Compare commits

...
42 Commits
Author SHA1 Message Date
ZyyandCodex 7a32c6cec2 chore(release): v3.38.2
Co-authored-by: Codex <noreply@openai.com>
2026-11-02 13:44:36 +08:00
0ff1a72919 fix(retry): limit upstream Retry-After waits (#578)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-02 13:07:20 +08:00
79083a5632 fix(openai): reconnect before websocket search activation (#577)
Track hosted search initialization on pooled Codex sockets and retire incompatible connections before sending. Preserve existing client full replay recovery for connection-local deltas.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-02 12:28:14 +08:00
ZyyandCodex 90f7456db8 fix(ui): align CPR badge text with its icon
Keep the compact line height and offset the uppercase label by one pixel to align its visible center with the brand glyph across badge sizes.

Co-authored-by: Codex <noreply@openai.com>
2026-11-02 12:27:51 +08:00
ZyyandCodex 6f71b27c3c fix(peer): include quota-backed plans in pool summaries
Reuse account plan resolution when exporting peer pool summaries. Read saved quota observations for missing plans even when quota sharing is disabled, without exposing quota data or mutating account records.

Cover unknown and missing plans, subtype preservation, plan-only limits, and quota refresh from an unknown account plan.

Co-authored-by: Codex <noreply@openai.com>
2026-11-02 12:24:57 +08:00
ZyyandCodex 7b05b14a2b fix(openai): narrow model rejection cooldown detection
Require the official model_not_found code and supported status codes for
account-model cooldowns. Treat unsupported_value as a terminal request
error and preserve account eligibility across HTTP, SSE, and WebSocket.

Co-authored-by: Codex <noreply@openai.com>
2026-11-02 11:36:58 +08:00
f9b5b5ff77 chore(deps): bump node from 26.12.1-bookworm-slim to 26.13.0-bookworm-slim in /deploy in the minor-and-patch group across 1 directory (#573)
* chore(deps): bump node

Bumps the minor-and-patch group with 1 update in the /deploy directory: node.


Updates `node` from 26.12.1-bookworm-slim to 26.13.0-bookworm-slim

---
updated-dependencies:
- dependency-name: node
  dependency-version: 26.13.0-bookworm-slim
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* docs(development): align Node.js version with Docker

Co-authored-by: Codex <noreply@openai.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-02 10:33:44 +08:00
dependabot[bot] 4d5778a84f chore(deps): bump rust from 48b7f43 to 9181b56 in /deploy (#575)
Bumps rust from `48b7f43` to `9181b56`.

---
updated-dependencies:
- dependency-name: rust
  dependency-version: 1.99.1-bookworm
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 10:32:20 +08:00
dependabot[bot] 6d22d73506 chore(deps): bump debian from 7ff6ba3 to 99c0b25 in /deploy (#574)
Bumps debian from `7ff6ba3` to `99c0b25`.

---
updated-dependencies:
- dependency-name: debian
  dependency-version: bookworm-slim
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 10:32:10 +08:00
dependabot[bot] d8ab207282 chore(deps): bump the minor-and-patch group in /frontend with 7 updates (#572)
Bumps the minor-and-patch group in /frontend with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [@lucide/vue](https://github.com/lucide-icons/lucide/tree/HEAD/packages/vue) | `1.61.0` | `1.66.0` |
| [axios](https://github.com/axios/axios) | `1.20.0` | `1.21.0` |
| [es-toolkit](https://github.com/toss/es-toolkit) | `1.53.0` | `1.53.1` |
| [@intlify/unplugin-vue-i18n](https://github.com/intlify/bundle-tools/tree/HEAD/packages/unplugin-vue-i18n) | `11.2.7` | `11.2.8` |
| [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) | `26.6.5` | `26.6.6` |
| [eslint](https://github.com/eslint/eslint) | `10.13.0` | `10.14.0` |
| [vue-tsc](https://github.com/vuejs/language-tools/tree/HEAD/packages/tsc) | `3.3.13` | `3.3.14` |


Updates `@lucide/vue` from 1.61.0 to 1.66.0
- [Release notes](https://github.com/lucide-icons/lucide/releases)
- [Commits](https://github.com/lucide-icons/lucide/commits/1.66.0/packages/vue)

Updates `axios` from 1.20.0 to 1.21.0
- [Release notes](https://github.com/axios/axios/releases)
- [Changelog](https://github.com/axios/axios/blob/v1.x/CHANGELOG.md)
- [Commits](https://github.com/axios/axios/compare/v1.20.0...v1.21.0)

Updates `es-toolkit` from 1.53.0 to 1.53.1
- [Release notes](https://github.com/toss/es-toolkit/releases)
- [Changelog](https://github.com/toss/es-toolkit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/toss/es-toolkit/compare/v1.53.0...v1.53.1)

Updates `@intlify/unplugin-vue-i18n` from 11.2.7 to 11.2.8
- [Release notes](https://github.com/intlify/bundle-tools/releases)
- [Changelog](https://github.com/intlify/bundle-tools/blob/v11.2.8/CHANGELOG.md)
- [Commits](https://github.com/intlify/bundle-tools/commits/v11.2.8/packages/unplugin-vue-i18n)

Updates `@types/node` from 26.6.5 to 26.6.6
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

Updates `eslint` from 10.13.0 to 10.14.0
- [Release notes](https://github.com/eslint/eslint/releases)
- [Commits](https://github.com/eslint/eslint/compare/v10.13.0...v10.14.0)

Updates `vue-tsc` from 3.3.13 to 3.3.14
- [Release notes](https://github.com/vuejs/language-tools/releases)
- [Changelog](https://github.com/vuejs/language-tools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vuejs/language-tools/commits/v3.3.14/packages/tsc)

---
updated-dependencies:
- dependency-name: "@lucide/vue"
  dependency-version: 1.66.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: axios
  dependency-version: 1.21.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: es-toolkit
  dependency-version: 1.53.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: "@intlify/unplugin-vue-i18n"
  dependency-version: 11.2.8
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: "@types/node"
  dependency-version: 26.6.6
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: eslint
  dependency-version: 10.14.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: vue-tsc
  dependency-version: 3.3.14
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 10:31:59 +08:00
dependabot[bot] 3816b46ba1 chore(deps): bump the minor-and-patch group in /backend with 8 updates (#571)
Bumps the minor-and-patch group in /backend with 8 updates:

| Package | From | To |
| --- | --- | --- |
| [uuid](https://github.com/uuid-rs/uuid) | `1.28.0` | `1.29.0` |
| [zip](https://github.com/zip-rs/zip2) | `9.0.5` | `9.1.3` |
| [jsonschema](https://github.com/Stranger6667/jsonschema) | `0.60.0` | `0.60.2` |
| [roxmltree](https://github.com/RazrFalcon/roxmltree) | `0.21.1` | `0.21.2` |
| [aws-sdk-s3](https://github.com/awslabs/aws-sdk-rust) | `1.155.0` | `1.156.0` |
| [aws-smithy-runtime-api](https://github.com/smithy-lang/smithy-rs) | `1.19.1` | `1.19.2` |
| [aws-smithy-types](https://github.com/smithy-lang/smithy-rs) | `1.9.0` | `1.9.1` |
| [redis](https://github.com/redis-rs/redis-rs) | `1.7.1` | `1.8.0` |


Updates `uuid` from 1.28.0 to 1.29.0
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/v1.28.0...v1.29.0)

Updates `zip` from 9.0.5 to 9.1.3
- [Release notes](https://github.com/zip-rs/zip2/releases)
- [Changelog](https://github.com/zip-rs/zip2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/zip-rs/zip2/compare/v9.0.5...v9.1.3)

Updates `jsonschema` from 0.60.0 to 0.60.2
- [Release notes](https://github.com/Stranger6667/jsonschema/releases)
- [Changelog](https://github.com/Stranger6667/jsonschema/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stranger6667/jsonschema/compare/ruby-v0.60.0...ruby-v0.60.2)

Updates `roxmltree` from 0.21.1 to 0.21.2
- [Changelog](https://github.com/RazrFalcon/roxmltree/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/roxmltree/compare/v0.21.1...v0.21.2)

Updates `aws-sdk-s3` from 1.155.0 to 1.156.0
- [Release notes](https://github.com/awslabs/aws-sdk-rust/releases)
- [Commits](https://github.com/awslabs/aws-sdk-rust/commits)

Updates `aws-smithy-runtime-api` from 1.19.1 to 1.19.2
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)

Updates `aws-smithy-types` from 1.9.0 to 1.9.1
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)

Updates `redis` from 1.7.1 to 1.8.0
- [Release notes](https://github.com/redis-rs/redis-rs/releases)
- [Commits](https://github.com/redis-rs/redis-rs/compare/redis-1.7.1...redis-1.8.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-version: 1.29.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: zip
  dependency-version: 9.1.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: jsonschema
  dependency-version: 0.60.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: roxmltree
  dependency-version: 0.21.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: aws-sdk-s3
  dependency-version: 1.156.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: aws-smithy-runtime-api
  dependency-version: 1.19.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: aws-smithy-types
  dependency-version: 1.9.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: redis
  dependency-version: 1.8.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 10:31:48 +08:00
dependabot[bot] 4a7c06f895 chore(deps): bump taiki-e/install-action (#569)
Bumps the minor-and-patch group with 1 update: [taiki-e/install-action](https://github.com/taiki-e/install-action).


Updates `taiki-e/install-action` from 2.87.31 to 2.88.1
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/1e29709c0ec63babbe8f266397a465f29a825f91...68a389c788b2cd04016ce8c80ee23b80d2ab3d68)

---
updated-dependencies:
- dependency-name: taiki-e/install-action
  dependency-version: 2.88.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-02 10:31:13 +08:00
ZyyandCodex e0b7a841ca chore(release): v3.38.1
Co-authored-by: Codex <noreply@openai.com>
2026-11-02 00:08:30 +08:00
ZyyandCodex 96a292dd0f fix(providers): align catalog-based account eligibility
Apply valid catalog membership alongside account model access for native OAuth, API key upstreams, and xAI. Preserve upstream validation for missing catalogs and unknown plans.

Co-authored-by: Codex <noreply@openai.com>
2026-11-02 00:02:08 +08:00
ZyyandCodex 6f138c8256 chore(release): v3.38.0
Co-authored-by: Codex <noreply@openai.com>
2026-11-01 23:32:37 +08:00
ZyyandCodex c8ffb1a8c8 perf(store): batch account lease runtime reads
Read up to 128 accounts per Redis script while preserving input order, pool isolation, shared scheduling timestamps, expiry cleanup, and error propagation. Extend real Redis coverage for mixed lease states and invalid batch data.

Co-authored-by: Codex <noreply@openai.com>
2026-11-01 23:23:49 +08:00
ZyyandCodex 79df82a4f0 fix(openai): respect model capabilities in account selection
Filter known OAuth models by fresh plan catalogs and retain account-scoped model rejections. Preserve credential, transport, adapter, and continuation boundaries, and combine cooldown expiry without dropping protection semantics.

Co-authored-by: Codex <noreply@openai.com>
2026-11-01 22:39:21 +08:00
ZyyandCodex 62516d9769 fix(ci): reduce Cargo cache snapshot disk usage
Keep Cargo snapshots compressed during transfer, extract directly into BuildKit cache mounts, and release restored runner snapshots. Export archives without an extra copy layer and use a new cache format key.

Co-authored-by: Codex <noreply@openai.com>
2026-11-01 22:05:10 +08:00
ZyyandCodex 892cba68ae test: streamline backend tests and speed up validation
Remove obsolete and redundant assertions while preserving current behavior.
Optimize zlib-rs in dev/test builds and use eight concurrent CI test workers.
Use explicit time bases and synchronization for integration fixtures.

Validated 4322 tests, 72 Redis ACL tests, and ten stress iterations of 102
concurrency-related tests. Rustfmt, strict Clippy, and actionlint passed.

Co-authored-by: Codex <noreply@openai.com>
2026-11-01 21:31:07 +08:00
ZyyandCodex fada9bc0bd refactor: clarify module boundaries and naming
Organize backend modules by responsibility, simplify runtime layers, and
keep shared contracts with their owning modules. Split frontend account
APIs and separate shared types from request implementations.

Strengthen architecture checks for module dependencies and test layout.

Co-authored-by: Codex <noreply@openai.com>
2026-11-01 20:35:59 +08:00
ZyyandCodex 27b9bbe994 fix: persist peer discovery and quiet expected cookie conflicts
Persist unsupported CPR Link detection for each connection and skip periodic
probes across restarts. Retain manual rediscovery, connection invalidation,
retry throttling, and existing quota access protections.

Log stale response-cookie revision conflicts at debug while retaining
warnings for actual storage failures.

Validation: 29 peer-related tests, 14 cookie tests, 16 architecture checks,
strict Clippy, Rustfmt, and diff checks passed.

Co-authored-by: Codex <noreply@openai.com>
2026-11-01 20:20:45 +08:00
ZyyandCodex db849c1251 feat(peer-link)!: simplify the CPR Link protocol
Make capability blocks optional and represent key budgets as named windows. Centralize validation and preserve unknown states for missing observations.

BREAKING CHANGE: CPR Link requires schema_version 2; v1 snapshots are no longer accepted.

Co-authored-by: Codex <noreply@openai.com>
2026-11-01 19:24:17 +08:00
ZyyandCodex c57af9b0dd fix(openai): strip sentry-trace and baggage from outbound requests
Remove tracing headers and their WebSocket metadata copies through the existing correlation cleanup. Cover cache identity stability and WebSocket reuse after privacy header changes.

Co-authored-by: Codex <noreply@openai.com>
2026-11-01 14:21:01 +08:00
ZyyandCodex 641f43b303 chore(release): v3.37.2
Co-authored-by: Codex <noreply@openai.com>
2026-11-01 03:41:24 +08:00
ZyyandCodex 6ce723ce93 fix(accounts): show account names in delete confirmation
Co-authored-by: Codex <noreply@openai.com>
2026-11-01 03:41:16 +08:00
ZyyandCodex 2ff43c73da fix(accounts): bound discovery refreshes and align import settings
Share a four-request limit across CPR discovery, manual refreshes, and
periodic refreshes, and avoid loading runtime settings for initial reads.

Reuse account name and notes fields for import and editing, apply custom
names only to single-account imports, and match API usage bars to quota
progress styling.

Co-authored-by: Codex <noreply@openai.com>
2026-11-01 03:35:29 +08:00
ZyyandCodex cb79f4d5a5 fix(accounts): start peer discovery after credential imports
Reuse quota routing for initial peer observation and preserve discovery state across ordinary account edits. Hide pending subscription badges and remove redundant provider quota labels without adding frontend polling.

Co-authored-by: Codex <noreply@openai.com>
2026-11-01 03:10:18 +08:00
ZyyandCodex 1e57805598 chore(release): v3.37.1
Co-authored-by: Codex <noreply@openai.com>
2026-11-01 02:20:01 +08:00
ZyyandCodex aca75490cb fix(accounts): preserve migrated identities and restore cumulative usage
Co-authored-by: Codex <noreply@openai.com>
2026-11-01 02:19:53 +08:00
ZyyandCodex 6a95869a72 docs: fix README links and add UI playground
Co-authored-by: Codex <noreply@openai.com>
2026-11-01 01:59:55 +08:00
ZyyandCodex dfc29d4709 chore(release): v3.37.0
Co-authored-by: Codex <noreply@openai.com>
2026-11-01 01:49:30 +08:00
ZyyandCodex 3cbebb5f9b feat(frontend): adopt UI v0.8.0 and refine form feedback
Unify theme namespaces and plugin theme delivery, improve validation and empty states, and pin UI v0.8.0 plus plugin workbench v0.9.0.

Co-authored-by: Codex <noreply@openai.com>
2026-11-01 01:40:09 +08:00
ZyyandCodex fbdfe22757 feat(settings): update default concurrency policy
Co-authored-by: Codex <noreply@openai.com>
2026-10-31 21:29:50 +08:00
7cf99769b1 feat(accounts): support editing optional account names (#568)
Allow local account names to be changed or cleared without replacing credentials or account settings. Show the name with notes and fall back to email when the name is empty.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-31 21:19:07 +08:00
ZyyandCodex 839054cdaf docs: refine GitHub reply tone
Co-authored-by: Codex <noreply@openai.com>
2026-10-31 21:09:49 +08:00
ZyyandCodex bcf0582814 refactor(peer): reuse account quota storage for CPR Link
Store peer snapshots in existing account quota fields and remove dedicated node and synchronization tables. Keep CPR protocol handling in Admin and native quota handling in Providers, sharing persistence and the quota refresh endpoint.

Use runtime refresh coordination and preserve credential, connection, and observation-time guards. Update the unreleased migration, API contracts, documentation, and regression coverage.

Validated with 354 selected tests, strict workspace Clippy, Rustfmt, frontend lint and build, and migration checksum verification. Browser interaction remains unverified.

Co-authored-by: Codex <noreply@openai.com>
2026-10-31 20:43:15 +08:00
ZyyandCodex a5d3d9f12d fix(peer): recover quota from explicit upstream budget state
Export authoritative budget access flags and keep monetary fields display-only
for peer consumers. Use the official usage_limit_reached error type for budget
exhaustion while preserving ordinary rate-limit cooldowns.

Apply fresh quota facts atomically with peer snapshots through the shared quota
writer, retaining connection, claim, and observation-time guards. Reuse the
existing sync worker and document the recovery contract.

Validated with 234 selected tests, strict workspace Clippy, and Rustfmt.

Co-authored-by: Codex <noreply@openai.com>
2026-10-31 15:58:02 +08:00
58a1f7541e feat(peer): add CPR Link discovery and scoped quota sharing (#566)
* feat(peer): add automatic discovery and scalable quota sharing

Export authorized direct-account pools and budgets through the CPR peer
protocol, with durable upstream discovery and immutable scheduling views.

Show shared-pool node quotas through common quota popovers and lazy account
details. Bound snapshot reads and export concurrency, reduce Redis and SQL
work, and keep only the latest observation for each upstream connection.

Co-authored-by: Codex <noreply@openai.com>

* refactor(peer): centralize upstream discovery and quota projections

Move peer synchronization into Admin, HTTP transport into Host, and expose
neutral Core scheduling signals through explicitly wired provider capabilities.

Preserve provider-visible model catalogs and independent quota disclosure,
with stale windows following the existing OAuth quota layout.

Cover cross-provider discovery, lifecycle fencing, and database leases, and
update the architecture and usage documentation.

Co-authored-by: Codex <noreply@openai.com>

* fix(ui): center usage card loading and empty states

Co-authored-by: Codex <noreply@openai.com>

* fix(accounts): unify peer subscription badges across account views

Share CPR identity rendering through AccountPlanBadge, project upstream metadata into dashboard and proxy account views, and align quota empty states and API badges.

Co-authored-by: Codex <noreply@openai.com>

* fix(peer): bound snapshot work and preserve quota windows

Keep summary reads off full snapshots, reject oversized model expansion early, and yield export work. Preserve daily and weekly unlimited budgets, hide empty token counters, and unify quota empty states.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-31 15:00:12 +08:00
ZyyandCodex 35e8b73075 fix(ui): center usage card loading and empty states
Co-authored-by: Codex <noreply@openai.com>
2026-10-31 13:03:30 +08:00
ZyyandCodex 73f0706f99 fix(deps): update follow-redirects to 1.16.2
Update the transitive Axios dependency to address GHSA-mrwp-f3q8-3g9p.

Co-authored-by: Codex <noreply@openai.com>
2026-10-31 12:11:14 +08:00
ZyyandCodex 76eecd771e fix(ui): correct account subscription and quota displays
Preserve authentication snapshots for diagnostic account badges and align
account note typography. Hide unavailable dashboard quotas consistently
and show available quota rates independently of local request counts.

Co-authored-by: Codex <noreply@openai.com>
2026-10-31 12:06:16 +08:00
Ke WangandCodex 7293d03d70 fix(models): separate reasoning summary capability from effort (#565)
Co-authored-by: Codex <noreply@openai.com>
2026-10-31 11:53:26 +08:00
646 changed files with 20146 additions and 6978 deletions
+3 -1
View File
@@ -44,7 +44,9 @@ Issue 可记录复现、排查和变更经过;涉及常规文档修改时执
草拟、发布或修改 Issue 回复时,按 [GitHub 回复口吻](../../../CONTRIBUTING.md#github-回复口吻)将核实结果写成面向提问者的交流答复,具体口吻规则集中维护在该节
发送或更新前从提问者视角通读:是否接住了他的目标和困难,建议是否便于尝试,补充材料的用途是否清楚,澄清或拒绝是否耐心解释了原因,能提供的替代办法是否说清。发现生硬时调整整句措辞与前后衔接,不能仅加缓和词;保留已核实的结论
按平常回帖的口吻写,语气自然、委婉,说清判断和原因,少用铺垫、转折和自我说明;暂不采纳或提出不同意见时说明具体原因
发送或更新前从提问者视角通读:判断和原因是否清楚,建议是否便于尝试,补充材料的用途是否明确。删去不影响理解的客套、重复解释和过程说明;发现生硬时调整整句,不能仅加缓和词,保留已核实的结论
## 发布与收尾
+9 -12
View File
@@ -7,21 +7,18 @@ RUN --mount=type=bind,target=/cache \
--mount=type=cache,target=/usr/local/cargo/git \
--mount=type=cache,target=/app/backend/target \
test -n "$CACHE_STAMP" \
&& cp -a /cache/registry/. /usr/local/cargo/registry/ \
&& cp -a /cache/git/. /usr/local/cargo/git/ \
&& cp -a /cache/target/. /app/backend/target/
&& tar -xzf /cache/registry.tar.gz -C /usr/local/cargo/registry \
&& tar -xzf /cache/git.tar.gz -C /usr/local/cargo/git \
&& tar -xzf /cache/target.tar.gz -C /app/backend/target
FROM busybox:1.37.0-musl@sha256:5cec3fc171c87218698e85a52af7087de727372aae264a787b8112901a5b0092 AS collect
FROM busybox:1.37.0-musl@sha256:5cec3fc171c87218698e85a52af7087de727372aae264a787b8112901a5b0092 AS snapshot
ARG CACHE_STAMP
# 一次导出三个 mount,避免逐目录创建镜像、载入 Docker 再复制文件。
# 直接压缩 mount 内容,构建层和 runner 只保存归档,避免多份展开的编译产物占满磁盘
RUN --mount=type=cache,target=/usr/local/cargo/registry \
--mount=type=cache,target=/usr/local/cargo/git \
--mount=type=cache,target=/app/backend/target \
test -n "$CACHE_STAMP" \
&& mkdir -p /cache/registry /cache/git /cache/target \
&& cp -a /usr/local/cargo/registry/. /cache/registry/ \
&& cp -a /usr/local/cargo/git/. /cache/git/ \
&& cp -a /app/backend/target/. /cache/target/
FROM scratch AS snapshot
COPY --from=collect /cache/ /
&& mkdir -p /cache \
&& tar -czf /cache/registry.tar.gz -C /usr/local/cargo/registry . \
&& tar -czf /cache/git.tar.gz -C /usr/local/cargo/git . \
&& tar -czf /cache/target.tar.gz -C /app/backend/target .
+32 -13
View File
@@ -14,24 +14,42 @@ case "$architecture" in amd64|arm64) ;; *) exit 2 ;; esac
dockerfile="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/Dockerfile"
usable() {
local root="$1" path crates=false libraries=false
# 目录或 buildstamp 存在不代表编译发生过,必须同时有下载包与目标依赖。
for path in "$root"/registry/cache/*/*.crate; do
if [ -f "$path" ] && [ -s "$path" ]; then crates=true; break; fi
done
for path in "$root"/target/"$architecture"/release/deps/*.rlib; do
if [ -f "$path" ] && [ -s "$path" ]; then libraries=true; break; fi
done
[ "$crates" = true ] && [ "$libraries" = true ]
# 流式检查归档中的非空依赖文件,校验时不在 runner 展开编译产物
python3 - "$1" "$architecture" <<'PY'
import pathlib
import re
import sys
import tarfile
root = pathlib.Path(sys.argv[1])
architecture = sys.argv[2]
def contains(filename, pattern):
with tarfile.open(root / filename, mode="r|gz") as archive:
return any(
member.isfile() and member.size > 0
and re.fullmatch(pattern, member.name.removeprefix("./"))
for member in archive
)
try:
valid = (
(root / "git.tar.gz").stat().st_size > 0
and contains("registry.tar.gz", r"cache/[^/]+/[^/]+\.crate")
and contains("target.tar.gz", rf"{architecture}/release/deps/[^/]+\.rlib")
)
except (OSError, tarfile.TarError):
valid = False
sys.exit(0 if valid else 1)
PY
}
command=(docker buildx build --builder "$builder" --file "$dockerfile" --progress plain)
# mount 的内容不参与层指纹,用时间戳重跑复制;--no-cache 会重置 cache mount。
# mount 的内容不参与层指纹,用时间戳重跑归档操作;--no-cache 会重置 cache mount
command+=(--build-arg "CACHE_STAMP=$(date +%s%N)" --target "$operation")
valid=false
if [ "$operation" = restore ]; then
if usable "$directory"; then
mkdir -p "$directory"/{registry,git,target}
"${command[@]}" --output type=cacheonly "$directory"
valid=true
else
@@ -43,10 +61,11 @@ else
temporary="$(mktemp -d "${directory}.XXXXXX")"
trap 'rm -rf "$temporary"' EXIT
mkdir "$temporary"/{context,snapshot}
# tar 流避开 local exporter 的逐文件同步阻塞,直接解包以免额外落盘整份归档。
# tar 流避开 local exporter 的逐文件同步阻塞,只接收 /cache 下的压缩归档
# 直接导出打包阶段,避免再创建一层归档副本
# https://github.com/moby/buildkit/issues/2950
"${command[@]}" --output type=tar,dest=- "$temporary/context" \
| tar -xf - -C "$temporary/snapshot"
| tar -xf - -C "$temporary/snapshot" --strip-components=1 cache/
if usable "$temporary/snapshot"; then
rm -rf "$directory"
mv "$temporary/snapshot" "$directory"
+5 -3
View File
@@ -36,10 +36,9 @@ jobs:
with:
path: ${{ runner.temp }}/container-cargo
# 同一平台和依赖复用快照,源码变化由 Cargo 指纹判定是否重编译
key: container-cargo-v3-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('backend/Cargo.lock', 'backend/rust-toolchain.toml', 'backend/**/Cargo.toml', 'backend/.cargo/config.toml', 'deploy/Dockerfile', '.github/actions/cargo-cache/Dockerfile') }}
key: container-cargo-v4-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('backend/Cargo.lock', 'backend/rust-toolchain.toml', 'backend/**/Cargo.toml', 'backend/.cargo/config.toml', 'deploy/Dockerfile', '.github/actions/cargo-cache/Dockerfile') }}
restore-keys: |
container-cargo-v3-${{ runner.os }}-${{ runner.arch }}-
container-cargo-v2-${{ runner.os }}-${{ runner.arch }}-
container-cargo-v4-${{ runner.os }}-${{ runner.arch }}-
- name: Restore Cargo mounts to BuildKit
id: cargo-restore
@@ -50,6 +49,9 @@ jobs:
path: ${{ runner.temp }}/container-cargo
architecture: amd64
- name: Release restored Cargo snapshot
run: rm -rf "$RUNNER_TEMP/container-cargo"
- name: Build runtime image
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
with:
+3 -3
View File
@@ -122,7 +122,7 @@ jobs:
run: cargo clippy --all-targets --all-features --locked -- -D warnings
- name: Setup nextest
uses: taiki-e/install-action@1e29709c0ec63babbe8f266397a465f29a825f91 # v2.87.31
uses: taiki-e/install-action@68a389c788b2cd04016ce8c80ee23b80d2ab3d68 # v2.88.1
with:
tool: cargo-nextest@0.9.143
fallback: none
@@ -176,7 +176,7 @@ jobs:
cargo nextest run \
--cargo-metadata "$RUNNER_TEMP/nextest-cargo-metadata.json" \
--binaries-metadata "$RUNNER_TEMP/nextest-binaries.json" \
--no-tests fail
--test-threads 8 --no-tests fail
- name: Store tests with Redis ACL authentication
working-directory: backend
@@ -195,7 +195,7 @@ jobs:
cargo nextest run \
--cargo-metadata "$RUNNER_TEMP/nextest-cargo-metadata.json" \
--binaries-metadata "$RUNNER_TEMP/nextest-binaries.json" \
--no-tests fail -E 'package(=gateway-store) & test(/^redis::/)'
--test-threads 8 --no-tests fail -E 'package(=gateway-store) & test(/^redis::/)'
frontend:
if: inputs.run_frontend
+1 -1
View File
@@ -28,7 +28,7 @@ jobs:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Install cargo-audit
uses: taiki-e/install-action@1e29709c0ec63babbe8f266397a465f29a825f91 # v2.87.31
uses: taiki-e/install-action@68a389c788b2cd04016ce8c80ee23b80d2ab3d68 # v2.88.1
with:
tool: cargo-audit@0.22.2
fallback: none
+1 -1
View File
@@ -26,7 +26,7 @@ AI 草拟、发布或修改 Issue、PR 讨论及审查意见的回复时,以
- 围绕问题、事实和结论展开,能自然省略主语时省略;需要说明个人判断或行动归属时用“我”,避免反复以“我”起句或强调自己,不用“我们”“本助手”代称自己
- 普通回复直接回应问题,不加问候语或称呼;向对方提问时称呼“佬”,同一条回复不重复称呼
- 先接住对方想完成的事和遇到的困难,再给出核实结果与建议;回复让对方感到问题被理解、有下一步可做,避免一上来归类、纠正或划定责任
- 建议用商量的口吻,说明尝试某个操作能帮助判断什么,给对方留出选择空间;“可以试试”“方便的话”等表达随语境使用,整句仍需自然,不靠堆叠缓和词修饰命令
- 给出操作建议时说明它能帮助判断什么,措辞自然、委婉,不靠堆叠缓和词修饰命令
- 回应已有能力、暂不支持或未采纳的建议时,结合场景说明可用入口或具体原因,有可行替代办法时一并说明,再交代必要边界;避免只回复“已有功能”“重复问题”“不在范围内”等结论
- 需要补充材料或配合操作时,先利用已有信息,只询问仍缺少的部分,说明用途和最小范围;例如“佬,方便再观察几次吗?如果还出现,可以附一下对应请求的诊断包,便于对照终态用量”
- 澄清误解或提出不同意见时,从观察和依据说起,照顾对方的使用环境与理解路径,避免责备、反问或居高临下的判断;已确认的事实保持明确,尚未确认的原因保留余地
+3 -7
View File
@@ -118,18 +118,14 @@ curl http://127.0.0.1:8080/v1/models \
| 部署与使用 | [部署、备份与恢复](deploy/README.md) · [客户端接入与生图](deploy/README.md#客户端配置) |
| 接口集成 | [API 参考](docs/api.md) · [模型定价](docs/api.md#模型定价) |
| 使用插件 | [安装、配置与使用](docs/plugins.md) |
| 开发插件 | [SDK 与合同](backend/crates/gateway-plugin/sdk/README.md) · [打包工具](backend/apps/plugin-cli/README.md) · [官方示例](modules/plugins/README.md) |
| 开发共用组件 | [组件库说明](modules/ui/README.md) · [组件参考](modules/ui/playground/components/index.md) |
| 开发插件 | [SDK 与合同](backend/crates/gateway-plugin/sdk/README.md) · [打包工具](backend/apps/plugin-cli/README.md) · [官方示例](https://github.com/zyycn/codex-proxy-plugins/blob/main/README.md) |
| 开发共用组件 | [组件库说明](https://github.com/zyycn/codex-proxy-ui/blob/main/README.md) · [组件参考](https://github.com/zyycn/codex-proxy-ui/blob/main/playground/components/index.md) · [Playground](https://zyycn.github.io/codex-proxy-ui/) |
| 开发宿主 | [开发与源码联调](docs/development.md) · [系统架构](docs/architecture.md) · [管理端主题](docs/theme.md) · [数据库迁移](backend/migrations/README.md) |
| 参与协作 | [贡献、审查与验证](CONTRIBUTING.md) |
## 社区
欢迎到 [Discussions 讨论区](https://github.com/zyycn/codex-proxy-rs/discussions)交流:
[使用问答](https://github.com/zyycn/codex-proxy-rs/discussions/categories/使用问答)、
[想法讨论](https://github.com/zyycn/codex-proxy-rs/discussions/categories/想法讨论)、
[实验反馈](https://github.com/zyycn/codex-proxy-rs/discussions/categories/实验反馈)与
[经验分享](https://github.com/zyycn/codex-proxy-rs/discussions/categories/经验分享)
问题反馈与功能建议请使用 [GitHub Issues](https://github.com/zyycn/codex-proxy-rs/issues),填写要求见 [贡献说明](CONTRIBUTING.md#提交-issue)
感谢 [LINUX DO](https://linux.do) 社区提供开放、友善的技术交流平台
+43 -38
View File
@@ -286,9 +286,9 @@ dependencies = [
[[package]]
name = "aws-sdk-s3"
version = "1.155.0"
version = "1.156.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a542f0e9261b9ce56708d591a62062079fa3c58a362ffbf49a2e48a4f6ced16"
checksum = "1072c9aa20159b9b417115342d38c4c53108fc98e5359fcb86b7d4575747eb59"
dependencies = [
"arc-swap",
"aws-credential-types",
@@ -322,9 +322,9 @@ dependencies = [
[[package]]
name = "aws-sigv4"
version = "1.6.3"
version = "1.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d3d48355cb86b15ccbb38b8bc6b87500c824ee4ed8cbc10a961b435242b5e16"
checksum = "f385da9b7757ed4c8214797ad663ff534fb622006143530aa80e21f8e06ff85c"
dependencies = [
"aws-credential-types",
"aws-smithy-eventstream",
@@ -387,9 +387,9 @@ dependencies = [
[[package]]
name = "aws-smithy-http"
version = "0.64.2"
version = "0.64.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba8960956e0ee0ca3f3161e62913a61bc65a7ef638ca3ef1d4c7159f8443e398"
checksum = "19af609dd5edf8b056cd0bb001c706c8b3b1458055328b334ec13d7f46f6aaa8"
dependencies = [
"aws-smithy-eventstream",
"aws-smithy-runtime-api",
@@ -408,9 +408,9 @@ dependencies = [
[[package]]
name = "aws-smithy-http-client"
version = "1.5.1"
version = "1.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aeca69f65362f7375891008ea525fd440ad6db1836458ac188c87b126804114e"
checksum = "859a81364f8185349c584fb41cd8765a400cf982f447adada590cf008a9f3b85"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
@@ -432,9 +432,9 @@ dependencies = [
[[package]]
name = "aws-smithy-json"
version = "0.65.0"
version = "0.65.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f6835fb38bc6fc8c91e584b0366eb4ff595e5c485134dbf4f0bdeaa18856666"
checksum = "b45f9fc77ffe92c92c341165a01a565bd30c39ecf5aa8c0719b41719009604f9"
dependencies = [
"aws-smithy-runtime-api",
"aws-smithy-schema",
@@ -452,9 +452,9 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime"
version = "1.16.1"
version = "1.16.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "79cef0c41e8ca40f74f68ed8ee4d1ddbde12bb847b95ef9b3c025681ddfae58a"
checksum = "4cb80fb19c084da5fca9d797845735e184445e5b958599440d67529bf6ef1b64"
dependencies = [
"aws-smithy-async",
"aws-smithy-http",
@@ -475,9 +475,9 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime-api"
version = "1.19.1"
version = "1.19.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "06788004dea79018413f381891da6dcca23196acd79c4d51b767fbdacc76eae4"
checksum = "8c4e85cffe9944b10d576b075432bd8f98dbc081e8681cb37d5b2d87dc27bc84"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api-macros",
@@ -492,9 +492,9 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime-api-macros"
version = "1.1.0"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "221eaa237ddf1ca79b60d1372aad77e47f9c0ea5b3ce5099da8c61d027dc77b3"
checksum = "ce237f890a2283c9b328955c3b0918d2d5bf66a1cb6ceccd36a0047246fbc6e6"
dependencies = [
"proc-macro2",
"quote",
@@ -514,9 +514,9 @@ dependencies = [
[[package]]
name = "aws-smithy-types"
version = "1.9.0"
version = "1.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "55752025c39c0e094962fd73f1c57014b191f678612e517f219468e575af9822"
checksum = "cf183d70774ea2cfcc101ca74a8884e4c76be8b9df65cd303f6a1175c88e8f80"
dependencies = [
"base64-simd",
"bytes",
@@ -537,9 +537,9 @@ dependencies = [
[[package]]
name = "aws-smithy-xml"
version = "0.64.0"
version = "0.64.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a6581618a2fd006159016d1c36881d94d61b4a017f4a8d631310875e18252413"
checksum = "590fdc1a70546b2f7f2061c42eaf173fb6f26d935330d069edc04dbd8f01d924"
dependencies = [
"aws-smithy-runtime-api",
"aws-smithy-schema",
@@ -1705,6 +1705,7 @@ dependencies = [
"cron",
"futures",
"gateway-core",
"gateway-protocol",
"heck",
"hex",
"hkdf",
@@ -1802,6 +1803,7 @@ dependencies = [
"futures",
"gateway-admin",
"gateway-core",
"gateway-protocol",
"hex",
"http",
"http-body-util",
@@ -2483,9 +2485,9 @@ dependencies = [
[[package]]
name = "jsonschema"
version = "0.60.0"
version = "0.60.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7e7305938cac5ee144352ceb71583ffc6cfeac7d20fcb20ef865a4d5c314e499"
checksum = "a42a11b47492e6a2cc76e9c7bcc8077776cb31dac444e0795d7f61bcc6441c08"
dependencies = [
"ahash",
"bytecount",
@@ -2511,18 +2513,18 @@ dependencies = [
[[package]]
name = "jsonschema-regex"
version = "0.60.1"
version = "0.60.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6f8a848331614144c03e815fb931f238c9e342bd7ab501b29e35d647e7cd5688"
checksum = "107aeb6ff14b8deb5610096a8ce5c554ec89806b521147eaeb3cae05b07edb7a"
dependencies = [
"regex-syntax",
]
[[package]]
name = "jsonschema-value"
version = "0.60.1"
version = "0.60.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a5a947ef798eeacacb43edf29b1d0ab64bdf5e2c9fbcaad10ee8663d922b544"
checksum = "e281d6b177bb4834f2ffd6f92aa0443cffe29c996b251c8d94eb6f456186feba"
dependencies = [
"ahash",
"bytecount",
@@ -3323,6 +3325,7 @@ dependencies = [
"futures",
"gateway-admin",
"gateway-core",
"gateway-host",
"gateway-protocol",
"getrandom 0.4.3",
"h2",
@@ -3575,9 +3578,9 @@ dependencies = [
[[package]]
name = "redis"
version = "1.7.1"
version = "1.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5b98ae99461e02895c3a96a25dbe4d852a3d5e5b815d31d5d9e4b5344d99e9"
checksum = "7acd809e80afa754c63f586955699d8990efe1c4b6613183ee5f027a11fbc8b5"
dependencies = [
"arc-swap",
"arcstr",
@@ -3631,9 +3634,9 @@ dependencies = [
[[package]]
name = "referencing"
version = "0.60.1"
version = "0.60.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "922268812b6f9e20b4ab0f1efe8e07036958eed7346ceddb31c777b5531fd11f"
checksum = "f42818036d3b37cda6bd4c494543158a36e431cb717015ef7c93acc6ba214d3b"
dependencies = [
"ahash",
"fluent-uri",
@@ -3756,9 +3759,9 @@ dependencies = [
[[package]]
name = "roxmltree"
version = "0.21.1"
version = "0.21.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1964b10c76125c36f8afe190065a4bf9a87bf324842c05701330bba9f1cacbb"
checksum = "9da7f59328b09405b181fee1018925fad50f157afb43c0c0f5c522c256dd5907"
dependencies = [
"memchr",
]
@@ -4950,9 +4953,9 @@ dependencies = [
[[package]]
name = "typed-path"
version = "0.12.3"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e28f89b80c87b8fb0cf04ab448d5dd0dd0ade2f8891bae878de66a75a28600e"
checksum = "9ce898823d8cd0f028662d2f349694b1af0a5cc9be83f1d213a00d3147445e1e"
[[package]]
name = "typenum"
@@ -5049,9 +5052,9 @@ checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "uuid"
version = "1.28.0"
version = "1.29.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7cc1186384beb7dd8eedea376413fd654937285ea6c9cfbb928dc3043ea4b606"
checksum = "4a0e2d11620c50695fe12bbbfde96b3477fe606e7284c22d505c12ce51214514"
dependencies = [
"getrandom 0.4.3",
"js-sys",
@@ -5643,16 +5646,18 @@ dependencies = [
[[package]]
name = "zip"
version = "9.0.5"
version = "9.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e04976da629d3b04204ba8105b396a193af8ad2583b2c79bfb3ab8af580a11e"
checksum = "1156b39d142a34b999c2310a50455e0cb3760bf7efad0122f796d813bee26ccb"
dependencies = [
"crc32fast",
"flate2",
"getrandom 0.4.3",
"indexmap",
"memchr",
"nix",
"typed-path",
"windows-sys 0.61.2",
]
[[package]]
+4
View File
@@ -49,6 +49,10 @@ opt-level = 2
[profile.dev.package."miniz_oxide@0.9.1"]
opt-level = 2
# workspace 的 WebSocket deflate 特性会让 flate2 选用 zlib-rs
[profile.dev.package.zlib-rs]
opt-level = 2
[profile.release]
debug = false
strip = "symbols"
@@ -100,7 +100,12 @@ async fn execute(
plugin_runtime,
plugin_runtime.bind_snapshots(&core.snapshots())
);
let management = ManagementPorts::new(&store, providers.admin.clone(), core.snapshot_control());
let management = ManagementPorts::new(
&store,
providers.admin.clone(),
core.snapshot_control(),
providers.peer.service(),
);
or_shutdown!(plugin_runtime, management.bind(&plugin_runtime));
let core = or_shutdown!(plugin_runtime, core.activate().await);
or_shutdown!(
@@ -132,6 +137,7 @@ async fn execute(
providers.admin.clone(),
core.snapshot_control(),
core.account_probe(),
Some(providers.peer.service()),
);
or_shutdown!(plugin_runtime, services.register_accounts(&accounts));
let usage_data = gateway_admin::initialize_usage_data(
@@ -42,6 +42,7 @@ impl ManagementPorts {
store: &gateway_store::StoreBundle,
providers: ProviderAdminRegistry,
snapshot: Arc<dyn SnapshotControl>,
upstream_info: Arc<dyn gateway_admin::ports::peer::PeerUpstreamService>,
) -> Self {
let ports = store.admin_ports();
Self {
@@ -49,6 +50,7 @@ impl ManagementPorts {
providers.clone(),
ports.accounts(),
snapshot.clone(),
Some(upstream_info),
),
keys: gateway_admin::initialize_plugin_client_keys(
providers,
+8 -1
View File
@@ -35,7 +35,12 @@ pub async fn run() -> Result<(), BootstrapError> {
plugin_runtime,
plugin_runtime.bind_snapshots(&core.snapshots())
);
let management = ManagementPorts::new(&store, providers.admin.clone(), core.snapshot_control());
let management = ManagementPorts::new(
&store,
providers.admin.clone(),
core.snapshot_control(),
providers.peer.service(),
);
or_shutdown!(plugin_runtime, management.bind(&plugin_runtime));
let mut core = or_shutdown!(plugin_runtime, core.activate().await);
or_shutdown!(
@@ -77,6 +82,7 @@ pub async fn run() -> Result<(), BootstrapError> {
config.client,
store.admin_ports(),
gateway_admin::AdminRuntimePorts {
upstream_info: Some(providers.peer.service()),
timezone,
service_middleware: service_middleware(core.snapshots(), &plugin_runtime),
providers: providers.admin,
@@ -165,6 +171,7 @@ pub async fn run() -> Result<(), BootstrapError> {
plan.extend(providers.openai.take_worker_contributions());
plan.extend(providers.xai.take_worker_contributions());
plan.extend(admin.take_worker_contributions());
plan.extend(providers.peer.take_worker_contributions());
plan.push(or_shutdown!(
plugin_runtime,
plugin_runtime.maintenance_worker(core.snapshots())
+20 -3
View File
@@ -67,6 +67,7 @@ pub(super) struct Providers {
pub(super) xai: provider_xai::ProviderBundle,
pub(super) core: ProviderRegistry,
pub(super) admin: ProviderAdminRegistry,
pub(super) peer: gateway_admin::PeerUpstreamBundle,
}
impl Providers {
@@ -75,16 +76,32 @@ impl Providers {
environment: &Environment,
) -> Result<Self, BootstrapError> {
let ports = environment.store.provider_ports();
let openai =
provider_openai::initialize(config.with_timezone(environment.timezone), ports.clone())
.await?;
let upstream_reader = Arc::new(gateway_core::account::UpstreamInfoReader::default());
let openai = provider_openai::initialize(
config.with_timezone(environment.timezone),
ports.clone(),
upstream_reader.clone(),
)
.await?;
environment.host.report_startup_ready("OpenAI Provider");
let xai = provider_xai::initialize(ports).await?;
environment.host.report_startup_ready("xAI Provider");
let core = ProviderRegistry::new([openai.core_provider(), xai.core_provider()])?;
let admin = ProviderAdminRegistry::new([openai.admin_provider(), xai.admin_provider()])
.map_err(|_| gateway_admin::model::AdminError::invalid("Provider 管理身份冲突"))?;
let peer = gateway_admin::initialize_peer_upstream(
environment.store.provider_ports().accounts(),
&admin,
Arc::new(gateway_host::peer::HttpPeerTransport::with_client_builder(
provider_openai::build_reqwest_client_with_custom_ca,
)),
)
.map_err(|_| gateway_admin::model::AdminError::invalid("互联后台任务配置无效"))?;
upstream_reader
.bind(&peer.scheduling_source())
.map_err(|_| gateway_admin::model::AdminError::invalid("互联观测端口重复绑定"))?;
Ok(Self {
peer,
openai,
xai,
core,
+189 -40
View File
@@ -70,6 +70,7 @@ fn gateway_core_depends_on_no_http_db_redis_or_provider_crate() {
#[test]
fn core_value_owners_do_not_depend_on_execution_or_routing() {
let root = backend_root().join("crates/gateway-core/src");
let reexports = read_root_reexport_owners(&root);
for relative in super::rust_files(&root) {
let allowed: Option<&[&str]> = match relative.to_str().expect("source path") {
"validation.rs" => Some(&[]),
@@ -96,7 +97,7 @@ fn core_value_owners_do_not_depend_on_execution_or_routing() {
let Some(allowed) = allowed else { continue };
let source = fs::read_to_string(root.join(&relative)).expect("read core source");
let syntax = syn::parse_file(&source).expect("parse core source");
let mut references = CrateReferences::in_module(&relative);
let mut references = CrateReferences::in_module(&relative).with_root_reexports(&reexports);
references.visit_file(&syntax);
// 相对路径中的 self 指向当前 owner,不是跨 owner 依赖
references.owners.remove(&references.module_path[0]);
@@ -111,17 +112,24 @@ fn core_value_owners_do_not_depend_on_execution_or_routing() {
}
#[test]
fn core_owner_paths_form_a_dag() {
let root = backend_root().join("crates/gateway-core/src");
fn domain_owner_paths_form_a_dag() {
for member in ["crates/gateway-core", "crates/gateway-admin"] {
assert_owner_paths_form_a_dag(member);
}
}
fn assert_owner_paths_form_a_dag(member: &str) {
let root = backend_root().join(member).join("src");
let reexports = read_root_reexport_owners(&root);
let mut graph = BTreeMap::<String, BTreeSet<String>>::new();
for relative in super::rust_files(&root) {
if relative == Path::new("lib.rs") {
continue;
}
let mut references = CrateReferences::in_module(&relative);
let mut references = CrateReferences::in_module(&relative).with_root_reexports(&reexports);
let owner = references.module_path[0].clone();
let source = fs::read_to_string(root.join(&relative)).expect("read core source");
references.visit_file(&syn::parse_file(&source).expect("parse core source"));
let source = fs::read_to_string(root.join(&relative)).expect("read owner source");
references.visit_file(&syn::parse_file(&source).expect("parse owner source"));
references.owners.remove(&owner);
graph.entry(owner).or_default().extend(references.owners);
}
@@ -131,7 +139,7 @@ fn core_owner_paths_form_a_dag() {
while let Some(dependency) = pending.pop() {
assert_ne!(
&dependency, owner,
"Core owner dependency cycle reaches {owner}"
"{member} owner dependency cycle reaches {owner}"
);
if visited.insert(dependency.clone())
&& let Some(next) = graph.get(&dependency)
@@ -143,7 +151,44 @@ fn core_owner_paths_form_a_dag() {
}
#[test]
fn core_reference_scan_resolves_groups_reexports_and_relative_paths() {
fn shared_owners_do_not_depend_on_their_callers() {
let boundaries: &[(&str, &str, &[&str])] = &[
("crates/providers/xai", "transport", &["credential"]),
("crates/providers/openai", "transport", &["provider"]),
("crates/gateway-api", "time", &["admin"]),
(
"crates/gateway-api",
"client_error",
&["admin", "openai", "middleware"],
),
("crates/gateway-api", "middleware", &["openai"]),
("crates/gateway-host", "system_update", &["config"]),
("crates/gateway-plugin/runtime", "package", &["callback"]),
];
for &(member, owner, forbidden) in boundaries {
let root = backend_root().join(member).join("src");
let reexports = read_root_reexport_owners(&root);
for relative in super::rust_files(&root) {
let mut references =
CrateReferences::in_module(&relative).with_root_reexports(&reexports);
if references.module_path.first().map(String::as_str) != Some(owner) {
continue;
}
let source = fs::read_to_string(root.join(&relative)).expect("read owner source");
references.visit_file(&syn::parse_file(&source).expect("parse owner source"));
for dependency in forbidden {
assert!(
!references.owners.contains(*dependency),
"{member}/src/{} must not depend on crate::{dependency}",
relative.display()
);
}
}
}
}
#[test]
fn owner_reference_scan_resolves_groups_reexports_and_relative_paths() {
let syntax = syn::parse_file(
r#"
pub use crate::{engine::AttemptContext, settings::SettingsValues as Values};
@@ -167,13 +212,86 @@ fn core_reference_scan_resolves_groups_reexports_and_relative_paths() {
);
}
#[derive(Default)]
struct CrateReferences {
owners: BTreeSet<String>,
module_path: Vec<String>,
#[test]
fn owner_reference_scan_resolves_crate_root_reexports() {
let root = syn::parse_file(
r#"
mod provider;
mod transport;
mod secret;
pub use provider::{OFFICIAL_URL, Options as ProviderOptions};
pub use self::transport::{self as network, Request};
pub use crate::secret::SecretValue;
pub use external::Unrelated;
"#,
)
.expect("parse root reexport fixture");
let syntax = syn::parse_file(
r#"
use crate::{OFFICIAL_URL, ProviderOptions, network::Client};
use super::super::SecretValue;
fn request() -> crate::Request { unimplemented!() }
"#,
)
.expect("parse reexport consumer fixture");
let reexports = root_reexport_owners(&root);
assert!(!reexports.contains_key("Unrelated"));
let mut references = CrateReferences::in_module(Path::new("consumer/request.rs"))
.with_root_reexports(&reexports);
references.visit_file(&syntax);
assert_eq!(
references.owners,
BTreeSet::from([
"provider".to_owned(),
"transport".to_owned(),
"secret".to_owned(),
])
);
}
impl<'ast> Visit<'ast> for CrateReferences {
fn read_root_reexport_owners(root: &Path) -> BTreeMap<String, String> {
let source = fs::read_to_string(root.join("lib.rs")).expect("read crate root");
root_reexport_owners(&syn::parse_file(&source).expect("parse crate root"))
}
// 只解析根模块的显式重导出,不推断 glob、宏展开或完整 Rust 名称解析
fn root_reexport_owners(syntax: &syn::File) -> BTreeMap<String, String> {
let modules = syntax
.items
.iter()
.filter_map(|item| match item {
Item::Mod(module) => Some(module.ident.to_string()),
_ => None,
})
.collect::<BTreeSet<_>>();
let mut reexports = BTreeMap::new();
for item in &syntax.items {
if let Item::Use(item) = item
&& !matches!(item.vis, syn::Visibility::Inherited)
{
visit_use_leaves(&item.tree, &mut Vec::new(), &mut |path, name| {
let owner = path
.iter()
.find(|segment| !matches!(segment.as_str(), "crate" | "self"));
if let (Some(owner), Some(name)) = (owner, name)
&& modules.contains(owner)
{
reexports.insert(name.to_owned(), owner.clone());
}
});
}
}
reexports
}
#[derive(Default)]
struct CrateReferences<'a> {
owners: BTreeSet<String>,
module_path: Vec<String>,
root_reexports: Option<&'a BTreeMap<String, String>>,
}
impl<'ast> Visit<'ast> for CrateReferences<'_> {
fn visit_path(&mut self, path: &'ast syn::Path) {
self.record(
&path
@@ -186,11 +304,13 @@ impl<'ast> Visit<'ast> for CrateReferences {
}
fn visit_item_use(&mut self, item: &'ast syn::ItemUse) {
self.use_tree(&item.tree, &mut Vec::new());
visit_use_leaves(&item.tree, &mut Vec::new(), &mut |path, _| {
self.record(path);
});
}
}
impl CrateReferences {
impl<'a> CrateReferences<'a> {
fn in_module(relative: &Path) -> Self {
let mut module = relative
.with_extension("")
@@ -208,13 +328,26 @@ impl CrateReferences {
Self {
owners: BTreeSet::new(),
module_path: module,
root_reexports: None,
}
}
fn with_root_reexports(mut self, reexports: &'a BTreeMap<String, String>) -> Self {
self.root_reexports = Some(reexports);
self
}
fn root_owner(&self, name: &str) -> String {
self.root_reexports
.and_then(|reexports| reexports.get(name))
.cloned()
.unwrap_or_else(|| name.to_owned())
}
fn record(&mut self, path: &[String]) {
let Some(first) = path.first() else { return };
let owner = if first == "crate" {
path.get(1).cloned()
path.get(1).map(|name| self.root_owner(name))
} else if first == "self" || first == "super" {
let mut module = self.module_path.clone();
let mut rest = path;
@@ -227,7 +360,10 @@ impl CrateReferences {
}
rest = &rest[1..];
}
module.first().or_else(|| rest.first()).cloned()
module
.first()
.cloned()
.or_else(|| rest.first().map(|name| self.root_owner(name)))
} else {
None
};
@@ -235,31 +371,41 @@ impl CrateReferences {
self.owners.insert(owner);
}
}
}
fn use_tree(&mut self, tree: &syn::UseTree, prefix: &mut Vec<String>) {
match tree {
syn::UseTree::Path(path) => {
prefix.push(path.ident.to_string());
self.use_tree(&path.tree, prefix);
prefix.pop();
}
syn::UseTree::Name(name) => {
prefix.push(name.ident.to_string());
self.record(prefix);
prefix.pop();
}
syn::UseTree::Rename(rename) => {
prefix.push(rename.ident.to_string());
self.record(prefix);
prefix.pop();
}
syn::UseTree::Group(group) => {
for item in &group.items {
self.use_tree(item, prefix);
}
}
syn::UseTree::Glob(_) => self.record(prefix),
fn visit_use_leaves(
tree: &syn::UseTree,
prefix: &mut Vec<String>,
visit: &mut impl FnMut(&[String], Option<&str>),
) {
match tree {
syn::UseTree::Path(path) => {
prefix.push(path.ident.to_string());
visit_use_leaves(&path.tree, prefix, visit);
prefix.pop();
}
syn::UseTree::Name(name) => {
let name = name.ident.to_string();
let export = if name == "self" {
prefix.last().cloned()
} else {
Some(name.clone())
};
prefix.push(name);
visit(prefix, export.as_deref());
prefix.pop();
}
syn::UseTree::Rename(rename) => {
prefix.push(rename.ident.to_string());
visit(prefix, Some(&rename.rename.to_string()));
prefix.pop();
}
syn::UseTree::Group(group) => {
for item in &group.items {
visit_use_leaves(item, prefix, visit);
}
}
syn::UseTree::Glob(_) => visit(prefix, None),
}
}
@@ -321,6 +467,7 @@ const ADAPTER_PUBLIC_MODULES: &[(&str, &[&str])] = &[
"client_distribution",
"config",
"outbound",
"peer",
"official_plugins",
"pricing",
"plugin_distribution",
@@ -358,11 +505,13 @@ const ALLOWED_INTERNAL_EDGES: &[(&str, &str)] = &[
("codex-proxy-rs", "provider-openai"),
("codex-proxy-rs", "provider-xai"),
("gateway-admin", "gateway-core"),
("gateway-admin", "gateway-protocol"),
("gateway-api", "gateway-admin"),
("gateway-api", "gateway-core"),
("gateway-api", "gateway-protocol"),
("gateway-host", "gateway-admin"),
("gateway-host", "gateway-core"),
("gateway-host", "gateway-protocol"),
("gateway-plugin-runtime", "gateway-host"),
("gateway-plugin-runtime", "gateway-core"),
("gateway-plugin-runtime", "gateway-admin"),
+1
View File
@@ -19,6 +19,7 @@ bytes = "1.11.1"
chrono = { version = "0.4.45", features = ["serde"] }
futures = "0.3.32"
gateway-core = { path = "../gateway-core" }
gateway-protocol = { path = "../gateway-protocol" }
heck = "0.5.0"
hex = "0.4.3"
hkdf = "0.13.0"
@@ -11,11 +11,15 @@ use chrono::{DateTime, Utc};
use futures::future::BoxFuture;
use gateway_core::{
lifecycle::CancellationToken,
task::{DaemonTask, WorkerDaemonContext, WorkerTaskError},
task::{
DaemonRestartPolicy, DaemonTask, WorkerContribution, WorkerDaemonContext, WorkerId,
WorkerKind, WorkerRegistration, WorkerRunnable, WorkerTaskError,
},
};
use tracing::{info, warn};
use crate::{
model::AdminError,
model::backup::{
BackupError, BackupObjectMetadata, BackupRecord, BackupStatus, BackupStatusTransition,
BackupStorageConfig, BackupTriggerKind, build_backup_seed, code,
@@ -34,6 +38,8 @@ impl From<BackupError> for WorkerTaskError {
use super::policy::{BackupSchedule, decide_retention};
const BACKUP_WORKER_OWNER: &str = "backup";
/// 循环间隔:管理员修改 Cron 后该时长内生效;无工作时可取消等待
const BACKUP_LOOP_INTERVAL: Duration = Duration::from_secs(30);
/// 每个循环最多执行的保留删除数量
@@ -72,6 +78,22 @@ impl BackupTask {
self.timezone = timezone;
self
}
pub(crate) fn into_worker(self) -> Result<WorkerContribution, AdminError> {
let id = WorkerId::try_new(WorkerKind::Backup, BACKUP_WORKER_OWNER)
.map_err(|_| AdminError::internal("备份 Worker ID 不合法"))?;
let restart = DaemonRestartPolicy::try_new(Duration::from_secs(1), Duration::from_secs(60))
.map_err(|_| AdminError::internal("备份 Worker 重启策略不合法"))?;
let registration = WorkerRegistration::try_new(
id,
WorkerRunnable::Daemon {
restart,
task: Box::new(self),
},
)
.map_err(|_| AdminError::internal("备份 Worker 注册信息不合法"))?;
Ok(WorkerContribution::Registration(registration))
}
}
impl DaemonTask for BackupTask {
@@ -0,0 +1,340 @@
//! 管理服务与后台任务的启动组装,窄端口供服务器和 CLI 复用
use std::{path::Path, sync::Arc};
use gateway_core::{engine::probe::AccountProbe, runtime::SnapshotControl};
use crate::{
AccountsService, AdminBundle, AdminConfig, AdminRuntimePorts, AdminServices, ClientConfig,
ClientKeyService, CredentialsService, PluginDistributionPorts, PluginManagementService,
PluginsService, SettingsService, UsageDataService, backup, freeze_recovery,
model::AdminError,
ports::{
self, plugin_accounts::PluginAccountAccess, plugin_client_keys::PluginClientKeyAccess,
provider::ProviderAdminRegistry, store::AdminStorePorts,
},
public_service, reset_credit_worker, use_case,
use_case::{
account_groups::DefaultAccountGroupService,
accounts::DefaultAccountsService,
auth::{AuthService, DefaultAuthService},
backup::DefaultBackupService,
client_distribution::DefaultClientDistributionService,
client_keys::DefaultClientKeyService,
observability::DefaultObservabilityService,
settings::DefaultSettingsService,
system::DefaultSystemService,
},
};
/// 校验配置、接入已组装的 Provider 注册表并完成默认管理员幂等初始化
///
/// # Errors
///
/// 配置非法或默认管理员初始化失败时返回错误
pub async fn initialize(
config: AdminConfig,
client_config: ClientConfig,
store: AdminStorePorts,
runtime: AdminRuntimePorts,
) -> Result<AdminBundle, AdminError> {
initialize_inner(config, client_config, store, runtime, None).await
}
/// 使用组合根已绑定给 Runtime 的同一账号窄端口,避免为完整 Admin 重建第二实例
pub async fn initialize_with_plugin_accounts(
config: AdminConfig,
client_config: ClientConfig,
store: AdminStorePorts,
runtime: AdminRuntimePorts,
plugin_accounts: Arc<dyn PluginAccountAccess>,
) -> Result<AdminBundle, AdminError> {
initialize_inner(config, client_config, store, runtime, Some(plugin_accounts)).await
}
async fn initialize_inner(
mut config: AdminConfig,
mut client_config: ClientConfig,
store: AdminStorePorts,
runtime: AdminRuntimePorts,
plugin_accounts: Option<Arc<dyn PluginAccountAccess>>,
) -> Result<AdminBundle, AdminError> {
let AdminRuntimePorts {
upstream_info,
timezone,
service_middleware,
plugin_preparation,
plugin_management,
published_snapshot,
plugin_distribution,
plugin_inspector,
pricing_source,
providers,
snapshot,
account_probe: probe,
proxy_probe,
client_distribution,
system,
client_key_verifier,
} = runtime;
config
.resolve_and_validate(Path::new("."))
.map_err(|error| AdminError::invalid(error.to_string()))?;
client_config
.resolve_and_validate(Path::new("."))
.map_err(|error| AdminError::invalid(error.to_string()))?;
let auth = Arc::new(DefaultAuthService::new(
&config,
client_config.session_ttl_minutes,
store.auth(),
client_key_verifier.clone(),
));
auth.ensure_default_admin(config.default_password.expose())
.await?;
let accounts = initialize_accounts(
store.accounts(),
store.account_runtime(),
store.settings(),
timezone,
providers.clone(),
snapshot.clone(),
probe.clone(),
upstream_info.clone(),
);
let backup_ports = store.backup();
let backups = Arc::new(DefaultBackupService::new(
backup_ports.repository(),
backup_ports.object_store(),
store.auth(),
snapshot.clone(),
timezone,
));
let backup_task = backup::task::BackupTask::new(
backup_ports.repository(),
backup_ports.dump(),
backup_ports.object_store(),
)
.with_timezone(timezone);
let system_preflight = Arc::new(
use_case::system_preflight::PluginSystemUpdatePreflight::new(
store.plugins(),
plugin_inspector.clone(),
),
);
let system = Arc::new(DefaultSystemService::new(system, system_preflight));
let observations: Arc<dyn ports::store::ObservabilityStore> = Arc::new(
use_case::observability::CachedObservabilityStore::new(store.observability()),
);
let key_usage = Arc::new(use_case::key_usage::DefaultKeyUsageService::new(
auth.clone(),
client_key_verifier,
store.client_keys(),
observations.clone(),
system.clone(),
timezone,
));
let credentials = Arc::new(CredentialsService::new(
providers.clone(),
store.accounts(),
store.proxies(),
snapshot.clone(),
upstream_info.clone(),
));
let plugin_accounts = plugin_accounts.unwrap_or_else(|| {
initialize_plugin_accounts(
providers.clone(),
store.accounts(),
snapshot.clone(),
upstream_info.clone(),
)
});
let import_tasks = use_case::import_tasks::DefaultImportTasksService::new(credentials.clone());
let import_task = use_case::import_tasks::ImportTaskWorker(import_tasks.clone());
let settings = initialize_settings(
store.settings(),
snapshot.clone(),
providers.clone(),
pricing_source,
);
let usage_data = initialize_usage_data(store.usage_data(), store.settings(), providers.clone());
let mut public_services = public_service::Registry::new(service_middleware);
public_services.register_settings(&settings)?;
public_services.register_accounts(&accounts)?;
public_services.register_usage_data(&usage_data)?;
let services = AdminServices {
peer_link: Arc::new(use_case::peer_link::DefaultPeerLinkService::new(
store.client_keys(),
store.accounts(),
store.account_runtime(),
providers.clone(),
)),
timezone,
public_services: Arc::new(public_services),
plugin_management: Arc::new(PluginManagementService::new(
plugin_management,
store.plugins(),
published_snapshot.clone(),
)),
plugins: Arc::new(PluginsService::new(
store.plugins(),
plugin_inspector,
PluginDistributionPorts::new(plugin_distribution, store.proxies()),
snapshot.clone(),
plugin_preparation,
published_snapshot,
store.plugin_state(),
)),
key_usage,
proxies: Arc::new(use_case::proxies::DefaultProxiesService::new(
store.proxies(),
proxy_probe,
snapshot.clone(),
providers.clone(),
upstream_info.clone(),
)),
auth,
accounts: accounts.clone(),
account_groups: Arc::new(DefaultAccountGroupService::new(
store.account_groups(),
store.account_runtime(),
snapshot.clone(),
)),
client_keys: Arc::new(DefaultClientKeyService::new(
store.client_keys(),
snapshot.clone(),
providers.clone(),
)),
client_distribution: Arc::new(DefaultClientDistributionService::new(client_distribution)),
observability: Arc::new(DefaultObservabilityService::new(
observations.clone(),
store.accounts(),
store.settings(),
providers.clone(),
timezone,
system.clone(),
upstream_info,
)),
settings,
usage_data,
system,
credentials,
_plugin_accounts: plugin_accounts,
import_tasks,
backups,
};
let freeze_recovery =
freeze_recovery::FreezeRecoveryTask::new(freeze_recovery::FreezeRecoveryDeps {
accounts: Arc::clone(&accounts) as Arc<dyn AccountsService>,
store: store.accounts(),
runtime: store.account_runtime(),
settings: store.settings(),
});
let worker_contributions = vec![
backup_task.into_worker()?,
import_task.into_worker()?,
freeze_recovery.into_worker()?,
reset_credit_worker::ResetCreditAutomationTask {
accounts,
store: store.accounts(),
settings: store.settings(),
}
.into_worker()?,
];
Ok(AdminBundle {
services,
worker_contributions,
})
}
/// CLI 只组合账号用例,不初始化管理员、管理服务或后台任务;写入仍复用同一事务与审计
pub fn initialize_plugin_accounts(
providers: ports::provider::ProviderAdminRegistry,
accounts: Arc<dyn ports::store::AccountStore>,
snapshot: Arc<dyn gateway_core::runtime::SnapshotControl>,
upstream_info: Option<Arc<dyn ports::peer::PeerUpstreamService>>,
) -> Arc<dyn PluginAccountAccess> {
Arc::new(use_case::plugin_accounts::DefaultPluginAccountAccess::new(
providers,
accounts,
snapshot,
upstream_info,
))
}
/// 为 Runtime 创建非秘密 Client Key 目录与预算重置的窄端口
#[must_use]
pub fn initialize_plugin_client_keys(
providers: ports::provider::ProviderAdminRegistry,
store: Arc<dyn ports::store::ClientKeyStore>,
snapshot: Arc<dyn SnapshotControl>,
) -> Arc<dyn PluginClientKeyAccess> {
let service: Arc<dyn ClientKeyService> =
Arc::new(DefaultClientKeyService::new(store, snapshot, providers));
Arc::new(use_case::plugin_client_keys::DefaultPluginClientKeyAccess::new(service))
}
/// 为 Runtime 组合实例自有资源写入;权限和归属在同一存储事务复核
#[must_use]
pub fn initialize_plugin_resources(
store: Arc<dyn ports::plugin_resources::PluginResourceStore>,
snapshot: Arc<dyn SnapshotControl>,
) -> Arc<dyn ports::plugin_resources::PluginResourceAccess> {
Arc::new(use_case::plugin_resources::DefaultPluginResourceAccess { store, snapshot })
}
/// 服务端与 CLI 复用账号用例,不随命令启动后台调度
#[must_use]
#[expect(
clippy::too_many_arguments,
reason = "账号存储、发布、探测与互联分别通过独立端口注入"
)]
pub fn initialize_accounts(
accounts: Arc<dyn ports::store::AccountStore>,
account_runtime: Arc<dyn ports::store::AccountRuntimeStore>,
settings: Arc<dyn ports::store::SettingsStore>,
timezone: gateway_core::time::DeploymentTimeZone,
providers: ProviderAdminRegistry,
snapshot: Arc<dyn SnapshotControl>,
probe: Arc<dyn AccountProbe>,
upstream_info: Option<Arc<dyn ports::peer::PeerUpstreamService>>,
) -> Arc<dyn AccountsService> {
Arc::new(DefaultAccountsService::new(
accounts,
account_runtime,
settings,
timezone,
providers,
snapshot,
probe,
upstream_info,
))
}
/// 设置服务不依赖 Web 管理会话,CLI 与服务器通过同一用例执行事务和发布
pub fn initialize_settings(
store: Arc<dyn ports::store::SettingsStore>,
snapshot: Arc<dyn SnapshotControl>,
providers: ProviderAdminRegistry,
pricing_source: Arc<dyn ports::pricing::PricingSource>,
) -> Arc<dyn SettingsService> {
Arc::new(DefaultSettingsService::new(
store,
snapshot,
providers,
pricing_source,
))
}
/// HTTP、插件及 CLI 共用的用量领域用例,宿主提供已验证的调用主体
#[must_use]
pub fn initialize_usage_data(
store: Arc<dyn ports::usage_data::UsageDataStore>,
settings: Arc<dyn ports::store::SettingsStore>,
providers: ProviderAdminRegistry,
) -> Arc<dyn UsageDataService> {
Arc::new(use_case::usage_data::DefaultUsageDataService::new(
store, settings, providers,
))
}
@@ -8,9 +8,13 @@ use chrono::{DateTime, Utc};
use futures::StreamExt as _;
use gateway_core::account::{AccountErrorProtection, ProviderAccountId};
use gateway_core::routing::UpstreamModelId;
use gateway_core::task::{ScheduledTask, WorkerCycleContext, WorkerTaskError};
use gateway_core::task::{
ScheduledTask, WorkerContribution, WorkerCycleContext, WorkerId, WorkerKind,
WorkerLeaseRequest, WorkerRegistration, WorkerRunnable, WorkerSchedule, WorkerTaskError,
};
use tracing::warn;
use crate::model::AdminError;
use crate::model::accounts::{
AccountConnectionTestEvent, AccountFreeze, AccountPageItem, AccountRuntimeSnapshot,
};
@@ -42,6 +46,34 @@ impl FreezeRecoveryTask {
Self { deps }
}
pub(crate) fn into_worker(self) -> Result<WorkerContribution, AdminError> {
let id = WorkerId::try_new(
WorkerKind::AccountFreezeRecovery,
FREEZE_RECOVERY_WORKER_OWNER,
)
.map_err(|_| AdminError::internal("冻结恢复 Worker ID 不合法"))?;
let schedule = WorkerSchedule::try_new(
FREEZE_RECOVERY_INTERVAL,
WORKER_INITIAL_BACKOFF,
WORKER_MAXIMUM_BACKOFF,
WORKER_LEASE_TTL,
WORKER_LEASE_RENEWAL,
)
.map_err(|_| AdminError::internal("冻结恢复 Worker 调度配置不合法"))?;
let lease = WorkerLeaseRequest::try_new(id.clone(), WORKER_LEASE_TTL)
.map_err(|_| AdminError::internal("冻结恢复 Worker 租约配置不合法"))?;
let registration = WorkerRegistration::try_new(
id,
WorkerRunnable::Scheduled {
schedule,
lease: Some(lease),
task: Box::new(self),
},
)
.map_err(|_| AdminError::internal("冻结恢复 Worker 注册信息不合法"))?;
Ok(WorkerContribution::Registration(registration))
}
/// 无法读取或校验配置时跳过本轮,不能把未知状态当作允许解冻
async fn freeze_policy(&self) -> Option<AccountErrorProtection> {
let settings = self.deps.settings.load_runtime_settings().await.ok()?;
+24 -386
View File
@@ -2,30 +2,27 @@
//!
//! 本 crate 不包含 HTTP wire、数据库实现或具体 Provider 实现
use std::{fmt, path::Path, sync::Arc, time::Duration};
use std::{fmt, path::Path, sync::Arc};
use gateway_core::{
engine::execution::ClientKeyVerifier,
engine::probe::AccountProbe,
runtime::SnapshotControl,
task::{
DaemonRestartPolicy, WorkerContribution, WorkerId, WorkerKind, WorkerLeaseRequest,
WorkerRegistration, WorkerRunnable, WorkerSchedule,
},
engine::execution::ClientKeyVerifier, engine::probe::AccountProbe, runtime::SnapshotControl,
task::WorkerContribution,
};
use secrecy::{ExposeSecret as _, SecretString};
use serde::Deserialize;
pub mod backup;
mod bootstrap;
pub mod freeze_recovery;
pub mod model;
pub mod ports;
pub mod public_service;
mod reset_credit_automation;
mod reset_credit_worker;
mod use_case;
pub use use_case::plugins::{PluginDistributionPorts, PluginManagementService, PluginsService};
pub use use_case::key_usage::KeyUsageService;
pub use use_case::peer_link::{PeerLinkError, PeerLinkService};
pub use use_case::{
account_groups::AccountGroupService,
@@ -43,18 +40,15 @@ pub use use_case::{
usage_data::UsageDataService,
};
use model::AdminError;
pub use bootstrap::{
initialize, initialize_accounts, initialize_plugin_accounts, initialize_plugin_client_keys,
initialize_plugin_resources, initialize_settings, initialize_usage_data,
initialize_with_plugin_accounts,
};
use ports::{
client_distribution::ClientDistributionResolver, plugin_accounts::PluginAccountAccess,
plugin_client_keys::PluginClientKeyAccess, provider::ProviderAdminRegistry,
store::AdminStorePorts, system::SystemOperations,
};
use use_case::{
account_groups::DefaultAccountGroupService, accounts::DefaultAccountsService,
auth::DefaultAuthService, backup::DefaultBackupService,
client_distribution::DefaultClientDistributionService, client_keys::DefaultClientKeyService,
observability::DefaultObservabilityService, settings::DefaultSettingsService,
system::DefaultSystemService,
provider::ProviderAdminRegistry, system::SystemOperations,
};
const MINIMUM_INITIAL_PASSWORD_BYTES: usize = 12;
@@ -69,7 +63,6 @@ const WEAK_ADMIN_PASSWORDS: &[&str] = &[
"codex-proxy-rs",
];
const BACKUP_WORKER_OWNER: &str = "backup";
const DEFAULT_CLIENT_SESSION_TTL_MINUTES: u64 = 24 * 60;
/// 只用于首次幂等创建默认管理员的启动密码
@@ -213,6 +206,7 @@ pub struct AdminServices {
proxies: Arc<dyn ProxiesService>,
auth: Arc<dyn AuthService>,
key_usage: Arc<dyn KeyUsageService>,
peer_link: Arc<dyn PeerLinkService>,
accounts: Arc<dyn AccountsService>,
account_groups: Arc<dyn AccountGroupService>,
client_keys: Arc<dyn ClientKeyService>,
@@ -258,6 +252,11 @@ impl AdminServices {
self.key_usage.as_ref()
}
#[must_use]
pub fn peer_link(&self) -> &dyn PeerLinkService {
self.peer_link.as_ref()
}
#[must_use]
pub fn proxies(&self) -> &dyn ProxiesService {
self.proxies.as_ref()
@@ -336,8 +335,13 @@ impl AdminBundle {
}
}
pub use use_case::peer_link::upstream::{
PeerUpstreamBundle, initialize as initialize_peer_upstream,
};
/// 组合根提供给控制面的运行能力;与配置和存储端口分别传入
pub struct AdminRuntimePorts {
pub upstream_info: Option<Arc<dyn ports::peer::PeerUpstreamService>>,
pub timezone: gateway_core::time::DeploymentTimeZone,
pub service_middleware: public_service::PlanSource,
pub plugin_preparation: Arc<dyn ports::plugins::PluginPreparation>,
@@ -354,369 +358,3 @@ pub struct AdminRuntimePorts {
pub system: Arc<dyn SystemOperations>,
pub client_key_verifier: Arc<dyn ClientKeyVerifier>,
}
/// 校验配置、接入已组装的 Provider 注册表并完成默认管理员幂等初始化
///
/// # Errors
///
/// 配置非法或默认管理员初始化失败时返回错误
pub async fn initialize(
config: AdminConfig,
client_config: ClientConfig,
store: AdminStorePorts,
runtime: AdminRuntimePorts,
) -> Result<AdminBundle, AdminError> {
initialize_inner(config, client_config, store, runtime, None).await
}
/// 使用组合根已绑定给 Runtime 的同一账号窄端口,避免为完整 Admin 重建第二实例
pub async fn initialize_with_plugin_accounts(
config: AdminConfig,
client_config: ClientConfig,
store: AdminStorePorts,
runtime: AdminRuntimePorts,
plugin_accounts: Arc<dyn PluginAccountAccess>,
) -> Result<AdminBundle, AdminError> {
initialize_inner(config, client_config, store, runtime, Some(plugin_accounts)).await
}
async fn initialize_inner(
mut config: AdminConfig,
mut client_config: ClientConfig,
store: AdminStorePorts,
runtime: AdminRuntimePorts,
plugin_accounts: Option<Arc<dyn PluginAccountAccess>>,
) -> Result<AdminBundle, AdminError> {
let AdminRuntimePorts {
timezone,
service_middleware,
plugin_preparation,
plugin_management,
published_snapshot,
plugin_distribution,
plugin_inspector,
pricing_source,
providers,
snapshot,
account_probe: probe,
proxy_probe,
client_distribution,
system,
client_key_verifier,
} = runtime;
config
.resolve_and_validate(Path::new("."))
.map_err(|error| AdminError::invalid(error.to_string()))?;
client_config
.resolve_and_validate(Path::new("."))
.map_err(|error| AdminError::invalid(error.to_string()))?;
let registry = providers;
let auth = Arc::new(DefaultAuthService::new(
&config,
client_config.session_ttl_minutes,
store.auth(),
client_key_verifier.clone(),
));
auth.ensure_default_admin(config.default_password.expose())
.await?;
let accounts = initialize_accounts(
store.accounts(),
store.account_runtime(),
store.settings(),
timezone,
registry.clone(),
snapshot.clone(),
probe.clone(),
);
let backup_ports = store.backup();
let backups = Arc::new(DefaultBackupService::new(
backup_ports.repository(),
backup_ports.object_store(),
store.auth(),
snapshot.clone(),
timezone,
));
let backup_task = backup::task::BackupTask::new(
backup_ports.repository(),
backup_ports.dump(),
backup_ports.object_store(),
)
.with_timezone(timezone);
let system_preflight = Arc::new(use_case::plugin_update::PluginSystemUpdatePreflight::new(
store.plugins(),
plugin_inspector.clone(),
));
let system = Arc::new(DefaultSystemService::new(system, system_preflight));
let observations: Arc<dyn ports::store::ObservabilityStore> = Arc::new(
use_case::observability::CachedObservabilityStore::new(store.observability()),
);
let key_usage = Arc::new(use_case::key_usage::DefaultKeyUsageService::new(
auth.clone(),
client_key_verifier,
store.client_keys(),
observations.clone(),
system.clone(),
timezone,
));
let credentials = Arc::new(CredentialsService::new(
registry.clone(),
store.accounts(),
store.proxies(),
snapshot.clone(),
));
let plugin_accounts = plugin_accounts.unwrap_or_else(|| {
initialize_plugin_accounts(registry.clone(), store.accounts(), snapshot.clone())
});
let import_tasks = use_case::import_tasks::DefaultImportTasksService::new(credentials.clone());
let import_task = use_case::import_tasks::ImportTaskWorker(import_tasks.clone());
let settings = initialize_settings(
store.settings(),
snapshot.clone(),
registry.clone(),
pricing_source,
);
let usage_data = initialize_usage_data(store.usage_data(), store.settings(), registry.clone());
let mut public_services = public_service::Registry::new(service_middleware);
public_services.register_settings(&settings)?;
public_services.register_accounts(&accounts)?;
public_services.register_usage_data(&usage_data)?;
let services = AdminServices {
timezone,
public_services: Arc::new(public_services),
plugin_management: Arc::new(PluginManagementService::new(
plugin_management,
store.plugins(),
published_snapshot.clone(),
)),
plugins: Arc::new(PluginsService::new(
store.plugins(),
plugin_inspector,
PluginDistributionPorts::new(plugin_distribution, store.proxies()),
snapshot.clone(),
plugin_preparation,
published_snapshot,
store.plugin_state(),
)),
key_usage,
proxies: Arc::new(use_case::proxies::DefaultProxiesService::new(
store.proxies(),
proxy_probe,
snapshot.clone(),
registry.clone(),
)),
auth,
accounts: accounts.clone(),
account_groups: Arc::new(DefaultAccountGroupService::new(
store.account_groups(),
store.account_runtime(),
snapshot.clone(),
)),
client_keys: Arc::new(DefaultClientKeyService::new(
store.client_keys(),
snapshot.clone(),
registry.clone(),
)),
client_distribution: Arc::new(DefaultClientDistributionService::new(client_distribution)),
observability: Arc::new(DefaultObservabilityService::new(
observations.clone(),
store.accounts(),
store.settings(),
registry.clone(),
timezone,
system.clone(),
)),
settings,
usage_data,
system,
credentials,
_plugin_accounts: plugin_accounts,
import_tasks,
backups,
};
let freeze_recovery =
freeze_recovery::FreezeRecoveryTask::new(freeze_recovery::FreezeRecoveryDeps {
accounts: Arc::clone(&accounts) as Arc<dyn AccountsService>,
store: store.accounts(),
runtime: store.account_runtime(),
settings: store.settings(),
});
let mut worker_contributions = backup_worker_contribution(backup_task)?;
let id = WorkerId::try_new(WorkerKind::AccountImport, "admin")
.map_err(|_| AdminError::internal("导入 Worker ID 不合法"))?;
let restart = DaemonRestartPolicy::try_new(Duration::from_secs(1), Duration::from_secs(60))
.map_err(|_| AdminError::internal("导入 Worker 重启策略不合法"))?;
let registration = WorkerRegistration::try_new(
id,
WorkerRunnable::Daemon {
restart,
task: Box::new(import_task),
},
)
.map_err(|_| AdminError::internal("导入 Worker 注册信息不合法"))?;
worker_contributions.push(WorkerContribution::Registration(registration));
worker_contributions.extend(freeze_recovery_worker_contribution(freeze_recovery)?);
let id = WorkerId::try_new(WorkerKind::ResetCreditAutomation, "admin")
.map_err(|_| AdminError::internal("重置卡 Worker ID 不合法"))?;
let schedule = WorkerSchedule::try_new(
Duration::from_secs(5),
Duration::from_secs(5),
Duration::from_secs(60),
Duration::from_secs(90),
Duration::from_secs(30),
)
.map_err(|_| AdminError::internal("重置卡 Worker 调度配置不合法"))?;
let lease = WorkerLeaseRequest::try_new(id.clone(), Duration::from_secs(90))
.map_err(|_| AdminError::internal("重置卡 Worker 租约配置不合法"))?;
worker_contributions.push(WorkerContribution::Registration(
WorkerRegistration::try_new(
id,
WorkerRunnable::Scheduled {
schedule,
lease: Some(lease),
task: Box::new(reset_credit_automation::ResetCreditAutomationTask {
accounts,
store: store.accounts(),
settings: store.settings(),
}),
},
)
.map_err(|_| AdminError::internal("重置卡 Worker 注册失败"))?,
));
Ok(AdminBundle {
services,
worker_contributions,
})
}
/// CLI 只组合账号用例,不初始化管理员、管理服务或后台任务;写入仍复用同一事务与审计
pub fn initialize_plugin_accounts(
providers: ports::provider::ProviderAdminRegistry,
accounts: Arc<dyn ports::store::AccountStore>,
snapshot: Arc<dyn gateway_core::runtime::SnapshotControl>,
) -> Arc<dyn PluginAccountAccess> {
Arc::new(use_case::plugin_accounts::DefaultPluginAccountAccess::new(
providers, accounts, snapshot,
))
}
/// 为 Runtime 创建非秘密 Client Key 目录与预算重置的窄端口
#[must_use]
pub fn initialize_plugin_client_keys(
providers: ports::provider::ProviderAdminRegistry,
store: Arc<dyn ports::store::ClientKeyStore>,
snapshot: Arc<dyn SnapshotControl>,
) -> Arc<dyn PluginClientKeyAccess> {
let service: Arc<dyn ClientKeyService> =
Arc::new(DefaultClientKeyService::new(store, snapshot, providers));
Arc::new(use_case::plugin_client_keys::DefaultPluginClientKeyAccess::new(service))
}
/// 为 Runtime 组合实例自有资源写入;权限和归属在同一存储事务复核
#[must_use]
pub fn initialize_plugin_resources(
store: Arc<dyn ports::plugin_resources::PluginResourceStore>,
snapshot: Arc<dyn SnapshotControl>,
) -> Arc<dyn ports::plugin_resources::PluginResourceAccess> {
Arc::new(use_case::plugin_resources::DefaultPluginResourceAccess { store, snapshot })
}
/// Backup Worker 注册:单个可取消 Daemon,owner 固定为 `backup`
fn backup_worker_contribution(
task: backup::task::BackupTask,
) -> Result<Vec<WorkerContribution>, AdminError> {
let id = WorkerId::try_new(WorkerKind::Backup, BACKUP_WORKER_OWNER)
.map_err(|_| AdminError::internal("备份 Worker ID 不合法"))?;
let restart = DaemonRestartPolicy::try_new(Duration::from_secs(1), Duration::from_secs(60))
.map_err(|_| AdminError::internal("备份 Worker 重启策略不合法"))?;
let registration = WorkerRegistration::try_new(
id,
WorkerRunnable::Daemon {
restart,
task: Box::new(task),
},
)
.map_err(|_| AdminError::internal("备份 Worker 注册信息不合法"))?;
Ok(vec![WorkerContribution::Registration(registration)])
}
/// 冻结恢复 Worker 注册:按固定周期扫描活跃冻结,owner 固定
fn freeze_recovery_worker_contribution(
task: freeze_recovery::FreezeRecoveryTask,
) -> Result<Vec<WorkerContribution>, AdminError> {
let id = WorkerId::try_new(
WorkerKind::AccountFreezeRecovery,
freeze_recovery::FREEZE_RECOVERY_WORKER_OWNER,
)
.map_err(|_| AdminError::internal("冻结恢复 Worker ID 不合法"))?;
let schedule = WorkerSchedule::try_new(
freeze_recovery::FREEZE_RECOVERY_INTERVAL,
freeze_recovery::WORKER_INITIAL_BACKOFF,
freeze_recovery::WORKER_MAXIMUM_BACKOFF,
freeze_recovery::WORKER_LEASE_TTL,
freeze_recovery::WORKER_LEASE_RENEWAL,
)
.map_err(|_| AdminError::internal("冻结恢复 Worker 调度配置不合法"))?;
let lease = WorkerLeaseRequest::try_new(id.clone(), freeze_recovery::WORKER_LEASE_TTL)
.map_err(|_| AdminError::internal("冻结恢复 Worker 租约配置不合法"))?;
let registration = WorkerRegistration::try_new(
id,
WorkerRunnable::Scheduled {
schedule,
lease: Some(lease),
task: Box::new(task),
},
)
.map_err(|_| AdminError::internal("冻结恢复 Worker 注册信息不合法"))?;
Ok(vec![WorkerContribution::Registration(registration)])
}
/// 服务端与 CLI 复用账号用例,不随命令启动后台调度
#[must_use]
pub fn initialize_accounts(
accounts: Arc<dyn ports::store::AccountStore>,
account_runtime: Arc<dyn ports::store::AccountRuntimeStore>,
settings: Arc<dyn ports::store::SettingsStore>,
timezone: gateway_core::time::DeploymentTimeZone,
providers: ProviderAdminRegistry,
snapshot: Arc<dyn SnapshotControl>,
probe: Arc<dyn AccountProbe>,
) -> Arc<dyn AccountsService> {
Arc::new(DefaultAccountsService::new(
accounts,
account_runtime,
settings,
timezone,
providers,
snapshot,
probe,
))
}
/// 设置服务不依赖 Web 管理会话,CLI 与服务器通过同一用例执行事务和发布
pub fn initialize_settings(
store: Arc<dyn ports::store::SettingsStore>,
snapshot: Arc<dyn SnapshotControl>,
providers: ProviderAdminRegistry,
pricing_source: Arc<dyn ports::pricing::PricingSource>,
) -> Arc<dyn SettingsService> {
Arc::new(DefaultSettingsService::new(
store,
snapshot,
providers,
pricing_source,
))
}
/// HTTP、插件及 CLI 共用的用量领域用例,宿主提供已验证的调用主体
#[must_use]
pub fn initialize_usage_data(
store: Arc<dyn ports::usage_data::UsageDataStore>,
settings: Arc<dyn ports::store::SettingsStore>,
providers: ProviderAdminRegistry,
) -> Arc<dyn UsageDataService> {
Arc::new(use_case::usage_data::DefaultUsageDataService::new(
store, settings, providers,
))
}
@@ -30,9 +30,10 @@ pub struct ProviderAccountCapabilities {
pub consume_reset_credit: bool,
}
/// 导入时统一应用的账号备注、调度与分组设置;缺省时保留原有导入语义
/// 导入时应用的账号设置;名称只覆盖单账号导入,其余设置统一应用
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct AccountImportSettings {
pub name: Option<String>,
pub notes: Option<String>,
pub enabled: bool,
pub concurrency_limit: Option<AccountConcurrencyLimit>,
@@ -226,6 +227,8 @@ pub struct AccountCapacity {
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct UpdateAccount {
pub account_id: String,
/// 缺省保留名称,空字符串清除名称并使用邮箱展示
pub name: Option<String>,
/// 缺省保留备注;空字符串清空备注
pub notes: Option<String>,
pub enabled: bool,
@@ -108,6 +108,7 @@ pub struct ClientKeyListQuery {
/// 不含完整明文 Key 的管理投影
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ClientKeyRecord {
pub peer_sharing: gateway_core::policy::ClientPeerSharing,
pub request_profile_overrides: ProviderRequestProfileOverrides,
pub id: ClientApiKeyId,
pub name: String,
@@ -167,6 +168,7 @@ impl fmt::Debug for ClientKeySecret {
/// API 提交的 Client Key 创建命令
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CreateClientKey {
pub peer_sharing: gateway_core::policy::ClientPeerSharing,
pub request_profile_overrides: ProviderRequestProfileOverrides,
pub custom_key: Option<PlaintextClientApiKey>,
pub name: String,
@@ -179,6 +181,7 @@ pub struct CreateClientKey {
/// 管理用例生成 ID 与明文后的持久化命令
#[derive(Clone, PartialEq, Eq)]
pub struct NewClientKey {
pub peer_sharing: gateway_core::policy::ClientPeerSharing,
pub request_profile_overrides: ProviderRequestProfileOverrides,
pub id: ClientApiKeyId,
pub name: String,
@@ -204,6 +207,7 @@ impl fmt::Debug for NewClientKey {
/// 修改 Client Key 的公开策略字段
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct UpdateClientKey {
pub peer_sharing: Option<gateway_core::policy::ClientPeerSharing>,
pub request_profile_override_updates: ProviderRequestProfileOverrideUpdates,
pub id: ClientApiKeyId,
pub name: String,
@@ -14,6 +14,7 @@ pub mod diagnostics;
pub mod import_tasks;
pub mod key_usage;
pub mod observability;
pub mod peer_link;
pub mod plugin_client_keys;
pub mod plugin_resources;
pub mod plugins;
@@ -26,6 +27,7 @@ pub mod reset_credit_automation;
pub mod retention;
pub mod settings;
pub mod system;
pub mod upstream_info;
pub mod usage_data;
/// 管理用例对外返回的稳定错误分类
@@ -648,6 +648,7 @@ pub struct AccountRequestBucket {
/// Dashboard 中一个账号的完整用量事实
#[derive(Debug, Clone, PartialEq)]
pub struct DashboardAccountUsage {
pub upstream: Option<super::upstream_info::ProviderUpstreamInfo>,
pub account_id: String,
pub provider_kind: String,
pub authentication_kind: String,
@@ -965,6 +966,7 @@ pub struct DiagnosticObservation {
pub key: String,
pub name: String,
pub account_provider_kind: Option<String>,
pub account_authentication_kind: Option<String>,
pub account_plan_type: Option<String>,
pub request_count: u64,
pub success_count: u64,
@@ -1435,6 +1437,7 @@ pub struct UsageInsight {
pub struct DiagnosticsItem {
pub key: String,
pub name: String,
pub account_authentication_kind: Option<String>,
pub account_plan_type: Option<String>,
pub account_plan_type_display: Option<String>,
pub request_count: u64,
@@ -0,0 +1,76 @@
//! 当前认证 Key 可公开的本地互联观测,不携带 HTTP 或 Provider 私有字段
use super::provider_credentials::ProviderQuotaWindowRole;
use chrono::{DateTime, Utc};
use gateway_core::engine::budget::{ClientBudgetAccess, ClientBudgetStatus};
/// 调用边界提供的响应资源预算,超限时整体拒绝而不截断授权范围
#[derive(Debug, Clone, Copy)]
pub struct PeerLinkLimits {
pub max_accounts: usize,
pub max_quota_windows: usize,
pub max_models: usize,
pub max_response_bytes: usize,
}
#[derive(Debug, Clone, PartialEq)]
pub struct PeerLinkExport {
pub observed_at: DateTime<Utc>,
pub expires_at: DateTime<Utc>,
pub budget: ClientBudgetStatus,
pub budget_access: ClientBudgetAccess,
pub budget_observed_at: DateTime<Utc>,
pub pool: Option<PeerPoolSummary>,
pub quota: Option<Vec<PeerAccountQuota>>,
pub models: Vec<PeerModelSummary>,
}
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct PeerPoolSummary {
pub ready: u32,
pub cooldown: u32,
pub exhausted: u32,
pub disabled: u32,
pub error: u32,
pub unknown: u32,
pub plans: Option<std::collections::BTreeMap<String, u32>>,
}
#[derive(Debug, Clone, PartialEq)]
pub struct PeerAccountQuota {
pub id: String,
pub models: Vec<String>,
pub observed_at: Option<DateTime<Utc>>,
pub expires_at: Option<DateTime<Utc>>,
pub windows: Vec<PeerQuotaWindow>,
}
#[derive(Debug, Clone, PartialEq)]
pub struct PeerQuotaWindow {
pub key: String,
pub group: String,
pub label: String,
pub limit_id: Option<String>,
pub limit_name: Option<String>,
pub role: Option<ProviderQuotaWindowRole>,
pub window_seconds: Option<u64>,
pub used_percent: Option<f64>,
pub reset_at: Option<DateTime<Utc>>,
pub limit_reached: bool,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum PeerModelState {
Available,
Exhausted,
Unknown,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PeerModelSummary {
pub model: String,
pub complete: bool,
pub state: PeerModelState,
pub observed_at: Option<DateTime<Utc>>,
pub expires_at: Option<DateTime<Utc>>,
}
@@ -790,6 +790,7 @@ impl ProviderQuotaWindowRole {
/// 一个 Provider quota 窗口的公共投影
#[derive(Debug, Clone, PartialEq)]
pub struct ProviderQuotaWindow {
pub upstream: Option<super::upstream_info::UpstreamQuotaWindow>,
pub key: String,
pub group: String,
pub label: String,
@@ -828,8 +829,9 @@ pub struct ProviderQuotaReadRequest<'a> {
}
/// Provider 已解析的 quota 结果及其不透明差异字段
#[derive(Debug, Clone, PartialEq)]
#[derive(Debug, Default, Clone, PartialEq)]
pub struct ProviderQuota {
pub upstream: Option<super::upstream_info::ProviderUpstreamInfo>,
/// 上游额度响应明确提供的套餐,可用于补全账号展示
pub plan_type: Option<String>,
pub observed_at: Option<DateTime<Utc>>,
@@ -1008,6 +1010,21 @@ impl ProviderQuota {
.min_by_key(|(_, period)| *period)
}
/// 互联账号按最长的 Key 消费周期统计本地用量,池内账号窗口不代表此 Key 的周期
#[must_use]
pub fn upstream_usage_window(&self) -> Option<&ProviderQuotaWindow> {
self.windows
.iter()
.filter(|window| {
window
.upstream
.as_ref()
.is_some_and(|upstream| upstream.amount.is_some())
&& window.window_seconds.is_some_and(|seconds| seconds > 0)
})
.max_by_key(|window| window.window_seconds)
}
/// 统计与预测共用窗口归属规则,避免把模型专属桶或短期限流当作账号容量
pub(crate) fn usage_windows(
&self,
@@ -28,6 +28,7 @@ pub struct ProxyListQuery {
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ProxyAccountRef {
pub upstream: Option<super::upstream_info::ProviderUpstreamInfo>,
pub id: String,
pub name: String,
pub email: Option<String>,
@@ -54,6 +54,7 @@ pub struct RuntimeSettingsValues {
pub max_waiting_per_key: u32,
pub max_waiting_per_account: u32,
pub concurrency_wait_timeout_seconds: u32,
pub max_retry_after_seconds: u32,
pub openai_guardian_reserved_concurrency: u32,
pub openai_account_affinity: gateway_core::account::AccountAffinity,
pub max_account_rotations: u32,
@@ -0,0 +1,52 @@
//! 管理端互联快照的中立展示事实,不携带 Provider 原始协议
use chrono::{DateTime, Utc};
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ProviderUpstreamInfo {
pub discovered: bool,
pub observed_at: Option<DateTime<Utc>>,
pub stale: bool,
pub connection_revision: String,
pub last_error: Option<String>,
pub pool: Option<UpstreamPool>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct UpstreamPool {
pub account_count: u32,
pub states: UpstreamPoolStates,
pub plans: Vec<UpstreamPlanCount>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct UpstreamPoolStates {
pub ready: u32,
pub cooldown: u32,
pub exhausted: u32,
pub disabled: u32,
pub error: u32,
pub unknown: u32,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct UpstreamPlanCount {
pub name: String,
pub count: u32,
}
#[derive(Debug, Clone, PartialEq)]
pub struct UpstreamQuotaWindow {
pub account_id: Option<String>,
pub observed_at: Option<DateTime<Utc>>,
pub stale: bool,
pub amount: Option<UpstreamQuotaAmount>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct UpstreamQuotaAmount {
pub unit: String,
pub used: String,
pub remaining: Option<String>,
pub total: Option<String>,
}
@@ -2,6 +2,7 @@
pub mod backup;
pub mod client_distribution;
pub mod peer;
pub mod plugin_accounts;
pub mod plugin_client_keys;
pub mod plugin_management;
@@ -0,0 +1,77 @@
//! CPR 连接材料与传输端口;Provider 解释凭据,Host 执行受限网络请求
use std::time::Duration;
use async_trait::async_trait;
use gateway_core::account::{ProviderAccount, UpstreamInfoFailure};
use secrecy::SecretString;
/// 仅供后台请求使用,不进入日志、持久观测或管理响应
pub struct PeerConnection {
pub base_url: String,
pub bearer_token: SecretString,
}
pub struct PeerFetchFailure {
pub kind: UpstreamInfoFailure,
pub retry_after: Option<Duration>,
}
impl From<UpstreamInfoFailure> for PeerFetchFailure {
fn from(kind: UpstreamInfoFailure) -> Self {
Self {
kind,
retry_after: None,
}
}
}
#[async_trait]
pub trait PeerTransport: Send + Sync {
async fn fetch(
&self,
account: &ProviderAccount,
connection: &PeerConnection,
) -> Result<Vec<u8>, PeerFetchFailure>;
}
/// 账号用例读取公共互联观测,显式刷新等待额度查询完成
#[async_trait]
pub trait PeerUpstreamService: gateway_core::account::UpstreamInfoSource {
async fn account_changed(
&self,
id: &gateway_core::account::ProviderAccountId,
) -> Result<(), gateway_core::error::StoreError>;
fn forget(&self, id: &gateway_core::account::ProviderAccountId);
async fn refresh_quota(
&self,
id: &gateway_core::account::ProviderAccountId,
) -> Result<
crate::model::provider_credentials::ProviderQuota,
super::provider::ProviderAdminError,
>;
async fn read_quota(
&self,
account_id: &gateway_core::account::ProviderAccountId,
) -> Result<
Option<crate::model::provider_credentials::ProviderQuota>,
gateway_core::error::StoreError,
>;
/// 读取版本化摘要,不展开远端节点窗口
async fn read_summary(
&self,
account_id: &gateway_core::account::ProviderAccountId,
) -> Result<
Option<crate::model::provider_credentials::ProviderQuota>,
gateway_core::error::StoreError,
>;
async fn read_quotas(
&self,
requests: &[crate::model::provider_credentials::ProviderQuotaReadRequest<'_>],
) -> Vec<
Result<
crate::model::provider_credentials::ProviderQuota,
super::provider::ProviderAdminError,
>,
>;
}
@@ -179,6 +179,19 @@ pub trait ProviderAdmin: Send + Sync {
}
fn provider_kind(&self) -> &ProviderKind;
/// 与执行 Provider 共用的直连资格,未知实现默认不公开账号
fn is_direct_account(&self, _authentication_kind: &str) -> bool {
false
}
/// 仅依据本地账号事实判断通用模型资格;未知实现不能宣称完整覆盖
fn peer_model_eligible(
&self,
_account: &crate::model::accounts::AccountRecord,
_model: &str,
) -> Option<bool> {
None
}
/// 只读取原生 Provider 能力及账号类型,不执行网络或数据库查询
fn account_capabilities(
@@ -309,6 +322,24 @@ pub trait ProviderAdmin: Send + Sync {
.await
}
/// 仅依据账号类型声明 CPR 连接能力,通用用例不解析 Provider 凭据
fn peer_authentication_kinds(&self) -> &[&str] {
&[]
}
fn supports_peer_connection(&self, authentication_kind: &str) -> bool {
self.peer_authentication_kinds()
.contains(&authentication_kind)
}
/// 从本轮加载的凭据提取连接材料,避免在校验连接版本后另读一份凭据
fn peer_connection(
&self,
_loaded: &gateway_core::account::LoadedCredential,
) -> Result<Option<super::peer::PeerConnection>, ProviderAdminError> {
Ok(None)
}
/// 历史观测的协议字段仅由具体 Provider 解释;不支持时保留累计估算
fn quota_forecast_observation(
&self,
@@ -385,6 +416,10 @@ pub struct ProviderAdminRegistry {
}
impl ProviderAdminRegistry {
pub(crate) fn registered(&self) -> BTreeMap<ProviderKind, Arc<dyn ProviderAdmin>> {
self.providers.as_ref().clone()
}
pub fn pricing_catalog(&self) -> gateway_core::metering::PricingOverrides {
self.providers
.iter()
@@ -131,8 +131,43 @@ pub trait AccountStore:
runtime: AccountRuntimeSnapshot,
) -> AdminStoreResult<Option<AccountPageItem>>;
/// 批量读取指定账号与当前启用分组的交集,分组只用于授权筛选,不加载展示信息
/// 不加载凭据正文或触发 Provider 查询
async fn load_accounts_in_enabled_groups(
&self,
account_ids: &[gateway_core::account::ProviderAccountId],
group_ids: &[gateway_core::routing::AccountGroupId],
) -> AdminStoreResult<Vec<crate::model::accounts::AccountRecord>> {
if group_ids.is_empty() {
return Ok(Vec::new());
}
let mut accounts = Vec::with_capacity(account_ids.len());
for id in account_ids {
if let Some(item) = self
.load_account(id.as_str(), AccountRuntimeSnapshot::default())
.await?
&& item
.account
.groups
.iter()
.any(|group| group.enabled && group_ids.contains(&group.id))
{
let mut account = item.account;
account.groups.clear();
accounts.push(account);
}
}
Ok(accounts)
}
async fn account_exists(&self, account_id: &str) -> AdminStoreResult<bool>;
/// 只读取账号记录,不加载运行配置、分组或额度展示
async fn load_account_record(
&self,
account_id: &gateway_core::account::ProviderAccountId,
) -> AdminStoreResult<Option<crate::model::accounts::AccountRecord>>;
async fn load_account_usage(
&self,
range: TimeRange,
@@ -252,6 +287,14 @@ pub trait AccountRuntimeStore: Send + Sync {
account_ids: &[String],
) -> AdminStoreResult<AccountRuntimeSnapshot>;
/// 只需账号可用状态时不读取在途租约,避免池规模放大容量查询
async fn account_rate_limits(
&self,
account_ids: &[String],
) -> AdminStoreResult<AccountRuntimeSnapshot> {
self.account_runtime(account_ids).await
}
/// 账号错误保护中的账号与其冻结快照;临时限流不包含在内
async fn active_freezes(
&self,
@@ -1,125 +0,0 @@
//! 每日错峰发现与已知卡片的临期执行,周期只读取本地到期计划
use crate::{
model::reset_credit_automation::{PlannedResetCredit, ResetCreditAutomation},
ports::store::{AccountStore, SettingsStore},
use_case::accounts::AccountsService,
};
use chrono::{DateTime, Utc};
use futures::StreamExt as _;
use gateway_core::{
account::ProviderAccountId,
task::{ScheduledTask, WorkerCycleContext, WorkerTaskError},
time::DeploymentTimeZone,
};
use sha2::{Digest as _, Sha256};
use std::{sync::Arc, time::Duration};
/// 当地次日随机发现时间;随机数由调用者提供,结果持久化后不会因重启重抽
pub(crate) fn next_discovery(
now: DateTime<Utc>,
timezone: DeploymentTimeZone,
random: u64,
) -> Option<DateTime<Utc>> {
let start = timezone.days_after(now, 1)?;
let end = timezone.days_after(now, 2)?;
let seconds = u64::try_from((end - start).num_seconds()).ok()?;
if seconds == 0 {
return None;
}
start.checked_add_signed(chrono::Duration::seconds(
i64::try_from(random % seconds).ok()?,
))
}
/// 提前窗口的前 10% 用于账号错峰,最多一分钟,其余时间保留给网络失败恢复
pub(crate) fn consume_at(
account_id: &str,
credit: &PlannedResetCredit,
lead_minutes: u32,
) -> DateTime<Utc> {
let lead_seconds = i64::from(lead_minutes) * 60;
let mut hash = Sha256::new();
hash.update(account_id.as_bytes());
hash.update([0]);
hash.update(credit.id.as_bytes());
let digest = hash.finalize();
let offset =
i64::from(u16::from_be_bytes([digest[0], digest[1]])) % (lead_seconds / 10).clamp(1, 60);
credit.expires_at - chrono::Duration::seconds(lead_seconds - offset)
}
pub(crate) fn next_consumption(
record: &ResetCreditAutomation,
lead_minutes: u32,
) -> Option<DateTime<Utc>> {
record
.state
.credits
.iter()
.map(|credit| consume_at(&record.account_id, credit, lead_minutes))
.min()
}
pub(crate) fn reschedule(
record: &mut ResetCreditAutomation,
lead_minutes: u32,
now: DateTime<Utc>,
) {
record.next_run_at = if let Some(retry) = record.state.retry_at {
retry
} else if record.state.pending.is_some() {
now
} else {
next_consumption(record, lead_minutes).map_or(record.state.next_discovery_at, |due| {
due.min(record.state.next_discovery_at)
})
};
}
pub(crate) struct ResetCreditAutomationTask {
pub(crate) accounts: Arc<dyn AccountsService>,
pub(crate) store: Arc<dyn AccountStore>,
pub(crate) settings: Arc<dyn SettingsStore>,
}
impl ScheduledTask for ResetCreditAutomationTask {
fn run_cycle(
&self,
context: WorkerCycleContext,
) -> futures::future::BoxFuture<'_, Result<(), WorkerTaskError>> {
Box::pin(async move {
let settings = self
.settings
.load_runtime_settings()
.await
.map_err(|_| WorkerTaskError::safe("重置卡设置读取失败"))?;
// 单轮有界,积压沿到期顺序继续;并发复用已有账号刷新配置
let due = self
.store
.due_reset_credit_automations(Utc::now(), 100)
.await
.map_err(|_| WorkerTaskError::safe("重置卡计划读取失败"))?;
let work = futures::stream::iter(due)
.map(|account_id| async move {
let id = ProviderAccountId::new(account_id.clone())
.map_err(|_| WorkerTaskError::safe("重置卡账号 ID 不合法"))?;
if let Err(error) = self.accounts.reconcile_reset_credit_automation(id).await {
tracing::warn!(
account_id,
error_kind = error.kind().as_str(),
"自动重置卡任务失败"
);
}
Ok::<_, WorkerTaskError>(())
})
.buffer_unordered(settings.values.refresh_concurrency.max(1) as usize)
.collect::<Vec<_>>();
tokio::select! {
() = context.cancellation().cancelled() => {},
_ = tokio::time::timeout(Duration::from_secs(30), work) => {},
}
Ok(())
})
}
}
@@ -0,0 +1,91 @@
//! 自动重置卡 Worker 的到期扫描、并发预算与取消,消费编排由账号用例负责
use crate::{
model::AdminError,
ports::store::{AccountStore, SettingsStore},
use_case::accounts::AccountsService,
};
use chrono::Utc;
use futures::StreamExt as _;
use gateway_core::{
account::ProviderAccountId,
task::{
ScheduledTask, WorkerContribution, WorkerCycleContext, WorkerId, WorkerKind,
WorkerLeaseRequest, WorkerRegistration, WorkerRunnable, WorkerSchedule, WorkerTaskError,
},
};
use std::{sync::Arc, time::Duration};
pub(crate) struct ResetCreditAutomationTask {
pub(crate) accounts: Arc<dyn AccountsService>,
pub(crate) store: Arc<dyn AccountStore>,
pub(crate) settings: Arc<dyn SettingsStore>,
}
impl ResetCreditAutomationTask {
pub(crate) fn into_worker(self) -> Result<WorkerContribution, AdminError> {
let id = WorkerId::try_new(WorkerKind::ResetCreditAutomation, "admin")
.map_err(|_| AdminError::internal("重置卡 Worker ID 不合法"))?;
let schedule = WorkerSchedule::try_new(
Duration::from_secs(5),
Duration::from_secs(5),
Duration::from_secs(60),
Duration::from_secs(90),
Duration::from_secs(30),
)
.map_err(|_| AdminError::internal("重置卡 Worker 调度配置不合法"))?;
let lease = WorkerLeaseRequest::try_new(id.clone(), Duration::from_secs(90))
.map_err(|_| AdminError::internal("重置卡 Worker 租约配置不合法"))?;
let registration = WorkerRegistration::try_new(
id,
WorkerRunnable::Scheduled {
schedule,
lease: Some(lease),
task: Box::new(self),
},
)
.map_err(|_| AdminError::internal("重置卡 Worker 注册失败"))?;
Ok(WorkerContribution::Registration(registration))
}
}
impl ScheduledTask for ResetCreditAutomationTask {
fn run_cycle(
&self,
context: WorkerCycleContext,
) -> futures::future::BoxFuture<'_, Result<(), WorkerTaskError>> {
Box::pin(async move {
let settings = self
.settings
.load_runtime_settings()
.await
.map_err(|_| WorkerTaskError::safe("重置卡设置读取失败"))?;
// 单轮有界,积压沿到期顺序继续;并发复用已有账号刷新配置
let due = self
.store
.due_reset_credit_automations(Utc::now(), 100)
.await
.map_err(|_| WorkerTaskError::safe("重置卡计划读取失败"))?;
let work = futures::stream::iter(due)
.map(|account_id| async move {
let id = ProviderAccountId::new(account_id.clone())
.map_err(|_| WorkerTaskError::safe("重置卡账号 ID 不合法"))?;
if let Err(error) = self.accounts.reconcile_reset_credit_automation(id).await {
tracing::warn!(
account_id,
error_kind = error.kind().as_str(),
"自动重置卡任务失败"
);
}
Ok::<_, WorkerTaskError>(())
})
.buffer_unordered(settings.values.refresh_concurrency.max(1) as usize)
.collect::<Vec<_>>();
tokio::select! {
() = context.cancellation().cancelled() => {},
_ = tokio::time::timeout(Duration::from_secs(30), work) => {},
}
Ok(())
})
}
}
@@ -191,12 +191,17 @@ pub(crate) struct DefaultAccountsService {
providers: ProviderAdminRegistry,
snapshot: Arc<dyn SnapshotControl>,
probe: Arc<dyn AccountProbe>,
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
reset_credit_locks:
Arc<futures::lock::Mutex<BTreeMap<ProviderAccountId, Arc<futures::lock::Mutex<()>>>>>,
}
impl DefaultAccountsService {
#[must_use]
#[expect(
clippy::too_many_arguments,
reason = "账号存储、发布、探测与互联分别通过独立端口注入"
)]
pub(crate) fn new(
accounts: Arc<dyn AccountStore>,
account_runtime: Arc<dyn AccountRuntimeStore>,
@@ -205,6 +210,7 @@ impl DefaultAccountsService {
providers: ProviderAdminRegistry,
snapshot: Arc<dyn SnapshotControl>,
probe: Arc<dyn AccountProbe>,
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
) -> Self {
Self {
accounts,
@@ -214,6 +220,7 @@ impl DefaultAccountsService {
providers,
snapshot,
probe,
upstream_info,
reset_credit_locks: Arc::new(futures::lock::Mutex::new(BTreeMap::new())),
}
}
@@ -231,10 +238,11 @@ impl DefaultAccountsService {
}
async fn read_quotas(
providers: &ProviderAdminRegistry,
&self,
items: &[AccountPageItem],
rolling_usage: &BTreeMap<String, AccountUsage>,
) -> Vec<ProviderQuota> {
let providers = &self.providers;
let mut groups = BTreeMap::<_, Vec<usize>>::new();
for (index, item) in items.iter().enumerate() {
groups
@@ -264,7 +272,12 @@ impl DefaultAccountsService {
}
})
.collect::<Vec<_>>();
let results = provider.read_quotas(&requests).await;
let results = super::peer_link::upstream::read_quotas(
self.upstream_info.as_deref(),
provider.as_ref(),
&requests,
)
.await;
if results.len() != indices.len() {
tracing::warn!(
provider = %kind,
@@ -381,32 +394,51 @@ impl DefaultAccountsService {
async fn load_api_key_usage(
&self,
accounts: &[AccountPageItem],
quotas: &[ProviderQuota],
) -> Result<BTreeMap<String, AccountUsage>, AdminError> {
let now = Utc::now();
// API Key 没有套餐周期;本地累计直接查询账号创建后仍保留的请求记录
let mut usage = BTreeMap::new();
// 互联账号复用已聚合的最长 Key 窗口,普通 API Key 保留累计口径
let windows = accounts
.iter()
.filter(|item| item.account.authentication_kind == "api_key")
.map(|item| AccountUsageWindowQuery {
account_id: item.account.id.clone(),
key: "account-lifetime".to_owned(),
range: TimeRange {
start: item.account.created_at,
end: now,
},
.zip(quotas)
.filter(|(item, _)| item.account.authentication_kind == "api_key")
.filter_map(|(item, quota)| {
if quota
.upstream
.as_ref()
.is_some_and(|upstream| upstream.discovered)
{
if let Some(local_usage) = quota
.upstream_usage_window()
.and_then(|window| window.local_usage.as_ref())
{
usage.insert(item.account.id.clone(), local_usage.clone());
}
return None;
}
Some(AccountUsageWindowQuery {
account_id: item.account.id.clone(),
key: "account-lifetime".to_owned(),
range: TimeRange {
start: item.account.created_at,
end: now,
},
})
})
.collect::<Vec<_>>();
if windows.is_empty() {
return Ok(BTreeMap::new());
return Ok(usage);
}
Ok(self
.accounts
.load_account_usage_by_windows(&windows)
.await
.map_err(|error| map_store_error(error, "API Key account usage"))?
.into_iter()
.map(|result| (result.account_id, result.usage))
.collect())
usage.extend(
self.accounts
.load_account_usage_by_windows(&windows)
.await
.map_err(|error| map_store_error(error, "API Key account usage"))?
.into_iter()
.map(|result| (result.account_id, result.usage)),
);
Ok(usage)
}
async fn load_directory_item(
@@ -428,13 +460,17 @@ impl DefaultAccountsService {
.await
.map_err(|error| map_store_error(error, "rolling account usage"))?;
let rolling_usage = rolling_usage.into_iter().next();
let mut quota = match provider
.quota(ProviderQuotaRequest {
let mut quota = match super::peer_link::upstream::read_quota(
self.upstream_info.as_deref(),
provider.as_ref(),
account,
ProviderQuotaRequest {
account_id: account_id.clone(),
refresh: refresh_quota,
rolling_usage: rolling_usage.clone(),
})
.await
},
)
.await
{
Ok(quota) => quota,
// 凭据更新后的账号投影不要求 Provider 提供额度;显式刷新仍须支持该操作
@@ -458,7 +494,7 @@ impl DefaultAccountsService {
)
.await?;
let usage = self
.load_api_key_usage(std::slice::from_ref(&stored))
.load_api_key_usage(std::slice::from_ref(&stored), std::slice::from_ref(&quota))
.await?
.remove(&stored.account.id)
.or_else(|| {
@@ -554,10 +590,10 @@ impl AccountsService for DefaultAccountsService {
.into_iter()
.map(|usage| (usage.account_id.clone(), usage))
.collect::<BTreeMap<_, _>>();
let mut quotas = Self::read_quotas(providers, &page.items, &rolling_usage).await;
let mut quotas = self.read_quotas(&page.items, &rolling_usage).await;
self.attach_quota_local_usage(&page.items, &mut quotas)
.await?;
let mut api_key_usage = self.load_api_key_usage(&page.items).await?;
let mut api_key_usage = self.load_api_key_usage(&page.items, &quotas).await?;
let items = page
.items
.into_iter()
@@ -684,9 +720,12 @@ impl AccountsService for DefaultAccountsService {
"provider credential refresh",
)
.await?;
provider
.account_facts_changed(std::slice::from_ref(&result.account_id))
.await;
super::notify_account_facts_changed(
self.upstream_info.as_deref(),
provider.as_ref(),
std::slice::from_ref(&result.account_id),
)
.await;
publish_committed(self.snapshot.as_ref(), result.config_revision).await?;
let account = self.load_directory_item(&result.account_id, false).await?;
Ok(AccountRefreshResult {
@@ -726,9 +765,12 @@ impl AccountsService for DefaultAccountsService {
.map_err(|error| map_store_error(error, "enable provider account"))?
.config_revision
};
provider
.account_facts_changed(std::slice::from_ref(&account_id))
.await;
super::notify_account_facts_changed(
self.upstream_info.as_deref(),
provider.as_ref(),
std::slice::from_ref(&account_id),
)
.await;
publish_committed(self.snapshot.as_ref(), config_revision).await?;
let account = self.load_directory_item(&account_id, false).await?;
Ok(AccountRefreshResult {
@@ -752,11 +794,19 @@ impl AccountsService for DefaultAccountsService {
.await
.map_err(|error| map_store_error(error, "provider account"))?;
if !enabled {
provider.account_unavailable(&account_id).await;
}
provider
.account_facts_changed(std::slice::from_ref(&account_id))
super::notify_account_unavailable(
self.upstream_info.as_deref(),
provider.as_ref(),
&account_id,
)
.await;
}
super::notify_account_facts_changed(
self.upstream_info.as_deref(),
provider.as_ref(),
std::slice::from_ref(&account_id),
)
.await;
publish_committed(self.snapshot.as_ref(), result.config_revision).await?;
Ok(result)
}
@@ -798,10 +848,20 @@ impl AccountsService for DefaultAccountsService {
for (provider, provider_ids) in providers.values() {
if enabled == Some(false) {
for account_id in provider_ids {
provider.account_unavailable(account_id).await;
super::notify_account_unavailable(
self.upstream_info.as_deref(),
provider.as_ref(),
account_id,
)
.await;
}
}
provider.account_facts_changed(provider_ids).await;
super::notify_account_facts_changed(
self.upstream_info.as_deref(),
provider.as_ref(),
provider_ids,
)
.await;
}
publish_committed(self.snapshot.as_ref(), result.config_revision).await?;
Ok(result)
@@ -831,14 +891,18 @@ impl AccountsService for DefaultAccountsService {
account_id: &ProviderAccountId,
) -> Result<AccountQuotaForecastReport, AdminError> {
let (stored, provider) = self.provider_for_account(account_id).await?;
let quota = provider
.quota(ProviderQuotaRequest {
let quota = super::peer_link::upstream::read_quota(
self.upstream_info.as_deref(),
provider.as_ref(),
&stored.account,
ProviderQuotaRequest {
account_id: account_id.clone(),
refresh: false,
rolling_usage: None,
})
.await
.map_err(|error| map_provider_error(error, "forecast quota snapshot"))?;
},
)
.await
.map_err(|error| map_provider_error(error, "forecast quota snapshot"))?;
let now = Utc::now();
let mut samples = Vec::new();
let mut selected_keys = BTreeSet::new();
@@ -1143,15 +1207,7 @@ fn map_reset_credits_error_after_refresh(
/// 账号目录中单个账号 quota 读取失败时使用的空额度投影
fn empty_quota() -> ProviderQuota {
ProviderQuota {
credits: None,
plan_type: None,
observed_at: None,
refresh_token_expires_at: None,
windows: Vec::new(),
limit_reached: false,
provider_data: None,
}
ProviderQuota::default()
}
fn quota_usage_window(
@@ -1,7 +1,13 @@
//! 重置卡自动使用的账号锁、持久幂等键、查询缓存与重试编排
//! 重置卡查询、消费与自动使用排程,统一账号锁、持久幂等键和重试编排
use super::*;
use crate::model::{MutationActor, reset_credit_automation::*};
use chrono::{DateTime, Utc};
use gateway_core::time::DeploymentTimeZone;
use rand_core::{OsRng, RngCore as _};
use sha2::{Digest as _, Sha256};
impl DefaultAccountsService {
pub(super) async fn reset_credits_upstream(
&self,
@@ -51,13 +57,6 @@ impl DefaultAccountsService {
}
}
use crate::{
model::{MutationActor, reset_credit_automation::*},
reset_credit_automation::{consume_at, next_consumption, next_discovery, reschedule},
};
use chrono::{DateTime, Utc};
use rand_core::{OsRng, RngCore as _};
impl DefaultAccountsService {
async fn automation_settings(
&self,
@@ -511,3 +510,54 @@ impl DefaultAccountsService {
Ok(())
}
}
/// 当地次日随机发现时间;随机数由调用者提供,结果持久化后不会因重启重抽
fn next_discovery(
now: DateTime<Utc>,
timezone: DeploymentTimeZone,
random: u64,
) -> Option<DateTime<Utc>> {
let start = timezone.days_after(now, 1)?;
let end = timezone.days_after(now, 2)?;
let seconds = u64::try_from((end - start).num_seconds()).ok()?;
if seconds == 0 {
return None;
}
start.checked_add_signed(chrono::Duration::seconds(
i64::try_from(random % seconds).ok()?,
))
}
/// 提前窗口的前 10% 用于账号错峰,最多一分钟,其余时间保留给网络失败恢复
fn consume_at(account_id: &str, credit: &PlannedResetCredit, lead_minutes: u32) -> DateTime<Utc> {
let lead_seconds = i64::from(lead_minutes) * 60;
let mut hash = Sha256::new();
hash.update(account_id.as_bytes());
hash.update([0]);
hash.update(credit.id.as_bytes());
let digest = hash.finalize();
let offset =
i64::from(u16::from_be_bytes([digest[0], digest[1]])) % (lead_seconds / 10).clamp(1, 60);
credit.expires_at - chrono::Duration::seconds(lead_seconds - offset)
}
fn next_consumption(record: &ResetCreditAutomation, lead_minutes: u32) -> Option<DateTime<Utc>> {
record
.state
.credits
.iter()
.map(|credit| consume_at(&record.account_id, credit, lead_minutes))
.min()
}
fn reschedule(record: &mut ResetCreditAutomation, lead_minutes: u32, now: DateTime<Utc>) {
record.next_run_at = if let Some(retry) = record.state.retry_at {
retry
} else if record.state.pending.is_some() {
now
} else {
next_consumption(record, lead_minutes).map_or(record.state.next_discovery_at, |due| {
due.min(record.state.next_discovery_at)
})
};
}
@@ -165,6 +165,7 @@ impl ClientKeyService for DefaultClientKeyService {
.and_then(|provider| provider.preview_client_profile(profile))
.map_err(|error| super::map_provider_error(error, "client profile"))?;
}
validate_peer_sharing(command.peer_sharing, &command.group_ids)?;
let id = ClientApiKeyId::new(format!("key_{}", Uuid::now_v7().simple()))
.map_err(|_| AdminError::internal("创建 Client API Key ID 失败"))?;
let plaintext = if let Some(key) = command.custom_key {
@@ -176,6 +177,7 @@ impl ClientKeyService for DefaultClientKeyService {
.store
.create_client_key(
NewClientKey {
peer_sharing: command.peer_sharing,
request_profile_overrides: command.request_profile_overrides,
id,
name: command.name,
@@ -211,6 +213,9 @@ impl ClientKeyService for DefaultClientKeyService {
.map_err(|error| super::map_provider_error(error, "client profile"))?;
}
}
if let Some(sharing) = command.peer_sharing {
validate_peer_sharing(sharing, &command.group_ids)?;
}
let id = command.id.clone();
let (config_revision, record) = self
.store
@@ -308,3 +313,16 @@ pub(super) fn generate_key() -> String {
OsRng.fill_bytes(&mut bytes);
format!("sk_{}", URL_SAFE_NO_PAD.encode(bytes))
}
fn validate_peer_sharing(
sharing: gateway_core::policy::ClientPeerSharing,
groups: &[gateway_core::routing::AccountGroupId],
) -> Result<(), AdminError> {
sharing.validate(!groups.is_empty()).map_err(|error| {
use gateway_core::policy::ClientPeerSharingError;
AdminError::invalid(match error {
ClientPeerSharingError::MissingGroupScope => "开启 CPR 互联需要指定账号分组",
ClientPeerSharingError::PlansWithoutPool => "公开套餐分布需要先公开账号池",
})
})
}
@@ -19,29 +19,54 @@ use gateway_core::{account::ProviderAccountId, routing::ProviderKind, runtime::S
use super::{map_store_error, publish_committed};
pub(super) async fn publish_credentials_and_observe_quota(
peer: Option<&std::sync::Arc<dyn crate::ports::peer::PeerUpstreamService>>,
provider: &std::sync::Arc<dyn crate::ports::provider::ProviderAdmin>,
accounts: &std::sync::Arc<dyn AccountStore>,
snapshot: &dyn SnapshotControl,
revision: crate::model::Revision,
account_ids: &[ProviderAccountId],
request_id: &str,
) -> Result<(), AdminError> {
provider.account_facts_changed(account_ids).await;
notify_account_facts_changed(peer.map(AsRef::as_ref), provider.as_ref(), account_ids).await;
publish_committed(snapshot, revision).await?;
// 凭据已经提交;额度只是可重建的观察,不能拖住管理请求或回滚提交结果
let provider = provider.clone();
let peer = peer.cloned();
let accounts = accounts.clone();
let account_ids = account_ids.to_vec();
let request_id = request_id.to_owned();
tokio::spawn(async move {
for account_id in account_ids {
if let Err(error) = provider
.quota(ProviderQuotaRequest {
account_id: account_id.clone(),
refresh: true,
rolling_usage: None,
})
let request = ProviderQuotaRequest {
account_id: account_id.clone(),
refresh: true,
rolling_usage: None,
};
let result = if let Some(peer) = peer.as_deref() {
let account = match accounts.load_account_record(&account_id).await {
Ok(Some(account)) => account,
Ok(None) => continue,
Err(error) => {
tracing::warn!(request_id, account_id = %account_id, error = %error, "initial quota account lookup failed");
continue;
}
};
if !account.enabled || &account.provider_kind != provider.provider_kind() {
continue;
}
// 首查与手动刷新共用认证能力分派,互联协议由 Admin 解释
super::peer_link::upstream::read_quota(
Some(peer),
provider.as_ref(),
&account,
request,
)
.await
{
} else {
provider.quota(request).await
};
if let Err(error) = result {
tracing::warn!(
request_id,
provider = %provider.provider_kind().as_str(),
@@ -343,6 +368,7 @@ pub(super) async fn commit_credential_refresh(
}
pub(super) async fn delete_credentials(
peer: Option<&dyn crate::ports::peer::PeerUpstreamService>,
accounts: &dyn AccountStore,
provider: &dyn ProviderAdmin,
command: CredentialDeletion,
@@ -365,11 +391,37 @@ pub(super) async fn delete_credentials(
.await
.map_err(|error| map_store_error(error, resource))?;
for account_id in &account_ids {
provider.account_unavailable(account_id).await;
notify_account_unavailable(peer, provider, account_id).await;
}
provider.account_facts_changed(&account_ids).await;
notify_account_facts_changed(peer, provider, &account_ids).await;
Ok(CredentialDeletionResult {
config_revision: revision,
account_ids,
})
}
pub(super) async fn notify_account_facts_changed(
peer: Option<&dyn crate::ports::peer::PeerUpstreamService>,
provider: &dyn ProviderAdmin,
ids: &[ProviderAccountId],
) {
provider.account_facts_changed(ids).await;
if let Some(peer) = peer {
for id in ids {
if let Err(error) = peer.account_changed(id).await {
tracing::warn!(account_id = %id, error = %error, "upstream information cache invalidation failed");
}
}
}
}
pub(super) async fn notify_account_unavailable(
peer: Option<&dyn crate::ports::peer::PeerUpstreamService>,
provider: &dyn ProviderAdmin,
id: &ProviderAccountId,
) {
if let Some(peer) = peer {
peer.forget(id);
}
provider.account_unavailable(id).await;
}
@@ -30,6 +30,7 @@ pub struct ProviderCredentials {
accounts: Arc<dyn AccountStore>,
proxies: Arc<dyn crate::ports::proxy::ProxyStore>,
snapshot: Arc<dyn SnapshotControl>,
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
}
impl ProviderCredentials {
@@ -39,12 +40,14 @@ impl ProviderCredentials {
accounts: Arc<dyn AccountStore>,
proxies: Arc<dyn crate::ports::proxy::ProxyStore>,
snapshot: Arc<dyn SnapshotControl>,
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
) -> Self {
Self {
provider,
accounts,
proxies,
snapshot,
upstream_info,
}
}
pub async fn import_document(
@@ -89,7 +92,9 @@ impl ProviderCredentials {
.map_err(|error| map_store_error(error, "provider credential import"))?;
drop(proxy_reservation);
publish_credentials_and_observe_quota(
self.upstream_info.as_ref(),
&self.provider,
&self.accounts,
self.snapshot.as_ref(),
result.config_revision,
&result.credential_ids,
@@ -160,11 +165,19 @@ impl ProviderCredentials {
)
.await?;
if disable_account {
self.provider.account_unavailable(&account_id).await;
}
self.provider
.account_facts_changed(std::slice::from_ref(&result.account_id))
super::super::notify_account_unavailable(
self.upstream_info.as_deref(),
self.provider.as_ref(),
&account_id,
)
.await;
}
super::super::notify_account_facts_changed(
self.upstream_info.as_deref(),
self.provider.as_ref(),
std::slice::from_ref(&result.account_id),
)
.await;
publish_committed(self.snapshot.as_ref(), result.config_revision).await?;
Ok(result)
}
@@ -174,6 +187,7 @@ impl ProviderCredentials {
command: CredentialDeletion,
) -> Result<CredentialDeletionResult, AdminError> {
let result = delete_credentials(
self.upstream_info.as_deref(),
self.accounts.as_ref(),
self.provider.as_ref(),
command,
@@ -236,7 +250,9 @@ impl super::CredentialsService {
let result = outcome.result;
if outcome.newly_committed {
publish_credentials_and_observe_quota(
self.upstream_info.as_ref(),
provider,
&self.accounts,
self.snapshot.as_ref(),
result.config_revision,
std::slice::from_ref(&result.account_id),
@@ -19,6 +19,7 @@ pub struct CredentialsService {
accounts: Arc<dyn AccountStore>,
proxies: Arc<dyn ProxyStore>,
snapshot: Arc<dyn SnapshotControl>,
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
}
impl CredentialsService {
@@ -28,12 +29,14 @@ impl CredentialsService {
accounts: Arc<dyn AccountStore>,
proxies: Arc<dyn ProxyStore>,
snapshot: Arc<dyn SnapshotControl>,
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
) -> Self {
Self {
providers,
accounts,
proxies,
snapshot,
upstream_info,
}
}
@@ -47,6 +50,7 @@ impl CredentialsService {
self.accounts.clone(),
self.proxies.clone(),
self.snapshot.clone(),
self.upstream_info.clone(),
))
}
}
@@ -9,7 +9,10 @@ use std::{
use chrono::Utc;
use futures::{FutureExt as _, StreamExt as _, future::BoxFuture, stream::FuturesUnordered};
use gateway_core::task::{DaemonTask, WorkerDaemonContext, WorkerTaskError};
use gateway_core::task::{
DaemonRestartPolicy, DaemonTask, WorkerContribution, WorkerDaemonContext, WorkerId, WorkerKind,
WorkerRegistration, WorkerRunnable, WorkerTaskError,
};
use tokio::{sync::Notify, time::Instant};
use uuid::Uuid;
@@ -328,6 +331,24 @@ impl ImportTasksService for DefaultImportTasksService {
pub(crate) struct ImportTaskWorker(pub Arc<DefaultImportTasksService>);
impl ImportTaskWorker {
pub(crate) fn into_worker(self) -> Result<WorkerContribution, AdminError> {
let id = WorkerId::try_new(WorkerKind::AccountImport, "admin")
.map_err(|_| AdminError::internal("导入 Worker ID 不合法"))?;
let restart = DaemonRestartPolicy::try_new(Duration::from_secs(1), Duration::from_secs(60))
.map_err(|_| AdminError::internal("导入 Worker 重启策略不合法"))?;
let registration = WorkerRegistration::try_new(
id,
WorkerRunnable::Daemon {
restart,
task: Box::new(self),
},
)
.map_err(|_| AdminError::internal("导入 Worker 注册信息不合法"))?;
Ok(WorkerContribution::Registration(registration))
}
}
impl DaemonTask for ImportTaskWorker {
fn run(&self, context: WorkerDaemonContext) -> BoxFuture<'_, Result<(), WorkerTaskError>> {
Box::pin(async move {
@@ -10,14 +10,15 @@ pub mod credentials;
pub mod import_tasks;
pub mod key_usage;
pub mod observability;
pub mod peer_link;
pub mod plugin_accounts;
pub mod plugin_client_keys;
pub mod plugin_resources;
pub(crate) mod plugin_update;
pub mod plugins;
pub mod proxies;
pub mod settings;
pub mod system;
pub(crate) mod system_preflight;
pub mod usage_data;
mod credential_mutation;
@@ -26,10 +27,10 @@ mod publication;
use credential_mutation::{
commit_authorization, commit_credential_refresh, commit_credential_rotation,
delete_credentials, import_proxy_binding, pending_authorization,
publish_credentials_and_observe_quota, required_credential, required_plugin_credential,
validate_authorization_commit, validate_prepared_import, validate_prepared_rotation,
validate_prepared_rotation_facts,
delete_credentials, import_proxy_binding, notify_account_facts_changed,
notify_account_unavailable, pending_authorization, publish_credentials_and_observe_quota,
required_credential, required_plugin_credential, validate_authorization_commit,
validate_prepared_import, validate_prepared_rotation, validate_prepared_rotation_facts,
};
use error::{map_provider_error, map_store_error};
use publication::publish_committed;
@@ -94,6 +94,7 @@ pub(crate) struct DefaultObservabilityService {
settings: Arc<dyn SettingsStore>,
providers: ProviderAdminRegistry,
system: Arc<dyn super::system::SystemService>,
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
}
impl DefaultObservabilityService {
@@ -105,6 +106,7 @@ impl DefaultObservabilityService {
providers: ProviderAdminRegistry,
timezone: gateway_core::time::DeploymentTimeZone,
system: Arc<dyn super::system::SystemService>,
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
) -> Self {
Self {
timezone,
@@ -113,6 +115,7 @@ impl DefaultObservabilityService {
settings,
providers,
system,
upstream_info,
}
}
@@ -344,6 +347,7 @@ impl ObservabilityService for DefaultObservabilityService {
DiagnosticsItem {
key: item.key,
name: item.name,
account_authentication_kind: item.account_authentication_kind,
account_plan_type_display: item.account_provider_kind.as_deref().and_then(
|provider| {
self.providers
@@ -932,6 +936,7 @@ impl DefaultObservabilityService {
.map(|usage| (usage.account_id.clone(), usage))
.collect::<HashMap<_, _>>();
let providers = self.providers.clone();
let peer = self.upstream_info.as_deref();
let quota_reads = accounts.iter().map(|account| {
let rolling_usage = rolling_usage.remove(&account.account_id);
let account_id = ProviderAccountId::new(account.account_id.clone()).ok();
@@ -942,6 +947,11 @@ impl DefaultObservabilityService {
let (Some(account_id), Some(provider)) = (account_id, provider) else {
return None;
};
if let Some(peer) = peer
&& let Ok(Some(quota)) = peer.read_summary(&account_id).await
{
return Some(quota);
}
provider
.quota(ProviderQuotaRequest {
account_id,
@@ -959,6 +969,7 @@ impl DefaultObservabilityService {
quota.apply_limit_reached_display();
account.quota_used_percent = quota.representative_used_percent();
account.quota_window = quota.representative_window().cloned();
account.upstream = quota.upstream;
}
}
}
@@ -0,0 +1,268 @@
//! CPR 互联只读取认证范围内的本地事实,不参与推理准入或上游刷新
mod projection;
mod rate_limit;
pub mod upstream;
use std::{collections::BTreeMap, sync::Arc};
use async_trait::async_trait;
use chrono::{TimeDelta, Utc};
use gateway_core::{engine::execution::AuthenticatedClient, policy::ClientPeerSharing};
use tokio::sync::Semaphore;
use crate::{
model::{
AdminError,
peer_link::{PeerLinkExport, PeerLinkLimits},
provider_credentials::{ProviderQuotaReadRequest, explicit_plan_type},
},
ports::{
provider::ProviderAdminRegistry,
store::{AccountRuntimeStore, AccountStore, ClientKeyStore},
},
};
use super::map_store_error;
const MAX_CONCURRENT_SNAPSHOTS: usize = 8;
/// 查询失败保持独立于推理错误,不改变账号或 Key 状态
#[derive(Debug, thiserror::Error)]
pub enum PeerLinkError {
#[error("client API key is unavailable")]
Unauthorized,
#[error("peer sharing is not allowed")]
Forbidden,
#[error("peer query rate limit reached")]
RateLimited { retry_after: u64 },
#[error("peer snapshot exceeds the supported bounds")]
SnapshotTooLarge,
#[error("peer snapshot is unavailable")]
Unavailable(#[from] AdminError),
}
#[async_trait]
pub trait PeerLinkService: Send + Sync {
async fn snapshot(
&self,
client: &AuthenticatedClient,
limits: PeerLinkLimits,
) -> Result<PeerLinkExport, PeerLinkError>;
}
pub(crate) struct DefaultPeerLinkService {
keys: Arc<dyn ClientKeyStore>,
accounts: Arc<dyn AccountStore>,
runtime: Arc<dyn AccountRuntimeStore>,
providers: ProviderAdminRegistry,
rate_limit: rate_limit::QueryRateLimit,
snapshot_slots: Semaphore,
}
impl DefaultPeerLinkService {
pub(crate) fn new(
keys: Arc<dyn ClientKeyStore>,
accounts: Arc<dyn AccountStore>,
runtime: Arc<dyn AccountRuntimeStore>,
providers: ProviderAdminRegistry,
) -> Self {
Self {
keys,
accounts,
runtime,
providers,
rate_limit: Default::default(),
snapshot_slots: Semaphore::new(MAX_CONCURRENT_SNAPSHOTS),
}
}
}
#[async_trait]
impl PeerLinkService for DefaultPeerLinkService {
async fn snapshot(
&self,
client: &AuthenticatedClient,
limits: PeerLinkLimits,
) -> Result<PeerLinkExport, PeerLinkError> {
self.rate_limit.acquire(client.policy().key_id())?;
let frozen_sharing = client.policy().peer_sharing();
// 当前权限只能收窄冻结范围,重查询在读取 Key 分组前拒绝且不排队
let _permit = if frozen_sharing.share_pool || frozen_sharing.share_quota {
Some(
self.snapshot_slots
.try_acquire()
.map_err(|_| PeerLinkError::RateLimited { retry_after: 1 })?,
)
} else {
None
};
let key = self
.keys
.get_client_key(client.policy().key_id())
.await
.map_err(|error| map_store_error(error, "client API key"))?
.filter(|key| key.enabled)
.ok_or(PeerLinkError::Unauthorized)?;
let budget_observed_at = Utc::now();
let sharing = intersect_sharing(frozen_sharing, key.peer_sharing);
if !sharing.enabled || key.groups.is_empty() {
return Err(PeerLinkError::Forbidden);
}
let now = Utc::now();
let mut result = PeerLinkExport {
observed_at: now,
expires_at: now + TimeDelta::minutes(5),
budget_access: key
.budget
.limits
.access(key.budget.daily_used_usd, key.budget.weekly_used_usd),
budget: key.budget,
budget_observed_at,
pool: None,
quota: None,
models: Vec::new(),
};
if !sharing.share_pool && !sharing.share_quota {
return Ok(result);
}
let ids = client
.policy()
.account_scope()
.allowed_account_ids()
.take(limits.max_accounts.saturating_add(1))
.cloned()
.collect::<Vec<_>>();
if ids.len() > limits.max_accounts {
return Err(PeerLinkError::SnapshotTooLarge);
}
let public_models = if sharing.share_quota {
client
.snapshot()
.public_models_for_scope(client.policy().account_scope())
} else {
Vec::new()
};
if public_models.len() > limits.max_models {
return Err(PeerLinkError::SnapshotTooLarge);
}
let groups = key
.groups
.iter()
.filter(|group| group.enabled)
.map(|group| group.id.clone())
.collect::<Vec<_>>();
let accounts = self
.accounts
.load_accounts_in_enabled_groups(&ids, &groups)
.await
.map_err(|error| map_store_error(error, "peer accounts"))?
.into_iter()
.filter(|account| {
self.providers
.require(&account.provider_kind)
.is_ok_and(|provider| provider.is_direct_account(&account.authentication_kind))
})
.collect::<Vec<_>>();
let account_ids = accounts
.iter()
.map(|account| account.id.clone())
.collect::<Vec<_>>();
let runtime = self
.runtime
.account_rate_limits(&account_ids)
.await
.map_err(|error| map_store_error(error, "peer account runtime"))?;
let mut quotas = BTreeMap::new();
if sharing.share_quota || sharing.share_plans {
let mut window_count = 0usize;
let mut by_provider = BTreeMap::new();
for account in &accounts {
// 仅公开套餐时只补读缺失套餐的本地额度,不回源刷新或扩大公开范围
if !sharing.share_quota
&& explicit_plan_type(account.plan_type.as_deref()).is_some()
{
continue;
}
by_provider
.entry(&account.provider_kind)
.or_insert_with(Vec::new)
.push(account);
}
for (kind, accounts) in by_provider {
let Ok(provider) = self.providers.require(kind) else {
continue;
};
let requests = accounts
.iter()
.map(|account| ProviderQuotaReadRequest {
account,
rolling_usage: None,
})
.collect::<Vec<_>>();
for (account, quota) in accounts
.into_iter()
.zip(provider.read_quotas(&requests).await)
{
if let Ok(quota) = quota {
if sharing.share_quota {
window_count = window_count.saturating_add(quota.windows.len());
if window_count > limits.max_quota_windows {
return Err(PeerLinkError::SnapshotTooLarge);
}
}
quotas.insert(account.id.clone(), quota);
}
}
}
}
let now = Utc::now();
result.observed_at = now;
result.expires_at = now + TimeDelta::minutes(5);
if sharing.share_pool {
result.pool = Some(projection::pool_summary(
&accounts,
&quotas,
&runtime,
&self.providers,
sharing.share_plans,
now,
));
}
if sharing.share_quota {
let (quota, mut models) = projection::quota_and_models(
client,
&accounts,
&quotas,
&self.providers,
&runtime,
&public_models,
now,
limits.max_response_bytes,
)
.await?;
// 发布窗口内缺失或已变更的账号不能被静默剔除后推导整个模型池耗尽
if accounts.len() != ids.len() {
for model in &mut models {
model.complete = false;
if model.state == crate::model::peer_link::PeerModelState::Exhausted {
model.state = crate::model::peer_link::PeerModelState::Unknown;
}
}
}
result.quota = Some(quota);
result.models = models;
}
Ok(result)
}
}
fn intersect_sharing(frozen: ClientPeerSharing, current: ClientPeerSharing) -> ClientPeerSharing {
ClientPeerSharing {
enabled: frozen.enabled && current.enabled,
share_pool: frozen.share_pool && current.share_pool,
share_quota: frozen.share_quota && current.share_quota,
share_plans: frozen.share_plans && current.share_plans,
}
}
@@ -0,0 +1,262 @@
//! 从已授权的本地账号事实派生公开汇总,保留原始观测时间
use std::collections::BTreeMap;
use chrono::{DateTime, TimeDelta, Utc};
use gateway_core::{
account::{
AccountStatus, AccountStatusFacts, ProviderAccountId, QuotaAccessState,
resolve_account_status,
},
engine::execution::AuthenticatedClient,
};
use sha2::{Digest, Sha256};
use crate::{
model::{
accounts::{AccountRecord, AccountRuntimeSnapshot},
peer_link::{
PeerAccountQuota, PeerModelState, PeerModelSummary, PeerPoolSummary, PeerQuotaWindow,
},
provider_credentials::ProviderQuota,
},
ports::provider::ProviderAdminRegistry,
};
pub(super) fn pool_summary(
accounts: &[AccountRecord],
quotas: &BTreeMap<String, ProviderQuota>,
runtime: &AccountRuntimeSnapshot,
providers: &ProviderAdminRegistry,
share_plans: bool,
now: DateTime<Utc>,
) -> PeerPoolSummary {
let mut summary = PeerPoolSummary {
plans: share_plans.then(BTreeMap::new),
..Default::default()
};
for account in accounts {
match status(account, runtime, now) {
AccountStatus::Normal => summary.ready += 1,
AccountStatus::RateLimited => summary.cooldown += 1,
AccountStatus::QuotaExhausted => summary.exhausted += 1,
AccountStatus::Disabled => summary.disabled += 1,
AccountStatus::Error => summary.error += 1,
}
if let Some(plans) = &mut summary.plans
&& let Some(plan) = providers.resolve_account_plan(
account.provider_kind.as_str(),
&mut account.plan_type.clone(),
quotas.get(&account.id),
)
{
*plans.entry(plan).or_default() += 1;
}
}
summary
}
#[expect(
clippy::too_many_arguments,
reason = "投影显式接收已授权事实和调用边界的响应预算"
)]
pub(super) async fn quota_and_models(
client: &AuthenticatedClient,
accounts: &[AccountRecord],
quotas: &BTreeMap<String, ProviderQuota>,
providers: &ProviderAdminRegistry,
runtime: &AccountRuntimeSnapshot,
models: &[gateway_core::routing::PublicModelId],
now: DateTime<Utc>,
max_response_bytes: usize,
) -> Result<(Vec<PeerAccountQuota>, Vec<PeerModelSummary>), super::PeerLinkError> {
let snapshot = client.snapshot();
let scope = client.policy().account_scope();
let mut observations = BTreeMap::<&str, ModelObservations>::new();
let mut model_bytes = 0usize;
let mut result = Vec::with_capacity(accounts.len());
for (index, account) in accounts.iter().enumerate() {
// 大池投影不能连续占住异步执行线程,取消请求也要能及时释放并发名额
if index % 16 == 0 {
tokio::task::yield_now().await;
}
let observation = model_observation(account, runtime, now);
let Ok(id) = ProviderAccountId::new(account.id.clone()) else {
continue;
};
let provider = providers.require(&account.provider_kind).ok();
let mut account_models = Vec::new();
for model in models {
let upstream = snapshot.mapped_model(model.as_str());
if !snapshot.allows_public_model_for_account(model, scope, &id)
|| !account.model_access.allows(&upstream)
{
continue;
}
let eligible = provider
.as_ref()
.and_then(|provider| provider.peer_model_eligible(account, &upstream));
if eligible == Some(false) {
continue;
}
observations
.entry(model.as_str())
.or_default()
.observe(eligible.and(observation));
if eligible == Some(true) {
// 模型名是响应字节数的下界;提前拒绝必然超限的交叉展开,不误拒稀疏授权
model_bytes = model_bytes.saturating_add(model.as_str().len());
if model_bytes > max_response_bytes {
return Err(super::PeerLinkError::SnapshotTooLarge);
}
account_models.push(model.as_str().to_owned());
}
}
let quota = quotas.get(&account.id);
let observed_at = quota.and_then(|quota| quota.observed_at);
let expires_at = observed_at.map(|observed| {
quota
.into_iter()
.flat_map(|quota| &quota.windows)
.filter_map(|window| window.reset_at)
.fold(observed + TimeDelta::minutes(30), DateTime::min)
});
// 标识只在当前 Key 内稳定,不能用原始账号 ID 跨节点或跨授权关联身份
let mut digest = Sha256::new();
digest.update(b"cpr.peer.account\0");
digest.update(client.policy().plaintext_key().expose_for_auth().as_bytes());
digest.update([0]);
digest.update(account.id.as_bytes());
let id = hex::encode(&digest.finalize()[..16]);
result.push(PeerAccountQuota {
id,
models: account_models,
observed_at,
expires_at,
windows: quota
.into_iter()
.flat_map(|quota| &quota.windows)
.map(|window| PeerQuotaWindow {
key: window.key.clone(),
group: window.group.clone(),
label: window.label.clone(),
limit_id: window.limit_id.clone(),
limit_name: window.limit_name.clone(),
role: window.role,
window_seconds: window.window_seconds,
used_percent: window.used_percent,
reset_at: window.reset_at,
limit_reached: window.limit_reached,
})
.collect(),
});
}
let summaries = models
.iter()
.map(|model| {
let values = observations.remove(model.as_str()).unwrap_or_default();
let complete = values.observed_at.is_some() && !values.incomplete;
let state = if values.available {
PeerModelState::Available
} else if complete {
PeerModelState::Exhausted
} else {
PeerModelState::Unknown
};
PeerModelSummary {
model: model.as_str().to_owned(),
complete,
state,
observed_at: values.observed_at,
expires_at: values.expires_at,
}
})
.collect();
Ok((result, summaries))
}
#[derive(Clone, Copy)]
struct ModelObservation {
state: PeerModelState,
observed_at: DateTime<Utc>,
expires_at: DateTime<Utc>,
}
#[derive(Default)]
struct ModelObservations {
incomplete: bool,
available: bool,
observed_at: Option<DateTime<Utc>>,
expires_at: Option<DateTime<Utc>>,
}
impl ModelObservations {
fn observe(&mut self, observation: Option<ModelObservation>) {
let Some(observation) = observation else {
self.incomplete = true;
return;
};
self.available |= observation.state == PeerModelState::Available;
self.observed_at = Some(self.observed_at.map_or(observation.observed_at, |current| {
current.min(observation.observed_at)
}));
self.expires_at = Some(self.expires_at.map_or(observation.expires_at, |current| {
current.min(observation.expires_at)
}));
}
}
fn model_observation(
account: &AccountRecord,
runtime: &AccountRuntimeSnapshot,
now: DateTime<Utc>,
) -> Option<ModelObservation> {
if !matches!(
status(account, runtime, now),
AccountStatus::Normal | AccountStatus::QuotaExhausted
) {
return None;
}
let state = match account.quota.access() {
QuotaAccessState::Allowed => PeerModelState::Available,
QuotaAccessState::Exhausted => PeerModelState::Exhausted,
QuotaAccessState::Unknown => return None,
};
let observed_at = DateTime::<Utc>::from(account.quota.observed_at()?);
let expires_at = account
.quota
.reset_at()
.map(DateTime::<Utc>::from)
.map_or(observed_at + TimeDelta::minutes(30), |reset| {
reset.min(observed_at + TimeDelta::minutes(30))
})
.min(now + TimeDelta::minutes(5));
if observed_at > now || expires_at <= now {
return None;
}
Some(ModelObservation {
state,
observed_at,
expires_at,
})
}
fn status(
account: &AccountRecord,
runtime: &AccountRuntimeSnapshot,
now: DateTime<Utc>,
) -> AccountStatus {
resolve_account_status(
&AccountStatusFacts {
enabled: account.enabled,
credential_state: account.credential_state,
access_token_expires_at: account.access_token_expires_at.map(Into::into),
quota: account.quota,
cooldown: runtime.cooldown.get(&account.id).copied(),
last_error_reason: account.last_error_reason,
last_error_message: None,
},
now.into(),
)
.status
}
@@ -0,0 +1,46 @@
//! 互联只读查询的短期限流,不占用推理请求配额
use super::PeerLinkError;
use gateway_core::policy::ClientApiKeyId;
use std::{
collections::{BTreeMap, VecDeque},
sync::Mutex,
time::{Duration, Instant},
};
const WINDOW: Duration = Duration::from_secs(60);
const REQUESTS_PER_WINDOW: usize = 30;
#[derive(Default)]
pub(super) struct QueryRateLimit(Mutex<BTreeMap<ClientApiKeyId, VecDeque<Instant>>>);
impl QueryRateLimit {
pub(super) fn acquire(&self, id: &ClientApiKeyId) -> Result<(), PeerLinkError> {
let now = Instant::now();
let mut entries = self
.0
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
entries.retain(|_, requests| {
while requests
.front()
.is_some_and(|time| now.duration_since(*time) >= WINDOW)
{
requests.pop_front();
}
!requests.is_empty()
});
let requests = entries.entry(id.clone()).or_default();
if requests.len() >= REQUESTS_PER_WINDOW {
let retry_after = requests.front().map_or(1, |time| {
WINDOW
.saturating_sub(now.duration_since(*time))
.as_secs()
.saturating_add(1)
});
return Err(PeerLinkError::RateLimited { retry_after });
}
requests.push_back(now);
Ok(())
}
}
@@ -0,0 +1,133 @@
//! 账号查询合并原生额度与公共互联投影,保持列表摘要和详情窗口边界
use crate::{
model::{
accounts::AccountRecord,
provider_credentials::{ProviderQuota, ProviderQuotaReadRequest, ProviderQuotaRequest},
},
ports::provider::{ProviderAdmin, ProviderAdminError, ProviderAdminErrorKind},
};
use gateway_core::account::{CredentialRevision, ProviderAccountId, UpstreamConnection};
fn connection(account: &AccountRecord) -> Result<UpstreamConnection, ProviderAdminError> {
let invalid = || ProviderAdminError::new(ProviderAdminErrorKind::Invalid);
Ok(UpstreamConnection::new(
ProviderAccountId::new(account.id.clone()).map_err(|_| invalid())?,
CredentialRevision::new(account.credential_revision.get()).map_err(|_| invalid())?,
account.effective_outbound_proxy(),
))
}
pub(crate) fn store_error(error: gateway_core::error::StoreError) -> ProviderAdminError {
use gateway_core::error::StoreErrorKind;
let kind = match error.kind() {
StoreErrorKind::Conflict => ProviderAdminErrorKind::Conflict,
StoreErrorKind::InvalidData | StoreErrorKind::InvalidState => {
ProviderAdminErrorKind::NotFound
}
StoreErrorKind::Unavailable => ProviderAdminErrorKind::Unavailable,
_ => ProviderAdminErrorKind::Internal,
};
ProviderAdminError::new(kind).with_source(error)
}
pub(crate) async fn read_quota(
peer: Option<&dyn crate::ports::peer::PeerUpstreamService>,
provider: &dyn ProviderAdmin,
account: &AccountRecord,
request: ProviderQuotaRequest,
) -> Result<ProviderQuota, ProviderAdminError> {
if provider.supports_peer_connection(&account.authentication_kind)
&& let Some(peer) = peer
{
if request.refresh {
return peer.refresh_quota(&request.account_id).await;
}
return peer
.read_quota(&request.account_id)
.await
.map(|quota| quota.unwrap_or_else(ProviderQuota::default))
.map_err(store_error);
}
provider.quota(request).await
}
pub(crate) async fn read_quotas(
peer: Option<&dyn crate::ports::peer::PeerUpstreamService>,
provider: &dyn ProviderAdmin,
requests: &[ProviderQuotaReadRequest<'_>],
) -> Vec<Result<ProviderQuota, ProviderAdminError>> {
let mut results = provider.read_quotas(requests).await;
let Some(peer) = peer else {
return results;
};
if results.len() != requests.len() {
return results;
}
let indices = requests
.iter()
.enumerate()
.filter_map(|(index, request)| {
provider
.supports_peer_connection(&request.account.authentication_kind)
.then_some(index)
})
.collect::<Vec<_>>();
let queries = indices
.iter()
.map(|&index| ProviderQuotaReadRequest {
account: requests[index].account,
rolling_usage: requests[index].rolling_usage,
})
.collect::<Vec<_>>();
for (index, quota) in indices.into_iter().zip(peer.read_quotas(&queries).await) {
results[index] = quota;
}
results
}
/// 列表只加载轻量摘要,连接身份过期时丢弃旧观测
pub(super) async fn read_peer_quotas(
service: &super::PeerInfoService,
requests: &[ProviderQuotaReadRequest<'_>],
) -> Vec<Result<ProviderQuota, ProviderAdminError>> {
let connections = requests
.iter()
.map(|request| connection(request.account))
.collect::<Vec<_>>();
let queries = connections
.iter()
.zip(requests)
.filter(|(_, request)| request.account.enabled)
.filter_map(|(connection, _)| connection.as_ref().ok().cloned())
.collect::<Vec<_>>();
if queries.is_empty() {
return connections
.into_iter()
.map(|connection| connection.map(|_| ProviderQuota::default()))
.collect();
}
let summaries = service
.read_summaries(&queries)
.await
.map(|records| {
records
.into_iter()
.map(|(connection, quota)| (connection.account_id.clone(), (connection, quota)))
.collect::<std::collections::BTreeMap<_, _>>()
})
.map_err(store_error);
connections
.into_iter()
.map(|connection| {
connection.and_then(|connection| {
Ok(summaries
.as_ref()
.map_err(Clone::clone)?
.get(&connection.account_id)
.filter(|(observed, _)| *observed == connection)
.map_or_else(ProviderQuota::default, |(_, quota)| quota.clone()))
})
})
.collect()
}
@@ -0,0 +1,156 @@
//! 按账号额度版本缓存互联摘要,完整正文仅在版本变化时分批读取
use super::{
PeerInfoService, scheduling::SchedulingObservation, service::CachedObservation, snapshot,
};
use crate::model::provider_credentials::ProviderQuota;
use gateway_core::{
account::{QuotaObservation, UpstreamConnection, UpstreamInfoFailure},
error::StoreError,
};
use std::{collections::BTreeMap, sync::Arc, time::SystemTime};
impl PeerInfoService {
pub(crate) async fn read_summaries(
&self,
connections: &[UpstreamConnection],
) -> Result<Vec<(UpstreamConnection, ProviderQuota)>, StoreError> {
let _refresh = self.cache_refresh.lock().await;
let ids = connections
.iter()
.map(|connection| connection.account_id.clone())
.collect::<Vec<_>>();
let versions = self
.accounts
.get_quota_versions(&ids)
.await?
.into_iter()
.map(|version| (version.account_id.clone(), version))
.collect::<BTreeMap<_, _>>();
let now = SystemTime::now();
let mut changed = Vec::new();
let mut replaced = Vec::new();
{
let mut observations = self
.observations
.write()
.unwrap_or_else(std::sync::PoisonError::into_inner);
for connection in connections {
let observed_at = versions
.get(&connection.account_id)
.filter(|version| version.credential_revision == connection.credential_revision)
.map(|version| version.observed_at);
if let Some(cached) = observations.get_mut(&connection.account_id)
&& cached.connection == *connection
&& cached.observed_at == observed_at
{
cached.checked_at = now;
} else if observed_at.is_some() {
changed.push(connection.clone());
} else {
replaced.push(observations.insert(
connection.account_id.clone(),
CachedObservation {
connection: connection.clone(),
observed_at: None,
checked_at: now,
scheduling: None,
summary: None,
unsupported: false,
},
));
}
}
}
drop(replaced);
// 远端窗口可能很大,逐批投影后释放正文,锁内只交换轻量 Arc
for batch in changed.chunks(2) {
let ids = batch
.iter()
.map(|connection| connection.account_id.clone())
.collect::<Vec<_>>();
let observations = self.accounts.get_quotas(&ids).await?;
for connection in batch {
let observation = observations.iter().find(|value| {
value.account_id == connection.account_id
&& value.expected_revision == connection.credential_revision
});
self.cache(connection, observation);
}
}
let summaries = {
let observations = self
.observations
.read()
.unwrap_or_else(std::sync::PoisonError::into_inner);
connections
.iter()
.map(|connection| {
let cached = observations
.get(&connection.account_id)
.filter(|cached| cached.connection == *connection);
(
connection.clone(),
cached.and_then(|cached| cached.summary.clone()),
cached.is_some_and(|cached| cached.unsupported),
)
})
.collect::<Vec<_>>()
};
Ok(summaries
.into_iter()
.map(|(connection, peer, unsupported)| {
let quota = Self::project_peer(
&connection,
peer.as_deref(),
self.last_error(&connection)
.or(unsupported.then_some(UpstreamInfoFailure::Unsupported)),
now,
);
(connection, quota)
})
.collect())
}
pub(super) fn cache(
&self,
connection: &UpstreamConnection,
observation: Option<&QuotaObservation>,
) {
let now = SystemTime::now();
let peer = observation.and_then(|value| snapshot::decode(&value.quota));
let scheduling = peer
.as_ref()
.and_then(|peer| SchedulingObservation::new(peer, now))
.filter(|observation| observation.fresh(now))
.map(Arc::new);
let summary = peer.map(|mut peer| {
peer.quota = Default::default();
peer.models.clear();
Arc::new(peer)
});
let cached = CachedObservation {
connection: connection.clone(),
observed_at: observation.map(|value| value.observed_at),
checked_at: now,
scheduling,
summary,
unsupported: observation
.is_some_and(|value| snapshot::is_unsupported(&value.quota, connection)),
};
let previous = self
.observations
.write()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.insert(connection.account_id.clone(), cached);
drop(previous);
}
pub(super) fn unsupported(&self, connection: &UpstreamConnection) -> bool {
self.observations
.read()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.get(&connection.account_id)
.is_some_and(|cached| cached.connection == *connection && cached.unsupported)
}
}
@@ -0,0 +1,100 @@
//! 公共 CPR 上游同步、管理投影与调度观测,连接凭据解释由 Provider 提供
mod accounts;
mod cache;
mod projection;
mod quota;
mod refresh;
mod scheduling;
mod service;
mod snapshot;
mod worker;
use std::sync::Arc;
use crate::{
model::provider_credentials::ProviderQuota,
ports::{
peer::{PeerTransport, PeerUpstreamService},
provider::ProviderAdminRegistry,
},
};
pub(crate) use accounts::{read_quota, read_quotas};
use gateway_core::{
account::{ProviderAccountId, ProviderAccountStore, UpstreamInfoSource},
error::StoreError,
task::{WorkerContribution, WorkerDefinitionError},
};
use service::PeerInfoService;
/// 组合根独立注册公共互联 Worker,不属于任何具体 Provider 的任务集合
pub struct PeerUpstreamBundle {
service: Arc<PeerInfoService>,
workers: Vec<WorkerContribution>,
}
impl PeerUpstreamBundle {
pub fn service(&self) -> Arc<dyn PeerUpstreamService> {
self.service.clone()
}
pub fn scheduling_source(&self) -> Arc<dyn UpstreamInfoSource> {
self.service.clone()
}
pub fn take_worker_contributions(&mut self) -> Vec<WorkerContribution> {
std::mem::take(&mut self.workers)
}
}
pub fn initialize(
accounts: Arc<dyn ProviderAccountStore>,
providers: &ProviderAdminRegistry,
transport: Arc<dyn PeerTransport>,
) -> Result<PeerUpstreamBundle, WorkerDefinitionError> {
let service = Arc::new(PeerInfoService::new(accounts, providers, transport));
let worker = service.worker()?;
Ok(PeerUpstreamBundle {
service,
workers: vec![worker],
})
}
#[async_trait::async_trait]
impl PeerUpstreamService for PeerInfoService {
async fn account_changed(&self, id: &ProviderAccountId) -> Result<(), StoreError> {
PeerInfoService::account_changed(self, id).await
}
fn forget(&self, id: &ProviderAccountId) {
PeerInfoService::forget(self, id);
}
async fn refresh_quota(
&self,
id: &ProviderAccountId,
) -> Result<ProviderQuota, crate::ports::provider::ProviderAdminError> {
PeerInfoService::refresh(self, id, true).await
}
async fn read_quota(
&self,
account_id: &ProviderAccountId,
) -> Result<Option<ProviderQuota>, StoreError> {
PeerInfoService::read_quota(self, account_id).await
}
async fn read_summary(
&self,
account_id: &ProviderAccountId,
) -> Result<Option<ProviderQuota>, StoreError> {
PeerInfoService::read_summary(self, account_id).await
}
async fn read_quotas(
&self,
requests: &[crate::model::provider_credentials::ProviderQuotaReadRequest<'_>],
) -> Vec<Result<ProviderQuota, crate::ports::provider::ProviderAdminError>> {
accounts::read_peer_quotas(self, requests).await
}
}
@@ -0,0 +1,201 @@
//! 将已保存的互联观测投影为管理端额度与同步事实
use std::time::SystemTime;
use crate::model::{
provider_credentials::{
ProviderQuota, ProviderQuotaWindow, ProviderQuotaWindowRole, QuotaLocalUsageAttribution,
},
upstream_info::{
ProviderUpstreamInfo, UpstreamPlanCount, UpstreamPool, UpstreamPoolStates,
UpstreamQuotaAmount, UpstreamQuotaWindow,
},
};
use gateway_core::account::{QuotaAccessState, UpstreamConnection, UpstreamInfoFailure};
use gateway_protocol::peer::{PeerComponentStatus, PeerLink};
use super::{PeerInfoService, quota, snapshot};
impl PeerInfoService {
pub(super) fn project_peer(
connection: &UpstreamConnection,
peer: Option<&PeerLink>,
last_error: Option<UpstreamInfoFailure>,
now: SystemTime,
) -> ProviderQuota {
let observed_at = peer
.as_ref()
.and_then(|peer| snapshot::datetime(&peer.observed_at));
let stale = peer.as_ref().is_none_or(|peer| {
!snapshot::component_fresh(Some(&peer.observed_at), Some(&peer.expires_at), now)
});
let info = ProviderUpstreamInfo {
discovered: peer.is_some(),
observed_at,
stale,
connection_revision: connection.revision.clone(),
last_error: last_error.map(|error| error.as_str().to_owned()),
pool: peer
.as_ref()
.and_then(|peer| peer.pool.as_ref())
.map(|pool| {
let states = &pool.states;
UpstreamPool {
account_count: pool.account_count,
states: UpstreamPoolStates {
ready: states.ready,
cooldown: states.cooldown,
exhausted: states.exhausted,
disabled: states.disabled,
error: states.error,
unknown: states.unknown,
},
plans: pool
.plans
.as_deref()
.unwrap_or_default()
.iter()
.map(|plan| UpstreamPlanCount {
name: plan.name.clone(),
count: plan.count,
})
.collect(),
}
}),
};
let mut windows = Vec::new();
if let Some(peer) = &peer {
if let Some(component) = &peer.usage
&& component.status == PeerComponentStatus::Available
&& let Some(usage) = &component.data
&& let Some(unit) = &usage.unit
{
for window in &usage.windows {
// 不限额仍展示已提供的账本窗口,只是不计算占用百分比
let total = window
.total
.as_deref()
.and_then(snapshot::decimal)
.filter(|value| *value > 0.0);
let used_percent = snapshot::decimal(&window.used)
.zip(total)
.map(|(used, total)| (used / total * 100.0).clamp(0.0, 100.0));
let window_observed = component
.observed_at
.as_deref()
.and_then(snapshot::datetime);
let window_stale = stale
|| !component
.observed_at
.as_deref()
.is_some_and(|value| snapshot::fresh(value, now))
|| window
.reset_at
.as_deref()
.and_then(snapshot::timestamp)
.is_some_and(|reset| reset <= now);
windows.push(ProviderQuotaWindow {
upstream: Some(UpstreamQuotaWindow {
account_id: None,
observed_at: window_observed,
stale: window_stale,
amount: Some(UpstreamQuotaAmount {
unit: unit.clone(),
used: window.used.clone(),
total: window.total.clone(),
remaining: window.remaining.clone(),
}),
}),
key: format!("peer:budget:{}", window.key),
group: "general".to_owned(),
label: window.label.as_deref().unwrap_or(&window.key).to_owned(),
limit_id: None,
limit_name: None,
role: match window.key.as_str() {
"daily" => Some(ProviderQuotaWindowRole::Primary),
"weekly" => Some(ProviderQuotaWindowRole::Secondary),
"monthly" => Some(ProviderQuotaWindowRole::Monthly),
_ => None,
},
// Key 消费窗口覆盖此连接的全部请求,本机用量按同一时间边界聚合
local_usage_attribution: QuotaLocalUsageAttribution::AccountWide,
window_seconds: window.window_seconds,
used_percent,
reset_at: window.reset_at.as_deref().and_then(snapshot::datetime),
limit_reached: !window_stale
&& quota::window_access(window, now) == QuotaAccessState::Exhausted,
local_usage: None,
provider_data: None,
});
}
}
// 池内窗口各自保留账号归属,不能将不同套餐相加为统一消费额度
if let Some(quota) = &peer.quota {
for (index, account) in quota.accounts.iter().enumerate() {
let window_observed =
account.observed_at.as_deref().and_then(snapshot::datetime);
let window_stale = stale
|| !snapshot::component_fresh(
account.observed_at.as_deref(),
account.expires_at.as_deref(),
now,
);
for window in &account.windows {
let group = window.group.as_deref().unwrap_or("general");
let window_stale = window_stale
|| window
.reset_at
.as_deref()
.and_then(snapshot::timestamp)
.is_some_and(|reset| reset <= now);
let scope = format!(
"peer:{}:{}:{}",
account.id.len(),
account.id,
window.limit_id.as_deref().unwrap_or(group)
);
windows.push(ProviderQuotaWindow {
upstream: Some(UpstreamQuotaWindow {
account_id: Some(account.id.clone()),
observed_at: window_observed,
stale: window_stale,
amount: None,
}),
key: format!("{scope}:{}", window.key),
group: group.to_owned(),
label: window.label.as_deref().unwrap_or(&window.key).to_owned(),
limit_id: Some(scope),
limit_name: Some(match window.limit_name.as_deref() {
Some(name) => format!("账号 {} · {name}", index + 1),
None => format!("账号 {}", index + 1),
}),
role: match window.role.as_deref() {
Some("primary") => Some(ProviderQuotaWindowRole::Primary),
Some("secondary") => Some(ProviderQuotaWindowRole::Secondary),
Some("monthly") => Some(ProviderQuotaWindowRole::Monthly),
_ => None,
},
local_usage_attribution: QuotaLocalUsageAttribution::Unavailable,
window_seconds: window.window_seconds,
used_percent: window.used_percent,
reset_at: window.reset_at.as_deref().and_then(snapshot::datetime),
limit_reached: !window_stale && window.limit_reached,
local_usage: None,
provider_data: None,
});
}
}
}
}
ProviderQuota {
upstream: Some(info),
plan_type: None,
observed_at,
refresh_token_expires_at: None,
windows,
credits: None,
limit_reached: false,
provider_data: None,
}
}
}
@@ -0,0 +1,82 @@
//! 消费互联明确的额度访问结论,金额只由展示层解释
use std::time::SystemTime;
use gateway_core::account::{QuotaAccessState, QuotaEvidence, QuotaState};
use gateway_protocol::peer::{PeerBudgetWindow, PeerComponentStatus, PeerKeyUsage, PeerLink};
use super::snapshot::{component_fresh, fresh, timestamp};
/// 原始时间与请求开始时间共同限制观测,迟到响应不能覆盖请求期间的新失败
pub(super) fn observe(
peer: &PeerLink,
started_at: SystemTime,
now: SystemTime,
) -> Option<QuotaState> {
let usage = peer.usage.as_ref()?;
if !fresh(&peer.observed_at, now)
|| !component_fresh(Some(&peer.observed_at), Some(&peer.expires_at), now)
|| usage.status != PeerComponentStatus::Available
|| !component_fresh(usage.observed_at.as_deref(), None, now)
{
return None;
}
let observed_at = started_at
.min(timestamp(&peer.observed_at)?)
.min(timestamp(usage.observed_at.as_deref()?)?);
let usage = usage.data.as_ref()?;
match access(usage, now) {
QuotaAccessState::Allowed => Some(QuotaState::allowed(observed_at)),
QuotaAccessState::Exhausted => {
let reset_at = usage
.windows
.iter()
.filter(|window| window_access(window, now) == QuotaAccessState::Exhausted)
.filter_map(|window| window.reset_at.as_deref().and_then(timestamp))
.min();
Some(QuotaState::exhausted(
if usage.allowed == Some(false) {
QuotaEvidence::ProviderDenied
} else {
QuotaEvidence::AccountLimitReached
},
observed_at,
reset_at,
))
}
QuotaAccessState::Unknown => None,
}
}
/// 与 OAuth 相同,明确的 allowed 优先,窗口耗尽仅补充缺失的访问结论
pub(super) fn access(usage: &PeerKeyUsage, now: SystemTime) -> QuotaAccessState {
match usage.allowed {
Some(true) => QuotaAccessState::Allowed,
Some(false) => QuotaAccessState::Exhausted,
None if usage
.windows
.iter()
.any(|window| window_access(window, now) == QuotaAccessState::Exhausted) =>
{
QuotaAccessState::Exhausted
}
None => QuotaAccessState::Unknown,
}
}
/// 过期窗口保持未知,不从重置时间或金额推断恢复
pub(super) fn window_access(window: &PeerBudgetWindow, now: SystemTime) -> QuotaAccessState {
if window
.reset_at
.as_deref()
.and_then(timestamp)
.is_some_and(|reset| reset <= now)
{
return QuotaAccessState::Unknown;
}
match window.limit_reached {
Some(true) => QuotaAccessState::Exhausted,
Some(false) => QuotaAccessState::Allowed,
None => QuotaAccessState::Unknown,
}
}
@@ -0,0 +1,217 @@
//! 互联请求的运行期节流、取消恢复与结果提交,协议判断留在 Admin
use super::{PeerInfoService, accounts::store_error, quota, snapshot};
use crate::{
model::provider_credentials::ProviderQuota,
ports::provider::{ProviderAdminError, ProviderAdminErrorKind},
};
use gateway_core::account::{
OpaqueProviderData, ProviderAccountId, QuotaObservation, QuotaState, QuotaWriteOutcome,
UpstreamConnection, UpstreamInfoFailure,
};
use std::{
sync::Arc,
time::{Duration, SystemTime},
};
use tokio::time::Instant;
pub(super) struct RefreshState {
connection: UpstreamConnection,
next_attempt: Instant,
manual_after: Instant,
last_error: Option<UpstreamInfoFailure>,
}
impl PeerInfoService {
fn refresh_state(
&self,
connection: &UpstreamConnection,
) -> Arc<tokio::sync::Mutex<RefreshState>> {
self.attempts
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.entry(connection.account_id.clone())
.or_insert_with(|| {
Arc::new(tokio::sync::Mutex::new(RefreshState {
connection: connection.clone(),
next_attempt: Instant::now(),
manual_after: Instant::now(),
last_error: None,
}))
})
.clone()
}
pub(super) fn refresh_due(&self, connection: &UpstreamConnection) -> bool {
if self.unsupported(connection) {
return false;
}
let state = self.refresh_state(connection);
state.try_lock().is_ok_and(|state| {
state.connection != *connection || Instant::now() >= state.next_attempt
})
}
pub(super) fn last_error(
&self,
connection: &UpstreamConnection,
) -> Option<UpstreamInfoFailure> {
let state = self
.attempts
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.get(&connection.account_id)?
.clone();
state
.try_lock()
.ok()
.filter(|state| state.connection == *connection)
.and_then(|state| state.last_error)
}
pub(crate) async fn refresh(
&self,
id: &ProviderAccountId,
manual: bool,
) -> Result<ProviderQuota, ProviderAdminError> {
// 首查、手动与周期刷新共享容量;排队后再读凭据,避免使用等待期间失效的连接
let _permit = self
.refresh_slots
.acquire()
.await
.map_err(|_| ProviderAdminError::new(ProviderAdminErrorKind::Unavailable))?;
let loaded = self
.accounts
.load_current_credential(id)
.await
.map_err(store_error)?;
let connection = UpstreamConnection::for_account(&loaded.account);
if !loaded.account.enabled() {
return Err(ProviderAdminError::new(ProviderAdminErrorKind::Conflict));
}
let authentication = self
.connection(&loaded)
.ok_or_else(|| ProviderAdminError::new(ProviderAdminErrorKind::Unsupported))?;
let slot = self.refresh_state(&connection);
let mut state = slot.lock().await;
if !manual && self.unsupported(&connection)
|| state.connection == connection
&& Instant::now()
< if manual {
state.manual_after
} else {
state.next_attempt
}
{
let error = state.last_error;
drop(state);
if let Some(error) = error.filter(|error| *error != UpstreamInfoFailure::Unsupported) {
return Err(refresh_error(error));
}
return self
.read_quota(id)
.await
.map_err(store_error)
.map(Option::unwrap_or_default);
}
if state.connection != connection {
state.last_error = None;
}
state.connection = connection.clone();
state.next_attempt = Instant::now() + Duration::from_secs(30);
state.manual_after = state.next_attempt;
let started_at = SystemTime::now();
let (result, retry_after) =
match self.transport.fetch(&loaded.account, &authentication).await {
Ok(bytes) => (snapshot::parse(&bytes, SystemTime::now()), None),
Err(failure) => (Err(failure.kind), failure.retry_after),
};
let now = SystemTime::now();
let failure = result.as_ref().err().copied();
let delay = match failure {
None => Duration::from_secs(240 + stable_jitter(id.as_str())),
Some(UpstreamInfoFailure::Unauthorized) => Duration::from_secs(1800),
Some(_) => Duration::from_secs(30),
}
.max(retry_after.unwrap_or_default());
state.next_attempt = Instant::now() + delay;
state.manual_after = Instant::now()
+ (if failure.is_some_and(|error| error != UpstreamInfoFailure::Unsupported) {
delay
} else {
Duration::from_secs(15)
})
.max(retry_after.unwrap_or_default());
state.last_error = failure;
let document = match result {
Ok(document) => Some(document),
// 空观测清除已撤销内容,并用共有观察时间阻止更早请求重新写回
Err(UpstreamInfoFailure::Unauthorized) => {
Some(OpaqueProviderData::new(Default::default()))
}
Err(UpstreamInfoFailure::Unsupported) => Some(snapshot::unsupported(&connection)),
Err(_) => None,
};
if let Some(document) = document {
let access = snapshot::decode(&document)
.and_then(|peer| quota::observe(&peer, started_at, now))
.unwrap_or(QuotaState::unknown());
let observation = QuotaObservation {
expected_connection: Some(connection.clone()),
account_id: id.clone(),
expected_revision: connection.credential_revision,
quota: document,
plan_type: None,
observed_at: started_at,
state: access,
};
if self
.accounts
.compare_and_swap_quota(observation)
.await
.map_err(store_error)?
== QuotaWriteOutcome::Conflict
{
return Err(ProviderAdminError::new(ProviderAdminErrorKind::Conflict));
}
// 与列表 hydration 串行,避免本地缓存发布旧版本覆盖新版本
let _refresh = self.cache_refresh.lock().await;
let observations = self
.accounts
.get_quotas(std::slice::from_ref(id))
.await
.map_err(store_error)?;
self.cache(
&connection,
observations.iter().find(|observation| {
observation.expected_revision == connection.credential_revision
}),
);
}
drop(state);
// 不支持协议是正常发现结果,管理端保留分类,后台不重复报警
if let Some(error) = failure.filter(|error| *error != UpstreamInfoFailure::Unsupported) {
return Err(refresh_error(error));
}
self.read_quota(id)
.await
.map_err(store_error)
.map(Option::unwrap_or_default)
}
}
fn stable_jitter(id: &str) -> u64 {
id.bytes().fold(0_u64, |hash, byte| {
hash.wrapping_mul(31).wrapping_add(u64::from(byte))
}) % 60
}
fn refresh_error(error: UpstreamInfoFailure) -> ProviderAdminError {
ProviderAdminError::new(ProviderAdminErrorKind::BadGateway).with_public_message(match error {
UpstreamInfoFailure::Unauthorized => "上游未授权读取互联信息",
UpstreamInfoFailure::Unsupported => "上游不支持互联信息查询",
UpstreamInfoFailure::RateLimited => "上游互联查询已限流",
UpstreamInfoFailure::InvalidResponse => "上游互联响应无效",
UpstreamInfoFailure::Unavailable => "上游互联服务暂不可用",
})
}
@@ -0,0 +1,208 @@
//! 将互联观测压缩为按模型索引的调度事实,保留各层独立的有效期
use std::{collections::HashMap, sync::Arc, time::SystemTime};
use gateway_protocol::peer::{PeerComponentStatus, PeerLink, PeerModelCoverage, PeerModelState};
use super::{quota, snapshot};
use gateway_core::account::{QuotaAccessState, UpstreamSchedulingSignal};
#[derive(Clone, Copy)]
struct Validity {
observed_at: SystemTime,
expires_at: SystemTime,
}
impl Validity {
fn component(observed: Option<&str>, expires: Option<&str>) -> Option<Self> {
let observed_at = observed.and_then(snapshot::timestamp)?;
let expires_at = match expires {
Some(value) => snapshot::timestamp(value)?,
None => observed_at + snapshot::FRESH_FOR,
};
(expires_at <= observed_at + snapshot::MAX_AGE).then_some(Self {
observed_at,
expires_at,
})
}
fn fresh(self, now: SystemTime) -> bool {
self.observed_at <= now + std::time::Duration::from_secs(60) && self.expires_at > now
}
}
struct QuotaWindow {
remaining_percent: f64,
reset_at: Option<SystemTime>,
}
struct AccountQuota {
validity: Validity,
windows: Vec<QuotaWindow>,
}
impl AccountQuota {
fn remaining_percent(&self, now: SystemTime) -> Option<f64> {
if !self.validity.fresh(now) {
return None;
}
self.windows
.iter()
.filter(|window| window.reset_at.is_none_or(|reset| reset > now))
.map(|window| window.remaining_percent)
.reduce(f64::min)
}
}
struct ModelObservation {
validity: Validity,
exhausted: bool,
account: Option<Arc<AccountQuota>>,
}
pub(super) struct SchedulingObservation {
validity: Validity,
exhausted_budgets: Vec<Validity>,
models: HashMap<String, Option<ModelObservation>>,
}
impl SchedulingObservation {
pub(super) fn new(peer: &PeerLink, now: SystemTime) -> Option<Self> {
let validity = Validity::component(Some(&peer.observed_at), Some(&peer.expires_at))?;
let mut exhausted_budgets = Vec::new();
if let Some(component) = &peer.usage
&& component.status == PeerComponentStatus::Available
&& let Some(usage) = &component.data
&& let Some(usage_validity) =
Validity::component(component.observed_at.as_deref(), None)
&& quota::access(usage, now) == QuotaAccessState::Exhausted
{
if usage.allowed.is_some() {
exhausted_budgets.push(usage_validity);
} else {
// 窗口提供的补充证据只在各自重置边界之前影响调度评分
for window in &usage.windows {
if quota::window_access(window, now) == QuotaAccessState::Exhausted {
let mut validity = usage_validity;
if let Some(reset) =
window.reset_at.as_deref().and_then(snapshot::timestamp)
{
validity.expires_at = validity.expires_at.min(reset);
}
exhausted_budgets.push(validity);
}
}
}
}
// 同一账号重复列出模型不增加候选数,多账号覆盖不计算池平均百分比
let mut model_accounts = HashMap::<&str, Option<usize>>::new();
let quota_accounts = peer
.quota
.as_ref()
.map_or(&[][..], |quota| quota.accounts.as_slice());
for (index, account) in quota_accounts.iter().enumerate() {
for model in &account.models {
model_accounts
.entry(model)
.and_modify(|candidate| {
if *candidate != Some(index) {
*candidate = None;
}
})
.or_insert(Some(index));
}
}
let mut accounts = HashMap::<usize, Option<Arc<AccountQuota>>>::new();
let mut models = HashMap::with_capacity(peer.models.len());
for model in &peer.models {
models.entry(model.model.clone()).or_insert_with(|| {
if model.coverage != PeerModelCoverage::Complete
|| model.state == PeerModelState::Unknown
{
return None;
}
let validity =
Validity::component(model.observed_at.as_deref(), model.expires_at.as_deref())?;
let account = model_accounts
.get(model.model.as_str())
.copied()
.flatten()
.and_then(|index| {
accounts
.entry(index)
.or_insert_with(|| {
let account = &quota_accounts[index];
let validity = Validity::component(
account.observed_at.as_deref(),
account.expires_at.as_deref(),
)?;
Some(Arc::new(AccountQuota {
validity,
windows: account
.windows
.iter()
.filter_map(|window| {
Some(QuotaWindow {
remaining_percent: 100.0 - window.used_percent?,
reset_at: window
.reset_at
.as_deref()
.and_then(snapshot::timestamp),
})
})
.collect(),
}))
})
.clone()
});
Some(ModelObservation {
validity,
exhausted: model.state == PeerModelState::Exhausted,
account,
})
});
}
Some(Self {
validity,
exhausted_budgets,
models,
})
}
pub(super) fn fresh(&self, now: SystemTime) -> bool {
self.validity.fresh(now)
&& now
.duration_since(self.validity.observed_at)
.unwrap_or_default()
<= snapshot::FRESH_FOR
}
pub(super) fn signal(&self, model: &str, now: SystemTime) -> Option<UpstreamSchedulingSignal> {
if !self.fresh(now) {
return None;
}
if self
.exhausted_budgets
.iter()
.any(|validity| validity.fresh(now))
{
return Some(UpstreamSchedulingSignal {
exhausted: true,
remaining_percent: None,
});
}
let model = self.models.get(model)?.as_ref()?;
if !model.validity.fresh(now) {
return None;
}
Some(UpstreamSchedulingSignal {
exhausted: model.exhausted,
remaining_percent: model
.account
.as_ref()
.and_then(|account| account.remaining_percent(now)),
})
}
}
@@ -0,0 +1,308 @@
//! 互联额度读取与后台刷新,复用账号额度存储和 Host Worker 生命周期
use super::{refresh::RefreshState, scheduling::SchedulingObservation, snapshot};
use crate::{
model::provider_credentials::ProviderQuota,
ports::{
peer::{PeerConnection, PeerTransport},
provider::{ProviderAdmin, ProviderAdminRegistry},
},
};
use futures::{StreamExt as _, stream};
use gateway_core::{
account::{
LoadedCredential, ProviderAccount, ProviderAccountId, ProviderAccountStore,
UpstreamConnection, UpstreamInfoFailure, UpstreamInfoSource, UpstreamSchedulingSignal,
},
error::{StoreError, StoreErrorKind},
routing::ProviderKind,
task::WorkerCycleContext,
};
use gateway_protocol::peer::PeerLink;
use std::{
collections::{BTreeMap, HashMap},
sync::{Arc, Mutex, RwLock},
time::SystemTime,
};
use tokio::sync::Semaphore;
const REFRESH_CONCURRENCY: usize = 4;
pub(super) struct CachedObservation {
pub connection: UpstreamConnection,
pub observed_at: Option<SystemTime>,
pub checked_at: SystemTime,
pub scheduling: Option<Arc<SchedulingObservation>>,
pub summary: Option<Arc<PeerLink>>,
pub unsupported: bool,
}
pub(crate) struct PeerInfoService {
pub(super) accounts: Arc<dyn ProviderAccountStore>,
providers: BTreeMap<ProviderKind, Arc<dyn ProviderAdmin>>,
pub(super) transport: Arc<dyn PeerTransport>,
pub(super) attempts: Mutex<HashMap<ProviderAccountId, Arc<tokio::sync::Mutex<RefreshState>>>>,
cursors: Mutex<BTreeMap<ProviderKind, usize>>,
pub(super) cache_refresh: tokio::sync::Mutex<()>,
pub(super) refresh_slots: Semaphore,
pub(super) observations: RwLock<HashMap<ProviderAccountId, CachedObservation>>,
}
impl PeerInfoService {
pub(crate) fn new(
accounts: Arc<dyn ProviderAccountStore>,
providers: &ProviderAdminRegistry,
transport: Arc<dyn PeerTransport>,
) -> Self {
Self {
accounts,
providers: providers.registered(),
transport,
attempts: Mutex::default(),
cursors: Mutex::default(),
cache_refresh: tokio::sync::Mutex::new(()),
refresh_slots: Semaphore::new(REFRESH_CONCURRENCY),
observations: RwLock::default(),
}
}
pub(crate) async fn account_changed(&self, id: &ProviderAccountId) -> Result<(), StoreError> {
let has_observation = self
.observations
.read()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.contains_key(id);
let has_attempt = self
.attempts
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.contains_key(id);
if !has_observation && !has_attempt {
return Ok(());
}
let Some(account) = self
.accounts
.get_account(id)
.await?
.filter(|account| self.supported(account))
else {
self.forget(id);
return Ok(());
};
let connection = UpstreamConnection::for_account(&account);
// 普通信息编辑保留观测与退避,连接变化由刷新状态中的身份比较处理
let mut observations = self
.observations
.write()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if observations
.get(id)
.is_some_and(|cached| cached.connection != connection)
{
observations.remove(id);
}
Ok(())
}
pub(crate) fn forget(&self, id: &ProviderAccountId) {
let previous = self
.observations
.write()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.remove(id);
drop(previous);
// 正在执行的请求继续持有同一把锁,结果仍由数据库版本校验决定是否接纳
let mut attempts = self
.attempts
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if attempts
.get(id)
.is_some_and(|state| Arc::strong_count(state) == 1)
{
attempts.remove(id);
}
}
pub(super) fn connection(&self, loaded: &LoadedCredential) -> Option<PeerConnection> {
self.providers
.get(loaded.account.provider())?
.peer_connection(loaded)
.ok()
.flatten()
}
fn supported(&self, account: &ProviderAccount) -> bool {
account.enabled()
&& self
.providers
.get(account.provider())
.is_some_and(|provider| {
provider.supports_peer_connection(account.authentication_kind())
})
}
async fn current_connection(
&self,
id: &ProviderAccountId,
) -> Result<Option<UpstreamConnection>, StoreError> {
let account = self
.accounts
.get_account(id)
.await?
.ok_or_else(|| StoreError::new(StoreErrorKind::InvalidData))?;
Ok(self
.supported(&account)
.then(|| UpstreamConnection::for_account(&account)))
}
pub(crate) async fn read_quota(
&self,
id: &ProviderAccountId,
) -> Result<Option<ProviderQuota>, StoreError> {
let Some(connection) = self.current_connection(id).await? else {
return Ok(None);
};
let observation = self
.accounts
.get_quotas(std::slice::from_ref(id))
.await?
.into_iter()
.find(|value| value.expected_revision == connection.credential_revision);
let peer = observation
.as_ref()
.and_then(|value| snapshot::decode(&value.quota));
let unsupported = observation
.as_ref()
.is_some_and(|value| snapshot::is_unsupported(&value.quota, &connection));
if self.current_connection(id).await?.as_ref() != Some(&connection) {
return Ok(None);
}
Ok(Some(Self::project_peer(
&connection,
peer.as_ref(),
self.last_error(&connection)
.or(unsupported.then_some(UpstreamInfoFailure::Unsupported)),
SystemTime::now(),
)))
}
pub(crate) async fn read_summary(
&self,
id: &ProviderAccountId,
) -> Result<Option<ProviderQuota>, StoreError> {
let Some(connection) = self.current_connection(id).await? else {
return Ok(None);
};
Ok(self
.read_summaries(&[connection])
.await?
.into_iter()
.next()
.map(|(_, quota)| quota))
}
pub(super) async fn synchronize(&self, context: &WorkerCycleContext) -> Result<(), StoreError> {
let mut failure = None;
let mut active = std::collections::HashSet::new();
for provider in self.providers.keys() {
if context.cancellation().is_cancelled() {
break;
}
let accounts = match self.accounts.list_for_provider(provider).await {
Ok(accounts) => accounts,
Err(error) => {
failure.get_or_insert(error);
continue;
}
};
let connections = accounts
.iter()
.filter(|account| self.supported(account))
.map(UpstreamConnection::for_account)
.collect::<Vec<_>>();
active.extend(
connections
.iter()
.map(|connection| connection.account_id.clone()),
);
let connections = {
let mut cursors = self
.cursors
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let offset = cursors.entry(provider.clone()).or_default();
if *offset >= connections.len() {
*offset = 0;
}
let batch = connections
.iter()
.skip(*offset)
.take(100)
.cloned()
.collect::<Vec<_>>();
*offset += batch.len();
batch
};
if let Err(error) = self.read_summaries(&connections).await {
failure.get_or_insert(error);
continue;
}
// 每轮有界并发,运行期到期记录在请求开始时预留,取消后可按普通重试恢复
let pending = connections
.into_iter()
.filter(|connection| self.refresh_due(connection))
.take(100)
.collect::<Vec<_>>();
stream::iter(pending).map(|connection| async move {
if let Err(error) = self.refresh(&connection.account_id, false).await {
tracing::warn!(account_id = %connection.account_id, error = %error, "Upstream quota refresh failed");
}
}).buffer_unordered(REFRESH_CONCURRENCY).collect::<Vec<_>>().await;
}
// 查询失败时保留该轮无法核对的账号,避免错误风暴清空所有节流记录
if failure.is_none() && !context.cancellation().is_cancelled() {
self.attempts
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.retain(|id, state| active.contains(id) || Arc::strong_count(state) > 1);
}
let now = SystemTime::now();
let removed = self
.observations
.write()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.extract_if(|_, cached| {
now.duration_since(cached.checked_at).unwrap_or_default() > snapshot::MAX_AGE
})
.collect::<Vec<_>>();
drop(removed);
failure.map_or(Ok(()), Err)
}
}
impl UpstreamInfoSource for PeerInfoService {
fn signal(
&self,
account: &ProviderAccount,
model: &str,
now: SystemTime,
) -> Option<UpstreamSchedulingSignal> {
if !account.enabled() {
return None;
}
let connection = UpstreamConnection::for_account(account);
let scheduling = {
let observations = self
.observations
.read()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let cached = observations.get(account.id())?;
if cached.connection != connection {
return None;
}
Arc::clone(cached.scheduling.as_ref()?)
};
scheduling.signal(model, now)
}
}
@@ -0,0 +1,292 @@
//! 互联响应的有界校验、能力授权与时间有效性解释
use std::{
collections::HashSet,
time::{Duration, SystemTime},
};
use chrono::{DateTime, Utc};
use gateway_core::account::{OpaqueProviderData, UpstreamConnection, UpstreamInfoFailure};
use gateway_protocol::peer::{
MAX_ACCOUNTS, MAX_MODELS, MAX_QUOTA_WINDOWS, OBJECT, PeerKeyUsage, PeerLink, PeerPool,
PeerQuota, SCHEMA_VERSION,
};
use serde::Deserialize as _;
pub(super) const MAX_AGE: Duration = Duration::from_secs(30 * 60);
pub(super) const FRESH_FOR: Duration = Duration::from_secs(5 * 60);
// 不支持协议是连接的识别结果,复用观测存储与版本保护,不伪造上游额度快照
pub(super) fn unsupported(connection: &UpstreamConnection) -> OpaqueProviderData {
OpaqueProviderData::new(serde_json::Map::from_iter([
("object".into(), "cpr.peer.unsupported".into()),
("schema_version".into(), 1.into()),
(
"connection_revision".into(),
connection.revision.clone().into(),
),
]))
}
pub(super) fn is_unsupported(data: &OpaqueProviderData, connection: &UpstreamConnection) -> bool {
let fields = data.expose_to_provider();
fields.get("object").and_then(serde_json::Value::as_str) == Some("cpr.peer.unsupported")
&& fields
.get("schema_version")
.and_then(serde_json::Value::as_u64)
== Some(1)
&& fields
.get("connection_revision")
.and_then(serde_json::Value::as_str)
== Some(connection.revision.as_str())
}
pub(super) fn timestamp(value: &str) -> Option<SystemTime> {
DateTime::parse_from_rfc3339(value)
.ok()
.map(SystemTime::from)
}
pub(super) fn datetime(value: &str) -> Option<DateTime<Utc>> {
DateTime::parse_from_rfc3339(value)
.ok()
.map(|value| value.with_timezone(&Utc))
}
pub(super) fn fresh(observed: &str, now: SystemTime) -> bool {
timestamp(observed).is_some_and(|observed| {
observed <= now + Duration::from_secs(60)
&& now.duration_since(observed).unwrap_or_default() <= FRESH_FOR
})
}
pub(super) fn component_fresh(
observed: Option<&str>,
expires: Option<&str>,
now: SystemTime,
) -> bool {
let Some(observed) = observed.and_then(timestamp) else {
return false;
};
let expiry = match expires {
Some(value) => match timestamp(value) {
Some(expiry) => expiry,
None => return false,
},
None => observed + FRESH_FOR,
};
observed <= now + Duration::from_secs(60) && expiry > now && expiry <= observed + MAX_AGE
}
pub(super) fn decode(data: &OpaqueProviderData) -> Option<PeerLink> {
let peer = PeerLink::deserialize(
serde::de::value::MapDeserializer::<_, serde_json::Error>::new(
data.expose_to_provider()
.iter()
.map(|(key, value)| (key.as_str(), value)),
),
)
.ok()?;
(peer.object == OBJECT && peer.schema_version == SCHEMA_VERSION).then_some(peer)
}
pub(super) fn parse(
bytes: &[u8],
now: SystemTime,
) -> Result<OpaqueProviderData, UpstreamInfoFailure> {
let mut peer: PeerLink =
serde_json::from_slice(bytes).map_err(|_| UpstreamInfoFailure::InvalidResponse)?;
let observed = timestamp(&peer.observed_at).ok_or(UpstreamInfoFailure::InvalidResponse)?;
let expires = timestamp(&peer.expires_at).ok_or(UpstreamInfoFailure::InvalidResponse)?;
let accounts = peer
.quota
.as_ref()
.map_or(&[][..], |quota| quota.accounts.as_slice());
let budget_windows = peer
.usage
.as_ref()
.and_then(|usage| usage.data.as_ref())
.map_or(0, |usage| usage.windows.len());
if peer.object != OBJECT
|| peer.schema_version != SCHEMA_VERSION
|| observed > now + Duration::from_secs(60)
|| now.duration_since(observed).unwrap_or_default() > MAX_AGE
|| expires <= observed
|| expires > observed + MAX_AGE
|| peer.capabilities.len() > 32
|| accounts
.iter()
.map(|account| account.windows.len())
.sum::<usize>()
> MAX_QUOTA_WINDOWS
|| budget_windows > MAX_QUOTA_WINDOWS
|| accounts.len() > MAX_ACCOUNTS
|| peer.models.len() > MAX_MODELS
{
return Err(UpstreamInfoFailure::InvalidResponse);
}
// 未授权能力不能借同名数据块越过逐层公开边界
let disclosed = |capability| peer.capabilities.iter().any(|value| value == capability);
if !disclosed("key_usage") {
peer.usage = None;
}
if !disclosed("pool_summary") {
peer.pool = None;
}
if !disclosed("quota_windows") {
peer.quota = None;
}
if !disclosed("model_availability") {
peer.models.clear();
}
if let Some(pool) = &peer.pool {
validate_pool(pool)?;
}
if let Some(usage) = peer.usage.as_ref().and_then(|usage| usage.data.as_ref()) {
validate_usage(usage)?;
}
if let Some(quota) = &peer.quota {
validate_quota(quota)?;
}
for model in &peer.models {
if !valid_text(&model.model, 256)
|| model
.observed_at
.as_deref()
.is_some_and(|value| timestamp(value).is_none())
|| model
.expires_at
.as_deref()
.is_some_and(|value| timestamp(value).is_none())
{
return Err(UpstreamInfoFailure::InvalidResponse);
}
}
match serde_json::to_value(peer) {
Ok(serde_json::Value::Object(object)) => Ok(OpaqueProviderData::new(object)),
_ => Err(UpstreamInfoFailure::InvalidResponse),
}
}
fn valid_text(value: &str, max: usize) -> bool {
!value.is_empty() && value.len() <= max && !value.chars().any(char::is_control)
}
fn validate_pool(pool: &PeerPool) -> Result<(), UpstreamInfoFailure> {
let count = pool.account_count;
let states = &pool.states;
let state_total = [
states.ready,
states.cooldown,
states.exhausted,
states.disabled,
states.error,
states.unknown,
]
.into_iter()
.map(u64::from)
.sum::<u64>();
if u64::from(count) > MAX_ACCOUNTS as u64 || state_total != u64::from(count) {
return Err(UpstreamInfoFailure::InvalidResponse);
}
if let Some(plans) = &pool.plans
&& (plans.len() > MAX_ACCOUNTS
|| plans
.iter()
.any(|plan| !valid_text(&plan.name, 256) || plan.count > count)
|| plans.iter().map(|plan| u64::from(plan.count)).sum::<u64>() > u64::from(count))
{
return Err(UpstreamInfoFailure::InvalidResponse);
}
Ok(())
}
fn validate_usage(usage: &PeerKeyUsage) -> Result<(), UpstreamInfoFailure> {
if usage
.unit
.as_deref()
.is_some_and(|unit| !valid_text(unit, 16))
|| !usage.windows.is_empty() && usage.unit.is_none()
{
return Err(UpstreamInfoFailure::InvalidResponse);
}
let mut keys = HashSet::new();
for window in &usage.windows {
if !valid_text(&window.key, 256)
|| !keys.insert(&window.key)
|| window
.label
.as_deref()
.is_some_and(|label| !valid_text(label, 256))
|| window.window_seconds == Some(0)
|| [
Some(&window.used),
window.total.as_ref(),
window.remaining.as_ref(),
]
.into_iter()
.flatten()
.any(|value| decimal(value).is_none())
|| window
.reset_at
.as_ref()
.is_some_and(|value| timestamp(value).is_none())
{
return Err(UpstreamInfoFailure::InvalidResponse);
}
}
Ok(())
}
fn validate_quota(quota: &PeerQuota) -> Result<(), UpstreamInfoFailure> {
for account in &quota.accounts {
if !valid_text(&account.id, 256)
|| account.models.len() > MAX_MODELS
|| account.windows.iter().any(|window| {
!valid_text(&window.key, 256)
|| window
.label
.as_deref()
.is_some_and(|label| !valid_text(label, 256))
|| window
.group
.as_deref()
.is_some_and(|group| !valid_text(group, 128))
|| window
.limit_id
.as_deref()
.is_some_and(|value| !valid_text(value, 256))
|| window
.limit_name
.as_deref()
.is_some_and(|value| !valid_text(value, 256))
|| window
.used_percent
.is_some_and(|value| !value.is_finite() || !(0.0..=100.0).contains(&value))
|| window
.reset_at
.as_ref()
.is_some_and(|value| timestamp(value).is_none())
})
{
return Err(UpstreamInfoFailure::InvalidResponse);
}
}
Ok(())
}
pub(super) fn decimal(value: &str) -> Option<f64> {
if value.is_empty()
|| value.len() > 64
|| value
.bytes()
.any(|byte| !byte.is_ascii_digit() && byte != b'.')
|| value.bytes().filter(|byte| *byte == b'.').count() > 1
{
return None;
}
value
.parse::<f64>()
.ok()
.filter(|value| value.is_finite() && *value >= 0.0)
}
@@ -0,0 +1,50 @@
//! 将互联同步交给 Host 周期监督并响应取消
use std::{sync::Arc, time::Duration};
use futures::future::BoxFuture;
use gateway_core::task::{
ScheduledTask, WorkerContribution, WorkerCycleContext, WorkerDefinitionError, WorkerId,
WorkerKind, WorkerLeaseRequest, WorkerRegistration, WorkerRunnable, WorkerSchedule,
WorkerTaskError,
};
use super::PeerInfoService;
impl PeerInfoService {
pub(crate) fn worker(self: &Arc<Self>) -> Result<WorkerContribution, WorkerDefinitionError> {
let id = WorkerId::try_new(WorkerKind::QuotaCatalogHealth, "peer-info")?;
let ttl = Duration::from_secs(120);
let schedule = WorkerSchedule::try_new(
Duration::from_secs(15),
Duration::from_secs(1),
Duration::from_secs(60),
ttl,
Duration::from_secs(30),
)?;
let lease = WorkerLeaseRequest::try_new(id.clone(), ttl)?;
Ok(WorkerContribution::Registration(
WorkerRegistration::try_new(
id,
WorkerRunnable::Scheduled {
schedule,
lease: Some(lease),
task: Box::new(PeerInfoTask(Arc::clone(self))),
},
)?,
))
}
}
struct PeerInfoTask(Arc<PeerInfoService>);
impl ScheduledTask for PeerInfoTask {
fn run_cycle(&self, context: WorkerCycleContext) -> BoxFuture<'_, Result<(), WorkerTaskError>> {
Box::pin(async move {
tokio::select! {
() = context.cancellation().cancelled() => Ok(()),
result = self.0.synchronize(&context) => result.map_err(|error| WorkerTaskError::safe("upstream information synchronization failed").with_source(error)),
}
})
}
}
@@ -29,6 +29,7 @@ pub(crate) struct DefaultPluginAccountAccess {
providers: ProviderAdminRegistry,
accounts: Arc<dyn AccountStore>,
snapshot: Arc<dyn SnapshotControl>,
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
}
impl DefaultPluginAccountAccess {
@@ -37,11 +38,13 @@ impl DefaultPluginAccountAccess {
providers: ProviderAdminRegistry,
accounts: Arc<dyn AccountStore>,
snapshot: Arc<dyn SnapshotControl>,
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
) -> Self {
Self {
providers,
accounts,
snapshot,
upstream_info,
}
}
@@ -51,14 +54,19 @@ impl DefaultPluginAccountAccess {
refresh: bool,
) -> Result<crate::model::provider_credentials::ProviderQuota, AdminError> {
let account = self.get_runtime(account_id).await?;
self.provider(&account.provider_kind)?
.quota(crate::model::provider_credentials::ProviderQuotaRequest {
let provider = self.provider(&account.provider_kind)?;
super::peer_link::upstream::read_quota(
self.upstream_info.as_deref(),
provider.as_ref(),
&account,
crate::model::provider_credentials::ProviderQuotaRequest {
account_id: account_id.clone(),
refresh,
rolling_usage: None,
})
.await
.map_err(|error| map_provider_error(error, "plugin quota facts"))
},
)
.await
.map_err(|error| map_provider_error(error, "plugin quota facts"))
}
fn provider(
@@ -165,7 +173,9 @@ impl PluginAccountAccess for DefaultPluginAccountAccess {
.credential
.credential_revision;
publish_credentials_and_observe_quota(
self.upstream_info.as_ref(),
&provider,
&self.accounts,
self.snapshot.as_ref(),
result.config_revision,
std::slice::from_ref(&account_id),
@@ -208,7 +218,9 @@ impl PluginAccountAccess for DefaultPluginAccountAccess {
.credential_revision
.ok_or_else(|| AdminError::internal("账号保存未返回凭据版本"))?;
publish_credentials_and_observe_quota(
self.upstream_info.as_ref(),
&provider,
&self.accounts,
self.snapshot.as_ref(),
result.config_revision,
std::slice::from_ref(&result.account_id),
@@ -83,6 +83,7 @@ impl PluginResourceAccess for DefaultPluginResourceAccess {
resource_key,
groups,
NewClientKey {
peer_sharing: Default::default(),
id,
name: command.name,
label: None,
@@ -87,6 +87,7 @@ pub(crate) struct DefaultProxiesService {
probe: Arc<dyn ProxyProbe>,
snapshot: Arc<dyn SnapshotControl>,
providers: ProviderAdminRegistry,
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
test_slots: Semaphore,
}
@@ -96,12 +97,14 @@ impl DefaultProxiesService {
probe: Arc<dyn ProxyProbe>,
snapshot: Arc<dyn SnapshotControl>,
providers: ProviderAdminRegistry,
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
) -> Self {
Self {
store,
probe,
snapshot,
providers,
upstream_info,
test_slots: Semaphore::new(4),
}
}
@@ -226,19 +229,30 @@ impl ProxiesService for DefaultProxiesService {
.map(|mut account| async {
// 与账号目录一致,缺失套餐时读取已有额度快照;限制并发且不触发上游刷新
let mut cached_quota = None;
if explicit_plan_type(account.plan_type.as_deref()).is_none()
&& let Ok(kind) = ProviderKind::new(account.provider_kind.clone())
if let Ok(kind) = ProviderKind::new(account.provider_kind.clone())
&& let Ok(provider) = self.providers.require(&kind)
&& let Ok(account_id) = ProviderAccountId::new(account.id.clone())
&& let Ok(quota) = provider
.quota(ProviderQuotaRequest {
account_id,
refresh: false,
rolling_usage: None,
})
.await
{
cached_quota = Some(quota);
if provider.supports_peer_connection(&account.authentication_kind)
&& let Some(peer) = self.upstream_info.as_deref()
{
account.upstream = peer
.read_summary(&account_id)
.await
.ok()
.flatten()
.and_then(|quota| quota.upstream);
}
if explicit_plan_type(account.plan_type.as_deref()).is_none() {
cached_quota = provider
.quota(ProviderQuotaRequest {
account_id,
refresh: false,
rolling_usage: None,
})
.await
.ok();
}
}
account.plan_type_display = self.providers.resolve_account_plan(
&account.provider_kind,
@@ -2,7 +2,7 @@
use std::sync::Arc;
use super::plugin_update::{ConfirmedPluginRestart, PluginSystemUpdatePreflight};
use super::system_preflight::{ConfirmedPluginRestart, PluginSystemUpdatePreflight};
use crate::model::{MutationContext, system::SystemRestartPlan};
use async_trait::async_trait;
@@ -107,6 +107,7 @@ fn provider_document_debug_should_not_expose_opaque_material() {
#[test]
fn representative_quota_should_prefer_short_window_and_highest_usage() {
let quota = ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: None,
@@ -129,6 +130,7 @@ fn representative_quota_should_prefer_account_wide_window_over_model_specific_wi
let mut model_specific = quota_window("shortTerm", Some(18_000), Some(64.0));
model_specific.local_usage_attribution = QuotaLocalUsageAttribution::Unavailable;
let quota = ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: None,
@@ -167,11 +169,13 @@ fn usage_window_should_not_use_daily_rolling_usage_for_weekly_statistics() {
models: Vec::new(),
};
let quota = ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: None,
refresh_token_expires_at: None,
windows: vec![ProviderQuotaWindow {
upstream: None,
key: "free-rolling-24h".to_owned(),
group: "shortTerm".to_owned(),
label: "日限额".to_owned(),
@@ -196,6 +200,7 @@ fn usage_window_should_not_use_daily_rolling_usage_for_weekly_statistics() {
#[test]
fn exhausted_quota_should_project_full_usage_to_only_the_representative_window() {
let mut quota = ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: None,
@@ -225,6 +230,7 @@ fn exhausted_quota_should_preserve_the_provider_identified_reached_window() {
let mut reached = quota_window("monthly", Some(2_592_000), Some(98.0));
reached.limit_reached = true;
let mut quota = ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: None,
@@ -247,6 +253,7 @@ fn quota_window(
used_percent: Option<f64>,
) -> ProviderQuotaWindow {
ProviderQuotaWindow {
upstream: None,
key: group.to_owned(),
group: group.to_owned(),
label: group.to_owned(),
@@ -86,6 +86,7 @@ fn selected_usage_window(quota: &ProviderQuota) -> Option<(&str, AccountUsagePer
fn usage_quota(windows: Vec<ProviderQuotaWindow>) -> ProviderQuota {
ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: None,
@@ -98,6 +99,7 @@ fn usage_quota(windows: Vec<ProviderQuotaWindow>) -> ProviderQuota {
fn usage_window(key: &str, group: &str, seconds: u64) -> ProviderQuotaWindow {
ProviderQuotaWindow {
upstream: None,
key: key.to_owned(),
group: group.to_owned(),
label: key.to_owned(),
@@ -136,6 +138,7 @@ fn usage_window(key: &str, group: &str, seconds: u64) -> ProviderQuotaWindow {
#[test]
fn dashboard_quota_should_preserve_unknown_and_actual_window_facts() {
let mut quota = ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: None,
@@ -148,6 +151,7 @@ fn dashboard_quota_should_preserve_unknown_and_actual_window_facts() {
assert!(quota.representative_used_percent().is_none());
let rolling = ProviderQuotaWindow {
upstream: None,
key: "provider_rolling".to_owned(),
group: "requests".to_owned(),
label: "Provider rolling window".to_owned(),
@@ -180,6 +184,7 @@ fn dashboard_quota_should_preserve_unknown_and_actual_window_facts() {
quota.windows.insert(
0,
ProviderQuotaWindow {
upstream: None,
key: "model_specific".to_owned(),
used_percent: Some(100.0),
local_usage_attribution: QuotaLocalUsageAttribution::Unavailable,
@@ -15,6 +15,7 @@ fn now() -> DateTime<Utc> {
fn window(key: &str, days: u64) -> ProviderQuotaWindow {
ProviderQuotaWindow {
upstream: None,
key: key.to_owned(),
group: if days > 7 { "monthly" } else { "shortTerm" }.to_owned(),
label: key.to_owned(),
@@ -58,6 +59,7 @@ fn window(key: &str, days: u64) -> ProviderQuotaWindow {
fn quota(windows: Vec<ProviderQuotaWindow>) -> ProviderQuota {
ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: Some(now()),
@@ -279,6 +279,10 @@ impl FakeProviderAdmin {
#[async_trait]
impl ProviderAdmin for FakeProviderAdmin {
fn peer_authentication_kinds(&self) -> &[&str] {
&["api_key"]
}
fn account_capabilities(
&self,
_account_id: &ProviderAccountId,
@@ -733,6 +737,20 @@ impl FakeAccountStore {
#[async_trait]
impl AccountStore for FakeAccountStore {
async fn load_account_record(
&self,
account_id: &ProviderAccountId,
) -> AdminStoreResult<Option<AccountRecord>> {
self.record("store.load_account_record");
Ok(self
.accounts
.lock()
.expect("accounts")
.iter()
.find(|account| account.id == account_id.as_str())
.cloned())
}
async fn account_exists(&self, account_id: &str) -> AdminStoreResult<bool> {
self.record("store.account_exists");
Ok(self
@@ -1148,6 +1166,7 @@ impl SettingsStore for StaticSettingsStore {
max_waiting_per_key: 0,
max_waiting_per_account: 0,
concurrency_wait_timeout_seconds: 30,
max_retry_after_seconds: 240,
openai_guardian_reserved_concurrency: 0,
openai_account_affinity: gateway_core::account::AccountAffinity::Relaxed,
max_account_rotations: 3,
@@ -1473,6 +1492,7 @@ async fn plugin_account_save_publishes_native_credential_change() {
ProviderAdminRegistry::new([provider.clone() as Arc<dyn ProviderAdmin>]).unwrap(),
store.clone(),
snapshot,
None,
);
let result = access
.save(
@@ -1516,6 +1536,7 @@ async fn plugin_account_reads_resolve_the_authoritative_provider_from_the_accoun
ProviderAdminRegistry::new([registered_provider as Arc<dyn ProviderAdmin>]).unwrap(),
store,
Arc::new(RecordingPluginAccountPublication(events.clone())),
None,
);
let account_id = ProviderAccountId::new("acct_test").expect("account ID");
@@ -1563,6 +1584,7 @@ async fn plugin_account_save_rejects_stale_authentication_kind_before_commit() {
ProviderAdminRegistry::new([provider as Arc<dyn ProviderAdmin>]).unwrap(),
store.clone(),
Arc::new(RecordingPluginAccountPublication(events.clone())),
None,
);
let error = access
@@ -1692,9 +1714,11 @@ async fn accounts_recover_disabled_should_only_enable_scheduling_and_preserve_qu
);
}
let quota = ProviderQuota {
upstream: None,
credits: None,
observed_at: Some(Utc::now()),
windows: vec![ProviderQuotaWindow {
upstream: None,
key: "primary".to_owned(),
group: "shortTerm".to_owned(),
label: "5小时限额".to_owned(),
@@ -1790,6 +1814,7 @@ async fn accounts_update_should_commit_then_release_disabled_account_and_publish
.update(
&context("update-request"),
UpdateAccount {
name: None,
notes: None,
model_access: Default::default(),
outbound_proxy: None,
@@ -1829,6 +1854,7 @@ async fn accounts_update_should_not_notify_provider_when_store_commit_fails() {
.update(
&context("update-failure"),
UpdateAccount {
name: None,
notes: None,
model_access: Default::default(),
outbound_proxy: None,
@@ -1996,6 +2022,7 @@ async fn accounts_should_fill_missing_plan_from_quota_without_overriding_known_s
] {
let provider = FakeProviderAdmin::new("openai", events());
provider.set_quota(ProviderQuota {
upstream: None,
plan_type: quota_plan.map(str::to_owned),
..empty_quota()
});
@@ -2039,11 +2066,13 @@ async fn accounts_list_should_degrade_quota_failure_to_empty_window_without_drop
let events = events();
let openai = FakeProviderAdmin::new("openai", events.clone());
openai.set_quota(ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: Some(Utc::now()),
refresh_token_expires_at: None,
windows: vec![ProviderQuotaWindow {
upstream: None,
key: "primary".to_owned(),
group: "shortTerm".to_owned(),
label: "5小时限额".to_owned(),
@@ -2197,11 +2226,13 @@ async fn accounts_list_should_map_unknown_credential_to_error_not_normal() {
async fn accounts_list_should_not_derive_rate_limited_from_provider_quota_view() {
let provider = FakeProviderAdmin::new("openai", events());
provider.set_quota(ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: Some(Utc::now()),
refresh_token_expires_at: None,
windows: vec![ProviderQuotaWindow {
upstream: None,
key: "primary".to_owned(),
group: "shortTerm".to_owned(),
label: "5小时限额".to_owned(),
@@ -2246,11 +2277,13 @@ async fn accounts_list_should_not_derive_rate_limited_from_provider_quota_view()
async fn accounts_list_should_not_derive_exhaustion_from_provider_quota_view() {
let provider = FakeProviderAdmin::new("openai", events());
provider.set_quota(ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: Some(Utc::now()),
refresh_token_expires_at: None,
windows: vec![ProviderQuotaWindow {
upstream: None,
key: "primary".to_owned(),
group: "shortTerm".to_owned(),
label: "5小时限额".to_owned(),
@@ -2298,10 +2331,12 @@ async fn quota_forecast_reads_raw_snapshot_and_limits_usage_to_observation_time(
let observed = now - TimeDelta::hours(1);
let reset = now + TimeDelta::days(1);
provider.set_quota(ProviderQuota {
upstream: None,
credits: None,
observed_at: Some(observed),
limit_reached: true,
windows: vec![ProviderQuotaWindow {
upstream: None,
key: "week".to_owned(),
group: "shortTerm".to_owned(),
label: "周额度".to_owned(),
@@ -2390,6 +2425,7 @@ async fn quota_forecast_only_queries_the_two_selected_source_windows() {
.map(|index| {
let monthly = index % 2 == 1;
ProviderQuotaWindow {
upstream: None,
key: if monthly {
format!("month-{index}")
} else {
@@ -2411,6 +2447,7 @@ async fn quota_forecast_only_queries_the_two_selected_source_windows() {
})
.collect();
provider.set_quota(ProviderQuota {
upstream: None,
plan_type: Some("pro".to_owned()),
observed_at: Some(observed),
windows,
@@ -2462,9 +2499,11 @@ async fn quota_forecast_same_window_reversal_keeps_recorded_usage_in_the_report(
let reset = now + TimeDelta::days(1);
let provider = FakeProviderAdmin::new("openai", events());
provider.set_quota(ProviderQuota {
upstream: None,
plan_type: Some("pro".to_owned()),
observed_at: Some(observed),
windows: vec![ProviderQuotaWindow {
upstream: None,
key: "week".to_owned(),
group: "shortTerm".to_owned(),
label: "周额度".to_owned(),
@@ -2540,10 +2579,12 @@ async fn quota_forecast_mid_cycle_sampling_accepts_small_reset_jitter_but_not_a_
let added = now - TimeDelta::hours(5);
let provider = FakeProviderAdmin::new("openai", events());
provider.set_quota(ProviderQuota {
upstream: None,
credits: None,
plan_type: Some("pro".to_owned()),
observed_at: Some(observed),
windows: vec![ProviderQuotaWindow {
upstream: None,
key: "week".to_owned(),
group: "shortTerm".to_owned(),
label: "周额度".to_owned(),
@@ -2636,6 +2677,156 @@ async fn quota_forecast_mid_cycle_sampling_accepts_small_reset_jitter_but_not_a_
assert_eq!(cycle.estimated_tokens, Some(679));
}
#[tokio::test]
async fn peer_account_usage_should_follow_budget_windows_without_borrowing_pool_windows() {
use gateway_admin::model::upstream_info::{
ProviderUpstreamInfo, UpstreamQuotaAmount, UpstreamQuotaWindow,
};
for longest in [86_400, 604_800] {
let provider = FakeProviderAdmin::new("openai", events());
let mut account = account_record("openai");
account.authentication_kind = "api_key".to_owned();
account.created_at = Utc::now() - TimeDelta::days(60);
let store = FakeAccountStore::with_account(account, events());
let reset_at = Utc::now() + TimeDelta::hours(1);
let mut quota = empty_quota();
quota.upstream = Some(ProviderUpstreamInfo {
discovered: true,
observed_at: Some(Utc::now()),
stale: false,
connection_revision: "1".to_owned(),
last_error: None,
pool: None,
});
quota.windows = [86_400, 604_800, 30 * 86_400]
.into_iter()
.filter(|seconds| *seconds <= longest || *seconds == 30 * 86_400)
.map(|seconds| ProviderQuotaWindow {
upstream: Some(UpstreamQuotaWindow {
account_id: None,
observed_at: Some(Utc::now()),
stale: false,
amount: (seconds <= longest).then(|| UpstreamQuotaAmount {
unit: "USD".to_owned(),
used: "1".to_owned(),
remaining: Some("9".to_owned()),
total: Some("10".to_owned()),
}),
}),
key: seconds.to_string(),
group: "general".to_owned(),
label: "额度".to_owned(),
limit_id: None,
limit_name: None,
role: None,
local_usage_attribution: if seconds <= longest {
QuotaLocalUsageAttribution::AccountWide
} else {
QuotaLocalUsageAttribution::Unavailable
},
window_seconds: Some(seconds),
used_percent: Some(10.0),
reset_at: Some(reset_at),
limit_reached: false,
local_usage: None,
provider_data: None,
})
.collect();
provider.set_quota(quota.clone());
let window_usage = [86_400, 604_800]
.into_iter()
.filter(|seconds| *seconds <= longest)
.map(|seconds| AccountUsageWindowResult {
account_id: "acct_test".to_owned(),
key: seconds.to_string(),
usage: quota_local_usage("acct_test", if seconds == longest { 1_234 } else { 123 }),
})
.collect::<Vec<_>>();
store.set_quota_window_usage(window_usage.clone());
let services = accounts_service(provider.clone(), store.clone()).await;
let page = services
.accounts()
.list(AccountListQuery {
page: 1,
page_size: gateway_admin::model::PageSize::new(20).unwrap(),
provider_kind: None,
group_filter: None,
search: None,
status: None,
sort: None,
})
.await
.unwrap();
assert_eq!(
page.items[0].usage.as_ref().unwrap().total_tokens,
Some(1_234)
);
let queries = store.quota_window_queries();
assert_eq!(queries.len(), window_usage.len());
for (query, expected) in queries.iter().zip(&window_usage) {
assert_eq!(query.key, expected.key);
assert_eq!(
query.range.start,
reset_at - TimeDelta::seconds(query.key.parse().unwrap())
);
assert_eq!(query.range.end, reset_at);
let window = page.items[0]
.quota
.windows
.iter()
.find(|window| window.key == query.key)
.unwrap();
assert_eq!(window.local_usage.as_ref(), Some(&expected.usage));
assert_eq!(window.used_percent, Some(10.0));
assert_eq!(
window
.upstream
.as_ref()
.unwrap()
.amount
.as_ref()
.unwrap()
.used,
"1"
);
}
assert!(
page.items[0]
.quota
.windows
.last()
.unwrap()
.local_usage
.is_none()
);
let id = ProviderAccountId::new("acct_test").unwrap();
let detail = services.accounts().quota(&id, false).await.unwrap();
assert_eq!(detail.usage, page.items[0].usage);
assert_eq!(detail.quota.windows, page.items[0].quota.windows);
assert_eq!(store.quota_window_queries()[0].range, queries[0].range);
quota
.windows
.iter_mut()
.find(|window| window.window_seconds == Some(longest))
.unwrap()
.reset_at = None;
provider.set_quota(quota);
store.set_quota_window_usage(
window_usage
.into_iter()
.filter(|usage| usage.key != longest.to_string())
.collect(),
);
let unopened = services.accounts().quota(&id, false).await.unwrap();
assert!(
unopened.usage.is_none(),
"未开启的最长窗口不能借用日窗口或历史累计"
);
}
}
#[tokio::test]
async fn api_key_list_and_detail_should_accumulate_local_usage_without_subscription_windows() {
let provider = FakeProviderAdmin::new("openai", events());
@@ -2687,11 +2878,13 @@ async fn accounts_list_should_attach_local_usage_to_quota_windows() {
let provider = FakeProviderAdmin::new("openai", events());
let reset_at = Utc::now() + TimeDelta::hours(1);
provider.set_quota(ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: Some(Utc::now()),
refresh_token_expires_at: None,
windows: vec![ProviderQuotaWindow {
upstream: None,
key: "primary".to_owned(),
group: "shortTerm".to_owned(),
label: "5小时限额".to_owned(),
@@ -2756,6 +2949,7 @@ async fn accounts_list_and_quota_refresh_should_select_the_same_weekly_or_monthl
let account_id = ProviderAccountId::new("acct_test").expect("account id");
let reset_at = Utc::now() + TimeDelta::days(1);
let short = ProviderQuotaWindow {
upstream: None,
key: "short".to_owned(),
group: "shortTerm".to_owned(),
label: "5小时限额".to_owned(),
@@ -2771,12 +2965,14 @@ async fn accounts_list_and_quota_refresh_should_select_the_same_weekly_or_monthl
provider_data: None,
};
let week = ProviderQuotaWindow {
upstream: None,
key: "week".to_owned(),
label: "周额度".to_owned(),
window_seconds: Some(7 * 86_400),
..short.clone()
};
let month = ProviderQuotaWindow {
upstream: None,
key: "month".to_owned(),
group: "monthly".to_owned(),
label: "月额度".to_owned(),
@@ -2788,6 +2984,7 @@ async fn accounts_list_and_quota_refresh_should_select_the_same_weekly_or_monthl
(vec![short, month], "month", 3000, 30),
] {
provider.set_quota(ProviderQuota {
upstream: None,
windows,
..empty_quota()
});
@@ -2845,12 +3042,14 @@ async fn accounts_list_should_not_attach_account_usage_to_model_specific_quota_w
let provider = FakeProviderAdmin::new("openai", events());
let reset_at = Utc::now() + TimeDelta::days(7);
provider.set_quota(ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: Some(Utc::now()),
refresh_token_expires_at: None,
windows: vec![
ProviderQuotaWindow {
upstream: None,
key: "core-primary".to_owned(),
group: "weekly".to_owned(),
label: "周额度".to_owned(),
@@ -2866,6 +3065,7 @@ async fn accounts_list_should_not_attach_account_usage_to_model_specific_quota_w
provider_data: None,
},
ProviderQuotaWindow {
upstream: None,
key: "codex-bengalfox-primary".to_owned(),
group: "weekly".to_owned(),
label: "周额度".to_owned(),
@@ -3147,6 +3347,7 @@ pub(super) fn document() -> ProviderDocument {
fn empty_quota() -> ProviderQuota {
ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: None,
@@ -3542,6 +3743,7 @@ fn unsupported() -> ProviderAdminError {
pub(super) fn import_settings() -> gateway_admin::model::accounts::AccountImportSettings {
gateway_admin::model::accounts::AccountImportSettings {
name: None,
notes: Some("团队备用".to_owned()),
model_access: Default::default(),
enabled: false,
@@ -3591,6 +3793,7 @@ async fn plugin_quota_refresh_uses_native_provider_without_changing_configuratio
ProviderAdminRegistry::new([provider.clone() as Arc<dyn ProviderAdmin>]).unwrap(),
store,
Arc::new(RecordingPluginAccountPublication(events.clone())),
None,
);
let id = ProviderAccountId::new("acct_test").unwrap();
access.get_quota(&id).await.unwrap();
@@ -726,7 +726,9 @@ async fn threshold_services(
fn weekly_quota(used_percent: f64) -> ProviderQuota {
ProviderQuota {
upstream: None,
windows: vec![ProviderQuotaWindow {
upstream: None,
key: "week".into(),
group: "shortTerm".into(),
label: "周额度".into(),
@@ -870,6 +872,7 @@ async fn automation_defers_missing_invalid_or_failed_weekly_quota() {
(empty_quota(), None),
(
ProviderQuota {
upstream: None,
windows: vec![short, model, month],
..empty_quota()
},
@@ -877,6 +880,7 @@ async fn automation_defers_missing_invalid_or_failed_weekly_quota() {
),
(
ProviderQuota {
upstream: None,
windows: vec![missing_percent],
..empty_quota()
},
@@ -96,6 +96,7 @@ impl ClientKeyStore for TestClientKeyStore {
}
self.plaintexts.lock().unwrap().push(key.plaintext);
let record = ClientKeyRecord {
peer_sharing: key.peer_sharing,
request_profile_overrides: Default::default(),
id: key.id,
name: key.name,
@@ -234,6 +235,7 @@ async fn plugin_client_key_list_projects_only_public_identity_and_uses_stable_na
list_response: Mutex::new(Some(ClientKeyPage {
config_revision: Revision::new(7).expect("revision"),
items: vec![ClientKeyRecord {
peer_sharing: Default::default(),
request_profile_overrides: Default::default(),
id: key_id.clone(),
name: "Public identity".to_owned(),
@@ -370,6 +372,7 @@ async fn duplicate_names_report_the_same_actionable_conflict_on_create_and_updat
.update(
&mutation_context(),
UpdateClientKey {
peer_sharing: None,
request_profile_override_updates: Default::default(),
id: ClientApiKeyId::new("key_existing").unwrap(),
name: "Migration".to_owned(),
@@ -390,6 +393,7 @@ async fn duplicate_names_report_the_same_actionable_conflict_on_create_and_updat
fn create_command(key: Option<&str>) -> CreateClientKey {
CreateClientKey {
peer_sharing: Default::default(),
request_profile_overrides: Default::default(),
custom_key: key.map(|value| PlaintextClientApiKey::new(value).unwrap()),
name: "Migration".to_owned(),
@@ -400,6 +404,53 @@ fn create_command(key: Option<&str>) -> CreateClientKey {
}
}
#[tokio::test]
async fn peer_sharing_requires_explicit_groups_and_independent_plan_authorization_before_writing() {
use gateway_core::policy::ClientPeerSharing;
let store = Arc::new(TestClientKeyStore::default());
let services = super::AdminHarness::new()
.client_keys(store.clone())
.build()
.await;
for sharing in [
ClientPeerSharing {
enabled: true,
..Default::default()
},
ClientPeerSharing {
share_plans: true,
..Default::default()
},
] {
let mut command = create_command(None);
command.peer_sharing = sharing;
let error = services
.client_keys()
.create(&mutation_context(), command)
.await
.unwrap_err();
assert_eq!(error.kind(), AdminErrorKind::Invalid);
}
assert!(store.plaintexts.lock().unwrap().is_empty());
let mut command = create_command(None);
command.peer_sharing = ClientPeerSharing {
enabled: true,
share_pool: true,
share_quota: true,
share_plans: true,
};
command.group_ids.push(
gateway_core::routing::AccountGroupId::new(format!("grp_{}", "1".repeat(32))).unwrap(),
);
let expected = command.peer_sharing;
let result = services
.client_keys()
.create(&mutation_context(), command)
.await
.unwrap();
assert_eq!(result.secret.record.peer_sharing, expected);
}
fn mutation_context() -> MutationContext {
MutationContext {
actor: MutationActor::System,
@@ -436,6 +487,7 @@ async fn duplicate_group_ids_are_rejected_before_key_creation_or_update() {
.update(
&mutation_context(),
UpdateClientKey {
peer_sharing: None,
request_profile_override_updates: Default::default(),
id: ClientApiKeyId::new("key_existing").unwrap(),
name: "Migration".to_owned(),
@@ -1,4 +1,5 @@
//! 按 Provider 组织凭据导入、授权与提交边界测试
mod openai;
mod peer;
mod xai;
@@ -0,0 +1,184 @@
//! 验证凭据提交后的互联首查、原生额度分派与异步失败边界
use std::sync::{Arc, Mutex};
use std::time::Duration;
use async_trait::async_trait;
use gateway_admin::{
model::provider_credentials::{ImportCredentials, ProviderQuota, ProviderQuotaReadRequest},
ports::{
peer::PeerUpstreamService,
provider::{ProviderAdminError, ProviderAdminErrorKind},
},
};
use gateway_core::{
account::{ProviderAccount, ProviderAccountId, UpstreamInfoSource, UpstreamSchedulingSignal},
error::StoreError,
routing::ProviderKind,
};
use super::super::{
AdminHarness,
accounts::{FakeAccountStore, FakeProviderAdmin, account_record, context, document, events},
};
#[tokio::test]
async fn imported_peer_accounts_should_start_observation_without_waiting_for_network() {
let events = events();
let provider = FakeProviderAdmin::new("openai", events.clone());
provider.set_import_account_ids(&["acct_first", "acct_second"]);
let mut first = account_record("openai");
first.id = "acct_first".into();
first.authentication_kind = "api_key".into();
let mut second = first.clone();
second.id = "acct_second".into();
let store = FakeAccountStore::with_account(first.clone(), events);
store.set_accounts(vec![first, second]);
let peer = Arc::new(PendingPeer::default());
let services = AdminHarness::new()
.provider(provider.clone())
.accounts(store)
.upstream_info(peer.clone())
.build()
.await;
let result = tokio::time::timeout(
Duration::from_secs(1),
services
.credentials()
.for_provider(&ProviderKind::new("openai").unwrap())
.unwrap()
.import_document(ImportCredentials {
outbound_proxy_id: None,
settings: None,
context: context("import-peer"),
document: document(),
}),
)
.await
.expect("import must not wait for peer response")
.expect("import committed");
peer.wait_for_requests(1).await;
assert_eq!(peer.changed.lock().unwrap().len(), 2);
assert_eq!(result.credential_ids.len(), 2);
assert!(provider.quota_requests().is_empty());
peer.gate.add_permits(2);
peer.wait_for_requests(2).await;
assert_eq!(
*peer.requests.lock().unwrap(),
["acct_first", "acct_second"]
);
assert!(provider.quota_requests().is_empty());
}
#[tokio::test]
async fn native_import_should_use_provider_quota_when_peer_service_is_available() {
let events = events();
let provider = FakeProviderAdmin::new("openai", events.clone());
provider.set_import_account_ids(&["acct_test"]);
let store = FakeAccountStore::new("openai", events);
let peer = Arc::new(PendingPeer::default());
let services = AdminHarness::new()
.provider(provider.clone())
.accounts(store)
.upstream_info(peer.clone())
.build()
.await;
services
.credentials()
.for_provider(&ProviderKind::new("openai").unwrap())
.unwrap()
.import_document(ImportCredentials {
outbound_proxy_id: None,
settings: None,
context: context("import-native"),
document: document(),
})
.await
.expect("native import");
provider.wait_for_quota_requests(1).await;
assert!(peer.requests.lock().unwrap().is_empty());
assert_eq!(
provider.quota_requests()[0].account_id.as_str(),
"acct_test"
);
}
struct PendingPeer {
changed: Mutex<Vec<String>>,
requests: Mutex<Vec<String>>,
started: tokio::sync::Notify,
gate: tokio::sync::Semaphore,
}
impl Default for PendingPeer {
fn default() -> Self {
Self {
changed: Mutex::default(),
requests: Mutex::default(),
started: tokio::sync::Notify::new(),
gate: tokio::sync::Semaphore::new(0),
}
}
}
impl PendingPeer {
async fn wait_for_requests(&self, count: usize) {
tokio::time::timeout(Duration::from_secs(1), async {
loop {
let notified = self.started.notified();
if self.requests.lock().unwrap().len() >= count {
return;
}
notified.await;
}
})
.await
.expect("initial peer observation started");
}
}
impl UpstreamInfoSource for PendingPeer {
fn signal(
&self,
_: &ProviderAccount,
_: &str,
_: std::time::SystemTime,
) -> Option<UpstreamSchedulingSignal> {
None
}
}
#[async_trait]
impl PeerUpstreamService for PendingPeer {
async fn account_changed(&self, id: &ProviderAccountId) -> Result<(), StoreError> {
self.changed.lock().unwrap().push(id.as_str().into());
Ok(())
}
fn forget(&self, _: &ProviderAccountId) {}
async fn refresh_quota(
&self,
id: &ProviderAccountId,
) -> Result<ProviderQuota, ProviderAdminError> {
self.requests.lock().unwrap().push(id.as_str().into());
self.started.notify_one();
self.gate.acquire().await.unwrap().forget();
Err(ProviderAdminError::new(ProviderAdminErrorKind::Unavailable))
}
async fn read_quota(&self, _: &ProviderAccountId) -> Result<Option<ProviderQuota>, StoreError> {
panic!("unexpected cached read")
}
async fn read_summary(
&self,
_: &ProviderAccountId,
) -> Result<Option<ProviderQuota>, StoreError> {
panic!("unexpected summary read")
}
async fn read_quotas(
&self,
_: &[ProviderQuotaReadRequest<'_>],
) -> Vec<Result<ProviderQuota, ProviderAdminError>> {
panic!("unexpected batch read")
}
}
@@ -41,6 +41,7 @@ fn runtime_settings(enabled: bool, probe_enabled: bool) -> RuntimeSettings {
max_waiting_per_key: 0,
max_waiting_per_account: 0,
concurrency_wait_timeout_seconds: 30,
max_retry_after_seconds: 240,
openai_guardian_reserved_concurrency: 0,
openai_account_affinity: gateway_core::account::AccountAffinity::Relaxed,
max_account_rotations: 3,
@@ -10,11 +10,12 @@ mod credentials;
mod freeze_recovery;
mod import_tasks;
mod observability;
mod plugin_update;
mod peer_link;
mod plugins;
mod proxies;
mod settings;
mod system;
mod system_preflight;
mod usage_data;
use std::{
@@ -111,6 +112,7 @@ pub(super) struct AdminHarness {
plugin_store: Arc<dyn gateway_admin::ports::plugins::PluginStore>,
plugin_inspector: Arc<dyn gateway_admin::ports::plugins::PluginPackageInspector>,
client_key_verifier: Arc<dyn ClientKeyVerifier>,
upstream_info: Option<Arc<dyn gateway_admin::ports::peer::PeerUpstreamService>>,
service_middleware: gateway_admin::public_service::PlanSource,
}
@@ -141,6 +143,7 @@ impl AdminHarness {
plugin_store: Arc::new(plugins::TestPluginPorts),
plugin_inspector: Arc::new(plugins::TestPluginPorts),
client_key_verifier: Arc::new(UnavailableClientKeyVerifier),
upstream_info: None,
service_middleware: Arc::new(|| None),
}
}
@@ -160,6 +163,14 @@ impl AdminHarness {
self
}
pub(super) fn upstream_info(
mut self,
peer: Arc<dyn gateway_admin::ports::peer::PeerUpstreamService>,
) -> Self {
self.upstream_info = Some(peer);
self
}
pub(super) fn account_runtime(mut self, store: Arc<dyn AccountRuntimeStore>) -> Self {
self.account_runtime = store;
self
@@ -286,6 +297,7 @@ impl AdminHarness {
Arc::new(plugins::TestPluginPorts),
),
gateway_admin::AdminRuntimePorts {
upstream_info: self.upstream_info,
timezone: Default::default(),
service_middleware: self.service_middleware,
plugin_preparation: Arc::new(plugins::TestPluginPorts),
@@ -485,6 +497,13 @@ impl AccountGroupStore for UnavailableAccountGroupStore {
#[async_trait]
impl AccountStore for UnavailableStore {
async fn load_account_record(
&self,
_: &ProviderAccountId,
) -> AdminStoreResult<Option<gateway_admin::model::accounts::AccountRecord>> {
Err(unavailable("account"))
}
async fn account_exists(&self, _: &str) -> AdminStoreResult<bool> {
Err(unavailable("account"))
}
@@ -909,6 +928,7 @@ impl SettingsStore for UnavailableStore {
}
struct UnavailableProvider {
peer_enabled: bool,
kind: ProviderKind,
dashboard_profile: Option<DashboardWireProfile>,
calculated_billing: Option<gateway_admin::model::observability::CalculatedBillingBreakdown>,
@@ -931,6 +951,7 @@ impl UnavailableProvider {
fn new(kind: &str) -> Self {
Self {
kind: ProviderKind::new(kind).expect("provider kind"),
peer_enabled: false,
dashboard_profile: None,
calculated_billing: None,
}
@@ -943,6 +964,28 @@ pub(super) fn test_provider(kind: &str) -> Arc<dyn ProviderAdmin> {
#[async_trait]
impl ProviderAdmin for UnavailableProvider {
fn peer_authentication_kinds(&self) -> &[&str] {
if self.peer_enabled {
&["peer_test"]
} else {
&[]
}
}
fn peer_connection(
&self,
loaded: &gateway_core::account::LoadedCredential,
) -> Result<Option<gateway_admin::ports::peer::PeerConnection>, ProviderAdminError> {
Ok(
(self.peer_enabled && loaded.account.authentication_kind() == "peer_test").then(|| {
gateway_admin::ports::peer::PeerConnection {
base_url: format!("https://{}.invalid/v1", self.kind),
bearer_token: secrecy::SecretString::from("test-peer-only"),
}
}),
)
}
fn provider_kind(&self) -> &ProviderKind {
&self.kind
}
@@ -1031,6 +1074,7 @@ impl ProviderAdmin for UnavailableProvider {
pub(super) fn dashboard_profile_provider() -> Arc<dyn ProviderAdmin> {
Arc::new(UnavailableProvider {
peer_enabled: false,
kind: ProviderKind::new("openai").expect("provider kind"),
dashboard_profile: Some(DashboardWireProfile {
provider: "openai".to_owned(),
@@ -1074,6 +1118,7 @@ pub(super) fn calculated_billing_provider() -> Arc<dyn ProviderAdmin> {
.expect("test billing amount"),
};
Arc::new(UnavailableProvider {
peer_enabled: false,
kind: ProviderKind::new("openai").expect("provider kind"),
dashboard_profile: None,
calculated_billing: Some(
@@ -995,6 +995,7 @@ async fn dashboard_should_preserve_provider_plan_names() {
*store.account_usage.lock().unwrap() = plans
.iter()
.map(|(plan, _)| DashboardAccountUsage {
upstream: None,
account_id: format!("account-{plan}"),
provider_kind: "openai".to_owned(),
authentication_kind: "oauth".to_owned(),
@@ -1383,6 +1384,7 @@ impl SettingsStore for FixtureSettingsStore {
max_waiting_per_key: 0,
max_waiting_per_account: 0,
concurrency_wait_timeout_seconds: 30,
max_retry_after_seconds: 240,
openai_guardian_reserved_concurrency: 0,
openai_account_affinity: gateway_core::account::AccountAffinity::Relaxed,
max_account_rotations: 3,
@@ -1559,6 +1561,7 @@ fn diagnostic(name: &str, request_count: u64) -> DiagnosticObservation {
retry_count: 0,
retried_request_count: 0,
account_provider_kind: None,
account_authentication_kind: None,
account_plan_type: None,
key: name.to_owned(),
name: name.to_owned(),
@@ -0,0 +1,3 @@
//! 互联共享与上游同步用例测试
mod upstream;
@@ -0,0 +1,534 @@
//! 公共互联按 Provider 声明发现连接,不将认证种类或调度观测绑在 OpenAI 上
mod refresh;
use super::super::UnavailableProvider;
use async_trait::async_trait;
use chrono::Utc;
use gateway_admin::{
initialize_peer_upstream,
ports::{
peer::{PeerConnection, PeerFetchFailure, PeerTransport},
provider::{ProviderAdmin, ProviderAdminRegistry},
},
};
use gateway_core::{
account::*,
error::{StoreError, StoreErrorKind},
lifecycle::CancellationToken,
routing::ProviderKind,
task::{WorkerContribution, WorkerCycleContext, WorkerRunnable},
};
use std::{
collections::BTreeMap,
sync::{
Arc, Mutex,
atomic::{AtomicUsize, Ordering},
},
time::{Duration, SystemTime},
};
#[tokio::test]
async fn manual_and_periodic_refreshes_share_capacity_and_release_it_after_failure_or_cancellation()
{
let providers = ["openai", "xai"].map(|kind| {
let mut provider = UnavailableProvider::new(kind);
provider.peer_enabled = true;
Arc::new(provider) as Arc<dyn ProviderAdmin>
});
let registry = ProviderAdminRegistry::new(providers).unwrap();
let records = (0..100)
.map(|index| {
account(
&format!("acct_{index}"),
if index % 2 == 0 { "openai" } else { "xai" },
"peer_test",
)
})
.collect::<Vec<_>>();
let accounts = Arc::new(TestAccounts {
accounts: Mutex::new(records.clone()),
records: Mutex::default(),
failed_provider: None,
});
let transport = Arc::new(GatedTransport::new());
let mut bundle = initialize_peer_upstream(accounts, &registry, transport.clone()).unwrap();
let service = bundle.service();
let mut requests = Vec::new();
for record in &records[..20] {
let service = service.clone();
let id = record.id().clone();
requests.push(tokio::spawn(
async move { service.refresh_quota(&id).await },
));
}
transport.wait_for_requests(4).await;
let WorkerContribution::Registration(worker) = bundle.take_worker_contributions().remove(0)
else {
panic!("scheduled worker")
};
let worker = tokio::spawn(async move {
let WorkerRunnable::Scheduled { task, .. } = worker.runnable else {
panic!("scheduled worker")
};
task.run_cycle(WorkerCycleContext::new(
worker.id,
None,
CancellationToken::new(),
))
.await
});
assert!(
tokio::time::timeout(Duration::from_millis(50), transport.wait_for_requests(5))
.await
.is_err()
);
requests.remove(0).abort();
transport.wait_for_requests(5).await;
transport.gate.add_permits(100);
tokio::time::timeout(Duration::from_secs(5), async {
for (index, request) in requests.into_iter().enumerate() {
let result = request.await.unwrap();
assert_eq!(result.is_err(), index == 0);
}
worker.await.unwrap().unwrap();
})
.await
.expect("all queued refreshes complete");
assert_eq!(transport.peak.load(Ordering::SeqCst), 4);
assert_eq!(transport.active.load(Ordering::SeqCst), 0);
assert_eq!(transport.started_count.load(Ordering::SeqCst), 100);
}
#[tokio::test]
async fn queued_refresh_reads_current_account_after_capacity_becomes_available() {
let mut provider = UnavailableProvider::new("openai");
provider.peer_enabled = true;
let registry =
ProviderAdminRegistry::new([Arc::new(provider) as Arc<dyn ProviderAdmin>]).unwrap();
let records = (0..5)
.map(|index| account(&format!("acct_{index}"), "openai", "peer_test"))
.collect::<Vec<_>>();
let accounts = Arc::new(TestAccounts {
accounts: Mutex::new(records.clone()),
records: Mutex::default(),
failed_provider: None,
});
let transport = Arc::new(GatedTransport::new());
let bundle = initialize_peer_upstream(accounts.clone(), &registry, transport.clone()).unwrap();
let service = bundle.service();
let mut requests = Vec::new();
for record in &records[..4] {
let service = service.clone();
let id = record.id().clone();
requests.push(tokio::spawn(
async move { service.refresh_quota(&id).await },
));
}
transport.wait_for_requests(4).await;
let queued = tokio::spawn(async move { service.refresh_quota(records[4].id()).await });
assert!(
tokio::time::timeout(Duration::from_millis(50), transport.wait_for_requests(5))
.await
.is_err()
);
accounts
.accounts
.lock()
.unwrap()
.retain(|account| account.id().as_str() != "acct_4");
transport.gate.add_permits(4);
for request in requests {
let _result = request.await.unwrap();
}
assert!(
tokio::time::timeout(Duration::from_secs(1), queued)
.await
.unwrap()
.unwrap()
.is_err()
);
assert_eq!(transport.started_count.load(Ordering::SeqCst), 4);
}
struct GatedTransport {
gate: tokio::sync::Semaphore,
started: tokio::sync::Notify,
started_count: AtomicUsize,
active: AtomicUsize,
peak: AtomicUsize,
success: TestTransport,
}
impl GatedTransport {
fn new() -> Self {
Self {
gate: tokio::sync::Semaphore::new(0),
started: tokio::sync::Notify::new(),
started_count: AtomicUsize::new(0),
active: AtomicUsize::new(0),
peak: AtomicUsize::new(0),
success: TestTransport(Mutex::default()),
}
}
async fn wait_for_requests(&self, count: usize) {
tokio::time::timeout(Duration::from_secs(1), async {
loop {
let notified = self.started.notified();
if self.started_count.load(Ordering::SeqCst) >= count {
return;
}
notified.await;
}
})
.await
.expect("refresh reached transport");
}
}
struct ActiveFetch<'a>(&'a AtomicUsize);
impl Drop for ActiveFetch<'_> {
fn drop(&mut self) {
self.0.fetch_sub(1, Ordering::SeqCst);
}
}
#[async_trait]
impl PeerTransport for GatedTransport {
async fn fetch(
&self,
account: &ProviderAccount,
connection: &PeerConnection,
) -> Result<Vec<u8>, PeerFetchFailure> {
let active = self.active.fetch_add(1, Ordering::SeqCst) + 1;
let _active = ActiveFetch(&self.active);
self.peak.fetch_max(active, Ordering::SeqCst);
self.started_count.fetch_add(1, Ordering::SeqCst);
self.started.notify_one();
self.gate.acquire().await.unwrap().forget();
if account.id().as_str() == "acct_1" {
return Err(UpstreamInfoFailure::Unavailable.into());
}
self.success.fetch(account, connection).await
}
}
#[tokio::test]
async fn discovery_uses_provider_capabilities_and_publishes_neutral_signals() {
for failed_provider in [None, Some("openai")] {
let providers = ["openai", "xai"].map(|kind| {
let mut provider = UnavailableProvider::new(kind);
provider.peer_enabled = true;
Arc::new(provider) as Arc<dyn ProviderAdmin>
});
let registry = ProviderAdminRegistry::new(providers).unwrap();
let accounts = Arc::new(TestAccounts {
accounts: Mutex::new(vec![
account("acct_first", "openai", "peer_test"),
account("acct_second", "xai", "peer_test"),
account("acct_native", "xai", "oauth"),
]),
records: Mutex::default(),
failed_provider,
});
let transport = Arc::new(TestTransport(Mutex::new(Vec::new())));
let mut bundle =
initialize_peer_upstream(accounts.clone(), &registry, transport.clone()).unwrap();
let initial_accounts = accounts.accounts.lock().unwrap().clone();
let reader = UpstreamInfoReader::default();
assert!(
reader
.signal(&initial_accounts[0], "model", SystemTime::now())
.is_none()
);
let source = bundle.scheduling_source();
reader.bind(&source).unwrap();
assert!(reader.bind(&source).is_err());
drop(source);
let WorkerContribution::Registration(worker) = bundle.take_worker_contributions().remove(0)
else {
panic!("scheduled worker")
};
let WorkerRunnable::Scheduled { ref task, .. } = worker.runnable else {
panic!("scheduled worker")
};
let result = task
.run_cycle(WorkerCycleContext::new(
worker.id.clone(),
None,
CancellationToken::new(),
))
.await;
assert_eq!(result.is_err(), failed_provider.is_some());
assert_eq!(
*transport.0.lock().unwrap(),
if failed_provider.is_some() {
vec!["acct_second"]
} else {
vec!["acct_first", "acct_second"]
}
);
for account in &initial_accounts[..2] {
if Some(account.provider().as_str()) == failed_provider {
assert!(reader.signal(account, "model", SystemTime::now()).is_none());
continue;
}
let observed = bundle
.service()
.read_quota(account.id())
.await
.unwrap()
.unwrap();
assert_eq!(
observed.upstream.unwrap().connection_revision,
UpstreamConnection::for_account(account).revision
);
assert!(
reader
.signal(account, "model", SystemTime::now())
.unwrap()
.exhausted
);
}
assert!(
reader
.signal(&initial_accounts[2], "model", SystemTime::now())
.is_none()
);
drop(worker);
drop(bundle);
assert!(
reader
.signal(&initial_accounts[0], "model", SystemTime::now())
.is_none()
);
}
}
#[tokio::test]
async fn account_edits_preserve_observation_and_refresh_budget_until_connection_changes() {
let mut provider = UnavailableProvider::new("openai");
provider.peer_enabled = true;
let registry =
ProviderAdminRegistry::new([Arc::new(provider) as Arc<dyn ProviderAdmin>]).unwrap();
let initial = account("acct_peer", "openai", "peer_test");
let accounts = Arc::new(TestAccounts {
accounts: Mutex::new(vec![initial.clone()]),
records: Mutex::default(),
failed_provider: None,
});
let transport = Arc::new(TestTransport(Mutex::new(Vec::new())));
let bundle = initialize_peer_upstream(accounts.clone(), &registry, transport.clone()).unwrap();
let service = bundle.service();
service.refresh_quota(initial.id()).await.unwrap();
let before = service.read_summary(initial.id()).await.unwrap().unwrap();
let edited = initial.clone().with_notes(Some("仅修改备注".into()));
*accounts.accounts.lock().unwrap() = vec![edited.clone()];
service.account_changed(edited.id()).await.unwrap();
assert!(
bundle
.scheduling_source()
.signal(&edited, "model", SystemTime::now())
.is_some()
);
let after = service.refresh_quota(edited.id()).await.unwrap();
assert_eq!(after.upstream, before.upstream);
assert_eq!(transport.0.lock().unwrap().len(), 1);
let reconnected = edited.with_outbound_proxy(Some(
OutboundProxy::parse("http://proxy.invalid:8080").unwrap(),
));
*accounts.accounts.lock().unwrap() = vec![reconnected.clone()];
service.account_changed(reconnected.id()).await.unwrap();
assert!(
bundle
.scheduling_source()
.signal(&reconnected, "model", SystemTime::now())
.is_none()
);
service.refresh_quota(reconnected.id()).await.unwrap();
assert_eq!(transport.0.lock().unwrap().len(), 2);
}
fn account(id: &str, provider: &str, authentication: &str) -> ProviderAccount {
ProviderAccount::new(
ProviderAccountId::new(id).unwrap(),
ProviderKind::new(provider).unwrap(),
id.to_owned(),
None,
authentication.to_owned(),
CredentialRevision::new(1).unwrap(),
None,
)
.with_account_facts(
true,
CredentialState::Ready,
QuotaState::unknown(),
None,
None,
)
}
struct TestTransport(Mutex<Vec<String>>);
#[async_trait]
impl PeerTransport for TestTransport {
async fn fetch(
&self,
account: &ProviderAccount,
connection: &PeerConnection,
) -> Result<Vec<u8>, PeerFetchFailure> {
assert_eq!(
connection.base_url,
format!("https://{}.invalid/v1", account.provider())
);
self.0
.lock()
.unwrap()
.push(account.id().as_str().to_owned());
let now = Utc::now();
Ok(serde_json::to_vec(&serde_json::json!({
"object":"cpr.peer", "schema_version":2,
"observed_at":now.to_rfc3339(),
"expires_at":(now + chrono::Duration::minutes(5)).to_rfc3339(),
"capabilities":["key_usage"],
"usage":{"status":"available","observed_at":now.to_rfc3339(),"data":{"unit":"USD","allowed":false,
"windows":[{"key":"daily","total":"1","used":"1","remaining":"0","reset_at":(now + chrono::Duration::days(1)).to_rfc3339()}]}},
"models":[]
})).unwrap())
}
}
struct TestAccounts {
accounts: Mutex<Vec<ProviderAccount>>,
records: Mutex<BTreeMap<ProviderAccountId, QuotaObservation>>,
failed_provider: Option<&'static str>,
}
fn loaded(account: &ProviderAccount) -> LoadedCredential {
LoadedCredential {
account: account.clone(),
credential: PlaintextCredential::new(Default::default()),
}
}
#[async_trait]
impl ProviderAccountStore for TestAccounts {
async fn create_account(&self, _account: NewProviderAccount) -> Result<(), StoreError> {
panic!("unused account operation")
}
async fn get_account(
&self,
account_id: &ProviderAccountId,
) -> Result<Option<ProviderAccount>, StoreError> {
Ok(self
.accounts
.lock()
.unwrap()
.iter()
.find(|account| account.id() == account_id)
.cloned())
}
async fn list_accounts(&self) -> Result<Vec<ProviderAccount>, StoreError> {
panic!("unused account operation")
}
async fn list_for_provider(
&self,
provider: &ProviderKind,
) -> Result<Vec<ProviderAccount>, StoreError> {
if Some(provider.as_str()) == self.failed_provider {
return Err(StoreError::new(StoreErrorKind::Unavailable));
}
Ok(self
.accounts
.lock()
.unwrap()
.iter()
.filter(|account| account.provider() == provider)
.cloned()
.collect())
}
async fn list_refresh_candidates(
&self,
_query: ProviderRefreshQuery,
) -> Result<Vec<LoadedCredential>, StoreError> {
panic!("unused account operation")
}
async fn load_credential(
&self,
_account: &ProviderAccountId,
_expected_revision: CredentialRevision,
) -> Result<LoadedCredential, StoreError> {
panic!("unused account operation")
}
async fn load_current_credential(
&self,
id: &ProviderAccountId,
) -> Result<LoadedCredential, StoreError> {
self.accounts
.lock()
.unwrap()
.iter()
.find(|account| account.id() == id)
.map(loaded)
.ok_or_else(|| StoreError::new(StoreErrorKind::InvalidData))
}
async fn compare_and_swap_credential(
&self,
_update: CredentialCasUpdate,
) -> Result<CredentialCasOutcome, StoreError> {
panic!("unused account operation")
}
async fn get_quotas(
&self,
accounts: &[ProviderAccountId],
) -> Result<Vec<QuotaObservation>, StoreError> {
let stored = self.records.lock().unwrap();
Ok(accounts
.iter()
.filter_map(|id| stored.get(id).cloned())
.collect())
}
async fn compare_and_swap_quota(
&self,
observation: QuotaObservation,
) -> Result<QuotaWriteOutcome, StoreError> {
self.records
.lock()
.unwrap()
.insert(observation.account_id.clone(), observation);
Ok(QuotaWriteOutcome::Updated)
}
async fn touch_quota_observation(
&self,
_touch: QuotaObservationTouch,
) -> Result<QuotaWriteOutcome, StoreError> {
panic!("unused account operation")
}
async fn apply_quota_access(
&self,
_change: QuotaAccessChange,
) -> Result<QuotaWriteOutcome, StoreError> {
panic!("unused account operation")
}
async fn apply_state_change(&self, _change: AccountStateChange) -> Result<(), StoreError> {
panic!("unused account operation")
}
async fn update_account(&self, _update: ProviderAccountUpdate) -> Result<(), StoreError> {
panic!("unused account operation")
}
async fn set_enabled(
&self,
_account: &ProviderAccountId,
_enabled: bool,
) -> Result<(), StoreError> {
panic!("unused account operation")
}
async fn delete_account(&self, _account: &ProviderAccountId) -> Result<(), StoreError> {
panic!("unused account operation")
}
}
@@ -0,0 +1,260 @@
//! 互联发现结果的持久恢复、连接失效、手动重查与临时失败退避
use super::*;
use gateway_admin::PeerUpstreamBundle;
use gateway_core::task::WorkerRegistration;
#[tokio::test(start_paused = true)]
async fn unsupported_detection_survives_restart_until_manual_rediscovery() {
let (mut bundle, accounts, transport, id) = fixture(UpstreamInfoFailure::Unsupported);
// 入库后的首查与手动刷新使用同一入口,不支持协议也是可保存的正常发现结果
let initial = bundle.service().refresh_quota(&id).await.unwrap();
assert!(!initial.upstream.unwrap().discovered);
let first_worker = worker(&mut bundle);
tokio::time::advance(Duration::from_secs(86400)).await;
cycle(&first_worker).await;
assert_eq!(transport.requests.load(Ordering::SeqCst), 1);
drop(first_worker);
drop(bundle);
let mut restarted = initialize(accounts, transport.clone());
let info = restarted
.service()
.read_quota(&id)
.await
.unwrap()
.unwrap()
.upstream
.unwrap();
assert!(!info.discovered);
assert_eq!(info.last_error.as_deref(), Some("unsupported"));
let resumed_worker = worker(&mut restarted);
cycle(&resumed_worker).await;
tokio::time::advance(Duration::from_secs(86400)).await;
cycle(&resumed_worker).await;
assert_eq!(transport.requests.load(Ordering::SeqCst), 1);
let summary = restarted
.service()
.read_summary(&id)
.await
.unwrap()
.unwrap();
assert_eq!(
summary.upstream.unwrap().last_error.as_deref(),
Some("unsupported")
);
*transport.failure.lock().unwrap() = None;
let discovered = restarted.service().refresh_quota(&id).await.unwrap();
assert!(discovered.upstream.unwrap().discovered);
assert_eq!(transport.requests.load(Ordering::SeqCst), 2);
tokio::time::advance(Duration::from_secs(360)).await;
cycle(&resumed_worker).await;
assert_eq!(transport.requests.load(Ordering::SeqCst), 3);
}
#[tokio::test(start_paused = true)]
async fn unsupported_detection_keeps_manual_refresh_throttling() {
for retry_after in [None, Some(Duration::from_secs(90))] {
let (bundle, _, transport, id) = fixture(UpstreamInfoFailure::Unsupported);
*transport.failure.lock().unwrap() = Some(PeerFetchFailure {
kind: UpstreamInfoFailure::Unsupported,
retry_after,
});
bundle.service().refresh_quota(&id).await.unwrap();
let delay = retry_after.unwrap_or(Duration::from_secs(15));
tokio::time::advance(delay - Duration::from_secs(1)).await;
bundle.service().refresh_quota(&id).await.unwrap();
assert_eq!(transport.requests.load(Ordering::SeqCst), 1);
tokio::time::advance(Duration::from_secs(1)).await;
*transport.failure.lock().unwrap() = None;
assert!(
bundle
.service()
.refresh_quota(&id)
.await
.unwrap()
.upstream
.unwrap()
.discovered
);
assert_eq!(transport.requests.load(Ordering::SeqCst), 2);
}
}
#[tokio::test(start_paused = true)]
async fn ordinary_edits_keep_detection_but_proxy_and_credential_changes_reprobe() {
let (mut bundle, accounts, transport, id) = fixture(UpstreamInfoFailure::Unsupported);
let worker = worker(&mut bundle);
// 无识别记录的存量账号由周期任务补查
cycle(&worker).await;
let initial = accounts.accounts.lock().unwrap()[0].clone();
let edited = initial.clone().with_notes(Some("修改备注".into()));
*accounts.accounts.lock().unwrap() = vec![edited.clone()];
bundle.service().account_changed(&id).await.unwrap();
tokio::time::advance(Duration::from_secs(86400)).await;
cycle(&worker).await;
assert_eq!(transport.requests.load(Ordering::SeqCst), 1);
let reconnected = edited.with_outbound_proxy(Some(
OutboundProxy::parse("http://proxy.invalid:8080").unwrap(),
));
*accounts.accounts.lock().unwrap() = vec![reconnected.clone()];
bundle.service().account_changed(&id).await.unwrap();
cycle(&worker).await;
assert_eq!(transport.requests.load(Ordering::SeqCst), 2);
let replaced = ProviderAccount::new(
id.clone(),
initial.provider().clone(),
"new credential".into(),
None,
"peer_test".into(),
CredentialRevision::new(2).unwrap(),
None,
)
.with_outbound_proxy(reconnected.outbound_proxy().cloned())
.with_account_facts(
true,
CredentialState::Ready,
QuotaState::unknown(),
None,
None,
);
*accounts.accounts.lock().unwrap() = vec![replaced];
bundle.service().account_changed(&id).await.unwrap();
cycle(&worker).await;
assert_eq!(transport.requests.load(Ordering::SeqCst), 3);
}
#[tokio::test(start_paused = true)]
async fn uncertain_discovery_failures_retry_without_becoming_unsupported() {
for failure in [
UpstreamInfoFailure::Unauthorized,
UpstreamInfoFailure::Unavailable,
UpstreamInfoFailure::RateLimited,
UpstreamInfoFailure::InvalidResponse,
] {
let (mut bundle, _, transport, id) = fixture(failure);
let worker = worker(&mut bundle);
cycle(&worker).await;
let info = bundle
.service()
.read_quota(&id)
.await
.unwrap()
.unwrap()
.upstream
.unwrap();
assert_eq!(info.last_error.as_deref(), Some(failure.as_str()));
assert!(!info.discovered);
cycle(&worker).await;
assert_eq!(transport.requests.load(Ordering::SeqCst), 1);
*transport.failure.lock().unwrap() = None;
tokio::time::advance(Duration::from_secs(
if failure == UpstreamInfoFailure::Unauthorized {
1800
} else {
30
},
))
.await;
cycle(&worker).await;
assert_eq!(transport.requests.load(Ordering::SeqCst), 2);
assert!(
bundle
.service()
.read_quota(&id)
.await
.unwrap()
.unwrap()
.upstream
.unwrap()
.discovered
);
}
}
fn fixture(
failure: UpstreamInfoFailure,
) -> (
PeerUpstreamBundle,
Arc<TestAccounts>,
Arc<DetectionTransport>,
ProviderAccountId,
) {
let record = account("acct_peer", "openai", "peer_test");
let id = record.id().clone();
let accounts = Arc::new(TestAccounts {
accounts: Mutex::new(vec![record]),
records: Mutex::default(),
failed_provider: None,
});
let transport = Arc::new(DetectionTransport {
requests: AtomicUsize::new(0),
failure: Mutex::new(Some(failure.into())),
success: TestTransport(Mutex::default()),
});
(
initialize(accounts.clone(), transport.clone()),
accounts,
transport,
id,
)
}
fn initialize(
accounts: Arc<TestAccounts>,
transport: Arc<DetectionTransport>,
) -> PeerUpstreamBundle {
let mut provider = UnavailableProvider::new("openai");
provider.peer_enabled = true;
let registry =
ProviderAdminRegistry::new([Arc::new(provider) as Arc<dyn ProviderAdmin>]).unwrap();
initialize_peer_upstream(accounts, &registry, transport).unwrap()
}
fn worker(bundle: &mut PeerUpstreamBundle) -> WorkerRegistration {
let WorkerContribution::Registration(worker) = bundle.take_worker_contributions().remove(0)
else {
panic!("scheduled worker")
};
worker
}
async fn cycle(worker: &WorkerRegistration) {
let WorkerRunnable::Scheduled { task, .. } = &worker.runnable else {
panic!("scheduled worker")
};
task.run_cycle(WorkerCycleContext::new(
worker.id.clone(),
None,
CancellationToken::new(),
))
.await
.unwrap();
}
struct DetectionTransport {
requests: AtomicUsize,
failure: Mutex<Option<PeerFetchFailure>>,
success: TestTransport,
}
#[async_trait]
impl PeerTransport for DetectionTransport {
async fn fetch(
&self,
account: &ProviderAccount,
connection: &PeerConnection,
) -> Result<Vec<u8>, PeerFetchFailure> {
self.requests.fetch_add(1, Ordering::SeqCst);
if let Some(failure) = self.failure.lock().unwrap().as_ref() {
return Err(PeerFetchFailure {
kind: failure.kind,
retry_after: failure.retry_after,
});
}
self.success.fetch(account, connection).await
}
}
@@ -203,6 +203,7 @@ async fn linked_accounts_share_plan_resolution_and_only_read_cached_quota() {
] {
let provider = FakeProviderAdmin::new("openai", events());
provider.set_quota(ProviderQuota {
upstream: None,
credits: None,
plan_type: cached.map(str::to_owned),
observed_at: None,
@@ -215,6 +216,7 @@ async fn linked_accounts_share_plan_resolution_and_only_read_cached_quota() {
.provider(provider.clone())
.proxies(Arc::new(TestProxies {
accounts: Some(vec![ProxyAccountRef {
upstream: None,
id: "acct_plan".to_owned(),
name: "套餐测试".to_owned(),
email: None,
@@ -334,6 +334,7 @@ async fn settings_should_reject_zero_refresh_margin_before_store_call() {
max_waiting_per_key: 0,
max_waiting_per_account: 0,
concurrency_wait_timeout_seconds: 30,
max_retry_after_seconds: 240,
openai_guardian_reserved_concurrency: 0,
openai_account_affinity: gateway_core::account::AccountAffinity::Relaxed,
max_account_rotations: 3,
@@ -18,10 +18,11 @@ fn parse_provider(value: &str) -> Result<ProviderKind, WireValidationError> {
ProviderKind::new(value.trim().to_owned()).map_err(|_| WireValidationError::new("provider"))
}
/// 导入统一设置,复用编辑账号的备注、调度和分组约束
/// 导入设置复用编辑账号的名称、备注、调度和分组约束
#[derive(Debug, Clone, Deserialize, Serialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct AccountImportSettingsRequest {
pub name: Option<String>,
pub notes: Option<String>,
pub enabled: bool,
#[serde(deserialize_with = "deserialize_required_nullable")]
@@ -33,6 +34,9 @@ pub struct AccountImportSettingsRequest {
impl AccountImportSettingsRequest {
fn validate(&self) -> Result<(), WireValidationError> {
if let Some(name) = &self.name {
validate_account_name(name.trim())?;
}
validate_account_notes(self.notes.as_deref())?;
parse_concurrency_limit(self.concurrency_limit)?;
parse_account_weight(self.weight)?;
@@ -44,6 +48,10 @@ impl AccountImportSettingsRequest {
self,
) -> Result<gateway_admin::model::accounts::AccountImportSettings, WireValidationError> {
Ok(gateway_admin::model::accounts::AccountImportSettings {
name: self
.name
.map(|name| name.trim().to_owned())
.filter(|name| !name.is_empty()),
notes: self.notes,
enabled: self.enabled,
concurrency_limit: parse_concurrency_limit(self.concurrency_limit)?,
@@ -116,7 +124,7 @@ pub struct StartAccountAuthorizationRequest {
impl StartAccountAuthorizationRequest {
pub fn validate(&self) -> Result<(), WireValidationError> {
parse_provider(&self.provider)?;
require_text(&self.name, MAX_NAME_BYTES, "name")?;
validate_account_name(&self.name)?;
if let Some(account_id) = self.account_id.as_deref() {
require_account_id(account_id, "accountId")?;
}
@@ -138,7 +146,7 @@ impl StartAccountAuthorizationRequest {
provider,
StartAuthorization {
context,
name: self.name,
name: self.name.trim().to_owned(),
reauthorization,
outbound_proxy: super::wire::proxy_selection(
self.outbound_proxy_id,
@@ -213,6 +221,7 @@ pub struct UpdateAccountRequest {
pub outbound_proxy_id: Option<String>,
pub outbound_proxy_url: Option<super::wire::AccountProxyUpdate>,
pub account_id: String,
pub name: Option<String>,
pub notes: Option<String>,
pub enabled: bool,
#[serde(deserialize_with = "deserialize_required_nullable")]
@@ -225,6 +234,9 @@ pub struct UpdateAccountRequest {
impl UpdateAccountRequest {
pub fn validate(&self) -> Result<(), WireValidationError> {
require_account_id(&self.account_id, "accountId")?;
if let Some(name) = &self.name {
validate_account_name(name)?;
}
if let Some(connection) = &self.connection {
connection.validate()?;
}
@@ -248,6 +260,7 @@ impl UpdateAccountRequest {
self.outbound_proxy_url,
)?,
account_id: self.account_id,
name: self.name.map(|name| name.trim().to_owned()),
notes: self.notes,
enabled: self.enabled,
concurrency_limit: parse_concurrency_limit(self.concurrency_limit)?,
@@ -513,6 +526,13 @@ fn require_wire_id(value: &str, field: &'static str) -> Result<(), WireValidatio
Ok(())
}
fn validate_account_name(value: &str) -> Result<(), WireValidationError> {
if value.len() > MAX_NAME_BYTES || value.chars().any(char::is_control) {
return Err(WireValidationError::new("name"));
}
Ok(())
}
fn require_text(
value: &str,
max_bytes: usize,
@@ -44,7 +44,9 @@ use serde::{Deserialize, Deserializer, Serialize};
use serde_json::{Map, Value};
use uuid::{Uuid, Version};
use super::presenter::{format_compact_number, format_decimal_currency, format_number};
use super::presenter::{
format_compact_number, format_decimal_currency, format_number, format_percentage,
};
use super::{
AdminAuth, AdminEnvelope, AdminError, AdminJson, AdminQuery, AdminResponse, PageMeta,
WireValidationError,
@@ -66,11 +68,14 @@ mod credentials;
mod handlers;
mod import_tasks;
mod presenter;
mod upstream_info;
mod wire;
pub use credentials::*;
pub use handlers::{profile_avatar_response, router};
pub(super) use presenter::quota_window_view;
pub(crate) use upstream_info::view as upstream_info_view;
pub use upstream_info::{AccountQuotaAmountView, AccountUpstreamInfoView};
pub use wire::*;
use credentials::{
@@ -92,11 +92,31 @@ pub(super) fn account_view(
let usage_period = quota.usage_window().map(|(_, period)| period);
let mut usage = account_usage_view(usage, usage_period, now, time);
if account.authentication_kind == "api_key" {
usage.window_kind = AccountUsageWindowKind::General;
usage.window_kind = if quota
.upstream
.as_ref()
.is_some_and(|upstream| upstream.discovered)
{
match quota
.upstream_usage_window()
.and_then(|window| window.window_seconds)
{
Some(86_400) => AccountUsageWindowKind::Daily,
Some(604_800) => AccountUsageWindowKind::Weekly,
_ => AccountUsageWindowKind::Quota,
}
} else {
AccountUsageWindowKind::General
};
}
let upstream = quota
.upstream
.clone()
.map(|value| super::upstream_info::view(value, now, time));
let (mut quota, refresh_token_expires_at) = account_quota_view(quota, cooldown, now, time);
quota.protection_rules = protection_rules;
AccountView {
upstream,
capabilities: capabilities.into(),
id: account.id.clone(),
name: account.name,
@@ -277,6 +297,7 @@ pub(crate) fn quota_window_view(
time: crate::time::TimePresenter,
) -> AccountQuotaWindowView {
let ProviderQuotaWindow {
upstream,
key,
group,
label,
@@ -295,6 +316,16 @@ pub(crate) fn quota_window_view(
.as_ref()
.map_or_else(|| label.clone(), |name| format!("{name} · {label}"));
AccountQuotaWindowView {
source_account_id: upstream.as_ref().and_then(|value| value.account_id.clone()),
amount: upstream
.as_ref()
.and_then(|value| value.amount.as_ref())
.map(super::upstream_info::amount_view),
observed_at_display: upstream
.as_ref()
.and_then(|value| value.observed_at)
.map(|value| time.datetime(&value)),
stale: upstream.as_ref().is_some_and(|value| value.stale),
label_display,
window_label_display: label,
key,
@@ -304,8 +335,7 @@ pub(crate) fn quota_window_view(
role: role.map(|value| value.as_str().to_owned()),
window_seconds,
used_percent,
used_percent_display: used_percent
.map_or_else(|| "—".to_owned(), |value| format!("{value:.1}%")),
used_percent_display: used_percent.map_or_else(|| "—".to_owned(), format_percentage),
limit_reached,
local_usage: local_usage
.as_ref()
@@ -0,0 +1,101 @@
//! 互联快照与金额的管理端展示投影
use gateway_admin::model::upstream_info::{ProviderUpstreamInfo, UpstreamQuotaAmount};
use super::*;
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct AccountUpstreamInfoView {
pub discovered: bool,
pub observed_at: Option<String>,
pub observed_at_display: String,
pub observed_at_relative: Option<crate::time::RelativeTimeView>,
pub stale: bool,
pub connection_revision: String,
pub last_error: Option<String>,
pub pool: Option<UpstreamPoolView>,
}
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct UpstreamPoolView {
pub account_count: u32,
pub states: UpstreamPoolStatesView,
pub plans: Vec<UpstreamPlanView>,
}
#[derive(Debug, Clone, Serialize)]
pub struct UpstreamPoolStatesView {
pub ready: u32,
pub cooldown: u32,
pub exhausted: u32,
pub disabled: u32,
pub error: u32,
pub unknown: u32,
}
#[derive(Debug, Clone, Serialize)]
pub struct UpstreamPlanView {
pub name: String,
pub count: u32,
}
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct AccountQuotaAmountView {
pub used_display: String,
pub remaining_display: String,
pub total_display: String,
}
pub(super) fn amount_view(amount: &UpstreamQuotaAmount) -> AccountQuotaAmountView {
let display = |value: Option<&str>| {
value.map_or_else(
|| "∞".to_owned(),
|value| crate::admin::presenter::format_decimal_currency(value, &amount.unit),
)
};
AccountQuotaAmountView {
used_display: display(Some(&amount.used)),
remaining_display: display(amount.remaining.as_deref()),
total_display: display(amount.total.as_deref()),
}
}
pub(crate) fn view(
info: ProviderUpstreamInfo,
now: DateTime<Utc>,
time: crate::time::TimePresenter,
) -> AccountUpstreamInfoView {
AccountUpstreamInfoView {
discovered: info.discovered,
observed_at: info.observed_at.map(|value| time.rfc3339(&value)),
observed_at_display: info
.observed_at
.map_or_else(|| "—".to_owned(), |value| time.datetime(&value)),
observed_at_relative: info.observed_at.map(|value| time.relative_view(value, now)),
stale: info.stale,
connection_revision: info.connection_revision,
last_error: info.last_error,
pool: info.pool.map(|pool| UpstreamPoolView {
account_count: pool.account_count,
states: UpstreamPoolStatesView {
ready: pool.states.ready,
cooldown: pool.states.cooldown,
exhausted: pool.states.exhausted,
disabled: pool.states.disabled,
error: pool.states.error,
unknown: pool.states.unknown,
},
plans: pool
.plans
.into_iter()
.map(|plan| UpstreamPlanView {
name: plan.name,
count: plan.count,
})
.collect(),
}),
}
}
@@ -266,6 +266,7 @@ pub struct AccountSummaryView {
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct AccountView {
pub upstream: Option<super::AccountUpstreamInfoView>,
pub capabilities: AccountCapabilitiesView,
pub outbound_proxy_id: Option<String>,
pub outbound_proxy_endpoint: Option<String>,
@@ -412,6 +413,11 @@ pub struct AccountQuotaCreditsView {
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct AccountQuotaWindowView {
#[serde(skip_serializing_if = "Option::is_none")]
pub source_account_id: Option<String>,
pub amount: Option<super::AccountQuotaAmountView>,
pub observed_at_display: Option<String>,
pub stale: bool,
pub key: String,
pub group: String,
pub limit_id: Option<String>,
@@ -433,6 +439,7 @@ pub struct AccountQuotaWindowView {
#[serde(rename_all = "snake_case")]
pub enum AccountUsageWindowKind {
General,
Daily,
Weekly,
Monthly,
Quota,
@@ -152,6 +152,8 @@ impl ClientKeySort {
#[derive(Clone, PartialEq, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct CreateClientKeyRequest {
#[serde(default)]
peer_sharing: gateway_core::policy::ClientPeerSharing,
#[serde(default)]
provider_request_profile_overrides: ProviderRequestProfileOverrides,
openai_client_profile_override: Option<serde_json::Map<String, serde_json::Value>>,
@@ -190,6 +192,7 @@ impl CreateClientKeyRequest {
self.xai_client_profile_override,
)?;
Ok(CreateClientKey {
peer_sharing: self.peer_sharing,
request_profile_overrides,
custom_key: self
.custom_key
@@ -217,6 +220,7 @@ impl CreateClientKeyRequest {
#[derive(Debug, Clone, PartialEq, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct UpdateClientKeyRequest {
peer_sharing: Option<gateway_core::policy::ClientPeerSharing>,
#[serde(default)]
provider_request_profile_overrides: ProviderRequestProfileOverrideUpdates,
#[serde(default, deserialize_with = "deserialize_profile_override")]
@@ -248,6 +252,7 @@ impl UpdateClientKeyRequest {
self.xai_client_profile_override,
)?;
Ok(UpdateClientKey {
peer_sharing: self.peer_sharing,
request_profile_override_updates,
id: client_key_id(self.id, "clientKeyMutationNotFound")?,
name: self.name,
@@ -334,6 +339,7 @@ impl ClientKeyMutationRequest {
#[derive(Debug, Clone, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct ClientKeyView {
peer_sharing: gateway_core::policy::ClientPeerSharing,
provider_request_profile_overrides: ProviderRequestProfileOverrides,
/// 固定兼容字段;值始终从 provider_request_profile_overrides 派生
openai_client_profile_override: Option<serde_json::Map<String, serde_json::Value>>,
@@ -384,6 +390,7 @@ impl From<(ClientKeyRecord, crate::time::TimePresenter)> for ClientKeyView {
.cloned(),
xai_client_profile_override: provider_request_profile_overrides.get("xai").cloned(),
provider_request_profile_overrides,
peer_sharing: record.peer_sharing,
id: record.id.to_string(),
name: record.name,
label: record.label,
@@ -919,6 +919,9 @@ pub(crate) fn dashboard_view(
),
};
account_usage_views.push(DashboardAccountUsageView {
upstream: credential.upstream.map(|info| {
crate::admin::accounts::upstream_info_view(info, chrono::Utc::now(), time)
}),
id: credential.account_id.clone(),
provider: credential.provider_kind.clone(),
authentication_kind: credential.authentication_kind.clone(),
@@ -1339,6 +1342,7 @@ pub(crate) fn diagnostics_view(
.into_iter()
.map(|item| DiagnosticItemView {
key: item.key,
authentication_kind: item.account_authentication_kind,
name: if item.name == "__none__" {
"未知".to_owned()
} else {
@@ -2,6 +2,8 @@
use super::*;
pub use crate::time::parse_datetime;
/// 观测列表默认页大小
pub const DEFAULT_PAGE_SIZE: u16 = 50;
/// 观测列表允许的最大页大小
@@ -256,16 +258,6 @@ impl DiagnosticDimension {
}
}
/// 解析 RFC3339 时间;错误不回显原始值
pub fn parse_datetime(value: Option<&str>) -> Result<Option<DateTime<Utc>>, WireValidationError> {
let Some(value) = trimmed(value) else {
return Ok(None);
};
DateTime::parse_from_rfc3339(value)
.map(|value| Some(value.with_timezone(&Utc)))
.map_err(|_| WireValidationError::new("timeRange"))
}
/// 解析 HTTP 状态码
pub fn parse_status(value: Option<i64>) -> Result<Option<u16>, WireValidationError> {
value
@@ -402,6 +402,7 @@ pub enum DashboardAccountMetric {
#[derive(Debug, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct DashboardAccountUsageView {
pub upstream: Option<crate::admin::accounts::AccountUpstreamInfoView>,
pub id: String,
pub provider: String,
pub authentication_kind: String,
@@ -845,6 +846,7 @@ pub struct UsageCacheView {
pub struct DiagnosticItemView {
pub key: String,
pub name: String,
pub authentication_kind: Option<String>,
pub account_plan_type: Option<String>,
pub account_plan_type_display: Option<String>,
pub request_count: u64,
@@ -41,11 +41,11 @@ pub fn format_compact_number(value: u64) -> String {
}
/// 货币展示格式化
/// USD 常规金额保留两位,小于 1 美元时最多保留四位
/// USD 常规金额最多保留两位,小于 1 美元时最多保留四位,小数末尾不补零
#[must_use]
pub fn format_decimal_currency(amount: &str, currency: &str) -> String {
if currency != "USD" {
return format!("{currency} {amount}");
return format!("{currency} {}", trim_decimal_zeros(amount));
}
const DECIMAL_SCALE: u128 = 10_000_000_000;
@@ -60,13 +60,21 @@ pub fn format_decimal_currency(amount: &str, currency: &str) -> String {
let display_scale = 10_u128.pow(precision);
let whole = rounded / display_scale;
let fraction = rounded % display_scale;
let mut display = format!("{whole}.{fraction:0width$}", width = precision as usize);
while display.ends_with('0')
&& display
.split_once('.')
.is_some_and(|(_, fraction)| fraction.len() > 2)
{
display.pop();
let display = format!("{whole}.{fraction:0width$}", width = precision as usize);
format!("${}", trim_decimal_zeros(&display))
}
/// 百分比最多保留一位小数,整数百分比不补零
#[must_use]
pub(super) fn format_percentage(value: f64) -> String {
let display = format!("{value:.1}");
format!("{}%", trim_decimal_zeros(&display))
}
fn trim_decimal_zeros(value: &str) -> &str {
if value.contains('.') {
value.trim_end_matches('0').trim_end_matches('.')
} else {
value
}
format!("${display}")
}
@@ -135,6 +135,7 @@ impl From<ProxyTestResult> for ProxyTestView {
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
struct ProxyAccountView {
upstream: Option<super::accounts::AccountUpstreamInfoView>,
id: String,
name: String,
email: Option<String>,
@@ -322,6 +323,13 @@ where
.items
.into_iter()
.map(|account| ProxyAccountView {
upstream: account.upstream.map(|info| {
super::accounts::upstream_info_view(
info,
chrono::Utc::now(),
crate::time::TimePresenter::new(state.admin_services().timezone()),
)
}),
id: account.id,
name: account.name,
email: account.email,
@@ -45,6 +45,7 @@ pub struct RuntimeSettingsFields {
pub max_waiting_per_key: u32,
pub max_waiting_per_account: u32,
pub concurrency_wait_timeout_seconds: u32,
pub max_retry_after_seconds: u32,
pub openai_guardian_reserved_concurrency: u32,
pub openai_account_affinity: gateway_core::account::AccountAffinity,
pub max_account_rotations: u32,
@@ -751,6 +752,7 @@ impl RuntimeSettingsFields {
max_waiting_per_key: self.max_waiting_per_key,
max_waiting_per_account: self.max_waiting_per_account,
concurrency_wait_timeout_seconds: self.concurrency_wait_timeout_seconds,
max_retry_after_seconds: self.max_retry_after_seconds,
openai_guardian_reserved_concurrency: self.openai_guardian_reserved_concurrency,
openai_account_affinity: self.openai_account_affinity,
max_account_rotations: self.max_account_rotations,
@@ -790,6 +792,7 @@ impl From<gateway_admin::model::settings::RuntimeSettingsValues> for RuntimeSett
max_waiting_per_key: settings.max_waiting_per_key,
max_waiting_per_account: settings.max_waiting_per_account,
concurrency_wait_timeout_seconds: settings.concurrency_wait_timeout_seconds,
max_retry_after_seconds: settings.max_retry_after_seconds,
openai_guardian_reserved_concurrency: settings.openai_guardian_reserved_concurrency,
openai_account_affinity: settings.openai_account_affinity,
max_account_rotations: settings.max_account_rotations,
+1 -19
View File
@@ -9,25 +9,7 @@ use axum::{
};
use serde::{Deserialize, Serialize};
/// 管理端 wire 字段校验错误;只携带稳定字段名,不回显输入值
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct WireValidationError {
field: &'static str,
}
impl WireValidationError {
/// 构造字段校验错误
#[must_use]
pub const fn new(field: &'static str) -> Self {
Self { field }
}
/// 返回未通过校验的字段名
#[must_use]
pub const fn field(self) -> &'static str {
self.field
}
}
pub use crate::WireValidationError;
/// 成功响应的稳定业务码
pub const ADMIN_OK_CODE: u32 = 200;
@@ -0,0 +1,228 @@
//! 数据面与公共 HTTP 中间件共用的客户端错误合同和重试投影
use axum::{
Json,
http::{HeaderValue, StatusCode},
response::{IntoResponse, Response},
};
use gateway_core::error::{GatewayError, GatewayErrorKind};
use serde::Serialize;
use serde_json::{Value, json};
/// OpenAI 风格的协议错误 JSON
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
pub struct ProtocolErrorBody {
/// 错误对象
pub error: ProtocolError,
}
/// 不包含请求正文或 prompt 的错误对象
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
pub struct ProtocolError {
/// OpenAI 错误类别
#[serde(rename = "type")]
pub kind: &'static str,
/// 稳定机器码
pub code: &'static str,
/// 安全的可读信息
pub message: String,
/// 出错字段;不包含字段值
#[serde(skip_serializing_if = "Option::is_none")]
pub param: Option<String>,
}
impl ProtocolErrorBody {
/// 序列化为 JSON value
#[must_use]
pub fn into_value(self) -> Value {
serde_json::json!({ "error": self.error })
}
}
/// OpenAI 风格错误响应
pub fn openai_error_response(
status: StatusCode,
message: &str,
error_type: &str,
code: &str,
) -> (StatusCode, Json<Value>) {
let body = json!({
"error": {"message": message, "type": error_type, "code": code}
});
match capacity_error_for_client(&body) {
Some(body) => (StatusCode::SERVICE_UNAVAILABLE, Json(body)),
None => (status, Json(body)),
}
}
/// 无重试提示的容量拒绝及旧版 Codex 限流错误需要可重试投影
/// 仅在交付边界转换,内部仍保留真实上游事实
pub(crate) fn client_error_code(code: &str) -> &str {
match code {
"server_is_overloaded" | "slow_down" => "server_error",
_ => code,
}
}
/// 为 HTTP 错误或流内失败保留原有字段,仅投影客户端需要的重试信号
pub(crate) fn capacity_error_for_client(data: &Value) -> Option<Value> {
const CODE_PATHS: [&str; 2] = ["/error/code", "/response/error/code"];
let capacity_code = |value: &Value| {
value
.as_str()
.is_some_and(|code| client_error_code(code) != code)
};
if !CODE_PATHS
.iter()
.any(|path| data.pointer(path).is_some_and(capacity_code))
{
return None;
}
let mut projected = data.clone();
for path in CODE_PATHS {
if let Some(code) = projected.pointer_mut(path)
&& capacity_code(code)
{
*code = Value::String("server_error".to_owned());
}
}
// WS 包装错误按 HTTP 状态分类;保留 400/429 会使客户端终止重试
for field in ["status", "status_code"] {
if let Some(status) = projected.get_mut(field)
&& status.is_number()
{
*status = json!(503);
}
}
Some(projected)
}
/// Gateway 错误的 OpenAI HTTP 表达
pub fn gateway_error_response(error: &GatewayError) -> Response {
let (status, default_type, default_code) = gateway_error_contract(error.kind());
let mut response = openai_error_response(
status,
error.client_message(),
error.client_error_type().unwrap_or(default_type),
error.client_error_code().unwrap_or(default_code),
)
.into_response();
if let Some(delay) = error.retry_after()
&& let Ok(value) = HeaderValue::from_str(
&delay
.as_secs()
.saturating_add(u64::from(delay.subsec_nanos() > 0))
.max(1)
.to_string(),
)
{
response
.headers_mut()
.insert(axum::http::header::RETRY_AFTER, value);
}
response
}
/// Gateway 错误稳定映射,供 HTTP、SSE 和 WebSocket 共用
#[must_use]
pub const fn gateway_error_contract(
kind: GatewayErrorKind,
) -> (StatusCode, &'static str, &'static str) {
match kind {
GatewayErrorKind::InvalidRequest => (
StatusCode::BAD_REQUEST,
"invalid_request_error",
"invalid_request",
),
GatewayErrorKind::Unsupported => (
StatusCode::BAD_REQUEST,
"invalid_request_error",
"unsupported_capability",
),
GatewayErrorKind::Unauthorized => (
StatusCode::UNAUTHORIZED,
"invalid_request_error",
"invalid_api_key",
),
GatewayErrorKind::PolicyDenied => (
StatusCode::FORBIDDEN,
"invalid_request_error",
"policy_denied",
),
GatewayErrorKind::ModelNotFound => (
StatusCode::NOT_FOUND,
"invalid_request_error",
"model_not_found",
),
GatewayErrorKind::NoAvailableProvider => (
StatusCode::SERVICE_UNAVAILABLE,
"server_error",
"no_available_provider",
),
GatewayErrorKind::AccountCapacityUnavailable => (
StatusCode::SERVICE_UNAVAILABLE,
"server_error",
"account_capacity_unavailable",
),
GatewayErrorKind::ProviderInfrastructureUnavailable => (
StatusCode::SERVICE_UNAVAILABLE,
"server_error",
"provider_infrastructure_unavailable",
),
GatewayErrorKind::MiddlewareCapacityUnavailable => (
StatusCode::SERVICE_UNAVAILABLE,
"server_error",
"middleware_capacity_unavailable",
),
GatewayErrorKind::ConcurrencyQueueFull => (
StatusCode::TOO_MANY_REQUESTS,
"rate_limit_error",
"concurrency_queue_full",
),
GatewayErrorKind::ConcurrencyQueueTimeout => (
StatusCode::TOO_MANY_REQUESTS,
"rate_limit_error",
"concurrency_queue_timeout",
),
GatewayErrorKind::RateLimited => (
StatusCode::TOO_MANY_REQUESTS,
"rate_limit_error",
"rate_limit_exceeded",
),
GatewayErrorKind::BudgetExhausted => (
StatusCode::TOO_MANY_REQUESTS,
"usage_limit_reached",
"usage_limit_reached",
),
GatewayErrorKind::UpstreamUnavailable => (
StatusCode::BAD_GATEWAY,
"server_error",
"upstream_unavailable",
),
GatewayErrorKind::MessageTooBig => (
StatusCode::PAYLOAD_TOO_LARGE,
"invalid_request_error",
"message_too_big",
),
GatewayErrorKind::Timeout => (
StatusCode::GATEWAY_TIMEOUT,
"server_error",
"request_timeout",
),
GatewayErrorKind::Cancelled => (
StatusCode::REQUEST_TIMEOUT,
"server_error",
"request_cancelled",
),
GatewayErrorKind::Internal => (
StatusCode::INTERNAL_SERVER_ERROR,
"server_error",
"internal_error",
),
_ => (
StatusCode::INTERNAL_SERVER_ERROR,
"server_error",
"internal_error",
),
}
}
+21
View File
@@ -29,6 +29,7 @@ use crate::openai::service::OpenAiService;
pub mod admin;
pub mod auth;
mod client_error;
mod health;
mod key_usage;
mod middleware;
@@ -40,6 +41,26 @@ mod time;
pub use time::{RequestBucketView, TimePresenter};
/// HTTP wire 字段校验错误;只携带稳定字段名,不回显输入值
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct WireValidationError {
field: &'static str,
}
impl WireValidationError {
/// 构造字段校验错误
#[must_use]
pub const fn new(field: &'static str) -> Self {
Self { field }
}
/// 返回未通过校验的字段名
#[must_use]
pub const fn field(self) -> &'static str {
self.field
}
}
/// API-owned HTTP 与静态资源配置
#[derive(Debug, Clone, Deserialize, PartialEq, Eq)]
pub struct ApiConfig {
@@ -262,7 +262,7 @@ async fn handle(
Err(MiddlewareError::Gateway(error))
if error.kind() == GatewayErrorKind::MiddlewareCapacityUnavailable =>
{
crate::openai::error::gateway_error_response(&error)
crate::client_error::gateway_error_response(&error)
}
Err(error) => {
let status = if error.is_rejected() {
@@ -15,7 +15,8 @@ use gateway_core::{
use gateway_protocol::openai::sse::{DONE_SSE_FRAME, SseEvent, SseEventDecoder, sse_frame_is_done};
use serde_json::Value;
use crate::openai::{middleware::error_response, responses::ProtocolErrorBody};
use crate::client_error::ProtocolErrorBody;
use crate::openai::middleware::error_response;
use super::response::{ChatIdentity, ChatStreamEncoder, project_response};
@@ -8,7 +8,7 @@ use gateway_protocol::openai::sse::encode_sse_event;
use serde::Serialize;
use serde_json::{Value, json};
use crate::openai::responses::{ProtocolError, ProtocolErrorBody};
use crate::client_error::{ProtocolError, ProtocolErrorBody};
#[derive(Debug, Clone, Serialize)]
pub(in crate::openai) struct ChatIdentity {
@@ -16,6 +16,7 @@ use gateway_core::{
operation::Operation,
};
use crate::client_error::{ProtocolError, ProtocolErrorBody};
use crate::middleware::headers::encode_headers;
use crate::openai::middleware::{
ExpectedBody, PendingExecution, RequestInput, buffered_response, error_response,
@@ -25,7 +26,6 @@ use gateway_core::event::{ProviderEvent, ProviderResponseHeader};
use super::{
error::{engine_error_response, gateway_error_response, protocol_error_response},
responses::{ProtocolError, ProtocolErrorBody},
service::OpenAiService,
};
+5 -184
View File
@@ -8,72 +8,18 @@ use axum::{
};
use gateway_core::{
engine::EngineError,
error::{GatewayError, GatewayErrorKind, ProviderErrorKind},
error::ProviderErrorKind,
policy::{ClientVersionRejection, CodexClientKind},
};
use serde_json::{Value, json};
pub use gateway_core::engine::execution::gateway_error_from_engine;
use super::responses::ProtocolErrorBody;
use crate::client_error::{ProtocolErrorBody, capacity_error_for_client};
/// OpenAI 风格错误响应
pub fn openai_error_response(
status: StatusCode,
message: &str,
error_type: &str,
code: &str,
) -> (StatusCode, Json<Value>) {
let body = json!({
"error": {"message": message, "type": error_type, "code": code}
});
match capacity_error_for_client(&body) {
Some(body) => (StatusCode::SERVICE_UNAVAILABLE, Json(body)),
None => (status, Json(body)),
}
}
/// 无重试提示的容量拒绝及旧版 Codex 限流错误需要可重试投影
/// 仅在交付边界转换,内部仍保留真实上游事实
pub(super) fn client_error_code(code: &str) -> &str {
match code {
"server_is_overloaded" | "slow_down" => "server_error",
_ => code,
}
}
/// 为 HTTP 错误或流内失败保留原有字段,仅投影客户端需要的重试信号
pub(super) fn capacity_error_for_client(data: &Value) -> Option<Value> {
const CODE_PATHS: [&str; 2] = ["/error/code", "/response/error/code"];
let capacity_code = |value: &Value| {
value
.as_str()
.is_some_and(|code| client_error_code(code) != code)
};
if !CODE_PATHS
.iter()
.any(|path| data.pointer(path).is_some_and(capacity_code))
{
return None;
}
let mut projected = data.clone();
for path in CODE_PATHS {
if let Some(code) = projected.pointer_mut(path)
&& capacity_code(code)
{
*code = Value::String("server_error".to_owned());
}
}
// WS 包装错误按 HTTP 状态分类;保留 400/429 会使客户端终止重试
for field in ["status", "status_code"] {
if let Some(status) = projected.get_mut(field)
&& status.is_number()
{
*status = json!(503);
}
}
Some(projected)
}
pub use crate::client_error::{
gateway_error_contract, gateway_error_response, openai_error_response,
};
/// 严格协议 decoder/encoder 返回的安全错误 body
pub fn protocol_error_response(status: StatusCode, body: ProtocolErrorBody) -> Response {
@@ -151,32 +97,6 @@ pub fn model_not_found_response() -> (StatusCode, Json<Value>) {
)
}
/// Gateway 错误的 OpenAI HTTP 表达
pub fn gateway_error_response(error: &GatewayError) -> Response {
let (status, default_type, default_code) = gateway_error_contract(error.kind());
let mut response = openai_error_response(
status,
error.client_message(),
error.client_error_type().unwrap_or(default_type),
error.client_error_code().unwrap_or(default_code),
)
.into_response();
if let Some(delay) = error.retry_after()
&& let Ok(value) = HeaderValue::from_str(
&delay
.as_secs()
.saturating_add(u64::from(delay.subsec_nanos() > 0))
.max(1)
.to_string(),
)
{
response
.headers_mut()
.insert(axum::http::header::RETRY_AFTER, value);
}
response
}
/// 在下游尚未提交时交付 Provider 的 HTTP 失败响应,并应用客户端容量恢复合同
pub fn engine_error_response(error: &EngineError) -> Response {
let capacity_unavailable = matches!(error, EngineError::Provider(error)
@@ -233,102 +153,3 @@ pub fn engine_error_response(error: &EngineError) -> Response {
}
response
}
/// Gateway 错误稳定映射,供 HTTP、SSE 和 WebSocket 共用
#[must_use]
pub const fn gateway_error_contract(
kind: GatewayErrorKind,
) -> (StatusCode, &'static str, &'static str) {
match kind {
GatewayErrorKind::InvalidRequest => (
StatusCode::BAD_REQUEST,
"invalid_request_error",
"invalid_request",
),
GatewayErrorKind::Unsupported => (
StatusCode::BAD_REQUEST,
"invalid_request_error",
"unsupported_capability",
),
GatewayErrorKind::Unauthorized => (
StatusCode::UNAUTHORIZED,
"invalid_request_error",
"invalid_api_key",
),
GatewayErrorKind::PolicyDenied => (
StatusCode::FORBIDDEN,
"invalid_request_error",
"policy_denied",
),
GatewayErrorKind::ModelNotFound => (
StatusCode::NOT_FOUND,
"invalid_request_error",
"model_not_found",
),
GatewayErrorKind::NoAvailableProvider => (
StatusCode::SERVICE_UNAVAILABLE,
"server_error",
"no_available_provider",
),
GatewayErrorKind::AccountCapacityUnavailable => (
StatusCode::SERVICE_UNAVAILABLE,
"server_error",
"account_capacity_unavailable",
),
GatewayErrorKind::ProviderInfrastructureUnavailable => (
StatusCode::SERVICE_UNAVAILABLE,
"server_error",
"provider_infrastructure_unavailable",
),
GatewayErrorKind::MiddlewareCapacityUnavailable => (
StatusCode::SERVICE_UNAVAILABLE,
"server_error",
"middleware_capacity_unavailable",
),
GatewayErrorKind::ConcurrencyQueueFull => (
StatusCode::TOO_MANY_REQUESTS,
"rate_limit_error",
"concurrency_queue_full",
),
GatewayErrorKind::ConcurrencyQueueTimeout => (
StatusCode::TOO_MANY_REQUESTS,
"rate_limit_error",
"concurrency_queue_timeout",
),
GatewayErrorKind::RateLimited => (
StatusCode::TOO_MANY_REQUESTS,
"rate_limit_error",
"rate_limit_exceeded",
),
GatewayErrorKind::UpstreamUnavailable => (
StatusCode::BAD_GATEWAY,
"server_error",
"upstream_unavailable",
),
GatewayErrorKind::MessageTooBig => (
StatusCode::PAYLOAD_TOO_LARGE,
"invalid_request_error",
"message_too_big",
),
GatewayErrorKind::Timeout => (
StatusCode::GATEWAY_TIMEOUT,
"server_error",
"request_timeout",
),
GatewayErrorKind::Cancelled => (
StatusCode::REQUEST_TIMEOUT,
"server_error",
"request_cancelled",
),
GatewayErrorKind::Internal => (
StatusCode::INTERNAL_SERVER_ERROR,
"server_error",
"internal_error",
),
_ => (
StatusCode::INTERNAL_SERVER_ERROR,
"server_error",
"internal_error",
),
}
}
@@ -8,6 +8,7 @@ pub mod images;
pub(crate) mod live;
pub(crate) mod middleware;
pub mod models;
mod peer_link;
pub mod responses;
pub mod router;
pub mod search;

Some files were not shown because too many files have changed in this diff Show More