Compare commits

...
55 Commits
Author SHA1 Message Date
ZyyandCodex 5671a57dc8 chore: bump version to 3.0.26
Co-authored-by: Codex <noreply@openai.com>
2026-07-29 19:44:44 +08:00
Zyy 166e8754a2 fix(xai): restore tool replay and billing details 2026-07-29 18:28:22 +08:00
ZyyandCodex fbaf6c1bdd chore: bump version to 3.0.25
Co-authored-by: Codex <noreply@openai.com>
2026-07-29 17:43:34 +08:00
Zyy 7cdc2dba6f fix(xai): align account state and session replay 2026-07-29 17:36:09 +08:00
ZyyandCodex a321ccf737 chore: bump version to 3.0.24
Co-authored-by: Codex <noreply@openai.com>
2026-07-29 14:49:27 +08:00
CodexandCodex 4533ae8850 fix(openai): preserve resettable quota exhaustion state
Co-authored-by: Codex <noreply@openai.com>
2026-07-29 14:48:23 +08:00
zyycnandCodex 7542ed365e chore: bump version to 3.0.23
Co-authored-by: Codex <noreply@openai.com>
2026-07-29 03:52:17 +08:00
zyycnandCodex bf4da7a3e7 fix(accounts): unify credential and quota state
Apply OpenAI reauthorization to the selected local account ID while atomically replacing its verified upstream identity. Converge successful quota observations across refresh paths and preserve explicitly disabled account state during connection checks, quota refreshes, and credential rotation.

Co-authored-by: Codex <noreply@openai.com>
2026-07-29 03:51:54 +08:00
zyycnandCodex 0fe354b083 fix(openai): preserve requested service tier
Restore 2.x semantics: a request-specified tier drives persisted display and pricing, while the upstream response tier remains separately observable. Cover HTTP SSE and WebSocket conflicts plus the response-only fallback.

Co-authored-by: Codex <noreply@openai.com>
2026-07-29 03:09:22 +08:00
zyycnandCodex 6d0c583599 chore: bump version to 3.0.22
Co-authored-by: Codex <noreply@openai.com>
2026-07-29 02:36:04 +08:00
zyycnandCodex 2e861c03c4 fix(settings): apply refresh margin immediately
Schedule token refreshes at the exact configured lead time and reschedule existing refreshable accounts when that setting changes. Preserve current retry schedules when unrelated settings are saved.

Co-authored-by: Codex <noreply@openai.com>
2026-07-29 02:34:25 +08:00
zyycnandCodex d5fc778ea7 fix(openai): reconcile account connection failures
Expose exact upstream responses to authenticated connection tests, reload the resulting account state, and classify usage-limit windows as resettable quota exhaustion. Restore non-quota availability after verified credential rotation and refresh quota after reauthorization.

Co-authored-by: Codex <noreply@openai.com>
2026-07-29 02:31:38 +08:00
zyycnandCodex 404cd41ff4 docs: refresh runtime and account behavior
Document current OAuth reauthorization, quota state synchronization, response service tiers, transparent wire behavior, buffered observations, and deployment boundaries.

Co-authored-by: Codex <noreply@openai.com>
2026-07-29 01:38:03 +08:00
zyycnandCodex 851f473b88 fix(store): persist continuation affinity before delivery
Remove the asynchronous continuation queue so Core waits for the bounded Redis write before exposing a reusable response ID. Keep Redis index maintenance bounded per record to protect terminal response latency.

Co-authored-by: Codex <noreply@openai.com>
2026-07-29 01:35:05 +08:00
zyycnandCodex e97da4f551 chore: bump version to 3.0.21
Co-authored-by: Codex <noreply@openai.com>
2026-07-29 01:09:42 +08:00
zyycnandCodex 5544b0fb29 fix(openai): separate quota and credential auth state
Keep quota probes from terminally invalidating OAuth accounts on 401 or 403, and stop manual quota refreshes from querying upstream with an expired access token. Model quota-owned transitions explicitly and cover both boundaries with regression tests.

Co-authored-by: Codex <noreply@openai.com>
2026-07-29 01:09:07 +08:00
zyycnandCodex 428e744710 test(providers): remove destructive live fixtures
Delete secret-dependent integration tests that consume live quota, register upstream tasks, or rotate OAuth refresh tokens. Keep the default workspace suite entirely runnable without ignored cases.

Co-authored-by: Codex <noreply@openai.com>
2026-07-29 01:06:09 +08:00
zyycnandCodex 7b406765d6 fix(store): harden execution observation buffering
Log direct Store failures without changing client-visible results, emit connection-test diagnostics before asynchronous persistence, and keep Provider errors intact when observation writes fail. Add a process-wide byte budget, cumulative queue statistics, ordered failure isolation, and bounded shutdown draining to the PostgreSQL observation worker.

Co-authored-by: Codex <noreply@openai.com>
2026-07-29 00:55:33 +08:00
zyycnandCodex 92ae7df115 fix(openai): bind reauthorization to stable account identity
Co-authored-by: Codex <noreply@openai.com>
2026-07-29 00:45:43 +08:00
zyycnandCodex cc42e78334 refactor(store): buffer data-plane side effects
Move PostgreSQL execution observations and Redis release, circuit, and continuation writes onto bounded non-blocking queues. Supervise independent flush workers so database or coordination failures can drop recoverable side effects without delaying protocol delivery.

Co-authored-by: Codex <noreply@openai.com>
2026-07-29 00:11:26 +08:00
zyycnandCodex 0c90ed30fb fix(proxy): restore transparent wire semantics
Preserve raw request and response data across HTTP, SSE, and WebSocket adapters, including opaque headers and upstream error envelopes. Remove private payload and catalog gates, keep account switching behind the pre-delivery barrier, and make observation failures unable to replace client-visible protocol data.

Co-authored-by: Codex <noreply@openai.com>
2026-07-29 00:10:35 +08:00
zyycnandCodex 5a81856510 fix(openai): preserve opaque response IDs
Treat Codex response handles as opaque strings across request decoding, continuation affinity, and upstream metadata. Store raw UTF-8 bytes without the PostgreSQL text index so long or NUL-containing handles cannot fail after a successful response.

Co-authored-by: Codex <noreply@openai.com>
2026-07-29 00:07:07 +08:00
zyycnandCodex 3110b71512 chore: bump version to 3.0.20
Co-authored-by: Codex <noreply@openai.com>
2026-07-28 18:46:20 +08:00
zyycnandCodex eac1b2c39e fix(xai): classify upstream failures by scope
Co-authored-by: Codex <noreply@openai.com>
2026-07-28 18:31:06 +08:00
zyycnandCodex 337bb80230 fix(openai): restore streaming failure fallback
Restore the v2.x stream commit policies so ordinary requests forward the first event immediately while continuation and same-batch failures remain behind the retry boundary. Preserve typed upstream failure batches through Core and both streaming adapters, with a 64 KiB prefetch limit and end-to-end regression coverage.

Co-authored-by: Codex <noreply@openai.com>
2026-07-28 17:29:15 +08:00
zyycnandCodex 82f2a0da06 fix(openai): preserve quota exhaustion state
Prefer explicit quota evidence over generic 429 feedback, keep exhausted and terminal accounts from being downgraded by temporary rate limits, and reconcile historical cooldown states from current quota observations.

Co-authored-by: Codex <noreply@openai.com>
2026-07-28 14:47:21 +08:00
zyycnandCodex f0d8dad71e chore: bump version to 3.0.19
Co-authored-by: Codex <noreply@openai.com>
2026-07-28 13:05:04 +08:00
zyycnandCodex f5c1d773d7 fix(accounts): use current credential revisions
Load the current credential revision server-side for reauthorization and manual refresh so background token refreshes do not make admin mutations fail stale client fences.

Co-authored-by: Codex <noreply@openai.com>
2026-07-28 13:04:12 +08:00
zyycnandCodex 9148e6b11e chore: bump version to 3.0.18
Co-authored-by: Codex <noreply@openai.com>
2026-07-28 09:48:21 +08:00
zyycnandCodex a7d8c3d3df fix(accounts): make fixed diagnostics resilient
Co-authored-by: Codex <noreply@openai.com>
2026-07-28 09:45:35 +08:00
zyycnandCodex 997eb86aa0 fix(openai): refresh expired OAuth access tokens
Force refresh scheduling for OAuth accounts whose access token has already expired, even when a persisted next_refresh_at points to the future.

Co-authored-by: Codex <noreply@openai.com>
2026-07-28 09:41:44 +08:00
zyycnandCodex 3866acc269 chore: bump version to 3.0.17
Co-authored-by: Codex <noreply@openai.com>
2026-07-28 00:06:15 +08:00
zyycnandCodex 38477f4328 fix(openai): normalize OAuth import credentials
Import OAuth accounts from access and refresh tokens only, then verify identity and observe the initial quota through official endpoints.

Co-authored-by: Codex <noreply@openai.com>
2026-07-27 23:54:59 +08:00
zyycnandCodex 5168f6a924 fix(openai): replace stale passive rate-limit snapshots
Treat each passive response-header observation as a complete rate-limit bucket snapshot, preventing stale secondary windows from surviving into a newer primary-only update.\n\nExtract duplicate OAuth callback parameter rejection to keep the release quality gate warning-free.

Co-authored-by: Codex <noreply@openai.com>
2026-07-27 23:12:17 +08:00
zyycnandCodex 3a407c635a chore: bump version to 3.0.16
Co-authored-by: Codex <noreply@openai.com>
2026-07-27 22:39:31 +08:00
zyycnandCodex 6901e0ed45 fix(openai): accept standard OAuth callback parameters
Accept standard OAuth callback parameters while keeping one-shot flow and duplicate security parameter rejection.

Classify unavailable identity verification correctly and add redacted OAuth/import failure diagnostics.

Co-authored-by: Codex <noreply@openai.com>
2026-07-27 22:39:01 +08:00
zyycnandClaude Fable 5 9758c420fa refactor(availability): converge account availability on the gateway-core enum
- gateway-admin: drop the mirrored 7-variant enum; re-export the core type
- gateway-api: serialize wire availability via core as_str (bytes unchanged)
- gateway-store: drop ProviderAccountAvailability and four identity mappers;
  rows parse through core's codec with the same error string; redis state
  validation reuses core parse instead of a literal 7-string set
- providers/openai+xai: drop core<->admin identity mappers made moot by the
  admin re-export
- providers/xai: a single exhaustive From<GrokCredentialAvailability> impl
  is now the only Grok->core translation point

No behavior change: DB strings, wire JSON, and the AccountStatus fold
(including Invalid->Expired display) are byte-identical. 1244 tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 20:40:47 +08:00
ZyyandCodex 6d2bf06ac7 chore: bump version to 3.0.15
Co-authored-by: Codex <noreply@openai.com>
2026-07-27 17:29:36 +08:00
ZyyandCodex b8faaf250f fix(frontend): align dashboard account usage metrics
Add a metric layout variant for account usage windows so dashboard account rows align request bars with adjacent recent-usage text without changing the shared compact account-table presentation.

Co-authored-by: Codex <noreply@openai.com>
2026-07-27 17:22:28 +08:00
ZyyandCodex 40137b1205 fix(backend): preserve safe upstream error details
Keep admin account connection tests routed through the provider probe even when the stored account is already marked unavailable so clicking test can refresh state from the upstream response.

Preserve provider-declared safe upstream details when OpenAI-compatible API errors collapse local provider unavailability to the stable no-provider contract.

Co-authored-by: Codex <noreply@openai.com>
2026-07-27 17:09:59 +08:00
ZyyandCodex c67cde4e90 chore: bump version to 3.0.14
Co-authored-by: Codex <noreply@openai.com>
2026-07-27 16:28:29 +08:00
ZyyandCodex 2e9213a784 fix(backend): derive Codex metadata from provider catalogs
Share Codex Responses request semantics across OpenAI and xAI providers so reasoning effort, presets, request kind, subagent kind, and compaction observations stay consistent.

Read xAI reasoning efforts from official catalog feature options instead of model-name hardcoding, and make Codex service tiers data-driven through ModelPresentation rather than API adapter defaults.

Co-authored-by: Codex <noreply@openai.com>
2026-07-27 16:27:41 +08:00
ZyyandCodex 87496b5656 fix(frontend): polish dashboard layout
Keep dashboard header actions pinned to the action column, widen the xAI runtime field, and align active-account usage metrics without changing the shared usage window component.

Co-authored-by: Codex <noreply@openai.com>
2026-07-27 14:55:36 +08:00
ZyyandCodex c856e4306f fix(xai): mark rate limits as quota exhausted
Classify xAI 429 rate-limit responses as quota exhaustion so the scheduler removes the account from active rotation until the normal recovery flow makes it available again.

Cover the provider replay path and transport classification with tests.

Co-authored-by: Codex <noreply@openai.com>
2026-07-27 14:55:30 +08:00
ZyyandCodex 92090f28bf docs(backend): unify provider comments to Chinese
Convert the remaining English doc comments in the openai and xai provider
crates to Chinese, function-oriented wording so the whole backend follows
one convention. Explanatory narration is compressed into what the item does
and what its errors mean; code identifiers, route paths, rustdoc links and
`# Errors` headings stay as-is.

Also fixes two half-translated comments left in the websocket protocol
module and merges a duplicated doc block on the OIDC verifier constructor.
No code, signature or `#[error(...)]` string changes.

Co-authored-by: Codex <noreply@openai.com>
2026-07-27 13:13:44 +08:00
ZyyandCodex 12c2b612de chore(db): fold retention indexes into initial migration
Keep the database baseline as a single initial migration now that deployed instances have already been upgraded. The retention delete indexes move into 0001, the standalone 0002 file is removed, and the migration tests now expect one applied migration.

Co-authored-by: Codex <noreply@openai.com>
2026-07-27 12:05:49 +08:00
e8af0b9c66 chore(deps): Bump ed25519-dalek from 2.2.0 to 3.0.0 in /backend (#26)
* chore(deps): Bump ed25519-dalek from 2.2.0 to 3.0.0 in /backend

Bumps [ed25519-dalek](https://github.com/dalek-cryptography/curve25519-dalek) from 2.2.0 to 3.0.0.
- [Release notes](https://github.com/dalek-cryptography/curve25519-dalek/releases)
- [Changelog](https://github.com/dalek-cryptography/curve25519-dalek/blob/3.0.0/CHANGELOG.md)
- [Commits](https://github.com/dalek-cryptography/curve25519-dalek/compare/ed25519-2.2.0...3.0.0)

---
updated-dependencies:
- dependency-name: ed25519-dalek
  dependency-version: 3.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: enable ed25519 PKCS8 allocation support

ed25519-dalek 3 gates private-key PKCS#8 encoding behind the alloc feature. Enable it alongside pkcs8 so existing credential export tests continue to compile and exercise the same behavior.

Co-authored-by: Codex <noreply@openai.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zyy <No Email>
Co-authored-by: Codex <noreply@openai.com>
2026-07-27 12:04:48 +08:00
80776e8e03 chore(deps): Bump jsonwebtoken from 10.3.0 to 11.0.0 in /backend (#24)
* chore(deps): Bump jsonwebtoken from 10.3.0 to 11.0.0 in /backend

Bumps [jsonwebtoken](https://github.com/Keats/jsonwebtoken) from 10.3.0 to 11.0.0.
- [Changelog](https://github.com/Keats/jsonwebtoken/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Keats/jsonwebtoken/compare/v10.3.0...v11.0.0)

---
updated-dependencies:
- dependency-name: jsonwebtoken
  dependency-version: 11.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: adapt jsonwebtoken 11 header extras API

jsonwebtoken 11 wraps non-standard JWT header fields in Extras. Keep the existing strict header validation by checking the wrapped inner map instead of calling is_empty directly.

Co-authored-by: Codex <noreply@openai.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zyy <No Email>
Co-authored-by: Codex <noreply@openai.com>
2026-07-27 11:54:06 +08:00
dependabot[bot] b296c795ac chore(deps): Bump the minor-and-patch group across 1 directory with 3 updates (#22)
Bumps the minor-and-patch group with 3 updates in the / directory: [actions/checkout](https://github.com/actions/checkout), [github/codeql-action/upload-sarif](https://github.com/github/codeql-action) and [docker/login-action](https://github.com/docker/login-action).


Updates `actions/checkout` from 7.0.0 to 7.0.1
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0...3d3c42e5aac5ba805825da76410c181273ba90b1)

Updates `github/codeql-action/upload-sarif` from 4.37.1 to 4.37.3
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/7188fc363630916deb702c7fdcf4e481b751f97a...e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81)

Updates `docker/login-action` from 4.4.0 to 4.5.1
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/af1e73f918a031802d376d3c8bbc3fe56130a9b0...abd2ef45e78c5afb21d64d4ca52ee8550d9572c7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: docker/login-action
  dependency-version: 4.5.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: github/codeql-action/upload-sarif
  dependency-version: 4.37.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 11:41:57 +08:00
dependabot[bot] 9fc162a82f chore(deps): Bump base64 from 0.22.1 to 0.23.0 in /backend (#25)
Bumps [base64](https://github.com/marshallpierce/rust-base64) from 0.22.1 to 0.23.0.
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.22.1...v0.23.0)

---
updated-dependencies:
- dependency-name: base64
  dependency-version: 0.23.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 11:41:53 +08:00
dependabot[bot] 9532bd438b chore(deps): Bump getrandom from 0.3.4 to 0.4.3 in /backend (#27)
Bumps [getrandom](https://github.com/rust-random/getrandom) from 0.3.4 to 0.4.3.
- [Changelog](https://github.com/rust-random/getrandom/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-random/getrandom/compare/v0.3.4...v0.4.3)

---
updated-dependencies:
- dependency-name: getrandom
  dependency-version: 0.4.3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 11:41:49 +08:00
dependabot[bot] 73319f1943 chore(deps): Bump syn from 2.0.118 to 3.0.3 in /backend (#28)
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.118 to 3.0.3.
- [Release notes](https://github.com/dtolnay/syn/releases)
- [Commits](https://github.com/dtolnay/syn/compare/2.0.118...3.0.3)

---
updated-dependencies:
- dependency-name: syn
  dependency-version: 3.0.3
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 11:41:45 +08:00
dependabot[bot] ee56d6a1d2 chore(deps): Bump the minor-and-patch group across 1 directory with 8 updates (#29)
Bumps the minor-and-patch group with 8 updates in the /frontend directory:

| Package | From | To |
| --- | --- | --- |
| [@fontsource-variable/inter](https://github.com/fontsource/font-files/tree/HEAD/fonts/variable/inter) | `5.2.8` | `5.3.0` |
| [@fontsource-variable/jetbrains-mono](https://github.com/fontsource/font-files/tree/HEAD/fonts/variable/jetbrains-mono) | `5.2.8` | `5.3.0` |
| [@lucide/vue](https://github.com/lucide-icons/lucide/tree/HEAD/packages/vue) | `1.26.0` | `1.27.0` |
| [es-toolkit](https://github.com/toss/es-toolkit) | `1.49.0` | `1.50.0` |
| [marked](https://github.com/markedjs/marked) | `18.0.6` | `18.0.7` |
| [@antfu/eslint-config](https://github.com/antfu/eslint-config) | `9.1.0` | `9.2.0` |
| [eslint](https://github.com/eslint/eslint) | `10.7.0` | `10.8.0` |
| [vue-tsc](https://github.com/vuejs/language-tools/tree/HEAD/packages/tsc) | `3.3.7` | `3.3.8` |



Updates `@fontsource-variable/inter` from 5.2.8 to 5.3.0
- [Changelog](https://github.com/fontsource/font-files/blob/main/CHANGELOG.md)
- [Commits](https://github.com/fontsource/font-files/commits/HEAD/fonts/variable/inter)

Updates `@fontsource-variable/jetbrains-mono` from 5.2.8 to 5.3.0
- [Changelog](https://github.com/fontsource/font-files/blob/main/CHANGELOG.md)
- [Commits](https://github.com/fontsource/font-files/commits/HEAD/fonts/variable/jetbrains-mono)

Updates `@lucide/vue` from 1.26.0 to 1.27.0
- [Release notes](https://github.com/lucide-icons/lucide/releases)
- [Commits](https://github.com/lucide-icons/lucide/commits/1.27.0/packages/vue)

Updates `es-toolkit` from 1.49.0 to 1.50.0
- [Release notes](https://github.com/toss/es-toolkit/releases)
- [Changelog](https://github.com/toss/es-toolkit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/toss/es-toolkit/compare/v1.49.0...v1.50.0)

Updates `marked` from 18.0.6 to 18.0.7
- [Release notes](https://github.com/markedjs/marked/releases)
- [Commits](https://github.com/markedjs/marked/compare/v18.0.6...v18.0.7)

Updates `@antfu/eslint-config` from 9.1.0 to 9.2.0
- [Release notes](https://github.com/antfu/eslint-config/releases)
- [Commits](https://github.com/antfu/eslint-config/compare/v9.1.0...v9.2.0)

Updates `eslint` from 10.7.0 to 10.8.0
- [Release notes](https://github.com/eslint/eslint/releases)
- [Commits](https://github.com/eslint/eslint/compare/v10.7.0...v10.8.0)

Updates `vue-tsc` from 3.3.7 to 3.3.8
- [Release notes](https://github.com/vuejs/language-tools/releases)
- [Changelog](https://github.com/vuejs/language-tools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vuejs/language-tools/commits/v3.3.8/packages/tsc)

---
updated-dependencies:
- dependency-name: "@fontsource-variable/inter"
  dependency-version: 5.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: "@fontsource-variable/jetbrains-mono"
  dependency-version: 5.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: "@lucide/vue"
  dependency-version: 1.27.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: es-toolkit
  dependency-version: 1.50.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: marked
  dependency-version: 18.0.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: "@antfu/eslint-config"
  dependency-version: 9.2.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: eslint
  dependency-version: 10.8.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: vue-tsc
  dependency-version: 3.3.8
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-27 11:41:41 +08:00
ZyyandCodex 5d29ddfffc chore: bump version to 3.0.13
Co-authored-by: Codex <noreply@openai.com>
2026-07-27 11:33:55 +08:00
ZyyandCodex 45903b684b fix(admin): remove config revision fences from API writes
Operator mutations no longer require callers to carry configRevision values through accounts, client keys, provider credentials, or settings. The store and admin use cases now apply current state directly, which removes stale-revision failures from normal create, update, and delete flows.

xAI Responses catalog entries now let the upstream validate feature support instead of rejecting requests locally when optional catalog metadata is absent, matching the Build routing behavior used by grok2api.

Co-authored-by: Codex <noreply@openai.com>
2026-07-27 11:30:59 +08:00
236 changed files with 20840 additions and 6303 deletions
+2 -2
View File
@@ -40,7 +40,7 @@ jobs:
} >> "$GITHUB_ENV"
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ inputs.ref }}
@@ -76,7 +76,7 @@ jobs:
${{ hashFiles('trivy-results.sarif') != '' &&
(github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository) }}
uses: github/codeql-action/upload-sarif@7188fc363630916deb702c7fdcf4e481b751f97a # v4.37.1
uses: github/codeql-action/upload-sarif@e4fba868fa4b1b91e1fdab776edc8cfbe6e9fb81 # v4.37.3
with:
sarif_file: trivy-results.sarif
category: runtime-container
+3 -3
View File
@@ -61,7 +61,7 @@ jobs:
--health-retries 10
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ inputs.ref }}
# PR 场景需要 base 提交做冻结清单的 append-only 比对。
@@ -152,7 +152,7 @@ jobs:
CI: 'true'
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ inputs.ref }}
@@ -191,7 +191,7 @@ jobs:
timeout-minutes: 5
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ inputs.ref }}
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
container: ${{ steps.filter.outputs.container }}
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
fetch-depth: 0
+8 -8
View File
@@ -41,7 +41,7 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event_name == 'workflow_dispatch' && inputs.tag || github.ref }}
fetch-depth: 0
@@ -177,7 +177,7 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ needs.release-info.outputs.git_sha }}
@@ -237,7 +237,7 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ needs.release-info.outputs.git_sha }}
@@ -289,7 +289,7 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ needs.release-info.outputs.git_sha }}
@@ -408,7 +408,7 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ needs.release-info.outputs.git_sha }}
@@ -454,7 +454,7 @@ jobs:
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
- name: Login to GHCR
uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4.4.0
uses: docker/login-action@abd2ef45e78c5afb21d64d4ca52ee8550d9572c7 # v4.5.1
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -508,7 +508,7 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ needs.release-info.outputs.git_sha }}
@@ -525,7 +525,7 @@ jobs:
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
- name: Login to GHCR
uses: docker/login-action@af1e73f918a031802d376d3c8bbc3fe56130a9b0 # v4.4.0
uses: docker/login-action@abd2ef45e78c5afb21d64d4ca52ee8550d9572c7 # v4.5.1
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
+2 -2
View File
@@ -23,7 +23,7 @@ jobs:
timeout-minutes: 20
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
@@ -47,7 +47,7 @@ jobs:
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
+18 -11
View File
@@ -21,15 +21,16 @@
## 能力
| 领域 | 实现 |
| -------- | --------------------------------------------------------------------------- |
| 协议 | OpenAI Responses JSON、SSE、WebSocket 与模型目录 |
| Provider | 固定 OpenAI、xAI 两个 Provider;各自管理 OAuth、账号、额度与模型目录 |
| 路由 | Client Key 绑定 Provider、全局模型映射、会话亲和与同 Provider 安全 fallback |
| 透明边界 | OpenAI SSE 与 WebSocket 均原样转发上游字节;xAI 在 `providers/xai` 内做 Grok 与 Responses 协议转换 |
| 延续 | OpenAI native/replay continuation;xAI 使用客户端提交的完整历史 |
| 管理 | Client Key、账号、模型目录、设置、观测、系统更新与 OAuth |
| 计量 | 模型请求 Token、费用、延迟、账号与 Provider 归因 |
| 领域 | 实现 |
| -------- | ----------------------------------------------------------------------------------------------------- |
| 协议 | OpenAI Responses JSON、SSE、WebSocket 与模型目录 |
| Provider | 固定 OpenAI、xAI 两个 Provider;各自管理 OAuth、账号、额度与模型目录 |
| 路由 | Client Key 绑定 Provider、全局模型映射、会话亲和与同 Provider 安全 fallback |
| 透明边界 | OpenAI 请求保留未知字段和字段顺序,SSE 与 WebSocket 原样转发业务字节;xAI 在 Provider 内做协议转换 |
| 延续 | OpenAI native/replay continuation;xAI 使用客户端提交的完整历史 |
| 账号 | 导入、OAuth、credential 刷新、主动额度刷新,以及正常响应中的被动额度/状态观测 |
| 管理 | Client Key、账号、模型目录、设置、观测、系统更新与 OAuth |
| 计量 | 模型请求 Token、费用、延迟、账号与 Provider 归因,并记录响应确认的实际 `service_tier` |
## 快速开始
@@ -93,6 +94,11 @@ curl -i http://127.0.0.1:8080/healthz
3. 按需配置客户端模型到上游模型的全局映射;未命中映射的模型名原样透传。
4. 创建 `sk_...` Client Key,绑定 `openai` 或 `xai`,并设置速率与并发限制。
OpenAI 账号导入和首次 OAuth 创建在 credential 提交后会立即尝试一次额度观测;额度接口临时失败不会
回滚已提交的账号。Free、K12 等套餐共用同一套额度与状态转换逻辑,套餐字段只作为账号事实并隔离
对应的模型目录 cache。credential 刷新、额度刷新和正常请求中的被动额度观测是三条独立链路,详见
[账号接口](docs/api.md#5-账号)。
> [!IMPORTANT]
> xAI 使用 OAuth session,不支持把 xAI API Key 作为上游 credential。
@@ -145,7 +151,7 @@ curl http://127.0.0.1:8080/v1/responses \
## 运维
```bash
# 升级
# 实例升级
docker compose -f deploy/compose.yaml pull codex-proxy-rs
docker compose -f deploy/compose.yaml up -d --no-build
@@ -179,7 +185,8 @@ release/publish <version>
```
脚本会更新 `release/version.yaml`、创建约定提交与带注释的 `v<version>` tag,并原子推送分支和 tag。
不要手工修改版本后单独打 tag。
不要手工修改版本后单独打 tag。该脚本只发布仓库版本和触发 Release 构建,不会登录服务器,也不会
改变任何运行实例;源码提交、仓库发版和实例升级是三个独立状态。
## 文档
+127 -59
View File
@@ -142,7 +142,7 @@ checksum = "c7c24de15d275a1ecfd47a380fb4d5ec9bfe0933f309ed5e705b775596a3574d"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
]
[[package]]
@@ -208,7 +208,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "31b698c5f9a010f6573133b09e0de5408834d0c82f8d7475a89fc1867a71cd90"
dependencies = [
"axum-core",
"base64",
"base64 0.22.1",
"bytes",
"form_urlencoded",
"futures-util",
@@ -275,6 +275,12 @@ version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
[[package]]
name = "base64"
version = "0.23.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b25655df2c3cdd83c5e5b293b88acd880332b2ddadd7c30ac43144fdc0033da9"
[[package]]
name = "base64ct"
version = "1.8.3"
@@ -444,7 +450,7 @@ dependencies = [
"provider-openai",
"provider-xai",
"serde",
"syn 2.0.118",
"syn 3.0.3",
"tempfile",
"thiserror",
"tokio",
@@ -669,7 +675,7 @@ dependencies = [
"aead",
"blake2",
"crypto_secretbox",
"curve25519-dalek",
"curve25519-dalek 4.1.3",
"salsa20",
"subtle",
"zeroize",
@@ -708,8 +714,23 @@ dependencies = [
"cfg-if",
"cpufeatures 0.2.17",
"curve25519-dalek-derive",
"digest 0.10.7",
"fiat-crypto",
"fiat-crypto 0.2.9",
"rustc_version",
"subtle",
"zeroize",
]
[[package]]
name = "curve25519-dalek"
version = "5.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b5eed333089e2e1c1ac8c6c0398e5e2497b4c9926ca6d0365ed1e099afa5bc23"
dependencies = [
"cfg-if",
"cpufeatures 0.3.0",
"curve25519-dalek-derive",
"digest 0.11.3",
"fiat-crypto 0.3.0",
"rustc_version",
"subtle",
"zeroize",
@@ -723,7 +744,7 @@ checksum = "f46882e17999c6cc590af592290432be3bce0428cb0d5f8b6715e4dc7b383eb3"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
]
[[package]]
@@ -761,6 +782,16 @@ dependencies = [
"zeroize",
]
[[package]]
name = "der"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a69dedd701da44b0536442edf09c81a64b0ab97a7a4a5e3d1971f00027cbc63d"
dependencies = [
"const-oid 0.10.2",
"zeroize",
]
[[package]]
name = "deranged"
version = "0.5.8"
@@ -799,7 +830,7 @@ checksum = "1ac70aa55017e108007fbaf5aa0f54b021c98f92ff8af59d42eda9da96e3dd4f"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
]
[[package]]
@@ -829,34 +860,35 @@ version = "0.16.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ee27f32b5c5292967d2d4a9d7f1e0b0aed2c15daded5a60300e4abb9d8020bca"
dependencies = [
"der",
"der 0.7.10",
"digest 0.10.7",
"elliptic-curve",
"rfc6979",
"signature",
"spki",
"signature 2.2.0",
"spki 0.7.3",
]
[[package]]
name = "ed25519"
version = "2.2.3"
version = "3.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "115531babc129696a58c64a4fef0a8bf9e9698629fb97e9e40767d235cfbcd53"
checksum = "29fcf32e6c73d1079f83ab4d782de2d81620346a5f38c6237a86a22f8368980a"
dependencies = [
"pkcs8",
"signature",
"pkcs8 0.11.0",
"signature 3.0.0",
]
[[package]]
name = "ed25519-dalek"
version = "2.2.0"
version = "3.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70e796c081cee67dc755e1a36a0a172b897fab85fc3f6bc48307991f64e4eca9"
checksum = "6ebaa1a2bf1290ab3bfe5a7b771d050ebffab2711c19a81691c683a5144a25de"
dependencies = [
"curve25519-dalek",
"curve25519-dalek 5.0.0",
"ed25519",
"serde",
"sha2 0.10.9",
"sha2 0.11.0",
"signature 3.0.0",
"subtle",
"zeroize",
]
@@ -883,7 +915,7 @@ dependencies = [
"generic-array",
"group",
"pem-rfc7468",
"pkcs8",
"pkcs8 0.10.2",
"rand_core 0.6.4",
"sec1",
"subtle",
@@ -968,6 +1000,12 @@ version = "0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d"
[[package]]
name = "fiat-crypto"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "64cd1e32ddd350061ae6edb1b082d7c54915b5c672c389143b9a63403a109f24"
[[package]]
name = "filetime"
version = "0.2.29"
@@ -1100,7 +1138,7 @@ checksum = "2d6d3cde68c518367be28956066ddfef33813991b77a55005a69dae04bf3b10b"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
]
[[package]]
@@ -1144,7 +1182,7 @@ version = "0.0.0"
dependencies = [
"argon2",
"async-trait",
"base64",
"base64 0.23.0",
"chrono",
"futures",
"gateway-core",
@@ -1156,6 +1194,7 @@ dependencies = [
"subtle",
"thiserror",
"tokio",
"tracing",
"uuid",
]
@@ -1165,7 +1204,7 @@ version = "0.1.0"
dependencies = [
"async-trait",
"axum",
"base64",
"base64 0.23.0",
"bytes",
"chrono",
"futures",
@@ -1176,6 +1215,7 @@ dependencies = [
"serde_json",
"thiserror",
"tokio",
"tokio-tungstenite 0.29.0",
"tower",
"tower-http 0.7.0",
"tracing",
@@ -1194,6 +1234,7 @@ dependencies = [
"serde",
"serde_json",
"thiserror",
"tracing",
"uuid",
]
@@ -1371,7 +1412,7 @@ version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b3314d5adb5d94bcdf56771f2e50dbbc80bb4bdf88967526706205ac9eff24eb"
dependencies = [
"base64",
"base64 0.22.1",
"bytes",
"headers-core",
"http",
@@ -1538,7 +1579,7 @@ version = "0.1.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96547c2556ec9d12fb1578c4eaf448b04993e7fb79cbaad930a656880a6bdfa0"
dependencies = [
"base64",
"base64 0.22.1",
"bytes",
"futures-channel",
"futures-util",
@@ -1736,19 +1777,20 @@ dependencies = [
[[package]]
name = "jsonwebtoken"
version = "10.3.0"
version = "11.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0529410abe238729a60b108898784df8984c87f6054c9c4fcacc47e4803c1ce1"
checksum = "881733cbc631fc9e472e24447ce32a64bedf2da498d6d8570b08edc87de71f65"
dependencies = [
"aws-lc-rs",
"base64",
"base64 0.22.1",
"getrandom 0.2.17",
"js-sys",
"pem",
"serde",
"serde_json",
"signature",
"signature 2.2.0",
"simple_asn1",
"zeroize",
]
[[package]]
@@ -2015,7 +2057,7 @@ version = "3.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d30c53c26bc5b31a98cd02d20f25a7c8567146caf63ed593a9d87b2775291be"
dependencies = [
"base64",
"base64 0.22.1",
"serde_core",
]
@@ -2046,8 +2088,18 @@ version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7"
dependencies = [
"der",
"spki",
"der 0.7.10",
"spki 0.7.3",
]
[[package]]
name = "pkcs8"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "451913da69c775a56034ea8d9003d27ee8948e12443eae7c038ba100a4f21cb7"
dependencies = [
"der 0.8.1",
"spki 0.8.0",
]
[[package]]
@@ -2115,7 +2167,7 @@ version = "0.0.0"
dependencies = [
"async-stream",
"async-trait",
"base64",
"base64 0.23.0",
"bytes",
"chrono",
"cookie",
@@ -2125,7 +2177,7 @@ dependencies = [
"gateway-admin",
"gateway-core",
"gateway-protocol",
"getrandom 0.3.4",
"getrandom 0.4.3",
"hex",
"jsonwebtoken",
"reqwest",
@@ -2157,14 +2209,14 @@ version = "0.0.0"
dependencies = [
"async-stream",
"async-trait",
"base64",
"base64 0.23.0",
"bytes",
"chrono",
"futures",
"gateway-admin",
"gateway-core",
"gateway-protocol",
"getrandom 0.3.4",
"getrandom 0.4.3",
"jsonwebtoken",
"p256",
"reqwest",
@@ -2409,7 +2461,7 @@ version = "0.12.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eddd3ca559203180a307f12d114c268abf583f59b03cb906fd0b3ff8646c1147"
dependencies = [
"base64",
"base64 0.22.1",
"bytes",
"cookie",
"cookie_store",
@@ -2598,9 +2650,9 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3e97a565f76233a6003f9f5c54be1d9c5bdfa3eccfb189469f11ec4901c47dc"
dependencies = [
"base16ct",
"der",
"der 0.7.10",
"generic-array",
"pkcs8",
"pkcs8 0.10.2",
"subtle",
"zeroize",
]
@@ -2796,6 +2848,12 @@ dependencies = [
"rand_core 0.6.4",
]
[[package]]
name = "signature"
version = "3.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "28d567dcbaf0049cb8ac2608a76cd95ff9e4412e1899d389ee400918ca7537f5"
[[package]]
name = "simd-adler32"
version = "0.3.9"
@@ -2855,7 +2913,17 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d"
dependencies = [
"base64ct",
"der",
"der 0.7.10",
]
[[package]]
name = "spki"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d9efca8738c78ee9484207732f728b1ef517bbb1833d6fc0879ca898a522f6f"
dependencies = [
"base64ct",
"der 0.8.1",
]
[[package]]
@@ -2877,7 +2945,7 @@ version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05b44e85bf579a8eeb4ceaa77a3a523baf2bf0e9bac7e40f405d537b5d2d5ccb"
dependencies = [
"base64",
"base64 0.22.1",
"bytes",
"cfg-if",
"chrono",
@@ -2918,7 +2986,7 @@ dependencies = [
"quote",
"sqlx-core",
"sqlx-macros-core",
"syn 2.0.118",
"syn 2.0.119",
]
[[package]]
@@ -2941,7 +3009,7 @@ dependencies = [
"sqlx-mysql",
"sqlx-postgres",
"sqlx-sqlite",
"syn 2.0.118",
"syn 2.0.119",
"thiserror",
"tokio",
"url",
@@ -2981,7 +3049,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "87a2bdd6e83f6b3ea525ca9fee568030508b58355a43d0b2c1674d5f79dcd65e"
dependencies = [
"atoi",
"base64",
"base64 0.22.1",
"bitflags",
"byteorder",
"chrono",
@@ -3066,9 +3134,9 @@ checksum = "a7973cce6668464ea31f176d85b13c7ab3bba2cb3b77a2ed26abd7801688010a"
[[package]]
name = "syn"
version = "2.0.118"
version = "2.0.119"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1b9ae57f904213ebb649ce6895b8a66c66f0203b9319718f69a5612a065b1422"
checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
dependencies = [
"proc-macro2",
"quote",
@@ -3103,7 +3171,7 @@ checksum = "728a70f3dbaf5bab7f0c4b1ac8d7ae5ea60a4b5549c8a5914361c99147a709d2"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
]
[[package]]
@@ -3124,7 +3192,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom 0.3.4",
"getrandom 0.4.3",
"once_cell",
"rustix",
"windows-sys 0.61.2",
@@ -3238,7 +3306,7 @@ checksum = "385a6cb71ab9ab790c5fe8d67f1645e6c450a7ce006a33de03daa956cf70a496"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
]
[[package]]
@@ -3414,7 +3482,7 @@ checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
]
[[package]]
@@ -3691,7 +3759,7 @@ dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
"wasm-bindgen-shared",
]
@@ -3773,7 +3841,7 @@ checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
]
[[package]]
@@ -3784,7 +3852,7 @@ checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
]
[[package]]
@@ -3909,7 +3977,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08db1edfb05d9b3c1542e521aea074442088292f00b5f28e435c714a98f85031"
dependencies = [
"assert-json-diff",
"base64",
"base64 0.22.1",
"deadpool",
"futures",
"http",
@@ -3983,7 +4051,7 @@ checksum = "de844c262c8848816172cef550288e7dc6c7b7814b4ee56b3e1553f275f1858e"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
"synstructure",
]
@@ -4004,7 +4072,7 @@ checksum = "e2e817b7b52d0c7358d3246da9d69935ebb18116b2b102b4230dac079b4862f5"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
]
[[package]]
@@ -4024,7 +4092,7 @@ checksum = "11532158c46691caf0f2593ea8358fed6bbf68a0315e80aae9bd41fbade684a1"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
"synstructure",
]
@@ -4045,7 +4113,7 @@ checksum = "3c50655cbb0fe3fc43170059e702f1ce5e19b84cec58dc87b037a09935c2f328"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
]
[[package]]
@@ -4078,7 +4146,7 @@ checksum = "625dc425cab0dca6dc3c3319506e6593dcb08a9f387ea3b284dbd52a92c40555"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.118",
"syn 2.0.119",
]
[[package]]
+1 -1
View File
@@ -25,7 +25,7 @@ tokio = { version = "1.52.3", default-features = false, features = ["rt-multi-th
[dev-dependencies]
config = { version = "0.15.23", default-features = false, features = ["yaml"] }
syn = { version = "2.0.118", features = ["full"] }
syn = { version = "3.0.3", features = ["full"] }
tempfile = "3.27.0"
[lints]
+3 -2
View File
@@ -14,7 +14,7 @@ path = "tests/main.rs"
[dependencies]
argon2 = "0.5.3"
async-trait = "0.1.89"
base64 = "0.22.1"
base64 = "0.23.0"
chrono = "0.4.45"
futures = "0.3.32"
gateway-core = { path = "../gateway-core" }
@@ -24,10 +24,11 @@ serde = { version = "1.0.228", features = ["derive"] }
secrecy = { version = "0.10.3", features = ["serde"] }
subtle = "2.6.1"
thiserror = "2.0.18"
tracing = "0.1.44"
uuid = { version = "1.23.3", features = ["v7"] }
[dev-dependencies]
serde_json = { version = "1.0.150", features = ["preserve_order"] }
serde_json = { version = "1.0.150", features = ["arbitrary_precision", "preserve_order"] }
tokio = { version = "1.52.3", default-features = false, features = ["macros", "rt"] }
[lints]
@@ -9,17 +9,8 @@ use gateway_core::routing::ProviderKind;
use super::{PageSize, Revision, observability::TimeRange};
/// Provider 账号当前可用性。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AccountAvailability {
Unknown,
Ready,
Cooldown,
QuotaExhausted,
Expired,
Banned,
Invalid,
}
/// Provider 账号当前可用性;权威定义与字符串编解码在 gateway-core,此处仅复用。
pub use gateway_core::engine::credential::AccountAvailability;
/// 管理页使用的归一化账号状态。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
@@ -189,7 +180,6 @@ pub struct AccountSummary {
/// 账号启停写入命令。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SetAccountEnabled {
pub expected_config_revision: Revision,
pub account_id: String,
pub enabled: bool,
}
@@ -197,7 +187,6 @@ pub struct SetAccountEnabled {
/// 账号批量删除命令。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DeleteAccounts {
pub expected_config_revision: Revision,
pub account_ids: Vec<String>,
}
@@ -223,6 +212,9 @@ pub enum AccountConnectionTestEvent {
message: String,
provider_error_code: Option<String>,
provider_error_type: Option<String>,
upstream_status: Option<u16>,
upstream_content_type: Option<String>,
upstream_body: Option<String>,
account_status: AccountStatus,
},
}
@@ -141,7 +141,6 @@ impl fmt::Debug for ClientKeySecret {
/// API 提交的 Client Key 创建命令。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CreateClientKey {
pub expected_config_revision: Revision,
pub name: String,
pub label: Option<String>,
pub provider_kind: ProviderKind,
@@ -151,7 +150,6 @@ pub struct CreateClientKey {
/// 管理用例生成 ID 与明文后的持久化命令。
#[derive(Clone, PartialEq, Eq)]
pub struct NewClientKey {
pub expected_config_revision: Revision,
pub id: ClientApiKeyId,
pub name: String,
pub label: Option<String>,
@@ -175,7 +173,6 @@ impl fmt::Debug for NewClientKey {
/// 修改 Client Key 的公开策略字段。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct UpdateClientKey {
pub expected_config_revision: Revision,
pub id: ClientApiKeyId,
pub name: String,
pub label: Option<String>,
@@ -186,7 +183,6 @@ pub struct UpdateClientKey {
/// 修改 Client Key 启用状态。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SetClientKeyEnabled {
pub expected_config_revision: Revision,
pub id: ClientApiKeyId,
pub enabled: bool,
}
@@ -194,7 +190,6 @@ pub struct SetClientKeyEnabled {
/// 删除 Client Key。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DeleteClientKey {
pub expected_config_revision: Revision,
pub id: ClientApiKeyId,
}
@@ -330,6 +330,7 @@ pub struct LatencyPercentiles {
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ProviderBillingInput {
pub upstream_model_id: String,
pub service_tier: Option<String>,
pub input_tokens: Option<u64>,
pub output_tokens: Option<u64>,
pub cached_tokens: Option<u64>,
@@ -345,6 +346,7 @@ pub struct UsageCalculatedBillingFact {
pub bucket_start: DateTime<Utc>,
pub provider_kind: String,
pub upstream_model_id: String,
pub service_tier: Option<String>,
pub input_tokens: Option<u64>,
pub output_tokens: Option<u64>,
pub cached_tokens: Option<u64>,
@@ -574,6 +576,7 @@ pub struct UsageRecord {
pub upstream_transport: Option<String>,
pub http_version: Option<String>,
pub websocket_pool: Option<String>,
pub service_tier: Option<String>,
/// Provider 已筛选的专有观测 JSON;管理领域保持不透明。
pub provider_metadata_json: Option<String>,
pub attempt_count: u32,
@@ -2,14 +2,15 @@
use std::fmt;
use async_trait::async_trait;
use chrono::{DateTime, Utc};
use gateway_core::{
engine::credential::{OpaqueProviderData, ProviderAccountId},
engine::credential::{OpaqueProviderData, ProviderAccountId, ProviderAccountIdentity},
routing::{ProviderKind, UpstreamModelId},
};
use super::{
MutationActor, MutationContext, PageSize, Revision,
AdminError, MutationActor, MutationContext, PageSize, Revision,
accounts::{AccountAvailability, AccountRecord, AccountStatus, AccountSummary, AccountUsage},
};
@@ -101,7 +102,6 @@ pub struct CredentialDetails {
/// Provider 正式文档批量导入命令。
pub struct ImportCredentials {
pub context: MutationContext,
pub expected_config_revision: Revision,
pub document: ProviderDocument,
}
@@ -110,7 +110,6 @@ impl fmt::Debug for ImportCredentials {
formatter
.debug_struct("ImportCredentials")
.field("context", &self.context)
.field("expected_config_revision", &self.expected_config_revision)
.field("document", &self.document)
.finish()
}
@@ -169,17 +168,9 @@ pub struct PreparedCredentialImport {
/// Admin 交给 Store 的导入事务命令。
#[derive(Debug, Clone, PartialEq)]
pub struct CredentialImportCommit {
pub expected_config_revision: Revision,
pub prepared: PreparedCredentialImport,
}
/// 可选的重新授权目标。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ReauthorizationTarget {
pub account_id: ProviderAccountId,
pub credential_revision: Revision,
}
/// OAuth pending owner 的中立身份;不编码具体 Provider 的 Redis key 或 JSON。
#[derive(Clone, PartialEq, Eq)]
pub enum AuthorizationOwner {
@@ -240,21 +231,18 @@ impl fmt::Debug for AuthorizationOwnerBinding {
}
/// OAuth 完成时应创建新账号还是 CAS 更新既有 credential。
///
/// 重新授权只绑定稳定的账号身份:credential revision 由服务端在临近写入时读取,
/// 长流程期间的后台刷新不得让恢复操作失效。
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum AuthorizationMutationTarget {
Create {
name: String,
},
Reauthorize {
account_id: ProviderAccountId,
expected_credential_revision: Revision,
},
Create { name: String },
Reauthorize { account_id: ProviderAccountId },
}
/// 必须完整进入 Provider opaque pending payload、并在 complete 后原样恢复的事务信封。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PendingAuthorizationMutation {
expected_config_revision: Revision,
provider_kind: ProviderKind,
target: AuthorizationMutationTarget,
owner_binding: AuthorizationOwnerBinding,
@@ -263,24 +251,17 @@ pub struct PendingAuthorizationMutation {
impl PendingAuthorizationMutation {
#[must_use]
pub const fn new(
expected_config_revision: Revision,
provider_kind: ProviderKind,
target: AuthorizationMutationTarget,
owner_binding: AuthorizationOwnerBinding,
) -> Self {
Self {
expected_config_revision,
provider_kind,
target,
owner_binding,
}
}
#[must_use]
pub const fn expected_config_revision(&self) -> Revision {
self.expected_config_revision
}
#[must_use]
pub const fn provider_kind(&self) -> &ProviderKind {
&self.provider_kind
@@ -301,9 +282,8 @@ impl PendingAuthorizationMutation {
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct StartAuthorization {
pub context: MutationContext,
pub expected_config_revision: Revision,
pub name: String,
pub reauthorization: Option<ReauthorizationTarget>,
pub reauthorization: Option<ProviderAccountId>,
}
/// OAuth 流程启动结果。
@@ -322,6 +302,20 @@ pub trait CredentialCommitGuard: Send + 'static {
fn finish(self: Box<Self>);
}
/// OAuth pending claim 在 Store 事务结果确定后执行的显式结算动作。
///
/// OAuth provider 必须在 credential 准备阶段持有 claim:Store 提交成功后消费 flow,提交或
/// 校验失败后释放 flow。异步结算不能依赖 `Drop`,否则请求提前返回会让同一授权流无谓地
/// 进入短暂的不可重试状态。
#[async_trait]
pub trait AuthorizationCommitGuard: Send + 'static {
/// Store 已提交 OAuth credential 后消费对应的一次性 flow。
async fn commit(self: Box<Self>) -> Result<(), AdminError>;
/// Store 未提交 OAuth credential 时释放 claim,允许同一 flow 重试。
async fn abort(self: Box<Self>) -> Result<(), AdminError>;
}
/// OAuth complete 后由 Provider 返回的准备结果;Store 仍是唯一提交者。
pub enum PreparedAuthorizationCredential {
Create(PreparedCredentialCreate),
@@ -332,6 +326,7 @@ pub enum PreparedAuthorizationCredential {
pub struct PreparedAuthorizationCommit {
pub pending: PendingAuthorizationMutation,
pub credential: PreparedAuthorizationCredential,
authorization_guard: Option<Box<dyn AuthorizationCommitGuard>>,
}
impl fmt::Debug for PreparedAuthorizationCommit {
@@ -358,9 +353,37 @@ pub struct AuthorizationCommit {
pub credential: AuthorizationCredentialCommit,
}
/// OAuth 准备结果拆解后的 Store 命令与两个结算 guard。
///
/// Store 成功时先结束 credential guard,再消费 OAuth claim;失败时丢弃 credential guard 并
/// 释放 OAuth claim,使同一授权流可以重试。
pub(crate) struct AuthorizationCommitSettlement {
pub(crate) command: AuthorizationCommit,
pub(crate) credential_guard: Option<Box<dyn CredentialCommitGuard>>,
pub(crate) authorization_guard: Option<Box<dyn AuthorizationCommitGuard>>,
}
impl PreparedAuthorizationCommit {
#[must_use]
pub fn into_commit(self) -> (AuthorizationCommit, Option<Box<dyn CredentialCommitGuard>>) {
pub fn new(
pending: PendingAuthorizationMutation,
credential: PreparedAuthorizationCredential,
) -> Self {
Self {
pending,
credential,
authorization_guard: None,
}
}
/// 让 OAuth claim 覆盖准备完成到 Store 事务结算之间的全部窗口。
#[must_use]
pub fn with_authorization_guard(mut self, guard: Box<dyn AuthorizationCommitGuard>) -> Self {
self.authorization_guard = Some(guard);
self
}
pub(crate) fn into_commit(self) -> AuthorizationCommitSettlement {
let (credential, guard) = match self.credential {
PreparedAuthorizationCredential::Create(credential) => {
(AuthorizationCredentialCommit::Create(credential), None)
@@ -373,13 +396,27 @@ impl PreparedAuthorizationCommit {
)
}
};
(
AuthorizationCommit {
AuthorizationCommitSettlement {
command: AuthorizationCommit {
pending: self.pending,
credential,
},
guard,
)
credential_guard: guard,
authorization_guard: self.authorization_guard,
}
}
pub(crate) async fn abort(self) -> Result<(), AdminError> {
let AuthorizationCommitSettlement {
credential_guard,
authorization_guard,
..
} = self.into_commit();
drop(credential_guard);
if let Some(guard) = authorization_guard {
guard.abort().await?;
}
Ok(())
}
}
@@ -406,7 +443,6 @@ impl fmt::Debug for CompleteAuthorization {
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CredentialMutation {
pub context: MutationContext,
pub expected_config_revision: Revision,
pub account_id: ProviderAccountId,
}
@@ -422,7 +458,6 @@ pub struct CredentialMutationResult {
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CredentialDeletion {
pub context: MutationContext,
pub expected_config_revision: Revision,
pub account_ids: Vec<ProviderAccountId>,
}
@@ -436,14 +471,12 @@ pub struct CredentialDeletionResult {
/// Provider-owned token 轮换命令。
pub struct RotateCredential {
pub mutation: CredentialMutation,
pub expected_credential_revision: Revision,
pub provider_material: ProviderDocument,
}
/// Provider 校验手工轮换材料时所需的非事务输入。
pub struct PrepareCredentialRotation {
pub account: AccountRecord,
pub expected_credential_revision: Revision,
pub provider_material: ProviderDocument,
}
@@ -452,10 +485,6 @@ impl fmt::Debug for PrepareCredentialRotation {
formatter
.debug_struct("PrepareCredentialRotation")
.field("account", &self.account)
.field(
"expected_credential_revision",
&self.expected_credential_revision,
)
.field("provider_material", &self.provider_material)
.finish()
}
@@ -467,6 +496,7 @@ pub struct PreparedCredentialRotationFacts {
pub account_id: ProviderAccountId,
pub provider_kind: ProviderKind,
pub expected_credential_revision: Revision,
pub replacement_identity: Option<ProviderAccountIdentity>,
pub name: String,
pub email: Option<String>,
pub plan_type: Option<String>,
@@ -520,7 +550,6 @@ impl fmt::Debug for PreparedCredentialRotation {
/// Admin 交给 Store 的轮换或 refresh 事务命令。
#[derive(Debug, Clone, PartialEq)]
pub struct CredentialRotationCommit {
pub expected_config_revision: Revision,
pub prepared: PreparedCredentialRotationFacts,
}
@@ -529,10 +558,6 @@ impl fmt::Debug for RotateCredential {
formatter
.debug_struct("RotateCredential")
.field("mutation", &self.mutation)
.field(
"expected_credential_revision",
&self.expected_credential_revision,
)
.field("provider_material", &self.provider_material)
.finish()
}
@@ -39,7 +39,6 @@ pub struct RuntimeSettings {
/// 原子替换运行设置的命令。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ReplaceRuntimeSettings {
pub expected_config_revision: Revision,
pub model_mappings: ModelMappings,
pub refresh_margin_seconds: u64,
pub refresh_concurrency: u32,
@@ -64,6 +64,12 @@ pub trait ProviderAdmin: Send + Sync {
/// 成功之后,不参与事务成败,也不得恢复或改写已经提交的账号状态。
async fn account_unavailable(&self, account_id: &ProviderAccountId);
/// 账号资格事实已经由控制面提交,失效 Provider 持有的可重建派生状态。
///
/// 该通知发生在 Store 事务成功之后、下一份 RuntimeSnapshot 编译之前;通知
/// 不参与已提交事务成败。没有账号派生状态的 Provider 可使用默认空实现。
async fn account_facts_changed(&self, _account_ids: &[ProviderAccountId]) {}
/// 生成一次连接测试所需的 Provider-owned operation;Core 负责实际执行与落账。
fn connection_test_operation(
&self,
@@ -179,7 +179,7 @@ pub trait AuthStore: Send + Sync {
async fn append_audit_event(&self, event: AdminAuditEvent) -> AdminStoreResult<()>;
}
/// Client API Key 管理及其 revision CAS 写入。
/// Client API Key 管理写入。
#[async_trait]
pub trait ClientKeyStore: Send + Sync {
async fn list_client_keys(&self, query: ClientKeyListQuery) -> AdminStoreResult<ClientKeyPage>;
@@ -265,7 +265,7 @@ pub trait ObservabilityStore: Send + Sync {
async fn list_ops_errors(&self, query: OpsErrorQuery) -> AdminStoreResult<OpsErrorPage>;
}
/// Runtime settings 与管理员 API Key 的 revision CAS 写入。
/// Runtime settings 与管理员 API Key 写入。
#[async_trait]
pub trait SettingsStore: Send + Sync {
async fn load_runtime_settings(&self) -> AdminStoreResult<RuntimeSettings>;
@@ -280,14 +280,12 @@ pub trait SettingsStore: Send + Sync {
async fn replace_admin_api_key(
&self,
expected_config_revision: Revision,
key: AdminApiKey,
context: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation>;
async fn delete_admin_api_key(
&self,
expected_config_revision: Revision,
context: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation>;
}
@@ -15,7 +15,7 @@ use gateway_core::{
use crate::{
model::{
AdminError, MutationContext, Revision,
AdminError, MutationContext,
accounts::{
AccountConnectionTestEvent, AccountConnectionTestEventStream, AccountListQuery,
AccountRecord, AccountStatus, AccountUsageWindowQuery,
@@ -54,7 +54,6 @@ pub trait AccountsService: Send + Sync {
async fn refresh(
&self,
context: &MutationContext,
expected_config_revision: Revision,
account_id: ProviderAccountId,
) -> Result<AccountRefreshResult, AdminError>;
@@ -354,7 +353,6 @@ impl AccountsService for DefaultAccountsService {
async fn refresh(
&self,
context: &MutationContext,
expected_config_revision: Revision,
account_id: ProviderAccountId,
) -> Result<AccountRefreshResult, AdminError> {
let (account, provider) = self.provider_for_account(&account_id).await?;
@@ -367,12 +365,14 @@ impl AccountsService for DefaultAccountsService {
validate_prepared_rotation(&account, &prepared, "provider credential refresh")?;
let result = commit_credential_refresh(
self.accounts.as_ref(),
expected_config_revision,
prepared,
context,
"provider credential refresh",
)
.await?;
provider
.account_facts_changed(std::slice::from_ref(&result.account_id))
.await;
publish_committed(self.snapshot.as_ref(), result.config_revision).await?;
let account = self.load_directory_item(&result.account_id, false).await?;
Ok(AccountRefreshResult {
@@ -452,12 +452,33 @@ impl AccountsService for DefaultAccountsService {
account_status: status,
}))
.collect(),
Err(error) => vec![AccountConnectionTestEvent::Failed {
message: error.client_message().to_owned(),
provider_error_code: error.client_error_code().map(ToOwned::to_owned),
provider_error_type: error.client_error_type().map(ToOwned::to_owned),
account_status: status,
}],
Err(error) => {
let upstream_status = error
.upstream_response()
.map(gateway_core::engine::probe::AccountProbeUpstreamResponse::status);
let upstream_content_type = error
.upstream_response()
.and_then(|response| response.content_type())
.and_then(|value| std::str::from_utf8(value).ok())
.map(ToOwned::to_owned);
let upstream_body = error
.upstream_response()
.map(|response| String::from_utf8_lossy(response.body()).into_owned());
let message = upstream_body
.as_deref()
.filter(|body| !body.is_empty())
.unwrap_or_else(|| error.client_message())
.to_owned();
vec![AccountConnectionTestEvent::Failed {
message,
provider_error_code: error.client_error_code().map(ToOwned::to_owned),
provider_error_type: error.client_error_type().map(ToOwned::to_owned),
upstream_status,
upstream_content_type,
upstream_body,
account_status: status,
}]
}
}
})
.flat_map(futures::stream::iter);
@@ -468,22 +489,25 @@ impl AccountsService for DefaultAccountsService {
fn account_status(account: &AccountRecord, now: chrono::DateTime<Utc>) -> AccountStatus {
if !account.enabled {
AccountStatus::Disabled
} else if account.availability == crate::model::accounts::AccountAvailability::Banned {
AccountStatus::Banned
} else if account.availability == crate::model::accounts::AccountAvailability::QuotaExhausted
|| (account.availability == crate::model::accounts::AccountAvailability::Cooldown
&& account.availability_reason.as_deref() == Some("usage_limit_exhausted")
&& account.cooldown_until.is_none_or(|until| until > now))
{
AccountStatus::QuotaExhausted
} else if matches!(
account.availability,
crate::model::accounts::AccountAvailability::Expired
| crate::model::accounts::AccountAvailability::Invalid
) || account
.access_token_expires_at
.is_some_and(|expires_at| expires_at <= now)
{
AccountStatus::Expired
} else {
match account.availability {
crate::model::accounts::AccountAvailability::Banned => AccountStatus::Banned,
crate::model::accounts::AccountAvailability::QuotaExhausted => {
AccountStatus::QuotaExhausted
}
crate::model::accounts::AccountAvailability::Expired
| crate::model::accounts::AccountAvailability::Invalid => AccountStatus::Expired,
_ if account
.access_token_expires_at
.is_some_and(|expires_at| expires_at <= now) =>
{
AccountStatus::Expired
}
_ => AccountStatus::Active,
}
AccountStatus::Active
}
}
@@ -94,7 +94,6 @@ impl ClientKeyService for DefaultClientKeyService {
.store
.create_client_key(
NewClientKey {
expected_config_revision: command.expected_config_revision,
id,
name: command.name,
label: command.label,
@@ -90,17 +90,10 @@ async fn pending_authorization(
resource: &'static str,
) -> Result<PendingAuthorizationMutation, AdminError> {
let target = match &command.reauthorization {
Some(target) => {
let details =
required_credential(accounts, provider_kind, &target.account_id, resource).await?;
if details.credential.credential_revision != target.credential_revision {
return Err(AdminError::conflict(format!(
"{resource} reauthorization target is stale"
)));
}
Some(account_id) => {
required_credential(accounts, provider_kind, account_id, resource).await?;
AuthorizationMutationTarget::Reauthorize {
account_id: target.account_id.clone(),
expected_credential_revision: target.credential_revision,
account_id: account_id.clone(),
}
}
None => AuthorizationMutationTarget::Create {
@@ -108,7 +101,6 @@ async fn pending_authorization(
},
};
Ok(PendingAuthorizationMutation::new(
command.expected_config_revision,
provider_kind.clone(),
target,
crate::model::provider_credentials::AuthorizationOwnerBinding::from_context(
@@ -143,7 +135,6 @@ fn validate_prepared_rotation(
let facts = prepared.facts();
if facts.account_id.as_str() != account.id.as_str()
|| facts.provider_kind != account.provider_kind
|| facts.expected_credential_revision != account.credential_revision
{
return Err(AdminError::conflict(format!(
"{resource} prepared facts do not match the current credential"
@@ -152,18 +143,27 @@ fn validate_prepared_rotation(
Ok(())
}
fn validate_authorization_commit(
async fn validate_authorization_commit(
provider_kind: &ProviderKind,
context: &MutationContext,
prepared: &PreparedAuthorizationCommit,
prepared: PreparedAuthorizationCommit,
resource: &'static str,
) -> Result<(), AdminError> {
) -> Result<PreparedAuthorizationCommit, AdminError> {
if prepared.pending.provider_kind() != provider_kind
|| !prepared.pending.owner_binding().matches_context(context)
{
return Err(AdminError::conflict(format!(
let validation_error = AdminError::conflict(format!(
"{resource} pending authorization binding is invalid"
)));
));
if let Err(error) = prepared.abort().await {
tracing::warn!(
resource,
settlement_error = %error,
"OAuth authorization claim release failed after preparation validation"
);
return Err(error);
}
return Err(validation_error);
}
let matches_target = match (prepared.pending.target(), &prepared.credential) {
(
@@ -171,25 +171,29 @@ fn validate_authorization_commit(
PreparedAuthorizationCredential::Create(credential),
) => credential.provider_kind == *provider_kind,
(
AuthorizationMutationTarget::Reauthorize {
account_id,
expected_credential_revision,
},
AuthorizationMutationTarget::Reauthorize { account_id },
PreparedAuthorizationCredential::Reauthorize(credential),
) => {
let facts = credential.facts();
facts.provider_kind == *provider_kind
&& facts.account_id == *account_id
&& facts.expected_credential_revision == *expected_credential_revision
facts.provider_kind == *provider_kind && facts.account_id == *account_id
}
_ => false,
};
if !matches_target {
return Err(AdminError::conflict(format!(
let validation_error = AdminError::conflict(format!(
"{resource} prepared credential does not match its pending target"
)));
));
if let Err(error) = prepared.abort().await {
tracing::warn!(
resource,
settlement_error = %error,
"OAuth authorization claim release failed after preparation validation"
);
return Err(error);
}
return Err(validation_error);
}
Ok(())
Ok(prepared)
}
async fn commit_authorization(
@@ -198,16 +202,34 @@ async fn commit_authorization(
context: &MutationContext,
resource: &'static str,
) -> Result<CredentialMutationResult, AdminError> {
let (command, guard) = prepared.into_commit();
let crate::model::provider_credentials::AuthorizationCommitSettlement {
command,
credential_guard,
authorization_guard,
} = prepared.into_commit();
match accounts.commit_authorization(command, context).await {
Ok(result) => {
if let Some(guard) = guard {
if let Some(guard) = credential_guard {
guard.finish();
}
if let Some(guard) = authorization_guard {
guard.commit().await?;
}
Ok(result)
}
Err(error) => {
drop(guard);
drop(credential_guard);
if let Some(guard) = authorization_guard
&& let Err(settlement_error) = guard.abort().await
{
tracing::warn!(
resource,
store_error = %error,
settlement_error = %settlement_error,
"OAuth authorization claim release failed after Store commit failure"
);
return Err(settlement_error);
}
Err(map_store_error(error, resource))
}
}
@@ -215,20 +237,13 @@ async fn commit_authorization(
async fn commit_credential_rotation(
accounts: &dyn AccountStore,
expected_config_revision: crate::model::Revision,
prepared: PreparedCredentialRotation,
context: &MutationContext,
resource: &'static str,
) -> Result<CredentialMutationResult, AdminError> {
let (facts, guard) = prepared.into_parts();
match accounts
.commit_credential_rotation(
CredentialRotationCommit {
expected_config_revision,
prepared: facts,
},
context,
)
.commit_credential_rotation(CredentialRotationCommit { prepared: facts }, context)
.await
{
Ok(result) => {
@@ -244,20 +259,13 @@ async fn commit_credential_rotation(
async fn commit_credential_refresh(
accounts: &dyn AccountStore,
expected_config_revision: crate::model::Revision,
prepared: PreparedCredentialRotation,
context: &MutationContext,
resource: &'static str,
) -> Result<CredentialMutationResult, AdminError> {
let (facts, guard) = prepared.into_parts();
match accounts
.commit_credential_refresh(
CredentialRotationCommit {
expected_config_revision,
prepared: facts,
},
context,
)
.commit_credential_refresh(CredentialRotationCommit { prepared: facts }, context)
.await
{
Ok(result) => {
@@ -289,7 +297,6 @@ async fn set_credential_enabled(
let revision = accounts
.set_account_enabled(
SetAccountEnabled {
expected_config_revision: command.expected_config_revision,
account_id: account_id.as_str().to_owned(),
enabled,
},
@@ -300,6 +307,9 @@ async fn set_credential_enabled(
if !enabled {
provider.account_unavailable(&account_id).await;
}
provider
.account_facts_changed(std::slice::from_ref(&account_id))
.await;
Ok(CredentialMutationResult {
config_revision: revision,
account_id,
@@ -320,7 +330,6 @@ async fn delete_credentials(
let revision = accounts
.delete_accounts(
DeleteAccounts {
expected_config_revision: command.expected_config_revision,
account_ids: account_ids
.iter()
.map(|account_id| account_id.as_str().to_owned())
@@ -333,6 +342,7 @@ async fn delete_credentials(
for account_id in &account_ids {
provider.account_unavailable(account_id).await;
}
provider.account_facts_changed(&account_ids).await;
Ok(CredentialDeletionResult {
config_revision: revision,
account_ids,
@@ -428,6 +428,7 @@ fn recover_standard_costs(
};
let input = ProviderBillingInput {
upstream_model_id: fact.upstream_model_id,
service_tier: fact.service_tier,
input_tokens: fact.input_tokens,
output_tokens: fact.output_tokens,
cached_tokens: fact.cached_tokens,
@@ -491,14 +492,14 @@ impl DefaultObservabilityService {
}
}
/// 逐条尽力把 calculated 总额升级为完整分解;单条脏数据(非法 Provider kind
/// 或费用规则失败)只保留该条已存的总额,不影响整页返回。
/// 逐条尽力把可校验的总额升级为完整分解;单条脏数据(非法 Provider kind、
/// 不支持的来源或费用规则失败)只保留该条已存的总额,不影响整页返回。
fn enrich_billing(&self, records: &mut [crate::model::observability::UsageRecord]) {
for record in records {
let Some(UsageBilling::Total { source, total }) = record.billing.as_ref() else {
continue;
};
if source != "calculated" {
if !matches!(source.as_str(), "calculated" | "provider_reported") {
continue;
}
let (Some(provider), Some(upstream_model_id)) = (
@@ -512,6 +513,7 @@ impl DefaultObservabilityService {
};
let input = ProviderBillingInput {
upstream_model_id: upstream_model_id.clone(),
service_tier: record.service_tier.clone(),
input_tokens: record.input_tokens,
output_tokens: record.output_tokens,
cached_tokens: record.cached_tokens,
@@ -83,6 +83,27 @@ impl DefaultOpenAiService {
snapshot,
}
}
async fn observe_initial_quotas(&self, account_ids: &[ProviderAccountId], request_id: &str) {
for account_id in account_ids {
if let Err(error) = self
.provider
.quota(ProviderQuotaRequest {
account_id: account_id.clone(),
refresh: true,
rolling_usage: None,
})
.await
{
tracing::warn!(
request_id,
account_id = %account_id.as_str(),
quota_error = ?error.kind(),
"OpenAI initial quota observation failed"
);
}
}
}
}
#[async_trait]
@@ -112,7 +133,6 @@ impl OpenAiService for DefaultOpenAiService {
command: ImportCredentials,
) -> Result<CredentialImportResult, AdminError> {
let context = command.context;
let expected_config_revision = command.expected_config_revision;
let prepared = self
.provider
.prepare_import(PrepareCredentialImport {
@@ -127,27 +147,16 @@ impl OpenAiService for DefaultOpenAiService {
)?;
let result = self
.accounts
.commit_credential_import(
CredentialImportCommit {
expected_config_revision,
prepared,
},
&context,
)
.commit_credential_import(CredentialImportCommit { prepared }, &context)
.await
.map_err(|error| map_store_error(error, "OpenAI credential import"))?;
self.provider
.account_facts_changed(&result.credential_ids)
.await;
publish_committed(self.snapshot.as_ref(), result.config_revision).await?;
// 导入已经提交,额度属于可重建观察事实;单个账号查询失败由周期任务重试。
for account_id in &result.credential_ids {
let _ = self
.provider
.quota(ProviderQuotaRequest {
account_id: account_id.clone(),
refresh: true,
rolling_usage: None,
})
.await;
}
// 导入完成后立即做一次观察;失败不回滚已经提交的 credential。
self.observe_initial_quotas(&result.credential_ids, &context.request_id)
.await;
Ok(result)
}
@@ -178,12 +187,13 @@ impl OpenAiService for DefaultOpenAiService {
.complete_authorization(command)
.await
.map_err(|error| map_provider_error(error, "OpenAI authorization"))?;
validate_authorization_commit(
let prepared = validate_authorization_commit(
self.provider.provider_kind(),
&context,
&prepared,
prepared,
"OpenAI authorization",
)?;
)
.await?;
let result = commit_authorization(
self.accounts.as_ref(),
prepared,
@@ -191,7 +201,15 @@ impl OpenAiService for DefaultOpenAiService {
"OpenAI authorization",
)
.await?;
self.provider
.account_facts_changed(std::slice::from_ref(&result.account_id))
.await;
publish_committed(self.snapshot.as_ref(), result.config_revision).await?;
self.observe_initial_quotas(
std::slice::from_ref(&result.account_id),
&context.request_id,
)
.await;
Ok(result)
}
@@ -200,7 +218,6 @@ impl OpenAiService for DefaultOpenAiService {
command: RotateCredential,
) -> Result<CredentialMutationResult, AdminError> {
let context = command.mutation.context;
let expected_config_revision = command.mutation.expected_config_revision;
let account_id = command.mutation.account_id;
let details = required_credential(
self.accounts.as_ref(),
@@ -209,17 +226,11 @@ impl OpenAiService for DefaultOpenAiService {
"OpenAI credential rotation",
)
.await?;
if details.credential.credential_revision != command.expected_credential_revision {
return Err(AdminError::conflict(
"OpenAI credential rotation revision is stale",
));
}
let account = details.credential;
let prepared = self
.provider
.prepare_rotation(PrepareCredentialRotation {
account: account.clone(),
expected_credential_revision: command.expected_credential_revision,
provider_material: command.provider_material,
})
.await
@@ -227,12 +238,14 @@ impl OpenAiService for DefaultOpenAiService {
validate_prepared_rotation(&account, &prepared, "OpenAI credential rotation")?;
let result = commit_credential_rotation(
self.accounts.as_ref(),
expected_config_revision,
prepared,
&context,
"OpenAI credential rotation",
)
.await?;
self.provider
.account_facts_changed(std::slice::from_ref(&result.account_id))
.await;
publish_committed(self.snapshot.as_ref(), result.config_revision).await?;
Ok(result)
}
@@ -86,13 +86,12 @@ impl SettingsService for DefaultSettingsService {
&self,
context: &MutationContext,
) -> Result<RegeneratedAdminApiKey, AdminError> {
let settings = self.load().await?;
let mut bytes = [0_u8; 32];
OsRng.fill_bytes(&mut bytes);
let key = AdminApiKey::new(format!("admin-{}", hex::encode(bytes)));
let mutation = self
.store
.replace_admin_api_key(settings.config_revision, key.clone(), context)
.replace_admin_api_key(key.clone(), context)
.await
.map_err(|error| map_store_error(error, "administrator API key"))?;
publish_committed(self.snapshot.as_ref(), mutation.config_revision).await?;
@@ -103,10 +102,9 @@ impl SettingsService for DefaultSettingsService {
&self,
context: &MutationContext,
) -> Result<AdminApiKeyMutation, AdminError> {
let settings = self.load().await?;
let mutation = self
.store
.delete_admin_api_key(settings.config_revision, context)
.delete_admin_api_key(context)
.await
.map_err(|error| map_store_error(error, "administrator API key"))?;
publish_committed(self.snapshot.as_ref(), mutation.config_revision).await?;
@@ -89,7 +89,6 @@ impl XaiService for DefaultXaiService {
command: ImportCredentials,
) -> Result<CredentialImportResult, AdminError> {
let context = command.context;
let expected_config_revision = command.expected_config_revision;
let prepared = self
.provider
.prepare_import(PrepareCredentialImport {
@@ -104,13 +103,7 @@ impl XaiService for DefaultXaiService {
)?;
let result = self
.accounts
.commit_credential_import(
CredentialImportCommit {
expected_config_revision,
prepared,
},
&context,
)
.commit_credential_import(CredentialImportCommit { prepared }, &context)
.await
.map_err(|error| map_store_error(error, "xAI credential import"))?;
publish_committed(self.snapshot.as_ref(), result.config_revision).await?;
@@ -155,12 +148,13 @@ impl XaiService for DefaultXaiService {
.complete_authorization(command)
.await
.map_err(|error| map_provider_error(error, "xAI authorization"))?;
validate_authorization_commit(
let prepared = validate_authorization_commit(
self.provider.provider_kind(),
&context,
&prepared,
prepared,
"xAI authorization",
)?;
)
.await?;
let result = commit_authorization(
self.accounts.as_ref(),
prepared,
@@ -6,7 +6,7 @@ use futures::{StreamExt as _, future::BoxFuture};
use gateway_core::{
engine::{
credential::{OpaqueProviderData, ProviderAccountId},
probe::{AccountProbe, AccountProbeRequest, AccountProbeResult},
probe::{AccountProbe, AccountProbeError, AccountProbeRequest, AccountProbeResult},
},
error::{ClientVisibleUpstreamError, GatewayError, GatewayErrorKind},
operation::{GenerateRequest, Operation, ProtocolPayload},
@@ -16,7 +16,7 @@ use gateway_core::{
use gateway_admin::{
AdminServices,
model::{
MutationContext, Revision,
AdminError, MutationContext, Revision,
accounts::{
AccountAvailability, AccountConnectionTestEvent, AccountListQuery, AccountPage,
AccountRecord, AccountSummary, AccountUsage, AccountUsageWindowQuery,
@@ -24,12 +24,12 @@ use gateway_admin::{
},
observability::TimeRange,
provider_credentials::{
AuthorizationCommit, AuthorizationCredentialCommit, AuthorizationMutationTarget,
AuthorizationStarted, CompleteAuthorization, CredentialCommitGuard, CredentialDetails,
CredentialImportCommit, CredentialImportResult, CredentialListQuery,
CredentialMutationResult, CredentialPage, CredentialRotationCommit,
PendingAuthorizationMutation, PrepareCredentialImport, PrepareCredentialRefresh,
PrepareCredentialRotation, PreparedAuthorizationCommit,
AuthorizationCommit, AuthorizationCommitGuard, AuthorizationCredentialCommit,
AuthorizationMutationTarget, AuthorizationStarted, CompleteAuthorization,
CredentialCommitGuard, CredentialDetails, CredentialImportCommit,
CredentialImportResult, CredentialListQuery, CredentialMutationResult, CredentialPage,
CredentialRotationCommit, PendingAuthorizationMutation, PrepareCredentialImport,
PrepareCredentialRefresh, PrepareCredentialRotation, PreparedAuthorizationCommit,
PreparedAuthorizationCredential, PreparedCredentialCreate, PreparedCredentialImport,
PreparedCredentialRotation, PreparedCredentialRotationFacts, ProviderDocument,
ProviderExport, ProviderExportCredentialInput, ProviderModels, ProviderQuota,
@@ -58,11 +58,13 @@ pub(super) struct FakeProviderAdmin {
events: EventLog,
failure: Mutex<Option<ProviderAdminErrorKind>>,
quota_failure: Mutex<Option<ProviderAdminErrorKind>>,
pending: Mutex<Option<PendingAuthorizationMutation>>,
pending: Arc<Mutex<Option<PendingAuthorizationMutation>>>,
retry_authorization_after_abort: Mutex<bool>,
export_inputs: Mutex<Vec<ProviderExportCredentialInput>>,
import_account_ids: Mutex<Vec<String>>,
quota_requests: Mutex<Vec<ProviderQuotaRequest>>,
quota: Mutex<ProviderQuota>,
current_credential_revision: Mutex<Revision>,
}
impl FakeProviderAdmin {
@@ -72,11 +74,13 @@ impl FakeProviderAdmin {
events,
failure: Mutex::new(None),
quota_failure: Mutex::new(None),
pending: Mutex::new(None),
pending: Arc::new(Mutex::new(None)),
retry_authorization_after_abort: Mutex::new(false),
export_inputs: Mutex::new(Vec::new()),
import_account_ids: Mutex::new(vec!["acct_prepared".to_owned()]),
quota_requests: Mutex::new(Vec::new()),
quota: Mutex::new(empty_quota()),
current_credential_revision: Mutex::new(revision(1)),
})
}
@@ -113,6 +117,20 @@ impl FakeProviderAdmin {
self.pending.lock().expect("pending authorization").clone()
}
pub(super) fn retry_authorization_after_abort(&self) {
*self
.retry_authorization_after_abort
.lock()
.expect("authorization retry") = true;
}
pub(super) fn set_current_credential_revision(&self, revision: Revision) {
*self
.current_credential_revision
.lock()
.expect("current credential revision") = revision;
}
fn export_inputs(&self) -> Vec<ProviderExportCredentialInput> {
self.export_inputs.lock().expect("export inputs").clone()
}
@@ -133,7 +151,11 @@ impl FakeProviderAdmin {
PreparedCredentialRotationFacts {
account_id: ProviderAccountId::new(account.id.clone()).expect("account ID"),
provider_kind: account.provider_kind.clone(),
expected_credential_revision: account.credential_revision,
expected_credential_revision: *self
.current_credential_revision
.lock()
.expect("current credential revision"),
replacement_identity: None,
name: account.name.clone(),
email: account.email.clone(),
plan_type: account.plan_type.clone(),
@@ -159,6 +181,10 @@ impl ProviderAdmin for FakeProviderAdmin {
self.record("provider.account_unavailable");
}
async fn account_facts_changed(&self, _: &[ProviderAccountId]) {
self.record("provider.account_facts_changed");
}
fn connection_test_operation(
&self,
model: &gateway_core::routing::UpstreamModelId,
@@ -246,23 +272,31 @@ impl ProviderAdmin for FakeProviderAdmin {
if !pending.owner_binding().matches_context(&command.context) {
return Err(ProviderAdminError::new(ProviderAdminErrorKind::NotFound));
}
let retry_pending = (*self
.retry_authorization_after_abort
.lock()
.expect("authorization retry"))
.then(|| pending.clone());
let credential = match pending.target() {
AuthorizationMutationTarget::Create { name } => {
PreparedAuthorizationCredential::Create(prepared_create(self.kind.clone(), name))
}
AuthorizationMutationTarget::Reauthorize {
account_id,
expected_credential_revision,
} => {
AuthorizationMutationTarget::Reauthorize { account_id } => {
let mut account = account_record(self.kind.as_str());
account.id = account_id.as_str().to_owned();
account.credential_revision = *expected_credential_revision;
PreparedAuthorizationCredential::Reauthorize(self.prepared_rotation(&account))
}
};
Ok(PreparedAuthorizationCommit {
pending,
credential,
let prepared = PreparedAuthorizationCommit::new(pending, credential);
Ok(match retry_pending {
Some(pending) => {
prepared.with_authorization_guard(Box::new(RetryableAuthorizationGuard::new(
self.events.clone(),
Arc::clone(&self.pending),
pending,
)))
}
None => prepared,
})
}
@@ -342,18 +376,20 @@ pub(super) struct FakeAccountStore {
account: AccountRecord,
fail_commit: Mutex<bool>,
audit_requests: Mutex<Vec<String>>,
authorization_revision: Mutex<Option<Revision>>,
quota_window_usage: Mutex<Vec<AccountUsageWindowResult>>,
}
impl FakeAccountStore {
pub(super) fn new(kind: &str, events: EventLog) -> Arc<Self> {
Self::with_account(account_record(kind), events)
}
fn with_account(account: AccountRecord, events: EventLog) -> Arc<Self> {
Arc::new(Self {
events,
account: account_record(kind),
account,
fail_commit: Mutex::new(false),
audit_requests: Mutex::new(Vec::new()),
authorization_revision: Mutex::new(None),
quota_window_usage: Mutex::new(Vec::new()),
})
}
@@ -366,13 +402,6 @@ impl FakeAccountStore {
self.audit_requests.lock().expect("audit requests").clone()
}
pub(super) fn authorization_revision(&self) -> Option<Revision> {
*self
.authorization_revision
.lock()
.expect("authorization revision")
}
pub(super) fn set_quota_window_usage(&self, usage: Vec<AccountUsageWindowResult>) {
*self.quota_window_usage.lock().expect("quota window usage") = usage;
}
@@ -519,10 +548,6 @@ impl AccountStore for FakeAccountStore {
) -> AdminStoreResult<CredentialMutationResult> {
self.record("store.commit_authorization");
self.record_context(context);
*self
.authorization_revision
.lock()
.expect("authorization revision") = Some(command.pending.expected_config_revision());
self.require_commit()?;
let (account_id, credential_revision) = match command.credential {
AuthorizationCredentialCommit::Create(credential) => (credential.account_id, None),
@@ -627,7 +652,6 @@ impl SettingsStore for StaticSettingsStore {
async fn replace_admin_api_key(
&self,
_: Revision,
_: AdminApiKey,
_: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation> {
@@ -636,7 +660,6 @@ impl SettingsStore for StaticSettingsStore {
async fn delete_admin_api_key(
&self,
_: Revision,
_: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation> {
Err(store_unavailable())
@@ -675,6 +698,51 @@ impl Drop for RecordingGuard {
}
}
struct RetryableAuthorizationGuard {
events: EventLog,
pending: Arc<Mutex<Option<PendingAuthorizationMutation>>>,
authorization: PendingAuthorizationMutation,
}
impl RetryableAuthorizationGuard {
fn new(
events: EventLog,
pending: Arc<Mutex<Option<PendingAuthorizationMutation>>>,
authorization: PendingAuthorizationMutation,
) -> Self {
Self {
events,
pending,
authorization,
}
}
}
#[async_trait]
impl AuthorizationCommitGuard for RetryableAuthorizationGuard {
async fn commit(self: Box<Self>) -> Result<(), AdminError> {
self.events
.lock()
.expect("authorization guard events")
.push("authorization_guard.commit");
Ok(())
}
async fn abort(self: Box<Self>) -> Result<(), AdminError> {
let Self {
events,
pending,
authorization,
} = *self;
*pending.lock().expect("pending authorization") = Some(authorization);
events
.lock()
.expect("authorization guard events")
.push("authorization_guard.abort");
Ok(())
}
}
#[test]
fn provider_registry_should_resolve_custom_kind_without_central_match() {
let provider = FakeProviderAdmin::new("custom-provider", events());
@@ -701,9 +769,11 @@ fn provider_registry_should_reject_duplicate_kind() {
}
#[tokio::test]
async fn connection_test_should_preserve_safe_provider_error_details() {
async fn connection_test_should_probe_unavailable_account() {
let provider = FakeProviderAdmin::new("xai", events());
let store = FakeAccountStore::new("xai", events());
let mut account = account_record("xai");
account.availability = AccountAvailability::QuotaExhausted;
let store = FakeAccountStore::with_account(account, events());
let services =
accounts_service_with_probe(provider, store, Arc::new(FailingAccountProbe)).await;
@@ -724,6 +794,7 @@ async fn connection_test_should_preserve_safe_provider_error_details() {
message,
provider_error_code: Some(code),
provider_error_type: Some(error_type),
account_status: gateway_admin::model::accounts::AccountStatus::QuotaExhausted,
..
}) if message == "included usage exhausted"
&& code == "usage_exhausted"
@@ -731,6 +802,35 @@ async fn connection_test_should_preserve_safe_provider_error_details() {
));
}
#[tokio::test]
async fn connection_test_should_preserve_disabled_account_status() {
let provider = FakeProviderAdmin::new("xai", events());
let mut account = account_record("xai");
account.enabled = false;
let store = FakeAccountStore::with_account(account, events());
let services =
accounts_service_with_probe(provider, store, Arc::new(FailingAccountProbe)).await;
let events = services
.accounts()
.test_connection(
ProviderAccountId::new("acct_test").expect("account ID"),
gateway_core::routing::UpstreamModelId::new("grok-4.5").expect("model"),
)
.await
.expect("connection test stream")
.collect::<Vec<_>>()
.await;
assert!(matches!(
events.last(),
Some(AccountConnectionTestEvent::Failed {
account_status: gateway_admin::model::accounts::AccountStatus::Disabled,
..
})
));
}
#[tokio::test]
async fn accounts_export_should_pass_store_loaded_timestamps_and_material_to_provider() {
let provider = FakeProviderAdmin::new("openai", events());
@@ -769,7 +869,6 @@ async fn accounts_refresh_should_keep_guard_through_store_commit() {
.accounts()
.refresh(
&context("refresh-request"),
revision(1),
ProviderAccountId::new("acct_test").expect("account ID"),
)
.await
@@ -788,6 +887,7 @@ async fn accounts_refresh_should_keep_guard_through_store_commit() {
"provider.prepare_refresh",
"store.commit_refresh",
"guard.finish",
"provider.account_facts_changed",
"store.load_account",
]
);
@@ -822,6 +922,36 @@ async fn accounts_list_should_return_complete_directory_semantics() {
);
}
#[tokio::test]
async fn accounts_list_should_prefer_usage_limit_exhaustion_over_stale_token_expiry() {
let provider = FakeProviderAdmin::new("openai", events());
let mut account = account_record("openai");
account.availability = AccountAvailability::Cooldown;
account.availability_reason = Some("usage_limit_exhausted".to_owned());
account.cooldown_until = Some(Utc::now() + TimeDelta::minutes(5));
account.access_token_expires_at = Some(Utc::now() - TimeDelta::minutes(5));
let store = FakeAccountStore::with_account(account, events());
let page = accounts_service(provider, store)
.await
.accounts()
.list(AccountListQuery {
page: 1,
page_size: gateway_admin::model::PageSize::new(20).expect("page size"),
provider_kind: None,
search: None,
status: None,
sort: None,
})
.await
.expect("usage-limited account directory");
assert_eq!(
page.items.first().expect("account item").status,
gateway_admin::model::accounts::AccountStatus::QuotaExhausted,
);
}
#[tokio::test]
async fn accounts_list_should_attach_local_usage_to_quota_windows() {
let provider = FakeProviderAdmin::new("openai", events());
@@ -882,7 +1012,6 @@ async fn accounts_refresh_provider_failure_should_not_call_store_commit() {
.accounts()
.refresh(
&context("refresh-provider-error"),
revision(1),
ProviderAccountId::new("acct_test").expect("account ID"),
)
.await
@@ -895,7 +1024,7 @@ async fn accounts_refresh_provider_failure_should_not_call_store_commit() {
}
#[tokio::test]
async fn accounts_refresh_store_failure_should_drop_guard_after_cas_attempt() {
async fn accounts_refresh_store_failure_should_drop_guard_after_commit_attempt() {
let events = events();
let provider = FakeProviderAdmin::new("openai", events.clone());
let store = FakeAccountStore::new("openai", events.clone());
@@ -906,11 +1035,10 @@ async fn accounts_refresh_store_failure_should_drop_guard_after_cas_attempt() {
.accounts()
.refresh(
&context("refresh-store-error"),
revision(1),
ProviderAccountId::new("acct_test").expect("account ID"),
)
.await
.expect_err("Store CAS must fail");
.expect_err("Store commit must fail");
assert_eq!(
recorded(&events),
@@ -1039,7 +1167,7 @@ fn prepared_create_with_id(
fn rotation_result(command: CredentialRotationCommit) -> CredentialMutationResult {
CredentialMutationResult {
config_revision: revision(command.expected_config_revision.get() + 1),
config_revision: revision(2),
account_id: command.prepared.account_id,
credential_revision: Some(revision(
command.prepared.expected_credential_revision.get() + 1,
@@ -1074,7 +1202,7 @@ impl AccountProbe for SuccessfulAccountProbe {
fn probe(
&self,
_: AccountProbeRequest,
) -> BoxFuture<'_, Result<AccountProbeResult, GatewayError>> {
) -> BoxFuture<'_, Result<AccountProbeResult, AccountProbeError>> {
Box::pin(async {
Ok(AccountProbeResult {
text: vec!["OK".to_owned()],
@@ -1089,19 +1217,19 @@ impl AccountProbe for FailingAccountProbe {
fn probe(
&self,
_: AccountProbeRequest,
) -> BoxFuture<'_, Result<AccountProbeResult, GatewayError>> {
) -> BoxFuture<'_, Result<AccountProbeResult, AccountProbeError>> {
Box::pin(async {
let detail = ClientVisibleUpstreamError::new(
"included usage exhausted",
Some("usage_exhausted".to_owned()),
Some("invalid_request_error".to_owned()),
)
.expect("safe provider error");
);
Err(GatewayError::new(
GatewayErrorKind::RateLimited,
"upstream capacity is temporarily unavailable",
)
.with_client_visible_upstream_error(detail))
.with_client_visible_upstream_error(detail)
.into())
})
}
}
@@ -60,7 +60,7 @@ use gateway_admin::{
use gateway_core::{
engine::{
credential::ProviderAccountId,
probe::{AccountProbe, AccountProbeRequest, AccountProbeResult},
probe::{AccountProbe, AccountProbeError, AccountProbeRequest, AccountProbeResult},
},
error::{GatewayError, GatewayErrorKind},
policy::ClientApiKeyId,
@@ -449,7 +449,6 @@ impl SettingsStore for UnavailableStore {
async fn replace_admin_api_key(
&self,
_: Revision,
_: AdminApiKey,
_: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation> {
@@ -458,7 +457,6 @@ impl SettingsStore for UnavailableStore {
async fn delete_admin_api_key(
&self,
_: Revision,
_: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation> {
Err(unavailable("admin API key"))
@@ -649,12 +647,13 @@ impl AccountProbe for UnavailableProbe {
fn probe(
&self,
_: AccountProbeRequest,
) -> BoxFuture<'_, Result<AccountProbeResult, GatewayError>> {
) -> BoxFuture<'_, Result<AccountProbeResult, AccountProbeError>> {
Box::pin(async {
Err(GatewayError::new(
GatewayErrorKind::Internal,
"test account probe is unavailable",
))
)
.into())
})
}
}
@@ -249,6 +249,7 @@ async fn observability_services_should_calculate_usage_insights_and_diagnostic_s
bucket_start: quarter_hour_start(now),
provider_kind: "openai".to_owned(),
upstream_model_id: "gpt-5.5".to_owned(),
service_tier: None,
input_tokens: Some(800),
output_tokens: Some(200),
cached_tokens: Some(0),
@@ -326,9 +327,19 @@ async fn usage_records_should_tolerate_records_that_fail_billing_enrichment() {
let store = Arc::new(FixtureObservabilityStore::new(observation_range(now)));
let invalid_kind = "x".repeat(65);
store.replace_usage_records(vec![
calculated_total_record("request_invalid_kind", Some(&invalid_kind), now),
calculated_total_record("request_unregistered_kind", Some("anthropic"), now),
calculated_total_record("request_enriched", Some("openai"), now),
total_record(
"request_invalid_kind",
Some(&invalid_kind),
"calculated",
now,
),
total_record(
"request_unregistered_kind",
Some("anthropic"),
"calculated",
now,
),
total_record("request_enriched", Some("openai"), "calculated", now),
]);
let services = observability_services_with_calculated_billing(store).await;
@@ -353,6 +364,30 @@ async fn usage_records_should_tolerate_records_that_fail_billing_enrichment() {
);
}
#[tokio::test]
async fn usage_records_should_enrich_provider_reported_totals_when_pricing_matches() {
let now = Utc::now();
let store = Arc::new(FixtureObservabilityStore::new(observation_range(now)));
store.replace_usage_records(vec![total_record(
"request_provider_reported",
Some("openai"),
"provider_reported",
now,
)]);
let services = observability_services_with_calculated_billing(store).await;
let page = services
.observability()
.usage_records(usage_query(now))
.await
.expect("usage records");
assert!(matches!(
page.items[0].billing,
Some(UsageBilling::Calculated(_))
));
}
struct FixtureObservabilityStore {
trend: Mutex<Vec<RequestMetricPoint>>,
overview: Mutex<UsageOverview>,
@@ -518,7 +553,6 @@ impl SettingsStore for FixtureSettingsStore {
async fn replace_admin_api_key(
&self,
_: Revision,
_: AdminApiKey,
_: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation> {
@@ -527,7 +561,6 @@ impl SettingsStore for FixtureSettingsStore {
async fn delete_admin_api_key(
&self,
_: Revision,
_: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation> {
Err(super::unavailable("admin API key"))
@@ -568,9 +601,10 @@ fn usage_query(now: DateTime<Utc>) -> UsageQuery {
}
}
fn calculated_total_record(
fn total_record(
id: &str,
provider_kind: Option<&str>,
source: &str,
now: DateTime<Utc>,
) -> UsageRecord {
UsageRecord {
@@ -591,6 +625,7 @@ fn calculated_total_record(
upstream_transport: None,
http_version: None,
websocket_pool: None,
service_tier: None,
provider_metadata_json: None,
attempt_count: 1,
upstream_send_state: "sent".to_owned(),
@@ -613,11 +648,11 @@ fn calculated_total_record(
image_input_tokens: None,
image_output_tokens: None,
total_tokens: Some(1_000),
cost_source: "calculated".to_owned(),
cost_source: source.to_owned(),
cost_amount: None,
cost_currency: None,
billing: Some(UsageBilling::Total {
source: "calculated".to_owned(),
source: source.to_owned(),
total: CurrencyCost {
currency: "USD".to_owned(),
amount: gateway_admin::model::observability::DecimalAmount::from_str("1.25")
@@ -6,7 +6,7 @@ use gateway_admin::{
AdminServices,
model::provider_credentials::{
AuthorizationMutationTarget, CompleteAuthorization, CredentialDeletion, CredentialMutation,
ImportCredentials, ProviderQuotaRequest, ReauthorizationTarget, StartAuthorization,
ImportCredentials, ProviderQuotaRequest, StartAuthorization,
},
ports::provider::ProviderAdminErrorKind,
};
@@ -30,7 +30,11 @@ async fn openai_enable_should_use_scoped_store_without_provider_mutation() {
assert_eq!(
recorded(&events),
["store.credential_details", "store.set_enabled"]
[
"store.credential_details",
"store.set_enabled",
"provider.account_facts_changed",
]
);
assert_eq!(store.audit_requests(), ["request-openai"]);
}
@@ -54,6 +58,7 @@ async fn openai_disable_should_commit_then_release_provider_resources() {
"store.credential_details",
"store.set_enabled",
"provider.account_unavailable",
"provider.account_facts_changed",
]
);
assert_eq!(store.audit_requests(), ["request-openai"]);
@@ -78,6 +83,7 @@ async fn openai_delete_should_commit_then_release_provider_resources() {
"store.credential_details",
"store.delete",
"provider.account_unavailable",
"provider.account_facts_changed",
]
);
assert_eq!(store.audit_requests(), ["request-openai"]);
@@ -114,7 +120,6 @@ async fn openai_import_should_prepare_before_atomic_store_commit() {
.openai()
.import_document(ImportCredentials {
context: context("import-openai"),
expected_config_revision: revision(1),
document: document(),
})
.await
@@ -125,6 +130,7 @@ async fn openai_import_should_prepare_before_atomic_store_commit() {
[
"provider.prepare_import",
"store.commit_import",
"provider.account_facts_changed",
"provider.quota",
]
);
@@ -151,7 +157,6 @@ async fn openai_import_should_refresh_quota_for_every_imported_account() {
.openai()
.import_document(ImportCredentials {
context: context("import-openai-batch"),
expected_config_revision: revision(1),
document: document(),
})
.await
@@ -193,7 +198,6 @@ async fn openai_import_should_remain_successful_when_quota_refresh_fails() {
.openai()
.import_document(ImportCredentials {
context: context("import-openai-quota-failure"),
expected_config_revision: revision(1),
document: document(),
})
.await
@@ -206,6 +210,135 @@ async fn openai_import_should_remain_successful_when_quota_refresh_fails() {
assert_eq!(provider.quota_requests().len(), 1);
}
#[tokio::test]
async fn openai_authorization_create_should_observe_initial_quota() {
let events = events();
let provider = FakeProviderAdmin::new("openai", events.clone());
let store = FakeAccountStore::new("openai", events.clone());
let services = service(provider.clone(), store.clone()).await;
services
.openai()
.start_authorization(StartAuthorization {
context: context("oauth-start-openai-create"),
name: "new OpenAI credential".to_owned(),
reauthorization: None,
})
.await
.expect("start authorization");
services
.openai()
.complete_authorization(CompleteAuthorization {
context: context("oauth-start-openai-create"),
flow_id: "flow-test".to_owned(),
callback_url: "http://localhost/callback?code=test&state=test".to_owned(),
})
.await
.expect("complete authorization");
assert_eq!(
recorded(&events),
[
"provider.start_authorization",
"provider.complete_authorization",
"store.commit_authorization",
"provider.account_facts_changed",
"provider.quota",
]
);
assert_eq!(
provider.quota_requests(),
[ProviderQuotaRequest {
account_id: ProviderAccountId::new("acct_prepared").expect("account ID"),
refresh: true,
rolling_usage: None,
}]
);
assert_eq!(store.audit_requests(), ["oauth-start-openai-create"]);
}
#[tokio::test]
async fn openai_authorization_create_should_remain_successful_when_initial_quota_fails() {
let events = events();
let provider = FakeProviderAdmin::new("openai", events.clone());
provider.fail_next_quota(ProviderAdminErrorKind::Unavailable);
let store = FakeAccountStore::new("openai", events);
let services = service(provider.clone(), store).await;
services
.openai()
.start_authorization(StartAuthorization {
context: context("oauth-start-openai-quota-failure"),
name: "new OpenAI credential".to_owned(),
reauthorization: None,
})
.await
.expect("start authorization");
let result = services
.openai()
.complete_authorization(CompleteAuthorization {
context: context("oauth-start-openai-quota-failure"),
flow_id: "flow-test".to_owned(),
callback_url: "http://localhost/callback?code=test&state=test".to_owned(),
})
.await
.expect("quota observation must not fail authorization");
assert_eq!(result.account_id.as_str(), "acct_prepared");
assert_eq!(provider.quota_requests().len(), 1);
}
#[tokio::test]
async fn openai_authorization_store_failure_should_release_claim_for_retry() {
let events = events();
let provider = FakeProviderAdmin::new("openai", events.clone());
provider.retry_authorization_after_abort();
let store = FakeAccountStore::new("openai", events.clone());
store.fail_next_commit();
let services = service(provider.clone(), store).await;
let command = || CompleteAuthorization {
context: context("oauth-store-retry"),
flow_id: "flow-test".to_owned(),
callback_url: "http://localhost/callback?code=test&state=test".to_owned(),
};
services
.openai()
.start_authorization(StartAuthorization {
context: context("oauth-store-retry"),
name: "new OpenAI credential".to_owned(),
reauthorization: None,
})
.await
.expect("start authorization");
services
.openai()
.complete_authorization(command())
.await
.expect_err("Store commit must fail");
services
.openai()
.complete_authorization(command())
.await
.expect("released claim must permit retry");
assert_eq!(
recorded(&events),
[
"provider.start_authorization",
"provider.complete_authorization",
"store.commit_authorization",
"authorization_guard.abort",
"provider.complete_authorization",
"store.commit_authorization",
"authorization_guard.commit",
"provider.account_facts_changed",
"provider.quota",
]
);
assert!(provider.pending().is_none());
}
#[tokio::test]
async fn openai_import_provider_error_should_not_touch_store_transaction() {
let events = events();
@@ -218,7 +351,6 @@ async fn openai_import_provider_error_should_not_touch_store_transaction() {
.openai()
.import_document(ImportCredentials {
context: context("import-openai-error"),
expected_config_revision: revision(1),
document: document(),
})
.await
@@ -228,7 +360,7 @@ async fn openai_import_provider_error_should_not_touch_store_transaction() {
}
#[tokio::test]
async fn openai_reauthorization_should_restore_pending_envelope_and_hold_guard_through_commit() {
async fn openai_reauthorization_should_commit_after_credential_revision_advances() {
let events = events();
let provider = FakeProviderAdmin::new("openai", events.clone());
let store = FakeAccountStore::new("openai", events.clone());
@@ -237,18 +369,13 @@ async fn openai_reauthorization_should_restore_pending_envelope_and_hold_guard_t
.openai()
.start_authorization(StartAuthorization {
context: context("oauth-start-openai"),
expected_config_revision: revision(7),
name: "reauthorize".to_owned(),
reauthorization: Some(ReauthorizationTarget {
account_id: ProviderAccountId::new("acct_test").expect("account ID"),
credential_revision: revision(1),
}),
reauthorization: Some(ProviderAccountId::new("acct_test").expect("account ID")),
})
.await
.expect("start reauthorization");
let pending = provider.pending().expect("pending envelope");
assert_eq!(pending.expected_config_revision(), revision(7));
assert_eq!(pending.provider_kind().as_str(), "openai");
assert_eq!(
pending.owner_binding().started_request_id(),
@@ -256,15 +383,14 @@ async fn openai_reauthorization_should_restore_pending_envelope_and_hold_guard_t
);
assert!(matches!(
pending.target(),
AuthorizationMutationTarget::Reauthorize {
account_id,
expected_credential_revision,
} if account_id.as_str() == "acct_test"
&& *expected_credential_revision == revision(1)
AuthorizationMutationTarget::Reauthorize { account_id }
if account_id.as_str() == "acct_test"
));
assert!(!format!("{pending:?}").contains("admin-test"));
services
provider.set_current_credential_revision(revision(2));
let result = services
.openai()
.complete_authorization(CompleteAuthorization {
context: context("oauth-complete-openai"),
@@ -274,6 +400,8 @@ async fn openai_reauthorization_should_restore_pending_envelope_and_hold_guard_t
.await
.expect("complete reauthorization");
assert_eq!(result.credential_revision, Some(revision(3)));
assert_eq!(
recorded(&events),
[
@@ -282,9 +410,11 @@ async fn openai_reauthorization_should_restore_pending_envelope_and_hold_guard_t
"provider.complete_authorization",
"store.commit_authorization",
"guard.finish",
"provider.account_facts_changed",
"provider.quota",
]
);
assert_eq!(store.authorization_revision(), Some(revision(7)));
assert_eq!(provider.quota_requests().len(), 1);
assert_eq!(store.audit_requests(), ["oauth-complete-openai"]);
}
@@ -299,7 +429,6 @@ async fn service(provider: Arc<FakeProviderAdmin>, store: Arc<FakeAccountStore>)
fn mutation(account_id: &str) -> CredentialMutation {
CredentialMutation {
context: context("request-openai"),
expected_config_revision: revision(1),
account_id: ProviderAccountId::new(account_id).expect("account ID"),
}
}
@@ -307,7 +436,6 @@ fn mutation(account_id: &str) -> CredentialMutation {
fn deletion(account_id: &str) -> CredentialDeletion {
CredentialDeletion {
context: context("request-openai"),
expected_config_revision: revision(1),
account_ids: vec![ProviderAccountId::new(account_id).expect("account ID")],
}
}
@@ -2,7 +2,7 @@ use async_trait::async_trait;
use gateway_admin::{
model::{
AdminErrorKind, MutationContext, Revision,
AdminErrorKind, MutationContext,
settings::{
AdminApiKey, AdminApiKeyMutation, ReplaceRuntimeSettings, RotationStrategy,
RuntimeSettings,
@@ -33,7 +33,6 @@ impl SettingsStore for UnusedSettingsStore {
async fn replace_admin_api_key(
&self,
_: Revision,
_: AdminApiKey,
_: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation> {
@@ -42,7 +41,6 @@ impl SettingsStore for UnusedSettingsStore {
async fn delete_admin_api_key(
&self,
_: Revision,
_: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation> {
Err(unused())
@@ -63,7 +61,6 @@ async fn settings_should_reject_zero_refresh_margin_before_store_call() {
request_id: "request-settings".to_owned(),
},
ReplaceRuntimeSettings {
expected_config_revision: Revision::new(1).expect("revision"),
model_mappings: Default::default(),
refresh_margin_seconds: 0,
refresh_concurrency: 1,
@@ -8,9 +8,7 @@ use gateway_admin::{
ports::provider::ProviderAdminErrorKind,
};
use super::accounts::{
FakeAccountStore, FakeProviderAdmin, context, document, events, recorded, revision,
};
use super::accounts::{FakeAccountStore, FakeProviderAdmin, context, document, events, recorded};
#[tokio::test]
async fn xai_disable_should_commit_then_notify_the_stateless_provider() {
@@ -31,6 +29,7 @@ async fn xai_disable_should_commit_then_notify_the_stateless_provider() {
"store.credential_details",
"store.set_enabled",
"provider.account_unavailable",
"provider.account_facts_changed",
]
);
assert_eq!(store.audit_requests(), ["request-xai"]);
@@ -47,7 +46,6 @@ async fn xai_import_should_prepare_before_atomic_store_commit() {
.xai()
.import_document(ImportCredentials {
context: context("import-xai"),
expected_config_revision: revision(3),
document: document(),
})
.await
@@ -84,7 +82,6 @@ async fn xai_import_should_refresh_quota_for_every_imported_account() {
.xai()
.import_document(ImportCredentials {
context: context("import-xai-batch"),
expected_config_revision: revision(1),
document: document(),
})
.await
@@ -119,7 +116,6 @@ async fn xai_import_should_remain_successful_when_quota_refresh_fails() {
.xai()
.import_document(ImportCredentials {
context: context("import-xai-quota-failure"),
expected_config_revision: revision(1),
document: document(),
})
.await
@@ -140,7 +136,6 @@ async fn service(provider: Arc<FakeProviderAdmin>, store: Arc<FakeAccountStore>)
fn mutation(account_id: &str) -> CredentialMutation {
CredentialMutation {
context: context("request-xai"),
expected_config_revision: revision(1),
account_id: ProviderAccountId::new(account_id).expect("account ID"),
}
}
+5 -3
View File
@@ -13,7 +13,7 @@ path = "tests/main.rs"
[dependencies]
axum = { version = "0.8.9", default-features = false, features = ["json", "query", "ws"] }
base64 = "0.22.1"
base64 = "0.23.0"
bytes = "1.11.1"
chrono = { version = "0.4.45", features = ["serde"] }
futures = "0.3.32"
@@ -21,7 +21,7 @@ gateway-admin = { path = "../gateway-admin" }
gateway-core = { path = "../gateway-core" }
gateway-protocol = { path = "../gateway-protocol" }
serde = { version = "1.0.228", features = ["derive"] }
serde_json = { version = "1.0.150", features = ["preserve_order"] }
serde_json = { version = "1.0.150", features = ["arbitrary_precision", "preserve_order"] }
thiserror = "2.0.18"
tokio = { version = "1.52.3", default-features = false, features = ["macros", "time"] }
tower-http = { version = "0.7.0", features = ["cors", "fs", "request-id", "timeout", "trace"] }
@@ -31,7 +31,9 @@ uuid = { version = "1.23.3", features = ["v7"] }
[dev-dependencies]
async-trait = "0.1.89"
tokio = { version = "1.52.3", default-features = false, features = ["macros", "rt"] }
axum = { version = "0.8.9", default-features = false, features = ["http1", "tokio"] }
tokio = { version = "1.52.3", default-features = false, features = ["macros", "net", "rt-multi-thread"] }
tokio-tungstenite = "0.29.0"
tower = { version = "0.5.3", features = ["util"] }
[lints]
+29 -127
View File
@@ -16,18 +16,18 @@ use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use chrono::{DateTime, FixedOffset, Utc};
use futures::{Stream, StreamExt as _};
use gateway_admin::model::{
AdminError as AdminServiceError, AdminErrorKind, PageSize, Revision,
AdminError as AdminServiceError, AdminErrorKind, PageSize,
accounts::{
AccountAvailability, AccountConnectionTestEvent as DomainConnectionTestEvent, AccountCost,
AccountListQuery, AccountModelUsage, AccountSort, AccountSortField,
AccountStatus as DomainAccountStatus, AccountUsage, SortDirection,
AccountConnectionTestEvent as DomainConnectionTestEvent, AccountCost, AccountListQuery,
AccountModelUsage, AccountSort, AccountSortField, AccountStatus as DomainAccountStatus,
AccountUsage, SortDirection,
},
provider_credentials::{
AccountDirectoryItem, AccountDirectoryPage, AccountExportBundle, AccountRefreshResult,
AuthorizationStarted, CompleteAuthorization, CredentialDeletion, CredentialDeletionResult,
CredentialImportResult, CredentialMutation, CredentialMutationResult, ImportCredentials,
ProviderDocument, ProviderModels, ProviderQuota, ProviderQuotaWindow,
ReauthorizationTarget, RotateCredential, StartAuthorization,
ProviderDocument, ProviderModels, ProviderQuota, ProviderQuotaWindow, RotateCredential,
StartAuthorization,
},
};
use gateway_core::{
@@ -161,7 +161,6 @@ fn parse_sort_direction(value: &str) -> Option<SortDirection> {
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct AccountPageData {
pub config_revision: u64,
pub items: Vec<AccountView>,
pub page: PageMeta,
pub summary: AccountSummaryView,
@@ -197,8 +196,6 @@ pub struct AccountView {
pub token_refreshing: bool,
pub availability: String,
pub enabled: bool,
pub credential_revision: u64,
pub state_revision: Option<u64>,
pub access_token_expires_at: Option<String>,
pub access_token_expires_at_display: Option<String>,
pub refresh_token_expires_at: Option<String>,
@@ -391,31 +388,21 @@ impl AccountActionRequest {
}
}
/// 手工 OAuth 刷新会变更持久 credential,因此必须携带配置 CAS revision。
/// 手工 OAuth 刷新会变更持久 credential。
#[derive(Debug, Clone, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct AccountRefreshRequest {
pub account_id: String,
pub expected_config_revision: u64,
}
impl AccountRefreshRequest {
pub fn validate(&self) -> Result<(), WireValidationError> {
require_account_id(&self.account_id, "accountId")?;
if self.expected_config_revision == 0 {
return Err(WireValidationError::new("expectedConfigRevision"));
}
Ok(())
require_account_id(&self.account_id, "accountId")
}
fn into_command(self) -> Result<(Revision, ProviderAccountId), WireValidationError> {
fn into_command(self) -> Result<ProviderAccountId, WireValidationError> {
self.validate()?;
Ok((
Revision::new(self.expected_config_revision)
.map_err(|_| WireValidationError::new("expectedConfigRevision"))?,
ProviderAccountId::new(self.account_id)
.map_err(|_| WireValidationError::new("accountId"))?,
))
ProviderAccountId::new(self.account_id).map_err(|_| WireValidationError::new("accountId"))
}
}
@@ -460,7 +447,6 @@ pub struct AccountModelsData {
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct AccountRefreshData {
pub config_revision: u64,
pub account: AccountView,
}
@@ -545,12 +531,18 @@ impl From<DomainConnectionTestEvent> for AccountConnectionTestEvent {
message,
provider_error_code,
provider_error_type,
upstream_status,
upstream_content_type,
upstream_body,
account_status,
} => serde_json::json!({
"type": "error",
"error": message,
"providerErrorCode": provider_error_code,
"providerErrorType": provider_error_type,
"upstreamStatus": upstream_status,
"upstreamContentType": upstream_content_type,
"upstreamBody": upstream_body,
"accountStatus": account_status_name(account_status)
}),
};
@@ -579,14 +571,12 @@ impl AccountProvider {
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct AccountImportRequest {
pub provider: String,
pub expected_config_revision: u64,
pub data: Value,
}
impl AccountImportRequest {
pub fn validate(&self) -> Result<(), WireValidationError> {
AccountProvider::parse(&self.provider)?;
require_positive(self.expected_config_revision, "expectedConfigRevision")?;
if !self.data.is_object()
|| serde_json::to_vec(&self.data)
.map_or(true, |encoded| encoded.len() > MAX_IMPORT_DATA_BYTES)
@@ -606,10 +596,6 @@ impl AccountImportRequest {
provider,
ImportCredentials {
context,
expected_config_revision: revision(
self.expected_config_revision,
"expectedConfigRevision",
)?,
document: provider_document(self.data, "data")?,
},
))
@@ -620,28 +606,18 @@ impl AccountImportRequest {
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct StartAccountAuthorizationRequest {
pub provider: String,
pub expected_config_revision: u64,
pub name: String,
pub account_id: Option<String>,
pub expected_credential_revision: Option<u64>,
}
impl StartAccountAuthorizationRequest {
pub fn validate(&self) -> Result<(), WireValidationError> {
AccountProvider::parse(&self.provider)?;
require_positive(self.expected_config_revision, "expectedConfigRevision")?;
require_text(&self.name, MAX_NAME_BYTES, "name")?;
match (
self.account_id.as_deref(),
self.expected_credential_revision,
) {
(None, None) => Ok(()),
(Some(account_id), Some(credential_revision)) => {
require_account_id(account_id, "accountId")?;
require_positive(credential_revision, "expectedCredentialRevision")
}
_ => Err(WireValidationError::new("reauthorization")),
if let Some(account_id) = self.account_id.as_deref() {
require_account_id(account_id, "accountId")?;
}
Ok(())
}
fn into_command(
@@ -650,23 +626,15 @@ impl StartAccountAuthorizationRequest {
) -> Result<(AccountProvider, StartAuthorization), WireValidationError> {
self.validate()?;
let provider = AccountProvider::parse(&self.provider)?;
let reauthorization = match (self.account_id, self.expected_credential_revision) {
(None, None) => None,
(Some(account_id), Some(credential_revision)) => Some(ReauthorizationTarget {
account_id: ProviderAccountId::new(account_id)
.map_err(|_| WireValidationError::new("accountId"))?,
credential_revision: revision(credential_revision, "expectedCredentialRevision")?,
}),
_ => return Err(WireValidationError::new("reauthorization")),
};
let reauthorization = self
.account_id
.map(ProviderAccountId::new)
.transpose()
.map_err(|_| WireValidationError::new("accountId"))?;
Ok((
provider,
StartAuthorization {
context,
expected_config_revision: revision(
self.expected_config_revision,
"expectedConfigRevision",
)?,
name: self.name,
reauthorization,
},
@@ -721,14 +689,12 @@ impl CompleteAccountAuthorizationRequest {
pub struct AccountMutationRequest {
pub provider: String,
pub account_id: String,
pub expected_config_revision: u64,
}
impl AccountMutationRequest {
pub fn validate(&self) -> Result<(), WireValidationError> {
AccountProvider::parse(&self.provider)?;
require_account_id(&self.account_id, "accountId")?;
require_positive(self.expected_config_revision, "expectedConfigRevision")
require_account_id(&self.account_id, "accountId")
}
fn into_command(
@@ -741,10 +707,6 @@ impl AccountMutationRequest {
provider,
CredentialMutation {
context,
expected_config_revision: revision(
self.expected_config_revision,
"expectedConfigRevision",
)?,
account_id: ProviderAccountId::new(self.account_id)
.map_err(|_| WireValidationError::new("accountId"))?,
},
@@ -757,7 +719,6 @@ impl AccountMutationRequest {
pub struct AccountDeletionRequest {
pub provider: String,
pub account_ids: Vec<String>,
pub expected_config_revision: u64,
}
impl AccountDeletionRequest {
@@ -773,7 +734,7 @@ impl AccountDeletionRequest {
return Err(WireValidationError::new("accountIds"));
}
}
require_positive(self.expected_config_revision, "expectedConfigRevision")
Ok(())
}
fn into_command(
@@ -786,10 +747,6 @@ impl AccountDeletionRequest {
provider,
CredentialDeletion {
context,
expected_config_revision: revision(
self.expected_config_revision,
"expectedConfigRevision",
)?,
account_ids: self
.account_ids
.into_iter()
@@ -806,8 +763,6 @@ impl AccountDeletionRequest {
pub struct RotateAccountRequest {
pub provider: String,
pub account_id: String,
pub expected_config_revision: u64,
pub expected_credential_revision: u64,
pub access_token: String,
pub refresh_token: Option<String>,
}
@@ -818,11 +773,6 @@ impl RotateAccountRequest {
return Err(WireValidationError::new("provider"));
}
require_account_id(&self.account_id, "accountId")?;
require_positive(self.expected_config_revision, "expectedConfigRevision")?;
require_positive(
self.expected_credential_revision,
"expectedCredentialRevision",
)?;
validate_oauth_material(&self.access_token, self.refresh_token.as_deref())
}
@@ -840,17 +790,9 @@ impl RotateAccountRequest {
Ok(RotateCredential {
mutation: CredentialMutation {
context,
expected_config_revision: revision(
self.expected_config_revision,
"expectedConfigRevision",
)?,
account_id: ProviderAccountId::new(self.account_id)
.map_err(|_| WireValidationError::new("accountId"))?,
},
expected_credential_revision: revision(
self.expected_credential_revision,
"expectedCredentialRevision",
)?,
provider_material: ProviderDocument::new(OpaqueProviderData::new(material)),
})
}
@@ -859,7 +801,6 @@ impl RotateAccountRequest {
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct AccountImportData {
pub config_revision: u64,
pub imported_count: usize,
pub account_ids: Vec<String>,
}
@@ -872,7 +813,6 @@ impl From<CredentialImportResult> for AccountImportData {
.map(|account_id| account_id.to_string())
.collect::<Vec<_>>();
Self {
config_revision: result.config_revision.get(),
imported_count: account_ids.len(),
account_ids,
}
@@ -900,18 +840,13 @@ impl From<AuthorizationStarted> for AccountAuthorizationData {
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct AccountMutationData {
pub config_revision: u64,
pub account_id: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub credential_revision: Option<u64>,
}
impl From<CredentialMutationResult> for AccountMutationData {
fn from(result: CredentialMutationResult) -> Self {
Self {
config_revision: result.config_revision.get(),
account_id: result.account_id.to_string(),
credential_revision: result.credential_revision.map(Revision::get),
}
}
}
@@ -919,7 +854,6 @@ impl From<CredentialMutationResult> for AccountMutationData {
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct AccountDeletionData {
pub config_revision: u64,
pub deleted_count: usize,
pub account_ids: Vec<String>,
}
@@ -932,7 +866,6 @@ impl From<CredentialDeletionResult> for AccountDeletionData {
.map(|account_id| account_id.to_string())
.collect::<Vec<_>>();
Self {
config_revision: result.config_revision.get(),
deleted_count: account_ids.len(),
account_ids,
}
@@ -1264,15 +1197,11 @@ async fn refresh_account<S>(
where
S: AdminSessionState + Send + Sync,
{
let (expected_config_revision, account_id) = request.into_command().map_err(map_wire_error)?;
let account_id = request.into_command().map_err(map_wire_error)?;
let result = state
.admin_services()
.accounts()
.refresh(
&auth.context().mutation_context(),
expected_config_revision,
account_id,
)
.refresh(&auth.context().mutation_context(), account_id)
.await
.map_err(map_service_error)?;
let data = account_refresh_data(result, Utc::now());
@@ -1394,7 +1323,6 @@ fn account_page_data(
u32::try_from(result.total.div_ceil(u64::from(page_size))).unwrap_or(u32::MAX)
};
AccountPageData {
config_revision: result.config_revision.get(),
items: result
.items
.into_iter()
@@ -1412,7 +1340,6 @@ fn account_page_data(
fn account_refresh_data(result: AccountRefreshResult, now: DateTime<Utc>) -> AccountRefreshData {
AccountRefreshData {
config_revision: result.config_revision.get(),
account: account_view(result.account, now),
}
}
@@ -1460,10 +1387,8 @@ fn account_view(item: AccountDirectoryItem, now: DateTime<Utc>) -> AccountView {
status: status.clone(),
display_status: status,
token_refreshing: false,
availability: account_availability_name(account.availability).to_owned(),
availability: account.availability.as_str().to_owned(),
enabled: account.enabled,
credential_revision: account.credential_revision.get(),
state_revision: None,
access_token_expires_at: expires_at,
access_token_expires_at_display: account
.access_token_expires_at
@@ -1714,10 +1639,6 @@ fn empty_account_usage() -> AccountUsageView {
}
}
fn revision(value: u64, field: &'static str) -> Result<Revision, WireValidationError> {
Revision::new(value).map_err(|_| WireValidationError::new(field))
}
fn provider_document(
value: Value,
field: &'static str,
@@ -1751,13 +1672,6 @@ fn require_text(
Ok(())
}
fn require_positive(value: u64, field: &'static str) -> Result<(), WireValidationError> {
if value == 0 || i64::try_from(value).is_err() {
return Err(WireValidationError::new(field));
}
Ok(())
}
fn validate_oauth_material(
access_token: &str,
refresh_token: Option<&str>,
@@ -1805,18 +1719,6 @@ fn account_status_name(status: DomainAccountStatus) -> &'static str {
}
}
fn account_availability_name(availability: AccountAvailability) -> &'static str {
match availability {
AccountAvailability::Unknown => "unknown",
AccountAvailability::Ready => "ready",
AccountAvailability::Cooldown => "cooldown",
AccountAvailability::QuotaExhausted => "quota_exhausted",
AccountAvailability::Expired => "expired",
AccountAvailability::Banned => "banned",
AccountAvailability::Invalid => "invalid",
}
}
fn format_number(value: u64) -> String {
let text = value.to_string();
let mut output = String::with_capacity(text.len() + text.len() / 3);
@@ -4,14 +4,11 @@ use std::fmt;
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use chrono::{DateTime, Utc};
use gateway_admin::model::{
Revision,
client_keys::{
ClientKeyCursor, ClientKeyCursorValue as DomainCursorValue, ClientKeyListQuery,
ClientKeyMutation, ClientKeyPage, ClientKeyPageSize, ClientKeyRecord, ClientKeySecret,
ClientKeySort as DomainSort, ClientKeySortField as DomainSortField, CreateClientKey,
CreatedClientKey, DeleteClientKey, SetClientKeyEnabled, SortDirection, UpdateClientKey,
},
use gateway_admin::model::client_keys::{
ClientKeyCursor, ClientKeyCursorValue as DomainCursorValue, ClientKeyListQuery,
ClientKeyMutation, ClientKeyPage, ClientKeyPageSize, ClientKeyRecord, ClientKeySecret,
ClientKeySort as DomainSort, ClientKeySortField as DomainSortField, CreateClientKey,
CreatedClientKey, DeleteClientKey, SetClientKeyEnabled, SortDirection, UpdateClientKey,
};
use gateway_core::{
policy::{ClientApiKeyId, RateLimits},
@@ -138,7 +135,6 @@ impl ClientKeySort {
#[derive(Debug, Clone, PartialEq, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct CreateClientKeyRequest {
expected_config_revision: u64,
name: String,
label: Option<String>,
provider_kind: String,
@@ -149,14 +145,12 @@ pub struct CreateClientKeyRequest {
impl CreateClientKeyRequest {
/// 校验 wire 边界并直接构造管理用例命令。
pub fn into_command(self) -> Result<CreateClientKey, WireValidationError> {
validate_revision(self.expected_config_revision)?;
validate_required_text(&self.name, "name")?;
validate_optional_text(self.label.as_deref(), "label")?;
validate_provider_kind(&self.provider_kind)?;
validate_limit(self.max_concurrency, "maxConcurrency")?;
validate_limit(self.requests_per_minute, "requestsPerMinute")?;
Ok(CreateClientKey {
expected_config_revision: revision(self.expected_config_revision)?,
name: self.name,
label: self.label,
provider_kind: ProviderKind::new(self.provider_kind)
@@ -174,7 +168,6 @@ impl CreateClientKeyRequest {
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct UpdateClientKeyRequest {
id: String,
expected_config_revision: u64,
name: String,
label: Option<String>,
provider_kind: String,
@@ -186,14 +179,12 @@ impl UpdateClientKeyRequest {
/// 校验 wire 边界并直接构造管理用例命令。
pub fn into_command(self) -> Result<UpdateClientKey, WireValidationError> {
validate_required_text(&self.id, "id")?;
validate_revision(self.expected_config_revision)?;
validate_required_text(&self.name, "name")?;
validate_optional_text(self.label.as_deref(), "label")?;
validate_provider_kind(&self.provider_kind)?;
validate_limit(self.max_concurrency, "maxConcurrency")?;
validate_limit(self.requests_per_minute, "requestsPerMinute")?;
Ok(UpdateClientKey {
expected_config_revision: revision(self.expected_config_revision)?,
id: client_key_id(self.id, "clientKeyMutationNotFound")?,
name: self.name,
label: self.label,
@@ -207,12 +198,11 @@ impl UpdateClientKeyRequest {
}
}
/// 只携带 ID 与配置 revision 的 Client Key mutation 请求。
/// 只携带 ID 的 Client Key mutation 请求。
#[derive(Debug, Clone, PartialEq, Eq, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct ClientKeyRevisionRequest {
pub struct ClientKeyMutationRequest {
id: String,
expected_config_revision: u64,
}
/// 读取一次完整 Key 的 ID query。
@@ -233,20 +223,15 @@ impl ClientKeyIdQuery {
}
}
impl ClientKeyRevisionRequest {
/// 校验请求并取出 ID 与 revision。
pub fn into_parts(self) -> Result<(String, u64), WireValidationError> {
impl ClientKeyMutationRequest {
/// 校验请求并取出 ID。
pub fn into_id(self) -> Result<String, WireValidationError> {
validate_required_text(&self.id, "id")?;
validate_revision(self.expected_config_revision)?;
Ok((self.id, self.expected_config_revision))
Ok(self.id)
}
fn into_command_parts(self) -> Result<(ClientApiKeyId, Revision), WireValidationError> {
let (id, expected_config_revision) = self.into_parts()?;
Ok((
client_key_id(id, "clientKeyMutationNotFound")?,
revision(expected_config_revision)?,
))
fn into_domain_id(self) -> Result<ClientApiKeyId, WireValidationError> {
client_key_id(self.into_id()?, "clientKeyMutationNotFound")
}
}
@@ -289,7 +274,6 @@ impl From<ClientKeyRecord> for ClientKeyView {
#[derive(Debug, Clone, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct ClientKeyListData {
config_revision: u64,
items: Vec<ClientKeyView>,
next_cursor: Option<String>,
total: u64,
@@ -298,14 +282,8 @@ pub struct ClientKeyListData {
impl ClientKeyListData {
/// 构造 Client Key 列表响应。
#[must_use]
pub fn new(
config_revision: u64,
items: Vec<ClientKeyView>,
next_cursor: Option<String>,
total: u64,
) -> Self {
pub fn new(items: Vec<ClientKeyView>, next_cursor: Option<String>, total: u64) -> Self {
Self {
config_revision,
items,
next_cursor,
total,
@@ -325,7 +303,6 @@ impl TryFrom<ClientKeyPage> for ClientKeyListData {
.map(encode_client_key_cursor)
.transpose()?;
Ok(Self::new(
page.config_revision.get(),
page.items.into_iter().map(Into::into).collect(),
next_cursor,
page.total,
@@ -337,7 +314,6 @@ impl TryFrom<ClientKeyPage> for ClientKeyListData {
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
pub struct CreatedClientKeyData {
config_revision: u64,
id: String,
prefix: String,
plaintext_key: String,
@@ -346,9 +322,8 @@ pub struct CreatedClientKeyData {
impl CreatedClientKeyData {
/// 构造一次性创建响应。
#[must_use]
pub fn new(config_revision: u64, id: String, prefix: String, plaintext_key: String) -> Self {
pub fn new(id: String, prefix: String, plaintext_key: String) -> Self {
Self {
config_revision,
id,
prefix,
plaintext_key,
@@ -359,7 +334,6 @@ impl CreatedClientKeyData {
impl From<CreatedClientKey> for CreatedClientKeyData {
fn from(created: CreatedClientKey) -> Self {
Self::new(
created.config_revision.get(),
created.secret.record.id.to_string(),
created.secret.record.prefix.clone(),
created.secret.expose_for_response().to_owned(),
@@ -371,7 +345,6 @@ impl fmt::Debug for CreatedClientKeyData {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("CreatedClientKeyData")
.field("config_revision", &self.config_revision)
.field("id", &self.id)
.field("prefix", &self.prefix)
.field("plaintext_key", &"[REDACTED]")
@@ -417,24 +390,20 @@ impl fmt::Debug for RevealedClientKeyData {
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct MutatedClientKeyData {
config_revision: u64,
id: String,
}
impl MutatedClientKeyData {
/// 构造 mutation 响应。
#[must_use]
pub fn new(config_revision: u64, id: String) -> Self {
Self {
config_revision,
id,
}
pub fn new(id: String) -> Self {
Self { id }
}
}
impl From<ClientKeyMutation> for MutatedClientKeyData {
fn from(mutation: ClientKeyMutation) -> Self {
Self::new(mutation.config_revision.get(), mutation.id.to_string())
Self::new(mutation.id.to_string())
}
}
@@ -582,18 +551,6 @@ fn wire_cursor(cursor: ClientKeyCursor) -> Result<ClientKeyCursorData, WireValid
Ok(cursor)
}
fn validate_revision(value: u64) -> Result<(), WireValidationError> {
if value == 0 {
return Err(WireValidationError::new("expectedConfigRevision"));
}
Ok(())
}
fn revision(value: u64) -> Result<Revision, WireValidationError> {
validate_revision(value)?;
Revision::new(value).map_err(|_| WireValidationError::new("expectedConfigRevision"))
}
fn client_key_id(
value: String,
field: &'static str,
@@ -768,23 +725,19 @@ where
async fn disable_client_key<S>(
auth: AdminAuth,
State(state): State<S>,
Json(payload): Json<ClientKeyRevisionRequest>,
Json(payload): Json<ClientKeyMutationRequest>,
) -> Result<impl IntoResponse, AdminError>
where
S: AdminSessionState + Send + Sync,
{
let (id, expected_config_revision) = payload.into_command_parts().map_err(map_wire_error)?;
let id = payload.into_domain_id().map_err(map_wire_error)?;
mutation_response(
state
.admin_services()
.client_keys()
.set_enabled(
&auth.context().mutation_context(),
SetClientKeyEnabled {
expected_config_revision,
id,
enabled: false,
},
SetClientKeyEnabled { id, enabled: false },
)
.await,
)
@@ -793,23 +746,19 @@ where
async fn enable_client_key<S>(
auth: AdminAuth,
State(state): State<S>,
Json(payload): Json<ClientKeyRevisionRequest>,
Json(payload): Json<ClientKeyMutationRequest>,
) -> Result<impl IntoResponse, AdminError>
where
S: AdminSessionState + Send + Sync,
{
let (id, expected_config_revision) = payload.into_command_parts().map_err(map_wire_error)?;
let id = payload.into_domain_id().map_err(map_wire_error)?;
mutation_response(
state
.admin_services()
.client_keys()
.set_enabled(
&auth.context().mutation_context(),
SetClientKeyEnabled {
expected_config_revision,
id,
enabled: true,
},
SetClientKeyEnabled { id, enabled: true },
)
.await,
)
@@ -818,23 +767,17 @@ where
async fn delete_client_key<S>(
auth: AdminAuth,
State(state): State<S>,
Json(payload): Json<ClientKeyRevisionRequest>,
Json(payload): Json<ClientKeyMutationRequest>,
) -> Result<impl IntoResponse, AdminError>
where
S: AdminSessionState + Send + Sync,
{
let (id, expected_config_revision) = payload.into_command_parts().map_err(map_wire_error)?;
let id = payload.into_domain_id().map_err(map_wire_error)?;
mutation_response(
state
.admin_services()
.client_keys()
.delete(
&auth.context().mutation_context(),
DeleteClientKey {
expected_config_revision,
id,
},
)
.delete(&auth.context().mutation_context(), DeleteClientKey { id })
.await,
)
}
@@ -854,9 +797,6 @@ fn map_wire_error(error: WireValidationError) -> AdminError {
"cursor" => AdminError::bad_request("Invalid client key cursor"),
"clientKeyRevealNotFound" => AdminError::not_found("Client API key was not found"),
"clientKeyMutationNotFound" => AdminError::not_found("client API key was not found"),
"expectedConfigRevision" => {
AdminError::bad_request("expectedConfigRevision must be positive")
}
_ => AdminError::bad_request("Invalid client API key request"),
}
}
@@ -1800,7 +1800,7 @@ fn usage_record_view(record: domain::UsageRecord) -> UsageRecordView {
model,
requested_model: Some(record.requested_model_id),
upstream_model: record.upstream_model_id,
service_tier: None,
service_tier: record.service_tier,
status_code,
transport,
attempt_index: None,
@@ -10,12 +10,8 @@ use axum::{
routing::{get, post},
};
use chrono::{DateTime, Utc};
use gateway_admin::model::{
Revision,
settings::{
ModelMappings as DomainModelMappings, ReplaceRuntimeSettings, RotationStrategy,
RuntimeSettings,
},
use gateway_admin::model::settings::{
ModelMappings as DomainModelMappings, ReplaceRuntimeSettings, RotationStrategy, RuntimeSettings,
};
use gateway_core::routing::{PublicModelId, UpstreamModelId};
use serde::{Deserialize, Serialize};
@@ -32,7 +28,6 @@ pub type ModelMappings = BTreeMap<String, String>;
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct RuntimeSettingsView {
pub config_revision: u64,
pub model_mappings: ModelMappings,
pub refresh_margin_seconds: u64,
pub refresh_concurrency: u64,
@@ -49,7 +44,6 @@ pub struct RuntimeSettingsView {
#[derive(Debug, Clone, PartialEq, Eq, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct UpdateRuntimeSettingsRequest {
pub expected_config_revision: u64,
pub model_mappings: ModelMappings,
pub refresh_margin_seconds: u64,
pub refresh_concurrency: u64,
@@ -64,9 +58,6 @@ pub struct UpdateRuntimeSettingsRequest {
impl UpdateRuntimeSettingsRequest {
/// 校验公共运行参数。
pub fn validate(&self) -> Result<(), WireValidationError> {
if self.expected_config_revision == 0 {
return Err(WireValidationError::new("expectedConfigRevision"));
}
validate_model_mappings(&self.model_mappings)?;
for (value, field) in [
(self.refresh_margin_seconds, "refreshMarginSeconds"),
@@ -93,8 +84,6 @@ impl UpdateRuntimeSettingsRequest {
fn into_command(self) -> Result<ReplaceRuntimeSettings, WireValidationError> {
self.validate()?;
Ok(ReplaceRuntimeSettings {
expected_config_revision: Revision::new(self.expected_config_revision)
.map_err(|_| WireValidationError::new("settingsRevisionOverflow"))?,
model_mappings: domain_model_mappings(self.model_mappings)?,
refresh_margin_seconds: self.refresh_margin_seconds,
refresh_concurrency: u32::try_from(self.refresh_concurrency)
@@ -117,7 +106,6 @@ impl UpdateRuntimeSettingsRequest {
impl From<RuntimeSettings> for RuntimeSettingsView {
fn from(settings: RuntimeSettings) -> Self {
Self {
config_revision: settings.config_revision.get(),
model_mappings: wire_model_mappings(settings.model_mappings),
refresh_margin_seconds: settings.refresh_margin_seconds,
refresh_concurrency: u64::from(settings.refresh_concurrency),
@@ -358,7 +346,6 @@ fn valid_model_name(value: &str, max_len: usize) -> bool {
fn map_wire_error(error: WireValidationError) -> AdminError {
let message = match error.field() {
"settingsRevisionOverflow" => "expectedConfigRevision must be positive".to_owned(),
"settingsRefreshConcurrencyOverflow" => "Invalid refreshConcurrency".to_owned(),
"settingsMaxConcurrencyOverflow" => "Invalid maxConcurrentPerAccount".to_owned(),
"settingsUsageRetentionOverflow" => "Invalid usageRetentionDays".to_owned(),
+35 -3
View File
@@ -2,7 +2,8 @@
use axum::{
Json,
http::StatusCode,
body::Body,
http::{HeaderName, HeaderValue, StatusCode, header::CONTENT_TYPE},
response::{IntoResponse, Response},
};
use gateway_core::{
@@ -75,10 +76,14 @@ pub fn gateway_error_from_engine(error: &EngineError) -> GatewayError {
if error.kind() == ProviderErrorKind::Unavailable
&& error.send_state() == UpstreamSendState::NotSent =>
{
GatewayError::new(
let gateway = GatewayError::new(
GatewayErrorKind::NoAvailableProvider,
"no upstream provider is currently available for this request",
)
);
match error.client_visible_upstream_error() {
Some(upstream) => gateway.with_client_visible_upstream_error(upstream.clone()),
None => gateway,
}
}
EngineError::Provider(error) => GatewayError::from_provider(error),
EngineError::Cancelled => {
@@ -117,6 +122,33 @@ pub fn gateway_error_response(error: &GatewayError) -> Response {
.into_response()
}
/// 在下游尚未提交时优先交付 Provider 的原始 HTTP 失败响应。
pub fn engine_error_response(error: &EngineError) -> Response {
if let EngineError::Provider(error) = error
&& let Some(upstream) = error.client_visible_upstream_response()
&& let Ok(status) = StatusCode::from_u16(upstream.status())
{
let mut response = Response::new(Body::from(upstream.body().clone()));
*response.status_mut() = status;
if let Some(content_type) = upstream.content_type()
&& let Ok(content_type) = HeaderValue::from_bytes(content_type)
{
response.headers_mut().insert(CONTENT_TYPE, content_type);
}
for header in upstream.headers() {
let Ok(name) = HeaderName::from_bytes(header.name().as_bytes()) else {
continue;
};
let Ok(value) = HeaderValue::from_bytes(header.value()) else {
continue;
};
response.headers_mut().append(name, value);
}
return response;
}
gateway_error_response(&gateway_error_from_engine(error))
}
/// Gateway 错误稳定映射,供 HTTP、SSE 和 WebSocket 共用。
#[must_use]
pub const fn gateway_error_contract(
@@ -189,6 +189,22 @@ fn codex_model_json(profile: &PublicModelProfile, index: usize) -> Value {
} else {
vec!["text"]
};
let additional_speed_tiers = presentation
.service_tiers()
.iter()
.filter_map(|tier| tier.speed_tier())
.collect::<Vec<_>>();
let service_tiers = presentation
.service_tiers()
.iter()
.map(|tier| {
json!({
"id": tier.id(),
"name": tier.name(),
"description": tier.description(),
})
})
.collect::<Vec<_>>();
json!({
"slug": id,
@@ -200,8 +216,8 @@ fn codex_model_json(profile: &PublicModelProfile, index: usize) -> Value {
"visibility": if presentation.hidden() { "hide" } else { "list" },
"supported_in_api": true,
"priority": index.saturating_add(1),
"additional_speed_tiers": [],
"service_tiers": [],
"additional_speed_tiers": additional_speed_tiers,
"service_tiers": service_tiers,
"availability_nux": Value::Null,
"upgrade": Value::Null,
"base_instructions": CODEX_BASE_INSTRUCTIONS,
@@ -152,9 +152,6 @@ pub enum ResponseEncodeError {
/// 协议原生流缺少可返回的终态 response object。
#[error("responses wire stream has no terminal response")]
MissingWireTerminal,
/// 协议原生事件携带了不一致的响应身份。
#[error("responses wire stream changed its response identity")]
WireIdentityChanged,
/// 终态 response object 无法序列化为完整 HTTP JSON 响应。
#[error("responses wire terminal serialization failed")]
Serialization,
@@ -167,7 +164,7 @@ impl ResponseEncodeError {
ProtocolErrorBody {
error: ProtocolError {
kind: "server_error",
code: "invalid_canonical_response",
code: "invalid_upstream_response",
message: "The gateway could not forward the upstream response.".to_owned(),
param: None,
},
@@ -8,7 +8,7 @@ use axum::{
body::{Body, Bytes},
extract::{Extension, State, connect_info::ConnectInfo},
http::{
HeaderMap, HeaderValue, StatusCode,
HeaderMap, HeaderName, HeaderValue, StatusCode,
header::{CACHE_CONTROL, CONTENT_TYPE, USER_AGENT},
},
response::{IntoResponse, Response},
@@ -25,8 +25,8 @@ use crate::ApiState;
use crate::openai::{
auth::{authenticate_client, authentication_error_response},
error::{
gateway_error_contract, gateway_error_from_engine, gateway_error_response,
protocol_error_response, runtime_unavailable_response,
engine_error_response, gateway_error_contract, gateway_error_from_engine,
gateway_error_response, protocol_error_response, runtime_unavailable_response,
},
};
@@ -180,7 +180,7 @@ pub async fn collect_execution_response(session: Box<dyn ExecutionSession>) -> R
let events = match session.collect_uncommitted().await {
Ok(events) => events,
Err(error) => {
let response = gateway_error_response(&gateway_error_from_engine(&error));
let response = engine_error_response(&error);
return execution.record_response_status(response).await;
}
};
@@ -206,7 +206,7 @@ pub async fn collect_execution_response(session: Box<dyn ExecutionSession>) -> R
.await
{
execution.cancel_and_finalize().await;
let response = gateway_error_response(&gateway_error_from_engine(&error));
let response = engine_error_response(&error);
return execution.record_response_status(response).await;
}
if !session.is_finalized() {
@@ -237,9 +237,7 @@ fn encode_collected_events(
) -> Result<Vec<u8>, BufferedResponseEncodeError> {
let mut encoder = OpenAiResponsesEncoder::new();
for event in events {
encoder
.push_sse(event)
.map_err(BufferedResponseEncodeError::Canonical)?;
encoder.push_sse(event);
}
let response = encoder
.finish()
@@ -253,23 +251,25 @@ fn json_body_response(encoded: Vec<u8>, response_headers: &[ProviderResponseHead
response
.headers_mut()
.insert(CONTENT_TYPE, HeaderValue::from_static("application/json"));
apply_safe_response_headers(response, response_headers)
apply_response_headers(response, response_headers)
}
fn apply_safe_response_headers(
fn apply_response_headers(
mut response: Response,
response_headers: &[ProviderResponseHeader],
) -> Response {
let connection_options = super::response_connection_options(response_headers);
for header in response_headers {
let Some(name) = super::safe_response_header_name(header.name()) else {
if !super::response_header_is_forwardable(header.name(), &connection_options) {
continue;
}
let Ok(name) = HeaderName::from_bytes(header.name().as_bytes()) else {
continue;
};
let Ok(value) = HeaderValue::from_str(header.value().as_str()) else {
let Ok(value) = HeaderValue::from_bytes(header.value()) else {
continue;
};
response
.headers_mut()
.insert(axum::http::header::HeaderName::from_static(name), value);
response.headers_mut().append(name, value);
}
response
}
@@ -295,7 +295,7 @@ pub async fn stream_execution_response(
return response;
}
Err(error) => {
let response = gateway_error_response(&gateway_error_from_engine(&error));
let response = engine_error_response(&error);
return execution.record_response_status(response).await;
}
};
@@ -310,18 +310,7 @@ pub async fn stream_execution_response(
let mut encoder = OpenAiResponsesEncoder::new();
let mut frames = Vec::new();
for event in &first_events {
match encoder.push_sse(event) {
Ok(encoded) => frames.extend(encoded),
Err(error) => {
let response = protocol_error_response(
StatusCode::INTERNAL_SERVER_ERROR,
error.protocol_body(),
);
let response = execution.record_response_status(response).await;
execution.cancel_and_finalize().await;
return response;
}
}
frames.extend(encoder.push_sse(event));
}
if frames.is_empty() {
let response = internal_gateway_response("gateway commit batch encoded no output");
@@ -338,7 +327,7 @@ pub async fn stream_execution_response(
.await
{
execution.cancel_and_finalize().await;
let response = gateway_error_response(&gateway_error_from_engine(&error));
let response = engine_error_response(&error);
return execution.record_response_status(response).await;
}
let Some(session) = execution.into_session() else {
@@ -371,7 +360,7 @@ fn event_stream_response(body: Body, response_headers: &[ProviderResponseHeader]
response
.headers_mut()
.insert(CACHE_CONTROL, HeaderValue::from_static("no-cache"));
apply_safe_response_headers(response, response_headers)
apply_response_headers(response, response_headers)
}
fn internal_gateway_response(message: &'static str) -> Response {
@@ -403,15 +392,10 @@ impl PendingExecution {
}
async fn record_response_status(&mut self, response: Response) -> Response {
let Some(session) = self.session.as_mut() else {
return internal_gateway_response("gateway response session is unavailable");
};
if session
.record_client_status(response.status().as_u16())
.await
.is_err()
{
return internal_gateway_response("gateway client status could not be persisted");
if let Some(session) = self.session.as_mut() {
let _ = session
.record_client_status(response.status().as_u16())
.await;
}
response
}
@@ -499,10 +483,16 @@ impl ResponsesStreamState {
.session
.as_ref()
.is_some_and(|session| session.is_finalized());
self.finish_with_gateway_error(GatewayError::new(
GatewayErrorKind::Internal,
"gateway response ended without a terminal event",
));
if self.execution_terminal {
self.pending
.push_back(Bytes::from_static(DONE_SSE_FRAME.as_bytes()));
self.output_finished = true;
} else {
self.finish_with_gateway_error(GatewayError::new(
GatewayErrorKind::Internal,
"gateway response ended without finalizing the execution",
));
}
}
Err(error) => {
self.execution_terminal = self
@@ -521,18 +511,11 @@ impl ResponsesStreamState {
}
async fn push_event(&mut self, event: ProviderEvent) {
match self.encoder.push_sse(&event) {
Ok(frames) => {
if self.encoder.is_completed() {
self.finish_completed(frames).await;
} else {
self.pending.extend(frames);
}
}
Err(error) => {
self.cancel_execution().await;
self.finish_with_encode_error(error);
}
let frames = self.encoder.push_sse(&event);
if self.encoder.is_completed() {
self.finish_completed(frames).await;
} else {
self.pending.extend(frames);
}
}
@@ -607,20 +590,6 @@ impl ResponsesStreamState {
.push_back(Bytes::from_static(DONE_SSE_FRAME.as_bytes()));
self.output_finished = true;
}
fn finish_with_encode_error(&mut self, error: ResponseEncodeError) {
let protocol = error.protocol_body().error;
self.pending
.push_back(Bytes::from(response_failed_sse_event_with_id(
self.encoder.response_id(),
protocol.kind,
protocol.code,
&protocol.message,
)));
self.pending
.push_back(Bytes::from_static(DONE_SSE_FRAME.as_bytes()));
self.output_finished = true;
}
}
impl Drop for ResponsesStreamState {
@@ -17,13 +17,63 @@ pub use response::OpenAiResponsesEncoder;
pub(crate) use websocket::responses_websocket;
pub use websocket::{ResponseCreateFrameError, decode_response_create_with_context};
pub(super) fn safe_response_header_name(name: &str) -> Option<&'static str> {
match name {
"x-request-id" => Some("x-request-id"),
"openai-model" => Some("openai-model"),
"x-models-etag" => Some("x-models-etag"),
"x-reasoning-included" => Some("x-reasoning-included"),
"openai-processing-ms" => Some("openai-processing-ms"),
_ => None,
use gateway_core::event::ProviderResponseHeader;
pub(super) fn response_connection_options(headers: &[ProviderResponseHeader]) -> Vec<String> {
headers
.iter()
.filter(|header| header.name().trim().eq_ignore_ascii_case("connection"))
.filter_map(|header| std::str::from_utf8(header.value()).ok())
.flat_map(|value| value.split(','))
.map(str::trim)
.filter(|name| !name.is_empty())
.map(str::to_ascii_lowercase)
.collect()
}
pub(super) fn response_header_is_forwardable(name: &str, connection_options: &[String]) -> bool {
let name = name.trim().to_ascii_lowercase();
if connection_options
.iter()
.any(|option| option.eq_ignore_ascii_case(&name))
|| name.starts_with("sec-websocket-")
{
return false;
}
!matches!(
name.as_str(),
"connection"
| "keep-alive"
| "proxy-connection"
| "te"
| "trailer"
| "transfer-encoding"
| "upgrade"
| "content-length"
| "content-type"
| "content-encoding"
| "authorization"
| "x-api-key"
| "www-authenticate"
| "authentication-info"
| "proxy-authenticate"
| "proxy-authorization"
| "proxy-authentication-info"
| "cookie"
| "cookie2"
| "set-cookie"
| "set-cookie2"
| "chatgpt-account-id"
| "chatgpt-organization-id"
| "chatgpt-org-id"
| "chatgpt-project-id"
| "openai-organization"
| "openai-project"
| "x-openai-organization"
| "x-openai-project"
| "x-codex-installation-id"
| "x-codex-turn-state"
| "x-codex-turn-metadata"
)
}
@@ -3,6 +3,7 @@
use std::{fmt, net::IpAddr};
use axum::http::HeaderMap;
use base64::{Engine as _, engine::general_purpose::STANDARD};
use gateway_core::operation::{GenerateRequest, Operation, ProtocolPayload, ProviderSessionState};
use gateway_protocol::openai::{
X_OPENAI_INTERNAL_CODEX_RESPONSES_LITE_HEADER, X_OPENAI_MEMGEN_REQUEST_HEADER,
@@ -14,6 +15,7 @@ use super::error::RequestDecodeError;
const OPENAI_PROTOCOL: &str = "openai";
const REVIEW_SUBAGENT: &str = "review";
const OPENAI_SUBAGENT_KEY: &str = "x-openai-subagent";
const PASSTHROUGH_HEADERS_CONTEXT_KEY: &str = "opaque_request_headers";
#[derive(Clone, Default)]
pub struct OpenAiRequestHeaders {
@@ -34,6 +36,7 @@ pub struct OpenAiRequestHeaders {
responses_lite: Option<String>,
memgen_request: Option<String>,
subagent: Option<String>,
passthrough_headers: Vec<Value>,
}
impl OpenAiRequestHeaders {
@@ -57,12 +60,8 @@ impl OpenAiRequestHeaders {
turn_id: header_string(headers, "x-codex-turn-id"),
responses_lite: header_string(headers, X_OPENAI_INTERNAL_CODEX_RESPONSES_LITE_HEADER),
memgen_request: header_string(headers, X_OPENAI_MEMGEN_REQUEST_HEADER),
subagent: header_string(headers, OPENAI_SUBAGENT_KEY).filter(|value| {
matches!(
value.as_str(),
"review" | "compact" | "memory_consolidation" | "collab_spawn"
)
}),
subagent: header_string(headers, OPENAI_SUBAGENT_KEY),
passthrough_headers: passthrough_headers(headers),
}
}
@@ -108,6 +107,12 @@ impl OpenAiRequestHeaders {
insert_protocol_context(&mut context, "turn_id", self.turn_id.as_ref());
insert_protocol_context(&mut context, "responses_lite", self.responses_lite.as_ref());
insert_protocol_context(&mut context, "memgen_request", self.memgen_request.as_ref());
if !self.passthrough_headers.is_empty() {
context.insert(
PASSTHROUGH_HEADERS_CONTEXT_KEY.to_owned(),
Value::Array(self.passthrough_headers.clone()),
);
}
if let Some(use_websocket) = use_websocket {
context.insert("use_websocket".to_owned(), Value::Bool(use_websocket));
}
@@ -322,13 +327,14 @@ pub(super) fn decode_request_object(
review: bool,
request_headers: &OpenAiRequestHeaders,
) -> Result<DecodedResponsesRequest, RequestDecodeError> {
// `use_websocket` 只影响本地 transport。无论值是否可识别,都不得进入上游 body。
let use_websocket = object
.remove("use_websocket")
.and_then(|value| value.as_bool());
// 仅消费已识别的本地 transport 开关;未知同名值保留给未来上游协议。
let use_websocket = object.get("use_websocket").and_then(Value::as_bool);
if use_websocket.is_some() {
object.remove("use_websocket");
}
let model = required_non_empty_string(&object, "model", "model")?.trim();
if model.is_empty() {
let model = required_non_empty_string(&object, "model", "model")?;
if model.trim().is_empty() {
return Err(RequestDecodeError::EmptyField {
field: "model".to_owned(),
});
@@ -345,7 +351,6 @@ pub(super) fn decode_request_object(
let continuation = object
.get("previous_response_id")
.and_then(Value::as_str)
.filter(|response_id| !response_id.is_empty())
.map(|response_id| ContinuationIntent::PreviousResponseId(response_id.to_owned()))
.unwrap_or(ContinuationIntent::None);
@@ -376,26 +381,98 @@ fn inject_subagent_metadata(body: &mut Map<String, Value>, subagent: &str) {
let metadata = body
.entry("client_metadata".to_owned())
.or_insert_with(|| Value::Object(Map::new()));
if !metadata.is_object() {
*metadata = Value::Object(Map::new());
}
if let Some(metadata) = metadata.as_object_mut() {
metadata.insert(
OPENAI_SUBAGENT_KEY.to_owned(),
Value::String(subagent.to_owned()),
);
}
let Some(metadata) = metadata.as_object_mut() else {
return;
};
metadata.insert(
OPENAI_SUBAGENT_KEY.to_owned(),
Value::String(subagent.to_owned()),
);
}
fn header_string(headers: &HeaderMap, name: &str) -> Option<String> {
headers
.get(name)
.and_then(|value| value.to_str().ok())
.map(str::trim)
.filter(|value| !value.is_empty())
.map(ToOwned::to_owned)
}
fn passthrough_headers(headers: &HeaderMap) -> Vec<Value> {
let connection_headers = headers
.get_all("connection")
.iter()
.filter_map(|value| value.to_str().ok())
.flat_map(|value| value.split(','))
.map(str::trim)
.filter(|name| !name.is_empty())
.map(str::to_ascii_lowercase)
.collect::<Vec<_>>();
headers
.iter()
.filter(|(name, _)| passthrough_header_name(name.as_str(), &connection_headers))
.map(|(name, value)| {
Value::Array(vec![
Value::String(name.as_str().to_owned()),
Value::String(STANDARD.encode(value.as_bytes())),
])
})
.collect()
}
fn passthrough_header_name(name: &str, connection_headers: &[String]) -> bool {
if connection_headers
.iter()
.any(|connection_header| connection_header.eq_ignore_ascii_case(name))
|| name.starts_with("sec-websocket-")
|| name.starts_with("x-grok-")
|| name.starts_with("x-xai-")
{
return false;
}
!matches!(
name,
// 逐跳头、上游 authority 和重序列化后的实体长度由 transport 重建。
"connection"
| "keep-alive"
| "proxy-connection"
| "proxy-authenticate"
| "proxy-authorization"
| "te"
| "trailer"
| "transfer-encoding"
| "upgrade"
| "host"
| "content-length"
// 下游连接诊断事实只用于本地观测,不能伪装成 OpenAI 请求事实。
| "forwarded"
| "x-forwarded-for"
| "x-forwarded-host"
| "x-forwarded-proto"
| "x-forwarded-port"
| "x-real-ip"
| "true-client-ip"
| "cf-connecting-ip"
| "x-request-id"
// 下游鉴权和账号 cookie 绝不能成为上游账号身份。
| "authorization"
| "x-api-key"
| "cookie"
| "cookie2"
| "chatgpt-account-id"
| "chatgpt-organization-id"
| "chatgpt-org-id"
| "chatgpt-project-id"
| "openai-organization"
| "openai-project"
| "x-openai-organization"
| "x-openai-project"
// installation ID 始终由当前 lease 重建。
| "x-codex-installation-id"
)
}
fn required_non_empty_string<'a>(
object: &'a Map<String, Value>,
key: &str,
@@ -11,15 +11,14 @@ const OPENAI_PROTOCOL: &str = "openai";
/// OpenAI Responses wire 的转发器。
///
/// OpenAI Provider 与 xAI adapter 都必须交付 OpenAI wire。这里仅保留同一
/// 响应的身份和终态校验,不再从 canonical facts 重建客户端 JSON;因此
/// OpenAI 上游的未知字段、字段顺序和事件语义不会被 API 层二次解释。
/// OpenAI Provider 与 xAI adapter 都必须交付 OpenAI wire。wire 是客户端交付
/// 与终态判断的事实来源;canonical facts 只在 wire 暂未携带 ID 时提供旁路观测,
/// 不得反过来拒绝或改写上游事件。
#[derive(Debug, Default)]
pub struct OpenAiResponsesEncoder {
response_id: Option<String>,
wire_terminal: Option<Value>,
wire_failure: bool,
canonical_completed: bool,
}
impl OpenAiResponsesEncoder {
@@ -30,53 +29,41 @@ impl OpenAiResponsesEncoder {
response_id: None,
wire_terminal: None,
wire_failure: false,
canonical_completed: false,
}
}
/// 消费一个 Provider event,并返回 SSE frames。
///
/// # Errors
///
/// 同一响应身份变化时返回错误。
pub fn push_sse(&mut self, event: &ProviderEvent) -> Result<Vec<Bytes>, ResponseEncodeError> {
self.observe_identity(event)?;
pub fn push_sse(&mut self, event: &ProviderEvent) -> Vec<Bytes> {
self.observe_canonical_identity(event);
let Some(wire) = openai_wire(event) else {
return Ok(Vec::new());
return Vec::new();
};
self.observe_wire_terminal(wire);
self.observe_wire(wire);
if let Some(raw_sse_frame) = wire.raw_sse_frame() {
return Ok(vec![raw_sse_frame.clone()]);
return vec![raw_sse_frame.clone()];
}
Ok(vec![Bytes::from(encode_sse_event_with_metadata(
vec![Bytes::from(encode_sse_event_with_metadata(
wire.event_type().unwrap_or_default(),
&wire.data().to_string(),
wire.sse_id(),
wire.sse_retry(),
))])
))]
}
/// 消费一个 Provider event,并返回 WebSocket JSON messages。
///
/// # Errors
///
/// 同一响应身份变化时返回错误。
pub fn push_websocket(
&mut self,
event: &ProviderEvent,
) -> Result<Vec<String>, ResponseEncodeError> {
self.observe_identity(event)?;
pub fn push_websocket(&mut self, event: &ProviderEvent) -> Vec<String> {
self.observe_canonical_identity(event);
let Some(wire) = openai_wire(event).filter(|wire| wire.has_json_data()) else {
return Ok(Vec::new());
return Vec::new();
};
self.observe_wire_terminal(wire);
Ok(vec![wire.data().to_string()])
self.observe_wire(wire);
vec![wire.data().to_string()]
}
/// 返回是否已经看到同一响应的 canonical 与 wire 终态。
/// 返回是否已经看到客户端可见的 wire 终态。
#[must_use]
pub fn is_completed(&self) -> bool {
self.wire_terminal.is_some() && self.canonical_completed
self.wire_terminal.is_some()
}
/// 返回是否已把 Provider 原生失败 event 交付给客户端。
@@ -95,39 +82,31 @@ impl OpenAiResponsesEncoder {
///
/// # Errors
///
/// 缺少 wire 或 canonical 终态时返回错误。
/// 缺少可转换为非流式响应的 wire 终态时返回错误。
pub fn finish(self) -> Result<Value, ResponseEncodeError> {
if self.canonical_completed {
self.wire_terminal
.ok_or(ResponseEncodeError::MissingWireTerminal)
} else {
Err(ResponseEncodeError::MissingWireTerminal)
}
self.wire_terminal
.ok_or(ResponseEncodeError::MissingWireTerminal)
}
fn observe_identity(&mut self, event: &ProviderEvent) -> Result<(), ResponseEncodeError> {
fn observe_canonical_identity(&mut self, event: &ProviderEvent) {
for fact in event.canonical_facts() {
let metadata = match fact {
GatewayEvent::Started(metadata) => metadata,
GatewayEvent::Completed(metadata) => {
self.canonical_completed = true;
metadata
}
GatewayEvent::Started(metadata) | GatewayEvent::Completed(metadata) => metadata,
_ => continue,
};
if self
.response_id
.as_deref()
.is_some_and(|current| current != metadata.response_id())
{
return Err(ResponseEncodeError::WireIdentityChanged);
}
self.response_id = Some(metadata.response_id().to_owned());
}
Ok(())
}
fn observe_wire_terminal(&mut self, wire: &ProtocolWireEvent) {
fn observe_wire(&mut self, wire: &ProtocolWireEvent) {
if let Some(response_id) = wire
.data()
.pointer("/response/id")
.or_else(|| wire.data().get("response_id"))
.and_then(Value::as_str)
{
self.response_id = Some(response_id.to_owned());
}
let effective_type = wire
.event_type()
.or_else(|| wire.data().get("type").and_then(Value::as_str));
@@ -9,7 +9,7 @@ use axum::{
connect_info::ConnectInfo,
ws::{CloseFrame, Message, WebSocket, WebSocketUpgrade, close_code},
},
http::{HeaderMap, StatusCode},
http::{HeaderMap, HeaderName, StatusCode},
response::{IntoResponse, Response},
};
use futures::StreamExt;
@@ -27,7 +27,6 @@ use crate::{
openai::{
auth::{authenticate_client, authentication_error_response},
error::{gateway_error_contract, gateway_error_from_engine, runtime_unavailable_response},
router::MAX_CLIENT_REQUEST_BODY_BYTES,
service::OpenAiService,
},
};
@@ -102,8 +101,10 @@ impl ResponsesWebSocketAdapter {
_connection_guard: connection_guard,
};
websocket
.max_message_size(MAX_CLIENT_REQUEST_BODY_BYTES)
.max_frame_size(MAX_CLIENT_REQUEST_BODY_BYTES)
// 覆盖 axum/tungstenite 的私有 64 MiB message 与 16 MiB frame 默认值。
// Responses JSON 的协议可接受性由上游决定,代理不另设 wire 长度上限。
.max_message_size(usize::MAX)
.max_frame_size(usize::MAX)
.on_upgrade(move |socket| async move {
serve_responses_websocket(socket, session).await;
})
@@ -303,18 +304,7 @@ async fn forward_execution(
if let Some(update) = event.take_session_update() {
provider_state = Some(update);
}
match encoder.push_websocket(event) {
Ok(messages) => first_messages.extend(messages),
Err(error) => {
return send_protocol_error(
socket,
StatusCode::INTERNAL_SERVER_ERROR,
error.protocol_body(),
&request_id,
)
.await;
}
}
first_messages.extend(encoder.push_websocket(event));
}
if first_messages.is_empty() {
let error = GatewayError::new(
@@ -336,9 +326,7 @@ async fn forward_execution(
{
return outcome;
}
if let Err(error) = commit_connection_replay(replay, &encoder, provider_state.take()) {
return send_gateway_error(socket, &error, &request_id).await;
}
commit_connection_replay(replay, &encoder, provider_state.take());
if !send_text(
socket,
response_metadata_event(&request_id, &response_headers),
@@ -378,18 +366,7 @@ async fn forward_execution(
if let Some(update) = event.take_session_update() {
provider_state = Some(update);
}
match encoder.push_websocket(event) {
Ok(encoded) => messages.extend(encoded),
Err(error) => {
return send_protocol_error(
socket,
StatusCode::INTERNAL_SERVER_ERROR,
error.protocol_body(),
&request_id,
)
.await;
}
}
messages.extend(encoder.push_websocket(event));
}
if encoder.is_completed() {
if let Err(outcome) =
@@ -397,11 +374,7 @@ async fn forward_execution(
{
return outcome;
}
if let Err(error) =
commit_connection_replay(replay, &encoder, provider_state.take())
{
return send_gateway_error(socket, &error, &request_id).await;
}
commit_connection_replay(replay, &encoder, provider_state.take());
if !send_messages(socket, messages).await {
return ForwardOutcome::Disconnect;
}
@@ -413,9 +386,17 @@ async fn forward_execution(
}
}
ActiveInput::Event(Ok(None)) => {
if execution
.session_mut()
.is_some_and(|session| session.is_finalized())
{
commit_connection_replay(replay, &encoder, provider_state.take());
execution.disarm();
return ForwardOutcome::Continue;
}
let error = GatewayError::new(
GatewayErrorKind::Internal,
"gateway response ended without a terminal event",
"gateway response ended without finalizing the execution",
);
return send_gateway_error(socket, &error, &request_id).await;
}
@@ -441,15 +422,10 @@ fn commit_connection_replay(
replay: &mut ConnectionReplaySnapshot,
encoder: &OpenAiResponsesEncoder,
provider_state: Option<ProviderSessionState>,
) -> Result<(), GatewayError> {
let response_id = encoder.response_id().ok_or_else(|| {
GatewayError::new(
GatewayErrorKind::Internal,
"gateway completed a response without an identity",
)
})?;
replay.commit(response_id.to_owned(), provider_state);
Ok(())
) {
if let Some(response_id) = encoder.response_id() {
replay.commit(response_id.to_owned(), provider_state);
}
}
async fn confirm_completed_execution(
@@ -595,21 +571,24 @@ fn response_metadata_event(
request_id: &str,
response_headers: &[ProviderResponseHeader],
) -> String {
let mut headers = response_headers
.iter()
.filter_map(|header| {
super::safe_response_header_name(header.name()).map(|name| {
(
name.to_owned(),
Value::String(header.value().as_str().to_owned()),
)
})
})
.collect::<Map<String, Value>>();
headers.insert(
"x-request-id".to_owned(),
Value::String(request_id.to_owned()),
);
let connection_options = super::response_connection_options(response_headers);
let mut headers = Map::new();
for header in response_headers {
if !super::response_header_is_forwardable(header.name(), &connection_options) {
continue;
}
let Ok(name) = HeaderName::from_bytes(header.name().as_bytes()) else {
continue;
};
let Ok(value) = std::str::from_utf8(header.value()) else {
continue;
};
// 官方 response.metadata 使用 string map,无法表达同名多值;后值保持既有覆盖语义。
headers.insert(name.as_str().to_owned(), Value::String(value.to_owned()));
}
headers
.entry("x-request-id".to_owned())
.or_insert_with(|| Value::String(request_id.to_owned()));
json!({
"type": "response.metadata",
"headers": headers,
@@ -13,9 +13,6 @@ use super::{
use crate::ApiState;
/// OpenAI 请求正文上限。
pub const MAX_CLIENT_REQUEST_BODY_BYTES: usize = 16 * 1024 * 1024;
/// 构造 OpenAI 客户端协议路由。
pub(crate) fn router() -> Router<ApiState> {
Router::new()
@@ -26,5 +23,7 @@ pub(crate) fn router() -> Router<ApiState> {
.route("/v1/models/{model_id}/info", get(model_info))
// 官方 OpenAI 模型详情合同使用 path ID;它不属于 Admin API 约束。
.route("/v1/models/{model_id}", get(model_detail))
.layer(DefaultBodyLimit::max(MAX_CLIENT_REQUEST_BODY_BYTES))
// Responses 正文属于 OpenAI/Codex 协议;代理不能用私有大小上限提前拒绝
// 上游本可接受的未来 payload。
.layer(DefaultBodyLimit::disable())
}
@@ -81,16 +81,10 @@ impl OpenAiService {
"requested model was not found",
)
})?;
if !self.execution.contains_public_model(&client, &public_model) {
return Err(GatewayError::new(
GatewayErrorKind::ModelNotFound,
"requested model was not found",
));
}
let previous_response_id = match metadata.continuation() {
ContinuationIntent::None => None,
ContinuationIntent::PreviousResponseId(value) => {
PreviousResponseId::new(value.clone()).ok()
Some(PreviousResponseId::new(value.clone()))
}
};
self.execution
@@ -105,12 +105,15 @@ mod actions {
use gateway_admin::model::{
Revision,
accounts::{AccountConnectionTestEvent as DomainConnectionTestEvent, AccountStatus},
provider_credentials::{CredentialDeletionResult, CredentialImportResult},
provider_credentials::{
CredentialDeletionResult, CredentialImportResult, CredentialMutationResult,
},
};
use gateway_api::admin::accounts::{
AccountActionRequest, AccountConnectionTestEvent, AccountDeletionData,
AccountDeletionRequest, AccountExportData, AccountExportQuery, AccountIdQuery,
AccountImportData, AccountImportRequest, AccountRefreshRequest, AccountTestQuery,
AccountImportData, AccountImportRequest, AccountMutationData, AccountRefreshRequest,
AccountTestQuery, RotateAccountRequest, StartAccountAuthorizationRequest,
};
use gateway_core::engine::credential::ProviderAccountId;
use serde_json::json;
@@ -147,7 +150,7 @@ mod actions {
}
#[test]
fn account_actions_should_require_frozen_account_ids_and_revision() {
fn account_actions_should_require_frozen_account_ids_and_reject_unknown_revision() {
let id: AccountIdQuery =
serde_json::from_value(json!({ "accountId": "acct_1" })).expect("decode ID query");
assert!(id.validate().is_ok());
@@ -155,13 +158,68 @@ mod actions {
serde_json::from_value(json!({ "accountId": "legacy-id" })).expect("decode action");
assert_eq!(action.validate().unwrap_err().field(), "accountId");
let refresh: AccountRefreshRequest = serde_json::from_value(json!({
"accountId": "acct_1",
"expectedConfigRevision": 0
"accountId": "acct_1"
}))
.expect("decode refresh");
assert!(refresh.validate().is_ok());
assert!(
serde_json::from_value::<AccountRefreshRequest>(json!({
"accountId": "acct_1",
"expectedConfigRevision": 0
}))
.is_err()
);
}
#[test]
fn credential_recovery_requests_should_not_accept_client_revision_fences() {
let authorization: StartAccountAuthorizationRequest = serde_json::from_value(json!({
"provider": "openai",
"name": "reauthorize",
"accountId": "acct_1"
}))
.expect("decode reauthorization");
assert!(authorization.validate().is_ok());
assert!(
serde_json::from_value::<StartAccountAuthorizationRequest>(json!({
"provider": "openai",
"name": "reauthorize",
"accountId": "acct_1",
"expectedCredentialRevision": 1
}))
.is_err()
);
let rotation: RotateAccountRequest = serde_json::from_value(json!({
"provider": "openai",
"accountId": "acct_1",
"accessToken": "header.payload.signature",
"refreshToken": "refresh-token"
}))
.expect("decode rotation");
assert!(rotation.validate().is_ok());
assert!(
serde_json::from_value::<RotateAccountRequest>(json!({
"provider": "openai",
"accountId": "acct_1",
"expectedCredentialRevision": 1,
"accessToken": "header.payload.signature"
}))
.is_err()
);
}
#[test]
fn credential_mutation_response_should_not_expose_internal_revision() {
let response = AccountMutationData::from(CredentialMutationResult {
config_revision: Revision::new(8).expect("config revision"),
account_id: ProviderAccountId::new("acct_1").expect("account ID"),
credential_revision: Some(Revision::new(9).expect("credential revision")),
});
assert_eq!(
refresh.validate().unwrap_err().field(),
"expectedConfigRevision"
serde_json::to_value(response).expect("serialize credential mutation"),
json!({ "accountId": "acct_1" })
);
}
@@ -169,7 +227,6 @@ mod actions {
fn account_import_should_use_provider_and_opaque_data_fields() {
let valid: AccountImportRequest = serde_json::from_value(json!({
"provider": "openai",
"expectedConfigRevision": 7,
"data": {
"providerOwnedUnknownField": {"nested": [1, 2, 3]},
"accounts": [{"credentials": {"access_token": "provider-validates-this"}}]
@@ -180,7 +237,6 @@ mod actions {
let invalid: AccountImportRequest = serde_json::from_value(json!({
"provider": "xai",
"expectedConfigRevision": 7,
"data": []
}))
.expect("decode invalid account import");
@@ -188,27 +244,32 @@ mod actions {
assert!(
serde_json::from_value::<AccountImportRequest>(json!({
"provider": "openai",
"expectedConfigRevision": 7,
"document": {}
}))
.is_err()
);
assert!(
serde_json::from_value::<AccountImportRequest>(json!({
"provider": "openai",
"expectedConfigRevision": 7,
"data": {}
}))
.is_err()
);
}
#[test]
fn account_deletion_should_validate_one_provider_batch_and_emit_account_ids() {
let request: AccountDeletionRequest = serde_json::from_value(json!({
"provider": "xai",
"accountIds": ["acct_1", "acct_2"],
"expectedConfigRevision": 8
"accountIds": ["acct_1", "acct_2"]
}))
.expect("decode account deletion");
assert!(request.validate().is_ok());
let duplicate: AccountDeletionRequest = serde_json::from_value(json!({
"provider": "xai",
"accountIds": ["acct_1", "acct_1"],
"expectedConfigRevision": 8
"accountIds": ["acct_1", "acct_1"]
}))
.expect("decode duplicate account deletion");
assert_eq!(duplicate.validate().unwrap_err().field(), "accountIds");
@@ -223,7 +284,6 @@ mod actions {
assert_eq!(
serde_json::to_value(response).expect("serialize account deletion"),
json!({
"configRevision": 9,
"deletedCount": 2,
"accountIds": ["acct_1", "acct_2"]
})
@@ -239,7 +299,6 @@ mod actions {
assert_eq!(
serde_json::to_value(response).expect("serialize account import"),
json!({
"configRevision": 8,
"importedCount": 1,
"accountIds": ["acct_imported"]
})
@@ -278,6 +337,9 @@ mod actions {
message: "upstream unavailable".to_owned(),
provider_error_code: Some("usage_exhausted".to_owned()),
provider_error_type: Some("invalid_request_error".to_owned()),
upstream_status: Some(429),
upstream_content_type: Some("application/json".to_owned()),
upstream_body: Some(r#"{"error":{"type":"usage_limit_reached"}}"#.to_owned()),
account_status: AccountStatus::Active,
},
]
@@ -309,6 +371,9 @@ mod actions {
"error": "upstream unavailable",
"providerErrorCode": "usage_exhausted",
"providerErrorType": "invalid_request_error",
"upstreamStatus": 429,
"upstreamContentType": "application/json",
"upstreamBody": r#"{"error":{"type":"usage_limit_reached"}}"#,
"accountStatus": "active"
}),
]
@@ -1,6 +1,6 @@
use chrono::{DateTime, TimeZone as _, Utc};
use gateway_api::admin::client_keys::{
self, ClientKeyCursorData, ClientKeyCursorValue, ClientKeyListData, ClientKeyRevisionRequest,
self, ClientKeyCursorData, ClientKeyCursorValue, ClientKeyListData, ClientKeyMutationRequest,
ClientKeySort, ClientKeySortDirection, ClientKeySortField, ClientKeyView,
CreateClientKeyRequest, CreatedClientKeyData, ListClientKeysQuery, MutatedClientKeyData,
RevealedClientKeyData, UpdateClientKeyRequest, decode_client_key_cursor,
@@ -101,9 +101,8 @@ fn client_key_queries_should_reject_unknown_zero_and_oversized_fields() {
}
#[test]
fn client_key_mutations_should_validate_revision_text_limits_and_unknown_fields() {
fn client_key_mutations_should_validate_text_limits_and_unknown_fields() {
let valid = serde_json::from_value::<CreateClientKeyRequest>(json!({
"expectedConfigRevision": 7,
"name": "terminal key",
"label": "production",
"providerKind": "openai",
@@ -113,22 +112,11 @@ fn client_key_mutations_should_validate_revision_text_limits_and_unknown_fields(
.expect("deserialize create")
.into_command()
.expect("validate create");
assert_eq!(valid.expected_config_revision.get(), 7);
assert_eq!(valid.name, "terminal key");
for (payload, field) in [
(
json!({
"expectedConfigRevision": 0,
"name": "key",
"providerKind": "openai",
"maxConcurrency": 0,
"requestsPerMinute": 0
}),
"expectedConfigRevision",
),
(
json!({
"expectedConfigRevision": 1,
"name": " ",
"providerKind": "openai",
"maxConcurrency": 0,
@@ -138,7 +126,6 @@ fn client_key_mutations_should_validate_revision_text_limits_and_unknown_fields(
),
(
json!({
"expectedConfigRevision": 1,
"name": "key",
"providerKind": "openai",
"maxConcurrency": u64::MAX,
@@ -170,14 +157,23 @@ fn client_key_mutations_should_validate_revision_text_limits_and_unknown_fields(
}))
.is_err()
);
let revision = serde_json::from_value::<ClientKeyRevisionRequest>(json!({
"id": "key_1",
"expectedConfigRevision": 3
assert!(
serde_json::from_value::<CreateClientKeyRequest>(json!({
"expectedConfigRevision": 7,
"name": "terminal key",
"providerKind": "openai",
"maxConcurrency": 2,
"requestsPerMinute": 60
}))
.is_err()
);
let mutation_id = serde_json::from_value::<ClientKeyMutationRequest>(json!({
"id": "key_1"
}))
.expect("deserialize revision mutation")
.into_parts()
.expect("validate revision mutation");
assert_eq!(revision, ("key_1".to_owned(), 3));
.expect("deserialize mutation")
.into_id()
.expect("validate mutation");
assert_eq!(mutation_id, "key_1");
}
#[test]
@@ -225,9 +221,9 @@ fn client_key_responses_should_keep_shape_and_redact_creation_debug() {
updated_at: created_at,
last_used_at: Some(created_at),
});
let list = serde_json::to_value(ClientKeyListData::new(9, vec![view], None, 1))
.expect("serialize list");
assert_eq!(list["configRevision"], 9);
let list =
serde_json::to_value(ClientKeyListData::new(vec![view], None, 1)).expect("serialize list");
assert!(list.get("configRevision").is_none());
assert_eq!(list["total"], 1);
assert_eq!(list["items"][0]["id"], "key_visible");
assert_eq!(list["items"][0]["providerKind"], "openai");
@@ -248,7 +244,6 @@ fn client_key_responses_should_keep_shape_and_redact_creation_debug() {
let plaintext = format!("sk_{}", "a".repeat(43));
let created = CreatedClientKeyData::new(
10,
"key_created".to_owned(),
"sk_aaaaaaaaa".to_owned(),
plaintext.clone(),
@@ -263,9 +258,9 @@ fn client_key_responses_should_keep_shape_and_redact_creation_debug() {
json!({ "id": "key_created", "plaintextKey": plaintext })
);
assert_eq!(
serde_json::to_value(MutatedClientKeyData::new(11, "key_created".to_owned()))
serde_json::to_value(MutatedClientKeyData::new("key_created".to_owned()))
.expect("serialize mutation"),
json!({ "configRevision": 11, "id": "key_created" })
json!({ "id": "key_created" })
);
}
@@ -58,9 +58,8 @@ use gateway_api::admin::AdminSessionState;
use gateway_core::{
engine::{
credential::ProviderAccountId,
probe::{AccountProbe, AccountProbeRequest, AccountProbeResult},
probe::{AccountProbe, AccountProbeError, AccountProbeRequest, AccountProbeResult},
},
error::GatewayError,
policy::{ClientApiKeyId, RateLimits},
routing::{
ConfigRevision, ProviderKind, PublicModelId, UpstreamModelId, snapshot::SnapshotControl,
@@ -279,8 +278,9 @@ impl SettingsStore for MemorySettingsStore {
command: ReplaceRuntimeSettings,
_: &MutationContext,
) -> AdminStoreResult<RuntimeSettings> {
let mut settings = self.settings.lock().expect("settings");
let updated = RuntimeSettings {
config_revision: next_revision(command.expected_config_revision),
config_revision: next_revision(settings.config_revision),
model_mappings: command.model_mappings,
refresh_margin_seconds: command.refresh_margin_seconds,
refresh_concurrency: command.refresh_concurrency,
@@ -292,31 +292,31 @@ impl SettingsStore for MemorySettingsStore {
audit_retention_days: command.audit_retention_days,
updated_at: Utc::now(),
};
*self.settings.lock().expect("settings") = updated.clone();
*settings = updated.clone();
Ok(updated)
}
async fn replace_admin_api_key(
&self,
expected_config_revision: Revision,
key: AdminApiKey,
_: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation> {
*self.api_key.lock().expect("API key") = Some(key);
let revision = self.settings.lock().expect("settings").config_revision;
Ok(AdminApiKeyMutation {
config_revision: next_revision(expected_config_revision),
config_revision: next_revision(revision),
exists: true,
})
}
async fn delete_admin_api_key(
&self,
expected_config_revision: Revision,
_: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation> {
*self.api_key.lock().expect("API key") = None;
let revision = self.settings.lock().expect("settings").config_revision;
Ok(AdminApiKeyMutation {
config_revision: next_revision(expected_config_revision),
config_revision: next_revision(revision),
exists: false,
})
}
@@ -674,7 +674,7 @@ impl AccountProbe for NoopProbe {
fn probe(
&self,
_: AccountProbeRequest,
) -> BoxFuture<'_, Result<AccountProbeResult, GatewayError>> {
) -> BoxFuture<'_, Result<AccountProbeResult, AccountProbeError>> {
Box::pin(async { panic!("unused account probe") })
}
}
@@ -326,6 +326,7 @@ async fn usage_route_should_expose_image_and_websocket_facts() {
upstream_transport: Some("http_sse".to_owned()),
http_version: Some("h2".to_owned()),
websocket_pool: Some("reuse".to_owned()),
service_tier: Some("priority".to_owned()),
provider_metadata_json: Some(
serde_json::json!({
"effectiveModel": "grok-4.5",
@@ -402,6 +403,7 @@ async fn usage_route_should_expose_image_and_websocket_facts() {
assert_eq!(
serde_json::json!({
"route": value["data"]["items"][0]["route"],
"serviceTier": value["data"]["items"][0]["serviceTier"],
"authenticationKind": value["data"]["items"][0]["authenticationKind"],
"imageInputTokens": value["data"]["items"][0]["tokenDetails"]["imageInputTokens"],
"imageOutputTokens": value["data"]["items"][0]["tokenDetails"]["imageOutputTokens"],
@@ -428,6 +430,7 @@ async fn usage_route_should_expose_image_and_websocket_facts() {
}),
serde_json::json!({
"route": "/v1/responses",
"serviceTier": "priority",
"authenticationKind": "oauth",
"imageInputTokens": 31,
"imageOutputTokens": 9,
@@ -37,7 +37,6 @@ async fn response_json(response: axum::response::Response) -> Value {
fn update_body() -> Value {
json!({
"expectedConfigRevision": 7,
"modelMappings": {
"gpt-5.4": "gpt-5.5",
"grok-latest": "grok-4.5"
@@ -64,22 +63,11 @@ fn settings_request_should_reject_unknown_rotation_strategy() {
}
#[test]
fn settings_request_should_require_positive_revision() {
fn settings_request_should_reject_unknown_revision_field() {
let mut body = update_body();
body["expectedConfigRevision"] = json!(0);
let request: UpdateRuntimeSettingsRequest =
serde_json::from_value(body).expect("decode settings");
assert_eq!(
request.validate().unwrap_err().field(),
"expectedConfigRevision"
);
body["expectedConfigRevision"] = json!(7);
let mut missing = update_body();
missing
.as_object_mut()
.expect("settings object")
.remove("expectedConfigRevision");
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(missing).is_err());
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(body).is_err());
}
#[test]
@@ -124,7 +112,7 @@ async fn settings_post_should_replace_global_model_mappings() {
.expect("settings update response");
let data = response_json(response).await["data"].clone();
assert_eq!(data["configRevision"], 8);
assert!(data.get("configRevision").is_none());
assert_eq!(data["modelMappings"]["gpt-5.4"], "gpt-5.5");
assert_eq!(data["modelMappings"]["grok-latest"], "grok-4.5");
}
@@ -135,6 +135,35 @@ fn local_provider_capacity_exhaustion_should_map_to_service_unavailable() {
);
}
#[test]
fn local_provider_capacity_exhaustion_should_preserve_safe_provider_detail() {
let detail = ClientVisibleUpstreamError::new(
"no account is eligible for the requested model",
Some("no_eligible_account".to_owned()),
Some("account_unavailable_error".to_owned()),
);
let error = EngineError::Provider(
ProviderError::new(ProviderErrorKind::Unavailable, UpstreamSendState::NotSent)
.with_client_visible_upstream_error(detail),
);
let gateway = gateway_error_from_engine(&error);
assert_eq!(gateway.kind(), GatewayErrorKind::NoAvailableProvider);
assert_eq!(
gateway.client_message(),
"no account is eligible for the requested model"
);
assert_eq!(gateway.client_error_code(), Some("no_eligible_account"));
assert_eq!(
gateway.client_error_type(),
Some("account_unavailable_error")
);
assert_eq!(
gateway.safe_message(),
"no upstream provider is currently available for this request"
);
}
#[test]
fn no_eligible_provider_account_should_map_to_service_unavailable() {
let error = EngineError::Provider(ProviderError::new(
@@ -297,14 +326,11 @@ fn openai_error_response_should_preserve_only_safe_contract_fields() {
async fn gateway_error_response_should_expose_only_structured_client_visible_upstream_fields() {
let error = GatewayError::from_provider(
&ProviderError::new(ProviderErrorKind::QuotaExhausted, UpstreamSendState::Sent)
.with_client_visible_upstream_error(
ClientVisibleUpstreamError::new(
"Your Codex quota is exhausted",
Some("quota_exhausted".to_owned()),
Some("rate_limit_error".to_owned()),
)
.expect("safe structured upstream error"),
),
.with_client_visible_upstream_error(ClientVisibleUpstreamError::new(
"Your Codex quota is exhausted",
Some("quota_exhausted".to_owned()),
Some("rate_limit_error".to_owned()),
)),
);
let response = gateway_error_response(&error);
@@ -10,7 +10,9 @@ use gateway_core::engine::execution::{
StartedExecution,
};
use gateway_core::error::{GatewayError, GatewayErrorKind};
use gateway_core::routing::{ModelPresentation, PublicModelId, PublicModelProfile};
use gateway_core::routing::{
ModelPresentation, ModelServiceTier, PublicModelId, PublicModelProfile,
};
use tower::ServiceExt;
use super::{api_router, authenticated_client};
@@ -73,7 +75,15 @@ impl ExecutionService for ModelsExecution {
)
.with_context_window_tokens(Some(500_000))
.with_image_input(true)
.with_agent_tools(true, true),
.with_agent_tools(true, true)
.with_service_tiers(vec![
ModelServiceTier::new(
"priority",
"Fast",
"Route the request through Codex fast mode.",
)
.with_speed_tier("fast"),
]),
)]
}
@@ -145,7 +155,15 @@ async fn models_should_encode_provider_profiles_for_current_codex_clients() {
assert_eq!(model["default_reasoning_level"], "medium");
assert_eq!(model["context_window"], 500_000);
assert_eq!(model["apply_patch_tool_type"], "freeform");
assert_eq!(model["service_tiers"], serde_json::json!([]));
assert_eq!(model["additional_speed_tiers"], serde_json::json!(["fast"]));
assert_eq!(
model["service_tiers"],
serde_json::json!([{
"id": "priority",
"name": "Fast",
"description": "Route the request through Codex fast mode."
}])
);
assert_eq!(
model["supported_reasoning_levels"],
serde_json::json!([
@@ -8,7 +8,10 @@ use std::sync::{
use axum::{
body::{Body, to_bytes},
extract::connect_info::ConnectInfo,
http::{HeaderMap, Request, StatusCode, header::AUTHORIZATION},
http::{
HeaderMap, HeaderValue, Request, StatusCode,
header::{AUTHORIZATION, CONTENT_TYPE},
},
};
use bytes::Bytes;
use futures::future::{BoxFuture, pending};
@@ -18,8 +21,8 @@ use gateway_core::engine::execution::{
};
use gateway_core::engine::{CommitRequirement, CoordinatedEvent, EngineError, UpstreamSendState};
use gateway_core::error::{
ClientVisibleUpstreamError, GatewayError, GatewayErrorKind, ProviderError, ProviderErrorKind,
SafeUpstreamValue,
ClientVisibleUpstreamError, ClientVisibleUpstreamResponse, GatewayError, GatewayErrorKind,
ProviderError, ProviderErrorKind,
};
use gateway_core::event::{
ContentItem, ContentKind, GatewayEvent, ProtocolWireEvent, ProviderEvent,
@@ -101,6 +104,8 @@ struct ContextCaptureExecution {
#[derive(Debug, Clone, PartialEq)]
struct CapturedClientContext {
public_model: String,
wire_model: Option<String>,
client_ip: Option<IpAddr>,
user_agent: Option<String>,
endpoint: String,
@@ -109,6 +114,7 @@ struct CapturedClientContext {
client_metadata: Option<Value>,
protocol_context: Option<Value>,
prompt_cache_key: Option<String>,
previous_response_id: Option<String>,
}
impl ExecutionService for ContextCaptureExecution {
@@ -124,11 +130,14 @@ impl ExecutionService for ContextCaptureExecution {
}
fn public_models(&self, _: &AuthenticatedClient) -> Vec<PublicModelId> {
Vec::new()
["model-a", "model-b"]
.into_iter()
.map(|model| PublicModelId::new(model).expect("catalog model"))
.collect()
}
fn contains_public_model(&self, _: &AuthenticatedClient, _: &PublicModelId) -> bool {
true
fn contains_public_model(&self, _: &AuthenticatedClient, model: &PublicModelId) -> bool {
matches!(model.as_str(), "model-a" | "model-b")
}
fn start(
@@ -136,11 +145,16 @@ impl ExecutionService for ContextCaptureExecution {
request: StartExecution,
) -> BoxFuture<'_, Result<StartedExecution, GatewayError>> {
Box::pin(async move {
let public_model = request.public_model.as_str().to_owned();
let operation_kind = request.operation.kind();
let generation = match &request.operation {
Operation::Generate(generation) => Some(generation),
_ => None,
};
let wire_model = generation
.and_then(|generation| generation.protocol_payload().body().get("model"))
.and_then(Value::as_str)
.map(ToOwned::to_owned);
let (client_metadata, protocol_context, prompt_cache_key, input) =
generation.map_or((None, None, None, None), |generation| {
let payload = generation.protocol_payload();
@@ -152,7 +166,14 @@ impl ExecutionService for ContextCaptureExecution {
payload.body().get("input").cloned(),
)
});
let previous_response_id = request
.metadata
.previous_response_id
.as_ref()
.map(|value| value.as_str().to_owned());
*self.observed.lock().expect("context capture lock") = Some(CapturedClientContext {
public_model,
wire_model,
client_ip: request.metadata.client_ip,
user_agent: request.metadata.user_agent,
endpoint: request.metadata.endpoint,
@@ -161,6 +182,7 @@ impl ExecutionService for ContextCaptureExecution {
client_metadata,
protocol_context,
prompt_cache_key,
previous_response_id,
});
Err(GatewayError::new(
GatewayErrorKind::Internal,
@@ -503,14 +525,19 @@ async fn request_context_should_resolve_forwarded_precedence_and_peer_fallback()
assert_eq!(
captured_client_context(headers, peer).await,
CapturedClientContext {
public_model: "smart-code".to_owned(),
wire_model: Some("smart-code".to_owned()),
client_ip: Some("198.51.100.1".parse().expect("expected IP")),
user_agent: Some("Codex-CLI/1.0".to_owned()),
endpoint: "/v1/responses".to_owned(),
operation_kind: OperationKind::Generate,
input: Some(json!("hello")),
client_metadata: None,
protocol_context: None,
protocol_context: Some(json!({
"opaque_request_headers": [["user-agent", "IENvZGV4LUNMSS8xLjAg"]]
})),
prompt_cache_key: None,
previous_response_id: None,
}
);
@@ -533,14 +560,58 @@ async fn request_context_should_resolve_forwarded_precedence_and_peer_fallback()
}
#[tokio::test]
async fn http_request_should_forward_safe_codex_headers_without_projecting_xai_headers() {
async fn stale_model_catalog_should_not_block_a_new_model_request() {
let model = " gpt-future-codex ";
let captured = captured_http_request(
"openai",
json!({"model": model, "input": "hello"}),
HeaderMap::new(),
None,
)
.await;
assert_eq!(captured.public_model, model);
assert_eq!(captured.wire_model.as_deref(), Some(model));
}
#[tokio::test]
async fn http_request_should_pass_opaque_previous_response_id_to_core() {
for response_id in [
format!("resp_{}", "x".repeat(257)),
"resp_control\0opaque".to_owned(),
String::new(),
] {
let captured = captured_http_request(
"openai",
json!({
"model": "smart-code",
"input": "continue",
"previous_response_id": response_id.clone()
}),
HeaderMap::new(),
None,
)
.await;
assert_eq!(
captured.previous_response_id.as_deref(),
Some(response_id.as_str())
);
}
}
#[tokio::test]
async fn http_request_should_forward_codex_headers_without_projecting_xai_headers() {
let peer = "192.0.2.10:443".parse().expect("peer address");
let mut headers = HeaderMap::new();
headers.insert("x-codex-turn-state", "turn-state".parse().expect("header"));
headers.insert("conversation-id", "conversation-1".parse().expect("header"));
headers.insert("session-id", "session-1".parse().expect("header"));
headers.insert("x-grok-turn-idx", "7".parse().expect("header"));
headers.insert("x-openai-subagent", "compact".parse().expect("header"));
headers.insert(
"x-openai-subagent",
"future_codex_mode".parse().expect("header"),
);
headers.insert(
"x-openai-internal-codex-responses-lite",
"true".parse().expect("header"),
@@ -566,10 +637,33 @@ async fn http_request_should_forward_safe_codex_headers_without_projecting_xai_h
.client_metadata
.as_ref()
.and_then(|metadata| metadata.get("x-openai-subagent")),
Some(&json!("compact"))
Some(&json!("future_codex_mode"))
);
}
#[tokio::test]
async fn subagent_header_should_not_replace_non_object_client_metadata() {
let mut headers = HeaderMap::new();
headers.insert(
"x-openai-subagent",
"future_codex_mode".parse().expect("header"),
);
let captured = captured_http_request(
"openai",
json!({
"model": "smart-code",
"input": "preserve metadata",
"client_metadata": "opaque-client-value"
}),
headers,
None,
)
.await;
assert_eq!(captured.client_metadata, Some(json!("opaque-client-value")));
}
#[tokio::test]
async fn xai_private_headers_should_not_enter_openai_request_facts() {
let peer = "192.0.2.10:443".parse().expect("peer address");
@@ -632,6 +726,43 @@ async fn review_route_should_inject_review_subagent_and_record_its_endpoint() {
);
}
#[tokio::test]
async fn review_route_should_not_replace_non_object_client_metadata() {
let observed = Arc::new(Mutex::new(None));
let execution = Arc::new(ContextCaptureExecution {
observed: Arc::clone(&observed),
client: authenticated_client("sk_context_test"),
});
let request = Request::post("/v1/responses/review")
.header(AUTHORIZATION, "Bearer sk_context_test")
.body(Body::from(
json!({
"model": "smart-code",
"input": "review this",
"client_metadata": ["opaque", "client", "value"]
})
.to_string(),
))
.expect("review request");
let response = api_router(execution)
.await
.oneshot(request)
.await
.expect("review response");
let captured = observed
.lock()
.expect("review capture lock")
.clone()
.expect("captured review request");
assert_eq!(response.status(), StatusCode::INTERNAL_SERVER_ERROR);
assert_eq!(
captured.client_metadata,
Some(json!(["opaque", "client", "value"]))
);
}
#[tokio::test]
async fn streaming_encodes_first_frame_before_commit_and_http_delivery() {
let trace = Arc::new(Trace::default());
@@ -653,19 +784,22 @@ async fn streaming_encodes_first_frame_before_commit_and_http_delivery() {
}
#[tokio::test]
async fn streaming_response_forwards_only_allowlisted_provider_headers() {
async fn streaming_response_should_append_ordinary_headers_and_skip_unrepresentable_or_sensitive_ones()
{
let trace = Arc::new(Trace::default());
let headers = vec![
ProviderResponseHeader::new(
"x-models-etag",
SafeUpstreamValue::new("models-v2").expect("etag"),
)
.expect("safe header"),
ProviderResponseHeader::new("x-models-etag", Bytes::from_static(b"models-v2")),
ProviderResponseHeader::new("x-future-multi", Bytes::from_static(b"first")),
ProviderResponseHeader::new("x-future-multi", Bytes::from_static(b"second")),
ProviderResponseHeader::new("x-future-bytes", Bytes::from_static(b"\xffopaque")),
ProviderResponseHeader::new(
"authorization",
SafeUpstreamValue::new("should-not-cross-boundary").expect("header value"),
)
.expect("syntactically safe header"),
Bytes::from_static(b"should-not-cross-boundary"),
),
ProviderResponseHeader::new("connection", Bytes::from_static(b"x-hop-secret")),
ProviderResponseHeader::new("x-hop-secret", Bytes::from_static(b"hop-secret")),
ProviderResponseHeader::new("content-type", Bytes::from_static(b"application/private")),
ProviderResponseHeader::new("bad\0name", Bytes::from_static(b"unrepresentable")),
];
let session = FakeSession::streaming(
Arc::clone(&trace),
@@ -685,7 +819,29 @@ async fn streaming_response_forwards_only_allowlisted_provider_headers() {
.and_then(|value| value.to_str().ok()),
Some("models-v2")
);
assert_eq!(
response
.headers()
.get_all("x-future-multi")
.iter()
.map(HeaderValue::as_bytes)
.collect::<Vec<_>>(),
vec![b"first".as_slice(), b"second".as_slice()]
);
assert_eq!(
response
.headers()
.get("x-future-bytes")
.map(HeaderValue::as_bytes),
Some(b"\xffopaque".as_slice())
);
assert!(response.headers().get("authorization").is_none());
assert!(response.headers().get("connection").is_none());
assert!(response.headers().get("x-hop-secret").is_none());
assert_eq!(
response.headers().get(CONTENT_TYPE),
Some(&HeaderValue::from_static("text/event-stream"))
);
std::mem::forget(response);
}
@@ -801,7 +957,107 @@ async fn streaming_rate_limit_before_first_frame_should_persist_the_returned_429
}
#[tokio::test]
async fn streaming_postcommit_encode_failure_cancels_without_second_commit() {
async fn streaming_upstream_http_failure_before_first_frame_should_preserve_raw_response() {
let trace = Arc::new(Trace::default());
let raw_body = Bytes::from_static(
b"{\"error\":{\"message\":\"rate limited\",\"future_field\":{\"kept\":true}},\"top_level\":17}\x00",
);
let error = ProviderError::new(ProviderErrorKind::RateLimited, UpstreamSendState::Sent)
.with_client_visible_upstream_response(
ClientVisibleUpstreamResponse::new(
429,
Some(b"application/problem+json; charset=utf-8".to_vec()),
raw_body.clone(),
)
.with_headers(vec![
ProviderResponseHeader::new("retry-after", Bytes::from_static(b"17")),
ProviderResponseHeader::new("x-request-id", Bytes::from_static(b"req-failure")),
ProviderResponseHeader::new("x-future-error", Bytes::from_static(b"first")),
ProviderResponseHeader::new("x-future-error", Bytes::from_static(b"second")),
ProviderResponseHeader::new("x-future-bytes", Bytes::from_static(b"\xffopaque")),
]),
);
let session = FakeSession::streaming(
Arc::clone(&trace),
vec![NextStep::Error(EngineError::Provider(error))],
);
let response = stream_execution_response(Box::new(session), None).await;
assert_eq!(response.status(), StatusCode::TOO_MANY_REQUESTS);
assert_eq!(
response
.headers()
.get(CONTENT_TYPE)
.map(|value| value.as_bytes()),
Some(b"application/problem+json; charset=utf-8".as_slice())
);
assert_eq!(
response
.headers()
.get("retry-after")
.map(|value| value.as_bytes()),
Some(b"17".as_slice())
);
assert_eq!(
response
.headers()
.get("x-request-id")
.map(|value| value.as_bytes()),
Some(b"req-failure".as_slice())
);
assert_eq!(
response
.headers()
.get_all("x-future-error")
.iter()
.map(|value| value.as_bytes())
.collect::<Vec<_>>(),
vec![b"first".as_slice(), b"second".as_slice()]
);
assert_eq!(
response
.headers()
.get("x-future-bytes")
.map(|value| value.as_bytes()),
Some(b"\xffopaque".as_slice())
);
assert_eq!(
to_bytes(response.into_body(), usize::MAX)
.await
.expect("read raw error body"),
raw_body
);
assert_eq!(trace.client_statuses(), vec![429]);
}
#[tokio::test]
async fn buffered_upstream_http_failure_should_preserve_raw_response() {
let trace = Arc::new(Trace::default());
let raw_body = Bytes::from_static(b"not-json\xffstill-upstream");
let error = ProviderError::new(ProviderErrorKind::Unavailable, UpstreamSendState::Sent)
.with_client_visible_upstream_response(ClientVisibleUpstreamResponse::new(
503,
Some(b"application/octet-stream".to_vec()),
raw_body.clone(),
));
let session = FakeSession::buffered(Arc::clone(&trace), vec![started(), completed()])
.with_collect_error(EngineError::Provider(error));
let response = collect_execution_response(Box::new(session)).await;
assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE);
assert_eq!(
to_bytes(response.into_body(), usize::MAX)
.await
.expect("read raw error body"),
raw_body
);
assert_eq!(trace.client_statuses(), vec![503]);
}
#[tokio::test]
async fn streaming_canonical_identity_change_does_not_interrupt_wire_delivery() {
let trace = Arc::new(Trace::default());
let session = FakeSession::streaming(
Arc::clone(&trace),
@@ -811,7 +1067,7 @@ async fn streaming_postcommit_encode_failure_cancels_without_second_commit() {
mismatched_terminal_event(),
CommitRequirement::AlreadyCommitted,
)),
NextStep::FinalizeCancelled,
NextStep::FinalizeSuccess,
],
);
@@ -820,12 +1076,16 @@ async fn streaming_postcommit_encode_failure_cancels_without_second_commit() {
.await
.expect("read SSE body");
assert!(String::from_utf8_lossy(&body).contains("response.failed"));
let body = String::from_utf8_lossy(&body);
assert!(body.contains("response.completed"));
assert!(body.contains("resp_other"));
assert!(!body.contains("response.failed"));
assert!(body.ends_with("data: [DONE]\n\n"));
assert_eq!(trace.client_statuses(), vec![200]);
assert!(trace.is_cancelled());
assert!(!trace.is_cancelled());
assert_eq!(
trace.snapshot(),
vec!["next_event", "commit", "next_event", "cancel_finalize"]
vec!["next_event", "commit", "next_event", "next_end"]
);
}
@@ -874,6 +1134,38 @@ async fn streaming_success_should_emit_terminal_event_and_done_marker() {
);
}
#[tokio::test]
async fn streaming_finalized_unknown_wire_should_end_cleanly_without_synthetic_failure() {
let trace = Arc::new(Trace::default());
let raw = Bytes::from_static(
b"event: response.future_terminal\ndata: {\"type\":\"response.future_terminal\",\"opaque\":true}\n\n",
);
let event = ProviderEvent::wire(
ProtocolWireEvent::raw_sse("openai", raw.clone()).expect("raw future event"),
);
let session = FakeSession::streaming(
Arc::clone(&trace),
vec![
NextStep::Event(delivery_provider(
event,
CommitRequirement::CommitBeforeDelivery,
)),
NextStep::FinalizeSuccess,
],
);
let response = stream_execution_response(Box::new(session), None).await;
let body = to_bytes(response.into_body(), usize::MAX)
.await
.expect("read transparent SSE body");
assert!(body.as_ref().starts_with(raw.as_ref()));
assert!(body.as_ref().ends_with(b"data: [DONE]\n\n"));
assert!(!String::from_utf8_lossy(&body).contains("response.failed"));
assert!(!trace.is_cancelled());
assert_eq!(trace.snapshot(), vec!["next_event", "commit", "next_end"]);
}
#[tokio::test]
async fn streaming_completed_event_without_finalized_execution_should_fail_closed() {
let trace = Arc::new(Trace::default());
@@ -942,14 +1234,11 @@ async fn streaming_empty_terminal_should_emit_failure_done_and_cancel_execution(
async fn streaming_error_should_emit_client_visible_upstream_details() {
let trace = Arc::new(Trace::default());
let error = ProviderError::new(ProviderErrorKind::RateLimited, UpstreamSendState::Sent)
.with_client_visible_upstream_error(
ClientVisibleUpstreamError::new(
"Your Codex quota is exhausted",
Some("quota_exhausted".to_owned()),
Some("rate_limit_error".to_owned()),
)
.expect("safe structured upstream error"),
);
.with_client_visible_upstream_error(ClientVisibleUpstreamError::new(
"Your Codex quota is exhausted",
Some("quota_exhausted".to_owned()),
Some("rate_limit_error".to_owned()),
));
let session = FakeSession::streaming(
Arc::clone(&trace),
vec![
@@ -1055,6 +1344,86 @@ async fn streaming_upstream_wire_failure_should_not_be_rewritten_as_a_gateway_er
);
}
#[tokio::test]
async fn atomic_uncommitted_upstream_failure_batch_should_be_forwarded_once() {
let trace = Arc::new(Trace::default());
let raw_created = Bytes::from_static(
b"event: response.created\r\ndata: {\"type\":\"response.created\",\"response\":{\"id\":\"resp_atomic_api\",\"model\":\"public-model\",\"status\":\"in_progress\"}}\r\n\r\n",
);
let raw_failure = Bytes::from_static(
b"event: response.failed\r\ndata: {\"type\":\"response.failed\",\"response\":{\"id\":\"resp_atomic_api\",\"status\":\"failed\",\"error\":{\"code\":\"rate_limit_exceeded\",\"message\":\"atomic upstream marker\"}}}\r\n\r\n",
);
let started_wire = ProviderEvent::canonical_with_wire(
vec![started()],
ProtocolWireEvent::json_with_raw_sse_metadata(
"openai",
Some("response.created".to_owned()),
json!({
"type": "response.created",
"response": {
"id": "resp_atomic_api",
"model": "public-model",
"status": "in_progress"
}
}),
raw_created.clone(),
None,
None,
)
.expect("valid upstream started wire"),
);
let failed_wire = ProviderEvent::wire(
ProtocolWireEvent::json_with_raw_sse_metadata(
"openai",
Some("response.failed".to_owned()),
json!({
"type": "response.failed",
"response": {
"id": "resp_atomic_api",
"status": "failed",
"error": {
"code": "rate_limit_exceeded",
"message": "atomic upstream marker"
}
}
}),
raw_failure.clone(),
None,
None,
)
.expect("valid upstream failure wire"),
);
let batch = CoordinatedEvent::try_batch(
vec![started_wire, failed_wire],
CommitRequirement::CommitBeforeDelivery,
)
.expect("atomic failure delivery");
let session = FakeSession::streaming(
Arc::clone(&trace),
vec![
NextStep::Event(batch),
NextStep::Error(EngineError::Provider(ProviderError::new(
ProviderErrorKind::RateLimited,
UpstreamSendState::Sent,
))),
],
);
let response = stream_execution_response(Box::new(session), None).await;
let body = to_bytes(response.into_body(), usize::MAX)
.await
.expect("read SSE body");
let body = String::from_utf8(body.to_vec()).expect("SSE is UTF-8");
assert_eq!(body.matches("atomic upstream marker").count(), 1);
assert_eq!(body.matches("event: response.failed").count(), 1);
assert!(body.contains(std::str::from_utf8(&raw_created).expect("created frame")));
assert!(body.contains(std::str::from_utf8(&raw_failure).expect("failure frame")));
assert!(!body.contains("upstream capacity is temporarily unavailable"));
assert!(body.ends_with("data: [DONE]\n\n"));
assert_eq!(trace.snapshot(), vec!["next_event", "commit", "next_error"]);
}
#[tokio::test]
async fn streaming_second_commit_request_should_fail_and_finalize_once() {
let trace = Arc::new(Trace::default());
@@ -1132,13 +1501,10 @@ async fn buffered_response_commits_only_after_complete_json_is_encoded() {
}
#[tokio::test]
async fn buffered_response_forwards_allowlisted_provider_headers() {
async fn buffered_response_forwards_long_ordinary_provider_header_values() {
let trace = Arc::new(Trace::default());
let header = ProviderResponseHeader::new(
"openai-model",
SafeUpstreamValue::new("gpt-5.5-effective").expect("model header"),
)
.expect("safe header");
let model = format!("gpt-{}", "x".repeat(512));
let header = ProviderResponseHeader::new("openai-model", Bytes::from(model.clone()));
let session = FakeSession::buffered(Arc::clone(&trace), vec![started(), completed()])
.with_response_headers(vec![header]);
@@ -1149,7 +1515,7 @@ async fn buffered_response_forwards_allowlisted_provider_headers() {
.headers()
.get("openai-model")
.and_then(|value| value.to_str().ok()),
Some("gpt-5.5-effective")
Some(model.as_str())
);
}
@@ -2,6 +2,7 @@ mod http;
mod websocket;
use axum::http::{HeaderMap, HeaderValue};
use base64::{Engine as _, engine::general_purpose::STANDARD};
use bytes::Bytes;
use gateway_api::openai::responses::{
ContinuationIntent, DecodedResponsesRequest, OpenAiRequestHeaders, OpenAiResponsesEncoder,
@@ -117,6 +118,28 @@ fn decoder_should_preserve_the_openai_body_and_only_derive_stable_routing_facts(
assert!(decoded.metadata().store());
}
#[test]
fn decoder_should_preserve_opaque_client_model_values() {
for model in [
format!("future-{}", "x".repeat(512)),
"future\0model".to_owned(),
" future-model ".to_owned(),
] {
let decoded = generate_request(json!({
"model": model.clone(),
"input": "hello"
}));
assert_eq!(decoded.metadata().public_model(), model);
assert_eq!(
openai_wire_body(&decoded)
.get("model")
.and_then(Value::as_str),
Some(model.as_str())
);
}
}
#[test]
fn decoder_should_preserve_compaction_trigger_for_the_openai_provider() {
let decoded = generate_request(json!({
@@ -149,6 +172,21 @@ fn decoder_should_keep_transport_override_out_of_the_openai_wire_body() {
);
}
#[test]
fn decoder_should_preserve_unrecognized_use_websocket_values() {
let decoded = generate_request(json!({
"model": "smart-code",
"input": "hello",
"use_websocket": {"future": "transport-mode"}
}));
assert_eq!(
openai_wire_body(&decoded).get("use_websocket"),
Some(&json!({"future": "transport-mode"}))
);
assert!(!openai_protocol_context(&decoded).contains_key("use_websocket"));
}
#[test]
fn decoder_should_preserve_connection_metadata_outside_the_openai_wire_body() {
let mut headers = HeaderMap::new();
@@ -170,10 +208,142 @@ fn decoder_should_preserve_connection_metadata_outside_the_openai_wire_body() {
("turn_state".to_owned(), json!("turn-state")),
("turn_metadata".to_owned(), json!("{\"kind\":\"review\"}")),
("conversation_id".to_owned(), json!("conversation")),
(
"opaque_request_headers".to_owned(),
json!([
["x-codex-turn-state", STANDARD.encode(b"turn-state")],
[
"x-codex-turn-metadata",
STANDARD.encode(br#"{"kind":"review"}"#)
],
["conversation-id", STANDARD.encode(b"conversation")]
]),
),
])
);
}
#[test]
fn decoder_should_preserve_ordinary_request_headers_as_opaque_multivalues() {
let mut headers = HeaderMap::new();
headers.append(
"x-openai-future-mode",
HeaderValue::from_static("future-ascii"),
);
headers.append(
"x-openai-future-mode",
HeaderValue::from_bytes(b"\x80\xff").expect("opaque header bytes"),
);
headers.append("version", HeaderValue::from_static("future-v1"));
headers.append(
"version",
HeaderValue::from_bytes(b"future-\x80").expect("opaque known header bytes"),
);
headers.insert(
"accept",
HeaderValue::from_static("application/vnd.openai.responses+json"),
);
headers.insert(
"content-type",
HeaderValue::from_static("application/vnd.openai.responses.request+json"),
);
headers.insert(
"user-agent",
HeaderValue::from_static("Codex future-client"),
);
headers.insert("originator", HeaderValue::from_static("future-codex"));
headers.append(
"openai-beta",
HeaderValue::from_static("future_responses=v2"),
);
headers.append("openai-beta", HeaderValue::from_static("future_tools=v3"));
headers.insert(
"x-openai-internal-codex-residency",
HeaderValue::from_static("future-region"),
);
headers.insert(
"x-codex-turn-state",
HeaderValue::from_static("same-account-turn"),
);
headers.insert(
"connection",
HeaderValue::from_static("keep-alive, x-private-hop"),
);
headers.insert("x-private-hop", HeaderValue::from_static("drop-me"));
headers.insert("host", HeaderValue::from_static("downstream.invalid"));
headers.insert("content-length", HeaderValue::from_static("999"));
headers.insert("authorization", HeaderValue::from_static("Bearer client"));
headers.insert("x-api-key", HeaderValue::from_static("client-key"));
headers.insert("cookie", HeaderValue::from_static("client=cookie"));
headers.insert(
"chatgpt-account-id",
HeaderValue::from_static("client-account"),
);
headers.insert(
"chatgpt-project-id",
HeaderValue::from_static("client-project"),
);
headers.insert(
"x-codex-installation-id",
HeaderValue::from_static("client-installation"),
);
let decoded =
decode_request_with_headers(br#"{"model":"smart-code","input":"hello"}"#, &headers)
.expect("opaque request headers should not affect body decoding");
let entries = openai_protocol_context(&decoded)
.get("opaque_request_headers")
.and_then(Value::as_array)
.expect("opaque request header context");
let values = |target: &str| {
entries
.iter()
.filter_map(Value::as_array)
.filter(|entry| entry.first().and_then(Value::as_str) == Some(target))
.filter_map(|entry| entry.get(1).and_then(Value::as_str))
.map(|encoded| STANDARD.decode(encoded).expect("valid base64 header value"))
.collect::<Vec<_>>()
};
assert_eq!(
values("x-openai-future-mode"),
vec![b"future-ascii".to_vec(), b"\x80\xff".to_vec()]
);
assert_eq!(
values("version"),
vec![b"future-v1".to_vec(), b"future-\x80".to_vec()]
);
assert_eq!(
values("openai-beta"),
vec![b"future_responses=v2".to_vec(), b"future_tools=v3".to_vec()]
);
for preserved in [
"accept",
"content-type",
"user-agent",
"originator",
"x-openai-internal-codex-residency",
"x-codex-turn-state",
] {
assert_eq!(values(preserved).len(), 1, "missing {preserved}");
}
for excluded in [
"connection",
"x-private-hop",
"host",
"content-length",
"authorization",
"x-api-key",
"cookie",
"chatgpt-account-id",
"chatgpt-project-id",
"x-codex-installation-id",
] {
assert!(values(excluded).is_empty(), "unexpected {excluded}");
}
}
#[test]
fn decoder_should_preserve_unknown_nested_values_without_debug_disclosure() {
let secret = "nested-private-value";
@@ -191,25 +361,51 @@ fn decoder_should_preserve_unknown_nested_values_without_debug_disclosure() {
}
#[test]
fn decoder_should_preserve_continuation_and_mark_the_routing_requirement() {
let decoded = generate_request(json!({
"model": "smart-code",
"input": "continue",
"previous_response_id": "resp_private_continuation"
}));
fn decoder_should_preserve_unknown_arbitrary_precision_numbers() {
let number = "12345678901234567890123456789012345678901234567890";
let body = format!(r#"{{"model":"smart-code","input":"hello","future_number":{number}}}"#);
assert!(matches!(
decoded.metadata().continuation(),
ContinuationIntent::PreviousResponseId(value) if value == "resp_private_continuation"
));
assert!(
decoded
.operation()
.capability_requirements()
.features()
.contains(&Feature::NativeContinuation)
);
assert!(!format!("{decoded:?}").contains("resp_private_continuation"));
let decoded = decode_request_with_headers(body.as_bytes(), &HeaderMap::new())
.expect("opaque numeric field should decode");
let encoded = serde_json::to_string(openai_wire_body(&decoded)).expect("encode request body");
assert!(encoded.contains(&format!(r#""future_number":{number}"#)));
}
#[test]
fn decoder_should_preserve_opaque_continuation_and_mark_the_routing_requirement() {
for response_id in [
"resp_private_continuation".to_owned(),
format!("resp_{}\0opaque", "x".repeat(257)),
String::new(),
] {
let decoded = generate_request(json!({
"model": "smart-code",
"input": "continue",
"previous_response_id": response_id.clone()
}));
assert!(matches!(
decoded.metadata().continuation(),
ContinuationIntent::PreviousResponseId(value) if value == &response_id
));
assert_eq!(
openai_wire_body(&decoded)
.get("previous_response_id")
.and_then(Value::as_str),
Some(response_id.as_str())
);
assert!(
decoded
.operation()
.capability_requirements()
.features()
.contains(&Feature::NativeContinuation)
);
if !response_id.is_empty() {
assert!(!format!("{decoded:?}").contains(&response_id));
}
}
}
#[test]
@@ -316,7 +512,7 @@ fn transparent_encoder_should_forward_raw_sse_frames_unknown_events_and_terminal
let mut encoder = OpenAiResponsesEncoder::new();
let frames = events
.iter()
.flat_map(|event| encoder.push_sse(event).expect("wire event should encode"))
.flat_map(|event| encoder.push_sse(event))
.collect::<Vec<_>>();
let response = encoder.finish().expect("wire response should finish");
let body = frames
@@ -370,15 +566,11 @@ fn transparent_encoder_should_use_identical_json_for_sse_and_websocket() {
let sse = events
.iter()
.flat_map(|event| sse_encoder.push_sse(event).expect("SSE wire event"))
.flat_map(|event| sse_encoder.push_sse(event))
.collect::<Vec<_>>();
let websocket = events
.iter()
.flat_map(|event| {
websocket_encoder
.push_websocket(event)
.expect("WebSocket wire event")
})
.flat_map(|event| websocket_encoder.push_websocket(event))
.collect::<Vec<_>>();
let sse_text = String::from_utf8(
sse.iter()
@@ -400,8 +592,40 @@ fn transparent_encoder_should_use_identical_json_for_sse_and_websocket() {
}
#[test]
fn transparent_encoder_should_require_matching_canonical_and_wire_terminals() {
fn transparent_encoder_should_follow_wire_when_canonical_identity_changes() {
let mut encoder = OpenAiResponsesEncoder::new();
let started = openai_wire_event(
vec![GatewayEvent::Started(ResponseMeta::new(
"resp_1", "gpt-test",
))],
"response.created",
json!({"type": "response.created", "response": {"id": "wire_resp_1"}}),
);
encoder.push_sse(&started);
let changed_identity = openai_wire_event(
vec![GatewayEvent::Completed(ResponseMeta::new(
"resp_2", "gpt-test",
))],
"response.completed",
json!({"type": "response.completed", "response": {"id": "wire_resp_2"}}),
);
let completed = encoder.push_sse(&changed_identity);
assert_eq!(completed.len(), 1);
assert!(encoder.is_completed());
assert_eq!(encoder.response_id(), Some("wire_resp_2"));
assert_eq!(
encoder
.finish()
.expect("wire terminal remains authoritative"),
json!({"id": "wire_resp_2"})
);
}
#[test]
fn transparent_encoder_should_only_require_wire_terminal_for_buffered_conversion() {
let mut missing_terminal = OpenAiResponsesEncoder::new();
let started = openai_wire_event(
vec![GatewayEvent::Started(ResponseMeta::new(
"resp_1", "gpt-test",
@@ -409,24 +633,8 @@ fn transparent_encoder_should_require_matching_canonical_and_wire_terminals() {
"response.created",
json!({"type": "response.created", "response": {"id": "resp_1"}}),
);
encoder.push_sse(&started).expect("started event");
let changed_identity = openai_wire_event(
vec![GatewayEvent::Completed(ResponseMeta::new(
"resp_2", "gpt-test",
))],
"response.completed",
json!({"type": "response.completed", "response": {"id": "resp_2"}}),
);
assert_eq!(
encoder
.push_sse(&changed_identity)
.expect_err("identity changes must fail"),
ResponseEncodeError::WireIdentityChanged
);
let mut missing_terminal = OpenAiResponsesEncoder::new();
missing_terminal.push_sse(&started).expect("started event");
missing_terminal.push_sse(&started);
assert_eq!(
missing_terminal
.finish()
@@ -1,14 +1,35 @@
use std::sync::{
Arc,
atomic::{AtomicBool, AtomicUsize, Ordering},
};
use std::time::{Duration, SystemTime};
use axum::{
body::Body,
http::{Request, StatusCode, header::AUTHORIZATION},
};
use bytes::Bytes;
use futures::{SinkExt, StreamExt, future::BoxFuture};
use gateway_api::openai::responses::ResponseCreateFrameError;
use gateway_core::engine::execution::{
AuthenticatedClient, ClientAuthenticationError, ExecutionService, ExecutionSession,
StartExecution, StartedExecution,
};
use gateway_core::engine::{CommitRequirement, CoordinatedEvent, EngineError, UpstreamSendState};
use gateway_core::error::{GatewayError, ProviderError, ProviderErrorKind};
use gateway_core::event::{
GatewayEvent, ProtocolWireEvent, ProviderEvent, ProviderResponseHeader, ResponseMeta,
};
use gateway_core::operation::Operation;
use serde_json::json;
use gateway_core::routing::PublicModelId;
use serde_json::{Value, json};
use tokio_tungstenite::connect_async;
use tokio_tungstenite::tungstenite::Message as ClientMessage;
use tokio_tungstenite::tungstenite::client::IntoClientRequest;
use tower::ServiceExt;
use super::decode_response_create;
use crate::openai::{api_router, models::ModelsExecution};
use crate::openai::{api_router, authenticated_client, models::ModelsExecution};
#[test]
fn response_create_should_default_to_the_websocket_streaming_contract() {
@@ -148,11 +169,403 @@ fn response_create_should_reject_non_boolean_stream_without_disclosing_body_valu
assert!(!rendered.contains(opaque_stream_value));
}
// `serve_responses_websocket`/`forward_execution` 只有拿到 `hyper::upgrade::OnUpgrade`
// 的真实 HTTP/1.1 升级连接才会运行;本 crate 的依赖图按 axum `ws` feature 编译
// hyper(无 `http1`),测试进程内不存在能完成升级的服务端,`hyper::upgrade` 也不
// 提供公开构造器。集成测试因此止步于 upgrade 边界:会话循环内的事件转发、错误帧
// 与断连取消仍未被覆盖,需要 dev 依赖开启 `http1` 或 socket 泛化后才能落地。
#[derive(Default)]
struct AtomicFailureTrace {
next_calls: AtomicUsize,
committed: AtomicBool,
finalized: AtomicBool,
}
struct AtomicFailureSession {
trace: Arc<AtomicFailureTrace>,
response_headers: Vec<ProviderResponseHeader>,
}
impl ExecutionSession for AtomicFailureSession {
fn next_event(&mut self) -> BoxFuture<'_, Result<Option<CoordinatedEvent>, EngineError>> {
Box::pin(async move {
match self.trace.next_calls.fetch_add(1, Ordering::AcqRel) {
0 => Ok(Some(atomic_failure_batch())),
1 => {
self.trace.finalized.store(true, Ordering::Release);
Err(EngineError::Provider(ProviderError::new(
ProviderErrorKind::RateLimited,
UpstreamSendState::Sent,
)))
}
_ => Ok(None),
}
})
}
fn collect_uncommitted(&mut self) -> BoxFuture<'_, Result<Vec<ProviderEvent>, EngineError>> {
Box::pin(async { Err(EngineError::InvalidDeliveryState) })
}
fn response_headers(&self) -> &[ProviderResponseHeader] {
&self.response_headers
}
fn commit_downstream(&mut self, _: Option<u16>) -> BoxFuture<'_, Result<(), EngineError>> {
Box::pin(async move {
self.trace.committed.store(true, Ordering::Release);
Ok(())
})
}
fn record_client_status(&mut self, _: u16) -> BoxFuture<'_, Result<(), EngineError>> {
Box::pin(async { Ok(()) })
}
fn is_finalized(&self) -> bool {
self.trace.finalized.load(Ordering::Acquire)
}
fn cancel(&self) {}
fn detach_finalize(self: Box<Self>) -> BoxFuture<'static, ()> {
Box::pin(async move {
self.trace.finalized.store(true, Ordering::Release);
})
}
}
struct AtomicFailureExecution {
client: AuthenticatedClient,
trace: Arc<AtomicFailureTrace>,
response_headers: Vec<ProviderResponseHeader>,
}
impl ExecutionService for AtomicFailureExecution {
fn authenticate(
&self,
plaintext: &str,
) -> Result<AuthenticatedClient, ClientAuthenticationError> {
(plaintext == "sk_ws_atomic")
.then(|| self.client.clone())
.ok_or(ClientAuthenticationError::InvalidKey)
}
fn public_models(&self, _: &AuthenticatedClient) -> Vec<PublicModelId> {
vec![PublicModelId::new("model-a").expect("public model")]
}
fn contains_public_model(&self, _: &AuthenticatedClient, model: &PublicModelId) -> bool {
model.as_str() == "model-a"
}
fn start(
&self,
request: StartExecution,
) -> BoxFuture<'_, Result<StartedExecution, GatewayError>> {
Box::pin(async move {
Ok(StartedExecution {
request_id: gateway_core::engine::ModelRequestId::new("req_ws_atomic")
.expect("request id"),
created_at: SystemTime::now(),
stream: request.metadata.stream,
session: Box::new(AtomicFailureSession {
trace: Arc::clone(&self.trace),
response_headers: self.response_headers.clone(),
}),
})
})
}
}
fn atomic_failure_batch() -> CoordinatedEvent {
let started = ProviderEvent::canonical_with_wire(
vec![GatewayEvent::Started(ResponseMeta::new(
"resp_ws_atomic",
"model-a",
))],
ProtocolWireEvent::json(
"openai",
Some("response.created".to_owned()),
json!({
"type": "response.created",
"response": {
"id": "resp_ws_atomic",
"model": "model-a",
"status": "in_progress"
}
}),
)
.expect("created wire"),
);
let failed = ProviderEvent::wire(
ProtocolWireEvent::json(
"openai",
Some("response.failed".to_owned()),
json!({
"type": "response.failed",
"response": {
"id": "resp_ws_atomic",
"status": "failed",
"error": {
"code": "rate_limit_exceeded",
"message": "websocket atomic upstream marker"
}
}
}),
)
.expect("failed wire"),
);
CoordinatedEvent::try_batch(
vec![started, failed],
CommitRequirement::CommitBeforeDelivery,
)
.expect("atomic WebSocket batch")
}
#[tokio::test]
async fn websocket_atomic_upstream_failure_batch_should_be_forwarded_once() {
let trace = Arc::new(AtomicFailureTrace::default());
let execution = Arc::new(AtomicFailureExecution {
client: authenticated_client("sk_ws_atomic"),
trace: Arc::clone(&trace),
response_headers: Vec::new(),
});
let app = api_router(execution).await;
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind WebSocket test server");
let address = listener.local_addr().expect("test server address");
let server = tokio::spawn(async move {
axum::serve(listener, app)
.await
.expect("serve WebSocket test router");
});
let mut request = format!("ws://{address}/v1/responses")
.into_client_request()
.expect("WebSocket request");
request.headers_mut().insert(
AUTHORIZATION,
"Bearer sk_ws_atomic".parse().expect("authorization"),
);
let (mut socket, response) = connect_async(request).await.expect("upgrade WebSocket");
assert_eq!(response.status(), StatusCode::SWITCHING_PROTOCOLS);
socket
.send(ClientMessage::Text(
json!({
"type": "response.create",
"model": "model-a",
"input": "hello"
})
.to_string()
.into(),
))
.await
.expect("send response.create");
let mut messages = Vec::<Value>::new();
loop {
let message = tokio::time::timeout(Duration::from_secs(2), socket.next())
.await
.expect("WebSocket response timeout")
.expect("WebSocket remains open")
.expect("valid WebSocket frame");
let ClientMessage::Text(text) = message else {
continue;
};
let value: Value = serde_json::from_str(&text).expect("JSON WebSocket event");
let failed = value.get("type").and_then(Value::as_str) == Some("response.failed");
messages.push(value);
if failed {
break;
}
}
assert_eq!(
messages
.iter()
.filter(|message| message.get("type").and_then(Value::as_str) == Some("response.failed"))
.count(),
1
);
let failed = messages
.iter()
.find(|message| message.get("type").and_then(Value::as_str) == Some("response.failed"))
.expect("upstream failed event");
assert_eq!(
failed
.pointer("/response/error/message")
.and_then(Value::as_str),
Some("websocket atomic upstream marker")
);
assert!(trace.committed.load(Ordering::Acquire));
assert_eq!(trace.next_calls.load(Ordering::Acquire), 2);
assert!(trace.finalized.load(Ordering::Acquire));
socket.close(None).await.expect("close WebSocket");
server.abort();
}
#[tokio::test]
async fn websocket_response_metadata_should_preserve_ordinary_headers_without_blocking_events() {
let trace = Arc::new(AtomicFailureTrace::default());
let execution = Arc::new(AtomicFailureExecution {
client: authenticated_client("sk_ws_atomic"),
trace,
response_headers: vec![
ProviderResponseHeader::new("x-future-header", Bytes::from_static(b"future-value")),
ProviderResponseHeader::new("x-future-multi", Bytes::from_static(b"first")),
ProviderResponseHeader::new("x-future-multi", Bytes::from_static(b"second")),
ProviderResponseHeader::new("x-future-bytes", Bytes::from_static(b"\xffopaque")),
ProviderResponseHeader::new("bad\0name", Bytes::from_static(b"unrepresentable")),
ProviderResponseHeader::new(
"authorization",
Bytes::from_static(b"should-not-cross-boundary"),
),
ProviderResponseHeader::new("connection", Bytes::from_static(b"x-private-hop")),
ProviderResponseHeader::new("x-private-hop", Bytes::from_static(b"private-hop")),
ProviderResponseHeader::new("content-type", Bytes::from_static(b"application/private")),
ProviderResponseHeader::new("x-request-id", Bytes::from_static(b"req_upstream")),
],
});
let app = api_router(execution).await;
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind WebSocket test server");
let address = listener.local_addr().expect("test server address");
let server = tokio::spawn(async move {
axum::serve(listener, app)
.await
.expect("serve WebSocket test router");
});
let mut request = format!("ws://{address}/v1/responses")
.into_client_request()
.expect("WebSocket request");
request.headers_mut().insert(
AUTHORIZATION,
"Bearer sk_ws_atomic".parse().expect("authorization"),
);
let (mut socket, response) = connect_async(request).await.expect("upgrade WebSocket");
assert_eq!(response.status(), StatusCode::SWITCHING_PROTOCOLS);
socket
.send(ClientMessage::Text(
json!({
"type": "response.create",
"model": "model-a",
"input": "hello"
})
.to_string()
.into(),
))
.await
.expect("send response.create");
let mut metadata = None;
let mut saw_failed = false;
while metadata.is_none() || !saw_failed {
let message = tokio::time::timeout(Duration::from_secs(2), socket.next())
.await
.expect("WebSocket response timeout")
.expect("WebSocket remains open")
.expect("valid WebSocket frame");
let ClientMessage::Text(text) = message else {
continue;
};
let value: Value = serde_json::from_str(&text).expect("JSON WebSocket event");
match value.get("type").and_then(Value::as_str) {
Some("response.metadata") => metadata = Some(value),
Some("response.failed") => saw_failed = true,
_ => {}
}
}
let headers = metadata
.as_ref()
.and_then(|value| value.get("headers"))
.and_then(Value::as_object)
.expect("response metadata headers");
assert_eq!(headers.get("x-future-header"), Some(&json!("future-value")));
// 官方 metadata 是 string map,同名多值只能保持既有的后值覆盖语义。
assert_eq!(headers.get("x-future-multi"), Some(&json!("second")));
assert_eq!(headers.get("x-request-id"), Some(&json!("req_upstream")));
for omitted in [
"x-future-bytes",
"bad\0name",
"authorization",
"connection",
"x-private-hop",
"content-type",
] {
assert!(
!headers.contains_key(omitted),
"unexpected metadata header: {omitted:?}"
);
}
assert!(saw_failed);
socket.close(None).await.expect("close WebSocket");
server.abort();
}
#[tokio::test]
async fn websocket_response_create_should_not_have_a_private_16_mib_frame_limit() {
let trace = Arc::new(AtomicFailureTrace::default());
let execution = Arc::new(AtomicFailureExecution {
client: authenticated_client("sk_ws_atomic"),
trace,
response_headers: Vec::new(),
});
let app = api_router(execution).await;
let listener = tokio::net::TcpListener::bind("127.0.0.1:0")
.await
.expect("bind WebSocket test server");
let address = listener.local_addr().expect("test server address");
let server = tokio::spawn(async move {
axum::serve(listener, app)
.await
.expect("serve WebSocket test router");
});
let mut request = format!("ws://{address}/v1/responses")
.into_client_request()
.expect("WebSocket request");
request.headers_mut().insert(
AUTHORIZATION,
"Bearer sk_ws_atomic".parse().expect("authorization"),
);
let (mut socket, response) = connect_async(request).await.expect("upgrade WebSocket");
assert_eq!(response.status(), StatusCode::SWITCHING_PROTOCOLS);
let payload = json!({
"type": "response.create",
"model": "model-a",
"input": "x".repeat(16 * 1024 * 1024 + 1),
})
.to_string();
assert!(payload.len() > 16 * 1024 * 1024);
socket
.send(ClientMessage::Text(payload.into()))
.await
.expect("send response.create above the former frame limit");
let mut saw_upstream_event = false;
for _ in 0..3 {
let message = tokio::time::timeout(Duration::from_secs(10), socket.next())
.await
.expect("WebSocket response timeout")
.expect("WebSocket remains open")
.expect("valid WebSocket frame");
let ClientMessage::Text(text) = message else {
continue;
};
let value: Value = serde_json::from_str(&text).expect("JSON WebSocket event");
if matches!(
value.get("type").and_then(Value::as_str),
Some("response.created" | "response.failed")
) {
saw_upstream_event = true;
break;
}
}
assert!(saw_upstream_event);
socket.close(None).await.expect("close WebSocket");
server.abort();
}
const TEST_WEBSOCKET_KEY: &str = "AAAAAAAAAAAAAAAAAAAAAA==";
@@ -1,57 +1,29 @@
use axum::{
Router,
body::{Body, Bytes},
extract::DefaultBodyLimit,
body::Body,
http::{Method, Request, StatusCode},
routing::post,
};
use gateway_api::openai::router::MAX_CLIENT_REQUEST_BODY_BYTES;
use tower::ServiceExt;
use super::api_router_with_origins;
use super::models::ModelsExecution;
fn body_limit_app() -> Router {
Router::new()
.route(
"/v1/responses",
post(|body: Bytes| async move {
if body.len() == MAX_CLIENT_REQUEST_BODY_BYTES {
StatusCode::NO_CONTENT
} else {
StatusCode::INTERNAL_SERVER_ERROR
}
}),
)
.layer(DefaultBodyLimit::max(MAX_CLIENT_REQUEST_BODY_BYTES))
}
const REMOVED_BODY_LIMIT_BYTES: usize = 16 * 1024 * 1024;
#[tokio::test]
async fn responses_body_limit_should_accept_exactly_sixteen_mibibytes() {
let response = body_limit_app()
async fn responses_body_should_accept_payload_above_the_removed_private_limit() {
let router = api_router_with_origins(ModelsExecution::new(), Vec::new()).await;
let response = router
.oneshot(
Request::post("/v1/responses")
.body(Body::from(vec![b'a'; MAX_CLIENT_REQUEST_BODY_BYTES]))
.expect("build exact-limit request"),
.body(Body::from(vec![b'a'; REMOVED_BODY_LIMIT_BYTES + 1]))
.expect("build request above the removed limit"),
)
.await
.expect("route exact-limit request");
.expect("route request above the removed limit");
assert_eq!(response.status(), StatusCode::NO_CONTENT);
}
#[tokio::test]
async fn responses_body_limit_should_reject_one_byte_over_sixteen_mibibytes() {
let response = body_limit_app()
.oneshot(
Request::post("/v1/responses")
.body(Body::from(vec![b'a'; MAX_CLIENT_REQUEST_BODY_BYTES + 1]))
.expect("build over-limit request"),
)
.await
.expect("route over-limit request");
assert_eq!(response.status(), StatusCode::PAYLOAD_TOO_LARGE);
// The handler sees the body and rejects the missing credentials. A restored body limit
// would return 413 before authentication runs.
assert_eq!(response.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
+2 -1
View File
@@ -17,8 +17,9 @@ bytes = "1.11.1"
futures = "0.3.32"
futures-timer = "3.0.3"
serde = { version = "1.0.228", features = ["derive"] }
serde_json = { version = "1.0.150", features = ["preserve_order"] }
serde_json = { version = "1.0.150", features = ["arbitrary_precision", "preserve_order"] }
thiserror = "2.0.18"
tracing = "0.1.44"
uuid = { version = "1.23.3", features = ["v7"] }
[lints]
@@ -8,24 +8,21 @@ use std::fmt;
use futures::future::BoxFuture;
use crate::engine::credential::ProviderAccountId;
use crate::error::{IdentifierError, SafeUpstreamValue, validate_text};
use crate::operation::ProviderSessionState;
use crate::routing::ProviderKind;
/// 客户端传入的 previous response ID。
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
/// 客户端或 Provider 传递的 opaque response handle。
///
/// Codex 定义这个值的语义和格式。网关只将其作为亲和查找键或同账号上游
/// continuation 的载体,不得按私有长度、字符集或前缀规则拒绝、归一化或改写。
#[derive(Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct PreviousResponseId(String);
impl PreviousResponseId {
/// 校验并创建 previous response ID。
///
/// # Errors
///
/// ID 为空、过长或包含控制字符时返回错误。
pub fn new(value: impl Into<String>) -> Result<Self, IdentifierError> {
let value = value.into();
validate_text(&value, 256, false, None)?;
Ok(Self(value))
/// 按原样创建 opaque response handle。
#[must_use]
pub fn new(value: impl Into<String>) -> Self {
Self(value.into())
}
#[must_use]
@@ -34,6 +31,12 @@ impl PreviousResponseId {
}
}
impl fmt::Debug for PreviousResponseId {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.write_str("PreviousResponseId(<redacted>)")
}
}
/// Provider 原生 response handle 的续接范围。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum NativeContinuationScope {
@@ -52,7 +55,7 @@ pub struct NativeContinuationPin {
/// 客户端提交、用于查找可丢失会话亲和记录的 response ID。
previous_response_id: PreviousResponseId,
/// Provider 原生 response handle。
upstream_response_id: SafeUpstreamValue,
upstream_response_id: PreviousResponseId,
provider: ProviderKind,
account: ProviderAccountId,
scope: NativeContinuationScope,
@@ -63,7 +66,7 @@ impl NativeContinuationPin {
#[must_use]
pub const fn new(
previous_response_id: PreviousResponseId,
upstream_response_id: SafeUpstreamValue,
upstream_response_id: PreviousResponseId,
provider: ProviderKind,
account: ProviderAccountId,
) -> Self {
@@ -98,7 +101,7 @@ impl NativeContinuationPin {
/// 返回只允许发送给已冻结 Provider 目标的原生上游 handle。
#[must_use]
pub const fn upstream_response_id(&self) -> &SafeUpstreamValue {
pub const fn upstream_response_id(&self) -> &PreviousResponseId {
&self.upstream_response_id
}
@@ -1,6 +1,7 @@
//! 唯一的账号重试、发送与下游 commit barrier owner。
use std::collections::BTreeSet;
use std::future::Future;
use std::num::NonZeroU32;
use std::sync::Arc;
use std::time::{Duration, Instant, SystemTime};
@@ -17,7 +18,7 @@ use crate::engine::{
ModelRequestTimings, NewModelRequest, ProviderAccountStateOwner, ProviderAttemptOutcome,
RequestAttemptContext, UpstreamSendState,
};
use crate::error::{GatewayError, GatewayErrorKind, ProviderError, ProviderErrorKind};
use crate::error::{GatewayError, GatewayErrorKind, ProviderError, ProviderErrorKind, StoreError};
use crate::event::{
GatewayEvent, ProviderEvent, ProviderResponseHeader, ProviderResponseObservation,
};
@@ -32,6 +33,21 @@ pub struct AttemptCoordinator<S: ?Sized> {
engine: Arc<GatewayEngine<S>>,
}
#[derive(Debug, Clone)]
enum AccountSelection {
Scheduled(Option<crate::engine::credential::ProviderAccountId>),
Diagnostic(crate::engine::credential::ProviderAccountId),
}
impl AccountSelection {
fn required_account(&self) -> Option<&crate::engine::credential::ProviderAccountId> {
match self {
Self::Scheduled(account) => account.as_ref(),
Self::Diagnostic(account) => Some(account),
}
}
}
impl<S: ?Sized> AttemptCoordinator<S>
where
S: ExecutionStore,
@@ -56,6 +72,49 @@ where
required_account: Option<crate::engine::credential::ProviderAccountId>,
continuation: Option<ContinuationBinding>,
cancellation: CancellationToken,
) -> Result<ResponseExecutionSession<S>, EngineError> {
self.start_with_account_selection(
request,
operation,
plan,
AccountSelection::Scheduled(required_account),
continuation,
cancellation,
)
.await
}
/// 对固定账号执行诊断请求。
///
/// 仅跳过该账号的本地可用性投影;账号租约、并发和请求间隔仍必须满足。
pub async fn start_diagnostic(
&self,
request: NewModelRequest,
operation: Operation,
plan: RoutingPlan,
required_account: crate::engine::credential::ProviderAccountId,
continuation: Option<ContinuationBinding>,
cancellation: CancellationToken,
) -> Result<ResponseExecutionSession<S>, EngineError> {
self.start_with_account_selection(
request,
operation,
plan,
AccountSelection::Diagnostic(required_account),
continuation,
cancellation,
)
.await
}
async fn start_with_account_selection(
&self,
request: NewModelRequest,
operation: Operation,
plan: RoutingPlan,
account_selection: AccountSelection,
continuation: Option<ContinuationBinding>,
cancellation: CancellationToken,
) -> Result<ResponseExecutionSession<S>, EngineError> {
let request_id = request.id.clone();
let client_api_key_ref = request.client_api_key_ref.clone();
@@ -84,7 +143,7 @@ where
request_persisted: false,
operation,
plan,
required_account,
account_selection,
continuation,
continuation_attempt,
account_state_owner,
@@ -107,7 +166,9 @@ where
client_response_id: None,
upstream_response_id: None,
last_account_exhaustion: None,
last_account_exhaustion_events: Vec::new(),
provider_attempt_outcomes: Vec::new(),
pending_terminal_failure: None,
};
if session.cancellation.is_cancelled() {
@@ -158,7 +219,7 @@ pub struct ResponseExecutionSession<S: ?Sized> {
request_persisted: bool,
operation: Operation,
plan: RoutingPlan,
required_account: Option<crate::engine::credential::ProviderAccountId>,
account_selection: AccountSelection,
continuation: Option<ContinuationBinding>,
continuation_attempt: ContinuationAttempt,
account_state_owner: Option<ProviderAccountStateOwner>,
@@ -185,7 +246,16 @@ pub struct ResponseExecutionSession<S: ?Sized> {
client_response_id: Option<String>,
upstream_response_id: Option<String>,
last_account_exhaustion: Option<ProviderError>,
/// 最近一次账号耗尽在重试时被丢弃的原始失败批次;只用于后续空选路终态。
last_account_exhaustion_events: Vec<ProviderEvent>,
provider_attempt_outcomes: Vec<ProviderAttemptOutcome>,
/// 原子失败批次已交给协议层、但尚待下游提交后收敛的原 Provider 错误。
pending_terminal_failure: Option<PendingTerminalFailure>,
}
struct PendingTerminalFailure {
error: ProviderError,
send_state: UpstreamSendState,
}
impl<S: ?Sized> ResponseExecutionSession<S>
@@ -201,6 +271,11 @@ where
if self.delivery_pending {
return Err(EngineError::DownstreamCommitRequired);
}
if let Some(pending) = self.pending_terminal_failure.take() {
self.finish_provider_error_with_send_state(&pending.error, pending.send_state)
.await?;
return Err(provider_engine_error(pending.error));
}
if self.finalized {
return Ok(None);
}
@@ -220,6 +295,21 @@ where
)));
}
PullOutcome::AttemptDiscarded => {}
PullOutcome::TerminalFailure {
events,
error,
send_state,
} => {
let requirement = if self.downstream_committed_at.is_some() {
CommitRequirement::AlreadyCommitted
} else {
self.delivery_pending = true;
CommitRequirement::CommitBeforeDelivery
};
self.pending_terminal_failure =
Some(PendingTerminalFailure { error, send_state });
return CoordinatedEvent::try_batch(events, requirement).map(Some);
}
PullOutcome::End => {
if self.downstream_committed_at.is_some() {
self.finish_success().await?;
@@ -248,6 +338,13 @@ where
match self.pull().await? {
PullOutcome::Event(event) => events.push(event),
PullOutcome::AttemptDiscarded => events.clear(),
PullOutcome::TerminalFailure {
error, send_state, ..
} => {
self.finish_provider_error_with_send_state(&error, send_state)
.await?;
return Err(provider_engine_error(error));
}
PullOutcome::End => {
if events.is_empty() {
return Err(EngineError::InvalidDeliveryState);
@@ -259,11 +356,11 @@ where
}
}
/// 在协议 adapter 真正写出首字节前持久化下游不可撤回边界。
/// 在协议 adapter 真正写出首字节前记录下游不可撤回边界。
///
/// # Errors
///
/// 没有待提交结果、重复提交或 Store 失败时返回错误。
/// 没有待提交结果或重复提交时返回错误;观测 Store 失败只记服务端告警。
pub async fn commit_downstream(
&mut self,
client_status_code: Option<u16>,
@@ -272,10 +369,18 @@ where
return Err(EngineError::InvalidDeliveryState);
}
let committed_at = SystemTime::now();
self.engine
.store()
.mark_downstream_committed(&self.request_id, committed_at, client_status_code)
.await?;
if self.request_persisted {
best_effort_store_write(
"mark_downstream_committed",
&self.request_id,
self.engine.store().mark_downstream_committed(
&self.request_id,
committed_at,
client_status_code,
),
)
.await;
}
self.downstream_committed_at = Some(committed_at);
self.client_status_code = client_status_code;
self.delivery_pending = false;
@@ -289,7 +394,7 @@ where
///
/// # Errors
///
/// 状态已经写入或 Store 无法写回时返回错误。
/// 状态已经写入时返回错误;观测 Store 失败只记服务端告警。
pub async fn record_client_status(
&mut self,
client_status_code: u16,
@@ -298,10 +403,14 @@ where
return Err(EngineError::InvalidDeliveryState);
}
if self.request_persisted {
self.engine
.store()
.record_client_status(&self.request_id, client_status_code)
.await?;
best_effort_store_write(
"record_client_status",
&self.request_id,
self.engine
.store()
.record_client_status(&self.request_id, client_status_code),
)
.await;
}
self.client_status_code = Some(client_status_code);
Ok(())
@@ -345,9 +454,8 @@ where
}
let account = current.metadata.provider_account_id()?.clone();
let response_id = self.upstream_response_id.as_deref()?;
let previous_response_id = PreviousResponseId::new(response_id.to_owned()).ok()?;
let upstream_response_id =
crate::error::SafeUpstreamValue::new(response_id.to_owned()).ok()?;
let previous_response_id = PreviousResponseId::new(response_id.to_owned());
let upstream_response_id = PreviousResponseId::new(response_id.to_owned());
Some(
NativeContinuationPin::new(
previous_response_id,
@@ -376,13 +484,20 @@ where
return Ok(());
}
self.delivery_pending = false;
if let Some(pending) = self.pending_terminal_failure.take() {
return self
.finish_provider_error_with_send_state(&pending.error, pending.send_state)
.await;
}
self.finish_interruption(EngineError::Cancelled).await
}
async fn pull(&mut self) -> Result<PullOutcome, EngineError> {
loop {
if self.current.is_none() {
self.prepare_attempt().await?;
if self.current.is_none()
&& let Some(outcome) = self.prepare_attempt().await?
{
return Ok(outcome);
}
let boundary = {
@@ -409,32 +524,17 @@ where
return Err(EngineError::Deadline);
}
PollBoundary::Item(Some(Ok(mut event))) => {
if let Some(observation) = event.take_observation()
&& let Err(error) = self.observe_response(observation)
{
if self.handle_stream_error(error).await? {
return Ok(PullOutcome::AttemptDiscarded);
}
continue;
}
let mut identity_error = None;
for fact in event.canonical_facts() {
if let Err(error) = self.observe_response_identity(fact) {
identity_error = Some(error);
break;
}
}
if let Some(error) = identity_error {
if self.handle_stream_error(error).await? {
return Ok(PullOutcome::AttemptDiscarded);
}
continue;
if let Some(observation) = event.take_observation() {
self.observe_response(observation);
}
for fact in event.canonical_facts() {
self.observe_event(fact).await?;
self.observe_response_identity(fact);
}
for fact in event.canonical_facts() {
self.observe_event(fact).await;
}
if event.wire_event().is_some() && !event.has_canonical_facts() {
self.observe_wire_event().await?;
self.observe_wire_event().await;
}
if !event.has_client_event() {
continue;
@@ -442,8 +542,13 @@ where
return Ok(PullOutcome::Event(event));
}
PollBoundary::Item(Some(Err(error))) => {
if self.handle_stream_error(error).await? {
return Ok(PullOutcome::AttemptDiscarded);
match self.handle_stream_error(error).await? {
StreamErrorOutcome::AttemptDiscarded => {
return Ok(PullOutcome::AttemptDiscarded);
}
terminal @ StreamErrorOutcome::TerminalFailure { .. } => {
return Ok(terminal.into_pull_outcome());
}
}
}
PollBoundary::Item(None) => {
@@ -455,7 +560,7 @@ where
}
}
async fn prepare_attempt(&mut self) -> Result<(), EngineError> {
async fn prepare_attempt(&mut self) -> Result<Option<PullOutcome>, EngineError> {
if self.attempts >= self.plan.max_attempts().get() {
return Err(EngineError::EmptyRoutingPlan);
}
@@ -483,27 +588,36 @@ where
.ok_or(EngineError::EmptyRoutingPlan)?;
// recovery 钉选在此被一次性消费,只绑定本次 replay attempt;
// 外部 required_account 每次 attempt 都重新生效。
let pinned_account = self
.recovery_account
.take()
.or_else(|| self.required_account.clone());
let pinned_account = match &self.account_selection {
AccountSelection::Diagnostic(account) => Some(account.clone()),
AccountSelection::Scheduled(_) => self
.recovery_account
.take()
.or_else(|| self.account_selection.required_account().cloned()),
};
let account_context = match &self.account_selection {
AccountSelection::Diagnostic(account) => AccountAttemptContext::diagnostic(
self.excluded_accounts.clone(),
account.clone(),
self.account_state_owner.clone(),
),
AccountSelection::Scheduled(_) => AccountAttemptContext::new(
self.excluded_accounts.clone(),
pinned_account.clone(),
self.account_state_owner.clone(),
),
}
.with_credential_recovery_attempted(pinned_account.as_ref().is_some_and(|account| {
self.credential_recovery_attempted_accounts
.contains(account)
}));
let context = AttemptContext::new(
RequestAttemptContext::new(self.request_id.clone(), self.client_api_key_ref.clone())
.with_timing_started_at(self.timing_started_at),
next_attempt,
self.deadline,
self.plan.account_selection_policy(),
AccountAttemptContext::new(
self.excluded_accounts.clone(),
pinned_account.clone(),
self.account_state_owner.clone(),
)
.with_credential_recovery_attempted(pinned_account.as_ref().is_some_and(
|account| {
self.credential_recovery_attempted_accounts
.contains(account)
},
)),
account_context,
self.continuation.clone(),
self.cancellation.clone(),
)
@@ -546,11 +660,20 @@ where
Ok(stream) => stream,
Err(error) => {
if error.kind() == ProviderErrorKind::NoEligibleAccount
&& let Some(exhaustion) = self.last_account_exhaustion.clone()
&& let Some(exhaustion) = self.last_account_exhaustion.take()
{
let engine_error = provider_engine_error(&exhaustion);
let send_state = self.current_send_state();
let mut events = std::mem::take(&mut self.last_account_exhaustion_events);
if !events.is_empty() {
self.observe_atomic_terminal_events(&mut events);
return Ok(Some(PullOutcome::TerminalFailure {
events,
error: exhaustion,
send_state,
}));
}
self.finish_provider_error(&exhaustion).await?;
return Err(engine_error);
return Err(provider_engine_error(exhaustion));
}
if !(matches!(
error.kind(),
@@ -559,9 +682,8 @@ where
{
self.record_provider_failure(candidate.provider().clone(), error.kind());
}
let engine_error = provider_engine_error(&error);
self.finish_provider_error(&error).await?;
return Err(engine_error);
return Err(provider_engine_error(error));
}
},
};
@@ -650,17 +772,30 @@ where
http_version: None,
};
if self.request_persisted {
self.engine.store().record_attempt(attempt_record).await?;
best_effort_store_write(
"record_attempt",
&self.request_id,
self.engine.store().record_attempt(attempt_record),
)
.await;
} else {
let request = self
.pending_request
.take()
.as_ref()
.ok_or(EngineError::InvalidDeliveryState)?;
self.engine
.store()
.create_model_request_with_attempt(request, attempt_record)
.await?;
self.request_persisted = true;
if best_effort_store_write(
"create_model_request_with_attempt",
&self.request_id,
self.engine
.store()
.create_model_request_with_attempt(request.clone(), attempt_record),
)
.await
.is_some()
{
self.pending_request = None;
self.request_persisted = true;
}
}
self.attempts = next_attempt.get();
self.current = Some(CurrentAttempt {
@@ -672,10 +807,15 @@ where
send_observed: false,
response_observation: None,
});
Ok(())
Ok(None)
}
async fn observe_event(&mut self, event: &GatewayEvent) -> Result<(), EngineError> {
async fn observe_event(&mut self, event: &GatewayEvent) {
self.observe_event_facts(event);
self.mark_send_observed().await;
}
fn observe_event_facts(&mut self, event: &GatewayEvent) {
let elapsed = elapsed_ms(self.timing_started_at);
observe_event_timing(&mut self.timings, event, elapsed);
if let GatewayEvent::Usage(observed) = event {
@@ -689,46 +829,42 @@ where
if let GatewayEvent::ProviderCost(observed) = event {
self.cost = observed.into_estimate();
}
let current = self.current.as_mut().ok_or(EngineError::NoActiveAttempt)?;
}
async fn observe_wire_event(&mut self) {
self.mark_send_observed().await;
}
async fn mark_send_observed(&mut self) {
let Some(current) = self.current.as_mut() else {
return;
};
if !current.send_observed {
self.engine
.store()
.mark_send_state(&self.request_id, UpstreamSendState::Sent)
.await?;
if self.request_persisted {
best_effort_store_write(
"mark_send_state",
&self.request_id,
self.engine
.store()
.mark_send_state(&self.request_id, UpstreamSendState::Sent),
)
.await;
}
current.send_observed = true;
self.send_state_watermark = UpstreamSendState::Sent;
}
Ok(())
}
async fn observe_wire_event(&mut self) -> Result<(), EngineError> {
let current = self.current.as_mut().ok_or(EngineError::NoActiveAttempt)?;
if !current.send_observed {
self.engine
.store()
.mark_send_state(&self.request_id, UpstreamSendState::Sent)
.await?;
current.send_observed = true;
self.send_state_watermark = UpstreamSendState::Sent;
}
Ok(())
}
fn observe_response(
&mut self,
observation: ProviderResponseObservation,
) -> Result<(), ProviderError> {
let send_state = self.current_send_state();
let current = self
.current
.as_mut()
.ok_or_else(|| ProviderError::new(ProviderErrorKind::Protocol, send_state))?;
fn observe_response(&mut self, observation: ProviderResponseObservation) {
let Some(current) = self.current.as_mut() else {
return;
};
if current
.response_observation
.as_ref()
.is_some_and(|existing| existing.transport() != observation.transport())
{
return Err(ProviderError::new(ProviderErrorKind::Protocol, send_state));
return;
}
let observed = observation.timings();
if let Some(value) = observed.transport_decision_wait_ms {
@@ -756,47 +892,31 @@ where
self.timings.provider_processing_ms = Some(value);
}
current.response_observation = Some(observation);
Ok(())
}
fn observe_response_identity(&mut self, event: &GatewayEvent) -> Result<(), ProviderError> {
fn observe_response_identity(&mut self, event: &GatewayEvent) {
let metadata = match event {
GatewayEvent::Started(metadata) | GatewayEvent::Completed(metadata) => metadata,
_ => return Ok(()),
_ => return,
};
let response_id = crate::error::SafeUpstreamValue::new(metadata.response_id().to_owned())
.map_err(|_| {
ProviderError::new(ProviderErrorKind::Protocol, UpstreamSendState::Sent)
})?;
let response_id = response_id.as_str();
if response_id.is_empty() {
return Ok(());
}
if self
.upstream_response_id
.as_deref()
.is_some_and(|expected| expected != response_id)
{
return Err(ProviderError::new(
ProviderErrorKind::Protocol,
UpstreamSendState::Sent,
));
}
let response_id = response_id.to_owned();
let response_id = metadata.response_id().to_owned();
self.client_response_id = Some(response_id.clone());
self.upstream_response_id = Some(response_id);
Ok(())
}
/// 返回 `true` 表示调用方必须丢弃本 attempt 已收集的未提交事件。
async fn handle_stream_error(&mut self, error: ProviderError) -> Result<bool, EngineError> {
/// 失败可重试时要求丢弃本 attempt;预算耗尽时可返回最后一批原始失败事件。
async fn handle_stream_error(
&mut self,
mut error: ProviderError,
) -> Result<StreamErrorOutcome, EngineError> {
// 原始 wire 只能活在本次决策栈内;后续 clone、attempt 记录与终态对象均只
// 接触已剥离的稳定错误字段。
let mut atomic_client_events = error.take_atomic_client_events();
let current = self.current.take().ok_or(EngineError::NoActiveAttempt)?;
if matches!(
let account_exhausted = matches!(
error.kind(),
ProviderErrorKind::RateLimited | ProviderErrorKind::QuotaExhausted
) {
self.last_account_exhaustion = Some(error.clone());
}
);
self.record_provider_failure(current.metadata.provider().clone(), error.kind());
// attempt_send_state 是本 attempt 自身的发送事实,驱动重试门;
// 持久化与终态用请求级水位,二者不可混用(水位会把早先 attempt 的
@@ -807,10 +927,16 @@ where
error.send_state()
};
let send_state = self.raise_send_watermark(attempt_send_state);
self.engine
.store()
.mark_send_state(&self.request_id, send_state)
.await?;
if self.request_persisted {
best_effort_store_write(
"mark_send_state",
&self.request_id,
self.engine
.store()
.mark_send_state(&self.request_id, send_state),
)
.await;
}
let provider_proved_replay_safe = provider_proved_replay_safe(&error);
let continuation_retry = self.prepare_continuation_retry(
&current,
@@ -818,7 +944,13 @@ where
attempt_send_state,
provider_proved_replay_safe,
);
let ordinary_retry = self.required_account.is_none()
let pre_delivery_retry = self.account_selection.required_account().is_none()
&& self.continuation_attempt == ContinuationAttempt::None
&& self.downstream_committed_at.is_none()
&& !self.delivery_pending
&& error.allows_pre_delivery_retry()
&& self.attempts < self.plan.max_attempts().get();
let ordinary_retry = self.account_selection.required_account().is_none()
&& self.continuation_attempt == ContinuationAttempt::None
&& self.downstream_committed_at.is_none()
&& !self.delivery_pending
@@ -840,9 +972,13 @@ where
.credential_recovery_attempted_accounts
.contains(account)
});
let retryable = continuation_retry || same_account_retry || ordinary_retry;
let retryable =
continuation_retry || same_account_retry || ordinary_retry || pre_delivery_retry;
if retryable {
// 普通 clone 只保留稳定事实;原始失败响应只能随最后一次客户端可见
// 的账号耗尽错误继续,不能进入中间失败记录。
let persistence_error = self.request_persisted.then(|| error.clone());
if same_account_retry {
if let Some(account) = current.metadata.provider_account_id() {
self.credential_recovery_attempted_accounts
@@ -856,37 +992,66 @@ where
{
self.excluded_accounts.insert(account.clone());
}
self.engine
.store()
.record_intermediate_failure(IntermediateFailure {
request_id: self.request_id.clone(),
attempt_index: current.index,
trigger: current.trigger,
provider_kind: current.metadata.provider().clone(),
account_id: current.metadata.provider_account_id().cloned(),
upstream_model_id: current.metadata.upstream_model().clone(),
upstream_status_code: current
.response_observation
.as_ref()
.and_then(ProviderResponseObservation::status_code),
upstream_request_id: current
.response_observation
.as_ref()
.and_then(ProviderResponseObservation::request_id)
.map(|value| value.as_str().to_owned()),
latency: current.started_at.elapsed().unwrap_or_default(),
error,
})
.await?;
if account_exhausted {
self.last_account_exhaustion_events = atomic_client_events;
self.last_account_exhaustion = Some(error);
}
if let Some(error) = persistence_error {
best_effort_store_write(
"record_intermediate_failure",
&self.request_id,
self.engine
.store()
.record_intermediate_failure(IntermediateFailure {
request_id: self.request_id.clone(),
attempt_index: current.index,
trigger: current.trigger,
provider_kind: current.metadata.provider().clone(),
account_id: current.metadata.provider_account_id().cloned(),
upstream_model_id: current.metadata.upstream_model().clone(),
upstream_status_code: current
.response_observation
.as_ref()
.and_then(ProviderResponseObservation::status_code),
upstream_request_id: current
.response_observation
.as_ref()
.and_then(ProviderResponseObservation::request_id)
.map(|value| value.as_str().to_owned()),
latency: current.started_at.elapsed().unwrap_or_default(),
error,
}),
)
.await;
}
self.reset_uncommitted_observations();
return Ok(true);
return Ok(StreamErrorOutcome::AttemptDiscarded);
}
let engine_error = provider_engine_error(&error);
self.current = Some(current);
if !atomic_client_events.is_empty() {
self.observe_atomic_terminal_events(&mut atomic_client_events);
return Ok(StreamErrorOutcome::TerminalFailure {
events: atomic_client_events,
error,
send_state,
});
}
self.finish_provider_error_with_send_state(&error, send_state)
.await?;
Err(engine_error)
Err(provider_engine_error(error))
}
fn observe_atomic_terminal_events(&mut self, events: &mut [ProviderEvent]) {
for event in events {
if let Some(observation) = event.take_observation() {
self.observe_response(observation);
}
for fact in event.canonical_facts() {
self.observe_response_identity(fact);
self.observe_event_facts(fact);
}
}
}
fn prepare_continuation_retry(
@@ -896,7 +1061,7 @@ where
send_state: UpstreamSendState,
provider_proved_replay_safe: bool,
) -> bool {
if self.required_account.is_some()
if self.account_selection.required_account().is_some()
|| self.continuation_attempt == ContinuationAttempt::None
|| self.downstream_committed_at.is_some()
|| self.delivery_pending
@@ -976,34 +1141,42 @@ where
.and_then(ProviderResponseObservation::status_code);
let (upstream_transport, http_version, websocket_pool) =
self.current_transport_observation();
let service_tier = self.current_service_tier();
let provider_metadata_json = self.current_provider_metadata_json();
self.engine
.store()
.finalize_model_request(ModelRequestFinalization {
request_id: self.request_id.clone(),
outcome: ExecutionOutcome::Succeeded,
send_state: UpstreamSendState::Sent,
attempt_count: self.attempts,
downstream_committed_at: self.downstream_committed_at,
client_status_code: self.client_status_code,
upstream_status_code,
client_response_id: self.client_response_id.clone(),
upstream_request_id,
upstream_response_id: self.upstream_response_id.clone(),
upstream_transport,
http_version,
websocket_pool,
provider_metadata_json,
error: None,
provider_error_code: None,
retry_after_ms: None,
usage: self.usage.clone(),
image_generation_succeeded: self.image_generation_succeeded(),
cost: self.cost.clone(),
timings: self.timings.clone(),
completed_at,
})
.await?;
if self.request_persisted {
best_effort_store_write(
"finalize_model_request",
&self.request_id,
self.engine
.store()
.finalize_model_request(ModelRequestFinalization {
request_id: self.request_id.clone(),
outcome: ExecutionOutcome::Succeeded,
send_state: UpstreamSendState::Sent,
attempt_count: self.attempts,
downstream_committed_at: self.downstream_committed_at,
client_status_code: self.client_status_code,
upstream_status_code,
client_response_id: self.client_response_id.clone(),
upstream_request_id,
upstream_response_id: self.upstream_response_id.clone(),
upstream_transport,
http_version,
websocket_pool,
service_tier,
provider_metadata_json,
error: None,
provider_error_code: None,
retry_after_ms: None,
usage: self.usage.clone(),
image_generation_succeeded: self.image_generation_succeeded(),
cost: self.cost.clone(),
timings: self.timings.clone(),
completed_at,
}),
)
.await;
}
self.finalized = true;
Ok(())
}
@@ -1072,11 +1245,15 @@ where
UpstreamSendState::Ambiguous
};
let send_state = self.raise_send_watermark(send_state);
if self.attempts > 0 {
self.engine
.store()
.mark_send_state(&self.request_id, send_state)
.await?;
if self.attempts > 0 && self.request_persisted {
best_effort_store_write(
"mark_send_state",
&self.request_id,
self.engine
.store()
.mark_send_state(&self.request_id, send_state),
)
.await;
}
self.finish_failure(FailureFinalization {
outcome,
@@ -1126,36 +1303,44 @@ where
.and_then(ProviderResponseObservation::status_code);
let (upstream_transport, http_version, websocket_pool) =
self.current_transport_observation();
let service_tier = self.current_service_tier();
let provider_metadata_json = self.current_provider_metadata_json();
self.engine
.store()
.finalize_model_request(ModelRequestFinalization {
request_id: self.request_id.clone(),
outcome: finalization.outcome,
send_state: finalization.send_state,
attempt_count: self.attempts,
downstream_committed_at: self.downstream_committed_at,
client_status_code: self.client_status_code,
upstream_status_code: finalization.upstream_status_code.or(observed_status_code),
client_response_id: self.client_response_id.clone(),
upstream_request_id,
upstream_response_id: finalization
.provider_response_id
.or_else(|| self.upstream_response_id.clone()),
upstream_transport,
http_version,
websocket_pool,
provider_metadata_json,
error: Some(finalization.error),
provider_error_code: finalization.provider_error_code,
retry_after_ms: finalization.retry_after_ms,
usage: self.usage.clone(),
image_generation_succeeded: self.image_generation_succeeded(),
cost: self.cost.clone(),
timings: self.timings.clone(),
completed_at,
})
.await?;
best_effort_store_write(
"finalize_model_request",
&self.request_id,
self.engine
.store()
.finalize_model_request(ModelRequestFinalization {
request_id: self.request_id.clone(),
outcome: finalization.outcome,
send_state: finalization.send_state,
attempt_count: self.attempts,
downstream_committed_at: self.downstream_committed_at,
client_status_code: self.client_status_code,
upstream_status_code: finalization
.upstream_status_code
.or(observed_status_code),
client_response_id: self.client_response_id.clone(),
upstream_request_id,
upstream_response_id: finalization
.provider_response_id
.or_else(|| self.upstream_response_id.clone()),
upstream_transport,
http_version,
websocket_pool,
service_tier,
provider_metadata_json,
error: Some(finalization.error),
provider_error_code: finalization.provider_error_code,
retry_after_ms: finalization.retry_after_ms,
usage: self.usage.clone(),
image_generation_succeeded: self.image_generation_succeeded(),
cost: self.cost.clone(),
timings: self.timings.clone(),
completed_at,
}),
)
.await;
self.finalized = true;
Ok(())
}
@@ -1186,6 +1371,14 @@ where
.map(|metadata| metadata.as_json().to_owned())
}
fn current_service_tier(&self) -> Option<String> {
self.current
.as_ref()
.and_then(|current| current.response_observation.as_ref())
.and_then(ProviderResponseObservation::service_tier)
.map(str::to_owned)
}
fn image_generation_succeeded(&self) -> Option<bool> {
self.image_generation_requested
.then(|| self.usage.image_output_tokens.unwrap_or_default() > 0)
@@ -1235,12 +1428,62 @@ where
}
}
async fn best_effort_store_write<T>(
operation: &'static str,
request_id: &ModelRequestId,
write: impl Future<Output = Result<T, StoreError>>,
) -> Option<T> {
match write.await {
Ok(value) => Some(value),
Err(error) => {
tracing::warn!(
operation,
request_id = request_id.as_str(),
error_kind = ?error.kind(),
"执行观测写入失败,数据面不受影响"
);
None
}
}
}
enum PullOutcome {
Event(ProviderEvent),
AttemptDiscarded,
TerminalFailure {
events: Vec<ProviderEvent>,
error: ProviderError,
send_state: UpstreamSendState,
},
End,
}
enum StreamErrorOutcome {
AttemptDiscarded,
TerminalFailure {
events: Vec<ProviderEvent>,
error: ProviderError,
send_state: UpstreamSendState,
},
}
impl StreamErrorOutcome {
fn into_pull_outcome(self) -> PullOutcome {
match self {
Self::AttemptDiscarded => PullOutcome::AttemptDiscarded,
Self::TerminalFailure {
events,
error,
send_state,
} => PullOutcome::TerminalFailure {
events,
error,
send_state,
},
}
}
}
enum PollBoundary {
Item(Option<Result<ProviderEvent, ProviderError>>),
Cancelled,
@@ -1313,11 +1556,11 @@ fn initial_continuation_attempt(
}
}
fn provider_engine_error(error: &ProviderError) -> EngineError {
fn provider_engine_error(error: ProviderError) -> EngineError {
if error.kind() == ProviderErrorKind::Cancelled {
EngineError::Cancelled
} else {
EngineError::Provider(error.clone())
EngineError::Provider(error)
}
}
@@ -44,6 +44,46 @@ impl fmt::Display for ProviderAccountId {
}
}
/// credential 轮换时可选替换的上游账号身份。
#[derive(Clone, PartialEq, Eq)]
pub struct ProviderAccountIdentity {
upstream_user_id: String,
upstream_account_id: Option<String>,
}
impl fmt::Debug for ProviderAccountIdentity {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("ProviderAccountIdentity")
.field("upstream_user_id", &"<redacted>")
.field(
"upstream_account_id",
&self.upstream_account_id.as_ref().map(|_| "<redacted>"),
)
.finish()
}
}
impl ProviderAccountIdentity {
#[must_use]
pub const fn new(upstream_user_id: String, upstream_account_id: Option<String>) -> Self {
Self {
upstream_user_id,
upstream_account_id,
}
}
#[must_use]
pub fn upstream_user_id(&self) -> &str {
&self.upstream_user_id
}
#[must_use]
pub fn upstream_account_id(&self) -> Option<&str> {
self.upstream_account_id.as_deref()
}
}
/// `provider_accounts.credential_revision` 的正数 CAS revision。
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct CredentialRevision(NonZeroU64);
@@ -602,6 +642,15 @@ pub trait ProviderAccountStore: Send + Sync {
expected_revision: CredentialRevision,
) -> Result<LoadedCredential, StoreError>;
/// 读取账号当前 credential 及其 revision,不做任何版本比对。
///
/// 管理写入必须在临近 CAS 时用它取 fence,而不是携带调用方持有的旧 revision:
/// 后台刷新随时会推进 revision,用陈旧快照会把正常的恢复操作误判为冲突。
async fn load_current_credential(
&self,
account: &ProviderAccountId,
) -> Result<LoadedCredential, StoreError>;
async fn compare_and_swap_credential(
&self,
update: CredentialCasUpdate,
@@ -901,6 +950,63 @@ pub struct AccountSelectionContext {
pub excluded_accounts: BTreeSet<ProviderAccountId>,
pub preferred_account: Option<ProviderAccountId>,
pub round_robin_cursor: u64,
pub availability: AccountAvailabilityPolicy,
}
/// 选择账号时是否信任本地可用性投影。
///
/// 管理端对指定账号执行诊断时,会直接向上游确认实际状态;该模式跳过
/// `enabled`、可用性、冷却和 token 到期等本地投影,仍保留租约、并发和
/// 请求间隔约束,且只能与固定账号约束组合使用。
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum AccountAvailabilityPolicy {
#[default]
Enforce,
BypassForDiagnostic,
}
impl AccountAvailabilityPolicy {
#[must_use]
pub const fn bypasses_local_availability(self) -> bool {
matches!(self, Self::BypassForDiagnostic)
}
}
/// 候选账号未进入调度池的约束。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AccountSchedulingBlocker {
LocalAvailability,
Excluded,
ConcurrencyLimit,
RequestInterval,
}
/// 优先账号在本次选择中的处理结果。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum PreferredAccountSelection {
NotRequested,
Hit,
Missing,
Blocked(AccountSchedulingBlocker),
}
/// 一次完整账号选择;同时保留优先账号决策,供 Provider 记录调度遥测。
#[derive(Debug, Clone, Copy)]
pub struct AccountSelection<'a> {
candidate: &'a AccountCandidate,
preferred: PreferredAccountSelection,
}
impl<'a> AccountSelection<'a> {
#[must_use]
pub const fn candidate(self) -> &'a AccountCandidate {
self.candidate
}
#[must_use]
pub const fn preferred(self) -> PreferredAccountSelection {
self.preferred
}
}
/// 同一 target 内唯一的账号排序器。
@@ -914,43 +1020,55 @@ impl AccountSelector {
&self,
candidates: &'a [AccountCandidate],
context: &AccountSelectionContext,
) -> Option<&'a AccountCandidate> {
) -> Option<AccountSelection<'a>> {
let preferred = if let Some(preferred) = context.preferred_account.as_ref() {
match candidates
.iter()
.find(|candidate| candidate.account.id() == preferred)
{
Some(candidate) => match self.scheduling_blocker(candidate, context) {
None => {
return Some(AccountSelection {
candidate,
preferred: PreferredAccountSelection::Hit,
});
}
Some(blocker) => PreferredAccountSelection::Blocked(blocker),
},
None => PreferredAccountSelection::Missing,
}
} else {
PreferredAccountSelection::NotRequested
};
let mut eligible = candidates
.iter()
.filter(|candidate| eligible(candidate, context))
.filter(|candidate| self.scheduling_blocker(candidate, context).is_none())
.collect::<Vec<_>>();
if eligible.is_empty() {
return None;
}
if let Some(preferred) = context.preferred_account.as_ref()
&& let Some(candidate) = eligible
.iter()
.find(|candidate| candidate.account.id() == preferred)
&& (context.policy.strategy() == RotationStrategy::Sticky
|| (context.policy.strategy() == RotationStrategy::Smart
&& !smart_preference_should_escape(&candidate.signals)))
{
return Some(candidate);
}
match context.policy.strategy() {
RotationStrategy::QuotaResetPriority => eligible.sort_by_key(|candidate| {
(
candidate.signals.quota_reset_at.is_none(),
candidate.signals.quota_reset_at,
candidate.signals.in_flight,
candidate.signals.last_started_at,
candidate.account.id().clone(),
)
}),
let candidate = match context.policy.strategy() {
RotationStrategy::QuotaResetPriority => {
eligible.sort_by_key(|candidate| {
(
candidate.signals.quota_reset_at.is_none(),
candidate.signals.quota_reset_at,
candidate.signals.in_flight,
candidate.signals.last_started_at,
candidate.account.id().clone(),
)
});
eligible.first().copied()?
}
RotationStrategy::RoundRobin => {
eligible.sort_by_key(|candidate| candidate.account.id().clone());
let index = context.round_robin_cursor as usize % eligible.len();
return Some(eligible[index]);
eligible.get(index).copied()?
}
RotationStrategy::Smart => {
return select_smart_candidate(&eligible, context.round_robin_cursor);
select_smart_candidate(&eligible, context.round_robin_cursor)?
}
RotationStrategy::Sticky => {
eligible.sort_by_key(|candidate| {
@@ -959,9 +1077,44 @@ impl AccountSelector {
candidate.account.id().clone(),
)
});
eligible.first().copied()?
}
};
Some(AccountSelection {
candidate,
preferred,
})
}
fn scheduling_blocker(
&self,
candidate: &AccountCandidate,
context: &AccountSelectionContext,
) -> Option<AccountSchedulingBlocker> {
if !context.availability.bypasses_local_availability()
&& !candidate.account.is_schedulable(context.now)
{
return Some(AccountSchedulingBlocker::LocalAvailability);
}
eligible.into_iter().next()
if context.excluded_accounts.contains(candidate.account.id()) {
return Some(AccountSchedulingBlocker::Excluded);
}
if candidate.signals.in_flight >= context.policy.max_concurrent_per_account().get() {
return Some(AccountSchedulingBlocker::ConcurrencyLimit);
}
if candidate
.signals
.last_started_at
.is_some_and(|last_started| {
!context
.now
.duration_since(last_started)
.is_ok_and(|elapsed| elapsed >= context.policy.request_interval())
})
{
return Some(AccountSchedulingBlocker::RequestInterval);
}
None
}
}
@@ -969,8 +1122,6 @@ const SMART_LOAD_WEIGHT: f64 = 1.0;
const SMART_QUOTA_WEIGHT: f64 = 0.8;
const SMART_FAILURE_WEIGHT: f64 = 1.0;
const SMART_LATENCY_WEIGHT: f64 = 0.5;
const SMART_PREFERENCE_FAILURE_ESCAPE_BASIS_POINTS: u16 = 5_000;
const SMART_PREFERENCE_LATENCY_ESCAPE_MS: u64 = 15_000;
struct SmartNormalization {
max_in_flight: u32,
@@ -1091,34 +1242,6 @@ fn higher_is_better(value: u64, minimum: u64, maximum: u64) -> f64 {
(value.saturating_sub(minimum) as f64 / (maximum - minimum) as f64).clamp(0.0, 1.0)
}
fn smart_preference_should_escape(signals: &AccountRuntimeSignals) -> bool {
signals
.failure_rate_basis_points
.is_some_and(|rate| rate > SMART_PREFERENCE_FAILURE_ESCAPE_BASIS_POINTS)
|| signals
.first_output_latency_ms
.is_some_and(|latency| latency > SMART_PREFERENCE_LATENCY_ESCAPE_MS)
}
fn eligible(candidate: &AccountCandidate, context: &AccountSelectionContext) -> bool {
if !candidate.account.is_schedulable(context.now)
|| context.excluded_accounts.contains(candidate.account.id())
|| candidate.signals.in_flight >= context.policy.max_concurrent_per_account().get()
{
return false;
}
candidate
.signals
.last_started_at
.is_none_or(|last_started| {
context
.now
.duration_since(last_started)
.is_ok_and(|elapsed| elapsed >= context.policy.request_interval())
})
}
/// Credential 值对象构造错误。
#[derive(Debug, Clone, Copy, PartialEq, Eq, Error)]
pub enum CredentialError {
@@ -17,7 +17,10 @@ use crate::engine::continuation::{
ContinuationBinding, NativeContinuationPin, NativeContinuationPort, PreviousResponseId,
};
use crate::engine::coordinator::ResponseExecutionSession;
use crate::engine::probe::{AccountProbe, AccountProbeRequest, AccountProbeResult};
use crate::engine::probe::{
AccountProbe, AccountProbeError, AccountProbeRequest, AccountProbeResult,
AccountProbeUpstreamResponse,
};
use crate::engine::provider::ProviderRegistry;
use crate::engine::{
AttemptCoordinator, AttemptRecord, CancellationToken, CoordinatedEvent, EngineError,
@@ -36,7 +39,7 @@ use crate::routing::{
};
const MODEL_REQUEST_DEADLINE: Duration = Duration::from_secs(10 * 60);
const CONTINUATION_AFFINITY_TIMEOUT: Duration = Duration::from_millis(100);
const REDIS_COORDINATION_TIMEOUT: Duration = Duration::from_millis(100);
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ClientTransport {
@@ -233,15 +236,6 @@ impl DefaultExecutionService {
request.client.policy.authorize().map_err(|_| {
GatewayError::new(GatewayErrorKind::PolicyDenied, "client API key is disabled")
})?;
if !request.client.snapshot.contains_public_model_for_provider(
&request.public_model,
request.client.policy.provider_kind(),
) {
return Err(GatewayError::new(
GatewayErrorKind::ModelNotFound,
"requested model was not found",
));
}
let started_at = SystemTime::now();
let deadline_at = started_at
.checked_add(MODEL_REQUEST_DEADLINE)
@@ -260,7 +254,7 @@ impl DefaultExecutionService {
let continuation = match request.metadata.previous_response_id.as_ref() {
Some(previous) => {
let resolve = self.continuation.resolve(previous).fuse();
let timeout = Delay::new(CONTINUATION_AFFINITY_TIMEOUT).fuse();
let timeout = Delay::new(REDIS_COORDINATION_TIMEOUT).fuse();
pin_mut!(resolve, timeout);
let pin = select_biased! {
result = resolve => result.ok().flatten(),
@@ -398,14 +392,33 @@ impl DefaultExecutionService {
&self,
provider_kind: &ProviderKind,
) -> Result<RoutingContext, GatewayError> {
let blocked_providers = match self.circuits.decision(provider_kind).await.map_err(|_| {
GatewayError::new(
GatewayErrorKind::NoAvailableProvider,
"provider health state is temporarily unavailable",
)
})? {
ProviderCircuitDecision::Allow => BTreeSet::new(),
ProviderCircuitDecision::BlockedUntil(_) => BTreeSet::from([provider_kind.clone()]),
let decision = self.circuits.decision(provider_kind).fuse();
let timeout = Delay::new(REDIS_COORDINATION_TIMEOUT).fuse();
pin_mut!(decision, timeout);
let decision = select_biased! {
result = decision => Some(result),
_ = timeout => None,
};
let blocked_providers = match decision {
Some(Ok(ProviderCircuitDecision::Allow)) => BTreeSet::new(),
Some(Ok(ProviderCircuitDecision::BlockedUntil(_))) => {
BTreeSet::from([provider_kind.clone()])
}
Some(Err(error)) => {
tracing::warn!(
provider = provider_kind.as_str(),
%error,
"Provider circuit 读取失败,按可重建协调状态 fail-open"
);
BTreeSet::new()
}
None => {
tracing::warn!(
provider = provider_kind.as_str(),
"Provider circuit 读取超时,按可重建协调状态 fail-open"
);
BTreeSet::new()
}
};
Ok(RoutingContext {
provider_kind: Some(provider_kind.clone()),
@@ -416,7 +429,7 @@ impl DefaultExecutionService {
async fn probe_inner(
&self,
request: AccountProbeRequest,
) -> Result<AccountProbeResult, GatewayError> {
) -> Result<AccountProbeResult, AccountProbeError> {
let AccountProbeRequest {
account_id,
provider_kind,
@@ -477,11 +490,11 @@ impl DefaultExecutionService {
};
let mut session = match self
.probe_coordinator
.start(
.start_diagnostic(
new_request,
operation,
plan,
Some(account_id),
account_id,
None,
CancellationToken::new(),
)
@@ -525,26 +538,55 @@ impl DefaultExecutionService {
})
}
/// 探测失败先落库再转成客户端错误;转换会丢掉上游状态码与 provider code。
/// 探测失败先记录脱敏分类事实,再把请求局部的原始上游响应交给认证管理端。
async fn observe_probe_failure(
&self,
observed: &ProbeObservation,
started_at: SystemTime,
error: &EngineError,
) -> GatewayError {
) -> AccountProbeError {
let upstream_response = match error {
EngineError::Provider(provider_error) => provider_error
.client_visible_upstream_response()
.map(AccountProbeUpstreamResponse::from_client_response),
_ => None,
};
if let EngineError::Provider(provider_error) = error {
let _ = self
let latency = started_at.elapsed().unwrap_or_default();
let latency_ms = u64::try_from(latency.as_millis()).unwrap_or(u64::MAX);
tracing::warn!(
target: "gateway_probe",
provider_kind = observed.provider_kind.as_str(),
account_id = observed.account_id.as_str(),
upstream_model = observed.upstream_model.as_str(),
failure_kind = provider_error.kind().as_str(),
send_state = ?provider_error.send_state(),
upstream_status = ?provider_error.upstream_status(),
provider_error_code = ?provider_error.upstream_code().map(|code| code.as_str()),
latency_ms,
"账号连接测试失败"
);
if let Err(store_error) = self
.observations
.record_probe_failure(ProbeFailure {
provider_kind: observed.provider_kind.clone(),
account_id: observed.account_id.clone(),
upstream_model_id: observed.upstream_model.clone(),
error: provider_error.clone(),
latency: started_at.elapsed().unwrap_or_default(),
latency,
})
.await;
.await
{
tracing::warn!(
operation = "record_probe_failure",
provider_kind = observed.provider_kind.as_str(),
account_id = observed.account_id.as_str(),
error_kind = ?store_error.kind(),
"账号连接测试观测写入失败,测试结果不受影响"
);
}
}
gateway_error_from_engine(error)
AccountProbeError::new(gateway_error_from_engine(error), upstream_response)
}
}
@@ -651,7 +693,7 @@ impl AccountProbe for DefaultExecutionService {
fn probe(
&self,
request: AccountProbeRequest,
) -> BoxFuture<'_, Result<AccountProbeResult, GatewayError>> {
) -> BoxFuture<'_, Result<AccountProbeResult, AccountProbeError>> {
Box::pin(async move { self.probe_inner(request).await })
}
}
@@ -664,10 +706,13 @@ struct AdmissionLease {
impl AdmissionLease {
async fn release(self) {
let _ = self
if let Err(error) = self
.port
.release(&self.client_api_key_id, &self.model_request_id)
.await;
.await
{
tracing::warn!(%error, "Client admission 释放失败,依赖租约 TTL 收敛");
}
}
}
@@ -742,13 +787,7 @@ impl DefaultExecutionSession {
return;
};
self.continuation_recorded = true;
let record = self.continuation.record(pin).fuse();
let timeout = Delay::new(CONTINUATION_AFFINITY_TIMEOUT).fuse();
pin_mut!(record, timeout);
select_biased! {
_ = record => {},
_ = timeout => {},
}
record_native_continuation(self.continuation.as_ref(), pin).await;
}
async fn finalize_detached(&mut self) {
@@ -756,8 +795,10 @@ impl DefaultExecutionSession {
return;
};
core.cancel();
if !core.is_finalized() {
let _ = core.cancel_and_finalize().await;
if !core.is_finalized()
&& let Err(error) = core.cancel_and_finalize().await
{
tracing::warn!(%error, "Detached execution 终态收敛失败");
}
let pending = core
.provider_attempt_outcomes()
@@ -878,7 +919,32 @@ async fn publish_provider_attempt_outcomes(
}
Some(_) => continue,
};
let _ = result;
if let Err(error) = result {
tracing::warn!(
provider = outcome.provider_kind().as_str(),
%error,
"Provider circuit feedback 写入失败,数据面不受影响"
);
}
}
}
async fn record_native_continuation(
continuation: &dyn NativeContinuationPort,
pin: NativeContinuationPin,
) {
let record = continuation.record(pin).fuse();
let timeout = Delay::new(REDIS_COORDINATION_TIMEOUT).fuse();
pin_mut!(record, timeout);
select_biased! {
result = record => {
if let Err(error) = result {
tracing::warn!(%error, "Continuation affinity 写入失败,后续请求将退化为外部续接");
}
},
_ = timeout => {
tracing::warn!("Continuation affinity 后台写入超时,已丢弃本次亲和记录");
},
}
}
@@ -232,6 +232,7 @@ pub struct AccountAttemptContext {
required_account: Option<ProviderAccountId>,
state_owner: Option<ProviderAccountStateOwner>,
credential_recovery_attempted: bool,
diagnostic_required_account: bool,
}
impl AccountAttemptContext {
@@ -246,6 +247,25 @@ impl AccountAttemptContext {
required_account,
state_owner,
credential_recovery_attempted: false,
diagnostic_required_account: false,
}
}
/// 为管理端对固定账号的上游诊断创建上下文。
///
/// 诊断只绕过本地的被动可用性投影,不能换号,并继续受账号租约保护。
#[must_use]
pub const fn diagnostic(
excluded_accounts: BTreeSet<ProviderAccountId>,
required_account: ProviderAccountId,
state_owner: Option<ProviderAccountStateOwner>,
) -> Self {
Self {
excluded_accounts,
required_account: Some(required_account),
state_owner,
credential_recovery_attempted: false,
diagnostic_required_account: true,
}
}
@@ -275,6 +295,12 @@ impl AccountAttemptContext {
pub const fn credential_recovery_attempted(&self) -> bool {
self.credential_recovery_attempted
}
/// 此尝试是否要由真实上游而非本地投影确认固定账号的可用性。
#[must_use]
pub const fn is_diagnostic_required_account(&self) -> bool {
self.diagnostic_required_account
}
}
/// 请求中 Provider 账号绑定状态的唯一归属。
@@ -474,6 +500,12 @@ impl AttemptContext {
self.account.credential_recovery_attempted()
}
/// 管理端固定账号诊断会跳过本地可用性投影,保留全部租约约束。
#[must_use]
pub const fn is_diagnostic_required_account(&self) -> bool {
self.account.is_diagnostic_required_account()
}
#[must_use]
pub const fn continuation(&self) -> Option<&ContinuationBinding> {
self.continuation.as_ref()
@@ -583,6 +615,7 @@ pub struct ModelRequestFinalization {
pub upstream_transport: Option<String>,
pub http_version: Option<String>,
pub websocket_pool: Option<String>,
pub service_tier: Option<String>,
/// Provider 已筛选的专有观测 JSON;Core 不解释字段。
pub provider_metadata_json: Option<String>,
pub error: Option<GatewayError>,
+107 -2
View File
@@ -1,8 +1,11 @@
//! 管理端账号连通性测试的真实执行链端口。
use std::fmt;
use bytes::Bytes;
use futures::future::BoxFuture;
use crate::error::GatewayError;
use crate::error::{ClientVisibleUpstreamResponse, GatewayError, GatewayErrorKind};
use crate::routing::{ProviderKind, UpstreamModelId};
use crate::{engine::credential::ProviderAccountId, operation::Operation};
@@ -19,9 +22,111 @@ pub struct AccountProbeResult {
pub text: Vec<String>,
}
/// 仅供当前管理端连接测试展示的原始上游失败响应。
///
/// 正文不进入 `Debug`、日志或持久化;Core 只在探测终态从原始 Provider 错误显式复制。
#[derive(PartialEq, Eq)]
pub struct AccountProbeUpstreamResponse {
status: u16,
content_type: Option<Vec<u8>>,
body: Bytes,
}
impl AccountProbeUpstreamResponse {
pub(crate) fn from_client_response(response: &ClientVisibleUpstreamResponse) -> Self {
Self {
status: response.status(),
content_type: response.content_type().map(<[u8]>::to_vec),
body: response.body().clone(),
}
}
#[must_use]
pub const fn status(&self) -> u16 {
self.status
}
#[must_use]
pub fn content_type(&self) -> Option<&[u8]> {
self.content_type.as_deref()
}
#[must_use]
pub const fn body(&self) -> &Bytes {
&self.body
}
}
impl fmt::Debug for AccountProbeUpstreamResponse {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("AccountProbeUpstreamResponse")
.field("status", &self.status)
.field(
"content_type",
&self.content_type.as_ref().map(|_| "<present>"),
)
.field("body", &"<redacted>")
.finish()
}
}
/// 管理端连接测试的终态错误。
///
/// `gateway` 保留稳定分类,`upstream_response` 只面向本次认证管理请求展示源响应。
#[derive(Debug)]
pub struct AccountProbeError {
gateway: GatewayError,
upstream_response: Option<AccountProbeUpstreamResponse>,
}
impl AccountProbeError {
#[must_use]
pub const fn new(
gateway: GatewayError,
upstream_response: Option<AccountProbeUpstreamResponse>,
) -> Self {
Self {
gateway,
upstream_response,
}
}
#[must_use]
pub const fn kind(&self) -> GatewayErrorKind {
self.gateway.kind()
}
#[must_use]
pub fn client_message(&self) -> &str {
self.gateway.client_message()
}
#[must_use]
pub fn client_error_type(&self) -> Option<&str> {
self.gateway.client_error_type()
}
#[must_use]
pub fn client_error_code(&self) -> Option<&str> {
self.gateway.client_error_code()
}
#[must_use]
pub const fn upstream_response(&self) -> Option<&AccountProbeUpstreamResponse> {
self.upstream_response.as_ref()
}
}
impl From<GatewayError> for AccountProbeError {
fn from(gateway: GatewayError) -> Self {
Self::new(gateway, None)
}
}
pub trait AccountProbe: Send + Sync {
fn probe(
&self,
request: AccountProbeRequest,
) -> BoxFuture<'_, Result<AccountProbeResult, GatewayError>>;
) -> BoxFuture<'_, Result<AccountProbeResult, AccountProbeError>>;
}
@@ -16,7 +16,7 @@ use crate::engine::credential::{
};
use crate::engine::{AttemptContext, UpstreamSendState};
use crate::error::{
IdentifierError, ProviderError, ProviderErrorKind, SafeUpstreamValue, validate_text,
IdentifierError, OpaqueUpstreamValue, ProviderError, ProviderErrorKind, validate_text,
};
use crate::event::{EventSequenceValidator, ProviderEvent};
use crate::operation::Operation;
@@ -125,7 +125,7 @@ pub struct ProviderCallMetadata {
provider: ProviderKind,
upstream_model: UpstreamModelId,
resource: ProviderResource,
upstream_request_id: Option<SafeUpstreamValue>,
upstream_request_id: Option<OpaqueUpstreamValue>,
transport: UpstreamTransport,
}
@@ -149,7 +149,7 @@ impl ProviderCallMetadata {
/// 设置 adapter 已分类为非 bearer 的 request ID。
#[must_use]
pub fn with_upstream_request_id(mut self, request_id: SafeUpstreamValue) -> Self {
pub fn with_upstream_request_id(mut self, request_id: OpaqueUpstreamValue) -> Self {
self.upstream_request_id = Some(request_id);
self
}
@@ -186,7 +186,7 @@ impl ProviderCallMetadata {
/// 返回安全上游 request ID。
#[must_use]
pub const fn upstream_request_id(&self) -> Option<&SafeUpstreamValue> {
pub const fn upstream_request_id(&self) -> Option<&OpaqueUpstreamValue> {
self.upstream_request_id.as_ref()
}
+240 -65
View File
@@ -3,9 +3,11 @@
use std::fmt;
use std::time::Duration;
use bytes::Bytes;
use thiserror::Error;
use crate::engine::UpstreamSendState;
use crate::event::{ProviderEvent, ProviderResponseHeader};
/// 应用层标识不满足约束。
#[derive(Debug, Clone, PartialEq, Eq, Error)]
@@ -174,22 +176,18 @@ pub enum ContinuationFailure {
HistoryUnavailable,
}
/// Adapter 已明确分类为非 bearer、可用于诊断的上游值。
/// Adapter 已明确分类为非 bearer 的不透明上游值。
///
/// Core 不解释或校验内容,只通过自定义 [`Debug`] 避免它意外进入日志。协议 adapter
/// 可在原客户端响应中读取原值。
#[derive(Clone, PartialEq, Eq, Hash)]
pub struct SafeUpstreamValue(String);
pub struct OpaqueUpstreamValue(String);
impl SafeUpstreamValue {
const MAX_BYTES: usize = 256;
/// 校验并创建诊断值。
///
/// # Errors
///
/// 空值、超长值或控制字符会返回 [`IdentifierError`]。
pub fn new(value: impl Into<String>) -> Result<Self, IdentifierError> {
let value = value.into();
validate_text(&value, Self::MAX_BYTES, false, None)?;
Ok(Self(value))
impl OpaqueUpstreamValue {
/// 原样保存上游值。
#[must_use]
pub fn new(value: impl Into<String>) -> Self {
Self(value.into())
}
/// 返回已经由 adapter 安全分类的原值。
@@ -199,9 +197,9 @@ impl SafeUpstreamValue {
}
}
impl fmt::Debug for SafeUpstreamValue {
impl fmt::Debug for OpaqueUpstreamValue {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.write_str("SafeUpstreamValue(<redacted-from-Debug>)")
formatter.write_str("OpaqueUpstreamValue(<redacted-from-Debug>)")
}
}
@@ -212,31 +210,23 @@ impl fmt::Debug for SafeUpstreamValue {
#[derive(Clone, PartialEq, Eq)]
pub struct ClientVisibleUpstreamError {
message: String,
code: Option<SafeUpstreamValue>,
error_type: Option<SafeUpstreamValue>,
code: Option<OpaqueUpstreamValue>,
error_type: Option<OpaqueUpstreamValue>,
}
impl ClientVisibleUpstreamError {
const MAX_MESSAGE_BYTES: usize = 8_192;
/// 从结构化上游 `message`、`code` 和 `type` 创建客户端可见错误。
///
/// # Errors
///
/// message 为空、过长或含控制字符时返回错误;可选 code/type 不满足安全
/// 约束时会被省略,不会把整段上游 body 重新暴露。
/// 原样保存结构化上游 `message`、`code` 和 `type`。
#[must_use]
pub fn new(
message: impl Into<String>,
code: Option<String>,
error_type: Option<String>,
) -> Result<Self, IdentifierError> {
let message = message.into();
validate_text(&message, Self::MAX_MESSAGE_BYTES, false, None)?;
Ok(Self {
message,
code: code.and_then(|value| SafeUpstreamValue::new(value).ok()),
error_type: error_type.and_then(|value| SafeUpstreamValue::new(value).ok()),
})
) -> Self {
Self {
message: message.into(),
code: code.map(OpaqueUpstreamValue::new),
error_type: error_type.map(OpaqueUpstreamValue::new),
}
}
/// 返回原上游的结构化 message。
@@ -248,13 +238,13 @@ impl ClientVisibleUpstreamError {
/// 返回原上游的结构化 code。
#[must_use]
pub fn code(&self) -> Option<&str> {
self.code.as_ref().map(SafeUpstreamValue::as_str)
self.code.as_ref().map(OpaqueUpstreamValue::as_str)
}
/// 返回原上游的结构化 type。
#[must_use]
pub fn error_type(&self) -> Option<&str> {
self.error_type.as_ref().map(SafeUpstreamValue::as_str)
self.error_type.as_ref().map(OpaqueUpstreamValue::as_str)
}
}
@@ -264,28 +254,110 @@ impl fmt::Debug for ClientVisibleUpstreamError {
}
}
/// 只供当前客户端请求使用的原始上游 HTTP 失败响应。
///
/// 该值不属于稳定诊断事实,不能进入日志或持久化。它刻意不实现 [`Clone`];
/// [`ProviderError`] 的普通 clone 会丢弃它,只有最终失败的原对象才能把响应交给
/// 原客户端协议,或由认证管理端的账号连接测试显式复制到请求局部结果。
#[derive(PartialEq, Eq)]
pub struct ClientVisibleUpstreamResponse {
status: u16,
content_type: Option<Vec<u8>>,
headers: Vec<ProviderResponseHeader>,
body: Bytes,
}
impl ClientVisibleUpstreamResponse {
/// 保存 transport 实际收到的状态码、Content-Type 原值和正文。
#[must_use]
pub fn new(status: u16, content_type: Option<Vec<u8>>, body: Bytes) -> Self {
Self {
status,
content_type,
headers: Vec::new(),
body,
}
}
/// 附加 Provider 已剔除账号绑定与 framing 信息的普通响应头。
#[must_use]
pub fn with_headers(mut self, headers: Vec<ProviderResponseHeader>) -> Self {
self.headers = headers;
self
}
#[must_use]
pub const fn status(&self) -> u16 {
self.status
}
#[must_use]
pub fn content_type(&self) -> Option<&[u8]> {
self.content_type.as_deref()
}
#[must_use]
pub fn headers(&self) -> &[ProviderResponseHeader] {
&self.headers
}
#[must_use]
pub const fn body(&self) -> &Bytes {
&self.body
}
}
impl fmt::Debug for ClientVisibleUpstreamResponse {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("ClientVisibleUpstreamResponse")
.field("status", &self.status)
.field(
"content_type",
&self.content_type.as_ref().map(|_| "<present>"),
)
.field("header_count", &self.headers.len())
.field("body", &"<redacted>")
.finish()
}
}
/// 单次 Provider 调用的稳定错误。
///
/// 该类型不接收原始响应正文,也不会在 `Debug` 或 `Display` 中打印上游
/// 稳定诊断字段不接收原始响应正文,也不会在 `Debug` 或 `Display` 中打印上游
/// code/request ID/response ID。Adapter 若捕获到可能含 secret 的上下文,只能调用
/// [`ProviderError::redact_sensitive_context`] 丢弃正文并留下脱敏标记。
#[derive(Clone)]
///
/// 唯一例外是 [`ProviderError::with_atomic_client_events`]:它只承载尚未交付客户端、
/// 必须与本错误一起完成重试判断的协议事件。Core 会在任何持久化或日志记录前取走
/// 这些事件,不能把它们当作诊断上下文使用。
pub struct ProviderError {
kind: ProviderErrorKind,
send_state: UpstreamSendState,
upstream_status: Option<u16>,
upstream_code: Option<SafeUpstreamValue>,
upstream_request_id: Option<SafeUpstreamValue>,
upstream_response_id: Option<SafeUpstreamValue>,
upstream_values: Option<Box<ProviderErrorUpstreamValues>>,
retry_after: Option<Duration>,
continuation_failure: Option<ContinuationFailure>,
replay_safe: bool,
pre_delivery_retry: bool,
credential_recovery_required: bool,
retry_same_account: bool,
sensitive_context_redacted: bool,
client_visible_upstream_error: Option<Box<ClientVisibleUpstreamError>>,
client_visible_upstream_response: Option<Box<ClientVisibleUpstreamResponse>>,
atomic_client_events: Option<Box<AtomicClientEvents>>,
}
#[derive(Clone, Default)]
struct ProviderErrorUpstreamValues {
code: Option<OpaqueUpstreamValue>,
request_id: Option<OpaqueUpstreamValue>,
response_id: Option<OpaqueUpstreamValue>,
}
#[derive(Clone)]
struct AtomicClientEvents(Vec<ProviderEvent>);
impl ProviderError {
/// 创建 Provider 错误。
#[must_use]
@@ -294,16 +366,17 @@ impl ProviderError {
kind,
send_state,
upstream_status: None,
upstream_code: None,
upstream_request_id: None,
upstream_response_id: None,
upstream_values: None,
retry_after: None,
continuation_failure: None,
replay_safe: false,
pre_delivery_retry: false,
credential_recovery_required: false,
retry_same_account: false,
sensitive_context_redacted: false,
client_visible_upstream_error: None,
client_visible_upstream_response: None,
atomic_client_events: None,
}
}
@@ -318,25 +391,30 @@ impl ProviderError {
/// 附加 adapter 已分类为安全的上游错误 code。
#[must_use]
pub fn with_upstream_code(mut self, code: SafeUpstreamValue) -> Self {
self.upstream_code = Some(code);
pub fn with_upstream_code(mut self, code: OpaqueUpstreamValue) -> Self {
self.upstream_values_mut().code = Some(code);
self
}
/// 附加 adapter 已分类为非 bearer 的上游 request ID。
#[must_use]
pub fn with_upstream_request_id(mut self, request_id: SafeUpstreamValue) -> Self {
self.upstream_request_id = Some(request_id);
pub fn with_upstream_request_id(mut self, request_id: OpaqueUpstreamValue) -> Self {
self.upstream_values_mut().request_id = Some(request_id);
self
}
/// 附加 adapter 已分类为非 bearer 的上游 response ID。
#[must_use]
pub fn with_upstream_response_id(mut self, response_id: SafeUpstreamValue) -> Self {
self.upstream_response_id = Some(response_id);
pub fn with_upstream_response_id(mut self, response_id: OpaqueUpstreamValue) -> Self {
self.upstream_values_mut().response_id = Some(response_id);
self
}
fn upstream_values_mut(&mut self) -> &mut ProviderErrorUpstreamValues {
self.upstream_values
.get_or_insert_with(|| Box::new(ProviderErrorUpstreamValues::default()))
}
/// 附加 Provider 建议的冷却时间。
#[must_use]
pub const fn with_retry_after(mut self, retry_after: Duration) -> Self {
@@ -358,6 +436,17 @@ impl ProviderError {
self
}
/// 允许 Core 仅在客户端尚未收到任何事件时执行一次受预算约束的换号恢复。
///
/// 该标记不证明上游未执行,也不等价于 [`Self::with_replay_safe`];它只表达
/// Provider 选择了“客户端无感恢复优先”的传输策略。Core 在 continuation、
/// 指定账号或下游已经进入提交状态时必须忽略它。
#[must_use]
pub const fn with_pre_delivery_retry(mut self) -> Self {
self.pre_delivery_retry = true;
self
}
/// 要求 Core 在账号凭据已恢复后仅对原账号重放一次。
#[must_use]
pub const fn with_same_account_retry(mut self) -> Self {
@@ -379,6 +468,27 @@ impl ProviderError {
self
}
/// 附加仅可返回给当前请求方的原始上游 HTTP 失败响应。
#[must_use]
pub fn with_client_visible_upstream_response(
mut self,
response: ClientVisibleUpstreamResponse,
) -> Self {
self.client_visible_upstream_response = Some(Box::new(response));
self
}
/// 附加必须先由 Core 判断重试、再决定丢弃或交付的客户端协议事件。
///
/// Provider 只能放入当前 attempt 尚未交付的事件;调用方应设置明确的有界缓冲,
/// 并保证失败事件本身位于这批事件的末尾。
#[must_use]
pub fn with_atomic_client_events(mut self, events: Vec<ProviderEvent>) -> Self {
self.atomic_client_events =
(!events.is_empty()).then(|| Box::new(AtomicClientEvents(events)));
self
}
/// 丢弃敏感正文,只记录“发生过脱敏”这一事实。
#[must_use]
pub fn redact_sensitive_context(mut self, _sensitive_context: impl AsRef<str>) -> Self {
@@ -406,20 +516,26 @@ impl ProviderError {
/// 返回安全分类的上游错误 code。
#[must_use]
pub fn upstream_code(&self) -> Option<&SafeUpstreamValue> {
self.upstream_code.as_ref()
pub fn upstream_code(&self) -> Option<&OpaqueUpstreamValue> {
self.upstream_values
.as_deref()
.and_then(|values| values.code.as_ref())
}
/// 返回安全分类的上游 request ID。
#[must_use]
pub fn upstream_request_id(&self) -> Option<&SafeUpstreamValue> {
self.upstream_request_id.as_ref()
pub fn upstream_request_id(&self) -> Option<&OpaqueUpstreamValue> {
self.upstream_values
.as_deref()
.and_then(|values| values.request_id.as_ref())
}
/// 返回安全分类的上游 response ID。
#[must_use]
pub fn upstream_response_id(&self) -> Option<&SafeUpstreamValue> {
self.upstream_response_id.as_ref()
pub fn upstream_response_id(&self) -> Option<&OpaqueUpstreamValue> {
self.upstream_values
.as_deref()
.and_then(|values| values.response_id.as_ref())
}
/// 返回 Provider 建议的冷却时间。
@@ -440,6 +556,12 @@ impl ProviderError {
self.replay_safe
}
/// 返回 Provider 是否允许在首个客户端事件前换号恢复传输失败。
#[must_use]
pub const fn allows_pre_delivery_retry(&self) -> bool {
self.pre_delivery_retry
}
/// 返回 Provider 是否已完成凭据恢复并要求原账号重放。
#[must_use]
pub const fn retries_same_account(&self) -> bool {
@@ -462,6 +584,50 @@ impl ProviderError {
pub fn client_visible_upstream_error(&self) -> Option<&ClientVisibleUpstreamError> {
self.client_visible_upstream_error.as_deref()
}
/// 返回只供当前协议响应使用的原始上游 HTTP 失败响应。
#[must_use]
pub fn client_visible_upstream_response(&self) -> Option<&ClientVisibleUpstreamResponse> {
self.client_visible_upstream_response.as_deref()
}
/// 取走只供本次重试/提交决策使用的原子客户端事件。
///
/// Core 必须在克隆错误、记录中间失败或构造终态前调用本方法,避免原始 wire
/// 进入诊断或持久化对象。
#[must_use]
pub fn take_atomic_client_events(&mut self) -> Vec<ProviderEvent> {
self.atomic_client_events
.take()
.map_or_else(Vec::new, |events| events.0)
}
/// 返回是否携带尚未提交的原子客户端事件;不暴露事件内容。
#[must_use]
pub fn has_atomic_client_events(&self) -> bool {
self.atomic_client_events.is_some()
}
}
impl Clone for ProviderError {
fn clone(&self) -> Self {
Self {
kind: self.kind,
send_state: self.send_state,
upstream_status: self.upstream_status,
upstream_values: self.upstream_values.clone(),
retry_after: self.retry_after,
continuation_failure: self.continuation_failure,
replay_safe: self.replay_safe,
pre_delivery_retry: self.pre_delivery_retry,
credential_recovery_required: self.credential_recovery_required,
retry_same_account: self.retry_same_account,
sensitive_context_redacted: self.sensitive_context_redacted,
client_visible_upstream_error: self.client_visible_upstream_error.clone(),
client_visible_upstream_response: None,
atomic_client_events: None,
}
}
}
impl fmt::Debug for ProviderError {
@@ -473,25 +639,20 @@ impl fmt::Debug for ProviderError {
.field("upstream_status", &self.upstream_status)
.field(
"upstream_code",
&self.upstream_code.as_ref().map(|_| "<classified-safe>"),
&self.upstream_code().map(|_| "<classified-safe>"),
)
.field(
"upstream_request_id",
&self
.upstream_request_id
.as_ref()
.map(|_| "<classified-safe>"),
&self.upstream_request_id().map(|_| "<classified-safe>"),
)
.field(
"upstream_response_id",
&self
.upstream_response_id
.as_ref()
.map(|_| "<classified-safe>"),
&self.upstream_response_id().map(|_| "<classified-safe>"),
)
.field("retry_after", &self.retry_after)
.field("continuation_failure", &self.continuation_failure)
.field("replay_safe", &self.replay_safe)
.field("pre_delivery_retry", &self.pre_delivery_retry)
.field(
"credential_recovery_required",
&self.credential_recovery_required,
@@ -508,6 +669,20 @@ impl fmt::Debug for ProviderError {
.as_ref()
.map(|_| "<present>"),
)
.field(
"client_visible_upstream_response",
&self
.client_visible_upstream_response
.as_ref()
.map(|_| "<present>"),
)
.field(
"atomic_client_events",
&self
.atomic_client_events
.as_ref()
.map_or(0, |events| events.0.len()),
)
.finish()
}
}
+48 -28
View File
@@ -9,7 +9,7 @@ use thiserror::Error;
use crate::accounting::{CalculatedCost, ProviderReportedCost, Usage};
use crate::engine::provider::UpstreamTransport;
use crate::error::{IdentifierError, SafeUpstreamValue, validate_text};
use crate::error::{IdentifierError, OpaqueUpstreamValue, validate_text};
use crate::operation::ProviderSessionState;
/// 一次响应的稳定元数据。
@@ -199,7 +199,7 @@ impl ProtocolWireEvent {
///
/// # Errors
///
/// 协议名或显式事件名为空、过长或含控制字符时返回错误。
/// 协议名不满足内部路由标识约束时返回错误。上游事件名保持不透明。
pub fn json(
protocol: impl Into<String>,
event_type: Option<String>,
@@ -212,7 +212,7 @@ impl ProtocolWireEvent {
///
/// # Errors
///
/// 协议名、显式事件名或 SSE ID 不满足 wire 安全约束时返回错误。
/// 协议名不满足内部路由标识约束时返回错误。SSE 元数据保持不透明。
pub fn json_with_sse_metadata(
protocol: impl Into<String>,
event_type: Option<String>,
@@ -222,15 +222,6 @@ impl ProtocolWireEvent {
) -> Result<Self, IdentifierError> {
let protocol = protocol.into();
validate_text(&protocol, 64, true, None)?;
if let Some(event_type) = event_type.as_deref() {
validate_text(event_type, 256, true, None)?;
}
if sse_id
.as_deref()
.is_some_and(|id| id.contains(['\0', '\r', '\n']))
{
return Err(IdentifierError::ControlCharacter);
}
Ok(Self {
protocol,
event_type,
@@ -440,24 +431,21 @@ impl fmt::Debug for ProviderResponseMetadata {
}
}
/// Provider 已筛选、可由协议 adapter 再次按白名单表达的响应头。
/// Provider 已筛选、可由协议 adapter 尝试表达的不透明响应头。
#[derive(Clone, PartialEq, Eq)]
pub struct ProviderResponseHeader {
name: String,
value: SafeUpstreamValue,
value: Bytes,
}
impl ProviderResponseHeader {
/// 创建规范化的小写响应头;非 ASCII token 名称直接拒绝。
/// 保存 Provider 交付的原始名称和值;HTTP 可表达性由最终 adapter 判断。
#[must_use]
pub fn new(name: impl Into<String>, value: SafeUpstreamValue) -> Option<Self> {
let name = name.into().trim().to_ascii_lowercase();
let valid = !name.is_empty()
&& name.len() <= 64
&& name
.bytes()
.all(|byte| byte.is_ascii_lowercase() || byte.is_ascii_digit() || byte == b'-');
valid.then_some(Self { name, value })
pub fn new(name: impl Into<String>, value: Bytes) -> Self {
Self {
name: name.into(),
value,
}
}
#[must_use]
@@ -466,7 +454,7 @@ impl ProviderResponseHeader {
}
#[must_use]
pub const fn value(&self) -> &SafeUpstreamValue {
pub const fn value(&self) -> &Bytes {
&self.value
}
}
@@ -475,7 +463,7 @@ impl fmt::Debug for ProviderResponseHeader {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("ProviderResponseHeader")
.field("name", &self.name)
.field("name", &"<redacted>")
.field("value", &"<redacted>")
.finish()
}
@@ -488,7 +476,8 @@ pub struct ProviderResponseObservation {
http_version: Option<UpstreamHttpVersion>,
websocket_pool: Option<WebSocketPoolKind>,
status_code: Option<u16>,
request_id: Option<SafeUpstreamValue>,
request_id: Option<OpaqueUpstreamValue>,
service_tier: Option<String>,
timings: ProviderResponseTimings,
client_headers: Vec<ProviderResponseHeader>,
provider_metadata: Option<ProviderResponseMetadata>,
@@ -503,6 +492,7 @@ impl ProviderResponseObservation {
websocket_pool: None,
status_code: None,
request_id: None,
service_tier: None,
timings: ProviderResponseTimings::default(),
client_headers: Vec::new(),
provider_metadata: None,
@@ -528,11 +518,36 @@ impl ProviderResponseObservation {
}
#[must_use]
pub fn with_request_id(mut self, request_id: SafeUpstreamValue) -> Self {
pub fn with_request_id(mut self, request_id: OpaqueUpstreamValue) -> Self {
self.request_id = Some(request_id);
self
}
/// 附加 Provider 在响应中确认的服务档位。
///
/// # Errors
///
/// 档位为空、超过 64 字节或包含控制字符时返回错误。
pub fn try_with_service_tier(
mut self,
service_tier: impl Into<String>,
) -> Result<Self, IdentifierError> {
let service_tier = service_tier.into();
validate_text(&service_tier, 64, false, None)?;
self.service_tier = Some(service_tier);
Ok(self)
}
/// 附加合法服务档位;不适合持久化的上游观测值会被忽略。
#[must_use]
pub fn with_service_tier_if_valid(mut self, service_tier: impl Into<String>) -> Self {
let service_tier = service_tier.into();
if validate_text(&service_tier, 64, false, None).is_ok() {
self.service_tier = Some(service_tier);
}
self
}
#[must_use]
pub const fn with_timings(mut self, timings: ProviderResponseTimings) -> Self {
self.timings = timings;
@@ -573,10 +588,15 @@ impl ProviderResponseObservation {
}
#[must_use]
pub const fn request_id(&self) -> Option<&SafeUpstreamValue> {
pub const fn request_id(&self) -> Option<&OpaqueUpstreamValue> {
self.request_id.as_ref()
}
#[must_use]
pub fn service_tier(&self) -> Option<&str> {
self.service_tier.as_deref()
}
#[must_use]
pub const fn timings(&self) -> ProviderResponseTimings {
self.timings
@@ -302,7 +302,7 @@ impl GenerateRequest {
self.body()
.get("previous_response_id")
.and_then(Value::as_str)
.is_some_and(|response_id| !response_id.is_empty())
.is_some()
}
/// 返回客户端是否请求了图片生成工具。
@@ -13,7 +13,7 @@ use crate::engine::credential::{
OpaqueProviderData, ProviderAccountId, ProviderAccountStore,
};
use crate::error::{IdentifierError, validate_text};
use crate::routing::ProviderKind;
use crate::routing::{ProviderKind, UpstreamModelId};
const MAX_PENDING_FLOW_TTL: Duration = Duration::from_secs(30 * 60);
@@ -527,6 +527,80 @@ impl ProviderCooldown {
}
}
/// 可丢失 cooldown 的细粒度作用域。
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum ProviderCooldownScope {
/// 只阻止同一账号调用指定上游模型。
UpstreamModel(UpstreamModelId),
}
impl ProviderCooldownScope {
#[must_use]
pub const fn upstream_model(model: UpstreamModelId) -> Self {
Self::UpstreamModel(model)
}
#[must_use]
pub const fn kind(&self) -> &'static str {
match self {
Self::UpstreamModel(_) => "model",
}
}
#[must_use]
pub fn value(&self) -> &str {
match self {
Self::UpstreamModel(model) => model.as_str(),
}
}
}
/// 不进入账号持久状态的账号+作用域 cooldown。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ProviderScopedCooldown {
account_id: ProviderAccountId,
credential_revision: CredentialRevision,
scope: ProviderCooldownScope,
until: SystemTime,
}
impl ProviderScopedCooldown {
#[must_use]
pub const fn new(
account_id: ProviderAccountId,
credential_revision: CredentialRevision,
scope: ProviderCooldownScope,
until: SystemTime,
) -> Self {
Self {
account_id,
credential_revision,
scope,
until,
}
}
#[must_use]
pub const fn account_id(&self) -> &ProviderAccountId {
&self.account_id
}
#[must_use]
pub const fn credential_revision(&self) -> CredentialRevision {
self.credential_revision
}
#[must_use]
pub const fn scope(&self) -> &ProviderCooldownScope {
&self.scope
}
#[must_use]
pub const fn until(&self) -> SystemTime {
self.until
}
}
pub trait ProviderCooldownPort: Send + Sync {
fn put_if_later(
&self,
@@ -543,6 +617,24 @@ pub trait ProviderCooldownPort: Send + Sync {
account_id: &'a ProviderAccountId,
through_revision: CredentialRevision,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>>;
fn put_scoped_if_later(
&self,
cooldown: ProviderScopedCooldown,
) -> BoxFuture<'_, Result<bool, ProviderStoreError>>;
fn read_scoped<'a>(
&'a self,
account_id: &'a ProviderAccountId,
scope: &'a ProviderCooldownScope,
) -> BoxFuture<'a, Result<Option<ProviderScopedCooldown>, ProviderStoreError>>;
fn clear_scoped<'a>(
&'a self,
account_id: &'a ProviderAccountId,
scope: &'a ProviderCooldownScope,
through_revision: CredentialRevision,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>>;
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
@@ -575,32 +667,21 @@ impl ProviderRefreshPolicy {
self.concurrency
}
/// 按账号稳定扰动刷新提前量;相同账号与过期时间跨进程得到同一执行时刻。
/// 按配置的精确提前量计算刷新时刻。
pub fn next_attempt_at(
self,
account_id: &ProviderAccountId,
_account_id: &ProviderAccountId,
access_token_expires_at: SystemTime,
observed_at: SystemTime,
) -> Result<SystemTime, ProviderStoreError> {
let remaining = access_token_expires_at
.duration_since(observed_at)
.map_err(|_| invalid_refresh_policy("schedule expired access token"))?;
let base_margin_seconds = self.margin.as_secs();
if base_margin_seconds == 0 {
if self.margin >= remaining {
return Ok(observed_at);
}
let factor = stable_factor(account_id.as_str(), "normal", 850, 1_150);
let jittered_seconds = base_margin_seconds
.saturating_mul(u64::from(factor))
.saturating_add(500)
/ 1_000;
if jittered_seconds >= remaining.as_secs() {
return Ok(observed_at);
}
let lead = jittered_seconds.max(1);
access_token_expires_at
.checked_sub(Duration::from_secs(lead))
.checked_sub(self.margin)
.ok_or_else(|| invalid_refresh_policy("schedule access token refresh"))
}
}
@@ -778,35 +859,75 @@ pub enum OAuthPendingPutOutcome {
AlreadyExists,
}
/// OAuth 回调处理取得临时 flow 的独占权结果。
///
/// 与一次性消费不同,Provider 在上游换取令牌失败时可以释放 claim,让同一 flow
/// 使用新的回调地址重试;只有完整校验成功后才会消费 flow。
#[derive(Clone, PartialEq)]
pub enum OAuthPendingTakeOutcome {
Taken(OpaqueProviderData),
pub enum OAuthPendingClaimOutcome {
Claimed(OpaqueProviderData),
NotFound,
OwnerMismatch,
InProgress,
}
impl fmt::Debug for OAuthPendingTakeOutcome {
impl fmt::Debug for OAuthPendingClaimOutcome {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Taken(_) => formatter.write_str("Taken([PROVIDER-OWNED])"),
Self::Claimed(_) => formatter.write_str("Claimed([PROVIDER-OWNED])"),
Self::NotFound => formatter.write_str("NotFound"),
Self::OwnerMismatch => formatter.write_str("OwnerMismatch"),
Self::InProgress => formatter.write_str("InProgress"),
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum OAuthPendingReleaseOutcome {
Released,
NotFound,
OwnerMismatch,
ClaimMismatch,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum OAuthPendingConsumeOutcome {
Consumed,
NotFound,
OwnerMismatch,
ClaimMismatch,
}
pub trait OAuthPendingFlowPort: Send + Sync {
fn put_if_absent(
&self,
flow: NewOAuthPendingFlow,
) -> BoxFuture<'_, Result<OAuthPendingPutOutcome, ProviderStoreError>>;
fn take_if_owner<'a>(
fn claim_if_owner<'a>(
&'a self,
provider_kind: &'a ProviderKind,
flow: &'a OAuthPendingBinding,
owner: &'a OAuthPendingBinding,
) -> BoxFuture<'a, Result<OAuthPendingTakeOutcome, ProviderStoreError>>;
claim: &'a OAuthPendingBinding,
claim_ttl: Duration,
) -> BoxFuture<'a, Result<OAuthPendingClaimOutcome, ProviderStoreError>>;
fn release_claim<'a>(
&'a self,
provider_kind: &'a ProviderKind,
flow: &'a OAuthPendingBinding,
owner: &'a OAuthPendingBinding,
claim: &'a OAuthPendingBinding,
) -> BoxFuture<'a, Result<OAuthPendingReleaseOutcome, ProviderStoreError>>;
fn consume_claim<'a>(
&'a self,
provider_kind: &'a ProviderKind,
flow: &'a OAuthPendingBinding,
owner: &'a OAuthPendingBinding,
claim: &'a OAuthPendingBinding,
) -> BoxFuture<'a, Result<OAuthPendingConsumeOutcome, ProviderStoreError>>;
}
/// Provider 只能按能力取用端口,无法取得 Redis client 或 repository 集合。
@@ -89,6 +89,18 @@ impl UpstreamModelId {
Ok(Self(value))
}
/// 从未命中显式映射的客户端模型名构造同名上游模型。
///
/// 目录和配置仍使用 [`Self::new`] 的内部标识约束;客户端 wire 只要求非空,
/// 具体模型字符串是否可用由绑定 Provider 决定。
pub fn from_client_wire(value: impl Into<String>) -> Result<Self, IdentifierError> {
let value = value.into();
if value.is_empty() {
return Err(IdentifierError::Empty);
}
Ok(Self(value))
}
#[must_use]
pub fn as_str(&self) -> &str {
&self.0
@@ -152,9 +164,61 @@ pub struct ModelPresentation {
search_tool: bool,
image_detail_original: bool,
verbosity: bool,
service_tiers: Vec<ModelServiceTier>,
hidden: bool,
}
/// Provider 声明给 Codex 客户端的服务档位。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ModelServiceTier {
id: String,
name: String,
description: String,
speed_tier: Option<String>,
}
impl ModelServiceTier {
#[must_use]
pub fn new(
id: impl Into<String>,
name: impl Into<String>,
description: impl Into<String>,
) -> Self {
Self {
id: id.into(),
name: name.into(),
description: description.into(),
speed_tier: None,
}
}
#[must_use]
pub fn with_speed_tier(mut self, speed_tier: impl Into<String>) -> Self {
self.speed_tier = Some(speed_tier.into());
self
}
#[must_use]
pub fn id(&self) -> &str {
&self.id
}
#[must_use]
pub fn name(&self) -> &str {
&self.name
}
#[must_use]
pub fn description(&self) -> &str {
&self.description
}
#[must_use]
pub fn speed_tier(&self) -> Option<&str> {
self.speed_tier.as_deref()
}
}
impl ModelPresentation {
#[must_use]
pub fn new(display_name: Option<String>, description: Option<String>) -> Self {
@@ -213,6 +277,12 @@ impl ModelPresentation {
self
}
#[must_use]
pub fn with_service_tiers(mut self, service_tiers: Vec<ModelServiceTier>) -> Self {
self.service_tiers = service_tiers;
self
}
#[must_use]
pub const fn with_hidden(mut self, hidden: bool) -> Self {
self.hidden = hidden;
@@ -274,6 +344,11 @@ impl ModelPresentation {
self.verbosity
}
#[must_use]
pub fn service_tiers(&self) -> &[ModelServiceTier] {
&self.service_tiers
}
#[must_use]
pub const fn hidden(&self) -> bool {
self.hidden
@@ -488,15 +488,11 @@ impl RuntimeSnapshot {
#[must_use]
pub fn mapped_model(&self, requested: &str) -> String {
let original = requested.trim();
let original = requested;
let mut current = original.to_owned();
let mut seen = BTreeSet::new();
for _ in 0..20 {
let Some(target) = self
.model_mappings
.get(&current)
.map(|target| target.trim())
else {
let Some(target) = self.model_mappings.get(&current).map(String::as_str) else {
return current;
};
if !seen.insert(current.clone()) || seen.contains(target) {
@@ -529,8 +525,14 @@ impl RuntimeSnapshot {
{
continue;
}
let upstream_model = UpstreamModelId::new(self.mapped_model(public_model.as_str()))
.map_err(|_| RoutingError::InvalidIdentifier)?;
let requested_model = public_model.as_str();
let mapped_model = self.mapped_model(requested_model);
let upstream_model = if self.model_mappings.contains_key(requested_model) {
UpstreamModelId::new(mapped_model)
} else {
UpstreamModelId::from_client_wire(mapped_model)
}
.map_err(|_| RoutingError::InvalidIdentifier)?;
let emulated_features = match self
.provider_models
.get(provider)
@@ -2,13 +2,12 @@ use gateway_core::engine::continuation::{
ContinuationBinding, NativeContinuationPin, PreviousResponseId,
};
use gateway_core::engine::credential::ProviderAccountId;
use gateway_core::error::SafeUpstreamValue;
use gateway_core::routing::ProviderKind;
fn pin() -> NativeContinuationPin {
NativeContinuationPin::new(
PreviousResponseId::new("response-private").expect("valid response"),
SafeUpstreamValue::new("upstream-private").expect("valid upstream response"),
PreviousResponseId::new("response-private"),
PreviousResponseId::new("upstream-private"),
ProviderKind::new("openai").expect("valid provider"),
ProviderAccountId::new("acct_codex").expect("valid account"),
)
@@ -23,13 +22,20 @@ fn native_pin_debug_should_redact_previous_response_id() {
#[test]
fn external_binding_debug_should_redact_previous_response_id() {
let binding = ContinuationBinding::External(
PreviousResponseId::new("external-private").expect("valid response"),
);
let binding = ContinuationBinding::External(PreviousResponseId::new("external-private"));
assert!(!format!("{binding:?}").contains("external-private"));
}
#[test]
fn opaque_response_id_preserves_any_codex_string_without_debug_disclosure() {
let value = format!("resp_{}\0opaque", "x".repeat(257));
let response_id = PreviousResponseId::new(value.clone());
assert_eq!(response_id.as_str(), value);
assert!(!format!("{response_id:?}").contains(&value));
}
#[test]
fn native_pin_should_reject_different_account() {
assert!(!pin().matches(
@@ -6,6 +6,7 @@ use std::sync::{Arc, Mutex};
use std::time::{Duration, SystemTime};
use async_trait::async_trait;
use bytes::Bytes;
use futures::StreamExt;
use futures::executor::block_on;
@@ -28,8 +29,8 @@ use gateway_core::engine::{
ProviderAttemptOutcome, RecoveryReport, UpstreamSendState,
};
use gateway_core::error::{
ContinuationFailure, GatewayErrorKind, ProviderError, ProviderErrorKind, SafeUpstreamValue,
StoreError,
ClientVisibleUpstreamResponse, ContinuationFailure, GatewayErrorKind, OpaqueUpstreamValue,
ProviderError, ProviderErrorKind, StoreError, StoreErrorKind,
};
use gateway_core::event::{
ContentItem, ContentKind, GatewayEvent, ProtocolWireEvent, ProviderEvent,
@@ -65,6 +66,7 @@ struct FinalState {
upstream_transport: Option<String>,
http_version: Option<String>,
websocket_pool: Option<String>,
service_tier: Option<String>,
upstream_request_id: Option<String>,
upstream_status_code: Option<u16>,
transport_decision_wait_ms: Option<u64>,
@@ -97,11 +99,37 @@ struct StoreState {
#[derive(Default)]
struct FakeStore {
state: Mutex<StoreState>,
failures: BTreeSet<StoreWriteFailure>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
enum StoreWriteFailure {
Create,
Commit,
Finalize,
}
impl FakeStore {
fn failing(failure: StoreWriteFailure) -> Self {
Self {
state: Mutex::new(StoreState::default()),
failures: BTreeSet::from([failure]),
}
}
fn fails(&self, failure: StoreWriteFailure) -> Result<(), StoreError> {
if self.failures.contains(&failure) {
Err(StoreError::new(StoreErrorKind::Unavailable))
} else {
Ok(())
}
}
}
#[async_trait]
impl ExecutionStore for FakeStore {
async fn create_model_request(&self, _request: NewModelRequest) -> Result<(), StoreError> {
self.fails(StoreWriteFailure::Create)?;
self.state.lock().expect("store lock").created += 1;
Ok(())
}
@@ -134,6 +162,7 @@ impl ExecutionStore for FakeStore {
_committed_at: SystemTime,
client_status_code: Option<u16>,
) -> Result<(), StoreError> {
self.fails(StoreWriteFailure::Commit)?;
let mut state = self.state.lock().expect("store lock");
state.commits += 1;
state.committed_statuses.push(client_status_code);
@@ -172,6 +201,7 @@ impl ExecutionStore for FakeStore {
&self,
finalization: ModelRequestFinalization,
) -> Result<(), StoreError> {
self.fails(StoreWriteFailure::Finalize)?;
self.state
.lock()
.expect("store lock")
@@ -194,6 +224,7 @@ impl ExecutionStore for FakeStore {
upstream_transport: finalization.upstream_transport,
http_version: finalization.http_version,
websocket_pool: finalization.websocket_pool,
service_tier: finalization.service_tier,
upstream_request_id: finalization.upstream_request_id,
upstream_status_code: finalization.upstream_status_code,
transport_decision_wait_ms: finalization.timings.transport_decision_wait_ms,
@@ -291,15 +322,11 @@ impl Provider for ScriptedProvider {
},
UpstreamTransport::new("http_sse").expect("transport"),
)
.with_upstream_request_id(
SafeUpstreamValue::new("upstream-request").expect("safe request id"),
);
.with_upstream_request_id(OpaqueUpstreamValue::new("upstream-request"));
Ok(ProviderStream::new(
metadata,
Box::pin(futures::stream::iter(
items
.into_iter()
.map(|event| event.map(ProviderEvent::from)),
items.into_iter().map(canonical_provider_event),
)),
(),
))
@@ -318,12 +345,8 @@ impl Provider for ScriptedProvider {
Ok(ProviderStream::new(
metadata,
Box::pin(
futures::stream::iter(
items
.into_iter()
.map(|event| event.map(ProviderEvent::from)),
)
.chain(futures::stream::pending()),
futures::stream::iter(items.into_iter().map(canonical_provider_event))
.chain(futures::stream::pending()),
),
(),
))
@@ -399,6 +422,75 @@ fn complete_stream(total_tokens: Option<u64>) -> Vec<Result<GatewayEvent, Provid
events
}
fn canonical_provider_event(
event: Result<GatewayEvent, ProviderError>,
) -> Result<ProviderEvent, ProviderError> {
event.map(ProviderEvent::canonical)
}
fn atomic_response_failed(response_id: &str, marker: &str) -> ProviderError {
let started = ProviderEvent::canonical_with_wire(
vec![GatewayEvent::Started(ResponseMeta::new(
response_id,
"gpt-5",
))],
ProtocolWireEvent::json(
"openai",
Some("response.created".to_owned()),
json!({
"type": "response.created",
"response": {"id": response_id, "model": "gpt-5", "status": "in_progress"}
}),
)
.expect("started wire"),
);
let failed = ProviderEvent::wire(
ProtocolWireEvent::json(
"openai",
Some("response.failed".to_owned()),
json!({
"type": "response.failed",
"response": {
"id": response_id,
"status": "failed",
"error": {"code": "rate_limit_exceeded", "message": marker}
}
}),
)
.expect("failed wire"),
);
ProviderError::new(ProviderErrorKind::RateLimited, UpstreamSendState::Sent)
.with_status(429)
.with_replay_safe()
.with_atomic_client_events(vec![started, failed])
}
fn bare_atomic_response_failed(response_id: &str) -> ProviderError {
let failed = ProviderEvent::canonical_with_wire(
vec![GatewayEvent::Started(ResponseMeta::new(
response_id,
"fallback-model",
))],
ProtocolWireEvent::json(
"openai",
Some("response.failed".to_owned()),
json!({
"type": "response.failed",
"response": {
"id": response_id,
"status": "failed",
"error": {"code": "rate_limit_exceeded", "message": "bare failure"}
}
}),
)
.expect("bare failed wire"),
);
ProviderError::new(ProviderErrorKind::RateLimited, UpstreamSendState::Sent)
.with_status(429)
.with_replay_safe()
.with_atomic_client_events(vec![failed])
}
fn image_stream(image_output_tokens: Option<u64>) -> Vec<Result<GatewayEvent, ProviderError>> {
let mut usage = Usage::new();
usage.input_tokens = Some(12);
@@ -484,6 +576,17 @@ fn coordinator(
Arc<ScriptedProvider>,
) {
let store = Arc::new(FakeStore::default());
coordinator_with_store(scripts, store)
}
fn coordinator_with_store(
scripts: Vec<Script>,
store: Arc<FakeStore>,
) -> (
AttemptCoordinator<FakeStore>,
Arc<FakeStore>,
Arc<ScriptedProvider>,
) {
let provider = Arc::new(ScriptedProvider::new(scripts));
let mut registry = ProviderRegistry::builder();
registry
@@ -587,6 +690,170 @@ fn success_updates_one_model_request_and_persists_usage() {
);
}
#[test]
fn observation_store_failures_never_block_successful_delivery() {
for failure in [
StoreWriteFailure::Create,
StoreWriteFailure::Commit,
StoreWriteFailure::Finalize,
] {
let operation = operation(RetrySafety::NonIdempotent);
let route_plan = plan(&operation);
let store = Arc::new(FakeStore::failing(failure));
let (coordinator, _, provider) = coordinator_with_store(
vec![Script::Stream {
account_id: "acct_one",
items: complete_stream(None),
}],
store,
);
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
None,
None,
CancellationToken::new(),
))
.expect("start execution");
let events = block_on(session.collect_uncommitted())
.unwrap_or_else(|error| panic!("{failure:?} blocked collection: {error}"));
assert_eq!(events.len(), 2);
block_on(session.commit_downstream(Some(200)))
.unwrap_or_else(|error| panic!("{failure:?} blocked commit: {error}"));
assert!(session.is_finalized());
assert_eq!(provider.contexts.lock().expect("contexts lock").len(), 1);
}
}
#[test]
fn terminal_provider_error_preserves_request_local_upstream_response() {
let operation = generate_operation();
let route_plan = plan(&operation);
let raw_body = Bytes::from_static(b"{\"upstream\":\"unchanged\"}\x00");
let provider_error =
ProviderError::new(ProviderErrorKind::InvalidRequest, UpstreamSendState::Sent)
.with_client_visible_upstream_response(ClientVisibleUpstreamResponse::new(
422,
Some(b"application/problem+json".to_vec()),
raw_body.clone(),
));
let (coordinator, _, _) = coordinator(vec![Script::Error(provider_error)]);
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
None,
None,
CancellationToken::new(),
))
.expect("start execution");
let error = block_on(session.collect_uncommitted()).expect_err("provider rejection");
let EngineError::Provider(error) = error else {
panic!("expected provider error");
};
let response = error
.client_visible_upstream_response()
.expect("request-local upstream response");
assert_eq!(response.status(), 422);
assert_eq!(response.body(), &raw_body);
}
#[test]
fn success_preserves_opaque_upstream_response_id_and_builds_native_pin() {
let response_id = format!("resp_{}\0opaque", "x".repeat(4_096));
let operation = generate_operation();
let route_plan = plan(&operation);
let (coordinator, store, _) = coordinator(vec![Script::Stream {
account_id: "acct_one",
items: vec![
Ok(GatewayEvent::Started(ResponseMeta::new(
response_id.clone(),
"gpt-5",
))),
Ok(GatewayEvent::Completed(ResponseMeta::new(
response_id.clone(),
"gpt-5",
))),
],
}]);
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
None,
None,
CancellationToken::new(),
))
.expect("start execution");
block_on(session.collect_uncommitted()).expect("collect opaque response");
let provider_state = ProviderSessionState::new("openai", Map::new()).expect("provider state");
let pin = session
.native_continuation_pin(&provider_state)
.expect("native continuation pin");
assert_eq!(pin.previous_response_id().as_str(), response_id);
assert_eq!(pin.upstream_response_id().as_str(), response_id);
block_on(session.commit_downstream(Some(200))).expect("commit response");
let state = store.state.lock().expect("store lock");
assert_eq!(
state.finalizations[0].client_response_id,
Some(response_id.clone())
);
assert_eq!(
state.finalizations[0].upstream_response_id,
Some(response_id)
);
}
#[test]
fn canonical_identity_change_and_empty_terminal_id_are_observational() {
let operation = generate_operation();
let route_plan = plan(&operation);
let (coordinator, store, _) = coordinator(vec![Script::Stream {
account_id: "acct_one",
items: vec![
Ok(GatewayEvent::Started(ResponseMeta::new(
"response-created",
"gpt-5",
))),
Ok(GatewayEvent::Completed(ResponseMeta::new("", "gpt-5"))),
],
}]);
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
None,
None,
CancellationToken::new(),
))
.expect("start execution");
block_on(session.collect_uncommitted()).expect("identity observation cannot fail delivery");
let provider_state = ProviderSessionState::new("openai", Map::new()).expect("provider state");
let pin = session
.native_continuation_pin(&provider_state)
.expect("empty opaque response ID remains recordable");
assert_eq!(pin.previous_response_id().as_str(), "");
assert_eq!(pin.upstream_response_id().as_str(), "");
block_on(session.commit_downstream(Some(200))).expect("commit response");
let state = store.state.lock().expect("store lock");
assert_eq!(
state.finalizations[0].client_response_id.as_deref(),
Some("")
);
assert_eq!(
state.finalizations[0].upstream_response_id.as_deref(),
Some("")
);
}
fn finalized_image_request(image_output_tokens: Option<u64>) -> FinalState {
let operation = image_generate_operation();
let route_plan = plan(&operation);
@@ -669,7 +936,9 @@ fn response_observation_is_persisted_but_never_delivered() {
.with_http_version(UpstreamHttpVersion::Http2)
.with_websocket_pool(WebSocketPoolKind::New)
.with_status_code(200)
.with_request_id(SafeUpstreamValue::new("upstream-observed").expect("request id"))
.with_request_id(OpaqueUpstreamValue::new("upstream-observed"))
.try_with_service_tier("priority")
.expect("service tier")
.with_timings(ProviderResponseTimings {
transport_decision_wait_ms: Some(7),
connect_ms: Some(11),
@@ -681,7 +950,7 @@ fn response_observation_is_persisted_but_never_delivered() {
items.extend(
complete_stream(Some(21))
.into_iter()
.map(|event| event.map(ProviderEvent::canonical)),
.map(canonical_provider_event),
);
let (coordinator, store, _) = coordinator(vec![Script::ObservedStream {
account_id: "acct_observed",
@@ -706,6 +975,7 @@ fn response_observation_is_persisted_but_never_delivered() {
assert_eq!(finalization.upstream_transport.as_deref(), Some("http_sse"));
assert_eq!(finalization.http_version.as_deref(), Some("HTTP/2"));
assert_eq!(finalization.websocket_pool.as_deref(), Some("new"));
assert_eq!(finalization.service_tier.as_deref(), Some("priority"));
assert_eq!(
finalization.upstream_request_id.as_deref(),
Some("upstream-observed")
@@ -717,6 +987,48 @@ fn response_observation_is_persisted_but_never_delivered() {
assert_eq!(finalization.first_event_ms, Some(17));
}
#[test]
fn mismatched_response_observation_should_not_interrupt_client_events() {
let operation = operation(RetrySafety::Idempotent);
let route_plan = plan(&operation);
let first = ProviderEvent::observation(ProviderResponseObservation::new(
UpstreamTransport::new("websocket").expect("first transport"),
));
let mismatched = ProviderEvent::observation(ProviderResponseObservation::new(
UpstreamTransport::new("http_sse").expect("mismatched transport"),
));
let mut items = vec![Ok(first), Ok(mismatched)];
items.extend(
complete_stream(None)
.into_iter()
.map(canonical_provider_event),
);
let (coordinator, store, _) = coordinator(vec![Script::ObservedStream {
account_id: "acct_observation_mismatch",
items,
}]);
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
None,
None,
CancellationToken::new(),
))
.expect("start execution");
let events = block_on(session.collect_uncommitted()).expect("client events survive");
block_on(session.commit_downstream(Some(200))).expect("commit response");
assert_eq!(events.len(), 2);
assert_eq!(
store.state.lock().expect("store lock").finalizations[0]
.upstream_transport
.as_deref(),
Some("websocket")
);
}
#[test]
fn response_observation_should_persist_provider_metadata_and_output_timings() {
let operation = operation(RetrySafety::Idempotent);
@@ -741,7 +1053,7 @@ fn response_observation_should_persist_provider_metadata_and_output_timings() {
items.extend(
complete_stream(Some(21))
.into_iter()
.map(|event| event.map(ProviderEvent::canonical)),
.map(canonical_provider_event),
);
let (coordinator, store, _) = coordinator(vec![Script::ObservedStream {
account_id: "acct_observed",
@@ -881,7 +1193,7 @@ fn discarded_attempt_observation_does_not_leak_into_retry_result() {
)
.with_http_version(UpstreamHttpVersion::Http11)
.with_status_code(503)
.with_request_id(SafeUpstreamValue::new("discarded-request").expect("request id"))
.with_request_id(OpaqueUpstreamValue::new("discarded-request"))
.with_timings(ProviderResponseTimings {
transport_decision_wait_ms: Some(7),
connect_ms: Some(11),
@@ -895,12 +1207,12 @@ fn discarded_attempt_observation_does_not_leak_into_retry_result() {
)
.with_http_version(UpstreamHttpVersion::Http2)
.with_status_code(200)
.with_request_id(SafeUpstreamValue::new("winning-request").expect("request id"));
.with_request_id(OpaqueUpstreamValue::new("winning-request"));
let mut second_items = vec![Ok(ProviderEvent::observation(second_observation))];
second_items.extend(
complete_stream(None)
.into_iter()
.map(|event| event.map(ProviderEvent::canonical)),
.map(canonical_provider_event),
);
let (coordinator, store, _) = coordinator(vec![
Script::ObservedStream {
@@ -1029,8 +1341,8 @@ fn authenticated_native_continuation_reaches_every_attempt_context() {
items: complete_stream(None),
}]);
let continuation = NativeContinuationPin::new(
PreviousResponseId::new("previous-secret-id").expect("previous response"),
SafeUpstreamValue::new("provider-native-id").expect("upstream response"),
PreviousResponseId::new("previous-secret-id"),
PreviousResponseId::new("provider-native-id"),
ProviderKind::new("openai").expect("provider"),
ProviderAccountId::new("acct_one").expect("account"),
);
@@ -1086,8 +1398,8 @@ fn streaming_native_continuation_delivers_started_event_before_later_output() {
items: complete_stream(None),
}]);
let continuation = NativeContinuationPin::new(
PreviousResponseId::new("previous-response").expect("previous response"),
SafeUpstreamValue::new("upstream-response").expect("upstream response"),
PreviousResponseId::new("previous-response"),
PreviousResponseId::new("upstream-response"),
ProviderKind::new("openai").expect("provider"),
ProviderAccountId::new("acct_one").expect("account"),
);
@@ -1162,8 +1474,8 @@ fn native_continuation_replays_owner_before_safely_switching_account() {
},
]);
let continuation = NativeContinuationPin::new(
PreviousResponseId::new("previous-secret-id").expect("previous response"),
SafeUpstreamValue::new("provider-native-id").expect("upstream response"),
PreviousResponseId::new("previous-secret-id"),
PreviousResponseId::new("provider-native-id"),
ProviderKind::new("openai").expect("provider"),
ProviderAccountId::new("acct_one").expect("account"),
);
@@ -1202,7 +1514,7 @@ fn native_continuation_replays_owner_before_safely_switching_account() {
}
#[test]
fn required_account_reaches_provider_and_matching_metadata_succeeds() {
fn diagnostic_account_reaches_provider_and_matching_metadata_succeeds() {
let operation = operation(RetrySafety::Idempotent);
let route_plan = plan(&operation);
let (coordinator, _, provider) = coordinator(vec![Script::Stream {
@@ -1210,11 +1522,11 @@ fn required_account_reaches_provider_and_matching_metadata_succeeds() {
items: complete_stream(None),
}]);
let required = ProviderAccountId::new("acct_required").expect("account id");
let mut session = block_on(coordinator.start(
let mut session = block_on(coordinator.start_diagnostic(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
Some(required.clone()),
required.clone(),
None,
CancellationToken::new(),
))
@@ -1225,6 +1537,7 @@ fn required_account_reaches_provider_and_matching_metadata_succeeds() {
let contexts = provider.contexts.lock().expect("contexts lock");
assert_eq!(contexts.len(), 1);
assert_eq!(contexts[0].required_account(), Some(&required));
assert!(contexts[0].is_diagnostic_required_account());
}
#[test]
@@ -1709,9 +2022,7 @@ fn rate_limited_account_exhaustion_survives_a_later_empty_selection() {
UpstreamSendState::Sent,
)
.with_status(429)
.with_upstream_code(
SafeUpstreamValue::new("rate_limit_exceeded").expect("provider code"),
)
.with_upstream_code(OpaqueUpstreamValue::new("rate_limit_exceeded"))
.with_retry_after(Duration::from_secs(30))
.with_replay_safe())],
},
@@ -1739,7 +2050,7 @@ fn rate_limited_account_exhaustion_survives_a_later_empty_selection() {
assert_eq!(error.kind(), ProviderErrorKind::RateLimited);
assert_eq!(error.upstream_status(), Some(429));
assert_eq!(
error.upstream_code().map(SafeUpstreamValue::as_str),
error.upstream_code().map(OpaqueUpstreamValue::as_str),
Some("rate_limit_exceeded")
);
assert_eq!(error.retry_after(), Some(Duration::from_secs(30)));
@@ -1836,11 +2147,282 @@ fn explicit_429_after_structural_event_should_retry_before_commit() {
assert_eq!(state.finalizations[0].outcome, ExecutionOutcome::Succeeded);
}
#[test]
fn atomic_response_failed_should_rotate_account_before_stream_commit() {
let operation = generate_operation();
let route_plan = plan(&operation);
let (coordinator, store, provider) = coordinator(vec![
Script::ObservedStream {
account_id: "acct_first",
items: vec![Err(atomic_response_failed(
"response-first-failed",
"first account failure",
))],
},
Script::Stream {
account_id: "acct_second",
items: complete_stream(None),
},
]);
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
None,
None,
CancellationToken::new(),
))
.expect("start execution");
let first = block_on(session.next_event())
.expect("second account should produce the first delivery")
.expect("started event");
let delivered = first.into_provider_events();
assert!(delivered.iter().flat_map(ProviderEvent::canonical_facts).any(
|event| matches!(event, GatewayEvent::Started(meta) if meta.response_id() == "response-1")
));
assert!(
!delivered
.iter()
.filter_map(ProviderEvent::wire_event)
.any(|wire| {
wire.data().pointer("/response/id").and_then(Value::as_str)
== Some("response-first-failed")
})
);
block_on(session.commit_downstream(Some(200))).expect("commit second account");
while block_on(session.next_event())
.expect("successful stream")
.is_some()
{}
assert_eq!(provider.contexts.lock().expect("contexts lock").len(), 2);
let state = store.state.lock().expect("store lock");
assert_eq!(state.commits, 1);
assert_eq!(state.intermediate_failures, 1);
assert_eq!(state.finalizations[0].outcome, ExecutionOutcome::Succeeded);
}
#[test]
fn bare_atomic_response_failed_should_rotate_before_the_first_delivery() {
let operation = generate_operation();
let route_plan = plan(&operation);
let (coordinator, store, provider) = coordinator(vec![
Script::ObservedStream {
account_id: "acct_first",
items: vec![Err(bare_atomic_response_failed("response-bare-failed"))],
},
Script::Stream {
account_id: "acct_second",
items: complete_stream(None),
},
]);
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
None,
None,
CancellationToken::new(),
))
.expect("start execution");
let first = block_on(session.next_event())
.expect("second account should replace the bare failure")
.expect("second account started event");
assert!(first
.into_provider_events()
.iter()
.flat_map(ProviderEvent::canonical_facts)
.any(|event| matches!(event, GatewayEvent::Started(meta) if meta.response_id() == "response-1")));
block_on(session.commit_downstream(Some(200))).expect("commit second account");
while block_on(session.next_event())
.expect("successful stream")
.is_some()
{}
assert_eq!(provider.contexts.lock().expect("contexts lock").len(), 2);
let state = store.state.lock().expect("store lock");
assert_eq!(state.intermediate_failures, 1);
assert_eq!(state.finalizations[0].outcome, ExecutionOutcome::Succeeded);
}
#[test]
fn exhausted_atomic_response_failed_should_deliver_only_the_last_failure_once() {
let operation = generate_operation();
let route_plan = plan(&operation);
let scripts = (1..=route_plan.max_attempts().get())
.map(|attempt| {
let response_id = format!("response-attempt-{attempt}-failed");
let marker = if attempt == route_plan.max_attempts().get() {
"last upstream failure".to_owned()
} else {
format!("discarded failure {attempt}")
};
Script::ObservedStream {
account_id: "acct_failed",
items: vec![Err(atomic_response_failed(&response_id, &marker))],
}
})
.collect::<Vec<_>>();
let expected_attempts = route_plan.max_attempts().get();
let expected_last_response = format!("response-attempt-{expected_attempts}-failed");
let (coordinator, store, provider) = coordinator(scripts);
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
None,
None,
CancellationToken::new(),
))
.expect("start execution");
let terminal = block_on(session.next_event())
.expect("last failure remains deliverable")
.expect("terminal failure batch");
assert_eq!(
terminal.commit_requirement(),
CommitRequirement::CommitBeforeDelivery
);
let events = terminal.into_provider_events();
let failures = events
.iter()
.filter_map(ProviderEvent::wire_event)
.filter(|wire| wire.event_type() == Some("response.failed"))
.collect::<Vec<_>>();
assert_eq!(failures.len(), 1);
assert_eq!(
failures[0]
.data()
.pointer("/response/id")
.and_then(Value::as_str),
Some(expected_last_response.as_str())
);
assert!(
failures[0]
.data()
.to_string()
.contains("last upstream failure")
);
block_on(session.commit_downstream(Some(200))).expect("commit original terminal failure");
let error = block_on(session.next_event()).expect_err("typed failure finalizes the request");
assert!(matches!(error, EngineError::Provider(_)));
assert!(session.is_finalized());
assert_eq!(
provider.contexts.lock().expect("contexts lock").len(),
usize::try_from(expected_attempts).expect("attempt count fits usize")
);
let state = store.state.lock().expect("store lock");
assert_eq!(
state.intermediate_failures,
usize::try_from(expected_attempts - 1).expect("attempt count fits usize")
);
assert_eq!(state.commits, 1);
assert_eq!(state.finalizations[0].outcome, ExecutionOutcome::Incomplete);
assert!(state.finalizations[0].committed);
}
#[test]
fn empty_selection_after_atomic_failure_should_deliver_the_last_upstream_wire() {
let operation = generate_operation();
let route_plan = plan(&operation);
let (coordinator, store, provider) = coordinator(vec![
Script::ObservedStream {
account_id: "acct_only",
items: vec![Err(bare_atomic_response_failed("response-only-failed"))],
},
Script::Error(ProviderError::new(
ProviderErrorKind::NoEligibleAccount,
UpstreamSendState::NotSent,
)),
]);
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
None,
None,
CancellationToken::new(),
))
.expect("start execution");
let terminal = block_on(session.next_event())
.expect("last account failure remains deliverable")
.expect("terminal failure batch");
let events = terminal.into_provider_events();
let failure = events
.iter()
.filter_map(ProviderEvent::wire_event)
.find(|wire| wire.event_type() == Some("response.failed"))
.expect("last upstream response.failed");
assert_eq!(
failure
.data()
.pointer("/response/id")
.and_then(Value::as_str),
Some("response-only-failed")
);
block_on(session.commit_downstream(Some(200))).expect("commit terminal failure");
assert!(matches!(
block_on(session.next_event()),
Err(EngineError::Provider(_))
));
assert_eq!(provider.contexts.lock().expect("contexts lock").len(), 2);
let state = store.state.lock().expect("store lock");
assert_eq!(state.attempts.len(), 1);
assert_eq!(state.intermediate_failures, 1);
assert_eq!(state.finalizations[0].outcome, ExecutionOutcome::Incomplete);
assert_eq!(
state.finalizations[0].client_response_id.as_deref(),
Some("response-only-failed")
);
assert_eq!(
state.finalizations[0].upstream_response_id.as_deref(),
Some("response-only-failed")
);
}
#[test]
fn atomic_response_failed_keeps_collect_uncommitted_error_semantics() {
let operation = generate_operation();
let route_plan = plan(&operation);
let (coordinator, store, provider) = coordinator(vec![Script::ObservedStream {
account_id: "acct_required",
items: vec![Err(atomic_response_failed(
"response-required-failed",
"required account failure",
))],
}]);
let required = ProviderAccountId::new("acct_required").expect("required account");
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
Some(required),
None,
CancellationToken::new(),
))
.expect("start execution");
let error = block_on(session.collect_uncommitted())
.expect_err("non-streaming collection must keep returning the typed failure");
assert!(matches!(error, EngineError::Provider(_)));
assert!(session.is_finalized());
assert_eq!(provider.contexts.lock().expect("contexts lock").len(), 1);
let state = store.state.lock().expect("store lock");
assert_eq!(state.commits, 0);
assert_eq!(state.intermediate_failures, 0);
assert_eq!(state.finalizations[0].outcome, ExecutionOutcome::Failed);
}
#[test]
fn native_continuation_explicit_429_is_not_retried() {
let continuation = NativeContinuationPin::new(
PreviousResponseId::new("previous-secret-id").expect("previous response"),
SafeUpstreamValue::new("provider-native-id").expect("upstream response"),
PreviousResponseId::new("previous-secret-id"),
PreviousResponseId::new("provider-native-id"),
ProviderKind::new("openai").expect("provider"),
ProviderAccountId::new("acct_first").expect("account"),
);
@@ -1861,9 +2443,8 @@ fn native_continuation_explicit_429_is_not_retried() {
#[test]
fn external_continuation_explicit_429_is_not_retried() {
let continuation = ContinuationBinding::External(
PreviousResponseId::new("external-provider-response").expect("previous response"),
);
let opaque_response_id = format!("resp_{}\0opaque", "x".repeat(257));
let continuation = ContinuationBinding::External(PreviousResponseId::new(opaque_response_id));
let (store, provider) = terminal_non_idempotent_failure(
vec![Err(ProviderError::new(
ProviderErrorKind::RateLimited,
@@ -1930,6 +2511,100 @@ fn ambiguous_send_state_stops_retry() {
assert!(finalization.latency_ms.is_some());
}
#[test]
fn pre_delivery_transport_failure_rotates_account_for_non_idempotent_generation() {
let operation = operation(RetrySafety::NonIdempotent);
let route_plan = plan(&operation);
let (coordinator, store, provider) = coordinator(vec![
Script::Stream {
account_id: "acct_first",
items: vec![Err(ProviderError::new(
ProviderErrorKind::Transport,
UpstreamSendState::Ambiguous,
)
.with_pre_delivery_retry())],
},
Script::Stream {
account_id: "acct_second",
items: complete_stream(None),
},
]);
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
None,
None,
CancellationToken::new(),
))
.expect("start execution");
block_on(session.collect_uncommitted()).expect("second account succeeds before delivery");
block_on(session.commit_downstream(Some(200))).expect("commit winning response");
let contexts = provider.contexts.lock().expect("contexts lock");
assert_eq!(contexts.len(), 2);
assert!(
contexts[1]
.excluded_accounts()
.contains(&ProviderAccountId::new("acct_first").expect("account id"))
);
let state = store.state.lock().expect("store lock");
assert_eq!(state.intermediate_failures, 1);
assert_eq!(state.finalizations[0].outcome, ExecutionOutcome::Succeeded);
}
#[test]
fn pre_delivery_retry_marker_is_ignored_after_downstream_commit() {
let operation = operation(RetrySafety::NonIdempotent);
let route_plan = plan(&operation);
let (coordinator, store, provider) = coordinator(vec![
Script::Stream {
account_id: "acct_first",
items: vec![
Ok(GatewayEvent::Started(ResponseMeta::new(
"response-visible",
"gpt-5",
))),
Err(
ProviderError::new(ProviderErrorKind::Transport, UpstreamSendState::Ambiguous)
.with_pre_delivery_retry(),
),
],
},
Script::Stream {
account_id: "acct_second",
items: complete_stream(None),
},
]);
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
None,
None,
CancellationToken::new(),
))
.expect("start execution");
let first = block_on(session.next_event())
.expect("first event")
.expect("visible event");
assert_eq!(
first.commit_requirement(),
CommitRequirement::CommitBeforeDelivery
);
block_on(session.commit_downstream(Some(200))).expect("commit first event");
let error = block_on(session.next_event()).expect_err("committed output cannot be replayed");
assert!(matches!(error, EngineError::Provider(_)));
assert_eq!(provider.contexts.lock().expect("contexts lock").len(), 1);
let state = store.state.lock().expect("store lock");
assert_eq!(state.intermediate_failures, 0);
assert_eq!(state.finalizations[0].outcome, ExecutionOutcome::Incomplete);
assert!(state.finalizations[0].committed);
}
#[test]
fn structural_event_before_replay_safe_failure_should_switch_account_before_commit() {
let operation = operation(RetrySafety::Idempotent);
@@ -5,10 +5,11 @@ use std::time::{Duration, SystemTime};
use serde_json::{Map, Value};
use gateway_core::engine::credential::{
AccountAttemptFeedback, AccountAvailability, AccountCandidate, AccountFeedbackStats,
AccountQuotaSignals, AccountRuntimeSignals, AccountSelectionContext, AccountSelectionPolicy,
AccountSelector, CredentialCasUpdate, CredentialRevision, OpaqueProviderData,
PlaintextCredential, ProviderAccount, ProviderAccountId, ProviderAccountUpdate,
AccountAttemptFeedback, AccountAvailability, AccountAvailabilityPolicy, AccountCandidate,
AccountFeedbackStats, AccountQuotaSignals, AccountRuntimeSignals, AccountSchedulingBlocker,
AccountSelectionContext, AccountSelectionPolicy, AccountSelector, CredentialCasUpdate,
CredentialRevision, OpaqueProviderData, PlaintextCredential, PreferredAccountSelection,
ProviderAccount, ProviderAccountId, ProviderAccountIdentity, ProviderAccountUpdate,
RotationStrategy,
};
use gateway_core::routing::ProviderKind;
@@ -51,6 +52,7 @@ fn context(strategy: RotationStrategy) -> AccountSelectionContext {
excluded_accounts: BTreeSet::new(),
preferred_account: None,
round_robin_cursor: 0,
availability: AccountAvailabilityPolicy::Enforce,
}
}
@@ -72,6 +74,18 @@ fn opaque_provider_data_should_not_expose_quota_values_in_debug() {
assert!(!format!("{quota:?}").contains("private-window"));
}
#[test]
fn provider_account_identity_debug_should_redact_upstream_ids() {
let identity = ProviderAccountIdentity::new(
"private-user-id".to_owned(),
Some("private-account-id".to_owned()),
);
let debug = format!("{identity:?}");
assert!(!debug.contains("private-user-id"));
assert!(!debug.contains("private-account-id"));
}
#[test]
fn availability_should_round_trip_all_database_values() {
assert_eq!(
@@ -80,6 +94,55 @@ fn availability_should_round_trip_all_database_values() {
);
}
#[test]
fn diagnostic_selection_bypasses_all_local_account_availability() {
let exhausted = AccountCandidate {
account: account("acct_exhausted").with_runtime_state(
true,
AccountAvailability::QuotaExhausted,
None,
),
signals: AccountRuntimeSignals {
in_flight: 0,
last_started_at: None,
quota_reset_at: None,
quota_remaining_rank: None,
failure_rate_basis_points: None,
first_output_latency_ms: None,
},
};
let disabled = AccountCandidate {
account: account("acct_disabled").with_runtime_state(
false,
AccountAvailability::QuotaExhausted,
None,
),
signals: AccountRuntimeSignals {
in_flight: 0,
last_started_at: None,
quota_reset_at: None,
quota_remaining_rank: None,
failure_rate_basis_points: None,
first_output_latency_ms: None,
},
};
let mut context = context(RotationStrategy::Sticky);
context.availability = AccountAvailabilityPolicy::BypassForDiagnostic;
assert_eq!(
AccountSelector
.select(std::slice::from_ref(&exhausted), &context)
.map(|selection| selection.candidate().account.id()),
Some(exhausted.account.id())
);
assert_eq!(
AccountSelector
.select(std::slice::from_ref(&disabled), &context)
.map(|selection| selection.candidate().account.id()),
Some(disabled.account.id())
);
}
#[test]
fn disabled_account_should_not_be_schedulable() {
let account =
@@ -119,7 +182,7 @@ fn smart_selector_should_prefer_lower_inflight_count() {
.select(&candidates, &context(RotationStrategy::Smart))
.expect("candidate available");
assert_eq!(selected.account.id().as_str(), "acct_idle");
assert_eq!(selected.candidate().account.id().as_str(), "acct_idle");
}
#[test]
@@ -136,15 +199,13 @@ fn smart_selector_should_balance_signals_instead_of_using_lexicographic_load() {
.select(&candidates, &context(RotationStrategy::Smart))
.expect("candidate available");
assert_eq!(selected.account.id().as_str(), "acct_healthy");
assert_eq!(selected.candidate().account.id().as_str(), "acct_healthy");
}
#[test]
fn smart_selector_should_keep_healthy_preferred_account_at_escape_boundary() {
fn smart_selector_should_report_a_schedulable_preferred_account_hit() {
let fallback = candidate("acct_fallback", 0, Some(100));
let mut preferred = candidate("acct_preferred", 0, Some(1));
preferred.signals.failure_rate_basis_points = Some(5_000);
preferred.signals.first_output_latency_ms = Some(15_000);
let preferred = candidate("acct_preferred", 0, Some(1));
let preferred_id = preferred.account.id().clone();
let mut selection = context(RotationStrategy::Smart);
selection.preferred_account = Some(preferred_id);
@@ -154,16 +215,22 @@ fn smart_selector_should_keep_healthy_preferred_account_at_escape_boundary() {
.select(&candidates, &selection)
.expect("candidate available");
assert_eq!(selected.account.id().as_str(), "acct_preferred");
assert_eq!(
(
selected.candidate().account.id().as_str(),
selected.preferred(),
),
("acct_preferred", PreferredAccountSelection::Hit)
);
}
#[test]
fn smart_selector_should_escape_preferred_account_after_failure_threshold() {
fn smart_selector_should_keep_preferred_account_despite_failure_signals() {
let mut fallback = candidate("acct_fallback", 0, Some(100));
fallback.signals.failure_rate_basis_points = Some(0);
fallback.signals.first_output_latency_ms = Some(100);
let mut preferred = candidate("acct_preferred", 0, Some(100));
preferred.signals.failure_rate_basis_points = Some(5_001);
preferred.signals.failure_rate_basis_points = Some(10_000);
preferred.signals.first_output_latency_ms = Some(100);
let preferred_id = preferred.account.id().clone();
let mut selection = context(RotationStrategy::Smart);
@@ -172,19 +239,19 @@ fn smart_selector_should_escape_preferred_account_after_failure_threshold() {
let selected = AccountSelector
.select(&candidates, &selection)
.expect("fallback candidate available");
.expect("preferred candidate available");
assert_eq!(selected.account.id().as_str(), "acct_fallback");
assert_eq!(selected.candidate().account.id().as_str(), "acct_preferred");
}
#[test]
fn smart_selector_should_escape_preferred_account_after_latency_threshold() {
fn smart_selector_should_keep_preferred_account_despite_latency_signals() {
let mut fallback = candidate("acct_fallback", 0, Some(100));
fallback.signals.failure_rate_basis_points = Some(0);
fallback.signals.first_output_latency_ms = Some(100);
let mut preferred = candidate("acct_preferred", 0, Some(100));
preferred.signals.failure_rate_basis_points = Some(0);
preferred.signals.first_output_latency_ms = Some(15_001);
preferred.signals.first_output_latency_ms = Some(120_000);
let preferred_id = preferred.account.id().clone();
let mut selection = context(RotationStrategy::Smart);
selection.preferred_account = Some(preferred_id);
@@ -192,9 +259,9 @@ fn smart_selector_should_escape_preferred_account_after_latency_threshold() {
let selected = AccountSelector
.select(&candidates, &selection)
.expect("fallback candidate available");
.expect("preferred candidate available");
assert_eq!(selected.account.id().as_str(), "acct_fallback");
assert_eq!(selected.candidate().account.id().as_str(), "acct_preferred");
}
#[test]
@@ -210,7 +277,16 @@ fn smart_selector_should_escape_preferred_account_at_concurrency_limit() {
.select(&candidates, &selection)
.expect("fallback candidate available");
assert_eq!(selected.account.id().as_str(), "acct_fallback");
assert_eq!(
(
selected.candidate().account.id().as_str(),
selected.preferred(),
),
(
"acct_fallback",
PreferredAccountSelection::Blocked(AccountSchedulingBlocker::ConcurrencyLimit),
)
);
}
#[test]
@@ -296,7 +372,10 @@ fn smart_selector_should_use_provider_quota_rank_after_load_is_equal() {
.select(&candidates, &context(RotationStrategy::Smart))
.expect("candidate available");
assert_eq!(selected.account.id().as_str(), "acct_high_quota");
assert_eq!(
selected.candidate().account.id().as_str(),
"acct_high_quota"
);
}
#[test]
@@ -312,7 +391,7 @@ fn quota_reset_selector_should_prefer_known_earliest_window() {
.select(&candidates, &context(RotationStrategy::QuotaResetPriority))
.expect("candidate available");
assert_eq!(selected.account.id().as_str(), "acct_earlier");
assert_eq!(selected.candidate().account.id().as_str(), "acct_earlier");
}
#[test]
@@ -325,7 +404,7 @@ fn smart_selector_should_prefer_known_quota_over_unknown_after_load_is_equal() {
.select(&candidates, &context(RotationStrategy::Smart))
.expect("candidate available");
assert_eq!(selected.account.id().as_str(), "acct_known");
assert_eq!(selected.candidate().account.id().as_str(), "acct_known");
}
#[test]
@@ -338,7 +417,7 @@ fn selector_should_reject_account_at_concurrency_limit() {
.select(&candidates, &context(RotationStrategy::Smart))
.expect("candidate available");
assert_eq!(selected.account.id().as_str(), "acct_available");
assert_eq!(selected.candidate().account.id().as_str(), "acct_available");
}
#[test]
@@ -360,7 +439,7 @@ fn selector_should_enforce_request_interval_until_boundary() {
.select(&candidates, &context)
.expect("candidate available");
assert_eq!(selected.account.id().as_str(), "acct_ready");
assert_eq!(selected.candidate().account.id().as_str(), "acct_ready");
}
#[test]
@@ -377,7 +456,7 @@ fn sticky_selector_should_fall_back_when_requested_account_is_excluded() {
.select(&candidates, &context)
.expect("fallback candidate available");
assert_eq!(selected.account.id().as_str(), "acct_fallback");
assert_eq!(selected.candidate().account.id().as_str(), "acct_fallback");
}
#[test]
@@ -397,11 +476,38 @@ fn round_robin_selector_should_be_stable_for_unsorted_candidates() {
.expect("second candidate available");
assert_eq!(
(first.account.id().as_str(), second.account.id().as_str()),
(
first.candidate().account.id().as_str(),
second.candidate().account.id().as_str(),
),
("acct_first", "acct_second")
);
}
#[test]
fn round_robin_selector_should_honor_a_schedulable_preferred_account_before_the_cursor() {
let candidates = vec![
candidate("acct_first", 0, None),
candidate("acct_second", 0, None),
];
let mut context = context(RotationStrategy::RoundRobin);
context.round_robin_cursor = 0;
context.preferred_account =
Some(ProviderAccountId::new("acct_second").expect("preferred account"));
let selected = AccountSelector
.select(&candidates, &context)
.expect("preferred account available");
assert_eq!(
(
selected.candidate().account.id().as_str(),
selected.preferred(),
),
("acct_second", PreferredAccountSelection::Hit)
);
}
#[test]
fn selector_should_honor_request_local_exclusion() {
let candidates = vec![
@@ -416,7 +522,7 @@ fn selector_should_honor_request_local_exclusion() {
.select(&candidates, &context)
.expect("second account available");
assert_eq!(selected.account.id().as_str(), "acct_second");
assert_eq!(selected.candidate().account.id().as_str(), "acct_second");
}
#[test]
@@ -432,7 +538,7 @@ fn sticky_selector_should_prefer_requested_account() {
.select(&candidates, &context)
.expect("sticky account available");
assert_eq!(selected.account.id().as_str(), "acct_second");
assert_eq!(selected.candidate().account.id().as_str(), "acct_second");
}
#[test]
@@ -447,7 +553,7 @@ fn sticky_selector_without_request_binding_should_reuse_latest_account() {
.select(&candidates, &context(RotationStrategy::Sticky))
.expect("sticky candidate available");
assert_eq!(selected.account.id().as_str(), "acct_latest");
assert_eq!(selected.candidate().account.id().as_str(), "acct_latest");
}
#[test]
@@ -462,7 +568,7 @@ fn round_robin_selector_should_use_frozen_cursor() {
.select(&candidates, &context)
.expect("candidate available");
assert_eq!(selected.account.id().as_str(), "acct_second");
assert_eq!(selected.candidate().account.id().as_str(), "acct_second");
}
#[test]
@@ -4,10 +4,11 @@ use std::{
Arc, Mutex,
atomic::{AtomicBool, Ordering},
},
time::{Duration, SystemTime},
time::{Duration, Instant, SystemTime},
};
use async_trait::async_trait;
use bytes::Bytes;
use futures::{executor::block_on, future::BoxFuture};
use gateway_core::engine::admission::{
ClientAdmissionDecision, ClientAdmissionError, ClientAdmissionPort, ClientAdmissionRecovery,
@@ -20,8 +21,9 @@ use gateway_core::engine::credential::{
AccountSelectionPolicy, ProviderAccountId, RotationStrategy,
};
use gateway_core::engine::execution::{
ClientApiKeyUsageSink, DefaultExecutionService, ExecutionService, ProviderCircuitDecision,
ProviderCircuitError, ProviderCircuitPort, provider_failure_affects_circuit,
ClientApiKeyUsageSink, ClientTransport, DefaultExecutionService, ExecutionRequestMetadata,
ExecutionService, ProviderCircuitDecision, ProviderCircuitError, ProviderCircuitPort,
StartExecution, provider_failure_affects_circuit,
};
use gateway_core::engine::probe::{AccountProbe, AccountProbeRequest};
use gateway_core::engine::provider::{
@@ -32,12 +34,15 @@ use gateway_core::engine::{
AttemptContext, AttemptRecord, ExecutionStore, IntermediateFailure, ModelRequestFinalization,
ModelRequestId, NewModelRequest, ProbeFailure, RecoveryReport, UpstreamSendState,
};
use gateway_core::error::{GatewayErrorKind, ProviderError, ProviderErrorKind, StoreError};
use gateway_core::error::{
ClientVisibleUpstreamResponse, GatewayErrorKind, ProviderError, ProviderErrorKind, StoreError,
StoreErrorKind,
};
use gateway_core::operation::{GenerateRequest, Operation, OperationKind, ProtocolPayload};
use gateway_core::policy::{ClientApiKeyId, ClientPolicy, PlaintextClientApiKey, RateLimits};
use gateway_core::routing::snapshot::RuntimeSnapshotHandle;
use gateway_core::routing::{
ConfigRevision, ModelCapabilities, ProviderKind, ProviderModel, RuntimeSnapshot,
ConfigRevision, ModelCapabilities, ProviderKind, ProviderModel, PublicModelId, RuntimeSnapshot,
UpstreamModelId,
};
use serde_json::json;
@@ -105,10 +110,54 @@ fn probe_failures_should_be_observable_without_a_model_request_row() {
.expect_err("the provider rejects every probe");
assert_eq!(error.kind(), GatewayErrorKind::UpstreamUnavailable);
let upstream = error
.upstream_response()
.expect("probe must preserve its request-local upstream response");
assert_eq!(upstream.status(), 502);
assert_eq!(
upstream.content_type(),
Some(b"application/json".as_slice())
);
assert_eq!(
upstream.body(),
&Bytes::from_static(br#"{"error":{"message":"source upstream failure"}}"#),
);
assert!(!format!("{error:?}").contains("source upstream failure"));
assert!(!store.touched.load(Ordering::SeqCst));
assert_eq!(store.probe_failures(), vec!["transport".to_owned()]);
}
#[test]
fn probe_observation_store_failure_preserves_the_provider_error() {
let store = Arc::new(TrackingExecutionStore {
fail_probe_observation: AtomicBool::new(true),
..TrackingExecutionStore::default()
});
let providers = ProviderRegistry::new([Arc::new(FailingProvider) as Arc<dyn Provider>])
.expect("provider registry");
let service = DefaultExecutionService::new(
RuntimeSnapshotHandle::new(probe_snapshot()),
store.clone(),
providers,
Arc::new(UnusedAdmissions),
Arc::new(UnusedCircuits),
Arc::new(UnusedContinuation),
Arc::new(RecordingClientApiKeyUsage::default()),
);
let error = block_on(service.probe(AccountProbeRequest {
account_id: ProviderAccountId::new("acct_probe").expect("account ID"),
provider_kind: ProviderKind::new("openai").expect("provider kind"),
upstream_model: UpstreamModelId::new("gpt-probe").expect("model ID"),
operation: probe_operation(),
}))
.expect_err("the provider error must survive observation failure");
assert_eq!(error.kind(), GatewayErrorKind::UpstreamUnavailable);
assert!(!store.touched.load(Ordering::SeqCst));
assert!(store.probe_failures().is_empty());
}
struct FailingProvider;
#[async_trait]
@@ -132,10 +181,14 @@ impl Provider for FailingProvider {
_request: ProviderRequest,
_context: AttemptContext,
) -> Result<ProviderStream, ProviderError> {
Err(ProviderError::new(
ProviderErrorKind::Transport,
UpstreamSendState::NotSent,
))
Err(
ProviderError::new(ProviderErrorKind::Transport, UpstreamSendState::NotSent)
.with_client_visible_upstream_response(ClientVisibleUpstreamResponse::new(
502,
Some(b"application/json".to_vec()),
Bytes::from_static(br#"{"error":{"message":"source upstream failure"}}"#),
)),
)
}
}
@@ -159,6 +212,111 @@ fn successful_authentication_should_record_client_key_usage() {
assert_eq!(usage.recorded(), vec!["key_usage_test".to_owned()]);
}
#[test]
fn circuit_store_failure_should_fail_open_during_request_start() {
let service = DefaultExecutionService::new(
RuntimeSnapshotHandle::new(start_snapshot()),
Arc::new(TrackingExecutionStore::default()),
ProviderRegistry::default(),
Arc::new(UnusedAdmissions),
Arc::new(FailingDecisionCircuits),
Arc::new(UnusedContinuation),
Arc::new(RecordingClientApiKeyUsage::default()),
);
let client = service
.authenticate("sk_start_test")
.expect("authenticated client");
let started = block_on(service.start(StartExecution {
client,
public_model: PublicModelId::new("gpt-start").expect("public model"),
operation: start_operation(),
metadata: ExecutionRequestMetadata {
protocol: "openai".to_owned(),
endpoint: "/v1/responses".to_owned(),
transport: ClientTransport::HttpJson,
stream: false,
client_ip: None,
user_agent: None,
previous_response_id: None,
},
}))
.expect("recoverable circuit state must not reject the request");
assert!(!started.session.is_finalized());
}
#[test]
fn slow_circuit_store_should_time_out_and_fail_open_during_request_start() {
let service = DefaultExecutionService::new(
RuntimeSnapshotHandle::new(start_snapshot()),
Arc::new(TrackingExecutionStore::default()),
ProviderRegistry::default(),
Arc::new(UnusedAdmissions),
Arc::new(PendingDecisionCircuits),
Arc::new(UnusedContinuation),
Arc::new(RecordingClientApiKeyUsage::default()),
);
let client = service
.authenticate("sk_start_test")
.expect("authenticated client");
let started_at = Instant::now();
let started = block_on(service.start(StartExecution {
client,
public_model: PublicModelId::new("gpt-start").expect("public model"),
operation: start_operation(),
metadata: ExecutionRequestMetadata {
protocol: "openai".to_owned(),
endpoint: "/v1/responses".to_owned(),
transport: ClientTransport::HttpJson,
stream: false,
client_ip: None,
user_agent: None,
previous_response_id: None,
},
}))
.expect("slow recoverable circuit state must not reject the request");
assert!(started_at.elapsed() < Duration::from_secs(2));
assert!(!started.session.is_finalized());
}
#[test]
fn stale_nonempty_catalog_should_not_block_a_new_model_during_request_start() {
let service = DefaultExecutionService::new(
RuntimeSnapshotHandle::new(start_snapshot()),
Arc::new(TrackingExecutionStore::default()),
ProviderRegistry::default(),
Arc::new(UnusedAdmissions),
Arc::new(UnusedCircuits),
Arc::new(UnusedContinuation),
Arc::new(RecordingClientApiKeyUsage::default()),
);
let client = service
.authenticate("sk_start_test")
.expect("authenticated client");
let model = "gpt-future-not-in-catalog";
let started = block_on(service.start(StartExecution {
client,
public_model: PublicModelId::from_client_wire(model).expect("client model"),
operation: start_operation_for_model(model),
metadata: ExecutionRequestMetadata {
protocol: "openai".to_owned(),
endpoint: "/v1/responses".to_owned(),
transport: ClientTransport::HttpJson,
stream: false,
client_ip: None,
user_agent: None,
previous_response_id: None,
},
}))
.expect("stale model catalog must not reject the request");
assert!(!started.session.is_finalized());
}
#[derive(Default)]
struct RecordingClientApiKeyUsage {
ids: Mutex<Vec<String>>,
@@ -183,6 +341,7 @@ impl ClientApiKeyUsageSink for RecordingClientApiKeyUsage {
struct TrackingExecutionStore {
touched: AtomicBool,
probe_failures: Mutex<Vec<String>>,
fail_probe_observation: AtomicBool,
}
impl TrackingExecutionStore {
@@ -240,6 +399,9 @@ impl ExecutionStore for TrackingExecutionStore {
}
async fn record_probe_failure(&self, failure: ProbeFailure) -> Result<(), StoreError> {
if self.fail_probe_observation.load(Ordering::SeqCst) {
return Err(StoreError::new(StoreErrorKind::Unavailable));
}
self.probe_failures
.lock()
.expect("probe failures lock")
@@ -309,6 +471,56 @@ impl ProviderCircuitPort for UnusedCircuits {
}
}
struct FailingDecisionCircuits;
impl ProviderCircuitPort for FailingDecisionCircuits {
fn decision<'a>(
&'a self,
_: &'a ProviderKind,
) -> BoxFuture<'a, Result<ProviderCircuitDecision, ProviderCircuitError>> {
Box::pin(async { Err(ProviderCircuitError) })
}
fn observe_failure<'a>(
&'a self,
_: &'a ProviderKind,
) -> BoxFuture<'a, Result<(), ProviderCircuitError>> {
Box::pin(async { Ok(()) })
}
fn observe_success<'a>(
&'a self,
_: &'a ProviderKind,
) -> BoxFuture<'a, Result<(), ProviderCircuitError>> {
Box::pin(async { Ok(()) })
}
}
struct PendingDecisionCircuits;
impl ProviderCircuitPort for PendingDecisionCircuits {
fn decision<'a>(
&'a self,
_: &'a ProviderKind,
) -> BoxFuture<'a, Result<ProviderCircuitDecision, ProviderCircuitError>> {
Box::pin(futures::future::pending())
}
fn observe_failure<'a>(
&'a self,
_: &'a ProviderKind,
) -> BoxFuture<'a, Result<(), ProviderCircuitError>> {
Box::pin(async { Ok(()) })
}
fn observe_success<'a>(
&'a self,
_: &'a ProviderKind,
) -> BoxFuture<'a, Result<(), ProviderCircuitError>> {
Box::pin(async { Ok(()) })
}
}
struct UnusedContinuation;
impl NativeContinuationPort for UnusedContinuation {
@@ -371,6 +583,34 @@ fn client_snapshot() -> RuntimeSnapshot {
.expect("client snapshot")
}
fn start_snapshot() -> RuntimeSnapshot {
let provider = ProviderKind::new("openai").expect("provider kind");
let capabilities =
ModelCapabilities::new(BTreeSet::from([OperationKind::Generate]), Some(16_000));
RuntimeSnapshot::new(
ConfigRevision::new(1).expect("config revision"),
AccountSelectionPolicy::new(
RotationStrategy::Smart,
std::num::NonZeroU32::new(1).expect("concurrency"),
Duration::from_millis(1),
),
vec![provider.clone()],
vec![ProviderModel::new(
provider.clone(),
UpstreamModelId::new("gpt-start").expect("model ID"),
capabilities,
)],
vec![ClientPolicy::new(
ClientApiKeyId::new("key_start_test").expect("client API key ID"),
PlaintextClientApiKey::new("sk_start_test").expect("plaintext client API key"),
provider,
true,
RateLimits::unlimited(),
)],
)
.expect("start snapshot")
}
fn probe_operation() -> Operation {
let body = json!({
"model": "gpt-probe",
@@ -381,3 +621,18 @@ fn probe_operation() -> Operation {
.expect("OpenAI payload"),
))
}
fn start_operation() -> Operation {
start_operation_for_model("gpt-start")
}
fn start_operation_for_model(model: &str) -> Operation {
let body = json!({
"model": model,
"input": [{"type": "message", "role": "user", "content": "ping"}],
});
Operation::Generate(GenerateRequest::from_protocol_payload(
ProtocolPayload::json_object("openai", body.as_object().expect("request object").clone())
.expect("OpenAI payload"),
))
}
+104 -10
View File
@@ -1,7 +1,11 @@
use bytes::Bytes;
use gateway_core::engine::UpstreamSendState;
use gateway_core::error::{
ClientVisibleUpstreamError, GatewayError, ProviderError, ProviderErrorKind, SafeUpstreamValue,
ClientVisibleUpstreamError, ClientVisibleUpstreamResponse, GatewayError, OpaqueUpstreamValue,
ProviderError, ProviderErrorKind,
};
use gateway_core::event::{ProtocolWireEvent, ProviderEvent, ProviderResponseHeader};
use serde_json::json;
#[test]
fn provider_error_debug_should_not_expose_sensitive_context() {
@@ -15,13 +19,22 @@ fn provider_error_debug_should_not_expose_sensitive_context() {
#[test]
fn provider_error_debug_should_not_print_classified_upstream_values() {
let diagnostic = "request-visible-only-through-explicit-accessor";
let value = SafeUpstreamValue::new(diagnostic).expect("test diagnostic is valid");
let value = OpaqueUpstreamValue::new(diagnostic);
let error = ProviderError::new(ProviderErrorKind::Unavailable, UpstreamSendState::Sent)
.with_upstream_request_id(value);
assert!(!format!("{error:?}").contains(diagnostic));
}
#[test]
fn opaque_upstream_value_should_preserve_arbitrary_text_without_logging_it() {
let original = format!("\0{}\n", "x".repeat(9_000));
let value = OpaqueUpstreamValue::new(original.clone());
assert_eq!(value.as_str(), original);
assert!(!format!("{value:?}").contains(&original));
}
#[test]
fn provider_error_replay_proof_should_default_to_false() {
let error = ProviderError::new(ProviderErrorKind::RateLimited, UpstreamSendState::Sent);
@@ -37,18 +50,82 @@ fn provider_error_replay_proof_should_be_explicit() {
assert!(error.replay_is_safe());
}
#[test]
fn provider_error_atomic_client_events_should_be_take_only_and_debug_redacted() {
let marker = "atomic-wire-must-not-enter-debug";
let event = ProviderEvent::wire(
ProtocolWireEvent::json(
"openai",
Some("response.failed".to_owned()),
json!({"type": "response.failed", "message": marker}),
)
.expect("atomic wire"),
);
let mut error = ProviderError::new(ProviderErrorKind::RateLimited, UpstreamSendState::Sent)
.with_atomic_client_events(vec![event]);
assert!(error.has_atomic_client_events());
assert!(!format!("{error:?}").contains(marker));
assert_eq!(error.take_atomic_client_events().len(), 1);
assert!(!error.has_atomic_client_events());
}
#[test]
fn provider_error_clone_should_drop_request_local_upstream_response() {
let marker = Bytes::from_static(b"raw-client-response-only");
let error = ProviderError::new(ProviderErrorKind::RateLimited, UpstreamSendState::Sent)
.with_client_visible_upstream_response(
ClientVisibleUpstreamResponse::new(
429,
Some(b"application/problem+json".to_vec()),
marker.clone(),
)
.with_headers(vec![ProviderResponseHeader::new(
"x-future-error",
Bytes::from_static(b"opaque-header-value"),
)]),
);
let cloned = error.clone();
assert_eq!(
error
.client_visible_upstream_response()
.map(ClientVisibleUpstreamResponse::body),
Some(&marker)
);
assert!(cloned.client_visible_upstream_response().is_none());
assert!(!format!("{error:?}").contains("raw-client-response-only"));
assert!(!format!("{error:?}").contains("opaque-header-value"));
}
#[test]
fn provider_error_clone_should_drop_atomic_client_events() {
let event = ProviderEvent::wire(
ProtocolWireEvent::json(
"openai",
Some("response.failed".to_owned()),
json!({"type": "response.failed"}),
)
.expect("atomic wire"),
);
let error = ProviderError::new(ProviderErrorKind::RateLimited, UpstreamSendState::Sent)
.with_atomic_client_events(vec![event]);
let cloned = error.clone();
assert!(error.has_atomic_client_events());
assert!(!cloned.has_atomic_client_events());
}
#[test]
fn gateway_error_should_keep_client_visible_upstream_fields_out_of_safe_diagnostics() {
let message = "Your Codex quota is exhausted";
let error = ProviderError::new(ProviderErrorKind::QuotaExhausted, UpstreamSendState::Sent)
.with_client_visible_upstream_error(
ClientVisibleUpstreamError::new(
message,
Some("quota_exhausted".to_owned()),
Some("rate_limit_error".to_owned()),
)
.expect("safe structured upstream error"),
);
.with_client_visible_upstream_error(ClientVisibleUpstreamError::new(
message,
Some("quota_exhausted".to_owned()),
Some("rate_limit_error".to_owned()),
));
let gateway = GatewayError::from_provider(&error);
assert_eq!(
@@ -60,3 +137,20 @@ fn gateway_error_should_keep_client_visible_upstream_fields_out_of_safe_diagnost
assert_eq!(gateway.client_error_type(), Some("rate_limit_error"));
assert!(!format!("{gateway:?}").contains(message));
}
#[test]
fn client_visible_upstream_error_should_preserve_opaque_structured_fields() {
let message = format!("\0{}\n", "m".repeat(9_000));
let code = format!("\0{}", "c".repeat(300));
let error_type = String::new();
let detail = ClientVisibleUpstreamError::new(
message.clone(),
Some(code.clone()),
Some(error_type.clone()),
);
assert_eq!(detail.message(), message);
assert_eq!(detail.code(), Some(code.as_str()));
assert_eq!(detail.error_type(), Some(error_type.as_str()));
assert!(!format!("{detail:?}").contains(&message));
}
+42 -2
View File
@@ -1,8 +1,9 @@
use bytes::Bytes;
use gateway_core::engine::provider::UpstreamTransport;
use gateway_core::event::{
ContentItem, ContentKind, EventSequenceError, EventSequenceValidator, GatewayEvent,
ProtocolWireEvent, ProviderEvent, ProviderResponseObservation, ReasoningDelta, ResponseMeta,
TextDelta, ToolCallDelta,
ProtocolWireEvent, ProviderEvent, ProviderResponseHeader, ProviderResponseObservation,
ReasoningDelta, ResponseMeta, TextDelta, ToolCallDelta,
};
use serde_json::json;
@@ -21,6 +22,28 @@ fn provider_event_keeps_large_wire_and_observation_payloads_behind_one_indirecti
assert!(!observation.has_client_event());
}
#[test]
fn provider_response_header_should_preserve_opaque_name_and_bytes_without_debug_disclosure() {
let name = format!("x-future-{}\0", "n".repeat(512));
let value = Bytes::from_static(b"\xffopaque-response-header-secret");
let header = ProviderResponseHeader::new(name.clone(), value.clone());
assert_eq!(header.name(), name);
assert_eq!(header.value(), &value);
let rendered = format!("{header:?}");
assert!(!rendered.contains(&name));
assert!(!rendered.contains("opaque-response-header-secret"));
}
#[test]
fn invalid_observed_service_tier_should_be_ignored() {
let observation =
ProviderResponseObservation::new(UpstreamTransport::new("http_sse").expect("transport"))
.with_service_tier_if_valid(format!("priority\0{}", "x".repeat(128)));
assert_eq!(observation.service_tier(), None);
}
#[test]
fn protocol_wire_event_should_preserve_sse_metadata_without_exposing_id_in_debug() {
let wire = ProtocolWireEvent::json_with_sse_metadata(
@@ -37,6 +60,23 @@ fn protocol_wire_event_should_preserve_sse_metadata_without_exposing_id_in_debug
assert!(!format!("{wire:?}").contains("upstream-event-id"));
}
#[test]
fn protocol_wire_event_should_preserve_opaque_sse_metadata() {
let event_type = format!("\0{}", "event".repeat(80));
let sse_id = "id\0with\r\ncontrols".to_owned();
let wire = ProtocolWireEvent::json_with_sse_metadata(
"openai",
Some(event_type.clone()),
json!({"type": "future.event"}),
Some(sse_id.clone()),
None,
)
.expect("internal protocol name is valid");
assert_eq!(wire.event_type(), Some(event_type.as_str()));
assert_eq!(wire.sse_id(), Some(sse_id.as_str()));
}
#[test]
fn validator_should_accept_started_content_delta_completed_sequence() {
let mut validator = EventSequenceValidator::new();
@@ -57,7 +57,7 @@ fn refresh_policy_requires_a_positive_margin() {
}
#[test]
fn refresh_schedule_is_stable_and_never_uses_query_time_margin() {
fn refresh_schedule_should_use_the_exact_configured_margin() {
let account = ProviderAccountId::new("acct_stable").expect("valid account");
let policy = ProviderRefreshPolicy::try_new(
Duration::from_secs(3_600),
@@ -70,13 +70,7 @@ fn refresh_schedule_is_stable_and_never_uses_query_time_margin() {
let first = policy
.next_attempt_at(&account, expires_at, observed_at)
.expect("schedule refresh");
let second = policy
.next_attempt_at(&account, expires_at, observed_at)
.expect("schedule refresh again");
assert_eq!(first, second);
assert!(first > observed_at);
assert!(first < expires_at);
assert_eq!(first, expires_at - Duration::from_secs(3_600));
}
#[test]
@@ -190,21 +190,36 @@ fn global_mapping_should_apply_to_every_provider() {
#[test]
fn unmapped_model_should_pass_through_unchanged() {
let snapshot = snapshot();
let plan = snapshot
.plan(
&PublicModelId::new("future-openai-model").expect("model"),
&operation(),
&RoutingContext {
provider_kind: Some(ProviderKind::new("openai").expect("provider")),
..RoutingContext::default()
},
)
.expect("unknown model remains transparent");
for requested in [
"future-openai-model".to_owned(),
format!("future-{}", "x".repeat(512)),
"future\0model".to_owned(),
" future-openai-model ".to_owned(),
] {
let plan = snapshot
.plan(
&PublicModelId::from_client_wire(requested.clone()).expect("client model"),
&operation(),
&RoutingContext {
provider_kind: Some(ProviderKind::new("openai").expect("provider")),
..RoutingContext::default()
},
)
.expect("unknown model remains transparent");
assert_eq!(
plan.candidates()[0].upstream_model().as_str(),
"future-openai-model"
);
assert_eq!(plan.candidates()[0].upstream_model().as_str(), requested);
}
}
#[test]
fn model_mapping_should_use_exact_client_keys() {
let snapshot = snapshot().with_model_mappings(BTreeMap::from([(
" exact-alias ".to_owned(),
"gpt-5.5".to_owned(),
)]));
assert_eq!(snapshot.mapped_model(" exact-alias "), "gpt-5.5");
assert_eq!(snapshot.mapped_model("exact-alias"), "exact-alias");
}
#[test]
+1 -1
View File
@@ -13,7 +13,7 @@ path = "tests/main.rs"
[dependencies]
serde = { version = "1.0.228", features = ["derive"] }
serde_json = { version = "1.0.150", features = ["preserve_order"] }
serde_json = { version = "1.0.150", features = ["arbitrary_precision", "preserve_order"] }
thiserror = "2.0.18"
uuid = { version = "1.23.3", features = ["v4"] }
@@ -0,0 +1,126 @@
use serde_json::{Map, Value};
const MULTI_AGENT_MODE_OPEN_TAG: &str = "<multi_agent_mode>";
const MULTI_AGENT_MODE_CLOSE_TAG: &str = "</multi_agent_mode>";
const PROACTIVE_MULTI_AGENT_MODE_PREFIX: &str = "Proactive multi-agent delegation is active.";
/// 从 OpenAI Responses 请求中提取的稳定 Codex 请求语义。
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct CodexResponsesRequestSemantics {
/// 客户端原始选择的推理强度。
pub reasoning_effort: Option<String>,
/// Codex turn metadata 中的请求类型。
pub request_kind: Option<String>,
/// Codex turn metadata 中的子代理类型。
pub subagent_kind: Option<String>,
/// 网关侧派生出的推理预设。
pub reasoning_preset: Option<&'static str>,
/// 请求是否为 Codex 压缩请求。
pub compact: bool,
}
/// 从 Responses body 与非 wire 协议上下文中提取 Codex 语义。
#[must_use]
pub fn codex_responses_request_semantics(
body: &Map<String, Value>,
context: &Map<String, Value>,
) -> CodexResponsesRequestSemantics {
codex_responses_request_semantics_with_turn_metadata(
body,
non_empty_string(context.get("turn_metadata")),
)
}
/// 调用方已解析权威 turn metadata 时,从 Responses body 中提取 Codex 语义。
#[must_use]
pub fn codex_responses_request_semantics_with_turn_metadata(
body: &Map<String, Value>,
turn_metadata: Option<&str>,
) -> CodexResponsesRequestSemantics {
let reasoning_effort = body
.get("reasoning")
.and_then(Value::as_object)
.and_then(|reasoning| non_empty_string(reasoning.get("effort")))
.map(ToOwned::to_owned);
let parsed_turn_metadata = turn_metadata
.or_else(|| request_turn_metadata(body))
.and_then(|value| serde_json::from_str::<Value>(value).ok());
let request_kind = parsed_turn_metadata
.as_ref()
.and_then(|metadata| non_empty_string(metadata.get("request_kind")))
.map(ToOwned::to_owned);
let subagent_kind = parsed_turn_metadata
.as_ref()
.and_then(|metadata| non_empty_string(metadata.get("subagent_kind")))
.map(ToOwned::to_owned);
let proactive_multi_agent = latest_multi_agent_mode(input_items(body).iter())
.is_some_and(|mode| mode.starts_with(PROACTIVE_MULTI_AGENT_MODE_PREFIX));
let compact_trigger = input_items(body)
.iter()
.any(|item| non_empty_string(item.get("type")) == Some("compaction_trigger"));
let reasoning_preset = (subagent_kind.is_none()
&& reasoning_effort.as_deref() == Some("max")
&& proactive_multi_agent)
.then_some("ultra");
let compact = request_kind.as_deref() == Some("compaction") || compact_trigger;
CodexResponsesRequestSemantics {
reasoning_effort,
request_kind,
subagent_kind,
reasoning_preset,
compact,
}
}
fn request_turn_metadata(body: &Map<String, Value>) -> Option<&str> {
non_empty_string(body.get("turnMetadata"))
.or_else(|| non_empty_string(body.get("turn_metadata")))
.or_else(|| non_empty_string(body.get("x-codex-turn-metadata")))
.or_else(|| client_metadata_string(body, "x-codex-turn-metadata"))
.or_else(|| client_metadata_string(body, "turnMetadata"))
.or_else(|| client_metadata_string(body, "turn_metadata"))
}
fn client_metadata_string<'a>(body: &'a Map<String, Value>, key: &str) -> Option<&'a str> {
body.get("client_metadata")
.and_then(Value::as_object)
.and_then(|metadata| non_empty_string(metadata.get(key)))
}
fn input_items(body: &Map<String, Value>) -> &[Value] {
body.get("input")
.and_then(Value::as_array)
.map_or(&[], Vec::as_slice)
}
fn latest_multi_agent_mode<'a>(
input: impl DoubleEndedIterator<Item = &'a Value>,
) -> Option<&'a str> {
input.rev().find_map(|item| {
if non_empty_string(item.get("role")) != Some("developer") {
return None;
}
item.get("content")?
.as_array()?
.iter()
.rev()
.filter_map(|content| non_empty_string(content.get("text")))
.find_map(multi_agent_mode_from_text)
})
}
fn multi_agent_mode_from_text(text: &str) -> Option<&str> {
let close = text.rfind(MULTI_AGENT_MODE_CLOSE_TAG)?;
let before_close = &text[..close];
let open = before_close.rfind(MULTI_AGENT_MODE_OPEN_TAG)?;
let body = &before_close[open + MULTI_AGENT_MODE_OPEN_TAG.len()..];
Some(body.trim())
}
fn non_empty_string(value: Option<&Value>) -> Option<&str> {
value
.and_then(Value::as_str)
.map(str::trim)
.filter(|value| !value.is_empty())
}
@@ -151,7 +151,7 @@ pub fn retry_after_seconds_from_value(value: &Value) -> Option<u64> {
.or_else(|| {
error
.get("resets_in_seconds")
.and_then(Value::as_u64)
.and_then(positive_u64)
.filter(|seconds| *seconds > 0)
})
.or_else(|| retry_after_seconds_from_resets_at(error))
@@ -317,11 +317,7 @@ pub fn is_rate_limit_header_name(name: &str) -> bool {
/// 从内部 `codex.rate_limits` 事件中解析限流信息。
pub fn parse_rate_limits_event(value: &Value) -> Option<ParsedRateLimits> {
if value
.get("type")
.and_then(Value::as_str)
.is_some_and(|event_type| event_type != "codex.rate_limits")
{
if value.get("type").and_then(Value::as_str) != Some("codex.rate_limits") {
return None;
}
@@ -396,6 +392,12 @@ pub fn rate_limits_to_header_pairs(rate_limits: &ParsedRateLimits) -> Vec<(Strin
&format!("{prefix}-secondary"),
details.secondary,
);
if let Some(allowed) = details.allowed {
headers.push((format!("{prefix}-allowed"), allowed.to_string()));
}
if let Some(limit_reached) = details.limit_reached {
headers.push((format!("{prefix}-limit-reached"), limit_reached.to_string()));
}
if let Some(limit_name) = &details.limit_name {
headers.push((format!("{prefix}-limit-name"), limit_name.clone()));
}
@@ -441,7 +443,7 @@ fn nested_number_field(value: &Value, path: &[&str]) -> Option<u64> {
}
fn retry_after_seconds_from_resets_at(error: &Value) -> Option<u64> {
let resets_at = error.get("resets_at").and_then(Value::as_u64)?;
let resets_at = error.get("resets_at").and_then(positive_u64)?;
let now = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.ok()?
@@ -449,6 +451,13 @@ fn retry_after_seconds_from_resets_at(error: &Value) -> Option<u64> {
(resets_at > now).then_some(resets_at - now)
}
fn positive_u64(value: &Value) -> Option<u64> {
value
.as_u64()
.or_else(|| value.as_str().and_then(|value| value.parse::<u64>().ok()))
.filter(|value| *value > 0)
}
fn retry_after_seconds_from_rate_limit_message(error: &Value) -> Option<u64> {
let code = error
.get("code")
@@ -9,7 +9,14 @@ pub const WS_REQUEST_HEADER_RESPONSES_LITE_CLIENT_METADATA_KEY: &str =
/// Codex memory consolidation 请求标记。
pub const X_OPENAI_MEMGEN_REQUEST_HEADER: &str = "x-openai-memgen-request";
/// OpenAI Responses 请求中携带的 Codex 专属语义。
pub mod codex;
/// OpenAI/Codex 事件语义、用量与限流字段编解码。
pub mod events;
/// Server-Sent Events 帧的解析与编码。
pub mod sse;
pub use codex::{
CodexResponsesRequestSemantics, codex_responses_request_semantics,
codex_responses_request_semantics_with_turn_metadata,
};
@@ -18,10 +18,11 @@ pub struct SseEvent {
pub retry: Option<u64>,
}
/// 保留完整原始字节与旁路解析结果的单个 SSE 帧。
/// 保留原始字节与旁路解析结果的 SSE wire 片段。
///
/// 这用于需要原样转发的 transport;`events` 只供旁路观测,不能反过来重建
/// `raw`。
/// 通常一个片段就是完整 SSE 帧。单帧超过旁路观测预算时会拆成多个原始片段,
/// 此时 `events` 为空;透明 transport 必须按顺序交付 `raw`,不能从 `events`
/// 反向重建或否决原始数据。
#[derive(Clone, PartialEq, Eq)]
pub struct SseFrame {
raw: Vec<u8>,
@@ -62,7 +63,7 @@ impl fmt::Debug for SseFrame {
}
}
/// 单事件缓冲上限。
/// 单事件旁路解析缓冲上限。
pub const MAX_SSE_EVENT_BUFFER_BYTES: usize = 64 * 1024 * 1024;
/// SSE 流结束标记帧。
@@ -75,6 +76,9 @@ pub struct SseEventDecoder {
/// `pending` 中已确认不含帧分隔符的前缀长度。新数据到达时从该边界
/// 回退分隔符最大跨界宽度继续扫描,大帧跨多个 chunk 不会被反复重扫。
scanned: usize,
/// 超大未完成帧已开始直接交付;在遇到帧分隔符前只保留跨 chunk 扫描尾部,
/// 不再尝试解析该帧。
opaque_frame: bool,
}
impl SseEventDecoder {
@@ -129,28 +133,35 @@ impl SseEventDecoder {
/// 与 [`Self::push`] 相比,此方法仅应由需要原样转发的 transport 使用;普通
/// 解析路径无需为原始帧额外分配内存。旁路解析失败时仍保留原始帧,并以空
/// event 列表表示不可观测,避免观测失败截断客户端字节流。
pub fn push_frames(&mut self, chunk: &[u8]) -> Result<Vec<SseFrame>, SseError> {
pub fn push_frames(&mut self, chunk: &[u8]) -> Vec<SseFrame> {
self.pending.extend_from_slice(chunk);
let mut frames = Vec::new();
if self.opaque_frame && !self.finish_opaque_frame(&mut frames) {
self.flush_opaque_prefix(&mut frames);
return frames;
}
let mut consumed = 0usize;
while let Some(end) = self.next_frame_end(consumed) {
let raw = self.pending[consumed..end].to_vec();
let events = std::str::from_utf8(&raw)
.ok()
.and_then(|frame| parse_sse_events(frame).ok())
.unwrap_or_default();
let events = if raw.len() > MAX_SSE_EVENT_BUFFER_BYTES {
Vec::new()
} else {
std::str::from_utf8(&raw)
.ok()
.and_then(|frame| parse_sse_events(frame).ok())
.unwrap_or_default()
};
frames.push(SseFrame::new(raw, events));
consumed = end;
}
self.commit_consumed(consumed);
if self.pending.len() > MAX_SSE_EVENT_BUFFER_BYTES {
return Err(SseError::BufferExceeded {
max_bytes: MAX_SSE_EVENT_BUFFER_BYTES,
});
self.opaque_frame = true;
self.flush_opaque_prefix(&mut frames);
}
Ok(frames)
frames
}
/// 流结束时解析尚未带空行分隔符的最后一帧。
@@ -168,17 +179,53 @@ impl SseEventDecoder {
/// 在流结束时返回未带空行分隔符的最后一帧及其旁路解析结果。
///
/// 与 [`Self::push_frames`] 一致,旁路解析失败不会丢弃原始帧。
pub fn finish_frames(&mut self) -> Result<Vec<SseFrame>, SseError> {
pub fn finish_frames(&mut self) -> Vec<SseFrame> {
self.scanned = 0;
if self.pending.is_empty() {
return Ok(Vec::new());
self.opaque_frame = false;
return Vec::new();
}
let raw = std::mem::take(&mut self.pending);
let events = std::str::from_utf8(&raw)
.ok()
.and_then(|frame| parse_sse_events(frame).ok())
.unwrap_or_default();
Ok(vec![SseFrame::new(raw, events)])
let events = if self.opaque_frame || raw.len() > MAX_SSE_EVENT_BUFFER_BYTES {
Vec::new()
} else {
std::str::from_utf8(&raw)
.ok()
.and_then(|frame| parse_sse_events(frame).ok())
.unwrap_or_default()
};
self.opaque_frame = false;
vec![SseFrame::new(raw, events)]
}
/// 继续透明交付一个已超出观测预算的帧,并在分隔符后恢复普通解析。
fn finish_opaque_frame(&mut self, frames: &mut Vec<SseFrame>) -> bool {
let Some((position, separator_len)) = sse_frame_separator_bytes(&self.pending) else {
return false;
};
let end = position + separator_len;
let remaining = self.pending.split_off(end);
let raw = std::mem::replace(&mut self.pending, remaining);
frames.push(SseFrame::new(raw, Vec::new()));
self.opaque_frame = false;
self.scanned = 0;
true
}
/// 仅保留分隔符可能跨 chunk 的最大前缀宽度,其余原始字节立即释放给下游。
fn flush_opaque_prefix(&mut self, frames: &mut Vec<SseFrame>) {
const SEPARATOR_CROSS_CHUNK_BYTES: usize = 3;
let emit_len = self
.pending
.len()
.saturating_sub(SEPARATOR_CROSS_CHUNK_BYTES);
if emit_len == 0 {
return;
}
let tail = self.pending.split_off(emit_len);
let raw = std::mem::replace(&mut self.pending, tail);
self.scanned = 0;
frames.push(SseFrame::new(raw, Vec::new()));
}
}
@@ -366,12 +413,12 @@ pub fn encode_sse_event_with_metadata(
retry: Option<u64>,
) -> String {
let mut frame = String::new();
if let Some(id) = id {
if let Some(id) = id.filter(|id| sse_id_is_representable(id)) {
frame.push_str("id: ");
frame.push_str(id);
frame.push('\n');
}
if !event.is_empty() {
if !event.is_empty() && sse_field_value_is_single_line(event) {
frame.push_str("event: ");
frame.push_str(event);
frame.push('\n');
@@ -390,6 +437,14 @@ pub fn encode_sse_event_with_metadata(
frame
}
fn sse_field_value_is_single_line(value: &str) -> bool {
!value.contains(['\r', '\n'])
}
fn sse_id_is_representable(value: &str) -> bool {
sse_field_value_is_single_line(value) && !value.contains('\0')
}
fn track_event_buffer_bytes(current_bytes: &mut usize, line: &str) -> Result<(), SseError> {
let line_separator_bytes = usize::from(*current_bytes != 0);
*current_bytes = current_bytes
@@ -0,0 +1,61 @@
use gateway_protocol::openai::codex_responses_request_semantics;
use serde_json::{Map, Value, json};
#[test]
fn codex_semantics_should_use_protocol_context_before_body_metadata() {
let body = json!({
"model": "grok-4.5",
"reasoning": {"effort": " xhigh "},
"client_metadata": {
"x-codex-turn-metadata": "{\"request_kind\":\"review\"}"
},
"input": [
{"type": "message", "role": "user", "content": "history"},
{"type": "compaction_trigger"}
]
});
let body = object(body);
let context = Map::from_iter([(
"turn_metadata".to_owned(),
json!("{\"request_kind\":\"compaction\",\"subagent_kind\":\"compact\"}"),
)]);
let semantics = codex_responses_request_semantics(&body, &context);
assert_eq!(semantics.reasoning_effort.as_deref(), Some("xhigh"));
assert_eq!(semantics.request_kind.as_deref(), Some("compaction"));
assert_eq!(semantics.subagent_kind.as_deref(), Some("compact"));
assert_eq!(semantics.reasoning_preset, None);
assert!(semantics.compact);
}
#[test]
fn codex_semantics_should_detect_ultra_preset_from_proactive_multi_agent() {
let body = object(json!({
"model": "gpt-test",
"reasoning": {"effort": "max"},
"input": [
{
"type": "message",
"role": "developer",
"content": [{
"type": "input_text",
"text": "<multi_agent_mode>Proactive multi-agent delegation is active.</multi_agent_mode>"
}]
}
]
}));
let semantics = codex_responses_request_semantics(&body, &Map::new());
assert_eq!(semantics.reasoning_effort.as_deref(), Some("max"));
assert_eq!(semantics.reasoning_preset, Some("ultra"));
assert!(!semantics.compact);
}
fn object(value: Value) -> Map<String, Value> {
let Value::Object(object) = value else {
panic!("test value must be an object");
};
object
}
@@ -226,6 +226,32 @@ fn retry_after_should_read_structured_response_delay() {
assert_eq!(retry_after_seconds_from_body(&body), Some(45));
}
#[test]
fn retry_after_should_read_string_reset_delay() {
let body = json!({
"error": {"type": "usage_limit_reached", "resets_in_seconds": "46"}
})
.to_string();
assert_eq!(retry_after_seconds_from_body(&body), Some(46));
}
#[test]
fn retry_after_should_read_string_reset_timestamp() {
let reset_at = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.expect("system clock")
.as_secs()
+ 60;
let body = json!({
"error": {"type": "usage_limit_reached", "resets_at": reset_at.to_string()}
})
.to_string();
let retry_after = retry_after_seconds_from_body(&body).expect("retry-after");
assert!((59..=60).contains(&retry_after));
}
#[test]
fn retry_after_should_read_nested_error_delay() {
let body = json!({
@@ -414,6 +440,8 @@ fn parse_rate_limits_event_should_extract_websocket_metered_limit() {
"metered_limit_name": "codex-other",
"limit_name": "Other requests",
"rate_limits": {
"allowed": false,
"limit_reached": true,
"primary": {
"used_percent": 33,
"window_minutes": 60,
@@ -424,7 +452,8 @@ fn parse_rate_limits_event_should_extract_websocket_metered_limit() {
"has_credits": true,
"unlimited": false,
"balance": "9.25"
}
},
"rate_limit_reached_type": "usage_limit_reached"
});
let parsed = parse_rate_limits_event(&event).expect("rate-limit event");
@@ -434,29 +463,39 @@ fn parse_rate_limits_event_should_extract_websocket_metered_limit() {
(
parsed.active_limit.as_deref(),
details.limit_name.as_deref(),
details.allowed,
details.limit_reached,
details.primary.expect("primary window").window_minutes,
parsed.plan_type.as_deref(),
parsed.credits.expect("credits").balance,
parsed.rate_limit_reached_type.as_deref(),
),
(
Some("codex_other"),
Some("Other requests"),
Some(false),
Some(true),
Some(60),
Some("team"),
Some("9.25".to_owned()),
Some("usage_limit_reached"),
)
);
}
#[test]
fn parse_rate_limits_event_should_reject_unrelated_event_type() {
assert_eq!(
parse_rate_limits_event(&json!({
for event in [
json!({
"type": "response.completed",
"rate_limits": {"primary": {"used_percent": 100}}
})),
None
);
}),
json!({
"rate_limits": {"primary": {"used_percent": 100}}
}),
] {
assert_eq!(parse_rate_limits_event(&event), None);
}
}
#[test]
@@ -468,9 +507,15 @@ fn rate_limit_header_round_trip_should_preserve_wire_facts() {
"300".to_owned(),
),
("x-codex-active-limit".to_owned(), "codex".to_owned()),
("x-codex-allowed".to_owned(), "false".to_owned()),
("x-codex-limit-reached".to_owned(), "true".to_owned()),
("x-codex-credits-has-credits".to_owned(), "true".to_owned()),
("x-codex-credits-unlimited".to_owned(), "false".to_owned()),
("x-codex-plan-type".to_owned(), "plus".to_owned()),
(
"x-codex-rate-limit-reached-type".to_owned(),
"usage_limit_reached".to_owned(),
),
];
let parsed = parse_rate_limit_headers(&source).expect("initial parse");
let encoded = rate_limits_to_header_pairs(&parsed);
@@ -1,2 +1,3 @@
mod codex;
mod events;
mod sse;
@@ -135,13 +135,8 @@ fn raw_frame_decoder_should_preserve_exact_frame_across_chunk_boundaries() {
let raw = b": keep-alive\r\nid: evt_1\r\nevent: response.future\r\nretry: 250\r\ndata: { \"type\": \"response.future\", \"opaque\": true }\r\n\r\n";
let mut decoder = SseEventDecoder::default();
assert!(
decoder
.push_frames(&raw[..37])
.expect("partial raw frame")
.is_empty()
);
let frames = decoder.push_frames(&raw[37..]).expect("complete raw frame");
assert!(decoder.push_frames(&raw[..37]).is_empty());
let frames = decoder.push_frames(&raw[37..]);
assert_eq!(frames.len(), 1);
assert_eq!(frames[0].raw(), raw);
@@ -161,15 +156,11 @@ fn raw_frame_decoder_should_preserve_unparseable_frames_for_transparent_transpor
let non_utf8 = b"data: \xff\n\n";
let mut decoder = SseEventDecoder::default();
let frames = decoder
.push_frames(invalid_retry)
.expect("raw frame remains deliverable");
let frames = decoder.push_frames(invalid_retry);
assert_eq!(frames[0].raw(), invalid_retry);
assert!(frames[0].events().is_empty());
let frames = decoder
.push_frames(non_utf8)
.expect("non-UTF-8 raw frame remains deliverable");
let frames = decoder.push_frames(non_utf8);
assert_eq!(frames[0].raw(), non_utf8);
assert!(frames[0].events().is_empty());
}
@@ -179,15 +170,8 @@ fn raw_frame_decoder_finish_should_preserve_unparseable_trailing_frame() {
let raw = b"retry: later\ndata: opaque";
let mut decoder = SseEventDecoder::default();
assert!(
decoder
.push_frames(raw)
.expect("unfinished raw frame remains buffered")
.is_empty()
);
let frames = decoder
.finish_frames()
.expect("trailing raw frame remains deliverable");
assert!(decoder.push_frames(raw).is_empty());
let frames = decoder.finish_frames();
assert_eq!(frames[0].raw(), raw);
assert!(frames[0].events().is_empty());
@@ -232,6 +216,32 @@ fn incremental_decoder_should_accept_exact_limit_and_reject_one_byte_over() {
);
}
#[test]
fn raw_frame_decoder_should_stream_oversized_unfinished_frame_and_resume_observation() {
let oversized = vec![b'x'; MAX_SSE_EVENT_BUFFER_BYTES + 1];
let mut decoder = SseEventDecoder::default();
let mut raw = decoder
.push_frames(&oversized)
.into_iter()
.flat_map(|frame| frame.into_parts().0)
.collect::<Vec<_>>();
assert!(!raw.is_empty());
let tail = decoder.push_frames(b"\n\n");
assert!(tail.iter().all(|frame| frame.events().is_empty()));
raw.extend(tail.into_iter().flat_map(|frame| frame.into_parts().0));
assert_eq!(raw.len(), oversized.len() + 2);
assert_eq!(&raw[..oversized.len()], oversized.as_slice());
assert_eq!(&raw[oversized.len()..], b"\n\n");
let observed = decoder.push_frames(b"event: response.future\ndata: {\"ok\":true}\n\n");
assert_eq!(observed.len(), 1);
assert_eq!(
observed[0].events()[0].event.as_deref(),
Some("response.future")
);
}
#[test]
fn encoder_should_round_trip_multiline_data() {
let frame = encode_sse_event("response.output_text.delta", "first\nsecond");
@@ -267,6 +277,20 @@ fn encoder_should_round_trip_sse_id_and_retry() {
);
}
#[test]
fn encoder_should_omit_metadata_that_sse_cannot_represent_as_one_field() {
let frame = encode_sse_event_with_metadata(
"future\nevent",
r#"{"type":"future\nevent"}"#,
Some("id\0with\nline"),
None,
);
assert!(!frame.contains("event:"));
assert!(!frame.contains("id:"));
assert!(frame.contains(r#"data: {"type":"future\nevent"}"#));
}
#[test]
fn response_failed_event_should_match_openai_failure_shape() {
let frame = response_failed_sse_event("server_error", "stream_disconnected", "closed early");
+1 -1
View File
@@ -20,7 +20,7 @@ gateway-core = { path = "../gateway-core" }
hex = "0.4.3"
redis = { version = "1.3.0", default-features = false, features = ["tokio-comp", "connection-manager", "script"] }
serde = { version = "1.0.228", features = ["derive"] }
serde_json = { version = "1.0.150", features = ["preserve_order"] }
serde_json = { version = "1.0.150", features = ["arbitrary_precision", "preserve_order"] }
sha2 = "0.11.0"
sqlx = { version = "0.9.0", default-features = false, features = ["runtime-tokio", "tls-rustls", "postgres", "chrono", "json", "macros", "migrate"] }
thiserror = "2.0.18"
+78 -33
View File
@@ -21,7 +21,7 @@ use gateway_core::CoreStorePorts;
use gateway_core::health::{HealthProbe, HealthState};
use gateway_core::provider_ports::ProviderStorePorts;
use gateway_core::task::{
ScheduledTask, WorkerContribution, WorkerCycleContext, WorkerDisabledReason, WorkerId,
DaemonRestartPolicy, ScheduledTask, WorkerContribution, WorkerCycleContext, WorkerId,
WorkerKind, WorkerLeaderLeasePort, WorkerLeaseRequest, WorkerRegistration, WorkerRunnable,
WorkerSchedule, WorkerTaskError,
};
@@ -318,28 +318,39 @@ pub async fn initialize(mut config: StoreConfig) -> StoreResult<StoreBundle> {
}),
);
let execution = Arc::new(postgres::PgExecutionStore::new(pool.clone()));
let execution_repository = Arc::new(postgres::PgExecutionStore::new(pool.clone()));
let (execution, execution_writer) =
postgres::BufferedExecutionStore::new(Arc::clone(&execution_repository));
let execution = Arc::new(execution);
let retention = Arc::new(postgres::PgRetentionRepository::new(pool.clone()));
let core_ports = CoreStorePorts::new(
execution.clone(),
(
Arc::new(redis::RedisClientAdmissionRepository::new(
redis_connection.clone(),
REDIS_NAMESPACE,
)?),
Arc::new(postgres::PgClientAdmissionRecoveryRepository::new(
pool.clone(),
)),
),
let admissions: Arc<dyn gateway_core::engine::admission::ClientAdmissionPort> = Arc::new(
redis::RedisClientAdmissionRepository::new(redis_connection.clone(), REDIS_NAMESPACE)?,
);
let circuits: Arc<dyn gateway_core::engine::execution::ProviderCircuitPort> =
Arc::new(redis::RedisProviderCircuitRepository::new(
redis_connection.clone(),
REDIS_NAMESPACE,
redis::ProviderCircuitPolicy::default(),
)?),
)?);
// Continuation affinity 是下一轮请求的路由事实,Core 必须直接等待 Redis 确认。
let continuation: Arc<dyn gateway_core::engine::continuation::NativeContinuationPort> =
Arc::new(redis::RedisNativeContinuationRepository::new(
redis_connection.clone(),
REDIS_NAMESPACE,
)?),
)?);
let (admissions, admission_release_writer) =
redis::BufferedClientAdmissionPort::new(admissions);
let (circuits, circuit_feedback_writer) = redis::BufferedProviderCircuitPort::new(circuits);
let core_ports = CoreStorePorts::new(
execution,
(
Arc::new(admissions),
Arc::new(postgres::PgClientAdmissionRecoveryRepository::new(
pool.clone(),
)),
),
Arc::new(circuits),
continuation,
(
Arc::new(postgres::PgRuntimeSnapshotRepository::new(pool.clone())),
Arc::new(redis::RedisRuntimeChangeRepository::new(
@@ -373,7 +384,13 @@ pub async fn initialize(mut config: StoreConfig) -> StoreResult<StoreBundle> {
connection: redis_connection,
}),
];
let worker_contributions = store_worker_contributions(execution, retention)?;
let worker_contributions = store_worker_contributions(
execution_repository,
execution_writer,
admission_release_writer,
circuit_feedback_writer,
retention,
)?;
Ok(StoreBundle {
admin_ports,
core_ports,
@@ -386,12 +403,24 @@ pub async fn initialize(mut config: StoreConfig) -> StoreResult<StoreBundle> {
fn store_worker_contributions(
execution: Arc<postgres::PgExecutionStore>,
execution_writer: postgres::ExecutionObservationWriter<postgres::PgExecutionStore>,
admission_release_writer: redis::ClientAdmissionReleaseWriter,
circuit_feedback_writer: redis::ProviderCircuitFeedbackWriter,
retention: Arc<postgres::PgRetentionRepository>,
) -> StoreResult<Vec<WorkerContribution>> {
let stale_id = WorkerId::try_new(WorkerKind::StaleModelRequestRecovery, "postgres")
.map_err(worker_definition_error)?;
let retention_id =
WorkerId::try_new(WorkerKind::Retention, "postgres").map_err(worker_definition_error)?;
let ops_flush_id =
WorkerId::try_new(WorkerKind::OpsFlush, "postgres").map_err(worker_definition_error)?;
let admission_flush_id = WorkerId::try_new(WorkerKind::OpsFlush, "redis_admission")
.map_err(worker_definition_error)?;
let circuit_flush_id = WorkerId::try_new(WorkerKind::OpsFlush, "redis_circuit")
.map_err(worker_definition_error)?;
let ops_flush_restart =
DaemonRestartPolicy::try_new(Duration::from_secs(1), Duration::from_secs(60))
.map_err(worker_definition_error)?;
Ok(vec![
WorkerContribution::Registration(scheduled_worker(
stale_id,
@@ -403,10 +432,36 @@ fn store_worker_contributions(
Duration::from_secs(60 * 60),
Box::new(RetentionTask { retention }),
)?),
WorkerContribution::Disabled {
kind: WorkerKind::OpsFlush,
reason: WorkerDisabledReason::NoBufferedOpsEvents,
},
WorkerContribution::Registration(
WorkerRegistration::try_new(
ops_flush_id,
WorkerRunnable::Daemon {
restart: ops_flush_restart,
task: Box::new(execution_writer),
},
)
.map_err(worker_definition_error)?,
),
WorkerContribution::Registration(
WorkerRegistration::try_new(
admission_flush_id,
WorkerRunnable::Daemon {
restart: ops_flush_restart,
task: Box::new(admission_release_writer),
},
)
.map_err(worker_definition_error)?,
),
WorkerContribution::Registration(
WorkerRegistration::try_new(
circuit_flush_id,
WorkerRunnable::Daemon {
restart: ops_flush_restart,
task: Box::new(circuit_feedback_writer),
},
)
.map_err(worker_definition_error)?,
),
])
}
@@ -582,7 +637,6 @@ impl SettingsStore for AdminSettingsStoreAdapter {
};
let snapshot = postgres::ControlPlaneRepository::replace_control_plane(
&self.control_plane,
store_revision(command.expected_config_revision)?,
replacement,
)
.await
@@ -592,32 +646,24 @@ impl SettingsStore for AdminSettingsStoreAdapter {
async fn replace_admin_api_key(
&self,
expected_config_revision: AdminRevision,
key: AdminApiKey,
context: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation> {
self.replace_admin_api_key_value(
expected_config_revision,
Some(key.expose_for_auth().to_owned()),
context,
)
.await
self.replace_admin_api_key_value(Some(key.expose_for_auth().to_owned()), context)
.await
}
async fn delete_admin_api_key(
&self,
expected_config_revision: AdminRevision,
context: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation> {
self.replace_admin_api_key_value(expected_config_revision, None, context)
.await
self.replace_admin_api_key_value(None, context).await
}
}
impl AdminSettingsStoreAdapter {
async fn replace_admin_api_key_value(
&self,
expected_config_revision: AdminRevision,
admin_api_key: Option<String>,
context: &MutationContext,
) -> AdminStoreResult<AdminApiKeyMutation> {
@@ -639,7 +685,6 @@ impl AdminSettingsStoreAdapter {
};
let snapshot = postgres::ControlPlaneRepository::replace_control_plane(
&self.control_plane,
store_revision(expected_config_revision)?,
postgres::ControlPlaneReplacement {
settings: update,
audit: mutation_audit(
@@ -36,7 +36,7 @@ use sqlx::{PgPool, Postgres, QueryBuilder, Transaction};
use crate::{
StoreError, StoreResult, admin_revision, admin_store_error, mutation_audit,
postgres_unavailable, require_nonempty, store_revision,
postgres_unavailable, require_nonempty,
};
use super::{ControlPlaneRepository, PgControlPlaneRepository};
@@ -633,7 +633,6 @@ impl ClientKeyStore for PgAdminClientKeyStore {
let revision = self
.control_plane
.create_client_api_key(
store_revision(command.expected_config_revision)?,
NewClientApiKey {
id: id.as_str().to_owned(),
name: command.name,
@@ -676,7 +675,6 @@ impl ClientKeyStore for PgAdminClientKeyStore {
let revision = self
.control_plane
.update_client_api_key(
store_revision(command.expected_config_revision)?,
UpdateClientApiKeyDetails {
id: id.as_str().to_owned(),
name: command.name,
@@ -716,7 +714,6 @@ impl ClientKeyStore for PgAdminClientKeyStore {
let revision = self
.control_plane
.set_client_api_key_enabled(
store_revision(command.expected_config_revision)?,
id.as_str(),
command.enabled,
mutation_audit(
@@ -739,7 +736,6 @@ impl ClientKeyStore for PgAdminClientKeyStore {
) -> AdminStoreResult<gateway_admin::model::Revision> {
self.control_plane
.delete_client_api_key(
store_revision(command.expected_config_revision)?,
command.id.as_str(),
mutation_audit(
context,
@@ -224,6 +224,7 @@ pub struct ModelRequestFinalization {
pub upstream_transport: Option<String>,
pub http_version: Option<String>,
pub websocket_pool: Option<String>,
pub service_tier: Option<String>,
pub provider_metadata_json: Option<Value>,
pub error_kind: Option<String>,
pub provider_error_code: Option<String>,
@@ -276,6 +277,12 @@ impl ModelRequestFinalization {
{
return Err(invalid("websocket pool must be new or reuse"));
}
if let Some(service_tier) = self.service_tier.as_deref() {
require_nonempty(ENTITY, "service_tier", service_tier)?;
if service_tier.len() > 64 || service_tier.chars().any(char::is_control) {
return Err(invalid("service tier is invalid"));
}
}
if self
.provider_metadata_json
.as_ref()
@@ -573,7 +580,7 @@ impl ModelRequestRepository for PgExecutionStore {
latency_ms = $35, completed_at = $36,
upstream_transport = coalesce($37, upstream_transport),
http_version = coalesce($38, http_version), websocket_pool = $39,
provider_observation_json = $40
service_tier = $40, provider_observation_json = $41
where id = $1 and outcome = 'running'",
)
.bind(&finalization.model_request_id)
@@ -586,9 +593,9 @@ impl ModelRequestRepository for PgExecutionStore {
.bind(finalization.downstream_committed_at)
.bind(finalization.client_status_code.map(i32::from))
.bind(finalization.upstream_status_code.map(i32::from))
.bind(finalization.client_response_id)
.bind(finalization.client_response_id.map(String::into_bytes))
.bind(finalization.upstream_request_id)
.bind(finalization.upstream_response_id)
.bind(finalization.upstream_response_id.map(String::into_bytes))
.bind(finalization.error_kind)
.bind(finalization.provider_error_code)
.bind(finalization.error_message)
@@ -660,6 +667,7 @@ impl ModelRequestRepository for PgExecutionStore {
.bind(finalization.upstream_transport)
.bind(finalization.http_version)
.bind(finalization.websocket_pool)
.bind(finalization.service_tier)
.bind(finalization.provider_metadata_json.map(sqlx::types::Json))
.execute(&self.pool)
.await
@@ -831,18 +839,6 @@ impl ExecutionStore for PgExecutionStore {
let latency_ms = u64::try_from(failure.latency.as_millis())
.map_err(|_| CoreStoreError::new(CoreStoreErrorKind::InvalidData))?;
let provider_error_code = error.upstream_code().map(|code| code.as_str().to_owned());
tracing::warn!(
target: "gateway_probe",
provider_kind = failure.provider_kind.as_str(),
account_id = failure.account_id.as_str(),
upstream_model = failure.upstream_model_id.as_str(),
failure_kind = error.kind().as_str(),
send_state = ?error.send_state(),
upstream_status = ?error.upstream_status(),
provider_error_code = ?provider_error_code,
latency_ms,
"账号连接测试失败"
);
super::OpsEventRepository::append_ops_event(
&super::PgOpsEventRepository::new(self.pool.clone()),
super::OpsEvent {
@@ -927,6 +923,7 @@ impl ExecutionStore for PgExecutionStore {
upstream_transport: finalization.upstream_transport,
http_version: finalization.http_version,
websocket_pool: finalization.websocket_pool,
service_tier: finalization.service_tier,
provider_metadata_json,
error_kind,
provider_error_code: finalization.provider_error_code,
@@ -0,0 +1,699 @@
//! 数据面执行观测的非阻塞 PostgreSQL 写入队列。
use std::mem::size_of;
use std::num::NonZeroUsize;
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, AtomicUsize, Ordering};
use std::time::{Duration, Instant, SystemTime};
use async_trait::async_trait;
use gateway_core::engine::{
AttemptRecord, CancellationToken, ExecutionStore, IntermediateFailure,
ModelRequestFinalization, ModelRequestId, NewModelRequest, ProbeFailure, RecoveryReport,
UpstreamSendState,
};
use gateway_core::error::{ProviderError, StoreError};
use gateway_core::task::{DaemonTask, WorkerTaskError};
use tokio::sync::{Mutex, mpsc};
const DEFAULT_QUEUE_CAPACITY: usize = 4_096;
const DEFAULT_QUEUE_BYTE_CAPACITY: usize = 64 * 1024 * 1024;
const SHUTDOWN_DRAIN_TIMEOUT: Duration = Duration::from_secs(2);
/// 执行观测缓冲区的进程内累计状态。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct ExecutionBufferStats {
/// 尚未完成落库的队列项和当前写入项数量。
pub queued_items: usize,
/// 尚未完成落库的观测对象估算字节数。
pub queued_bytes: usize,
/// 进程启动后成功入队的累计数量。
pub enqueued_total: u64,
/// 因队列、字节预算或关闭排空超时丢弃的累计数量。
pub dropped_total: u64,
/// 已成功写入底层 Store 的累计数量。
pub persisted_total: u64,
/// 底层 Store 返回失败的累计数量。
pub write_failure_total: u64,
}
struct ExecutionBufferState {
maximum_queued_bytes: usize,
queued_items: AtomicUsize,
queued_bytes: AtomicUsize,
enqueued_total: AtomicU64,
dropped_total: AtomicU64,
persisted_total: AtomicU64,
write_failure_total: AtomicU64,
}
impl ExecutionBufferState {
fn new(maximum_queued_bytes: NonZeroUsize) -> Self {
Self {
maximum_queued_bytes: maximum_queued_bytes.get(),
queued_items: AtomicUsize::new(0),
queued_bytes: AtomicUsize::new(0),
enqueued_total: AtomicU64::new(0),
dropped_total: AtomicU64::new(0),
persisted_total: AtomicU64::new(0),
write_failure_total: AtomicU64::new(0),
}
}
fn reserve(&self, bytes: usize) -> bool {
let reserved = self
.queued_bytes
.fetch_update(Ordering::AcqRel, Ordering::Acquire, |current| {
current
.checked_add(bytes)
.filter(|next| *next <= self.maximum_queued_bytes)
})
.is_ok();
if reserved {
self.queued_items.fetch_add(1, Ordering::Relaxed);
}
reserved
}
fn release(&self, bytes: usize) {
self.queued_bytes.fetch_sub(bytes, Ordering::AcqRel);
self.queued_items.fetch_sub(1, Ordering::AcqRel);
}
fn record_enqueued(&self) {
saturating_increment(&self.enqueued_total, 1);
}
fn record_dropped(&self, count: usize) {
saturating_increment(
&self.dropped_total,
u64::try_from(count).unwrap_or(u64::MAX),
);
}
fn record_persisted(&self) {
saturating_increment(&self.persisted_total, 1);
}
fn record_write_failure(&self) {
saturating_increment(&self.write_failure_total, 1);
}
fn snapshot(&self) -> ExecutionBufferStats {
ExecutionBufferStats {
queued_items: self.queued_items.load(Ordering::Acquire),
queued_bytes: self.queued_bytes.load(Ordering::Acquire),
enqueued_total: self.enqueued_total.load(Ordering::Acquire),
dropped_total: self.dropped_total.load(Ordering::Acquire),
persisted_total: self.persisted_total.load(Ordering::Acquire),
write_failure_total: self.write_failure_total.load(Ordering::Acquire),
}
}
}
fn saturating_increment(counter: &AtomicU64, increment: u64) {
let _ = counter.fetch_update(Ordering::Relaxed, Ordering::Relaxed, |current| {
Some(current.saturating_add(increment))
});
}
/// 将数据面观测写入转换为有界、非阻塞的进程内命令。
///
/// 队列满、worker 尚未启动或已经退出时只丢弃观测并记录告警;协议数据面不会
/// 等待 PostgreSQL,也不会看到 Store 错误。启动恢复仍直接访问底层 Store。
pub struct BufferedExecutionStore<S: ?Sized> {
inner: Arc<S>,
sender: mpsc::Sender<QueuedExecutionObservation>,
state: Arc<ExecutionBufferState>,
}
impl<S: ?Sized> BufferedExecutionStore<S> {
#[must_use]
pub fn new(inner: Arc<S>) -> (Self, ExecutionObservationWriter<S>) {
Self::with_capacity(
inner,
NonZeroUsize::new(DEFAULT_QUEUE_CAPACITY).expect("queue capacity is non-zero"),
)
}
#[must_use]
pub fn with_capacity(
inner: Arc<S>,
capacity: NonZeroUsize,
) -> (Self, ExecutionObservationWriter<S>) {
Self::with_limits(
inner,
capacity,
NonZeroUsize::new(DEFAULT_QUEUE_BYTE_CAPACITY)
.expect("queue byte capacity is non-zero"),
)
}
#[must_use]
pub fn with_limits(
inner: Arc<S>,
capacity: NonZeroUsize,
maximum_queued_bytes: NonZeroUsize,
) -> (Self, ExecutionObservationWriter<S>) {
let (sender, receiver) = mpsc::channel(capacity.get());
let state = Arc::new(ExecutionBufferState::new(maximum_queued_bytes));
(
Self {
inner: Arc::clone(&inner),
sender,
state: Arc::clone(&state),
},
ExecutionObservationWriter {
inner,
receiver: Mutex::new(receiver),
state,
},
)
}
#[must_use]
pub fn stats(&self) -> ExecutionBufferStats {
self.state.snapshot()
}
fn enqueue(&self, write: ExecutionObservationWrite) {
let estimated_bytes = write
.estimated_bytes()
.saturating_add(write.request_id().map_or(0, str::len))
.max(1);
if !self.state.reserve(estimated_bytes) {
self.state.record_dropped(1);
let stats = self.state.snapshot();
tracing::warn!(
operation = write.operation(),
request_id = ?write.request_id(),
reason = "byte_capacity",
estimated_bytes,
maximum_queued_bytes = self.state.maximum_queued_bytes,
dropped_total = stats.dropped_total,
"执行观测队列字节预算不足,已丢弃本次写入"
);
return;
}
let queued =
QueuedExecutionObservation::new(write, estimated_bytes, Arc::clone(&self.state));
match self.sender.try_send(queued) {
Ok(()) => self.state.record_enqueued(),
Err(error) => {
let (reason, queued) = match error {
mpsc::error::TrySendError::Full(queued) => ("full", queued),
mpsc::error::TrySendError::Closed(queued) => ("closed", queued),
};
let operation = queued.operation();
let request_id = queued.request_id().map(ToOwned::to_owned);
drop(queued);
self.state.record_dropped(1);
let stats = self.state.snapshot();
tracing::warn!(
operation,
request_id = ?request_id,
reason,
queued_items = stats.queued_items,
queued_bytes = stats.queued_bytes,
dropped_total = stats.dropped_total,
"执行观测队列不可用,已丢弃本次写入"
);
}
}
}
}
#[async_trait]
impl<S> ExecutionStore for BufferedExecutionStore<S>
where
S: ExecutionStore + ?Sized,
{
async fn create_model_request(&self, request: NewModelRequest) -> Result<(), StoreError> {
self.enqueue(ExecutionObservationWrite::Create(Box::new(request)));
Ok(())
}
async fn record_attempt(&self, attempt: AttemptRecord) -> Result<(), StoreError> {
self.enqueue(ExecutionObservationWrite::Attempt(Box::new(attempt)));
Ok(())
}
async fn create_model_request_with_attempt(
&self,
request: NewModelRequest,
attempt: AttemptRecord,
) -> Result<(), StoreError> {
self.enqueue(ExecutionObservationWrite::CreateWithAttempt(Box::new((
request, attempt,
))));
Ok(())
}
async fn mark_send_state(
&self,
request_id: &ModelRequestId,
state: UpstreamSendState,
) -> Result<(), StoreError> {
self.enqueue(ExecutionObservationWrite::MarkSendState {
request_id: request_id.clone(),
state,
});
Ok(())
}
async fn mark_downstream_committed(
&self,
request_id: &ModelRequestId,
committed_at: SystemTime,
client_status_code: Option<u16>,
) -> Result<(), StoreError> {
self.enqueue(ExecutionObservationWrite::MarkDownstreamCommitted {
request_id: request_id.clone(),
committed_at,
client_status_code,
});
Ok(())
}
async fn record_client_status(
&self,
request_id: &ModelRequestId,
client_status_code: u16,
) -> Result<(), StoreError> {
self.enqueue(ExecutionObservationWrite::RecordClientStatus {
request_id: request_id.clone(),
client_status_code,
});
Ok(())
}
async fn record_intermediate_failure(
&self,
failure: IntermediateFailure,
) -> Result<(), StoreError> {
self.enqueue(ExecutionObservationWrite::IntermediateFailure(Box::new(
failure,
)));
Ok(())
}
async fn record_probe_failure(&self, failure: ProbeFailure) -> Result<(), StoreError> {
self.enqueue(ExecutionObservationWrite::ProbeFailure(Box::new(failure)));
Ok(())
}
async fn finalize_model_request(
&self,
finalization: ModelRequestFinalization,
) -> Result<(), StoreError> {
self.enqueue(ExecutionObservationWrite::Finalize(Box::new(finalization)));
Ok(())
}
async fn recover_expired(&self, now: SystemTime) -> Result<RecoveryReport, StoreError> {
self.inner.recover_expired(now).await
}
}
/// 由 Host 监督的单消费者,保证同一请求的观测命令按入队顺序落库。
pub struct ExecutionObservationWriter<S: ?Sized> {
inner: Arc<S>,
receiver: Mutex<mpsc::Receiver<QueuedExecutionObservation>>,
state: Arc<ExecutionBufferState>,
}
impl<S: ?Sized> ExecutionObservationWriter<S> {
#[must_use]
pub fn stats(&self) -> ExecutionBufferStats {
self.state.snapshot()
}
}
impl<S> DaemonTask for ExecutionObservationWriter<S>
where
S: ExecutionStore + ?Sized,
{
fn run(
&self,
cancellation: CancellationToken,
) -> futures::future::BoxFuture<'_, Result<(), WorkerTaskError>> {
Box::pin(async move {
let mut receiver = self.receiver.lock().await;
loop {
let queued = tokio::select! {
() = cancellation.cancelled() => {
drain_on_shutdown(
&mut receiver,
self.inner.as_ref(),
self.state.as_ref(),
).await;
return Ok(());
},
queued = receiver.recv() => queued,
};
let Some(queued) = queued else {
return Err(WorkerTaskError::safe("execution observation queue closed"));
};
persist_queued(queued, self.inner.as_ref()).await;
}
})
}
}
async fn persist_queued<S>(mut queued: QueuedExecutionObservation, store: &S)
where
S: ExecutionStore + ?Sized,
{
let operation = queued.operation();
let request_id = queued.request_id().map(ToOwned::to_owned);
let state = Arc::clone(&queued.state);
let Some(write) = queued.take_write() else {
state.record_write_failure();
tracing::error!(
operation,
request_id = ?request_id,
"执行观测队列内部状态无效,已丢弃本次写入"
);
return;
};
// Store 写入没有统一幂等键;超时可能表示已提交,不能在这里盲目重试并
// 制造重复 ops_events。失败会被计数并丢弃,数据面始终不等待补偿。
match write.persist(store).await {
Ok(()) => state.record_persisted(),
Err(error) => {
state.record_write_failure();
let stats = state.snapshot();
tracing::warn!(
operation,
request_id = ?request_id,
error_kind = ?error.kind(),
write_failure_total = stats.write_failure_total,
"执行观测写入失败,数据面不受影响"
);
}
}
}
async fn drain_on_shutdown<S>(
receiver: &mut mpsc::Receiver<QueuedExecutionObservation>,
store: &S,
state: &ExecutionBufferState,
) where
S: ExecutionStore + ?Sized,
{
receiver.close();
let queued_at_shutdown = receiver.len();
let started_at = Instant::now();
let mut dropped = 0_usize;
while let Some(queued) = receiver.recv().await {
let remaining = SHUTDOWN_DRAIN_TIMEOUT.saturating_sub(started_at.elapsed());
if remaining.is_zero() {
drop(queued);
dropped = dropped.saturating_add(1);
break;
}
if tokio::time::timeout(remaining, persist_queued(queued, store))
.await
.is_err()
{
dropped = dropped.saturating_add(1);
break;
}
}
while let Ok(queued) = receiver.try_recv() {
drop(queued);
dropped = dropped.saturating_add(1);
}
if dropped > 0 {
state.record_dropped(dropped);
let stats = state.snapshot();
tracing::warn!(
queued_at_shutdown,
dropped,
dropped_total = stats.dropped_total,
drain_timeout_ms =
u64::try_from(SHUTDOWN_DRAIN_TIMEOUT.as_millis()).unwrap_or(u64::MAX),
"执行观测队列关闭排空超时,剩余写入已丢弃"
);
} else if queued_at_shutdown > 0 {
tracing::info!(queued_at_shutdown, "执行观测队列已在关闭前排空");
}
}
struct QueuedExecutionObservation {
write: Option<ExecutionObservationWrite>,
operation: &'static str,
request_id: Option<String>,
estimated_bytes: usize,
state: Arc<ExecutionBufferState>,
}
impl QueuedExecutionObservation {
fn new(
write: ExecutionObservationWrite,
estimated_bytes: usize,
state: Arc<ExecutionBufferState>,
) -> Self {
let operation = write.operation();
let request_id = write.request_id().map(ToOwned::to_owned);
Self {
write: Some(write),
operation,
request_id,
estimated_bytes,
state,
}
}
fn operation(&self) -> &'static str {
self.operation
}
fn request_id(&self) -> Option<&str> {
self.request_id.as_deref()
}
fn take_write(&mut self) -> Option<ExecutionObservationWrite> {
self.write.take()
}
}
impl Drop for QueuedExecutionObservation {
fn drop(&mut self) {
self.state.release(self.estimated_bytes);
}
}
enum ExecutionObservationWrite {
Create(Box<NewModelRequest>),
Attempt(Box<AttemptRecord>),
CreateWithAttempt(Box<(NewModelRequest, AttemptRecord)>),
MarkSendState {
request_id: ModelRequestId,
state: UpstreamSendState,
},
MarkDownstreamCommitted {
request_id: ModelRequestId,
committed_at: SystemTime,
client_status_code: Option<u16>,
},
RecordClientStatus {
request_id: ModelRequestId,
client_status_code: u16,
},
IntermediateFailure(Box<IntermediateFailure>),
ProbeFailure(Box<ProbeFailure>),
Finalize(Box<ModelRequestFinalization>),
}
impl ExecutionObservationWrite {
const fn operation(&self) -> &'static str {
match self {
Self::Create(_) => "create_model_request",
Self::Attempt(_) => "record_attempt",
Self::CreateWithAttempt(_) => "create_model_request_with_attempt",
Self::MarkSendState { .. } => "mark_send_state",
Self::MarkDownstreamCommitted { .. } => "mark_downstream_committed",
Self::RecordClientStatus { .. } => "record_client_status",
Self::IntermediateFailure(_) => "record_intermediate_failure",
Self::ProbeFailure(_) => "record_probe_failure",
Self::Finalize(_) => "finalize_model_request",
}
}
fn request_id(&self) -> Option<&str> {
match self {
Self::Create(request) => Some(request.id.as_str()),
Self::Attempt(attempt) => Some(attempt.request_id.as_str()),
Self::CreateWithAttempt(write) => Some(write.0.id.as_str()),
Self::MarkSendState { request_id, .. }
| Self::MarkDownstreamCommitted { request_id, .. }
| Self::RecordClientStatus { request_id, .. } => Some(request_id.as_str()),
Self::IntermediateFailure(failure) => Some(failure.request_id.as_str()),
Self::ProbeFailure(_) => None,
Self::Finalize(finalization) => Some(finalization.request_id.as_str()),
}
}
fn estimated_bytes(&self) -> usize {
size_of::<Self>().saturating_add(match self {
Self::Create(request) => new_request_bytes(request),
Self::Attempt(attempt) => attempt_bytes(attempt),
Self::CreateWithAttempt(write) => {
new_request_bytes(&write.0).saturating_add(attempt_bytes(&write.1))
}
Self::MarkSendState { request_id, .. }
| Self::MarkDownstreamCommitted { request_id, .. }
| Self::RecordClientStatus { request_id, .. } => request_id.as_str().len(),
Self::IntermediateFailure(failure) => text_bytes([
Some(failure.request_id.as_str()),
Some(failure.provider_kind.as_str()),
failure.account_id.as_ref().map(|value| value.as_str()),
Some(failure.upstream_model_id.as_str()),
failure.upstream_request_id.as_deref(),
])
.saturating_add(provider_error_bytes(&failure.error)),
Self::ProbeFailure(failure) => text_bytes([
Some(failure.provider_kind.as_str()),
Some(failure.account_id.as_str()),
Some(failure.upstream_model_id.as_str()),
])
.saturating_add(provider_error_bytes(&failure.error)),
Self::Finalize(finalization) => {
let error_bytes = finalization.error.as_ref().map_or(0, |error| {
text_bytes([
Some(error.client_message()),
error.client_error_code(),
error.client_error_type(),
])
});
text_bytes([
Some(finalization.request_id.as_str()),
finalization.client_response_id.as_deref(),
finalization.upstream_request_id.as_deref(),
finalization.upstream_response_id.as_deref(),
finalization.upstream_transport.as_deref(),
finalization.http_version.as_deref(),
finalization.websocket_pool.as_deref(),
finalization.service_tier.as_deref(),
finalization.provider_metadata_json.as_deref(),
finalization.provider_error_code.as_deref(),
])
.saturating_add(error_bytes)
.saturating_add(size_of::<ModelRequestFinalization>())
}
})
}
async fn persist<S>(self, store: &S) -> Result<(), StoreError>
where
S: ExecutionStore + ?Sized,
{
match self {
Self::Create(request) => store.create_model_request(*request).await,
Self::Attempt(attempt) => store.record_attempt(*attempt).await,
Self::CreateWithAttempt(write) => {
let (request, attempt) = *write;
store
.create_model_request_with_attempt(request, attempt)
.await
}
Self::MarkSendState { request_id, state } => {
store.mark_send_state(&request_id, state).await
}
Self::MarkDownstreamCommitted {
request_id,
committed_at,
client_status_code,
} => {
store
.mark_downstream_committed(&request_id, committed_at, client_status_code)
.await
}
Self::RecordClientStatus {
request_id,
client_status_code,
} => {
store
.record_client_status(&request_id, client_status_code)
.await
}
Self::IntermediateFailure(failure) => store.record_intermediate_failure(*failure).await,
Self::ProbeFailure(failure) => store.record_probe_failure(*failure).await,
Self::Finalize(finalization) => store.finalize_model_request(*finalization).await,
}
}
}
fn new_request_bytes(request: &NewModelRequest) -> usize {
size_of::<NewModelRequest>().saturating_add(text_bytes([
Some(request.id.as_str()),
request
.client_api_key_id
.as_ref()
.map(|value| value.as_str()),
Some(request.client_api_key_ref.as_str()),
Some(request.protocol.as_str()),
Some(request.endpoint.as_str()),
Some(request.client_transport.as_str()),
Some(request.requested_model.as_str()),
request.user_agent.as_deref(),
request.reasoning_effort.as_deref(),
request.reasoning_preset.as_deref(),
request.request_kind.as_deref(),
request.subagent_kind.as_deref(),
]))
}
fn attempt_bytes(attempt: &AttemptRecord) -> usize {
size_of::<AttemptRecord>().saturating_add(text_bytes([
Some(attempt.request_id.as_str()),
Some(attempt.provider_kind.as_str()),
attempt
.provider_account_id
.as_ref()
.map(|value| value.as_str()),
attempt
.provider_account_ref
.as_ref()
.map(|value| value.as_str()),
Some(attempt.upstream_model_id.as_str()),
Some(attempt.upstream_transport.as_str()),
attempt.http_version.as_deref(),
]))
}
fn provider_error_bytes(error: &ProviderError) -> usize {
let mut bytes = text_bytes([
error.upstream_code().map(|value| value.as_str()),
error.upstream_request_id().map(|value| value.as_str()),
error.upstream_response_id().map(|value| value.as_str()),
]);
if let Some(client_error) = error.client_visible_upstream_error() {
bytes = bytes.saturating_add(text_bytes([
Some(client_error.message()),
client_error.code(),
client_error.error_type(),
]));
}
if let Some(response) = error.client_visible_upstream_response() {
bytes = bytes
.saturating_add(response.body().len())
.saturating_add(response.content_type().map_or(0, <[u8]>::len));
for header in response.headers() {
bytes = bytes
.saturating_add(header.name().len())
.saturating_add(header.value().len());
}
}
bytes
}
fn text_bytes<const N: usize>(values: [Option<&str>; N]) -> usize {
values
.into_iter()
.flatten()
.fold(0, |total, value| total.saturating_add(value.len()))
}
@@ -11,6 +11,7 @@ mod admin_security_audit;
mod admission_recovery;
mod client_keys;
mod execution;
mod execution_buffer;
mod observability;
mod ops_events;
mod provider_accounts;
@@ -22,6 +23,7 @@ pub use admin_security_audit::*;
pub use admission_recovery::*;
pub use client_keys::*;
pub use execution::*;
pub use execution_buffer::*;
pub use observability::*;
pub use ops_events::*;
pub use provider_accounts::*;
@@ -74,27 +76,23 @@ pub trait ControlPlaneRepository: Send + Sync {
async fn replace_control_plane(
&self,
expected_revision: Revision,
replacement: ControlPlaneReplacement,
) -> StoreResult<ControlPlaneSnapshot>;
async fn create_client_api_key(
&self,
expected_revision: Revision,
key: NewClientApiKey,
audit: AdminAuditEvent,
) -> StoreResult<Revision>;
async fn update_client_api_key(
&self,
expected_revision: Revision,
key: UpdateClientApiKeyDetails,
audit: AdminAuditEvent,
) -> StoreResult<Revision>;
async fn set_client_api_key_enabled(
&self,
expected_revision: Revision,
id: &str,
enabled: bool,
audit: AdminAuditEvent,
@@ -102,7 +100,6 @@ pub trait ControlPlaneRepository: Send + Sync {
async fn delete_client_api_key(
&self,
expected_revision: Revision,
id: &str,
audit: AdminAuditEvent,
) -> StoreResult<Revision>;
@@ -142,7 +139,6 @@ impl ControlPlaneRepository for PgControlPlaneRepository {
async fn replace_control_plane(
&self,
expected_revision: Revision,
replacement: ControlPlaneReplacement,
) -> StoreResult<ControlPlaneSnapshot> {
replacement.settings.validate()?;
@@ -152,12 +148,9 @@ impl ControlPlaneRepository for PgControlPlaneRepository {
.await
.map_err(|_| postgres_unavailable("begin control plane replacement"))?;
let result = async {
let revision = update_runtime_settings_in_transaction(
&mut transaction,
expected_revision,
&replacement.settings,
)
.await?;
let revision =
update_runtime_settings_in_transaction(&mut transaction, &replacement.settings)
.await?;
append_admin_audit_event_in_transaction(&mut transaction, replacement.audit, revision)
.await?;
load_control_plane_in_transaction(&mut transaction).await
@@ -183,41 +176,29 @@ impl ControlPlaneRepository for PgControlPlaneRepository {
async fn create_client_api_key(
&self,
expected_revision: Revision,
key: NewClientApiKey,
audit: AdminAuditEvent,
) -> StoreResult<Revision> {
self.apply_targeted_mutation(
expected_revision,
ControlPlaneMutation::CreateClientApiKey(key),
audit,
)
.await
self.apply_targeted_mutation(ControlPlaneMutation::CreateClientApiKey(key), audit)
.await
}
async fn update_client_api_key(
&self,
expected_revision: Revision,
key: UpdateClientApiKeyDetails,
audit: AdminAuditEvent,
) -> StoreResult<Revision> {
self.apply_targeted_mutation(
expected_revision,
ControlPlaneMutation::UpdateClientApiKey(key),
audit,
)
.await
self.apply_targeted_mutation(ControlPlaneMutation::UpdateClientApiKey(key), audit)
.await
}
async fn set_client_api_key_enabled(
&self,
expected_revision: Revision,
id: &str,
enabled: bool,
audit: AdminAuditEvent,
) -> StoreResult<Revision> {
self.apply_targeted_mutation(
expected_revision,
ControlPlaneMutation::SetClientApiKeyEnabled {
id: id.to_owned(),
enabled,
@@ -229,12 +210,10 @@ impl ControlPlaneRepository for PgControlPlaneRepository {
async fn delete_client_api_key(
&self,
expected_revision: Revision,
id: &str,
audit: AdminAuditEvent,
) -> StoreResult<Revision> {
self.apply_targeted_mutation(
expected_revision,
ControlPlaneMutation::DeleteClientApiKey(id.to_owned()),
audit,
)
@@ -252,7 +231,6 @@ enum ControlPlaneMutation {
impl PgControlPlaneRepository {
async fn apply_targeted_mutation(
&self,
expected_revision: Revision,
mutation: ControlPlaneMutation,
audit: AdminAuditEvent,
) -> StoreResult<Revision> {
@@ -262,8 +240,7 @@ impl PgControlPlaneRepository {
.await
.map_err(|_| postgres_unavailable("begin targeted control plane mutation"))?;
let result = async {
let revision =
bump_config_revision_in_transaction(&mut transaction, expected_revision).await?;
let revision = bump_config_revision_in_transaction(&mut transaction).await?;
match mutation {
ControlPlaneMutation::CreateClientApiKey(key) => {
insert_client_api_key_in_transaction(&mut transaction, &key).await?;
@@ -125,7 +125,6 @@ impl UsageRecordFilter {
(self.model.as_deref(), "model filter"),
(self.outcome.as_deref(), "outcome filter"),
(self.transport.as_deref(), "transport filter"),
(self.response_id.as_deref(), "response ID filter"),
(
self.upstream_request_id.as_deref(),
"upstream request ID filter",
@@ -189,7 +188,6 @@ impl OpsErrorFilter {
(self.operation.as_deref(), "operation filter"),
(self.model.as_deref(), "model filter"),
(self.transport.as_deref(), "transport filter"),
(self.response_id.as_deref(), "response ID filter"),
(
self.upstream_request_id.as_deref(),
"upstream request ID filter",
@@ -371,6 +369,7 @@ pub struct CalculatedUsageBillingFact {
pub bucket_start: DateTime<Utc>,
pub provider_kind: String,
pub upstream_model_id: String,
pub service_tier: Option<String>,
pub input_tokens: Option<u64>,
pub output_tokens: Option<u64>,
pub cached_tokens: Option<u64>,
@@ -523,6 +522,7 @@ pub struct UsageRecord {
pub upstream_transport: Option<String>,
pub http_version: Option<String>,
pub websocket_pool: Option<String>,
pub service_tier: Option<String>,
pub provider_metadata_json: Option<String>,
pub attempt_count: u32,
pub upstream_send_state: String,
@@ -1380,6 +1380,7 @@ fn admin_calculated_usage_billing_fact(
bucket_start: fact.bucket_start,
provider_kind: fact.provider_kind,
upstream_model_id: fact.upstream_model_id,
service_tier: fact.service_tier,
input_tokens: fact.input_tokens,
output_tokens: fact.output_tokens,
cached_tokens: fact.cached_tokens,
@@ -1455,6 +1456,7 @@ fn admin_usage_record(record: UsageRecord) -> AdminStoreResult<admin_observabili
upstream_transport: record.upstream_transport,
http_version: record.http_version,
websocket_pool: record.websocket_pool,
service_tier: record.service_tier,
provider_metadata_json: record.provider_metadata_json,
attempt_count: record.attempt_count,
upstream_send_state: record.upstream_send_state,
@@ -1814,7 +1816,7 @@ async fn calculated_usage_billing_facts(
query.push_bind(granularity.sql_interval());
query.push(
"::interval, mr.started_at, timestamptz '1970-01-01 00:00:00+00') as bucket_start,
mr.provider_kind, mr.upstream_model_id,
mr.provider_kind, mr.upstream_model_id, mr.service_tier,
mr.input_tokens, mr.output_tokens, mr.cached_tokens, mr.cache_write_tokens,
mr.cost_currency, mr.cost_amount::text as amount
from model_requests mr where mr.started_at >= ",
@@ -1990,27 +1992,35 @@ async fn provider_account_metrics(
observed_at: DateTime<Utc>,
) -> StoreResult<ProviderAccountMetrics> {
let row = sqlx::query(
"select count(*)::bigint as total,
"with normalized as (
select enabled,
case
when not enabled then 'disabled'
when availability = 'banned' then 'banned'
when availability = 'quota_exhausted'
or (
availability = 'cooldown'
and availability_reason = 'usage_limit_exhausted'
and (cooldown_until is null or cooldown_until > $1)
)
then 'quota_exhausted'
when availability in ('expired', 'invalid')
or (access_token_expires_at is not null and access_token_expires_at <= $1)
then 'expired'
else 'active'
end as status
from provider_accounts
)
select count(*)::bigint as total,
count(*) filter (where enabled)::bigint as enabled,
count(*) filter (
where not (
enabled and availability = 'ready'
and (access_token_expires_at is null or access_token_expires_at > $1)
)
)::bigint as unavailable,
count(*) filter (
where enabled and availability = 'ready'
and (access_token_expires_at is null or access_token_expires_at > $1)
)::bigint as active,
count(*) filter (
where availability = 'expired'
or (access_token_expires_at is not null and access_token_expires_at <= $1)
)::bigint as expired,
count(*) filter (where availability = 'quota_exhausted')::bigint
as quota_exhausted,
count(*) filter (where not enabled)::bigint as disabled,
count(*) filter (where availability = 'banned')::bigint as banned
from provider_accounts",
count(*) filter (where status in ('expired', 'disabled', 'banned'))::bigint
as unavailable,
count(*) filter (where status = 'active')::bigint as active,
count(*) filter (where status = 'expired')::bigint as expired,
count(*) filter (where status = 'quota_exhausted')::bigint as quota_exhausted,
count(*) filter (where status = 'disabled')::bigint as disabled,
count(*) filter (where status = 'banned')::bigint as banned
from normalized",
)
.bind(observed_at)
.fetch_one(pool)
@@ -2033,11 +2043,29 @@ async fn active_provider_account_ids(
observed_at: DateTime<Utc>,
) -> StoreResult<Vec<String>> {
sqlx::query(
"select id
from provider_accounts
where enabled and availability = 'ready'
and (access_token_expires_at is null or access_token_expires_at > $1)
order by id",
"with normalized as (
select id,
case
when not enabled then 'disabled'
when availability = 'banned' then 'banned'
when availability = 'quota_exhausted'
or (
availability = 'cooldown'
and availability_reason = 'usage_limit_exhausted'
and (cooldown_until is null or cooldown_until > $1)
)
then 'quota_exhausted'
when availability in ('expired', 'invalid')
or (access_token_expires_at is not null and access_token_expires_at <= $1)
then 'expired'
else 'active'
end as status
from provider_accounts
)
select id
from normalized
where status = 'active'
order by id",
)
.bind(observed_at)
.fetch_all(pool)
@@ -2234,9 +2262,9 @@ fn push_usage_filter(query: &mut QueryBuilder<Postgres>, filter: &UsageRecordFil
}
if let Some(value) = &filter.response_id {
query.push(format!(" and ({alias}.client_response_id = "));
query.push_bind(value.clone());
query.push_bind(value.as_bytes().to_vec());
query.push(format!(" or {alias}.upstream_response_id = "));
query.push_bind(value.clone());
query.push_bind(value.as_bytes().to_vec());
query.push(")");
}
if let Some(value) = &filter.upstream_request_id {
@@ -2248,7 +2276,7 @@ fn push_usage_filter(query: &mut QueryBuilder<Postgres>, filter: &UsageRecordFil
" and lower(concat_ws(' ', {alias}.id, {alias}.client_api_key_ref,
{alias}.requested_model_id, {alias}.provider_kind,
{alias}.provider_account_ref, {alias}.upstream_model_id,
{alias}.client_response_id, {alias}.upstream_request_id,
encode({alias}.client_response_id, 'escape'), {alias}.upstream_request_id,
{alias}.error_kind, {alias}.provider_error_code,
{alias}.error_message)) like "
));
@@ -2611,7 +2639,7 @@ const USAGE_RECORD_SELECT: &str =
pa.name as provider_account_name, pa.email as provider_account_email,
pa.authentication_kind as provider_account_authentication_kind,
mr.upstream_model_id, mr.upstream_transport, mr.http_version, mr.websocket_pool,
mr.provider_observation_json,
mr.service_tier, mr.provider_observation_json,
mr.attempt_count, mr.upstream_send_state, mr.downstream_committed_at,
mr.outcome, mr.client_status_code, mr.upstream_status_code,
mr.client_response_id, mr.upstream_request_id, mr.upstream_response_id,
@@ -3055,7 +3083,6 @@ fn push_ops_filter(statement: &mut QueryBuilder<Postgres>, filter: &OpsErrorFilt
("provider_kind", &filter.provider_kind),
("operation", &filter.operation),
("upstream_transport", &filter.transport),
("client_response_id", &filter.response_id),
("upstream_request_id", &filter.upstream_request_id),
("failure_kind", &filter.failure_kind),
] {
@@ -3064,6 +3091,10 @@ fn push_ops_filter(statement: &mut QueryBuilder<Postgres>, filter: &OpsErrorFilt
statement.push_bind(value.clone());
}
}
if let Some(value) = &filter.response_id {
statement.push(" and e.client_response_id = ");
statement.push_bind(value.as_bytes().to_vec());
}
if let Some(model) = &filter.model {
statement.push(" and e.upstream_model_id = ");
statement.push_bind(model.clone());
@@ -3106,6 +3137,7 @@ fn usage_record_from_row(row: &sqlx::postgres::PgRow) -> StoreResult<UsageRecord
upstream_transport: get(row, "upstream_transport")?,
http_version: get(row, "http_version")?,
websocket_pool: get(row, "websocket_pool")?,
service_tier: get(row, "service_tier")?,
provider_metadata_json: get::<Option<serde_json::Value>>(row, "provider_observation_json")?
.map(|value| serde_json::to_string(&value))
.transpose()
@@ -3116,9 +3148,9 @@ fn usage_record_from_row(row: &sqlx::postgres::PgRow) -> StoreResult<UsageRecord
outcome: get(row, "outcome")?,
client_status_code: optional_status(row, "client_status_code")?,
upstream_status_code: optional_status(row, "upstream_status_code")?,
client_response_id: get(row, "client_response_id")?,
client_response_id: opaque_response_id(row, "client_response_id")?,
upstream_request_id: get(row, "upstream_request_id")?,
upstream_response_id: get(row, "upstream_response_id")?,
upstream_response_id: opaque_response_id(row, "upstream_response_id")?,
error_kind: get(row, "error_kind")?,
provider_error_code: get(row, "provider_error_code")?,
error_message: get(row, "error_message")?,
@@ -3177,7 +3209,7 @@ fn ops_error_from_row(row: &sqlx::postgres::PgRow) -> StoreResult<OpsErrorRecord
client_status_code: optional_status(row, "client_status_code")?,
upstream_status_code: optional_status(row, "upstream_status_code")?,
provider_error_code: get(row, "provider_error_code")?,
client_response_id: get(row, "client_response_id")?,
client_response_id: opaque_response_id(row, "client_response_id")?,
upstream_request_id: get(row, "upstream_request_id")?,
latency_ms: optional_unsigned(row, "latency_ms")?,
message: get(row, "message")?,
@@ -3201,6 +3233,7 @@ fn calculated_usage_billing_fact_from_row(
bucket_start: get(row, "bucket_start")?,
provider_kind: get(row, "provider_kind")?,
upstream_model_id: get(row, "upstream_model_id")?,
service_tier: get(row, "service_tier")?,
input_tokens: optional_unsigned(row, "input_tokens")?,
output_tokens: optional_unsigned(row, "output_tokens")?,
cached_tokens: optional_unsigned(row, "cached_tokens")?,
@@ -3218,6 +3251,16 @@ fn optional_decimal(
.transpose()
}
fn opaque_response_id(
row: &sqlx::postgres::PgRow,
column: &'static str,
) -> StoreResult<Option<String>> {
get::<Option<Vec<u8>>>(row, column)?
.map(String::from_utf8)
.transpose()
.map_err(|_| invalid(column))
}
fn validate_account_ids(account_ids: &[String]) -> StoreResult<()> {
let mut unique = BTreeSet::new();
for account_id in account_ids {
@@ -12,8 +12,7 @@ use gateway_admin::{
model::{
MutationContext, Revision as AdminRevision,
accounts::{
AccountAvailability as AdminAccountAvailability, AccountCost,
AccountListQuery as AdminAccountListQuery, AccountModelUsage, AccountPage,
AccountCost, AccountListQuery as AdminAccountListQuery, AccountModelUsage, AccountPage,
AccountRecord, AccountRequestBucket, AccountSort as AdminAccountSort,
AccountSortField as AdminAccountSortField, AccountStatus as AdminAccountStatus,
AccountSummary, AccountUsage, AccountUsageWindowQuery, AccountUsageWindowResult,
@@ -35,12 +34,12 @@ use gateway_admin::{
use sqlx::{PgPool, Postgres, Row, Transaction};
use gateway_core::engine::credential::{
AccountAvailability as CoreAccountAvailability, AccountStateChange, CredentialCasOutcome,
CredentialCasUpdate, CredentialRevision as CoreCredentialRevision, LoadedCredential,
AccountAvailability, AccountStateChange, CredentialCasOutcome, CredentialCasUpdate,
CredentialRevision as CoreCredentialRevision, LoadedCredential,
NewProviderAccount as CoreNewProviderAccount, OpaqueProviderData, PlaintextCredential,
ProviderAccount as CoreProviderAccount, ProviderAccountId as CoreProviderAccountId,
ProviderAccountStore, ProviderAccountUpdate as CoreProviderAccountUpdate, QuotaObservation,
QuotaWriteOutcome,
ProviderAccountIdentity, ProviderAccountStore,
ProviderAccountUpdate as CoreProviderAccountUpdate, QuotaObservation, QuotaWriteOutcome,
};
use gateway_core::error::{StoreError as CoreStoreError, StoreErrorKind as CoreStoreErrorKind};
use gateway_core::routing::ProviderKind;
@@ -121,43 +120,29 @@ impl ProviderAccountAdminScope {
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ProviderAccountAvailability {
Unknown,
Ready,
Cooldown,
QuotaExhausted,
Expired,
Banned,
Invalid,
/// 解析数据库 availability 列;字符串映射本体唯一归属 gateway-core。
fn parse_availability(value: &str) -> StoreResult<AccountAvailability> {
AccountAvailability::parse(value).ok_or_else(|| invalid("unknown availability value"))
}
impl ProviderAccountAvailability {
#[must_use]
pub const fn as_str(self) -> &'static str {
match self {
Self::Unknown => "unknown",
Self::Ready => "ready",
Self::Cooldown => "cooldown",
Self::QuotaExhausted => "quota_exhausted",
Self::Expired => "expired",
Self::Banned => "banned",
Self::Invalid => "invalid",
}
fn validate_availability_cooldown(
availability: AccountAvailability,
has_cooldown_until: bool,
) -> StoreResult<()> {
if availability == AccountAvailability::Cooldown && !has_cooldown_until {
return Err(invalid("cooldown availability requires cooldown_until"));
}
fn parse(value: &str) -> StoreResult<Self> {
match value {
"unknown" => Ok(Self::Unknown),
"ready" => Ok(Self::Ready),
"cooldown" => Ok(Self::Cooldown),
"quota_exhausted" => Ok(Self::QuotaExhausted),
"expired" => Ok(Self::Expired),
"banned" => Ok(Self::Banned),
"invalid" => Ok(Self::Invalid),
_ => Err(invalid("unknown availability value")),
}
if has_cooldown_until
&& !matches!(
availability,
AccountAvailability::Cooldown | AccountAvailability::QuotaExhausted
)
{
return Err(invalid(
"cooldown_until is only allowed for cooldown or quota_exhausted availability",
));
}
Ok(())
}
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -175,7 +160,7 @@ pub struct ProviderAccountSummary {
pub access_token_expires_at: Option<DateTime<Utc>>,
pub next_refresh_at: Option<DateTime<Utc>>,
pub enabled: bool,
pub availability: ProviderAccountAvailability,
pub availability: AccountAvailability,
pub availability_reason: Option<String>,
pub cooldown_until: Option<DateTime<Utc>>,
pub availability_observed_at: DateTime<Utc>,
@@ -220,7 +205,7 @@ pub struct NewProviderAccount {
pub access_token_expires_at: Option<DateTime<Utc>>,
pub next_refresh_at: Option<DateTime<Utc>>,
pub enabled: bool,
pub availability: ProviderAccountAvailability,
pub availability: AccountAvailability,
pub availability_reason: Option<String>,
pub cooldown_until: Option<DateTime<Utc>>,
pub availability_observed_at: DateTime<Utc>,
@@ -259,13 +244,7 @@ impl NewProviderAccount {
if !self.has_refresh_token && self.next_refresh_at.is_some() {
return Err(invalid("next_refresh_at requires a refresh token"));
}
if (self.availability == ProviderAccountAvailability::Cooldown)
!= self.cooldown_until.is_some()
{
return Err(invalid(
"cooldown_until must be present exactly for cooldown availability",
));
}
validate_availability_cooldown(self.availability, self.cooldown_until.is_some())?;
validate_object_size(
"provider_credentials_json",
&self.provider_credentials_json,
@@ -333,6 +312,7 @@ impl ImportProviderAccounts {
pub struct RotateProviderAccount {
pub scope: ProviderAccountAdminScope,
pub profile: UpdateProviderAccount,
pub replacement_identity: Option<ProviderAccountIdentity>,
pub credential: ProviderCredentialUpdate,
pub audit: AdminAuditEvent,
}
@@ -343,6 +323,7 @@ impl fmt::Debug for RotateProviderAccount {
.debug_struct("RotateProviderAccount")
.field("scope", &self.scope)
.field("profile", &self.profile)
.field("replacement_identity", &self.replacement_identity)
.field("credential", &self.credential)
.field("audit", &self.audit)
.finish()
@@ -394,7 +375,7 @@ impl fmt::Debug for ProviderCredentialUpdate {
#[derive(Debug, Clone, PartialEq)]
pub struct ProviderAccountObservation {
pub account_id: String,
pub availability: ProviderAccountAvailability,
pub availability: AccountAvailability,
pub availability_reason: Option<String>,
pub cooldown_until: Option<DateTime<Utc>>,
pub provider_quota_json: Option<JsonObject>,
@@ -406,7 +387,7 @@ pub struct ProviderAccountObservation {
pub struct ProviderAccountStateUpdate {
pub account_id: String,
pub expected_revision: Revision,
pub availability: ProviderAccountAvailability,
pub availability: AccountAvailability,
pub availability_reason: Option<String>,
pub cooldown_until: Option<DateTime<Utc>>,
pub availability_observed_at: DateTime<Utc>,
@@ -415,13 +396,7 @@ pub struct ProviderAccountStateUpdate {
impl ProviderAccountStateUpdate {
pub fn validate(&self) -> StoreResult<()> {
require_nonempty(ENTITY, "account_id", &self.account_id)?;
if (self.availability == ProviderAccountAvailability::Cooldown)
!= self.cooldown_until.is_some()
{
return Err(invalid(
"cooldown availability and cooldown_until must agree",
));
}
validate_availability_cooldown(self.availability, self.cooldown_until.is_some())?;
Ok(())
}
}
@@ -429,13 +404,7 @@ impl ProviderAccountStateUpdate {
impl ProviderAccountObservation {
pub fn validate(&self) -> StoreResult<()> {
require_nonempty(ENTITY, "account_id", &self.account_id)?;
if (self.availability == ProviderAccountAvailability::Cooldown)
!= self.cooldown_until.is_some()
{
return Err(invalid(
"cooldown availability and cooldown_until must agree",
));
}
validate_availability_cooldown(self.availability, self.cooldown_until.is_some())?;
if self.provider_quota_json.is_some() != self.quota_observed_at.is_some() {
return Err(invalid("quota JSON and quota_observed_at must agree"));
}
@@ -485,25 +454,21 @@ pub trait ProviderAccountAdminRepository: Send + Sync {
async fn import_provider_accounts(
&self,
expected_config_revision: Revision,
command: ImportProviderAccounts,
) -> StoreResult<ProviderAccountAdminImport>;
async fn rotate_provider_account(
&self,
expected_config_revision: Revision,
command: RotateProviderAccount,
) -> StoreResult<ProviderAccountAdminRotation>;
async fn set_provider_account_enabled_admin(
&self,
expected_config_revision: Revision,
command: SetProviderAccountEnabled,
) -> StoreResult<Revision>;
async fn delete_provider_accounts_admin(
&self,
expected_config_revision: Revision,
command: DeleteProviderAccounts,
) -> StoreResult<Revision>;
}
@@ -657,8 +622,14 @@ impl ProviderAccountRepository for PgProviderAccountRepository {
update.validate()?;
let result = sqlx::query(
"update provider_accounts
set availability = $3, availability_reason = $4, cooldown_until = $5,
availability_observed_at = $6, updated_at = greatest(now(), $6)
set availability = case when enabled then $3 else availability end,
availability_reason = case when enabled then $4 else availability_reason end,
cooldown_until = case when enabled then $5 else cooldown_until end,
availability_observed_at = case
when enabled then $6
else availability_observed_at
end,
updated_at = case when enabled then greatest(now(), $6) else updated_at end
where id = $1 and credential_revision = $2",
)
.bind(update.account_id)
@@ -767,7 +738,6 @@ impl ProviderAccountAdminRepository for PgProviderAccountRepository {
async fn import_provider_accounts(
&self,
expected_config_revision: Revision,
command: ImportProviderAccounts,
) -> StoreResult<ProviderAccountAdminImport> {
command.validate()?;
@@ -777,9 +747,7 @@ impl ProviderAccountAdminRepository for PgProviderAccountRepository {
.await
.map_err(|_| postgres_unavailable("begin provider account admin import"))?;
let result = async {
let revision =
bump_config_revision_in_transaction(&mut transaction, expected_config_revision)
.await?;
let revision = bump_config_revision_in_transaction(&mut transaction).await?;
let mut account_ids = Vec::with_capacity(command.accounts.len());
for account in &command.accounts {
account_ids
@@ -798,12 +766,17 @@ impl ProviderAccountAdminRepository for PgProviderAccountRepository {
async fn rotate_provider_account(
&self,
expected_config_revision: Revision,
command: RotateProviderAccount,
) -> StoreResult<ProviderAccountAdminRotation> {
command.scope.validate()?;
require_nonempty(ENTITY, "account_id", &command.profile.id)?;
require_nonempty(ENTITY, "name", &command.profile.name)?;
if let Some(identity) = &command.replacement_identity {
require_nonempty(ENTITY, "upstream_user_id", identity.upstream_user_id())?;
if let Some(account_id) = identity.upstream_account_id() {
require_nonempty(ENTITY, "upstream_account_id", account_id)?;
}
}
if command.profile.id != command.credential.account_id {
return Err(invalid("rotated profile and credential account IDs differ"));
}
@@ -814,13 +787,12 @@ impl ProviderAccountAdminRepository for PgProviderAccountRepository {
.await
.map_err(|_| postgres_unavailable("begin provider account admin rotation"))?;
let result = async {
let config_revision =
bump_config_revision_in_transaction(&mut transaction, expected_config_revision)
.await?;
let config_revision = bump_config_revision_in_transaction(&mut transaction).await?;
let credential_revision = rotate_provider_account_in_transaction(
&mut transaction,
&command.scope,
&command.profile,
command.replacement_identity.as_ref(),
&command.credential,
)
.await?;
@@ -841,7 +813,6 @@ impl ProviderAccountAdminRepository for PgProviderAccountRepository {
async fn set_provider_account_enabled_admin(
&self,
expected_config_revision: Revision,
command: SetProviderAccountEnabled,
) -> StoreResult<Revision> {
command.scope.validate()?;
@@ -852,9 +823,7 @@ impl ProviderAccountAdminRepository for PgProviderAccountRepository {
.await
.map_err(|_| postgres_unavailable("begin provider account admin state change"))?;
let result = async {
let revision =
bump_config_revision_in_transaction(&mut transaction, expected_config_revision)
.await?;
let revision = bump_config_revision_in_transaction(&mut transaction).await?;
set_provider_account_enabled_in_transaction(
&mut transaction,
&command.scope,
@@ -872,7 +841,6 @@ impl ProviderAccountAdminRepository for PgProviderAccountRepository {
async fn delete_provider_accounts_admin(
&self,
expected_config_revision: Revision,
command: DeleteProviderAccounts,
) -> StoreResult<Revision> {
command.scope.validate()?;
@@ -883,9 +851,7 @@ impl ProviderAccountAdminRepository for PgProviderAccountRepository {
.await
.map_err(|_| postgres_unavailable("begin provider account admin deletion"))?;
let result = async {
let revision =
bump_config_revision_in_transaction(&mut transaction, expected_config_revision)
.await?;
let revision = bump_config_revision_in_transaction(&mut transaction).await?;
delete_provider_accounts_in_transaction(
&mut transaction,
&command.scope,
@@ -1032,7 +998,6 @@ impl PgAdminAccountStore {
async fn commit_prepared_import(
&self,
expected_config_revision: AdminRevision,
prepared: PreparedCredentialImport,
context: &MutationContext,
action: &str,
@@ -1046,22 +1011,19 @@ impl PgAdminAccountStore {
.map_err(|error| admin_store_error(ENTITY, error))?;
let imported = self
.accounts
.import_provider_accounts(
store_revision(expected_config_revision)?,
ImportProviderAccounts {
scope: ProviderAccountAdminScope {
provider_kind: provider_kind.clone(),
},
accounts,
audit: mutation_audit(
context,
action,
"provider_account",
&provider_kind,
vec!["credentials".to_owned()],
),
.import_provider_accounts(ImportProviderAccounts {
scope: ProviderAccountAdminScope {
provider_kind: provider_kind.clone(),
},
)
accounts,
audit: mutation_audit(
context,
action,
"provider_account",
&provider_kind,
vec!["credentials".to_owned()],
),
})
.await
.map_err(|error| admin_store_error(ENTITY, error))?;
Ok(CredentialImportResult {
@@ -1083,7 +1045,6 @@ impl PgAdminAccountStore {
async fn commit_prepared_rotation(
&self,
expected_config_revision: AdminRevision,
prepared: PreparedCredentialRotationFacts,
context: &MutationContext,
action: &str,
@@ -1094,36 +1055,32 @@ impl PgAdminAccountStore {
};
let rotation = self
.accounts
.rotate_provider_account(
store_revision(expected_config_revision)?,
RotateProviderAccount {
scope,
profile: UpdateProviderAccount {
id: account_id.as_str().to_owned(),
name: prepared.name,
email: prepared.email,
plan_type: prepared.plan_type,
},
credential: ProviderCredentialUpdate {
account_id: account_id.as_str().to_owned(),
expected_revision: store_revision(prepared.expected_credential_revision)?,
provider_credentials_json: provider_document_json(
prepared.provider_material,
)
.map_err(|error| admin_store_error(ENTITY, error))?,
has_refresh_token: prepared.has_refresh_token,
access_token_expires_at: prepared.access_token_expires_at,
next_refresh_at: prepared.next_refresh_at,
},
audit: mutation_audit(
context,
action,
"provider_account",
account_id.as_str(),
vec!["credentials".to_owned()],
),
.rotate_provider_account(RotateProviderAccount {
scope,
profile: UpdateProviderAccount {
id: account_id.as_str().to_owned(),
name: prepared.name,
email: prepared.email,
plan_type: prepared.plan_type,
},
)
replacement_identity: prepared.replacement_identity,
credential: ProviderCredentialUpdate {
account_id: account_id.as_str().to_owned(),
expected_revision: store_revision(prepared.expected_credential_revision)?,
provider_credentials_json: provider_document_json(prepared.provider_material)
.map_err(|error| admin_store_error(ENTITY, error))?,
has_refresh_token: prepared.has_refresh_token,
access_token_expires_at: prepared.access_token_expires_at,
next_refresh_at: prepared.next_refresh_at,
},
audit: mutation_audit(
context,
action,
"provider_account",
account_id.as_str(),
vec!["credentials".to_owned()],
),
})
.await
.map_err(|error| admin_store_error(ENTITY, error))?;
Ok(CredentialMutationResult {
@@ -1251,9 +1208,10 @@ impl AccountStore for PgAdminAccountStore {
.into_iter()
.filter(|account| account.provider_kind == provider_kind.as_str())
.filter(|account| {
query.availability.as_ref().is_none_or(|expected| {
expected.matches(admin_account_availability(account.availability))
})
query
.availability
.as_ref()
.is_none_or(|expected| expected.matches(account.availability))
})
.filter(|account| {
query
@@ -1383,13 +1341,8 @@ impl AccountStore for PgAdminAccountStore {
command: CredentialImportCommit,
context: &MutationContext,
) -> AdminStoreResult<CredentialImportResult> {
self.commit_prepared_import(
command.expected_config_revision,
command.prepared,
context,
"import_document",
)
.await
self.commit_prepared_import(command.prepared, context, "import_document")
.await
}
async fn commit_authorization(
@@ -1397,13 +1350,11 @@ impl AccountStore for PgAdminAccountStore {
command: AuthorizationCommit,
context: &MutationContext,
) -> AdminStoreResult<CredentialMutationResult> {
let expected_config_revision = command.pending.expected_config_revision();
match command.credential {
AuthorizationCredentialCommit::Create(credential) => {
let account_id = credential.account_id.clone();
let result = self
.commit_prepared_import(
expected_config_revision,
PreparedCredentialImport {
provider_kind: credential.provider_kind.clone(),
credentials: vec![credential],
@@ -1431,13 +1382,8 @@ impl AccountStore for PgAdminAccountStore {
})
}
AuthorizationCredentialCommit::Reauthorize(prepared) => {
self.commit_prepared_rotation(
expected_config_revision,
prepared,
context,
"reauthorize",
)
.await
self.commit_prepared_rotation(prepared, context, "reauthorize")
.await
}
}
}
@@ -1447,13 +1393,8 @@ impl AccountStore for PgAdminAccountStore {
command: CredentialRotationCommit,
context: &MutationContext,
) -> AdminStoreResult<CredentialMutationResult> {
self.commit_prepared_rotation(
command.expected_config_revision,
command.prepared,
context,
"rotate_credential",
)
.await
self.commit_prepared_rotation(command.prepared, context, "rotate_credential")
.await
}
async fn commit_credential_refresh(
@@ -1461,13 +1402,8 @@ impl AccountStore for PgAdminAccountStore {
command: CredentialRotationCommit,
context: &MutationContext,
) -> AdminStoreResult<CredentialMutationResult> {
self.commit_prepared_rotation(
command.expected_config_revision,
command.prepared,
context,
"refresh_credential",
)
.await
self.commit_prepared_rotation(command.prepared, context, "refresh_credential")
.await
}
async fn set_account_enabled(
@@ -1478,21 +1414,18 @@ impl AccountStore for PgAdminAccountStore {
let scope = self.required_scope(&command.account_id).await?;
let action = if command.enabled { "enable" } else { "disable" };
self.accounts
.set_provider_account_enabled_admin(
store_revision(command.expected_config_revision)?,
SetProviderAccountEnabled {
scope,
account_id: command.account_id.clone(),
enabled: command.enabled,
audit: mutation_audit(
context,
action,
"provider_account",
&command.account_id,
vec!["enabled".to_owned()],
),
},
)
.set_provider_account_enabled_admin(SetProviderAccountEnabled {
scope,
account_id: command.account_id.clone(),
enabled: command.enabled,
audit: mutation_audit(
context,
action,
"provider_account",
&command.account_id,
vec!["enabled".to_owned()],
),
})
.await
.map_err(|error| admin_store_error(ENTITY, error))
.and_then(admin_revision)
@@ -1517,20 +1450,17 @@ impl AccountStore for PgAdminAccountStore {
"provider_accounts".to_owned()
};
self.accounts
.delete_provider_accounts_admin(
store_revision(command.expected_config_revision)?,
DeleteProviderAccounts {
scope,
account_ids: command.account_ids,
audit: mutation_audit(
context,
"delete",
"provider_account",
&audit_target,
Vec::new(),
),
},
)
.delete_provider_accounts_admin(DeleteProviderAccounts {
scope,
account_ids: command.account_ids,
audit: mutation_audit(
context,
"delete",
"provider_account",
&audit_target,
Vec::new(),
),
})
.await
.map_err(|error| admin_store_error(ENTITY, error))
.and_then(admin_revision)
@@ -1656,12 +1586,16 @@ fn admin_account_status(
) -> AdminAccountStatus {
if !account.enabled {
AdminAccountStatus::Disabled
} else if account.availability == ProviderAccountAvailability::Banned {
} else if account.availability == AccountAvailability::Banned {
AdminAccountStatus::Banned
} else if account.availability == ProviderAccountAvailability::QuotaExhausted {
} else if account.availability == AccountAvailability::QuotaExhausted
|| (account.availability == AccountAvailability::Cooldown
&& account.availability_reason.as_deref() == Some("usage_limit_exhausted")
&& account.cooldown_until.is_none_or(|until| until > now))
{
AdminAccountStatus::QuotaExhausted
} else if account.availability == ProviderAccountAvailability::Expired
|| account.availability == ProviderAccountAvailability::Invalid
} else if account.availability == AccountAvailability::Expired
|| account.availability == AccountAvailability::Invalid
|| account
.access_token_expires_at
.is_some_and(|expires_at| expires_at <= now)
@@ -1786,7 +1720,7 @@ fn admin_account_record(summary: ProviderAccountSummary) -> AdminStoreResult<Acc
access_token_expires_at: summary.access_token_expires_at,
next_refresh_at: summary.next_refresh_at,
enabled: summary.enabled,
availability: admin_account_availability(summary.availability),
availability: summary.availability,
availability_reason: summary.availability_reason,
cooldown_until: summary.cooldown_until,
availability_observed_at: summary.availability_observed_at,
@@ -1811,7 +1745,7 @@ fn prepared_account(credential: PreparedCredentialCreate) -> StoreResult<NewProv
access_token_expires_at: credential.access_token_expires_at,
next_refresh_at: credential.next_refresh_at,
enabled: credential.enabled,
availability: provider_account_availability(credential.availability),
availability: credential.availability,
availability_reason: credential.availability_reason,
cooldown_until: credential.cooldown_until,
availability_observed_at: credential.availability_observed_at,
@@ -1826,34 +1760,6 @@ fn provider_document_json(document: ProviderDocument) -> StoreResult<JsonObject>
)
}
const fn provider_account_availability(
availability: AdminAccountAvailability,
) -> ProviderAccountAvailability {
match availability {
AdminAccountAvailability::Unknown => ProviderAccountAvailability::Unknown,
AdminAccountAvailability::Ready => ProviderAccountAvailability::Ready,
AdminAccountAvailability::Cooldown => ProviderAccountAvailability::Cooldown,
AdminAccountAvailability::QuotaExhausted => ProviderAccountAvailability::QuotaExhausted,
AdminAccountAvailability::Expired => ProviderAccountAvailability::Expired,
AdminAccountAvailability::Banned => ProviderAccountAvailability::Banned,
AdminAccountAvailability::Invalid => ProviderAccountAvailability::Invalid,
}
}
const fn admin_account_availability(
availability: ProviderAccountAvailability,
) -> AdminAccountAvailability {
match availability {
ProviderAccountAvailability::Unknown => AdminAccountAvailability::Unknown,
ProviderAccountAvailability::Ready => AdminAccountAvailability::Ready,
ProviderAccountAvailability::Cooldown => AdminAccountAvailability::Cooldown,
ProviderAccountAvailability::QuotaExhausted => AdminAccountAvailability::QuotaExhausted,
ProviderAccountAvailability::Expired => AdminAccountAvailability::Expired,
ProviderAccountAvailability::Banned => AdminAccountAvailability::Banned,
ProviderAccountAvailability::Invalid => AdminAccountAvailability::Invalid,
}
}
fn admin_account_usage(usage: ProviderAccountUsageObservation) -> AdminStoreResult<AccountUsage> {
Ok(AccountUsage {
account_id: usage.account_id,
@@ -2093,8 +1999,13 @@ async fn rotate_provider_account_in_transaction(
transaction: &mut Transaction<'_, Postgres>,
scope: &ProviderAccountAdminScope,
profile: &UpdateProviderAccount,
replacement_identity: Option<&ProviderAccountIdentity>,
update: &ProviderCredentialUpdate,
) -> StoreResult<Revision> {
let replace_identity = replacement_identity.is_some();
let upstream_user_id = replacement_identity.map(ProviderAccountIdentity::upstream_user_id);
let upstream_account_id =
replacement_identity.and_then(ProviderAccountIdentity::upstream_account_id);
let next = sqlx::query_scalar::<_, i64>(
"update provider_accounts
set name = $4,
@@ -2105,6 +2016,28 @@ async fn rotate_provider_account_in_transaction(
has_refresh_token = $8,
access_token_expires_at = $9,
next_refresh_at = $10,
upstream_user_id = case when $11::boolean then $12::text else upstream_user_id end,
upstream_account_id = case when $11::boolean then $13::text else upstream_account_id end,
availability = case
when not enabled then availability
when availability <> 'quota_exhausted' then 'ready'
else availability
end,
availability_reason = case
when not enabled then availability_reason
when availability <> 'quota_exhausted' then null
else availability_reason
end,
cooldown_until = case
when not enabled then cooldown_until
when availability <> 'quota_exhausted' then null
else cooldown_until
end,
availability_observed_at = case
when not enabled then availability_observed_at
when availability <> 'quota_exhausted' then now()
else availability_observed_at
end,
updated_at = now()
where id = $1 and provider_kind = $2
and credential_revision = $3
@@ -2120,9 +2053,25 @@ async fn rotate_provider_account_in_transaction(
.bind(update.has_refresh_token)
.bind(update.access_token_expires_at)
.bind(update.next_refresh_at)
.bind(replace_identity)
.bind(upstream_user_id)
.bind(upstream_account_id)
.fetch_optional(&mut **transaction)
.await
.map_err(|_| postgres_unavailable("rotate provider account in admin transaction"))?
.map_err(|error| {
if error
.as_database_error()
.is_some_and(sqlx::error::DatabaseError::is_unique_violation)
{
StoreError::Conflict {
entity: ENTITY,
id: update.account_id.clone(),
kind: ConflictKind::InvalidTransition,
}
} else {
postgres_unavailable("rotate provider account in admin transaction")
}
})?
.ok_or_else(|| StoreError::Conflict {
entity: ENTITY,
id: update.account_id.clone(),
@@ -2268,7 +2217,7 @@ impl ProviderAccountStore for PgProviderAccountRepository {
.map(DateTime::<Utc>::from),
next_refresh_at: account.account.next_refresh_at().map(DateTime::<Utc>::from),
enabled: account.account.enabled(),
availability: availability_from_core(account.account.availability()),
availability: account.account.availability(),
availability_reason: None,
cooldown_until: account.account.cooldown_until().map(DateTime::<Utc>::from),
availability_observed_at: Utc::now(),
@@ -2313,15 +2262,23 @@ impl ProviderAccountStore for PgProviderAccountRepository {
&self,
account: &CoreProviderAccountId,
expected_revision: CoreCredentialRevision,
) -> Result<LoadedCredential, CoreStoreError> {
let loaded = self.load_current_credential(account).await?;
if loaded.account.revision().get() != expected_revision.get() {
return Err(CoreStoreError::new(CoreStoreErrorKind::Conflict));
}
Ok(loaded)
}
async fn load_current_credential(
&self,
account: &CoreProviderAccountId,
) -> Result<LoadedCredential, CoreStoreError> {
let record = self
.load_provider_account(account.as_str())
.await
.map_err(core_store_error)?
.ok_or_else(|| CoreStoreError::new(CoreStoreErrorKind::InvalidData))?;
if record.summary.credential_revision.get() != expected_revision.get() {
return Err(CoreStoreError::new(CoreStoreErrorKind::Conflict));
}
Ok(LoadedCredential {
account: core_account_from_summary(record.summary)?,
credential: PlaintextCredential::new(record.provider_credentials_json.fields().clone()),
@@ -2470,7 +2427,7 @@ impl ProviderAccountStore for PgProviderAccountRepository {
account_id: change.account_id.as_str().to_owned(),
expected_revision: Revision::new(change.expected_revision.get())
.map_err(core_store_error)?,
availability: availability_from_core(change.availability),
availability: change.availability,
availability_reason: change.reason,
cooldown_until: change.cooldown_until.map(DateTime::<Utc>::from),
availability_observed_at: DateTime::<Utc>::from(change.observed_at),
@@ -2581,7 +2538,7 @@ fn core_account_from_summary(
)
.with_runtime_state(
summary.enabled,
availability_to_core(summary.availability),
summary.availability,
summary.cooldown_until.map(Into::into),
)
.with_refresh_schedule(
@@ -2590,30 +2547,6 @@ fn core_account_from_summary(
))
}
const fn availability_to_core(value: ProviderAccountAvailability) -> CoreAccountAvailability {
match value {
ProviderAccountAvailability::Unknown => CoreAccountAvailability::Unknown,
ProviderAccountAvailability::Ready => CoreAccountAvailability::Ready,
ProviderAccountAvailability::Cooldown => CoreAccountAvailability::Cooldown,
ProviderAccountAvailability::QuotaExhausted => CoreAccountAvailability::QuotaExhausted,
ProviderAccountAvailability::Expired => CoreAccountAvailability::Expired,
ProviderAccountAvailability::Banned => CoreAccountAvailability::Banned,
ProviderAccountAvailability::Invalid => CoreAccountAvailability::Invalid,
}
}
const fn availability_from_core(value: CoreAccountAvailability) -> ProviderAccountAvailability {
match value {
CoreAccountAvailability::Unknown => ProviderAccountAvailability::Unknown,
CoreAccountAvailability::Ready => ProviderAccountAvailability::Ready,
CoreAccountAvailability::Cooldown => ProviderAccountAvailability::Cooldown,
CoreAccountAvailability::QuotaExhausted => ProviderAccountAvailability::QuotaExhausted,
CoreAccountAvailability::Expired => ProviderAccountAvailability::Expired,
CoreAccountAvailability::Banned => ProviderAccountAvailability::Banned,
CoreAccountAvailability::Invalid => ProviderAccountAvailability::Invalid,
}
}
fn core_store_error(error: StoreError) -> CoreStoreError {
let kind = match error {
StoreError::Unavailable { .. } => CoreStoreErrorKind::Unavailable,
@@ -2654,7 +2587,7 @@ fn account_summary_from_row(row: sqlx::postgres::PgRow) -> StoreResult<ProviderA
access_token_expires_at: get(&row, "access_token_expires_at")?,
next_refresh_at: get(&row, "next_refresh_at")?,
enabled: get(&row, "enabled")?,
availability: ProviderAccountAvailability::parse(&availability)?,
availability: parse_availability(&availability)?,
availability_reason: get(&row, "availability_reason")?,
cooldown_until: get(&row, "cooldown_until")?,
availability_observed_at: get(&row, "availability_observed_at")?,
@@ -12,7 +12,7 @@ use gateway_core::provider_ports::{
ProviderRefreshPolicy, ProviderRuntimePolicyPort, ProviderStoreError, ProviderStoreErrorKind,
};
use crate::{ConflictKind, Revision, StoreError, StoreResult, postgres_unavailable};
use crate::{Revision, StoreError, StoreResult, postgres_unavailable};
#[derive(Clone, PartialEq, Eq)]
pub struct RuntimeSettings {
@@ -111,11 +111,8 @@ impl RuntimeSettingsUpdate {
pub trait RuntimeSettingsRepository: Send + Sync {
async fn load_runtime_settings(&self) -> StoreResult<RuntimeSettings>;
async fn update_runtime_settings(
&self,
expected_revision: Revision,
update: RuntimeSettingsUpdate,
) -> StoreResult<Revision>;
async fn update_runtime_settings(&self, update: RuntimeSettingsUpdate)
-> StoreResult<Revision>;
}
#[derive(Clone)]
@@ -169,7 +166,6 @@ impl RuntimeSettingsRepository for PgRuntimeSettingsRepository {
async fn update_runtime_settings(
&self,
expected_revision: Revision,
update: RuntimeSettingsUpdate,
) -> StoreResult<Revision> {
let mut transaction = self
@@ -177,9 +173,7 @@ impl RuntimeSettingsRepository for PgRuntimeSettingsRepository {
.begin()
.await
.map_err(|_| postgres_unavailable("begin runtime settings update"))?;
let revision =
update_runtime_settings_in_transaction(&mut transaction, expected_revision, &update)
.await?;
let revision = update_runtime_settings_in_transaction(&mut transaction, &update).await?;
transaction
.commit()
.await
@@ -228,30 +222,43 @@ pub(crate) async fn load_runtime_settings_in_transaction(
pub(crate) async fn update_runtime_settings_in_transaction(
transaction: &mut Transaction<'_, Postgres>,
expected_revision: Revision,
update: &RuntimeSettingsUpdate,
) -> StoreResult<Revision> {
update.validate()?;
let refresh_margin_seconds =
i64::try_from(update.refresh_margin_seconds).map_err(|_| invalid_numeric())?;
let previous_refresh_margin_seconds = sqlx::query_scalar::<_, i64>(
"select refresh_margin_seconds
from runtime_settings
where id = 1
for update",
)
.fetch_optional(&mut **transaction)
.await
.map_err(|_| postgres_unavailable("lock runtime settings for update"))?
.ok_or_else(|| StoreError::NotFound {
entity: "runtime settings",
id: "1".to_owned(),
})?;
let next = sqlx::query_scalar::<_, i64>(
"update runtime_settings
set config_revision = config_revision + 1,
admin_api_key = $2,
refresh_margin_seconds = $3,
refresh_concurrency = $4,
max_concurrent_per_account = $5,
request_interval_ms = $6,
rotation_strategy = $7,
model_mappings_json = $8,
usage_retention_days = $9,
ops_event_retention_days = $10,
audit_retention_days = $11,
updated_at = now()
where id = 1 and config_revision = $1
returning config_revision",
admin_api_key = $1,
refresh_margin_seconds = $2,
refresh_concurrency = $3,
max_concurrent_per_account = $4,
request_interval_ms = $5,
rotation_strategy = $6,
model_mappings_json = $7,
usage_retention_days = $8,
ops_event_retention_days = $9,
audit_retention_days = $10,
updated_at = now()
where id = 1
returning config_revision",
)
.bind(i64::try_from(expected_revision.get()).map_err(|_| invalid_numeric())?)
.bind(update.admin_api_key.as_deref())
.bind(i64::try_from(update.refresh_margin_seconds).map_err(|_| invalid_numeric())?)
.bind(refresh_margin_seconds)
.bind(i64::from(update.refresh_concurrency))
.bind(i64::from(update.max_concurrent_per_account))
.bind(i64::try_from(update.request_interval_ms).map_err(|_| invalid_numeric())?)
@@ -263,32 +270,54 @@ pub(crate) async fn update_runtime_settings_in_transaction(
.fetch_optional(&mut **transaction)
.await
.map_err(|_| postgres_unavailable("update runtime settings in transaction"))?
.ok_or_else(|| StoreError::Conflict {
.ok_or_else(|| StoreError::NotFound {
entity: "runtime settings",
id: "1".to_owned(),
kind: ConflictKind::StaleRevision,
})?;
if previous_refresh_margin_seconds != refresh_margin_seconds {
reschedule_provider_refreshes_in_transaction(transaction, refresh_margin_seconds).await?;
}
Revision::new(u64::try_from(next).map_err(|_| invalid_numeric())?)
}
async fn reschedule_provider_refreshes_in_transaction(
transaction: &mut Transaction<'_, Postgres>,
refresh_margin_seconds: i64,
) -> StoreResult<()> {
sqlx::query(
"update provider_accounts
set next_refresh_at = case
when access_token_expires_at <= now()
or extract(epoch from (access_token_expires_at - now())) <= $1
then now()
else access_token_expires_at - ($1::double precision * interval '1 second')
end,
updated_at = now()
where has_refresh_token
and access_token_expires_at is not null",
)
.bind(refresh_margin_seconds)
.execute(&mut **transaction)
.await
.map_err(|_| postgres_unavailable("reschedule provider credential refreshes"))?;
Ok(())
}
pub(crate) async fn bump_config_revision_in_transaction(
transaction: &mut Transaction<'_, Postgres>,
expected_revision: Revision,
) -> StoreResult<Revision> {
let next = sqlx::query_scalar::<_, i64>(
"update runtime_settings
set config_revision = config_revision + 1, updated_at = now()
where id = 1 and config_revision = $1
where id = 1
returning config_revision",
)
.bind(i64::try_from(expected_revision.get()).map_err(|_| invalid_numeric())?)
.fetch_optional(&mut **transaction)
.await
.map_err(|_| postgres_unavailable("bump config revision in transaction"))?
.ok_or_else(|| StoreError::Conflict {
.ok_or_else(|| StoreError::NotFound {
entity: "runtime settings",
id: "1".to_owned(),
kind: ConflictKind::StaleRevision,
})?;
Revision::new(u64::try_from(next).map_err(|_| invalid_numeric())?)
}
@@ -0,0 +1,254 @@
//! 可由 TTL 或后续反馈收敛的 Redis 写副作用队列。
//!
//! Continuation affinity 决定下一轮 Provider 与账号,必须在响应 ID 可复用前
//! 获得 Redis 确认,因此不属于本模块的可丢失副作用。
use std::num::NonZeroUsize;
use std::sync::Arc;
use futures::future::{BoxFuture, ready};
use gateway_core::engine::admission::{
ClientAdmissionDecision, ClientAdmissionError, ClientAdmissionPort, ClientAdmissionRecovery,
ClientAdmissionRequest, ClientAdmissionRestoreResult,
};
use gateway_core::engine::execution::{
ProviderCircuitDecision, ProviderCircuitError, ProviderCircuitPort,
};
use gateway_core::engine::{CancellationToken, ModelRequestId};
use gateway_core::policy::ClientApiKeyId;
use gateway_core::routing::ProviderKind;
use gateway_core::task::{DaemonTask, WorkerTaskError};
use tokio::sync::{Mutex, mpsc};
const DEFAULT_QUEUE_CAPACITY: usize = 4_096;
/// 准入读取保持强一致;终态释放只做有界入队,Redis 失败由 worker 吸收。
#[derive(Clone)]
pub struct BufferedClientAdmissionPort {
inner: Arc<dyn ClientAdmissionPort>,
sender: mpsc::Sender<AdmissionRelease>,
}
impl BufferedClientAdmissionPort {
#[must_use]
pub fn new(inner: Arc<dyn ClientAdmissionPort>) -> (Self, ClientAdmissionReleaseWriter) {
Self::with_capacity(
inner,
NonZeroUsize::new(DEFAULT_QUEUE_CAPACITY).expect("queue capacity is non-zero"),
)
}
#[must_use]
pub fn with_capacity(
inner: Arc<dyn ClientAdmissionPort>,
capacity: NonZeroUsize,
) -> (Self, ClientAdmissionReleaseWriter) {
let (sender, receiver) = mpsc::channel(capacity.get());
(
Self {
inner: Arc::clone(&inner),
sender,
},
ClientAdmissionReleaseWriter {
inner,
receiver: Mutex::new(receiver),
},
)
}
fn enqueue(&self, release: AdmissionRelease) -> bool {
match self.sender.try_send(release) {
Ok(()) => true,
Err(error) => {
let reason = match error {
mpsc::error::TrySendError::Full(_) => "full",
mpsc::error::TrySendError::Closed(_) => "closed",
};
tracing::warn!(
operation = "release_client_admission",
reason,
"Redis 协调队列不可用,已丢弃本次准入释放;租约将由 TTL 收敛"
);
false
}
}
}
}
impl ClientAdmissionPort for BufferedClientAdmissionPort {
fn admit(
&self,
request: ClientAdmissionRequest,
) -> BoxFuture<'_, Result<ClientAdmissionDecision, ClientAdmissionError>> {
self.inner.admit(request)
}
fn release<'a>(
&'a self,
client_api_key_id: &'a ClientApiKeyId,
model_request_id: &'a ModelRequestId,
) -> BoxFuture<'a, Result<bool, ClientAdmissionError>> {
let enqueued = self.enqueue(AdmissionRelease {
client_api_key_id: client_api_key_id.clone(),
model_request_id: model_request_id.clone(),
});
Box::pin(ready(Ok(enqueued)))
}
fn restore(
&self,
recovery: ClientAdmissionRecovery,
) -> BoxFuture<'_, Result<ClientAdmissionRestoreResult, ClientAdmissionError>> {
self.inner.restore(recovery)
}
}
struct AdmissionRelease {
client_api_key_id: ClientApiKeyId,
model_request_id: ModelRequestId,
}
pub struct ClientAdmissionReleaseWriter {
inner: Arc<dyn ClientAdmissionPort>,
receiver: Mutex<mpsc::Receiver<AdmissionRelease>>,
}
impl DaemonTask for ClientAdmissionReleaseWriter {
fn run(&self, cancellation: CancellationToken) -> BoxFuture<'_, Result<(), WorkerTaskError>> {
Box::pin(async move {
let mut receiver = self.receiver.lock().await;
loop {
let release = tokio::select! {
() = cancellation.cancelled() => return Ok(()),
release = receiver.recv() => release,
};
let Some(release) = release else {
return Err(WorkerTaskError::safe(
"client admission release queue closed",
));
};
if let Err(error) = self
.inner
.release(&release.client_api_key_id, &release.model_request_id)
.await
{
tracing::warn!(%error, "Client admission 后台释放失败,依赖租约 TTL 收敛");
}
}
})
}
}
/// Circuit decision 仍直读 Redis;success/failure feedback 只做有界入队。
#[derive(Clone)]
pub struct BufferedProviderCircuitPort {
inner: Arc<dyn ProviderCircuitPort>,
sender: mpsc::Sender<CircuitFeedback>,
}
impl BufferedProviderCircuitPort {
#[must_use]
pub fn new(inner: Arc<dyn ProviderCircuitPort>) -> (Self, ProviderCircuitFeedbackWriter) {
Self::with_capacity(
inner,
NonZeroUsize::new(DEFAULT_QUEUE_CAPACITY).expect("queue capacity is non-zero"),
)
}
#[must_use]
pub fn with_capacity(
inner: Arc<dyn ProviderCircuitPort>,
capacity: NonZeroUsize,
) -> (Self, ProviderCircuitFeedbackWriter) {
let (sender, receiver) = mpsc::channel(capacity.get());
(
Self {
inner: Arc::clone(&inner),
sender,
},
ProviderCircuitFeedbackWriter {
inner,
receiver: Mutex::new(receiver),
},
)
}
fn enqueue(&self, feedback: CircuitFeedback) {
if let Err(error) = self.sender.try_send(feedback) {
let reason = match error {
mpsc::error::TrySendError::Full(_) => "full",
mpsc::error::TrySendError::Closed(_) => "closed",
};
tracing::warn!(
operation = "record_provider_circuit_feedback",
reason,
"Redis 协调队列不可用,已丢弃本次 circuit feedback"
);
}
}
}
impl ProviderCircuitPort for BufferedProviderCircuitPort {
fn decision<'a>(
&'a self,
provider_kind: &'a ProviderKind,
) -> BoxFuture<'a, Result<ProviderCircuitDecision, ProviderCircuitError>> {
self.inner.decision(provider_kind)
}
fn observe_failure<'a>(
&'a self,
provider_kind: &'a ProviderKind,
) -> BoxFuture<'a, Result<(), ProviderCircuitError>> {
self.enqueue(CircuitFeedback::Failure(provider_kind.clone()));
Box::pin(ready(Ok(())))
}
fn observe_success<'a>(
&'a self,
provider_kind: &'a ProviderKind,
) -> BoxFuture<'a, Result<(), ProviderCircuitError>> {
self.enqueue(CircuitFeedback::Success(provider_kind.clone()));
Box::pin(ready(Ok(())))
}
}
enum CircuitFeedback {
Failure(ProviderKind),
Success(ProviderKind),
}
pub struct ProviderCircuitFeedbackWriter {
inner: Arc<dyn ProviderCircuitPort>,
receiver: Mutex<mpsc::Receiver<CircuitFeedback>>,
}
impl DaemonTask for ProviderCircuitFeedbackWriter {
fn run(&self, cancellation: CancellationToken) -> BoxFuture<'_, Result<(), WorkerTaskError>> {
Box::pin(async move {
let mut receiver = self.receiver.lock().await;
loop {
let feedback = tokio::select! {
() = cancellation.cancelled() => return Ok(()),
feedback = receiver.recv() => feedback,
};
let Some(feedback) = feedback else {
return Err(WorkerTaskError::safe(
"provider circuit feedback queue closed",
));
};
let result = match feedback {
CircuitFeedback::Failure(provider) => {
self.inner.observe_failure(&provider).await
}
CircuitFeedback::Success(provider) => {
self.inner.observe_success(&provider).await
}
};
if let Err(error) = result {
tracing::warn!(%error, "Provider circuit feedback 后台写入失败");
}
}
})
}
}
@@ -13,7 +13,8 @@ use gateway_core::{
},
error::StoreError as CoreStoreError,
provider_ports::{
ProviderCooldown, ProviderCooldownPort, ProviderStoreError, ProviderStoreErrorKind,
ProviderCooldown, ProviderCooldownPort, ProviderCooldownScope, ProviderScopedCooldown,
ProviderStoreError, ProviderStoreErrorKind,
},
routing::ProviderKind,
};
@@ -104,24 +105,35 @@ impl RedisCredentialCooldownRepository {
let fingerprint = resource_fingerprint("credential cooldown", provider_account_id)?;
Ok(format!("{}:account:{fingerprint}:cooldown", self.namespace))
}
}
#[async_trait]
impl CredentialCooldownRepository for RedisCredentialCooldownRepository {
async fn cache_credential_cooldown(&self, cooldown: &CredentialCooldown) -> StoreResult<bool> {
require_nonempty(
"credential cooldown",
"provider_account_id",
&cooldown.provider_account_id,
)?;
let until_ms = cooldown.cooldown_until.timestamp_millis();
fn scoped_key(
&self,
provider_account_id: &str,
scope: &ProviderCooldownScope,
) -> StoreResult<String> {
let account_fingerprint = resource_fingerprint("credential cooldown", provider_account_id)?;
let scope_fingerprint = resource_fingerprint("credential cooldown scope", scope.value())?;
Ok(format!(
"{}:account:{account_fingerprint}:cooldown:{}:{scope_fingerprint}",
self.namespace,
scope.kind(),
))
}
async fn cache_at_key(
&self,
key: String,
credential_revision: Revision,
cooldown_until: DateTime<Utc>,
) -> StoreResult<bool> {
let until_ms = cooldown_until.timestamp_millis();
if until_ms <= 0 {
return Err(invalid("cooldown expiry must be positive"));
}
let mut connection = self.connection.clone();
let written = Script::new(WRITE_SCRIPT)
.key(self.key(&cooldown.provider_account_id)?)
.arg(cooldown.credential_revision.get())
.key(key)
.arg(credential_revision.get())
.arg(until_ms)
.invoke_async::<i64>(&mut connection)
.await
@@ -129,18 +141,10 @@ impl CredentialCooldownRepository for RedisCredentialCooldownRepository {
Ok(written == 1)
}
async fn read_credential_cooldown(
&self,
provider_account_id: &str,
) -> StoreResult<Option<CredentialCooldown>> {
require_nonempty(
"credential cooldown",
"provider_account_id",
provider_account_id,
)?;
async fn read_at_key(&self, key: String) -> StoreResult<Option<(Revision, DateTime<Utc>)>> {
let mut connection = self.connection.clone();
let (present, revision, until_ms): (i64, String, String) = Script::new(READ_SCRIPT)
.key(self.key(provider_account_id)?)
.key(key)
.invoke_async(&mut connection)
.await
.map_err(|_| redis_unavailable("read credential cooldown"))?;
@@ -155,11 +159,59 @@ impl CredentialCooldownRepository for RedisCredentialCooldownRepository {
.map_err(|_| invalid("cached cooldown expiry is invalid"))?;
let cooldown_until = DateTime::from_timestamp_millis(until_ms)
.ok_or_else(|| invalid("cached cooldown expiry is invalid"))?;
Ok(Some(CredentialCooldown {
provider_account_id: provider_account_id.to_owned(),
credential_revision: Revision::new(revision)?,
cooldown_until,
}))
Ok(Some((Revision::new(revision)?, cooldown_until)))
}
async fn invalidate_at_key(
&self,
key: String,
through_revision: Revision,
) -> StoreResult<bool> {
let mut connection = self.connection.clone();
let removed = Script::new(INVALIDATE_SCRIPT)
.key(key)
.arg(through_revision.get())
.invoke_async::<i64>(&mut connection)
.await
.map_err(|_| redis_unavailable("invalidate credential cooldown"))?;
Ok(removed == 1)
}
}
#[async_trait]
impl CredentialCooldownRepository for RedisCredentialCooldownRepository {
async fn cache_credential_cooldown(&self, cooldown: &CredentialCooldown) -> StoreResult<bool> {
require_nonempty(
"credential cooldown",
"provider_account_id",
&cooldown.provider_account_id,
)?;
self.cache_at_key(
self.key(&cooldown.provider_account_id)?,
cooldown.credential_revision,
cooldown.cooldown_until,
)
.await
}
async fn read_credential_cooldown(
&self,
provider_account_id: &str,
) -> StoreResult<Option<CredentialCooldown>> {
require_nonempty(
"credential cooldown",
"provider_account_id",
provider_account_id,
)?;
self.read_at_key(self.key(provider_account_id)?)
.await
.map(|value| {
value.map(|(credential_revision, cooldown_until)| CredentialCooldown {
provider_account_id: provider_account_id.to_owned(),
credential_revision,
cooldown_until,
})
})
}
async fn invalidate_credential_cooldown(
@@ -172,14 +224,8 @@ impl CredentialCooldownRepository for RedisCredentialCooldownRepository {
"provider_account_id",
provider_account_id,
)?;
let mut connection = self.connection.clone();
let removed = Script::new(INVALIDATE_SCRIPT)
.key(self.key(provider_account_id)?)
.arg(through_revision.get())
.invoke_async::<i64>(&mut connection)
self.invalidate_at_key(self.key(provider_account_id)?, through_revision)
.await
.map_err(|_| redis_unavailable("invalidate credential cooldown"))?;
Ok(removed == 1)
}
}
@@ -241,6 +287,69 @@ impl ProviderCooldownPort for RedisCredentialCooldownRepository {
.map_err(|_| provider_unavailable("clear credential cooldown"))
})
}
fn put_scoped_if_later(
&self,
cooldown: ProviderScopedCooldown,
) -> futures::future::BoxFuture<'_, Result<bool, ProviderStoreError>> {
Box::pin(async move {
let revision = Revision::new(cooldown.credential_revision().get())
.map_err(|_| provider_invalid("encode scoped credential cooldown"))?;
self.cache_at_key(
self.scoped_key(cooldown.account_id().as_str(), cooldown.scope())
.map_err(|_| provider_invalid("encode scoped credential cooldown"))?,
revision,
cooldown.until().into(),
)
.await
.map_err(|_| provider_unavailable("cache scoped credential cooldown"))
})
}
fn read_scoped<'a>(
&'a self,
account_id: &'a ProviderAccountId,
scope: &'a ProviderCooldownScope,
) -> futures::future::BoxFuture<'a, Result<Option<ProviderScopedCooldown>, ProviderStoreError>>
{
Box::pin(async move {
self.read_at_key(
self.scoped_key(account_id.as_str(), scope)
.map_err(|_| provider_invalid("encode scoped credential cooldown"))?,
)
.await
.map_err(|_| provider_unavailable("read scoped credential cooldown"))?
.map(|(revision, until)| {
Ok(ProviderScopedCooldown::new(
account_id.clone(),
CredentialRevision::new(revision.get())
.map_err(|_| provider_invalid("decode scoped credential cooldown"))?,
scope.clone(),
until.into(),
))
})
.transpose()
})
}
fn clear_scoped<'a>(
&'a self,
account_id: &'a ProviderAccountId,
scope: &'a ProviderCooldownScope,
through_revision: CredentialRevision,
) -> futures::future::BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async move {
let revision = Revision::new(through_revision.get())
.map_err(|_| provider_invalid("encode scoped cooldown revision"))?;
self.invalidate_at_key(
self.scoped_key(account_id.as_str(), scope)
.map_err(|_| provider_invalid("encode scoped credential cooldown"))?,
revision,
)
.await
.map_err(|_| provider_unavailable("clear scoped credential cooldown"))
})
}
}
/// 账号状态写入到可丢失 cooldown cache 的桥接 adapter。
@@ -348,6 +457,13 @@ impl ProviderAccountStore for CooldownCachingProviderAccountStore {
.await
}
async fn load_current_credential(
&self,
account: &ProviderAccountId,
) -> Result<LoadedCredential, CoreStoreError> {
self.authoritative.load_current_credential(account).await
}
async fn compare_and_swap_credential(
&self,
update: CredentialCasUpdate,
@@ -388,6 +504,12 @@ impl ProviderAccountStore for CooldownCachingProviderAccountStore {
let availability = change.availability;
let cooldown_until = change.cooldown_until;
self.authoritative.apply_state_change(change).await?;
let Some(account) = self.authoritative.get_account(&account_id).await? else {
return Ok(());
};
if !account.enabled() {
return Ok(());
}
self.mirror(
&account_id,
credential_revision,
@@ -55,10 +55,7 @@ impl CredentialStateCache {
"provider_account_id",
&self.provider_account_id,
)?;
if !matches!(
self.availability.as_str(),
"unknown" | "ready" | "cooldown" | "quota_exhausted" | "expired" | "banned" | "invalid"
) {
if AccountAvailability::parse(&self.availability).is_none() {
return Err(invalid("availability is not registered"));
}
Ok(())
@@ -8,6 +8,7 @@ use sha2::{Digest, Sha256};
use async_trait::async_trait;
mod client_admission;
mod coordination_buffer;
mod credential_cooldown;
mod credential_leases;
mod credential_state;
@@ -20,6 +21,7 @@ mod runtime_change;
pub(crate) mod worker_lease;
pub use client_admission::*;
pub use coordination_buffer::*;
pub use credential_cooldown::*;
pub use credential_leases::*;
pub use credential_state::*;
@@ -12,7 +12,6 @@ use gateway_core::engine::continuation::{
NativeContinuationStoreError, PreviousResponseId,
};
use gateway_core::engine::credential::ProviderAccountId;
use gateway_core::error::SafeUpstreamValue;
use gateway_core::operation::ProviderSessionState;
use gateway_core::routing::ProviderKind;
use redis::aio::ConnectionManager;
@@ -25,6 +24,7 @@ use super::{namespace, resource_fingerprint};
const CONTINUATION_TTL: Duration = Duration::from_secs(4 * 60 * 60);
const MAX_CONTINUATIONS: usize = 65_536;
const MAX_RECORD_BYTES: usize = 64 * 1024;
const MAX_INDEX_CLEANUP_PER_RECORD: usize = 64;
const RECORD_SCRIPT: &str = r#"
local entry_key = KEYS[1]
@@ -36,12 +36,13 @@ local ttl_seconds = tonumber(ARGV[4])
local cutoff = tonumber(ARGV[5])
local max_entries = tonumber(ARGV[6])
local entry_prefix = ARGV[7]
local cleanup_limit = tonumber(ARGV[8])
redis.call('SET', entry_key, payload, 'EX', ttl_seconds)
redis.call('ZADD', index_key, score, entry_id)
redis.call('EXPIRE', index_key, ttl_seconds)
local stale = redis.call('ZRANGEBYSCORE', index_key, '-inf', cutoff)
local stale = redis.call('ZRANGEBYSCORE', index_key, '-inf', cutoff, 'LIMIT', 0, cleanup_limit)
for _, stale_id in ipairs(stale) do
redis.call('UNLINK', entry_prefix .. '{' .. stale_id .. '}')
redis.call('ZREM', index_key, stale_id)
@@ -49,6 +50,9 @@ end
local count = redis.call('ZCARD', index_key)
local excess = count - max_entries
if excess > cleanup_limit then
excess = cleanup_limit
end
if excess > 0 then
local oldest = redis.call('ZRANGE', index_key, 0, excess - 1)
for _, old_id in ipairs(oldest) do
@@ -60,7 +64,7 @@ end
return 1
"#;
/// Redis-backed, best-effort native response continuation affinity.
/// 基于 Redis 的 best-effort 原生响应 continuation 亲和。
#[derive(Clone)]
pub struct RedisNativeContinuationRepository {
connection: ConnectionManager,
@@ -110,6 +114,9 @@ impl NativeContinuationPort for RedisNativeContinuationRepository {
previous_response_id: &'a PreviousResponseId,
) -> BoxFuture<'a, Result<Option<NativeContinuationPin>, NativeContinuationStoreError>> {
Box::pin(async move {
if previous_response_id.as_str().is_empty() {
return Ok(None);
}
let fingerprint = self.entry_fingerprint(previous_response_id.as_str())?;
let mut connection = self.connection.clone();
let payload = redis::cmd("GET")
@@ -128,6 +135,9 @@ impl NativeContinuationPort for RedisNativeContinuationRepository {
pin: NativeContinuationPin,
) -> BoxFuture<'a, Result<(), NativeContinuationStoreError>> {
Box::pin(async move {
if pin.previous_response_id().as_str().is_empty() {
return Ok(());
}
let fingerprint = self.entry_fingerprint(pin.previous_response_id().as_str())?;
let payload = encode_pin(&pin)?;
if payload.len() > MAX_RECORD_BYTES {
@@ -153,6 +163,7 @@ impl NativeContinuationPort for RedisNativeContinuationRepository {
.arg(cutoff)
.arg(MAX_CONTINUATIONS)
.arg(self.entry_prefix())
.arg(MAX_INDEX_CLEANUP_PER_RECORD)
.invoke_async::<i64>(&mut connection)
.await
.map_err(|_| NativeContinuationStoreError)?;
@@ -186,8 +197,7 @@ fn decode_pin(
serde_json::from_str(payload).map_err(|_| NativeContinuationStoreError)?;
let provider = ProviderKind::new(wire.provider).map_err(|_| NativeContinuationStoreError)?;
let account = ProviderAccountId::new(wire.account).map_err(|_| NativeContinuationStoreError)?;
let upstream_response_id = SafeUpstreamValue::new(wire.upstream_response_id)
.map_err(|_| NativeContinuationStoreError)?;
let upstream_response_id = PreviousResponseId::new(wire.upstream_response_id);
let scope = parse_scope(&wire.scope)?;
let mut pin = NativeContinuationPin::new(
previous_response_id.clone(),

Some files were not shown because too many files have changed in this diff Show More