Compare commits

...
67 Commits
Author SHA1 Message Date
ZyyandCodex 4127886584 chore: bump version to 3.1.21
Co-authored-by: Codex <noreply@openai.com>
2026-08-11 17:42:54 +08:00
ZyyandCodex ee3e5db6c2 fix(accounts): isolate model-specific quota usage
Co-authored-by: Codex <noreply@openai.com>
2026-08-11 17:42:02 +08:00
ZyyandCodex 153cb67b9c refactor(login): refine panel details
Co-authored-by: Codex <noreply@openai.com>
2026-08-11 17:21:31 +08:00
ZyyandCodex f370d355b6 fix(accounts): align quota ordering and summaries
Co-authored-by: Codex <noreply@openai.com>
2026-08-11 17:21:23 +08:00
ZyyandCodex a31dad992c fix(frontend): stabilize button loading indicator
Co-authored-by: Codex <noreply@openai.com>
2026-08-11 17:21:16 +08:00
ZyyandCodex 55a88f5f6b chore: bump version to 3.1.20
Co-authored-by: Codex <noreply@openai.com>
2026-08-11 15:13:40 +08:00
ZyyandCodex 5d90a92582 fix(logging): rename OAuth recovery log file
Co-authored-by: Codex <noreply@openai.com>
2026-08-11 15:06:40 +08:00
ZyyandCodex 93abe8efcd fix(logging): generalize OAuth recovery logs
Co-authored-by: Codex <noreply@openai.com>
2026-08-11 14:55:59 +08:00
ZyyandCodex f4b75c42ed fix(openai): isolate quota and recovery signals
Co-authored-by: Codex <noreply@openai.com>
2026-08-11 13:37:16 +08:00
ZyyandCodex 36ff038c78 chore: bump version to 3.1.19
Co-authored-by: Codex <noreply@openai.com>
2026-08-11 12:17:28 +08:00
ZyyandCodex 545009590c fix(openai): persist authoritative quota exhaustion
Co-authored-by: Codex <noreply@openai.com>
2026-08-11 12:16:34 +08:00
ZyyandCodex 5b7b9b21ee fix(openai): require low usage for quota recovery
Co-authored-by: Codex <noreply@openai.com>
2026-08-10 15:55:58 +08:00
ZyyandCodex 44a2a87b69 fix(frontend): center button loading spinner
Co-authored-by: Codex <noreply@openai.com>
2026-08-10 15:55:58 +08:00
ZyyandCodex 184e88083b chore: bump version to 3.1.18
Co-authored-by: Codex <noreply@openai.com>
2026-08-10 12:25:42 +08:00
ZyyandCodex 53b40aea46 chore(deps): ignore fingerprint-sensitive updates
Co-authored-by: Codex <noreply@openai.com>
2026-08-10 12:23:45 +08:00
ZyyandCodex aa328b6dbf fix(openai): restore official TLS fingerprint pins
Co-authored-by: Codex <noreply@openai.com>
2026-08-10 12:23:36 +08:00
dependabot[bot]andZyy 64231ff403 chore(deps): bump the minor-and-patch group in /backend with 4 updates (#37)
Bumps the minor-and-patch group in /backend with 4 updates: [base64](https://github.com/marshallpierce/rust-base64), [aws-sdk-s3](https://github.com/awslabs/aws-sdk-rust), [hyper-rustls](https://github.com/rustls/hyper-rustls) and [rustls](https://github.com/rustls/rustls).


Updates `base64` from 0.23.0 to 0.23.1
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md)
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.23.0...v0.23.1)

Updates `aws-sdk-s3` from 1.140.0 to 1.141.0
- [Release notes](https://github.com/awslabs/aws-sdk-rust/releases)
- [Commits](https://github.com/awslabs/aws-sdk-rust/commits)

Updates `hyper-rustls` from 0.27.7 to 0.27.9
- [Release notes](https://github.com/rustls/hyper-rustls/releases)
- [Commits](https://github.com/rustls/hyper-rustls/compare/v/0.27.7...v/0.27.9)

Updates `rustls` from 0.23.36 to 0.23.43
- [Release notes](https://github.com/rustls/rustls/releases)
- [Changelog](https://github.com/rustls/rustls/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rustls/rustls/compare/v/0.23.36...v/0.23.43)

---
updated-dependencies:
- dependency-name: base64
  dependency-version: 0.23.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: aws-sdk-s3
  dependency-version: 1.141.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: hyper-rustls
  dependency-version: 0.27.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: rustls
  dependency-version: 0.23.43
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zyy <39424122+zyycn@users.noreply.github.com>
2026-08-10 12:05:00 +08:00
dependabot[bot]andZyy d7b80a2066 chore(deps): bump the minor-and-patch group with 5 updates (#36)
Bumps the minor-and-patch group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [github/codeql-action/upload-sarif](https://github.com/github/codeql-action) | `4.37.4` | `4.37.6` |
| [Swatinem/rust-cache](https://github.com/swatinem/rust-cache) | `2.9.1` | `2.9.2` |
| [pnpm/action-setup](https://github.com/pnpm/action-setup) | `6.0.9` | `6.0.10` |
| [dorny/paths-filter](https://github.com/dorny/paths-filter) | `4.0.2` | `4.0.3` |
| [actions/attest](https://github.com/actions/attest) | `4.2.1` | `4.2.2` |


Updates `github/codeql-action/upload-sarif` from 4.37.4 to 4.37.6
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/f205ea1c3313d32999d8d6a48b4f6530d4437b38...5595ccaf912efad79be6eef63a5619ff05969be3)

Updates `Swatinem/rust-cache` from 2.9.1 to 2.9.2
- [Release notes](https://github.com/swatinem/rust-cache/releases)
- [Changelog](https://github.com/Swatinem/rust-cache/blob/master/CHANGELOG.md)
- [Commits](https://github.com/swatinem/rust-cache/compare/c19371144df3bb44fab255c43d04cbc2ab54d1c4...6323deb102c322ba6fcbdcafc7e3dddab59af2b6)

Updates `pnpm/action-setup` from 6.0.9 to 6.0.10
- [Release notes](https://github.com/pnpm/action-setup/releases)
- [Commits](https://github.com/pnpm/action-setup/compare/0ebf47130e4866e96fce0953f49152a61190b271...0977fd99725f1db4007ccb2928dbb4e90d06cc86)

Updates `dorny/paths-filter` from 4.0.2 to 4.0.3
- [Release notes](https://github.com/dorny/paths-filter/releases)
- [Changelog](https://github.com/dorny/paths-filter/blob/master/CHANGELOG.md)
- [Commits](https://github.com/dorny/paths-filter/compare/7b450fff21473bca461d4b92ce414b9d0420d706...ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d)

Updates `actions/attest` from 4.2.1 to 4.2.2
- [Release notes](https://github.com/actions/attest/releases)
- [Changelog](https://github.com/actions/attest/blob/main/RELEASE.md)
- [Commits](https://github.com/actions/attest/compare/508db95dd578ae2727ebd6217d5ba78e4fbda05d...1e69f48acb82d1966a394da916b4c1698aa569d6)

---
updated-dependencies:
- dependency-name: github/codeql-action/upload-sarif
  dependency-version: 4.37.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: Swatinem/rust-cache
  dependency-version: 2.9.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: pnpm/action-setup
  dependency-version: 6.0.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: dorny/paths-filter
  dependency-version: 4.0.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: actions/attest
  dependency-version: 4.2.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zyy <39424122+zyycn@users.noreply.github.com>
2026-08-10 12:01:38 +08:00
dependabot[bot] 08d63ea3f3 chore(deps): bump debian from 7b140f3 to abd67ff in /deploy (#39)
Bumps debian from `7b140f3` to `abd67ff`.

---
updated-dependencies:
- dependency-name: debian
  dependency-version: bookworm-slim
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-10 11:58:47 +08:00
dependabot[bot] ac9c4c85ad chore(deps): bump the minor-and-patch group in /frontend with 8 updates (#38)
Bumps the minor-and-patch group in /frontend with 8 updates:

| Package | From | To |
| --- | --- | --- |
| [@lucide/vue](https://github.com/lucide-icons/lucide/tree/HEAD/packages/vue) | `1.28.0` | `1.30.0` |
| [dompurify](https://github.com/cure53/DOMPurify) | `3.4.12` | `3.4.13` |
| [marked](https://github.com/markedjs/marked) | `18.0.7` | `18.0.9` |
| [vue](https://github.com/vuejs/core) | `3.5.40` | `3.5.41` |
| [@antfu/eslint-config](https://github.com/antfu/eslint-config) | `9.2.0` | `9.3.0` |
| [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) | `26.1.2` | `26.2.0` |
| [eslint](https://github.com/eslint/eslint) | `10.8.0` | `10.8.1` |
| [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) | `8.2.0` | `8.2.1` |


Updates `@lucide/vue` from 1.28.0 to 1.30.0
- [Release notes](https://github.com/lucide-icons/lucide/releases)
- [Commits](https://github.com/lucide-icons/lucide/commits/1.30.0/packages/vue)

Updates `dompurify` from 3.4.12 to 3.4.13
- [Release notes](https://github.com/cure53/DOMPurify/releases)
- [Commits](https://github.com/cure53/DOMPurify/compare/3.4.12...3.4.13)

Updates `marked` from 18.0.7 to 18.0.9
- [Release notes](https://github.com/markedjs/marked/releases)
- [Commits](https://github.com/markedjs/marked/compare/v18.0.7...v18.0.9)

Updates `vue` from 3.5.40 to 3.5.41
- [Release notes](https://github.com/vuejs/core/releases)
- [Changelog](https://github.com/vuejs/core/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vuejs/core/compare/v3.5.40...v3.5.41)

Updates `@antfu/eslint-config` from 9.2.0 to 9.3.0
- [Release notes](https://github.com/antfu/eslint-config/releases)
- [Commits](https://github.com/antfu/eslint-config/compare/v9.2.0...v9.3.0)

Updates `@types/node` from 26.1.2 to 26.2.0
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

Updates `eslint` from 10.8.0 to 10.8.1
- [Release notes](https://github.com/eslint/eslint/releases)
- [Commits](https://github.com/eslint/eslint/compare/v10.8.0...v10.8.1)

Updates `vite` from 8.2.0 to 8.2.1
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.2.1/packages/vite)

---
updated-dependencies:
- dependency-name: "@lucide/vue"
  dependency-version: 1.30.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: dompurify
  dependency-version: 3.4.13
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: marked
  dependency-version: 18.0.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: vue
  dependency-version: 3.5.41
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: "@antfu/eslint-config"
  dependency-version: 9.3.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: "@types/node"
  dependency-version: 26.2.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: eslint
  dependency-version: 10.8.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: vite
  dependency-version: 8.2.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-10 11:54:36 +08:00
ZyyandCodex 492c401599 fix(openai): preserve state on quota auth failures
Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Zyy <zyycn@users.noreply.github.com>
2026-08-10 11:46:19 +08:00
ZyyandCodex a063258a40 chore: bump version to 3.1.17
Co-authored-by: Codex <noreply@openai.com>
2026-08-09 15:14:06 +08:00
ZyyandCodex dc590f2b5f fix: make OAuth refresh margin runtime-only
Co-authored-by: Codex <noreply@openai.com>
2026-08-09 15:12:43 +08:00
ZyyandCodex ca52e2bfd9 chore: bump version to 3.1.16
Co-authored-by: Codex <noreply@openai.com>
2026-08-09 11:59:08 +08:00
ZyyandCodex 43604e36dc fix: correct first token timing
Co-authored-by: Codex <noreply@openai.com>
2026-08-07 18:56:13 +08:00
ZyyandCodex a4a997e351 fix: use lifetime dashboard card totals
Co-authored-by: Codex <noreply@openai.com>
2026-08-07 18:56:13 +08:00
ZyyandCodex 73ca463924 feat: distinguish pro plan badges
Co-authored-by: Codex <noreply@openai.com>
2026-08-07 18:56:13 +08:00
ZyyandCodex 9eeec2c78c chore: bump version to 3.1.15
Co-authored-by: Codex <noreply@openai.com>
2026-08-07 15:15:00 +08:00
ZyyandCodex 90890be2b4 fix(openai): persist OAuth tokens before quota sync
Align OAuth token persistence and ID token metadata handling with Codex, and retry initial quota observation asynchronously.

Co-authored-by: Codex <noreply@openai.com>
2026-08-07 14:59:42 +08:00
ZyyandCodex 2e4bab17a8 chore: bump version to 3.1.14
Co-authored-by: Codex <noreply@openai.com>
2026-08-06 16:01:07 +08:00
ZyyandCodex b052e79a16 fix(openai): preserve imported token expiry
Co-authored-by: Codex <noreply@openai.com>
2026-08-06 15:49:42 +08:00
ZyyandCodex 9775451e83 fix(web): preserve button labels while loading
Co-authored-by: Codex <noreply@openai.com>
2026-08-06 15:27:00 +08:00
ZyyandCodex c22a2ad144 test(logging): move recovery filter coverage
Co-authored-by: Codex <noreply@openai.com>
2026-08-06 15:14:26 +08:00
ZyyandCodex 7bd362d544 chore: bump version to 3.1.13
Co-authored-by: Codex <noreply@openai.com>
2026-08-06 14:56:08 +08:00
ZyyandCodex e7036a616f fix(web): center button loading indicators
Co-authored-by: Codex <noreply@openai.com>
2026-08-06 14:53:53 +08:00
ZyyandCodex 98652d4c68 fix(openai): write oauth recovery to app logs
Co-authored-by: Codex <noreply@openai.com>
2026-08-06 14:22:13 +08:00
ZyyandCodex d0dc7ef92b fix(openai): complete import profile from access token
Co-authored-by: Codex <noreply@openai.com>
2026-08-06 12:31:13 +08:00
ZyyandCodex 194fb20d5f chore: bump version to 3.1.12
Co-authored-by: Codex <noreply@openai.com>
2026-08-06 11:14:32 +08:00
ZyyandCodex 30f528f084 fix(openai): simplify import and oauth validation
Co-authored-by: Codex <noreply@openai.com>
2026-08-06 11:14:18 +08:00
ZyyandCodex 28d024ceda chore: bump version to 3.1.11
Co-authored-by: Codex <noreply@openai.com>
2026-08-06 09:54:05 +08:00
ZyyandCodex 6b9c98c59f fix(openai): persist successful refresh tokens directly
Co-authored-by: Codex <noreply@openai.com>
2026-08-06 09:50:03 +08:00
ZyyandCodex 1294e50b76 chore: bump version to 3.1.10
Co-authored-by: Codex <noreply@openai.com>
2026-08-05 12:22:46 +08:00
ZyyandCodex a0e540642b fix(oauth): bound refresh recovery and persist failure reasons
Co-authored-by: Codex <noreply@openai.com>
2026-08-05 12:21:16 +08:00
ZyyandCodex 14d6a7de31 fix(dashboard): align summary with daily trend
Co-authored-by: Codex <noreply@openai.com>
2026-08-05 10:46:14 +08:00
ZyyandCodex cd7b60a3ba chore: bump version to 3.1.9
Co-authored-by: Codex <noreply@openai.com>
2026-08-05 00:03:39 +08:00
ZyyandCodex ebbed22f36 fix(openai): keep Codex wire OS exact
Co-authored-by: Codex <noreply@openai.com>
2026-08-05 00:03:21 +08:00
ZyyandCodex f6e5d8f673 fix(openai): drop null secondary quota windows
Co-authored-by: Codex <noreply@openai.com>
2026-08-05 00:03:11 +08:00
ZyyandCodex bf1c6211b8 chore: bump version to 3.1.8
Co-authored-by: Codex <noreply@openai.com>
2026-08-04 22:30:40 +08:00
ZyyandCodex d388fa344a fix(openai): drop placeholder secondary quota windows
Co-authored-by: Codex <noreply@openai.com>
2026-08-04 22:30:04 +08:00
ZyyandCodex eabb61b08c fix(openai): restore Codex Core app-server identity
Co-authored-by: Codex <noreply@openai.com>
2026-08-04 22:30:01 +08:00
ZyyandCodex a5c3438927 fix(openai): isolate thread-spawn session affinity
Co-authored-by: Codex <noreply@openai.com>
2026-08-04 22:29:53 +08:00
ZyyandCodex b6948542bb fix: align Codex Desktop HTTP identity
- send the official Desktop HTTP user agent across OpenAI upstream paths
- keep the audited model catalog version separate from the wire identity
- remove CLI release drift and clarify the dashboard client identity

Co-authored-by: Codex <noreply@openai.com>
2026-08-04 18:36:03 +08:00
ZyyandCodex 0242bd3193 chore: bump version to 3.1.7
Co-authored-by: Codex <noreply@openai.com>
2026-08-04 16:19:58 +08:00
ZyyandCodex 229ef48470 fix: harden backup verification and detail UI
- read every multipart backup payload to its requested length
- make remote verification failures diagnosable
- refine status details, loading icons, and attempt-chain presentation

Co-authored-by: Codex <noreply@openai.com>
2026-08-04 16:18:53 +08:00
ZyyandCodex 19c35ba901 chore: bump version to 3.1.6
Co-authored-by: Codex <noreply@openai.com>
2026-08-04 15:12:29 +08:00
ZyyandCodex 3c06ae97e8 fix: reconcile quota, backup, and session behavior
- preserve confirmed quota exhaustion until its recorded reset advances
- normalize account quota presentation and subagent session affinity
- repair S3-compatible backup handling and related admin UI details

Co-authored-by: Codex <noreply@openai.com>
2026-08-04 15:12:15 +08:00
ZyyandCodex f34966e4c7 chore: bump version to 3.1.5
Co-authored-by: Codex <noreply@openai.com>
2026-08-04 08:54:00 +08:00
ZyyandCodex e9ebf45640 fix(accounts): reconcile quota status and dashboard metrics
Co-authored-by: Codex <noreply@openai.com>
2026-08-04 08:53:29 +08:00
ZyyandCodex c00e5bd243 chore: bump version to 3.1.4
Co-authored-by: Codex <noreply@openai.com>
2026-08-04 01:10:51 +08:00
ZyyandCodex ab5f18a437 fix(accounts): update admin list test for cooldown-driven rate_limited
quota_limit_reached without Redis cooldown maps to quota_exhausted
now that rate_limited comes only from 429 cooldown. Adjust summary
assertions: quota_exhausted 1→2, rate_limited 1→0.

Co-authored-by: Codex <noreply@openai.com>
2026-08-04 01:10:40 +08:00
ZyyandClaude Fable 5 330b222359 feat(accounts): separate rate-limited status into Redis cooldown
429 临时限流从 quota 快照分离为 Redis 冷却(ProviderCooldownPort):
- apply_rate_limit_429 只写 Redis(put_if_later,不缩短已有冷却),不再合成 quota 窗口,额度重置时间不被 429 污染
- 调度选择时排除冷却中的账号,到期自动恢复;诊断请求绕过
- 状态派生分离:rate_limited_until 存在 → 限流中;quota_limit_reached → 配额耗尽
- store 层(账号列表/筛选 + dashboard poolSummary)注入 cooldown 端口,精确区分限流中与配额耗尽,"正常"筛选不再混入限流中
- wire 透传 rate_limited_until,前端状态弹窗改为状态卡片(icon 块 + 状态名 + 原因/到期明细),BasePopover 加 hover delay
- 更新 429 相关测试断言,新增配额耗尽映射测试

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-04 00:39:38 +08:00
ZyyandCodex d81cb2d4f0 chore: bump version to 3.1.3
Co-authored-by: Codex <noreply@openai.com>
2026-08-03 21:29:16 +08:00
ZyyandClaude Opus 4.8 8e2edb680d feat(accounts): converge status classification and persist upstream errors
Unify account status derivation in gateway-admin via AccountStatusSignals,
removing the duplicated rules in store and use_case. AccountStatus now has
five operational classes (normal/quota_exhausted/rate_limited/disabled/error),
with AccountErrorReason carrying the concrete cause.

Add provider_accounts.last_error_message, written by record_failure with the
raw upstream error and cleared on recovery to ready. The admin error badge
shows the cause and upstream text on hover.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-03 21:23:28 +08:00
ZyyandCodex 00879be47d fix(accounts): separate rate-limited account status
Co-authored-by: Codex <noreply@openai.com>
2026-08-03 17:01:23 +08:00
ZyyandCodex 34656a3f07 refactor(ui): standardize card spacing and layouts
Co-authored-by: Codex <noreply@openai.com>
2026-08-03 17:01:23 +08:00
ZyyandCodex 27cf037309 fix(ui): center button loading spinner
Co-authored-by: Codex <noreply@openai.com>
2026-08-03 17:01:23 +08:00
ZyyandCodex 2198fa69ab refactor(migrations): consolidate schema into initial baseline
Co-authored-by: Codex <noreply@openai.com>
2026-08-03 17:01:23 +08:00
194 changed files with 8442 additions and 7115 deletions
+5
View File
@@ -21,6 +21,11 @@ updates:
day: monday
time: '09:10'
timezone: Asia/Shanghai
# 固定官方 Codex Desktop 的 TLS 指纹;升级前必须抓取握手并比对扩展顺序。
ignore:
- dependency-name: hyper-rustls
- dependency-name: reqwest
- dependency-name: rustls
groups:
minor-and-patch:
update-types:
+1 -1
View File
@@ -76,7 +76,7 @@ jobs:
${{ hashFiles('trivy-results.sarif') != '' &&
(github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository) }}
uses: github/codeql-action/upload-sarif@f205ea1c3313d32999d8d6a48b4f6530d4437b38 # v4.37.4
uses: github/codeql-action/upload-sarif@5595ccaf912efad79be6eef63a5619ff05969be3 # v4.37.6
with:
sarif_file: trivy-results.sarif
category: runtime-container
+2 -2
View File
@@ -106,7 +106,7 @@ jobs:
components: rustfmt,clippy
- name: Cache Rust
uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
workspaces: backend -> target
@@ -157,7 +157,7 @@ jobs:
ref: ${{ inputs.ref }}
- name: Setup pnpm
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
version: 11.7.0
+1 -1
View File
@@ -32,7 +32,7 @@ jobs:
- name: Detect affected checks
id: filter
uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2
uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v4.0.3
with:
filters: |
backend:
+2 -2
View File
@@ -375,7 +375,7 @@ jobs:
PY
- name: Generate release asset provenance
uses: actions/attest@508db95dd578ae2727ebd6217d5ba78e4fbda05d # v4.2.1
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-path: dist/release/*
@@ -581,7 +581,7 @@ jobs:
echo "digest=${digest}" >> "$GITHUB_OUTPUT"
- name: Generate container provenance attestation
uses: actions/attest@508db95dd578ae2727ebd6217d5ba78e4fbda05d # v4.2.1
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
subject-digest: ${{ steps.manifest.outputs.digest }}
+2 -2
View File
@@ -31,7 +31,7 @@ jobs:
uses: dtolnay/rust-toolchain@1a3a6d54512beeaffd394f8d516ca16f2c506a20 # 1.97
- name: Cache Rust
uses: Swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2.9.1
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
with:
workspaces: backend -> target
@@ -52,7 +52,7 @@ jobs:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Setup pnpm
uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
version: 11.7.0
+10 -44
View File
@@ -216,9 +216,9 @@ dependencies = [
[[package]]
name = "aws-sdk-s3"
version = "1.140.0"
version = "1.141.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e9660cf991e512fbe6094f1041ff3d3282bc5aeeeb6184e8de437ec2de024a10"
checksum = "d9f9420d3a2467eed22ed3635ca653653162c386a0b0f65c78189f9bd3c1379e"
dependencies = [
"arc-swap",
"aws-credential-types",
@@ -578,9 +578,9 @@ checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
[[package]]
name = "base64"
version = "0.23.0"
version = "0.23.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b25655df2c3cdd83c5e5b293b88acd880332b2ddadd7c30ac43144fdc0033da9"
checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
[[package]]
name = "base64-simd"
@@ -1493,7 +1493,7 @@ version = "0.0.0"
dependencies = [
"argon2",
"async-trait",
"base64 0.23.0",
"base64 0.23.1",
"chrono",
"chrono-tz",
"cron",
@@ -1519,7 +1519,7 @@ version = "0.1.0"
dependencies = [
"async-trait",
"axum",
"base64 0.23.0",
"base64 0.23.1",
"bytes",
"chrono",
"futures",
@@ -2125,11 +2125,9 @@ dependencies = [
"base64 0.22.1",
"getrandom 0.2.17",
"js-sys",
"pem",
"serde",
"serde_json",
"signature 2.2.0",
"simple_asn1",
"zeroize",
]
@@ -2280,16 +2278,6 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "num-bigint"
version = "0.4.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c89e69e7e0f03bea5ef08013795c25018e101932225a656383bd384495ecc367"
dependencies = [
"num-integer",
"num-traits",
]
[[package]]
name = "num-conv"
version = "0.2.2"
@@ -2407,16 +2395,6 @@ version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df94ce210e5bc13cb6651479fa48d14f601d9858cfe0467f43ae157023b938d3"
[[package]]
name = "pem"
version = "3.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d30c53c26bc5b31a98cd02d20f25a7c8567146caf63ed593a9d87b2775291be"
dependencies = [
"base64 0.22.1",
"serde_core",
]
[[package]]
name = "pem-rfc7468"
version = "1.0.0"
@@ -2597,7 +2575,7 @@ version = "0.0.0"
dependencies = [
"async-stream",
"async-trait",
"base64 0.23.0",
"base64 0.23.1",
"bytes",
"chrono",
"cookie",
@@ -2610,7 +2588,6 @@ dependencies = [
"getrandom 0.4.3",
"hex",
"hyper-rustls",
"jsonwebtoken",
"reqwest",
"roxmltree",
"rustls",
@@ -2628,6 +2605,7 @@ dependencies = [
"tokio-tungstenite 0.28.0",
"tokio-util",
"tracing",
"tracing-subscriber",
"tungstenite 0.27.0",
"url",
"uuid",
@@ -2641,7 +2619,7 @@ version = "0.0.0"
dependencies = [
"async-stream",
"async-trait",
"base64 0.23.0",
"base64 0.23.1",
"bytes",
"chrono",
"futures",
@@ -3323,18 +3301,6 @@ version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "703d5c7ef118737c72f1af64ad2f6f8c5e1921f818cdcb97b8fe6fc69bf66214"
[[package]]
name = "simple_asn1"
version = "0.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d585997b0ac10be3c5ee635f1bab02d512760d14b7c468801ac8a01d9ae5f1d"
dependencies = [
"num-bigint",
"num-traits",
"thiserror",
"time",
]
[[package]]
name = "siphasher"
version = "1.0.3"
@@ -3657,7 +3623,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom 0.4.3",
"getrandom 0.3.4",
"once_cell",
"rustix",
"windows-sys 0.61.2",
+1 -1
View File
@@ -134,7 +134,7 @@ fn config_loader_should_reject_removed_fingerprint_section() {
}
#[test]
fn config_loader_should_reject_invalid_codex_core_version() {
fn config_loader_should_reject_invalid_codex_cli_version() {
assert_rejected(valid_config().replace("codex_version: '0.146.0'", "codex_version: 'latest'"));
}
+59 -13
View File
@@ -220,7 +220,8 @@ impl BackupTask {
sha256: Some(artifact.sha256.clone()),
..Default::default()
};
self.repository
let Some(uploading_record) = self
.repository
.transition_status(
&record.id,
BackupStatus::Dumping,
@@ -229,9 +230,12 @@ impl BackupTask {
now,
)
.await
.map_err(repo_error)?;
.map_err(repo_error)?
else {
return Ok(());
};
self.upload_and_verify(
record,
&uploading_record,
&artifact.path,
artifact.size_bytes,
&artifact.sha256,
@@ -394,7 +398,8 @@ impl BackupTask {
sha256: Some(artifact.sha256.clone()),
..Default::default()
};
self.repository
let Some(uploading_record) = self
.repository
.transition_status(
&record.id,
BackupStatus::Dumping,
@@ -403,10 +408,13 @@ impl BackupTask {
now,
)
.await
.map_err(repo_error)?;
.map_err(repo_error)?
else {
return Ok(());
};
self.upload_and_verify(
record,
&uploading_record,
&artifact.path,
artifact.size_bytes,
&artifact.sha256,
@@ -448,8 +456,12 @@ impl BackupTask {
.await
{
let _ = store.delete_object(&config, &record.object_key).await;
self.fail_task(record, error.code(), "上传到对象存储失败")
.await?;
self.fail_task(
record,
error.code(),
format!("上传到对象存储失败:{}", error.message()),
)
.await?;
return Ok(());
}
@@ -475,19 +487,46 @@ impl BackupTask {
);
}
}
Ok(_) => {
Ok(Some(remote)) => {
let size_matches = remote.size_bytes == size_bytes;
let sha256_matches = remote.sha256 == sha256;
warn!(
backup_id = %record.id,
expected_size_bytes = size_bytes,
remote_size_bytes = remote.size_bytes,
sha256_matches,
"备份远端对象校验不一致"
);
let _ = store.delete_object(&config, &record.object_key).await;
self.fail_task(
record,
code::REMOTE_VERIFICATION_FAILED,
"远端对象大小或校验值不一致",
match (size_matches, sha256_matches) {
(false, true) => "远端对象大小不一致",
(true, false) => "远端对象 sha256 校验值不一致",
(false, false) => "远端对象大小和 sha256 校验值均不一致",
(true, true) => "远端对象校验状态异常",
},
)
.await?;
}
Ok(None) => {
warn!(backup_id = %record.id, "上传完成后未找到远端对象");
self.fail_task(
record,
code::REMOTE_VERIFICATION_FAILED,
"上传完成后未找到远端对象",
)
.await?;
}
Err(error) => {
let _ = store.delete_object(&config, &record.object_key).await;
self.fail_task(record, error.code(), "远端对象校验失败")
.await?;
self.fail_task(
record,
error.code(),
format!("远端对象校验失败:{}", error.message()),
)
.await?;
}
}
Ok(())
@@ -500,10 +539,17 @@ impl BackupTask {
error_code: &'static str,
message: impl Into<String>,
) -> Result<(), WorkerTaskError> {
let message = message.into();
warn!(
backup_id = %record.id,
error_code,
error_message = %message,
"备份任务失败"
);
let now = Utc::now();
let update = StatusTransitionUpdate {
error_code: Some(error_code.to_string()),
error_message: Some(message.into()),
error_message: Some(message),
completed_at: Some(now),
..Default::default()
};
@@ -1,6 +1,6 @@
//! 多 Provider 账号目录与连接测试的公共事实。
use std::pin::Pin;
use std::{pin::Pin, time::SystemTime};
use chrono::{DateTime, Utc};
use futures::Stream;
@@ -13,15 +13,146 @@ use super::{PageSize, Revision, observability::TimeRange};
pub use gateway_core::engine::credential::AccountAvailability;
/// 管理页使用的归一化账号状态。
///
/// 只回答两个问题:是否可调度、是否需要人工干预。具体错误原因不进入此枚举,
/// 由 [`AccountErrorReason`] 承载;新增原因只扩展 reason,不改变分类。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AccountStatus {
Active,
RateLimited,
/// 正常,可调度。
Normal,
/// 配额耗尽(持久化终态),等待额度恢复,无需干预。
QuotaExhausted,
Expired,
Invalid,
/// 限流中(quota 快照级信号),自动恢复,无需干预。
RateLimited,
/// 管理员手动停用(`enabled = false`)。
Disabled,
/// 错误(过期 / 失效 / 封禁等),需要人工干预。具体原因见 [`AccountErrorReason`]。
Error,
}
impl AccountStatus {
/// 返回稳定 wire 值。
#[must_use]
pub const fn as_str(self) -> &'static str {
match self {
Self::Normal => "normal",
Self::QuotaExhausted => "quota_exhausted",
Self::RateLimited => "rate_limited",
Self::Disabled => "disabled",
Self::Error => "error",
}
}
/// 解析稳定 wire 值。
#[must_use]
pub fn parse(value: &str) -> Option<Self> {
match value {
"normal" => Some(Self::Normal),
"quota_exhausted" => Some(Self::QuotaExhausted),
"rate_limited" => Some(Self::RateLimited),
"disabled" => Some(Self::Disabled),
"error" => Some(Self::Error),
_ => None,
}
}
}
/// 状态派生所需的最小事实;由 store / use_case 各自构造,派生规则唯一。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct AccountStatusSignals {
pub enabled: bool,
pub availability: AccountAvailability,
pub access_token_expired: bool,
pub quota_limit_reached: bool,
/// 429 临时限流(Redis 冷却)到期时间;`Some` 表示账号当前处于临时限流。
pub rate_limited_until: Option<SystemTime>,
}
impl AccountStatusSignals {
/// 派生产出运营分类。终态优先于瞬时信号:停用 > 配额耗尽 > 错误 > 限流。
#[must_use]
pub const fn derive(self) -> AccountStatus {
if !self.enabled {
AccountStatus::Disabled
} else if matches!(self.availability, AccountAvailability::QuotaExhausted) {
AccountStatus::QuotaExhausted
} else if self.access_token_expired
|| matches!(
self.availability,
AccountAvailability::Expired
| AccountAvailability::Invalid
| AccountAvailability::Banned
| AccountAvailability::Unknown
)
{
AccountStatus::Error
} else if self.rate_limited_until.is_some() {
AccountStatus::RateLimited
} else if self.quota_limit_reached {
AccountStatus::QuotaExhausted
} else {
AccountStatus::Normal
}
}
}
/// 错误分类下的具体原因;仅在 `AccountStatus::Error` 时存在。
///
/// 受控枚举用于展示与聚合,扩展不改变状态分类。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AccountErrorReason {
Expired,
TokenExpired,
Invalid,
Banned,
Unknown,
}
impl AccountErrorReason {
/// 返回稳定 wire 值。
#[must_use]
pub const fn as_str(self) -> &'static str {
match self {
Self::Expired => "expired",
Self::TokenExpired => "token_expired",
Self::Invalid => "invalid",
Self::Banned => "banned",
Self::Unknown => "unknown",
}
}
/// 解析稳定 wire 值。
#[must_use]
pub fn parse(value: &str) -> Option<Self> {
match value {
"expired" => Some(Self::Expired),
"token_expired" => Some(Self::TokenExpired),
"invalid" => Some(Self::Invalid),
"banned" => Some(Self::Banned),
"unknown" => Some(Self::Unknown),
_ => None,
}
}
}
/// 由信号派生错误原因;非错误分类(正常 / 配额耗尽 / 限流 / 停用)返回 `None`。
///
/// 与 [`AccountStatusSignals::derive`] 保持同一判定:`Error` 当且仅当此处返回 `Some`。
#[must_use]
pub fn derive_error_reason(signals: &AccountStatusSignals) -> Option<AccountErrorReason> {
if !signals.enabled {
return None;
}
match signals.availability {
AccountAvailability::QuotaExhausted => None,
AccountAvailability::Ready => signals
.access_token_expired
.then_some(AccountErrorReason::TokenExpired),
AccountAvailability::Expired => Some(AccountErrorReason::Expired),
AccountAvailability::Invalid => Some(AccountErrorReason::Invalid),
AccountAvailability::Banned => Some(AccountErrorReason::Banned),
AccountAvailability::Unknown => Some(AccountErrorReason::Unknown),
}
}
/// 账号列表排序字段。
@@ -67,7 +198,7 @@ pub struct AccountRecord {
pub provider_kind: ProviderKind,
pub name: String,
pub email: Option<String>,
pub upstream_user_id: String,
pub upstream_user_id: Option<String>,
pub upstream_account_id: Option<String>,
pub plan_type: Option<String>,
pub authentication_kind: String,
@@ -78,6 +209,8 @@ pub struct AccountRecord {
pub enabled: bool,
pub availability: AccountAvailability,
pub availability_observed_at: DateTime<Utc>,
/// 最近一次非 ready 状态变更的原始原因(上游错误原文 / 原因码);恢复 ready 后清空。
pub last_error_message: Option<String>,
pub quota_observed_at: Option<DateTime<Utc>>,
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
@@ -170,13 +303,15 @@ pub struct AccountPage {
/// 统一账号目录的全局状态计数,不受当前筛选和分页影响。
///
/// 配额耗尽是可恢复状态,始终单列;`unavailable` 只包含过期、禁用和封禁账号。
/// 计数与 [`AccountStatus`] 一一对应,由 store 按派生状态聚合。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct AccountSummary {
pub total: u64,
pub active: u64,
pub normal: u64,
pub quota_exhausted: u64,
pub unavailable: u64,
pub rate_limited: u64,
pub disabled: u64,
pub error: u64,
}
/// 账号启停写入命令。
@@ -531,12 +531,23 @@ pub struct DashboardRuntimeSlots {
pub used_slots: Option<u64>,
}
/// 仪表盘所需的公共观测事实。
/// 仪表盘卡片脚注展示的全历史累计。
#[derive(Debug, Clone, Default, PartialEq)]
pub struct DashboardTotals {
pub request_count: u64,
pub input_tokens: u64,
pub cached_tokens: u64,
pub total_tokens: u64,
pub billing_usd: Option<DecimalAmount>,
}
/// 仪表盘所需的当前区间事实及全历史累计。
#[derive(Debug, Clone, PartialEq)]
pub struct DashboardObservation {
pub range: TimeRange,
pub requests: RequestMetrics,
pub attempts: AttemptMetrics,
pub totals: DashboardTotals,
pub provider_accounts: AccountPoolMetrics,
pub trend: Vec<RequestMetricPoint>,
pub account_usage: Vec<DashboardAccountUsage>,
@@ -11,7 +11,10 @@ use gateway_core::{
use super::{
AdminError, MutationActor, MutationContext, PageSize, Revision,
accounts::{AccountAvailability, AccountRecord, AccountStatus, AccountSummary, AccountUsage},
accounts::{
AccountAvailability, AccountErrorReason, AccountRecord, AccountStatus, AccountSummary,
AccountUsage,
},
};
/// Provider-owned JSON;公共层只搬运且 Debug 不输出值。
@@ -143,7 +146,7 @@ pub struct PreparedCredentialCreate {
pub provider_kind: ProviderKind,
pub name: String,
pub email: Option<String>,
pub upstream_user_id: String,
pub upstream_user_id: Option<String>,
pub upstream_account_id: Option<String>,
pub plan_type: Option<String>,
pub authentication_kind: String,
@@ -561,6 +564,15 @@ impl fmt::Debug for RotateCredential {
}
}
/// 通用账号用量能否可靠归属到一个 Provider quota 窗口。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum QuotaLocalUsageAttribution {
/// 窗口覆盖账号的全部请求,可按账号与时间范围聚合。
AccountWide,
/// 窗口需要 Provider / 模型级归属,通用账号聚合不可用。
Unavailable,
}
/// 一个 Provider quota 窗口的公共投影。
#[derive(Debug, Clone, PartialEq)]
pub struct ProviderQuotaWindow {
@@ -568,6 +580,7 @@ pub struct ProviderQuotaWindow {
pub group: String,
pub label: String,
pub source: Option<String>,
pub local_usage_attribution: QuotaLocalUsageAttribution,
pub window_seconds: Option<u64>,
pub used_percent: Option<f64>,
pub reset_at: Option<DateTime<Utc>>,
@@ -592,6 +605,9 @@ pub struct ProviderQuota {
pub windows: Vec<ProviderQuotaWindow>,
/// 快照级限流事实(顶层或任一窗口触顶);Admin 状态派生用,前端不再遍历窗口猜测。
pub limit_reached: bool,
/// 429 临时限流(Redis 冷却)到期时间;`Some` 时账号处于临时限流,到期自动恢复。
/// 与 `limit_reached`(配额耗尽)是独立维度,不污染额度窗口。
pub rate_limited_until: Option<DateTime<Utc>>,
pub provider_data: Option<ProviderDocument>,
}
@@ -688,6 +704,8 @@ impl fmt::Debug for ProviderExport {
pub struct AccountDirectoryItem {
pub account: AccountRecord,
pub status: AccountStatus,
/// `status == Error` 时的具体原因;其余状态为 `None`。
pub error_reason: Option<AccountErrorReason>,
pub usage: Option<super::accounts::AccountUsage>,
pub quota: ProviderQuota,
}
@@ -1,6 +1,6 @@
//! 统一账号目录与跨 Provider 动态分派。
use std::{collections::BTreeMap, sync::Arc};
use std::{collections::BTreeMap, sync::Arc, time::SystemTime};
use async_trait::async_trait;
use chrono::{Duration, Utc};
@@ -18,13 +18,14 @@ use crate::{
AdminError, MutationContext,
accounts::{
AccountConnectionTestEvent, AccountConnectionTestEventStream, AccountListQuery,
AccountRecord, AccountStatus, AccountUsageWindowQuery,
AccountRecord, AccountStatus, AccountStatusSignals, AccountUsageWindowQuery,
derive_error_reason,
},
observability::TimeRange,
provider_credentials::{
AccountDirectoryItem, AccountDirectoryPage, AccountExportBundle, AccountRefreshResult,
PrepareCredentialRefresh, ProviderModels, ProviderQuota, ProviderQuotaRequest,
ProviderQuotaWindow,
ProviderQuotaWindow, QuotaLocalUsageAttribution,
},
},
ports::{
@@ -159,7 +160,9 @@ impl DefaultAccountsService {
.collect::<BTreeMap<_, _>>();
for (account, quota) in accounts.iter().zip(quotas) {
for window in &mut quota.windows {
if window.local_usage.is_none() {
if window.local_usage.is_none()
&& window.local_usage_attribution == QuotaLocalUsageAttribution::AccountWide
{
let key = (account.id.clone(), window.key.clone());
if let Some(usage) = usage_by_window.get(&key) {
window.local_usage = Some(usage.clone());
@@ -216,8 +219,11 @@ impl DefaultAccountsService {
std::slice::from_mut(&mut quota),
)
.await?;
let signals = account_signals(&account, &quota, now);
let error_reason = derive_error_reason(&signals);
Ok(AccountDirectoryItem {
status: account_status(&account, quota.limit_reached, now),
status: signals.derive(),
error_reason,
usage: usage.into_iter().next(),
account,
quota,
@@ -301,11 +307,16 @@ impl AccountsService for DefaultAccountsService {
.items
.into_iter()
.zip(quotas)
.map(|(account, quota)| AccountDirectoryItem {
status: account_status(&account, quota.limit_reached, now),
usage: usage.get(&account.id).cloned(),
account,
quota,
.map(|(account, quota)| {
let signals = account_signals(&account, &quota, now);
let error_reason = derive_error_reason(&signals);
AccountDirectoryItem {
status: signals.derive(),
error_reason,
usage: usage.get(&account.id).cloned(),
account,
quota,
}
})
.collect();
Ok(AccountDirectoryPage {
@@ -512,43 +523,39 @@ fn empty_quota() -> ProviderQuota {
refresh_token_expires_at: None,
windows: Vec::new(),
limit_reached: false,
rate_limited_until: None,
provider_data: None,
}
}
/// 互斥页面状态:availability 终态优先,quota 快照级 `limit_reached` 派生
/// `rate_limited`,`Unknown` 保守映射 `invalid`(不可调度,不显示为 active)。
/// 构造状态派生所需的信号快照。
fn account_signals(
account: &AccountRecord,
quota: &ProviderQuota,
now: chrono::DateTime<Utc>,
) -> AccountStatusSignals {
AccountStatusSignals {
enabled: account.enabled,
availability: account.availability,
access_token_expired: account
.access_token_expires_at
.is_some_and(|expires_at| expires_at <= now),
quota_limit_reached: quota.limit_reached,
rate_limited_until: quota.rate_limited_until.map(SystemTime::from),
}
}
/// 账号运营分类(互斥展示状态)。
fn account_status(
account: &AccountRecord,
quota_limit_reached: bool,
now: chrono::DateTime<Utc>,
) -> AccountStatus {
if !account.enabled {
AccountStatus::Disabled
} else if account.availability == crate::model::accounts::AccountAvailability::Banned {
AccountStatus::Banned
} else if account.availability == crate::model::accounts::AccountAvailability::QuotaExhausted {
AccountStatus::QuotaExhausted
} else if matches!(
account.availability,
crate::model::accounts::AccountAvailability::Invalid
) {
AccountStatus::Invalid
} else if matches!(
account.availability,
crate::model::accounts::AccountAvailability::Expired
) || account
.access_token_expires_at
.is_some_and(|expires_at| expires_at <= now)
{
AccountStatus::Expired
} else if quota_limit_reached {
AccountStatus::RateLimited
} else if account.availability == crate::model::accounts::AccountAvailability::Unknown {
AccountStatus::Invalid
} else {
AccountStatus::Active
}
let quota = ProviderQuota {
limit_reached: quota_limit_reached,
..empty_quota()
};
account_signals(account, &quota, now).derive()
}
fn retained_usage_range(now: chrono::DateTime<Utc>, retention_days: u32) -> TimeRange {
@@ -563,7 +570,9 @@ fn quota_usage_window(
account_id: &str,
window: &ProviderQuotaWindow,
) -> Option<AccountUsageWindowQuery> {
if window.local_usage.is_some() {
if window.local_usage.is_some()
|| window.local_usage_attribution != QuotaLocalUsageAttribution::AccountWide
{
return None;
}
let reset_at = window.reset_at?;
@@ -112,15 +112,10 @@ impl ObservabilityService for DefaultObservabilityService {
let today =
dashboard_period_metrics(&observation.trend, today_start, observation.range.end);
let yesterday = dashboard_period_metrics(&observation.trend, yesterday_start, today_start);
let total_billing_usd = observation
.attempts
.costs
.iter()
.find(|cost| cost.currency == "USD")
.map(|cost| cost.amount.clone());
let total_billing_usd = observation.totals.billing_usd.clone();
let total_cached_token_rate = rate_or_zero(
observation.requests.cached_tokens,
observation.requests.input_tokens,
observation.totals.cached_tokens,
observation.totals.input_tokens,
);
let average_first_token_latency_ms = average(
observation.requests.first_token_latency_sum_ms,
@@ -84,25 +84,29 @@ impl DefaultOpenAiService {
}
}
async fn observe_initial_quotas(&self, account_ids: &[ProviderAccountId], request_id: &str) {
for account_id in account_ids {
if let Err(error) = self
.provider
.quota(ProviderQuotaRequest {
account_id: account_id.clone(),
refresh: true,
rolling_usage: None,
})
.await
{
tracing::warn!(
request_id,
account_id = %account_id.as_str(),
quota_error = ?error.kind(),
"OpenAI initial quota observation failed"
);
fn observe_initial_quotas(&self, account_ids: &[ProviderAccountId], request_id: &str) {
let provider = Arc::clone(&self.provider);
let account_ids = account_ids.to_vec();
let request_id = request_id.to_owned();
tokio::spawn(async move {
for account_id in account_ids {
if let Err(error) = provider
.quota(ProviderQuotaRequest {
account_id: account_id.clone(),
refresh: true,
rolling_usage: None,
})
.await
{
tracing::warn!(
request_id,
account_id = %account_id.as_str(),
quota_error = ?error.kind(),
"OpenAI initial quota observation failed"
);
}
}
}
});
}
}
@@ -155,8 +159,7 @@ impl OpenAiService for DefaultOpenAiService {
.await;
publish_committed(self.snapshot.as_ref(), result.config_revision).await?;
// 导入完成后立即做一次观察;失败不回滚已经提交的 credential。
self.observe_initial_quotas(&result.credential_ids, &context.request_id)
.await;
self.observe_initial_quotas(&result.credential_ids, &context.request_id);
Ok(result)
}
@@ -208,8 +211,7 @@ impl OpenAiService for DefaultOpenAiService {
self.observe_initial_quotas(
std::slice::from_ref(&result.account_id),
&context.request_id,
)
.await;
);
Ok(result)
}
@@ -13,9 +13,10 @@ use gateway_admin::{
MutationActor, MutationContext, Revision,
auth::AdminAuditEvent,
backup::{
BackupObjectMetadata, BackupRecord, BackupRecordListQuery, BackupRecordPage,
BackupRecordSeed, BackupSettings, BackupStatus, BackupStorageConfig, BackupTriggerKind,
ConnectionTestResult, UpdateBackupScheduleCommand, UpdateBackupStorageCommand,
BackupError, BackupObjectMetadata, BackupRecord, BackupRecordListQuery,
BackupRecordPage, BackupRecordSeed, BackupSettings, BackupStatus, BackupStorageConfig,
BackupTriggerKind, ConnectionTestResult, UpdateBackupScheduleCommand,
UpdateBackupStorageCommand,
},
},
ports::{
@@ -497,15 +498,23 @@ impl DatabaseDumpPort for FakeDumpPort {
/// 内存版对象存储:把上传内容按 key 保存并支持 Head/Delete/预签名。
pub(crate) struct FakeObjectStore {
pub(crate) objects: Mutex<std::collections::HashMap<String, Vec<u8>>>,
upload_error: Option<BackupError>,
}
impl FakeObjectStore {
pub(crate) fn new() -> Self {
Self {
objects: Mutex::new(std::collections::HashMap::new()),
upload_error: None,
}
}
/// 让上传返回指定的脱敏 S3 错误。
pub(crate) fn fail_upload(mut self, code: &'static str, message: impl Into<String>) -> Self {
self.upload_error = Some(BackupError::new(code, message.into()));
self
}
pub(crate) fn object(&self, key: &str) -> Option<Vec<u8>> {
self.objects.lock().expect("objects").get(key).cloned()
}
@@ -530,6 +539,9 @@ impl BackupObjectStorePort for FakeObjectStore {
_config: &BackupStorageConfig,
request: UploadObjectRequest,
) -> Result<(), gateway_admin::model::backup::BackupError> {
if let Some(error) = &self.upload_error {
return Err(error.clone());
}
let content = std::fs::read(&request.source).map_err(|_| {
gateway_admin::model::backup::BackupError::new(
gateway_admin::model::backup::code::S3_UPLOAD_FAILED,
@@ -6,9 +6,9 @@ use chrono::{Duration, Utc};
use gateway_admin::{
backup::task::BackupTask,
model::backup::{
BackupRecordSeed, BackupStatus, BackupTriggerKind, UpdateBackupScheduleCommand,
BackupRecordSeed, BackupStatus, BackupTriggerKind, UpdateBackupScheduleCommand, code,
},
ports::backup::BackupRepository,
ports::backup::{BackupRepository, DatabaseDumpPort, DumpRequest},
};
use gateway_core::engine::CancellationToken;
@@ -89,6 +89,78 @@ async fn backup_task_marks_failed_when_dump_port_fails() {
assert!(object_store.object(&seed.object_key).is_none());
}
#[tokio::test]
async fn backup_task_marks_failed_with_the_upload_error() {
let repository = Arc::new(FakeBackupRepository::new(configured_settings()));
let dump = Arc::new(FakeDumpPort::new());
let object_store =
Arc::new(FakeObjectStore::new().fail_upload(code::S3_UPLOAD_FAILED, "对象存储拒绝上传"));
let seed = BackupRecordSeed {
id: backup_id("upload-failure"),
trigger_kind: BackupTriggerKind::Manual,
scheduled_at: None,
object_key: "codex/2026/08/01/upload-failure.dump".to_owned(),
expires_at: None,
};
repository
.insert_backup_record(seed.clone())
.await
.expect("insert queued");
let task = BackupTask::new(repository.clone(), dump, object_store);
task.run_cycle(&CancellationToken::new())
.await
.expect("run cycle");
let records = repository.all_records();
let failed = &records[0];
assert_eq!(failed.status, BackupStatus::Failed);
assert_eq!(failed.error_code.as_deref(), Some(code::S3_UPLOAD_FAILED));
assert_eq!(
failed.error_message.as_deref(),
Some("上传到对象存储失败:对象存储拒绝上传")
);
}
#[tokio::test]
async fn backup_task_recovery_marks_failed_with_the_upload_error() {
let repository = Arc::new(FakeBackupRepository::new(configured_settings()));
let dump = Arc::new(FakeDumpPort::new());
let object_store =
Arc::new(FakeObjectStore::new().fail_upload(code::S3_UPLOAD_FAILED, "对象存储拒绝上传"));
let seed = BackupRecordSeed {
id: backup_id("recovered-upload-failure"),
trigger_kind: BackupTriggerKind::Manual,
scheduled_at: None,
object_key: "codex/2026/08/01/recovered-upload-failure.dump".to_owned(),
expires_at: None,
};
repository
.insert_backup_record(seed.clone())
.await
.expect("insert queued");
let cancellation = CancellationToken::new();
dump.dump(DumpRequest {
backup_id: seed.id.clone(),
cancellation: cancellation.clone(),
})
.await
.expect("stage dump");
repository.force_status(&seed.id, BackupStatus::Dumping);
let task = BackupTask::new(repository.clone(), dump, object_store);
task.run_cycle(&cancellation).await.expect("run cycle");
let records = repository.all_records();
let failed = &records[0];
assert_eq!(failed.status, BackupStatus::Failed);
assert_eq!(failed.error_code.as_deref(), Some(code::S3_UPLOAD_FAILED));
assert_eq!(
failed.error_message.as_deref(),
Some("上传到对象存储失败:对象存储拒绝上传")
);
}
#[tokio::test]
async fn retention_cleans_expired_scheduled_backups() {
let repository = Arc::new(FakeBackupRepository::new(configured_settings()));
@@ -11,6 +11,7 @@ use gateway_admin::{
observability::{DecimalAmount, RequestOutcome, TimeRange},
provider_credentials::{
CredentialCommitGuard, ProviderDocument, ProviderQuota, ProviderQuotaWindow,
QuotaLocalUsageAttribution,
},
settings::AdminApiKey,
},
@@ -128,6 +129,7 @@ fn representative_quota_should_prefer_short_window_and_highest_usage() {
quota_window("shortTerm", Some(14_400), Some(45.0)),
],
limit_reached: false,
rate_limited_until: None,
provider_data: None,
};
@@ -144,6 +146,7 @@ fn quota_window(
group: group.to_owned(),
label: group.to_owned(),
source: None,
local_usage_attribution: QuotaLocalUsageAttribution::AccountWide,
window_seconds,
used_percent,
reset_at: None,
@@ -33,7 +33,7 @@ use gateway_admin::{
PreparedAuthorizationCredential, PreparedCredentialCreate, PreparedCredentialImport,
PreparedCredentialRotation, PreparedCredentialRotationFacts, ProviderDocument,
ProviderExport, ProviderExportCredentialInput, ProviderModels, ProviderQuota,
ProviderQuotaRequest, ProviderQuotaWindow,
ProviderQuotaRequest, ProviderQuotaWindow, QuotaLocalUsageAttribution,
},
settings::{
AdminApiKey, AdminApiKeyMutation, ReplaceRuntimeSettings, RotationStrategy,
@@ -451,9 +451,11 @@ impl AccountStore for FakeAccountStore {
total: accounts.len() as u64,
summary: AccountSummary {
total: accounts.len() as u64,
active: 1,
normal: 1,
quota_exhausted: 0,
unavailable: 0,
rate_limited: 0,
disabled: 0,
error: 0,
},
})
}
@@ -844,7 +846,7 @@ async fn connection_test_should_probe_unavailable_account() {
}
#[tokio::test]
async fn connection_test_rate_limited_probe_preserves_active_account_status() {
async fn connection_test_rate_limited_probe_preserves_normal_account_status() {
let events = events();
let provider = FakeProviderAdmin::new("xai", events.clone());
let store = FakeAccountStore::new("xai", events);
@@ -867,7 +869,7 @@ async fn connection_test_rate_limited_probe_preserves_active_account_status() {
assert!(matches!(
events.last(),
Some(AccountConnectionTestEvent::Failed {
account_status: gateway_admin::model::accounts::AccountStatus::Active,
account_status: gateway_admin::model::accounts::AccountStatus::Normal,
..
})
));
@@ -948,7 +950,7 @@ async fn accounts_refresh_should_keep_guard_through_store_commit() {
assert_eq!(result.account.account.provider_kind.as_str(), "openai");
assert_eq!(
result.account.status,
gateway_admin::model::accounts::AccountStatus::Active
gateway_admin::model::accounts::AccountStatus::Normal
);
assert_eq!(
@@ -984,12 +986,12 @@ async fn accounts_list_should_return_complete_directory_semantics() {
.expect("complete account directory");
assert_eq!(page.summary.total, 1);
assert_eq!(page.summary.active, 1);
assert_eq!(page.summary.normal, 1);
let account = page.items.first().expect("account item");
assert_eq!(account.account.provider_kind.as_str(), "openai");
assert_eq!(
account.status,
gateway_admin::model::accounts::AccountStatus::Active
gateway_admin::model::accounts::AccountStatus::Normal
);
}
@@ -1005,6 +1007,7 @@ async fn accounts_list_should_degrade_quota_failure_to_empty_window_without_drop
group: "shortTerm".to_owned(),
label: "5小时限额".to_owned(),
source: None,
local_usage_attribution: QuotaLocalUsageAttribution::AccountWide,
window_seconds: Some(5 * 60 * 60),
used_percent: Some(97.0),
reset_at: Some(Utc::now() + TimeDelta::hours(1)),
@@ -1013,6 +1016,7 @@ async fn accounts_list_should_degrade_quota_failure_to_empty_window_without_drop
provider_data: None,
}],
limit_reached: false,
rate_limited_until: None,
provider_data: None,
});
let failing = FakeProviderAdmin::new("xai", events.clone());
@@ -1088,8 +1092,8 @@ async fn accounts_list_should_prefer_quota_exhaustion_over_stale_token_expiry()
}
#[tokio::test]
async fn accounts_list_should_map_unknown_availability_to_invalid_not_active() {
// Unknown 不可调度,Admin 不得显示为 active。
async fn accounts_list_should_map_unknown_availability_to_error_not_normal() {
// Unknown 不可调度,Admin 不得显示为 normal。
let provider = FakeProviderAdmin::new("openai", events());
let mut account = account_record("openai");
account.availability = AccountAvailability::Unknown;
@@ -1111,13 +1115,14 @@ async fn accounts_list_should_map_unknown_availability_to_invalid_not_active() {
assert_eq!(
page.items.first().expect("account item").status,
gateway_admin::model::accounts::AccountStatus::Invalid,
gateway_admin::model::accounts::AccountStatus::Error,
);
}
#[tokio::test]
async fn accounts_list_should_map_rate_limited_from_quota_limit_reached() {
// ready + 快照级 quota.limitReached → rate_limited。
// ready + 快照级 quota.limitReached → quota_exhausted(配额耗尽);
// rate_limited 只来自 ProviderQuota.rate_limited_until(429 Redis 冷却)。
let provider = FakeProviderAdmin::new("openai", events());
provider.set_quota(ProviderQuota {
observed_at: Some(Utc::now()),
@@ -1127,6 +1132,7 @@ async fn accounts_list_should_map_rate_limited_from_quota_limit_reached() {
group: "shortTerm".to_owned(),
label: "5小时限额".to_owned(),
source: None,
local_usage_attribution: QuotaLocalUsageAttribution::AccountWide,
window_seconds: Some(5 * 60 * 60),
used_percent: Some(100.0),
reset_at: Some(Utc::now() + TimeDelta::hours(1)),
@@ -1135,6 +1141,7 @@ async fn accounts_list_should_map_rate_limited_from_quota_limit_reached() {
provider_data: None,
}],
limit_reached: true,
rate_limited_until: Some(Utc::now() + TimeDelta::minutes(5)),
provider_data: None,
});
let store = FakeAccountStore::new("openai", events());
@@ -1159,6 +1166,52 @@ async fn accounts_list_should_map_rate_limited_from_quota_limit_reached() {
);
}
#[tokio::test]
async fn accounts_list_should_map_quota_exhausted_when_only_limit_reached() {
// ready + 快照级 quota.limitReached、无 429 冷却 → quota_exhausted。
let provider = FakeProviderAdmin::new("openai", events());
provider.set_quota(ProviderQuota {
observed_at: Some(Utc::now()),
refresh_token_expires_at: None,
windows: vec![ProviderQuotaWindow {
key: "primary".to_owned(),
group: "shortTerm".to_owned(),
label: "5小时限额".to_owned(),
source: None,
local_usage_attribution: QuotaLocalUsageAttribution::AccountWide,
window_seconds: Some(5 * 60 * 60),
used_percent: Some(100.0),
reset_at: Some(Utc::now() + TimeDelta::hours(1)),
limit_reached: true,
local_usage: None,
provider_data: None,
}],
limit_reached: true,
rate_limited_until: None,
provider_data: None,
});
let store = FakeAccountStore::new("openai", events());
let page = accounts_service(provider, store)
.await
.accounts()
.list(AccountListQuery {
page: 1,
page_size: gateway_admin::model::PageSize::new(20).expect("page size"),
provider_kind: None,
search: None,
status: None,
sort: None,
})
.await
.expect("quota exhausted account directory");
assert_eq!(
page.items.first().expect("account item").status,
gateway_admin::model::accounts::AccountStatus::QuotaExhausted,
);
}
#[tokio::test]
async fn accounts_list_should_attach_local_usage_to_quota_windows() {
let provider = FakeProviderAdmin::new("openai", events());
@@ -1171,6 +1224,7 @@ async fn accounts_list_should_attach_local_usage_to_quota_windows() {
group: "shortTerm".to_owned(),
label: "5小时限额".to_owned(),
source: None,
local_usage_attribution: QuotaLocalUsageAttribution::AccountWide,
window_seconds: Some(5 * 60 * 60),
used_percent: Some(97.0),
reset_at: Some(reset_at),
@@ -1179,6 +1233,7 @@ async fn accounts_list_should_attach_local_usage_to_quota_windows() {
provider_data: None,
}],
limit_reached: false,
rate_limited_until: None,
provider_data: None,
});
let store = FakeAccountStore::new("openai", events());
@@ -1209,6 +1264,96 @@ async fn accounts_list_should_attach_local_usage_to_quota_windows() {
assert_eq!(usage.total_tokens, Some(4_330_000));
}
#[tokio::test]
async fn accounts_list_should_not_attach_account_usage_to_model_specific_quota_windows() {
let provider = FakeProviderAdmin::new("openai", events());
let reset_at = Utc::now() + TimeDelta::days(7);
provider.set_quota(ProviderQuota {
observed_at: Some(Utc::now()),
refresh_token_expires_at: None,
windows: vec![
ProviderQuotaWindow {
key: "core-primary".to_owned(),
group: "weekly".to_owned(),
label: "周额度".to_owned(),
source: Some("codex".to_owned()),
local_usage_attribution: QuotaLocalUsageAttribution::AccountWide,
window_seconds: Some(7 * 24 * 60 * 60),
used_percent: Some(1.0),
reset_at: Some(reset_at),
limit_reached: false,
local_usage: None,
provider_data: None,
},
ProviderQuotaWindow {
key: "codex-bengalfox-primary".to_owned(),
group: "weekly".to_owned(),
label: "GPT-5.3-Codex-Spark · 周额度".to_owned(),
source: Some("codex_bengalfox".to_owned()),
local_usage_attribution: QuotaLocalUsageAttribution::Unavailable,
window_seconds: Some(7 * 24 * 60 * 60),
used_percent: Some(0.0),
reset_at: Some(reset_at),
limit_reached: false,
local_usage: None,
provider_data: None,
},
],
limit_reached: false,
rate_limited_until: None,
provider_data: None,
});
let store = FakeAccountStore::new("openai", events());
store.set_quota_window_usage(vec![
AccountUsageWindowResult {
account_id: "acct_test".to_owned(),
key: "core-primary".to_owned(),
usage: quota_local_usage("acct_test", 12_818_806),
},
AccountUsageWindowResult {
account_id: "acct_test".to_owned(),
key: "codex-bengalfox-primary".to_owned(),
usage: quota_local_usage("acct_test", 127_926),
},
]);
let page = accounts_service(provider, store)
.await
.accounts()
.list(AccountListQuery {
page: 1,
page_size: gateway_admin::model::PageSize::new(20).expect("page size"),
provider_kind: None,
search: None,
status: None,
sort: None,
})
.await
.expect("quota window usage");
let local_tokens = page.items[0]
.quota
.windows
.iter()
.map(|window| {
(
window.key.as_str(),
window
.local_usage
.as_ref()
.and_then(|usage| usage.total_tokens),
)
})
.collect::<Vec<_>>();
assert_eq!(
local_tokens,
[
("core-primary", Some(12_818_806)),
("codex-bengalfox-primary", None),
],
);
}
#[tokio::test]
async fn accounts_refresh_provider_failure_should_not_call_store_commit() {
let events = events();
@@ -1287,6 +1432,7 @@ fn empty_quota() -> ProviderQuota {
refresh_token_expires_at: None,
windows: Vec::new(),
limit_reached: false,
rate_limited_until: None,
provider_data: None,
}
}
@@ -1321,7 +1467,7 @@ pub(super) fn account_record(kind: &str) -> AccountRecord {
provider_kind: ProviderKind::new(kind).expect("provider kind"),
name: "test account".to_owned(),
email: Some("test@example.invalid".to_owned()),
upstream_user_id: "upstream-user".to_owned(),
upstream_user_id: Some("upstream-user".to_owned()),
upstream_account_id: None,
plan_type: Some("test".to_owned()),
authentication_kind: "oauth".to_owned(),
@@ -1332,6 +1478,7 @@ pub(super) fn account_record(kind: &str) -> AccountRecord {
enabled: true,
availability: AccountAvailability::Ready,
availability_observed_at: now,
last_error_message: None,
quota_observed_at: None,
created_at: now,
updated_at: now,
@@ -1357,7 +1504,7 @@ fn prepared_create_with_id(
provider_kind,
name: name.to_owned(),
email: Some("prepared@example.invalid".to_owned()),
upstream_user_id: "prepared-user".to_owned(),
upstream_user_id: Some("prepared-user".to_owned()),
upstream_account_id: None,
plan_type: Some("test".to_owned()),
authentication_kind: "oauth".to_owned(),
@@ -449,6 +449,7 @@ impl ObservabilityStore for FixtureObservabilityStore {
range,
requests: RequestMetrics::default(),
attempts: AttemptMetrics::default(),
totals: Default::default(),
provider_accounts: AccountPoolMetrics::default(),
trend: self.trend.lock().expect("trend").clone(),
account_usage: Vec::new(),
@@ -125,6 +125,8 @@ async fn openai_import_should_prepare_before_atomic_store_commit() {
.await
.expect("import credential");
await_quota_requests(&provider, 1).await;
assert_eq!(
recorded(&events),
[
@@ -162,6 +164,8 @@ async fn openai_import_should_refresh_quota_for_every_imported_account() {
.await
.expect("import credentials");
await_quota_requests(&provider, 2).await;
assert_eq!(
result.credential_ids,
[
@@ -203,6 +207,8 @@ async fn openai_import_should_remain_successful_when_quota_refresh_fails() {
.await
.expect("committed import remains successful");
await_quota_requests(&provider, 1).await;
assert_eq!(
result.credential_ids,
[ProviderAccountId::new("acct_prepared").expect("account ID")]
@@ -236,6 +242,8 @@ async fn openai_authorization_create_should_observe_initial_quota() {
.await
.expect("complete authorization");
await_quota_requests(&provider, 1).await;
assert_eq!(
recorded(&events),
[
@@ -284,6 +292,8 @@ async fn openai_authorization_create_should_remain_successful_when_initial_quota
.await
.expect("quota observation must not fail authorization");
await_quota_requests(&provider, 1).await;
assert_eq!(result.account_id.as_str(), "acct_prepared");
assert_eq!(provider.quota_requests().len(), 1);
}
@@ -322,6 +332,8 @@ async fn openai_authorization_store_failure_should_release_claim_for_retry() {
.await
.expect("released claim must permit retry");
await_quota_requests(&provider, 1).await;
assert_eq!(
recorded(&events),
[
@@ -400,6 +412,8 @@ async fn openai_reauthorization_should_commit_after_credential_revision_advances
.await
.expect("complete reauthorization");
await_quota_requests(&provider, 1).await;
assert_eq!(result.credential_revision, Some(revision(3)));
assert_eq!(
@@ -426,6 +440,16 @@ async fn service(provider: Arc<FakeProviderAdmin>, store: Arc<FakeAccountStore>)
.await
}
async fn await_quota_requests(provider: &FakeProviderAdmin, expected: usize) {
for _ in 0..100 {
if provider.quota_requests().len() == expected {
return;
}
tokio::task::yield_now().await;
}
assert_eq!(provider.quota_requests().len(), expected);
}
fn mutation(account_id: &str) -> CredentialMutation {
CredentialMutation {
context: context("request-openai"),
@@ -91,7 +91,8 @@ impl ListQuery {
let status = match self.status.as_deref().map(str::trim) {
None | Some("") => None,
Some(value) => Some(
parse_account_status(value).ok_or_else(|| WireValidationError::new("status"))?,
DomainAccountStatus::parse(&value.to_ascii_lowercase())
.ok_or_else(|| WireValidationError::new("status"))?,
),
};
let sort = match (self.sort_by.as_deref(), self.sort_direction.as_deref()) {
@@ -127,17 +128,6 @@ fn parse_provider(value: &str) -> Result<Option<ProviderKind>, WireValidationErr
.map_err(|_| WireValidationError::new("provider"))
}
fn parse_account_status(value: &str) -> Option<DomainAccountStatus> {
match value.trim().to_ascii_lowercase().as_str() {
"active" => Some(DomainAccountStatus::Active),
"expired" => Some(DomainAccountStatus::Expired),
"quota_exhausted" => Some(DomainAccountStatus::QuotaExhausted),
"disabled" => Some(DomainAccountStatus::Disabled),
"banned" => Some(DomainAccountStatus::Banned),
_ => None,
}
}
fn parse_sort_field(value: &str) -> Option<AccountSortField> {
match value.trim() {
"email" => Some(AccountSortField::Email),
@@ -172,9 +162,11 @@ pub struct AccountPageData {
#[serde(rename_all = "camelCase")]
pub struct AccountSummaryView {
pub total: u64,
pub active: u64,
pub normal: u64,
pub quota_exhausted: u64,
pub unavailable: u64,
pub rate_limited: u64,
pub disabled: u64,
pub error: u64,
}
/// 一条安全账号视图。
@@ -194,6 +186,10 @@ pub struct AccountView {
pub has_refresh_token: bool,
pub status: String,
pub display_status: String,
/// `status == "error"` 时的具体原因;其余状态为 `null`。
pub error_reason: Option<String>,
/// 最近一次失败的上游错误描述;仅错误状态存在。
pub error_message: Option<String>,
pub availability: String,
pub enabled: bool,
pub access_token_expires_at: Option<String>,
@@ -214,6 +210,8 @@ pub struct AccountView {
pub struct AccountQuotaView {
pub refreshed_at_display: String,
pub limit_reached: bool,
/// 429 临时限流(Redis 冷却)到期时间展示;非限流中为 `null`。
pub rate_limited_until: Option<String>,
pub windows: Vec<AccountQuotaWindowView>,
}
@@ -527,7 +525,7 @@ impl From<DomainConnectionTestEvent> for AccountConnectionTestEvent {
DomainConnectionTestEvent::Completed { account_status } => serde_json::json!({
"type": "test_complete",
"success": true,
"accountStatus": account_status_name(account_status)
"accountStatus": account_status.as_str()
}),
DomainConnectionTestEvent::Failed {
message,
@@ -545,7 +543,7 @@ impl From<DomainConnectionTestEvent> for AccountConnectionTestEvent {
"upstreamStatus": upstream_status,
"upstreamContentType": upstream_content_type,
"upstreamBody": upstream_body,
"accountStatus": account_status_name(account_status)
"accountStatus": account_status.as_str()
}),
};
Self { data }
@@ -1333,9 +1331,11 @@ fn account_page_data(
page: PageMeta::new(page, u32::from(page_size), result.total, total_pages),
summary: AccountSummaryView {
total: result.summary.total,
active: result.summary.active,
normal: result.summary.normal,
quota_exhausted: result.summary.quota_exhausted,
unavailable: result.summary.unavailable,
rate_limited: result.summary.rate_limited,
disabled: result.summary.disabled,
error: result.summary.error,
},
}
}
@@ -1366,10 +1366,11 @@ fn account_view(item: AccountDirectoryItem, now: DateTime<Utc>) -> AccountView {
let AccountDirectoryItem {
account,
status,
error_reason,
usage,
quota,
} = item;
let status = account_status_name(status).to_owned();
let status = status.as_str().to_owned();
let expires_at = account.access_token_expires_at.as_ref().map(china_rfc3339);
let added_at = china_rfc3339(&account.created_at);
let updated_at = china_rfc3339(&account.updated_at);
@@ -1381,13 +1382,15 @@ fn account_view(item: AccountDirectoryItem, now: DateTime<Utc>) -> AccountView {
resource_ref: account.id,
email: account.email,
account_id: account.upstream_account_id,
user_id: Some(account.upstream_user_id),
user_id: account.upstream_user_id,
label: None,
plan_type: account.plan_type,
authentication_kind: account.authentication_kind,
has_refresh_token: account.has_refresh_token,
status: status.clone(),
display_status: status,
error_reason: error_reason.map(|reason| reason.as_str().to_owned()),
error_message: account.last_error_message,
availability: account.availability.as_str().to_owned(),
enabled: account.enabled,
access_token_expires_at: expires_at,
@@ -1417,10 +1420,12 @@ fn account_quota_view(
.observed_at
.map_or_else(|| "—".to_owned(), |value| relative_time(value, now));
let windows = quota.windows.into_iter().map(quota_window_view).collect();
let rate_limited_until = quota.rate_limited_until.map(|until| china_datetime(&until));
(
AccountQuotaView {
refreshed_at_display,
limit_reached: quota.limit_reached,
rate_limited_until,
windows,
},
refresh_token_expires_at,
@@ -1433,6 +1438,7 @@ fn quota_window_view(window: ProviderQuotaWindow) -> AccountQuotaWindowView {
group,
label,
source: _,
local_usage_attribution: _,
window_seconds,
used_percent,
reset_at,
@@ -1713,18 +1719,6 @@ fn provider_document_value(document: ProviderDocument) -> Value {
Value::Object(document.into_provider_data().into_inner())
}
fn account_status_name(status: DomainAccountStatus) -> &'static str {
match status {
DomainAccountStatus::Active => "active",
DomainAccountStatus::RateLimited => "rate_limited",
DomainAccountStatus::QuotaExhausted => "quota_exhausted",
DomainAccountStatus::Expired => "expired",
DomainAccountStatus::Invalid => "invalid",
DomainAccountStatus::Disabled => "disabled",
DomainAccountStatus::Banned => "banned",
}
}
fn relative_time(value: DateTime<Utc>, now: DateTime<Utc>) -> String {
let elapsed = now.signed_duration_since(value);
if elapsed.num_seconds() < 0 {
@@ -709,7 +709,7 @@ pub(crate) fn dashboard_view(
} = result;
let domain::DashboardObservation {
range,
requests,
totals,
provider_accounts,
account_usage,
recent_requests,
@@ -756,13 +756,13 @@ pub(crate) fn dashboard_view(
today_requests: format_compact_number(today.request_count),
today_requests_value: today.request_count,
yesterday_requests_value: yesterday.request_count,
total_requests: format_compact_number(requests.request_count),
total_requests: format_compact_number(totals.request_count),
},
tokens: DashboardTokensCardView {
today_tokens: format_compact_number(today.total_tokens),
today_tokens_value: today.total_tokens,
yesterday_tokens_value: yesterday.total_tokens,
total_tokens: format_compact_number(requests.total_tokens),
total_tokens: format_compact_number(totals.total_tokens),
total_billing_amount_usd: total_billing_usd
.as_ref()
.map_or_else(|| "—".to_owned(), |amount| format!("${}", amount.as_str())),
@@ -772,7 +772,7 @@ pub(crate) fn dashboard_view(
today_hit_rate_value: today.observed_cached_token_rate,
yesterday_hit_rate_value: yesterday.observed_cached_token_rate,
total_hit_rate: display_rate(total_cached_token_rate),
total_cached_tokens: format_compact_number(requests.cached_tokens),
total_cached_tokens: format_compact_number(totals.cached_tokens),
average_first_token_latency_ms: display_duration(average_first_token_latency_ms),
},
},
@@ -310,16 +310,6 @@ pub(crate) fn usage_range(
domain::TimeRange::new(start, end).map_err(|_| WireValidationError::new("timeRange"))
}
pub(crate) fn dashboard_range(
start: Option<&str>,
end: Option<&str>,
) -> Result<domain::TimeRange, WireValidationError> {
let end = parse_datetime(end)?.unwrap_or_else(Utc::now);
let start =
parse_datetime(start)?.unwrap_or_else(|| domain::china_day_start(end) - Duration::days(1));
domain::TimeRange::new(start, end).map_err(|_| WireValidationError::new("timeRange"))
}
pub(crate) fn dashboard_today_range(
start: Option<&str>,
end: Option<&str>,
@@ -38,7 +38,8 @@ where
S: AdminSessionState + Send + Sync,
{
let kind = query.trend_kind().map_err(map_wire_error)?;
let range = dashboard_range(query.start_time.as_deref(), query.end_time.as_deref())
// 概览默认按中国时区当日统计,与单独趋势接口保持同一口径。
let range = dashboard_today_range(query.start_time.as_deref(), query.end_time.as_deref())
.map_err(map_wire_error)?;
let result = state
.admin_services()
@@ -10,7 +10,7 @@ mod query {
"pageSize": 20,
"provider": "xai",
"search": " operator ",
"status": "active",
"status": "normal",
"sortBy": "lastUsedAt",
"sortDirection": "desc"
}))
@@ -23,7 +23,7 @@ mod query {
Some(ref provider) if provider.as_str() == "xai"
));
assert_eq!(query.search.as_deref(), Some("operator"));
assert_eq!(query.status, Some(AccountStatus::Active));
assert_eq!(query.status, Some(AccountStatus::Normal));
assert_eq!(
query.sort.expect("sort").field,
AccountSortField::LastUsedAt
@@ -34,6 +34,19 @@ mod query {
);
}
#[test]
fn account_query_should_parse_rate_limited_status() {
let query: ListQuery = serde_json::from_value(json!({
"status": "rate_limited"
}))
.expect("deserialize account query");
assert_eq!(
query.validate().expect("validate account query").status,
Some(AccountStatus::RateLimited)
);
}
#[test]
fn account_query_should_reject_unbounded_page_size() {
let query: ListQuery =
@@ -331,7 +344,7 @@ mod actions {
text: "OK".to_owned(),
},
DomainConnectionTestEvent::Completed {
account_status: AccountStatus::Active,
account_status: AccountStatus::Normal,
},
DomainConnectionTestEvent::Failed {
message: "upstream unavailable".to_owned(),
@@ -365,7 +378,7 @@ mod actions {
}
}),
json!({ "type": "content", "text": "OK" }),
json!({ "type": "test_complete", "success": true, "accountStatus": "active" }),
json!({ "type": "test_complete", "success": true, "accountStatus": "normal" }),
json!({
"type": "error",
"error": "upstream unavailable",
+10 -1
View File
@@ -83,6 +83,7 @@ pub(super) struct AdminTestFixture {
pub diagnostics: Arc<Mutex<Vec<DiagnosticObservation>>>,
pub ops_errors: Arc<Mutex<Vec<OpsError>>>,
pub dashboard_observation: Arc<Mutex<Option<DashboardObservation>>>,
pub dashboard_summary_range: Arc<Mutex<Option<TimeRange>>>,
}
impl AdminTestFixture {
@@ -100,12 +101,14 @@ impl AdminTestFixture {
let diagnostics = Arc::new(Mutex::new(Vec::new()));
let ops_errors = Arc::new(Mutex::new(Vec::new()));
let dashboard_observation = Arc::new(Mutex::new(None));
let dashboard_summary_range = Arc::new(Mutex::new(None));
let unused = Arc::new(UnusedStore {
usage_records: Arc::clone(&usage_records),
usage_detail: Arc::clone(&usage_detail),
diagnostics: Arc::clone(&diagnostics),
ops_errors: Arc::clone(&ops_errors),
dashboard_observation: Arc::clone(&dashboard_observation),
dashboard_summary_range: Arc::clone(&dashboard_summary_range),
});
let stores = AdminStorePorts::new(
unused.clone(),
@@ -142,6 +145,7 @@ impl AdminTestFixture {
diagnostics,
ops_errors,
dashboard_observation,
dashboard_summary_range,
}
}
@@ -413,6 +417,7 @@ struct UnusedStore {
diagnostics: Arc<Mutex<Vec<DiagnosticObservation>>>,
ops_errors: Arc<Mutex<Vec<OpsError>>>,
dashboard_observation: Arc<Mutex<Option<DashboardObservation>>>,
dashboard_summary_range: Arc<Mutex<Option<TimeRange>>>,
}
#[async_trait]
@@ -523,7 +528,11 @@ impl AccountStore for UnusedStore {
#[async_trait]
impl ObservabilityStore for UnusedStore {
async fn dashboard_summary(&self, _: TimeRange) -> AdminStoreResult<DashboardObservation> {
async fn dashboard_summary(&self, range: TimeRange) -> AdminStoreResult<DashboardObservation> {
*self
.dashboard_summary_range
.lock()
.expect("dashboard summary range") = Some(range);
self.dashboard_observation
.lock()
.expect("dashboard observation")
@@ -219,6 +219,7 @@ async fn dashboard_summary_should_include_quota_exhaustion_in_unavailable_headli
range: TimeRange::new(now - Duration::hours(1), now).expect("dashboard range"),
requests: RequestMetrics::default(),
attempts: AttemptMetrics::default(),
totals: Default::default(),
provider_accounts: AccountPoolMetrics {
total: 807,
enabled: 804,
@@ -270,6 +271,143 @@ async fn dashboard_summary_should_include_quota_exhaustion_in_unavailable_headli
);
}
#[tokio::test]
async fn dashboard_summary_should_use_lifetime_totals_for_card_footers() {
use axum::{
body::{Body, to_bytes},
http::{Request, StatusCode, header},
};
use chrono::{Duration, Utc};
use gateway_admin::model::observability::{
AccountPoolMetrics, AttemptMetrics, DashboardObservation, DashboardTotals, RequestMetrics,
TimeRange,
};
use gateway_api::admin::observability;
use tower::ServiceExt as _;
use crate::admin::{AdminTestFixture, AdminTestState};
let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session");
let now = Utc::now();
let range_requests = RequestMetrics {
request_count: 2,
input_tokens: 8,
cached_tokens: 2,
total_tokens: 12,
..RequestMetrics::default()
};
*fixture
.dashboard_observation
.lock()
.expect("dashboard observation") = Some(DashboardObservation {
range: TimeRange::new(now - Duration::hours(1), now).expect("dashboard range"),
requests: range_requests,
attempts: AttemptMetrics::default(),
totals: DashboardTotals {
request_count: 42,
input_tokens: 800,
cached_tokens: 200,
total_tokens: 840,
billing_usd: None,
},
provider_accounts: AccountPoolMetrics::default(),
trend: Vec::new(),
account_usage: Vec::new(),
recent_requests: Vec::new(),
});
let response = observability::router::<AdminTestState>()
.with_state(fixture.state())
.oneshot(
Request::builder()
.uri("/api/admin/dashboard/summary")
.header(header::COOKIE, "cpr_admin_session=valid-session")
.header("x-request-id", "req_dashboard_lifetime_totals")
.body(Body::empty())
.expect("dashboard request"),
)
.await
.expect("dashboard response");
let status = response.status();
let body = to_bytes(response.into_body(), 1024 * 1024)
.await
.expect("dashboard response body");
let value: serde_json::Value = serde_json::from_slice(&body).expect("dashboard response JSON");
assert_eq!(status, StatusCode::OK, "{value}");
assert_eq!(
(
&value["data"]["cards"]["traffic"]["totalRequests"],
&value["data"]["cards"]["tokens"]["totalTokens"],
&value["data"]["cards"]["cache"]["totalHitRate"],
&value["data"]["cards"]["cache"]["totalCachedTokens"],
),
(
&serde_json::json!("42"),
&serde_json::json!("840"),
&serde_json::json!("25.0%"),
&serde_json::json!("200"),
),
);
}
#[tokio::test]
async fn dashboard_summary_should_default_to_current_china_day() {
use axum::{
body::Body,
http::{Request, StatusCode, header},
};
use chrono::{Duration, Utc};
use gateway_admin::model::observability::{
AccountPoolMetrics, AttemptMetrics, DashboardObservation, RequestMetrics, TimeRange,
china_day_start,
};
use gateway_api::admin::observability;
use tower::ServiceExt as _;
use crate::admin::{AdminTestFixture, AdminTestState};
let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session");
let now = Utc::now();
*fixture
.dashboard_observation
.lock()
.expect("dashboard observation") = Some(DashboardObservation {
range: TimeRange::new(now - Duration::hours(1), now).expect("dashboard range"),
requests: RequestMetrics::default(),
attempts: AttemptMetrics::default(),
totals: Default::default(),
provider_accounts: AccountPoolMetrics::default(),
trend: Vec::new(),
account_usage: Vec::new(),
recent_requests: Vec::new(),
});
let response = observability::router::<AdminTestState>()
.with_state(fixture.state())
.oneshot(
Request::builder()
.uri("/api/admin/dashboard/summary")
.header(header::COOKIE, "cpr_admin_session=valid-session")
.header("x-request-id", "req_dashboard_today_range")
.body(Body::empty())
.expect("dashboard request"),
)
.await
.expect("dashboard response");
let range = fixture
.dashboard_summary_range
.lock()
.expect("dashboard summary range")
.take()
.expect("recorded dashboard summary range");
assert_eq!(response.status(), StatusCode::OK);
assert_eq!(range.start, china_day_start(range.end));
}
#[tokio::test]
async fn usage_detail_should_keep_attempt_snapshot_contract() {
use axum::{
@@ -660,6 +660,9 @@ where
ProviderBoundary::Result(result) => match *result {
Ok(stream) => stream,
Err(error) => {
if self.prepare_unavailable_native_continuation_replay(&error) {
return Ok(Some(PullOutcome::AttemptDiscarded));
}
if matches!(
error.kind(),
ProviderErrorKind::AccountCapacityUnavailable
@@ -1096,6 +1099,41 @@ where
true
}
/// 原生续写的原账号在本地调度阶段已不可用时,改用完整输入重放。
///
/// 这不是强制 Smart:下一次 attempt 仍使用请求配置的调度策略。只有网关持有
/// 对应 Provider 的可携带会话状态时,才可安全移除 native handle 并换号。
fn prepare_unavailable_native_continuation_replay(&mut self, error: &ProviderError) -> bool {
if self.account_selection.required_account().is_some()
|| self.continuation_attempt != ContinuationAttempt::Native
|| !matches!(
error.kind(),
ProviderErrorKind::NoEligibleAccount
| ProviderErrorKind::AccountCapacityUnavailable
)
{
return false;
}
let Some(pin) = self
.continuation
.as_ref()
.and_then(ContinuationBinding::pinned)
else {
return false;
};
if self
.operation
.provider_session_state(pin.provider().as_str())
.is_none()
{
return false;
}
self.continuation_attempt = ContinuationAttempt::ReplayAny;
self.excluded_accounts.insert(pin.account().clone());
true
}
fn reset_uncommitted_observations(&mut self) {
self.usage = Usage::new();
self.cost = CostEstimate::unavailable();
@@ -215,6 +215,20 @@ impl AccountAvailability {
}
}
/// 返回账号进入不可恢复错误态、但 Provider 未给出更具体上游原因时的稳定兜底。
///
/// Provider 应优先持久化已脱敏的上游 message 或原因码;该值只避免错误态沿用
/// 上一次无关的错误信息。
#[must_use]
pub const fn fallback_error_reason(self) -> Option<&'static str> {
match self {
Self::Expired => Some("credential_expired"),
Self::Banned => Some("account_banned"),
Self::Invalid => Some("credential_invalid"),
Self::Unknown | Self::Ready | Self::QuotaExhausted => None,
}
}
/// 解析数据库稳定值。
#[must_use]
pub fn parse(value: &str) -> Option<Self> {
@@ -237,7 +251,7 @@ pub struct ProviderAccount {
provider: ProviderKind,
name: String,
email: Option<String>,
upstream_user_id: String,
upstream_user_id: Option<String>,
upstream_account_id: Option<String>,
plan_type: Option<String>,
authentication_kind: String,
@@ -256,7 +270,7 @@ impl ProviderAccount {
id: ProviderAccountId,
provider: ProviderKind,
name: String,
upstream_user_id: String,
upstream_user_id: Option<String>,
authentication_kind: String,
revision: CredentialRevision,
access_token_expires_at: Option<SystemTime>,
@@ -299,10 +313,18 @@ impl ProviderAccount {
availability: AccountAvailability,
) -> Self {
self.enabled = enabled;
self.availability = availability;
self.availability = if self.upstream_user_id.is_some() {
availability
} else {
AccountAvailability::Unknown
};
self
}
/// 设置 RT 存在性与失败后的最早重试时刻。
///
/// 正常 OAuth 预刷新由 worker 使用 AT 原始过期时间与当前运行时策略动态判断,
/// 不应把提前量物化到 `next_refresh_at`。
#[must_use]
pub const fn with_refresh_schedule(
mut self,
@@ -340,8 +362,8 @@ impl ProviderAccount {
}
#[must_use]
pub fn upstream_user_id(&self) -> &str {
&self.upstream_user_id
pub fn upstream_user_id(&self) -> Option<&str> {
self.upstream_user_id.as_deref()
}
#[must_use]
@@ -374,6 +396,7 @@ impl ProviderAccount {
self.access_token_expires_at
}
/// 返回瞬态 OAuth 刷新失败后的最早重试时刻;正常账号为 `None`。
#[must_use]
pub const fn next_refresh_at(&self) -> Option<SystemTime> {
self.next_refresh_at
@@ -395,7 +418,8 @@ impl ProviderAccount {
| AccountAvailability::Banned
| AccountAvailability::Invalid => false,
};
self.enabled
self.upstream_user_id.is_some()
&& self.enabled
&& available
&& self
.access_token_expires_at
@@ -607,6 +631,8 @@ pub struct AccountStateChange {
pub expected_revision: CredentialRevision,
pub availability: AccountAvailability,
pub observed_at: SystemTime,
/// 上游原因(错误原文 / 原因码);`Some` 写入持久化错误信息,`None` 不更新,恢复 `Ready` 时清空。
pub message: Option<String>,
}
/// `provider_accounts` 的数据库中立端口。
@@ -674,22 +674,20 @@ impl ProviderRefreshPolicy {
self.concurrency
}
/// 按配置的精确提前量计算刷新时刻。
pub fn next_attempt_at(
/// 判断 AT 是否已经进入当前配置的提前刷新窗口。
///
/// `margin` 是运行时策略,不投影为账号的持久化时间字段;已过期 AT 也需要
/// 尝试用 RT 恢复,因此视为到期。
#[must_use]
pub fn is_refresh_due(
self,
_account_id: &ProviderAccountId,
access_token_expires_at: SystemTime,
observed_at: SystemTime,
) -> Result<SystemTime, ProviderStoreError> {
let remaining = access_token_expires_at
.duration_since(observed_at)
.map_err(|_| invalid_refresh_policy("schedule expired access token"))?;
if self.margin >= remaining {
return Ok(observed_at);
) -> bool {
match access_token_expires_at.duration_since(observed_at) {
Ok(remaining) => remaining <= self.margin,
Err(_) => true,
}
access_token_expires_at
.checked_sub(self.margin)
.ok_or_else(|| invalid_refresh_policy("schedule access token refresh"))
}
}
@@ -513,17 +513,27 @@ fn image_stream(image_output_tokens: Option<u64>) -> Vec<Result<GatewayEvent, Pr
}
fn plan(operation: &Operation) -> RoutingPlan {
plan_with_policy(
operation,
AccountSelectionPolicy::new(
RotationStrategy::Smart,
NonZeroU32::new(2).expect("account concurrency"),
Duration::from_millis(50),
),
)
}
fn plan_with_policy(
operation: &Operation,
account_selection_policy: AccountSelectionPolicy,
) -> RoutingPlan {
let provider = ProviderKind::new("openai").expect("provider");
let public_model = PublicModelId::new("gpt-5").expect("public model");
let capabilities = ModelCapabilities::new(BTreeSet::from([operation.kind()]), Some(32_000))
.with_upstream_feature_validation();
RuntimeSnapshot::new(
ConfigRevision::new(1).expect("config revision"),
AccountSelectionPolicy::new(
RotationStrategy::Smart,
NonZeroU32::new(2).expect("account concurrency"),
Duration::from_millis(50),
),
account_selection_policy,
vec![provider.clone()],
vec![ProviderModel::new(
provider.clone(),
@@ -1514,6 +1524,70 @@ fn native_continuation_replays_owner_before_safely_switching_account() {
);
}
#[test]
fn unavailable_native_continuation_replays_with_the_configured_account_policy() {
let Operation::Generate(generate) = generate_operation() else {
panic!("generate operation");
};
let operation = Operation::Generate(generate.with_provider_session_state(
ProviderSessionState::new("openai", Map::new()).expect("provider session state"),
));
let policy = AccountSelectionPolicy::new(
RotationStrategy::RoundRobin,
NonZeroU32::new(2).expect("account concurrency"),
Duration::from_millis(50),
);
let route_plan = plan_with_policy(&operation, policy);
let (coordinator, store, provider) = coordinator(vec![
Script::Error(ProviderError::new(
ProviderErrorKind::NoEligibleAccount,
UpstreamSendState::NotSent,
)),
Script::Stream {
account_id: "acct_two",
items: complete_stream(None),
},
]);
let original = ProviderAccountId::new("acct_one").expect("account");
let continuation = NativeContinuationPin::new(
PreviousResponseId::new("previous-response"),
PreviousResponseId::new("upstream-response"),
ProviderKind::new("openai").expect("provider"),
original.clone(),
);
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
None,
Some(ContinuationBinding::Pinned(continuation)),
CancellationToken::new(),
))
.expect("start execution");
block_on(session.collect_uncommitted()).expect("portable replay succeeds");
block_on(session.commit_downstream(Some(200))).expect("commit response");
let contexts = provider.contexts.lock().expect("contexts lock");
assert_eq!(contexts.len(), 2);
assert_eq!(
contexts[0].continuation_attempt(),
ContinuationAttempt::Native
);
assert_eq!(
contexts[1].continuation_attempt(),
ContinuationAttempt::ReplayAny
);
assert_eq!(
contexts[1].account_selection_policy().strategy(),
RotationStrategy::RoundRobin
);
assert!(contexts[1].excluded_accounts().contains(&original));
let state = store.state.lock().expect("store lock");
assert_eq!(state.attempts.len(), 1);
assert_eq!(state.finalizations[0].outcome, ExecutionOutcome::Succeeded);
}
#[test]
fn diagnostic_account_reaches_provider_and_matching_metadata_succeeds() {
let operation = operation(RetrySafety::Idempotent);
@@ -19,7 +19,7 @@ fn account(id: &str) -> ProviderAccount {
ProviderAccountId::new(id).expect("valid account"),
ProviderKind::new("openai").expect("valid provider"),
id.to_owned(),
format!("user-{id}"),
Some(format!("user-{id}")),
"oauth".to_owned(),
CredentialRevision::new(1).expect("valid revision"),
Some(SystemTime::now() + Duration::from_secs(3600)),
@@ -160,6 +160,28 @@ fn disabled_account_should_not_be_schedulable() {
assert!(!account.is_schedulable(SystemTime::now()));
}
#[test]
fn account_without_upstream_identity_should_stay_unknown_and_not_schedulable() {
let account = ProviderAccount::new(
ProviderAccountId::new("acct_pending_identity").expect("valid account"),
ProviderKind::new("openai").expect("valid provider"),
"pending identity".to_owned(),
None,
"oauth".to_owned(),
CredentialRevision::new(1).expect("valid revision"),
Some(SystemTime::now() + Duration::from_secs(3600)),
)
.with_runtime_state(true, AccountAvailability::Ready);
assert_eq!(
(
account.availability(),
account.is_schedulable(SystemTime::now())
),
(AccountAvailability::Unknown, false)
);
}
#[test]
fn smart_selector_should_prefer_lower_inflight_count() {
let candidates = vec![
@@ -57,8 +57,7 @@ fn refresh_policy_requires_a_positive_margin() {
}
#[test]
fn refresh_schedule_should_use_the_exact_configured_margin() {
let account = ProviderAccountId::new("acct_stable").expect("valid account");
fn refresh_policy_should_mark_tokens_due_at_the_exact_configured_margin() {
let policy = ProviderRefreshPolicy::try_new(
Duration::from_secs(3_600),
NonZeroU32::new(2).expect("positive concurrency"),
@@ -67,15 +66,12 @@ fn refresh_schedule_should_use_the_exact_configured_margin() {
let observed_at = SystemTime::UNIX_EPOCH + Duration::from_secs(10_000);
let expires_at = observed_at + Duration::from_secs(7_200);
let first = policy
.next_attempt_at(&account, expires_at, observed_at)
.expect("schedule refresh");
assert_eq!(first, expires_at - Duration::from_secs(3_600));
assert!(!policy.is_refresh_due(expires_at, observed_at));
assert!(policy.is_refresh_due(observed_at + Duration::from_secs(3_600), observed_at));
}
#[test]
fn refresh_schedule_uses_observed_time_when_token_is_inside_safe_window() {
let account = ProviderAccountId::new("acct_boundary").expect("valid account");
fn refresh_policy_should_mark_expired_tokens_due() {
let policy = ProviderRefreshPolicy::try_new(
Duration::from_secs(3_600),
NonZeroU32::new(1).expect("positive concurrency"),
@@ -83,12 +79,7 @@ fn refresh_schedule_uses_observed_time_when_token_is_inside_safe_window() {
.expect("valid policy");
let observed_at = SystemTime::UNIX_EPOCH + Duration::from_secs(10_000);
assert_eq!(
policy
.next_attempt_at(&account, observed_at + Duration::from_secs(1), observed_at,)
.expect("schedule immediate refresh"),
observed_at
);
assert!(policy.is_refresh_due(observed_at - Duration::from_secs(1), observed_at));
}
#[test]
+80 -23
View File
@@ -13,6 +13,9 @@ use tracing_subscriber::{Layer as _, layer::SubscriberExt as _, util::Subscriber
use crate::config::LoggingConfig;
const LOG_FILE_PREFIX: &str = "codex-proxy-rs";
const OAUTH_RECOVERY_LOG_FILE_PREFIX: &str = "codex-proxy-rs-oath";
/// OAuth 恢复记录含原始 AT/RT,文件日志开启时必须落到独立文件。
const OAUTH_RECOVERY_LOG_TARGET: &str = "oauth_recovery";
/// non-blocking 日志 writer 的进程级守卫。
pub struct LogGuard {
@@ -34,7 +37,8 @@ pub enum LogError {
/// 按自然日、单文件大小、保留天数和文件总数初始化结构化日志。
pub fn initialize_logging(config: &LoggingConfig) -> Result<LogGuard, LogError> {
let directive = env::var("RUST_LOG").unwrap_or_else(|_| config.level.clone());
let file_filter = EnvFilter::try_new(&directive).map_err(|_| LogError::InvalidFilter)?;
let file_filter = application_file_filter(&directive)?;
let recovery_file_filter = oauth_recovery_file_filter();
let stdout_filter =
EnvFilter::try_new(stdout_filter_directive(&directive, config.file.enabled))
.map_err(|_| LogError::InvalidFilter)?;
@@ -47,25 +51,37 @@ pub fn initialize_logging(config: &LoggingConfig) -> Result<LogGuard, LogError>
guards.push(guard);
writer
});
let file_writer = if config.file.enabled {
let (file_writer, recovery_file_writer) = if config.file.enabled {
let maximum_bytes = config
.file
.max_file_size_mb
.checked_mul(1024 * 1024)
.ok_or(LogError::SizeOverflow)?;
let writer = RotatingLogWriter::open(
let file_writer = RotatingLogWriter::open(
config.file.directory.clone(),
LOG_FILE_PREFIX,
maximum_bytes,
config.file.retention_days,
config.file.max_files,
)?;
let (writer, guard) = NonBlockingBuilder::default()
let recovery_file_writer = RotatingLogWriter::open(
config.file.directory.clone(),
OAUTH_RECOVERY_LOG_FILE_PREFIX,
maximum_bytes,
config.file.retention_days,
config.file.max_files,
)?;
let (file_writer, file_guard) = NonBlockingBuilder::default()
.thread_name("gateway-log-file")
.finish(writer);
guards.push(guard);
Some(writer)
.finish(file_writer);
let (recovery_file_writer, recovery_file_guard) = NonBlockingBuilder::default()
.thread_name("gateway-log-oauth-recovery-file")
.finish(recovery_file_writer);
guards.push(file_guard);
guards.push(recovery_file_guard);
(Some(file_writer), Some(recovery_file_writer))
} else {
None
(None, None)
};
let stdout_layer = stdout_writer.map(|writer| {
@@ -89,14 +105,43 @@ pub fn initialize_logging(config: &LoggingConfig) -> Result<LogGuard, LogError>
.with_span_list(true)
.with_filter(file_filter)
});
let recovery_file_layer = recovery_file_writer.map(|writer| {
tracing_subscriber::fmt::layer()
.json()
.with_writer(writer)
.with_target(true)
.with_file(true)
.with_line_number(true)
.with_thread_ids(true)
.with_thread_names(true)
.with_current_span(true)
.with_span_list(true)
.with_filter(recovery_file_filter)
});
tracing_subscriber::registry()
.with(stdout_layer)
.with(file_layer)
.with(recovery_file_layer)
.try_init()
.map_err(|_| LogError::AlreadyInitialized)?;
Ok(LogGuard { _writers: guards })
}
fn application_file_filter(directive: &str) -> Result<EnvFilter, LogError> {
let recovery_directive = format!("{OAUTH_RECOVERY_LOG_TARGET}=off")
.parse()
.expect("static OAuth recovery log directive is valid");
EnvFilter::try_new(directive)
.map(|filter| filter.add_directive(recovery_directive))
.map_err(|_| LogError::InvalidFilter)
}
fn oauth_recovery_file_filter() -> EnvFilter {
format!("off,{OAUTH_RECOVERY_LOG_TARGET}=info")
.parse()
.expect("static OAuth recovery log filter is valid")
}
fn stdout_filter_directive(directive: &str, persistent_log_enabled: bool) -> String {
if !persistent_log_enabled {
directive.to_owned()
@@ -107,6 +152,7 @@ fn stdout_filter_directive(directive: &str, persistent_log_enabled: bool) -> Str
struct RotatingLogWriter {
directory: PathBuf,
file_prefix: &'static str,
maximum_bytes: u64,
retention_days: usize,
maximum_files: usize,
@@ -119,17 +165,20 @@ struct RotatingLogWriter {
impl RotatingLogWriter {
fn open(
directory: PathBuf,
file_prefix: &'static str,
maximum_bytes: u64,
retention_days: usize,
maximum_files: usize,
) -> io::Result<Self> {
fs::create_dir_all(&directory)?;
let date = Utc::now().date_naive();
cleanup_log_files(&directory, date, retention_days, maximum_files)?;
let (segment, bytes_written) = writable_log_segment(&directory, date, maximum_bytes)?;
let file = open_log_segment(&directory, date, segment)?;
cleanup_log_files(&directory, file_prefix, date, retention_days, maximum_files)?;
let (segment, bytes_written) =
writable_log_segment(&directory, file_prefix, date, maximum_bytes)?;
let file = open_log_segment(&directory, file_prefix, date, segment)?;
Ok(Self {
directory,
file_prefix,
maximum_bytes,
retention_days,
maximum_files,
@@ -157,10 +206,11 @@ impl RotatingLogWriter {
} else {
self.segment = self.segment.saturating_add(1);
}
self.file = open_log_segment(&self.directory, self.date, self.segment)?;
self.file = open_log_segment(&self.directory, self.file_prefix, self.date, self.segment)?;
self.bytes_written = self.file.metadata()?.len();
cleanup_log_files(
&self.directory,
self.file_prefix,
self.date,
self.retention_days,
self.maximum_files,
@@ -192,10 +242,11 @@ struct ManagedLogFile {
fn writable_log_segment(
directory: &Path,
file_prefix: &str,
date: chrono::NaiveDate,
maximum_bytes: u64,
) -> io::Result<(usize, u64)> {
let latest = managed_log_files(directory)?
let latest = managed_log_files(directory, file_prefix)?
.into_iter()
.filter(|entry| entry.date == date)
.max_by_key(|entry| entry.segment);
@@ -210,26 +261,31 @@ fn writable_log_segment(
}
}
fn open_log_segment(directory: &Path, date: chrono::NaiveDate, segment: usize) -> io::Result<File> {
fn open_log_segment(
directory: &Path,
file_prefix: &str,
date: chrono::NaiveDate,
segment: usize,
) -> io::Result<File> {
OpenOptions::new()
.create(true)
.append(true)
.open(directory.join(log_file_name(date, segment)))
.open(directory.join(log_file_name(file_prefix, date, segment)))
}
fn log_file_name(date: chrono::NaiveDate, segment: usize) -> String {
fn log_file_name(file_prefix: &str, date: chrono::NaiveDate, segment: usize) -> String {
if segment == 0 {
format!("{LOG_FILE_PREFIX}.{date}.log")
format!("{file_prefix}.{date}.log")
} else {
format!("{LOG_FILE_PREFIX}.{date}.{segment}.log")
format!("{file_prefix}.{date}.{segment}.log")
}
}
fn managed_log_files(directory: &Path) -> io::Result<Vec<ManagedLogFile>> {
fn managed_log_files(directory: &Path, file_prefix: &str) -> io::Result<Vec<ManagedLogFile>> {
Ok(fs::read_dir(directory)?
.filter_map(Result::ok)
.filter_map(|entry| {
parse_log_file_name(entry.file_name().to_string_lossy().as_ref()).map(
parse_log_file_name(entry.file_name().to_string_lossy().as_ref(), file_prefix).map(
|(date, segment)| ManagedLogFile {
date,
segment,
@@ -240,9 +296,9 @@ fn managed_log_files(directory: &Path) -> io::Result<Vec<ManagedLogFile>> {
.collect())
}
fn parse_log_file_name(name: &str) -> Option<(chrono::NaiveDate, usize)> {
fn parse_log_file_name(name: &str, file_prefix: &str) -> Option<(chrono::NaiveDate, usize)> {
let body = name
.strip_prefix(&format!("{LOG_FILE_PREFIX}."))?
.strip_prefix(&format!("{file_prefix}."))?
.strip_suffix(".log")?;
let (date, segment) = body.split_once('.').map_or((body, 0), |(date, segment)| {
(date, segment.parse().ok().unwrap_or(usize::MAX))
@@ -257,6 +313,7 @@ fn parse_log_file_name(name: &str) -> Option<(chrono::NaiveDate, usize)> {
fn cleanup_log_files(
directory: &Path,
file_prefix: &str,
today: chrono::NaiveDate,
retention_days: usize,
maximum_files: usize,
@@ -266,7 +323,7 @@ fn cleanup_log_files(
.checked_sub_signed(chrono::Duration::days(retention_offset))
.unwrap_or(chrono::NaiveDate::MIN);
let mut retained = Vec::new();
for entry in managed_log_files(directory)? {
for entry in managed_log_files(directory, file_prefix)? {
if entry.date < cutoff {
fs::remove_file(entry.path)?;
} else {
+126 -1
View File
@@ -1,8 +1,21 @@
use std::path::PathBuf;
use std::env;
use std::fs;
use std::path::{Path, PathBuf};
use std::process::Command;
use gateway_host::config::{FileLoggingConfig, HostConfig, ListenConfig, LoggingConfig};
use gateway_host::system_update::SystemUpdateConfig;
const LOG_DIRECTORY_ENV: &str = "CPR_LOGGING_TEST_DIRECTORY";
const CHILD_PROCESS_ENV: &str = "CPR_LOGGING_TEST_CHILD";
const APPLICATION_LOG_FILE_PREFIX: &str = "codex-proxy-rs.";
const OAUTH_RECOVERY_LOG_FILE_PREFIX: &str = "codex-proxy-rs-oath.";
const APPLICATION_LOG_TARGET: &str = "logging_test_application";
const APPLICATION_LOG_MARKER: &str = "application-file-filter-test";
const OAUTH_RECOVERY_LOG_TARGET: &str = "oauth_recovery";
const OAUTH_RECOVERY_LOG_MARKER: &str = "oauth-recovery-file-filter-test";
const OAUTH_RECOVERY_PROVIDER: &str = "openai";
#[test]
fn logging_requires_at_least_one_sink() {
let mut config = HostConfig {
@@ -32,3 +45,115 @@ fn logging_requires_at_least_one_sink() {
.is_err()
);
}
#[test]
fn oauth_recovery_file_logging_is_separate_and_overrides_global_log_level() {
if env::var_os(CHILD_PROCESS_ENV).is_some() {
write_oauth_recovery_log(PathBuf::from(
env::var_os(LOG_DIRECTORY_ENV).expect("child log directory"),
));
return;
}
let directory = tempfile::tempdir().expect("create log directory");
let output = Command::new(env::current_exe().expect("current test executable"))
.args([
"--exact",
"logging::oauth_recovery_file_logging_is_separate_and_overrides_global_log_level",
"--nocapture",
])
.env(CHILD_PROCESS_ENV, "1")
.env(LOG_DIRECTORY_ENV, directory.path())
.env(
"RUST_LOG",
"off,logging_test_application=info,oauth_recovery=off",
)
.output()
.expect("run logging child process");
assert!(
output.status.success(),
"logging child process failed:\nstdout:\n{}\nstderr:\n{}",
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr),
);
let application_log = read_log_file_set(directory.path(), APPLICATION_LOG_FILE_PREFIX);
assert!(application_log.contains(&json_target_field(APPLICATION_LOG_TARGET)));
assert!(application_log.contains(APPLICATION_LOG_MARKER));
assert!(!application_log.contains(&json_target_field(OAUTH_RECOVERY_LOG_TARGET)));
assert!(!application_log.contains(OAUTH_RECOVERY_LOG_MARKER));
let recovery_log = read_log_file_set(directory.path(), OAUTH_RECOVERY_LOG_FILE_PREFIX);
assert!(recovery_log.contains(&json_target_field(OAUTH_RECOVERY_LOG_TARGET)));
assert!(recovery_log.contains(OAUTH_RECOVERY_LOG_MARKER));
assert!(!recovery_log.contains(&json_target_field(APPLICATION_LOG_TARGET)));
assert!(!recovery_log.contains(APPLICATION_LOG_MARKER));
assert!(recovery_log.contains(&format!(r#""provider":"{OAUTH_RECOVERY_PROVIDER}""#)));
}
fn write_oauth_recovery_log(directory: PathBuf) {
let config = HostConfig {
listen: ListenConfig {
host: "127.0.0.1".to_owned(),
port: 8080,
},
logging: LoggingConfig {
level: "off".to_owned(),
stdout: false,
file: FileLoggingConfig {
enabled: true,
directory,
retention_days: 1,
max_file_size_mb: 1,
max_files: 1,
},
},
system_update: SystemUpdateConfig::default(),
drain_timeout_seconds: 30,
worker_shutdown_timeout_seconds: 30,
};
let runtime = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.expect("create logging runtime");
let bundle = runtime
.block_on(gateway_host::initialize(config))
.expect("initialize logging");
tracing::info!(
target: APPLICATION_LOG_TARGET,
marker = APPLICATION_LOG_MARKER,
"application file test record"
);
tracing::info!(
target: OAUTH_RECOVERY_LOG_TARGET,
provider = OAUTH_RECOVERY_PROVIDER,
marker = OAUTH_RECOVERY_LOG_MARKER,
"OAuth recovery test record"
);
drop(bundle);
}
fn read_log_file_set(directory: &Path, file_prefix: &str) -> String {
let mut files = fs::read_dir(directory)
.expect("read log directory")
.map(|entry| entry.expect("read log entry").path())
.filter(|path| {
path.file_name()
.and_then(|name| name.to_str())
.is_some_and(|name| name.starts_with(file_prefix))
})
.collect::<Vec<_>>();
files.sort();
assert!(!files.is_empty(), "expected log file set {file_prefix}");
files
.into_iter()
.map(|path| fs::read_to_string(path).expect("read log file"))
.collect()
}
fn json_target_field(target: &str) -> String {
format!(r#""target":"{target}""#)
}
@@ -315,6 +315,26 @@ pub fn is_rate_limit_header_name(name: &str) -> bool {
|| rate_limit_id_from_header_name(&normalized).is_some()
}
/// 判断响应头是否为仅供代理本地观测的 Codex 额度信号。
///
/// 这类头会让下游官方 Codex 客户端显示账户额度提示,因此不能穿过账号隔离边界。
/// 通用限流头(例如 `retry-after`)和非额度的 Codex 头仍允许下发。
pub fn is_codex_quota_header_name(name: &str) -> bool {
let normalized = name.trim().to_ascii_lowercase();
matches!(
normalized.as_str(),
"x-codex-credits-has-credits"
| "x-codex-credits-unlimited"
| "x-codex-credits-balance"
| "x-codex-active-limit"
| "x-codex-plan-type"
| "x-codex-promo-message"
| "x-codex-rate-limit-reached-type"
| "x-codex-primary-over-secondary-limit-percent"
) || rate_limit_id_from_header_name(&normalized)
.is_some_and(|limit_id| limit_id == "codex" || limit_id.starts_with("codex_"))
}
/// 从内部 `codex.rate_limits` 事件中解析限流信息。
pub fn parse_rate_limits_event(value: &Value) -> Option<ParsedRateLimits> {
if value.get("type").and_then(Value::as_str) != Some("codex.rate_limits") {
@@ -1,7 +1,7 @@
use gateway_protocol::openai::events::{
RateLimitWindow, TokenUsage, extract_sse_usage, extract_usage, is_rate_limit_header_name,
parse_rate_limit_headers, parse_rate_limits_event, rate_limits_to_header_pairs,
retry_after_seconds_from_body,
RateLimitWindow, TokenUsage, extract_sse_usage, extract_usage, is_codex_quota_header_name,
is_rate_limit_header_name, parse_rate_limit_headers, parse_rate_limits_event,
rate_limits_to_header_pairs, retry_after_seconds_from_body,
};
use serde_json::json;
@@ -537,3 +537,21 @@ fn rate_limit_header_filter_should_accept_only_supported_domain_headers() {
[true, true, true, false, false]
);
}
#[test]
fn codex_quota_header_filter_should_only_match_client_quota_signals() {
assert_eq!(
[
"X-CoDeX-Primary-Used-Percent",
"x-codex-code-review-primary-reset-at",
"x-codex-credits-balance",
"x-codex-primary-over-secondary-limit-percent",
"retry-after",
"x-ratelimit-remaining",
"x-codex-safety-buffering-enabled",
"x-codex-turn-state",
]
.map(is_codex_quota_header_name),
[true, true, true, true, false, false, false, false]
);
}
+1 -1
View File
@@ -37,7 +37,7 @@ uuid = { version = "1.23.3", features = ["v4", "v7"] }
[dev-dependencies]
tempfile = "3.23"
tokio = { version = "1.52.3", default-features = false, features = ["macros", "rt-multi-thread", "fs", "io-util", "process", "time"] }
tokio = { version = "1.52.3", default-features = false, features = ["macros", "rt-multi-thread", "fs", "io-util", "net", "process", "time"] }
[lints]
workspace = true
+12 -7
View File
@@ -385,7 +385,13 @@ async fn upload_parts(
BackupError::new(code::S3_UPLOAD_FAILED, "上传并发闸门关闭".to_owned())
})?;
let offset = (part - 1) * (PART_SIZE as u64);
let bytes = read_part(&source, offset, PART_SIZE).await?;
let remaining = file_size.checked_sub(offset).ok_or_else(|| {
BackupError::new(code::S3_UPLOAD_FAILED, "分片偏移超出归档大小".to_owned())
})?;
let part_len = usize::try_from(remaining.min(PART_SIZE as u64)).map_err(|_| {
BackupError::new(code::S3_UPLOAD_FAILED, "分片大小溢出".to_owned())
})?;
let bytes = read_part(&source, offset, part_len).await?;
let part_number = i32::try_from(part).map_err(|_| {
BackupError::new(code::S3_UPLOAD_FAILED, "分片号溢出".to_owned())
})?;
@@ -450,20 +456,19 @@ async fn upload_parts(
.map_err(map_s3_error)
}
/// 从文件中读取一个分片(最多 `PART_SIZE` 字节)。
async fn read_part(source: &Path, offset: u64, max_len: usize) -> Result<Vec<u8>, BackupError> {
/// 从文件中读取指定长度的完整分片。
async fn read_part(source: &Path, offset: u64, len: usize) -> Result<Vec<u8>, BackupError> {
let mut file = tokio::fs::File::open(source)
.await
.map_err(|_| BackupError::new(code::S3_UPLOAD_FAILED, "打开暂存归档失败".to_owned()))?;
file.seek(SeekFrom::Start(offset))
.await
.map_err(|_| BackupError::new(code::S3_UPLOAD_FAILED, "定位暂存归档失败".to_owned()))?;
let mut buffer = vec![0_u8; max_len];
let n = file
.read(&mut buffer)
let mut buffer = vec![0_u8; len];
// Tokio 默认单次文件读取最多 2 MiB;分片必须读取完整,不能把短读当作末尾分片上传。
file.read_exact(&mut buffer)
.await
.map_err(|_| BackupError::new(code::S3_UPLOAD_FAILED, "读取暂存归档失败".to_owned()))?;
buffer.truncate(n);
Ok(buffer)
}
+6 -1
View File
@@ -1,6 +1,7 @@
//! 完成连接、迁移与 hydration 的 Store 能力集合与启动屏障。
use gateway_core::engine::credential::ProviderAccountStore;
use gateway_core::provider_ports::ProviderCooldownPort;
use super::*;
@@ -85,7 +86,10 @@ pub async fn initialize(mut config: StoreConfig) -> StoreResult<StoreBundle> {
)?);
let admin_ports = AdminStorePorts::new(
Arc::new(postgres::PgAdminAccountStore::new(pool.clone())),
Arc::new(postgres::PgAdminAccountStore::new(
pool.clone(),
Some(Arc::clone(&cooldowns) as Arc<dyn ProviderCooldownPort>),
)),
Arc::new(AdminAuthStoreAdapter {
security: postgres::PgAdminSecurityAuditRepository::new(pool.clone()),
settings: postgres::PgRuntimeSettingsRepository::new(pool.clone()),
@@ -98,6 +102,7 @@ pub async fn initialize(mut config: StoreConfig) -> StoreResult<StoreBundle> {
Arc::new(postgres::PgAdminObservabilityStore::new(
pool.clone(),
Some(credential_leases.clone()),
Some(Arc::clone(&cooldowns) as Arc<dyn ProviderCooldownPort>),
)),
Arc::new(AdminSettingsStoreAdapter {
control_plane: postgres::PgControlPlaneRepository::new(pool.clone()),
@@ -1,5 +1,10 @@
//! Pg 观测 adapter:实现 `ObservabilityRepository` 与 `AdminObservabilityStore`。
use std::sync::Arc;
use std::time::SystemTime;
use gateway_core::provider_ports::ProviderCooldownPort;
use super::*;
use crate::redis::{CredentialLeaseRepository as _, RedisCredentialLeaseRepository};
@@ -7,12 +12,82 @@ use crate::redis::{CredentialLeaseRepository as _, RedisCredentialLeaseRepositor
#[derive(Clone)]
pub struct PgObservabilityRepository {
pool: PgPool,
cooldowns: Option<Arc<dyn ProviderCooldownPort>>,
}
impl PgObservabilityRepository {
#[must_use]
pub const fn new(pool: PgPool) -> Self {
Self { pool }
pub const fn new(pool: PgPool, cooldowns: Option<Arc<dyn ProviderCooldownPort>>) -> Self {
Self { pool, cooldowns }
}
/// 账号池指标:SQL 聚合后,仅用 Redis 429 冷却重分类仍可调度的 ready 账号。
/// 冷却中从 `active` 或 `quota_exhausted` 移到 `rate_limited`;冷却不可用时保留 SQL 分类。
async fn provider_account_metrics_with_cooldowns(
&self,
observed_at: DateTime<Utc>,
) -> StoreResult<ProviderAccountMetrics> {
let base = provider_account_metrics(&self.pool, observed_at).await?;
let Some(cooldowns) = &self.cooldowns else {
return Ok(base);
};
let candidates = schedulable_metric_candidates(&self.pool, observed_at).await?;
if candidates.is_empty() {
return Ok(base);
}
// Redis 只能把 SQL 已归入 active/quota_exhausted 的候选移到冷却桶;
// 其余状态不重建,避免 Redis 失败或脏 ID 时丢失持久化计数。
let mut active = base.active;
let mut rate_limited = base.rate_limited;
let mut quota_exhausted = base.quota_exhausted;
let now = SystemTime::now();
for (account_id, revision, quota_reached) in candidates {
let Ok(account_id) =
gateway_core::engine::credential::ProviderAccountId::new(account_id)
else {
continue;
};
let cooling = cooldowns
.read(&account_id)
.await
.ok()
.flatten()
.filter(|cooldown| cooldown.credential_revision().get() as i64 == revision)
.filter(|cooldown| cooldown.until() > now)
.is_some();
if !cooling {
continue;
}
if quota_reached {
if quota_exhausted == 0 {
continue;
}
quota_exhausted -= 1;
} else {
if active == 0 {
continue;
}
active -= 1;
}
rate_limited += 1;
}
Ok(ProviderAccountMetrics {
total: base.total,
enabled: base.enabled,
unavailable: rate_limited
+ quota_exhausted
+ base.expired
+ base.invalid
+ base.disabled
+ base.banned,
active,
rate_limited,
expired: base.expired,
invalid: base.invalid,
quota_exhausted,
disabled: base.disabled,
banned: base.banned,
})
}
}
@@ -28,9 +103,13 @@ pub struct PgAdminObservabilityStore {
impl PgAdminObservabilityStore {
#[must_use]
pub fn new(pool: PgPool, runtime_signals: Option<RedisCredentialLeaseRepository>) -> Self {
pub fn new(
pool: PgPool,
runtime_signals: Option<RedisCredentialLeaseRepository>,
cooldowns: Option<Arc<dyn ProviderCooldownPort>>,
) -> Self {
Self {
repository: PgObservabilityRepository::new(pool),
repository: PgObservabilityRepository::new(pool, cooldowns),
runtime_signals,
}
}
@@ -60,13 +139,15 @@ impl ObservabilityRepository for PgObservabilityRepository {
page: ObservabilityPageNumber::new(1)?,
page_size: ObservabilityPageSize::new(10)?,
};
// 分两批并发:页面延迟从六路之和降为两批各自最慢者之和,同时单次
// 渲染最多占用 3 个池连接,不与数据面写路径争抢整个连接池。
let (requests, attempts, provider_accounts) = futures::try_join!(
// 每批最多三路并发,既缩短概览渲染路径,也不与数据面写路径争抢整个连接池。
let (requests, attempts, totals) = futures::try_join!(
request_metrics(&self.pool, range, &filter),
attempt_metrics(&self.pool, range, &filter),
provider_account_metrics(&self.pool, range.end),
dashboard_totals(&self.pool),
)?;
let provider_accounts = self
.provider_account_metrics_with_cooldowns(range.end)
.await?;
let (trend, account_usage, recent_requests) = futures::try_join!(
request_metric_series(&self.pool, range, &filter),
provider_account_usage(&self.pool, account_usage_query),
@@ -76,6 +157,7 @@ impl ObservabilityRepository for PgObservabilityRepository {
range,
requests,
attempts,
totals,
provider_accounts,
trend,
account_usage,
@@ -103,6 +103,7 @@ pub(crate) fn admin_dashboard_observation(
range,
requests,
attempts,
totals,
provider_accounts,
trend,
account_usage,
@@ -112,6 +113,7 @@ pub(crate) fn admin_dashboard_observation(
range: admin_range(range),
requests: admin_request_metrics(requests)?,
attempts: admin_attempt_metrics(attempts)?,
totals: admin_dashboard_totals(totals)?,
provider_accounts: admin_account_pool_metrics(provider_accounts),
trend: trend
.into_iter()
@@ -128,6 +130,18 @@ pub(crate) fn admin_dashboard_observation(
})
}
pub(crate) fn admin_dashboard_totals(
totals: DashboardTotals,
) -> AdminStoreResult<admin_observability::DashboardTotals> {
Ok(admin_observability::DashboardTotals {
request_count: totals.request_count,
input_tokens: totals.input_tokens,
cached_tokens: totals.cached_tokens,
total_tokens: totals.total_tokens,
billing_usd: admin_optional_decimal_amount(totals.billing_usd)?,
})
}
pub(crate) const fn admin_range(range: ObservabilityRange) -> admin_observability::TimeRange {
admin_observability::TimeRange {
start: range.start,
@@ -477,11 +477,21 @@ impl ProviderAccountUsageQuery {
}
}
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct DashboardTotals {
pub request_count: u64,
pub input_tokens: u64,
pub cached_tokens: u64,
pub total_tokens: u64,
pub billing_usd: Option<DecimalAmount>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DashboardObservation {
pub range: ObservabilityRange,
pub requests: RequestMetrics,
pub attempts: AttemptMetrics,
pub totals: DashboardTotals,
pub provider_accounts: ProviderAccountMetrics,
pub trend: Vec<RequestMetricPoint>,
pub account_usage: Vec<ProviderAccountUsageObservation>,
@@ -65,6 +65,33 @@ pub(crate) async fn request_metrics(
request_metrics_from_row(&row)
}
pub(crate) async fn dashboard_totals(pool: &PgPool) -> StoreResult<DashboardTotals> {
// 卡片脚注的“总计”覆盖全部历史;只聚合页面实际展示的字段,避免重复计算
// 当前区间指标所需的延迟分位数。
let fact = completed_usage_fact_predicate("mr");
let mut query = QueryBuilder::<Postgres>::new(format!(
"select count(*)::bigint as request_count,
coalesce(sum(input_tokens) filter (where {fact}), 0)::bigint as input_tokens,
coalesce(sum(cached_tokens) filter (where {fact}), 0)::bigint as cached_tokens,
coalesce(sum(total_tokens) filter (where {fact}), 0)::bigint as total_tokens,
sum(cost_amount) filter (where {fact} and cost_currency = 'USD')::text
as billing_usd
from model_requests mr"
));
let row = query
.build()
.fetch_one(pool)
.await
.map_err(|_| postgres_unavailable("load dashboard totals"))?;
Ok(DashboardTotals {
request_count: unsigned(&row, "request_count")?,
input_tokens: unsigned(&row, "input_tokens")?,
cached_tokens: unsigned(&row, "cached_tokens")?,
total_tokens: unsigned(&row, "total_tokens")?,
billing_usd: optional_decimal(&row, "billing_usd")?,
})
}
pub(crate) async fn request_metric_series(
pool: &PgPool,
range: ObservabilityRange,
@@ -357,8 +384,8 @@ pub(crate) async fn provider_account_metrics(
when availability = 'expired'
or (access_token_expires_at is not null and access_token_expires_at <= $1)
then 'expired'
when quota_limit_reached then 'rate_limited'
when availability = 'unknown' then 'invalid'
when quota_limit_reached then 'quota_exhausted'
else 'active'
end as status
from provider_accounts
@@ -425,6 +452,35 @@ pub(crate) async fn active_provider_account_ids(
.collect()
}
/// 查询可被 Redis 冷却重新归类的 ready 账号:id、credential revision 与 quota 标记。
/// 已过期或非 ready 的账号由 [`provider_account_metrics`] 直接计数。
pub(crate) async fn schedulable_metric_candidates(
pool: &PgPool,
observed_at: DateTime<Utc>,
) -> StoreResult<Vec<(String, i64, bool)>> {
sqlx::query(
"select id, credential_revision, quota_limit_reached
from provider_accounts
where enabled
and availability = 'ready'
and (access_token_expires_at is null or access_token_expires_at > $1)
order by id",
)
.bind(observed_at)
.fetch_all(pool)
.await
.map_err(|_| postgres_unavailable("load schedulable metric candidates"))?
.into_iter()
.map(|row| {
Ok((
get(&row, "id")?,
get(&row, "credential_revision")?,
get(&row, "quota_limit_reached")?,
))
})
.collect::<StoreResult<Vec<_>>>()
}
pub(crate) async fn provider_observations(
pool: &PgPool,
range: ObservabilityRange,
@@ -1,5 +1,11 @@
//! `gateway-admin` 账号端口的 PostgreSQL adapter。
use std::collections::BTreeMap;
use std::sync::Arc;
use std::time::SystemTime;
use gateway_core::provider_ports::ProviderCooldownPort;
use super::*;
/// Admin 账号用例所需的公共账号、留存观测与 revision 事务能力。
@@ -11,19 +17,49 @@ pub struct PgAdminAccountStore {
accounts: PgProviderAccountRepository,
observability: PgObservabilityRepository,
control_plane: PgControlPlaneRepository,
cooldowns: Option<Arc<dyn ProviderCooldownPort>>,
}
impl PgAdminAccountStore {
#[must_use]
pub fn new(pool: PgPool) -> Self {
pub fn new(pool: PgPool, cooldowns: Option<Arc<dyn ProviderCooldownPort>>) -> Self {
Self {
pool: pool.clone(),
accounts: PgProviderAccountRepository::new(pool.clone()),
observability: PgObservabilityRepository::new(pool.clone()),
observability: PgObservabilityRepository::new(pool.clone(), cooldowns.clone()),
control_plane: PgControlPlaneRepository::new(pool),
cooldowns,
}
}
/// 批量读取 Redis 429 冷却,返回 `account_id → rate_limited_until` 映射。
/// 冷却不可用(Redis 挂/无 cooldown 端口)时返回空映射,账号归入配额耗尽(近似)。
async fn load_rate_limited_until(
&self,
accounts: &[ProviderAccountSummary],
) -> AdminStoreResult<BTreeMap<String, SystemTime>> {
let Some(cooldowns) = &self.cooldowns else {
return Ok(BTreeMap::new());
};
let mut map = BTreeMap::new();
for account in accounts {
let Ok(account_id) = CoreProviderAccountId::new(account.id.clone()) else {
continue;
};
let Some(cooldown) = cooldowns.read(&account_id).await.ok().flatten() else {
continue;
};
if cooldown.credential_revision().get() != account.credential_revision.get() {
continue;
}
let until = cooldown.until();
if until > SystemTime::now() {
map.insert(account.id.clone(), until);
}
}
Ok(map)
}
async fn usage_observations(
&self,
range: ObservabilityRange,
@@ -242,11 +278,17 @@ impl AccountStore for PgAdminAccountStore {
)
.map_err(|error| admin_store_error(ENTITY, error))?;
let now = Utc::now();
let summary = admin_account_summary(&accounts, now);
// 批量读取 Redis 429 冷却,供列表状态派生与筛选使用。
let rate_limited_until = self
.load_rate_limited_until(&accounts)
.await
.unwrap_or_default();
let summary = admin_account_summary(&accounts, now, &rate_limited_until);
let mut items = accounts
.into_iter()
.filter_map(|account| {
let status = admin_account_status(&account, now);
let until = rate_limited_until.get(&account.id).copied();
let status = admin_account_status(&account, now, until);
account_matches_admin_query(&account, status, &query).then_some(
AdminAccountListItem {
account,
@@ -19,7 +19,7 @@ impl ProviderAccountStore for PgProviderAccountRepository {
provider_kind: account.account.provider().as_str().to_owned(),
name: account.account.name().to_owned(),
email: account.account.email().map(str::to_owned),
upstream_user_id: account.account.upstream_user_id().to_owned(),
upstream_user_id: account.account.upstream_user_id().map(str::to_owned),
upstream_account_id: account.account.upstream_account_id().map(str::to_owned),
plan_type: account.account.plan_type().map(str::to_owned),
authentication_kind: account.account.authentication_kind().to_owned(),
@@ -247,6 +247,7 @@ impl ProviderAccountStore for PgProviderAccountRepository {
.map_err(core_store_error)?,
availability: change.availability,
availability_observed_at: DateTime::<Utc>::from(change.observed_at),
message: change.message,
})
.await
.map_err(core_store_error)?;
@@ -2,6 +2,8 @@
use super::*;
use gateway_admin::model::accounts::AccountStatusSignals;
pub(crate) struct AdminAccountListItem {
pub(crate) account: ProviderAccountSummary,
pub(crate) status: AdminAccountStatus,
@@ -45,7 +47,7 @@ pub(crate) fn account_matches_admin_query(
Some(account.name.as_str()),
account.email.as_deref(),
account.upstream_account_id.as_deref(),
Some(account.upstream_user_id.as_str()),
account.upstream_user_id.as_deref(),
]
.into_iter()
.flatten()
@@ -58,52 +60,48 @@ pub(crate) fn account_matches_admin_query(
pub(crate) fn admin_account_status(
account: &ProviderAccountSummary,
now: DateTime<Utc>,
rate_limited_until: Option<SystemTime>,
) -> AdminAccountStatus {
if !account.enabled {
AdminAccountStatus::Disabled
} else if account.availability == AccountAvailability::Banned {
AdminAccountStatus::Banned
} else if account.availability == AccountAvailability::QuotaExhausted {
AdminAccountStatus::QuotaExhausted
} else if account.availability == AccountAvailability::Expired
|| account.availability == AccountAvailability::Invalid
|| account
// 派生规则唯一归属 gateway-admin 域层,此处只做类型适配。
AccountStatusSignals {
enabled: account.enabled,
availability: account.availability,
access_token_expired: account
.access_token_expires_at
.is_some_and(|expires_at| expires_at <= now)
{
AdminAccountStatus::Expired
} else if account.quota_limit_reached {
AdminAccountStatus::RateLimited
} else if account.availability == AccountAvailability::Unknown {
AdminAccountStatus::Invalid
} else {
AdminAccountStatus::Active
.is_some_and(|expires_at| expires_at <= now),
quota_limit_reached: account.quota_limit_reached,
rate_limited_until,
}
.derive()
}
pub(crate) fn admin_account_summary(
accounts: &[ProviderAccountSummary],
now: DateTime<Utc>,
rate_limited_until: &BTreeMap<String, SystemTime>,
) -> AccountSummary {
let mut summary = AccountSummary {
total: u64::try_from(accounts.len()).unwrap_or(u64::MAX),
active: 0,
normal: 0,
quota_exhausted: 0,
unavailable: 0,
rate_limited: 0,
disabled: 0,
error: 0,
};
for account in accounts {
match admin_account_status(account, now) {
AdminAccountStatus::Active => summary.active = summary.active.saturating_add(1),
let until = rate_limited_until.get(&account.id).copied();
match admin_account_status(account, now, until) {
AdminAccountStatus::Normal => summary.normal = summary.normal.saturating_add(1),
AdminAccountStatus::QuotaExhausted => {
summary.quota_exhausted = summary.quota_exhausted.saturating_add(1);
}
AdminAccountStatus::RateLimited
| AdminAccountStatus::Expired
| AdminAccountStatus::Invalid
| AdminAccountStatus::Disabled
| AdminAccountStatus::Banned => {
summary.unavailable = summary.unavailable.saturating_add(1);
AdminAccountStatus::RateLimited => {
summary.rate_limited = summary.rate_limited.saturating_add(1);
}
AdminAccountStatus::Disabled => {
summary.disabled = summary.disabled.saturating_add(1);
}
AdminAccountStatus::Error => summary.error = summary.error.saturating_add(1),
}
}
summary
@@ -120,9 +118,7 @@ pub(crate) fn sort_admin_account_items(
items.sort_by(|left, right| {
let ordering = match sort.field {
AdminAccountSortField::Email => left.account.email.cmp(&right.account.email),
AdminAccountSortField::Status => {
admin_account_status_name(left.status).cmp(admin_account_status_name(right.status))
}
AdminAccountSortField::Status => left.status.as_str().cmp(right.status.as_str()),
AdminAccountSortField::PlanType => left.account.plan_type.cmp(&right.account.plan_type),
AdminAccountSortField::Usage => left
.usage
@@ -152,18 +148,6 @@ pub(crate) fn sort_admin_account_items(
});
}
pub(crate) const fn admin_account_status_name(status: AdminAccountStatus) -> &'static str {
match status {
AdminAccountStatus::Active => "active",
AdminAccountStatus::RateLimited => "rate_limited",
AdminAccountStatus::Expired => "expired",
AdminAccountStatus::Invalid => "invalid",
AdminAccountStatus::QuotaExhausted => "quota_exhausted",
AdminAccountStatus::Disabled => "disabled",
AdminAccountStatus::Banned => "banned",
}
}
pub(crate) fn admin_account_record(
summary: ProviderAccountSummary,
) -> AdminStoreResult<AccountRecord> {
@@ -189,6 +173,7 @@ pub(crate) fn admin_account_record(
enabled: summary.enabled,
availability: summary.availability,
availability_observed_at: summary.availability_observed_at,
last_error_message: summary.last_error_message,
quota_observed_at: summary.quota_observed_at,
created_at: summary.created_at,
updated_at: summary.updated_at,
@@ -4,6 +4,7 @@ use std::{
collections::{BTreeMap, BTreeSet},
fmt,
str::FromStr,
time::SystemTime,
};
use async_trait::async_trait;
@@ -95,7 +95,7 @@ impl ProviderAccountRepository for PgProviderAccountRepository {
upstream_account_id, plan_type, authentication_kind, credential_revision, has_refresh_token,
access_token_expires_at, next_refresh_at, enabled, availability,
availability_observed_at,
quota_observed_at, quota_limit_reached, created_at, updated_at
quota_observed_at, quota_limit_reached, last_error_message, created_at, updated_at
from provider_accounts
where ($1::text is null or provider_kind = $1) and ($2 or enabled)
order by provider_kind, name, id",
@@ -110,6 +110,11 @@ impl ProviderAccountRepository for PgProviderAccountRepository {
async fn insert_provider_account(&self, account: NewProviderAccount) -> StoreResult<()> {
account.validate()?;
let availability = if account.upstream_user_id.is_some() {
account.availability
} else {
AccountAvailability::Unknown
};
sqlx::query(
"insert into provider_accounts (
id, provider_kind, name, email, upstream_user_id,
@@ -135,7 +140,7 @@ impl ProviderAccountRepository for PgProviderAccountRepository {
.bind(account.access_token_expires_at)
.bind(account.next_refresh_at)
.bind(account.enabled)
.bind(account.availability.as_str())
.bind(availability.as_str())
.bind(account.availability_observed_at)
.execute(&self.pool)
.await
@@ -209,11 +214,19 @@ impl ProviderAccountRepository for PgProviderAccountRepository {
update.validate()?;
let result = sqlx::query(
"update provider_accounts
set availability = case when enabled then $3 else availability end,
set availability = case
when enabled and upstream_user_id is not null then $3
else availability
end,
availability_observed_at = case
when enabled then $4
when enabled and upstream_user_id is not null then $4
else availability_observed_at
end,
last_error_message = case
when enabled and upstream_user_id is not null then
case when $3 = 'ready' then null else coalesce($5, last_error_message) end
else last_error_message
end,
updated_at = case when enabled then greatest(now(), $4) else updated_at end
where id = $1 and credential_revision = $2
and (availability_observed_at is null or availability_observed_at <= $4)",
@@ -222,6 +235,7 @@ impl ProviderAccountRepository for PgProviderAccountRepository {
.bind(to_i64(update.expected_revision.get())?)
.bind(update.availability.as_str())
.bind(update.availability_observed_at)
.bind(update.message.as_deref())
.execute(&self.pool)
.await
.map_err(|_| postgres_unavailable("apply provider account state"))?;
@@ -460,6 +474,11 @@ pub(crate) async fn upsert_provider_account_in_transaction(
account: &NewProviderAccount,
) -> StoreResult<String> {
account.validate()?;
let availability = if account.upstream_user_id.is_some() {
account.availability
} else {
AccountAvailability::Unknown
};
let imported_id = sqlx::query_scalar::<_, String>(
"insert into provider_accounts (
id, provider_kind, name, email, upstream_user_id,
@@ -506,7 +525,7 @@ pub(crate) async fn upsert_provider_account_in_transaction(
.bind(account.access_token_expires_at)
.bind(account.next_refresh_at)
.bind(account.enabled)
.bind(account.availability.as_str())
.bind(availability.as_str())
.bind(account.availability_observed_at)
.fetch_optional(&mut **transaction)
.await
@@ -557,8 +576,9 @@ pub(crate) async fn rotate_provider_account_in_transaction(
upstream_account_id = case when $11::boolean then $13::text else upstream_account_id end,
availability = case
when not enabled then availability
when availability <> 'quota_exhausted' then 'ready'
else availability
when availability = 'quota_exhausted' then availability
when coalesce($12::text, upstream_user_id) is not null then 'ready'
else 'unknown'
end,
availability_observed_at = case
when not enabled then availability_observed_at
@@ -76,7 +76,7 @@ pub struct ProviderAccountSummary {
pub provider_kind: String,
pub name: String,
pub email: Option<String>,
pub upstream_user_id: String,
pub upstream_user_id: Option<String>,
pub upstream_account_id: Option<String>,
pub plan_type: Option<String>,
pub authentication_kind: String,
@@ -90,6 +90,8 @@ pub struct ProviderAccountSummary {
pub quota_observed_at: Option<DateTime<Utc>>,
/// 快照级限流事实(Provider 物化)。
pub quota_limit_reached: bool,
/// 最近一次非 ready 状态变更的原始原因(上游错误原文 / 原因码);恢复 ready 后清空。
pub last_error_message: Option<String>,
pub created_at: DateTime<Utc>,
pub updated_at: DateTime<Utc>,
}
@@ -121,7 +123,7 @@ pub struct NewProviderAccount {
pub provider_kind: String,
pub name: String,
pub email: Option<String>,
pub upstream_user_id: String,
pub upstream_user_id: Option<String>,
pub upstream_account_id: Option<String>,
pub plan_type: Option<String>,
pub authentication_kind: String,
@@ -161,7 +163,9 @@ impl NewProviderAccount {
require_nonempty(ENTITY, "id", &self.id)?;
require_nonempty(ENTITY, "provider_kind", &self.provider_kind)?;
require_nonempty(ENTITY, "name", &self.name)?;
require_nonempty(ENTITY, "upstream_user_id", &self.upstream_user_id)?;
if let Some(upstream_user_id) = &self.upstream_user_id {
require_nonempty(ENTITY, "upstream_user_id", upstream_user_id)?;
}
require_nonempty(ENTITY, "authentication_kind", &self.authentication_kind)?;
if !self.has_refresh_token && self.next_refresh_at.is_some() {
return Err(invalid("next_refresh_at requires a refresh token"));
@@ -310,6 +314,8 @@ pub struct ProviderAccountStateUpdate {
pub expected_revision: Revision,
pub availability: AccountAvailability,
pub availability_observed_at: DateTime<Utc>,
/// 上游原因(错误原文 / 原因码);`None` 不更新错误信息列,恢复 `Ready` 时清空。
pub message: Option<String>,
}
impl ProviderAccountStateUpdate {
@@ -335,14 +341,14 @@ impl ProviderAccountObservation {
pub(crate) const ACCOUNT_SELECT: &str = "select id, provider_kind, name, email, upstream_user_id,
upstream_account_id, plan_type, authentication_kind, provider_credentials_json, credential_revision,
has_refresh_token, access_token_expires_at, next_refresh_at, enabled, availability,
provider_quota_json, quota_limit_reached,
provider_quota_json, quota_limit_reached, last_error_message,
availability_observed_at, quota_observed_at, created_at, updated_at
from provider_accounts where id = $1";
pub(crate) const ACCOUNT_SELECT_BY_IDS: &str = "select id, provider_kind, name, email, upstream_user_id,
upstream_account_id, plan_type, authentication_kind, provider_credentials_json, credential_revision,
has_refresh_token, access_token_expires_at, next_refresh_at, enabled, availability,
provider_quota_json, quota_limit_reached,
provider_quota_json, quota_limit_reached, last_error_message,
availability_observed_at, quota_observed_at, created_at, updated_at
from provider_accounts
where id = any($1::text[]) and provider_kind = $2
@@ -445,6 +451,7 @@ pub(crate) fn account_summary_from_row(
availability_observed_at: get(&row, "availability_observed_at")?,
quota_observed_at: get(&row, "quota_observed_at")?,
quota_limit_reached: get(&row, "quota_limit_reached")?,
last_error_message: get(&row, "last_error_message")?,
created_at: get(&row, "created_at")?,
updated_at: get(&row, "updated_at")?,
})
@@ -225,19 +225,6 @@ pub(crate) async fn update_runtime_settings_in_transaction(
update.validate()?;
let refresh_margin_seconds =
i64::try_from(update.refresh_margin_seconds).map_err(|_| invalid_numeric())?;
let previous_refresh_margin_seconds = sqlx::query_scalar::<_, i64>(
"select refresh_margin_seconds
from runtime_settings
where id = 1
for update",
)
.fetch_optional(&mut **transaction)
.await
.map_err(|_| postgres_unavailable("lock runtime settings for update"))?
.ok_or_else(|| StoreError::NotFound {
entity: "runtime settings",
id: "1".to_owned(),
})?;
let next = sqlx::query_scalar::<_, i64>(
"update runtime_settings
set config_revision = config_revision + 1,
@@ -272,35 +259,9 @@ pub(crate) async fn update_runtime_settings_in_transaction(
entity: "runtime settings",
id: "1".to_owned(),
})?;
if previous_refresh_margin_seconds != refresh_margin_seconds {
reschedule_provider_refreshes_in_transaction(transaction, refresh_margin_seconds).await?;
}
Revision::new(u64::try_from(next).map_err(|_| invalid_numeric())?)
}
async fn reschedule_provider_refreshes_in_transaction(
transaction: &mut Transaction<'_, Postgres>,
refresh_margin_seconds: i64,
) -> StoreResult<()> {
sqlx::query(
"update provider_accounts
set next_refresh_at = case
when access_token_expires_at <= now()
or extract(epoch from (access_token_expires_at - now())) <= $1
then now()
else access_token_expires_at - ($1::double precision * interval '1 second')
end,
updated_at = now()
where has_refresh_token
and access_token_expires_at is not null",
)
.bind(refresh_margin_seconds)
.execute(&mut **transaction)
.await
.map_err(|_| postgres_unavailable("reschedule provider credential refreshes"))?;
Ok(())
}
pub(crate) async fn bump_config_revision_in_transaction(
transaction: &mut Transaction<'_, Postgres>,
) -> StoreResult<Revision> {
@@ -1,3 +1,4 @@
mod postgres;
mod redis;
mod s3;
mod support;
@@ -156,7 +156,12 @@ async fn connect_and_migrate_should_apply_all_migrations_once_and_reopen_cleanly
.expect("drop migration test database");
admin.close().await;
assert_eq!((first_table_count, migration_count), (10, 4));
assert_eq!(first_table_count, 10);
assert_eq!(
migration_count,
i64::try_from(TEST_MIGRATOR.iter().count())
.expect("migration count fits PostgreSQL bigint")
);
assert_eq!(response_id_types, ["bytea", "bytea"]);
assert!(!raw_response_id_index_exists);
}
@@ -1,10 +1,23 @@
use std::num::NonZeroU32;
use std::{
collections::BTreeMap,
num::NonZeroU32,
sync::Arc,
time::{Duration, SystemTime},
};
use chrono::{TimeDelta, Utc};
use futures::future::BoxFuture;
use gateway_admin::{
model::{PageSize, observability as admin_observability},
ports::store::ObservabilityStore as AdminObservabilityStore,
};
use gateway_core::{
engine::credential::{CredentialRevision, ProviderAccountId},
provider_ports::{
ProviderCooldown, ProviderCooldownPort, ProviderCooldownScope, ProviderScopedCooldown,
ProviderStoreError,
},
};
use gateway_store::postgres::{
DiagnosticDimension, ObservabilityPageNumber, ObservabilityPageSize, ObservabilityRange,
ObservabilityRepository, OpsErrorFilter, OpsErrorQuery, PgAdminObservabilityStore,
@@ -83,7 +96,7 @@ async fn observability_preserves_and_filters_opaque_response_ids() {
.expect("persist opaque response IDs");
let range = ObservabilityRange::new(now - TimeDelta::hours(1), now + TimeDelta::hours(1))
.expect("observability range");
let repository = PgObservabilityRepository::new(database.pool.clone());
let repository = PgObservabilityRepository::new(database.pool.clone(), None);
let records = repository
.list_usage_records(UsageRecordQuery {
@@ -147,7 +160,7 @@ async fn dashboard_account_metrics_should_partition_account_statuses() {
.execute(&database.pool)
.await
.expect("seed account metric states");
let repository = PgObservabilityRepository::new(database.pool.clone());
let repository = PgObservabilityRepository::new(database.pool.clone(), None);
let range = ObservabilityRange::new(now - TimeDelta::hours(1), now + TimeDelta::hours(1))
.expect("dashboard range");
@@ -179,6 +192,184 @@ async fn dashboard_account_metrics_should_partition_account_statuses() {
database.close().await;
}
#[tokio::test]
async fn dashboard_account_metrics_with_cooldowns_should_only_reclassify_eligible_accounts() {
let Some(database) = TestDatabase::create("dashboard_account_metrics_cooldowns").await else {
return;
};
let now = Utc::now();
sqlx::query(
"insert into provider_accounts (
id, provider_kind, name, upstream_user_id, authentication_kind,
provider_credentials_json, credential_revision, has_refresh_token,
access_token_expires_at, enabled, availability, quota_limit_reached,
availability_observed_at, created_at, updated_at
) values
('acct_metrics_active', 'openai', 'active', 'user-active', 'oauth',
'{}'::jsonb, 1, false, $1 + interval '1 day', true, 'ready', false,
$1, $1, $1),
('acct_metrics_active_cooling', 'openai', 'active-cooling', 'user-active-cooling', 'oauth',
'{}'::jsonb, 2, false, $1 + interval '1 day', true, 'ready', false,
$1, $1, $1),
('acct_metrics_ready_quota', 'openai', 'ready-quota', 'user-ready-quota', 'oauth',
'{}'::jsonb, 1, false, $1 + interval '1 day', true, 'ready', true,
$1, $1, $1),
('acct_metrics_cooling', 'openai', 'cooling', 'user-cooling', 'oauth',
'{}'::jsonb, 3, false, $1 + interval '1 day', true, 'ready', true,
$1, $1, $1),
('acct_metrics_stale_cooldown', 'openai', 'stale-cooldown', 'user-stale-cooldown', 'oauth',
'{}'::jsonb, 4, false, $1 + interval '1 day', true, 'ready', true,
$1, $1, $1),
('acct_metrics_persistent_quota', 'openai', 'persistent-quota', 'user-persistent-quota', 'oauth',
'{}'::jsonb, 1, false, $1 + interval '1 day', true, 'quota_exhausted', true,
$1, $1, $1),
('acct_metrics_expired', 'openai', 'expired', 'user-expired', 'oauth',
'{}'::jsonb, 5, false, $1 - interval '1 second', true, 'ready', true,
$1, $1, $1),
('acct_metrics_unknown', 'openai', 'unknown', 'user-unknown', 'oauth',
'{}'::jsonb, 6, false, $1 + interval '1 day', true, 'unknown', false,
$1, $1, $1),
('acct_metrics_disabled', 'openai', 'disabled', 'user-disabled', 'oauth',
'{}'::jsonb, 1, false, $1 + interval '1 day', false, 'ready', false,
$1, $1, $1)",
)
.bind(now)
.execute(&database.pool)
.await
.expect("seed cooldown account metric states");
let cooldowns = StaticCooldowns::new([
test_cooldown("acct_metrics_active_cooling", 2),
test_cooldown("acct_metrics_cooling", 3),
test_cooldown("acct_metrics_expired", 5),
test_cooldown("acct_metrics_unknown", 6),
test_cooldown("acct_metrics_stale_cooldown", 3),
]);
let repository =
PgObservabilityRepository::new(database.pool.clone(), Some(Arc::new(cooldowns)));
let range = ObservabilityRange::new(now - TimeDelta::hours(1), now + TimeDelta::hours(1))
.expect("dashboard range");
let metrics = repository
.dashboard_summary(range)
.await
.expect("dashboard summary")
.provider_accounts;
assert_eq!(
(
metrics.total,
metrics.enabled,
metrics.active,
metrics.rate_limited,
metrics.quota_exhausted,
metrics.expired,
metrics.invalid,
metrics.disabled,
metrics.banned,
metrics.unavailable,
),
(9, 8, 1, 2, 3, 1, 1, 1, 0, 8),
);
assert_eq!(
metrics.total,
metrics.active
+ metrics.rate_limited
+ metrics.quota_exhausted
+ metrics.expired
+ metrics.invalid
+ metrics.disabled
+ metrics.banned
);
assert_eq!(
metrics.unavailable,
metrics.rate_limited
+ metrics.quota_exhausted
+ metrics.expired
+ metrics.invalid
+ metrics.disabled
+ metrics.banned
);
database.close().await;
}
struct StaticCooldowns {
cooldowns: BTreeMap<ProviderAccountId, ProviderCooldown>,
}
impl StaticCooldowns {
fn new(cooldowns: impl IntoIterator<Item = ProviderCooldown>) -> Self {
Self {
cooldowns: cooldowns
.into_iter()
.map(|cooldown| (cooldown.account_id().clone(), cooldown))
.collect(),
}
}
}
impl ProviderCooldownPort for StaticCooldowns {
fn put_if_later(
&self,
_cooldown: ProviderCooldown,
) -> BoxFuture<'_, Result<bool, ProviderStoreError>> {
Box::pin(async { Ok(false) })
}
fn read<'a>(
&'a self,
account_id: &'a ProviderAccountId,
) -> BoxFuture<'a, Result<Option<ProviderCooldown>, ProviderStoreError>> {
Box::pin(async move { Ok(self.cooldowns.get(account_id).cloned()) })
}
fn clear<'a>(
&'a self,
_account_id: &'a ProviderAccountId,
_through_revision: CredentialRevision,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async { Ok(false) })
}
fn put_scoped_if_later(
&self,
_cooldown: ProviderScopedCooldown,
) -> BoxFuture<'_, Result<bool, ProviderStoreError>> {
Box::pin(async { Ok(false) })
}
fn read_scoped<'a>(
&'a self,
_account_id: &'a ProviderAccountId,
_scope: &'a ProviderCooldownScope,
) -> BoxFuture<'a, Result<Option<ProviderScopedCooldown>, ProviderStoreError>> {
Box::pin(async { Ok(None) })
}
fn clear_scoped<'a>(
&'a self,
_account_id: &'a ProviderAccountId,
_scope: &'a ProviderCooldownScope,
_through_revision: CredentialRevision,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async { Ok(false) })
}
fn clear_all<'a>(
&'a self,
_account_id: &'a ProviderAccountId,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async { Ok(false) })
}
}
fn test_cooldown(account_id: &str, revision: u64) -> ProviderCooldown {
ProviderCooldown::new(
ProviderAccountId::new(account_id).expect("test account ID"),
CredentialRevision::new(revision).expect("test revision"),
SystemTime::now() + Duration::from_secs(60),
)
}
#[tokio::test]
async fn calculated_usage_billing_facts_keep_only_completed_calculated_costs() {
let Some(database) = TestDatabase::create("calculated_usage_billing_facts").await else {
@@ -193,7 +384,7 @@ async fn calculated_usage_billing_facts_keep_only_completed_calculated_costs() {
.expect("seed calculated billing facts");
let range = ObservabilityRange::new(now - TimeDelta::hours(1), now + TimeDelta::hours(1))
.expect("observability range");
let repository = PgObservabilityRepository::new(database.pool.clone());
let repository = PgObservabilityRepository::new(database.pool.clone(), None);
let facts = repository
.usage_calculated_billing_facts(range, UsageRecordFilter::default())
@@ -207,7 +398,7 @@ async fn calculated_usage_billing_facts_keep_only_completed_calculated_costs() {
assert_eq!(facts[0].service_tier.as_deref(), Some("priority"));
assert_eq!(facts[0].total.amount.as_str(), "1.25");
let store = PgAdminObservabilityStore::new(database.pool.clone(), None);
let store = PgAdminObservabilityStore::new(database.pool.clone(), None, None);
let facts = store
.usage_calculated_billing_facts(
admin_observability::TimeRange::new(range.start, range.end)
@@ -235,7 +426,7 @@ async fn admin_observability_adapter_preserves_utc_queries_metrics_costs_and_det
let range =
admin_observability::TimeRange::new(now - TimeDelta::hours(1), now + TimeDelta::hours(1))
.expect("admin observability range");
let store = PgAdminObservabilityStore::new(database.pool.clone(), None);
let store = PgAdminObservabilityStore::new(database.pool.clone(), None, None);
let dashboard = store
.dashboard_summary(range)
@@ -513,6 +704,54 @@ fn admin_page(value: u32) -> admin_observability::PageNumber {
admin_observability::PageNumber::new(NonZeroU32::new(value).expect("positive page"))
}
#[tokio::test]
async fn dashboard_summary_totals_include_history_outside_selected_range() {
let Some(database) = TestDatabase::create("observability_dashboard_totals").await else {
return;
};
let now = Utc::now();
seed_observability_facts(&database.pool, now)
.await
.expect("seed observability facts");
sqlx::query(
"update model_requests
set started_at = $1 - interval '2 days'
where id = 'req_observe_success'",
)
.bind(now)
.execute(&database.pool)
.await
.expect("move historical request outside selected range");
let range = ObservabilityRange::new(now - TimeDelta::hours(1), now + TimeDelta::hours(1))
.expect("observability range");
let repository = PgObservabilityRepository::new(database.pool.clone(), None);
let dashboard = repository
.dashboard_summary(range)
.await
.expect("dashboard summary");
assert_eq!(dashboard.requests.request_count, 2);
assert_eq!(
(
dashboard.totals.request_count,
dashboard.totals.input_tokens,
dashboard.totals.cached_tokens,
dashboard.totals.total_tokens,
),
(3, 100, 40, 120),
);
assert_eq!(
dashboard
.totals
.billing_usd
.as_ref()
.map(|amount| amount.as_str()),
Some("1.25"),
);
database.close().await;
}
#[tokio::test]
async fn observability_queries_preserve_request_account_cost_and_diagnostic_facts() {
let Some(database) = TestDatabase::create("observability").await else {
@@ -524,7 +763,7 @@ async fn observability_queries_preserve_request_account_cost_and_diagnostic_fact
.expect("seed observability facts");
let range = ObservabilityRange::new(now - TimeDelta::hours(1), now + TimeDelta::hours(1))
.expect("observability range");
let repository = PgObservabilityRepository::new(database.pool.clone());
let repository = PgObservabilityRepository::new(database.pool.clone(), None);
let dashboard = repository
.dashboard_summary(range)
@@ -49,6 +49,35 @@ fn postgres_provider_account_adapter_implements_core_port() {
assert_terminal_admin_port::<PgAdminAccountStore>();
}
#[tokio::test]
async fn provider_account_should_allow_missing_upstream_user_id() {
let Some(database) = TestDatabase::create("provider_account_missing_upstream_identity").await
else {
return;
};
let repository = PgProviderAccountRepository::new(database.pool.clone());
let mut pending = account("acct_pending_identity", "unused");
pending.upstream_user_id = None;
pending.email = None;
pending.availability = AccountAvailability::Ready;
repository
.insert_provider_account(pending)
.await
.expect("insert account without upstream identity");
let stored = repository
.load_provider_account("acct_pending_identity")
.await
.expect("load account")
.expect("stored account");
assert_eq!(
(stored.summary.upstream_user_id, stored.summary.availability,),
(None, AccountAvailability::Unknown)
);
database.close().await;
}
#[tokio::test]
async fn core_quota_batch_reads_only_observed_accounts_in_one_contract_call() {
let Some(database) = TestDatabase::create("provider_account_quota_batch").await else {
@@ -160,6 +189,7 @@ async fn delayed_provider_observations_cannot_overwrite_newer_state_or_quota() {
repository
.apply_state_change(AccountStateChange {
message: None,
account_id: account_id.clone(),
expected_revision: revision,
availability: AccountAvailability::QuotaExhausted,
@@ -170,6 +200,7 @@ async fn delayed_provider_observations_cannot_overwrite_newer_state_or_quota() {
assert!(
repository
.apply_state_change(AccountStateChange {
message: None,
account_id: account_id.clone(),
expected_revision: revision,
availability: AccountAvailability::Ready,
@@ -200,6 +231,61 @@ async fn delayed_provider_observations_cannot_overwrite_newer_state_or_quota() {
database.close().await;
}
#[tokio::test]
async fn account_state_message_is_persisted_and_cleared_on_recovery() {
let Some(database) = TestDatabase::create("provider_account_state_message").await else {
return;
};
let repository = PgProviderAccountRepository::new(database.pool.clone());
let account_id = ProviderAccountId::new("acct_state_message").expect("account ID");
repository
.insert_provider_account(account(account_id.as_str(), "user-state-message"))
.await
.expect("insert state message fixture");
let revision = CredentialRevision::new(1).expect("revision");
repository
.apply_state_change(AccountStateChange {
message: Some("upstream returned 402 payment required".to_owned()),
account_id: account_id.clone(),
expected_revision: revision,
availability: AccountAvailability::QuotaExhausted,
observed_at: SystemTime::now(),
})
.await
.expect("persist error message");
let stored: Option<String> =
sqlx::query_scalar("select last_error_message from provider_accounts where id = $1")
.bind(account_id.as_str())
.fetch_one(&database.pool)
.await
.expect("read error message");
assert_eq!(
stored.as_deref(),
Some("upstream returned 402 payment required")
);
repository
.apply_state_change(AccountStateChange {
message: None,
account_id: account_id.clone(),
expected_revision: revision,
availability: AccountAvailability::Ready,
observed_at: SystemTime::now(),
})
.await
.expect("recover account");
let cleared: Option<String> =
sqlx::query_scalar("select last_error_message from provider_accounts where id = $1")
.bind(account_id.as_str())
.fetch_one(&database.pool)
.await
.expect("read cleared error message");
assert_eq!(cleared, None);
database.close().await;
}
#[tokio::test]
async fn terminal_admin_list_filters_and_sorts_before_pagination_with_retained_usage() {
let Some(database) = TestDatabase::create("provider_account_terminal_list").await else {
@@ -231,6 +317,21 @@ async fn terminal_admin_list_filters_and_sorts_before_pagination_with_retained_u
.await
.expect("insert account list fixture");
}
repository
.compare_and_swap_quota(QuotaObservation {
account_id: ProviderAccountId::new("acct_charlie").expect("account id"),
expected_revision: CredentialRevision::new(1).expect("credential revision"),
quota: Some(OpaqueProviderData::new(
json!({"rate_limit": {"limit_reached": true}})
.as_object()
.expect("quota object")
.clone(),
)),
observed_at: Some(SystemTime::now()),
limit_reached: Some(true),
})
.await
.expect("seed rate-limited account");
seed_model_request(
&database.pool,
@@ -289,7 +390,7 @@ async fn terminal_admin_list_filters_and_sorts_before_pagination_with_retained_u
.await
.expect("seed charlie usage");
let store = PgAdminAccountStore::new(database.pool.clone());
let store = PgAdminAccountStore::new(database.pool.clone(), None);
let usage_page = store
.list_accounts(AccountListQuery {
page: 1,
@@ -307,14 +408,18 @@ async fn terminal_admin_list_filters_and_sorts_before_pagination_with_retained_u
assert_eq!(usage_page.config_revision.get(), 1);
assert_eq!(usage_page.total, 6);
assert_eq!(usage_page.summary.total, 6);
assert_eq!(usage_page.summary.active, 2);
assert_eq!(usage_page.summary.quota_exhausted, 1);
assert_eq!(usage_page.summary.unavailable, 3);
assert_eq!(usage_page.summary.normal, 1);
assert_eq!(usage_page.summary.quota_exhausted, 2);
assert_eq!(usage_page.summary.rate_limited, 0);
assert_eq!(usage_page.summary.disabled, 1);
assert_eq!(usage_page.summary.error, 2);
assert_eq!(
usage_page.summary.total,
usage_page.summary.active
usage_page.summary.normal
+ usage_page.summary.quota_exhausted
+ usage_page.summary.unavailable
+ usage_page.summary.rate_limited
+ usage_page.summary.disabled
+ usage_page.summary.error
);
assert_eq!(
usage_page
@@ -354,7 +459,7 @@ async fn terminal_admin_list_filters_and_sorts_before_pagination_with_retained_u
page_size: PageSize::new(10).expect("page size"),
provider_kind: Some(ProviderKind::new("openai").expect("Provider kind")),
search: Some("ALPHA@EXAMPLE".to_owned()),
status: Some(AccountStatus::Active),
status: Some(AccountStatus::Normal),
sort: None,
})
.await
@@ -364,18 +469,20 @@ async fn terminal_admin_list_filters_and_sorts_before_pagination_with_retained_u
assert_eq!(filtered.items[0].id, "acct_alpha");
assert_eq!(filtered.items[0].provider_kind.as_str(), "openai");
let banned = store
let error_accounts = store
.list_accounts(AccountListQuery {
page: 1,
page_size: PageSize::new(10).expect("page size"),
provider_kind: None,
search: None,
status: Some(AccountStatus::Banned),
status: Some(AccountStatus::Error),
sort: None,
})
.await
.expect("filter banned accounts");
assert_eq!(banned.items[0].id, "acct_beta");
.expect("filter error accounts");
assert_eq!(error_accounts.items.len(), 2);
assert_eq!(error_accounts.items[0].id, "acct_beta");
assert_eq!(error_accounts.items[1].id, "acct_invalid");
database.close().await;
}
@@ -403,7 +510,7 @@ async fn terminal_credential_list_preserves_grouped_filters_and_unpaged_collecti
.expect("insert xAI credential fixture");
}
let store = PgAdminAccountStore::new(database.pool.clone());
let store = PgAdminAccountStore::new(database.pool.clone(), None);
let provider = ProviderKind::new("xai").expect("xAI Provider kind");
let complete = store
.list_credentials(
@@ -484,7 +591,7 @@ async fn terminal_admin_usage_chunks_large_selections_and_preserves_exact_costs(
let mut account_ids = vec!["acct_usage_exact".to_owned()];
account_ids.extend((0..200).map(|index| format!("missing_account_{index}")));
let usage = PgAdminAccountStore::new(database.pool.clone())
let usage = PgAdminAccountStore::new(database.pool.clone(), None)
.load_account_usage(
TimeRange {
start: now - TimeDelta::hours(1),
@@ -576,7 +683,7 @@ async fn terminal_admin_quota_window_usage_prefers_provider_total_and_falls_back
.await
.expect("separate provider total from fallback components");
let mut usage = PgAdminAccountStore::new(database.pool.clone())
let mut usage = PgAdminAccountStore::new(database.pool.clone(), None)
.load_account_usage_by_windows(&[
AccountUsageWindowQuery {
account_id: "acct_quota_window".to_owned(),
@@ -616,7 +723,7 @@ async fn terminal_admin_mutations_keep_revision_account_and_audit_atomic() {
.insert_provider_account(account("acct_terminal_mutation", "user-terminal-mutation"))
.await
.expect("insert mutation account");
let store = PgAdminAccountStore::new(database.pool.clone());
let store = PgAdminAccountStore::new(database.pool.clone(), None);
let context = MutationContext {
actor: MutationActor::System,
request_id: "request_terminal_mutation".to_owned(),
@@ -689,7 +796,7 @@ async fn terminal_admin_delete_removes_enabled_accounts_in_one_transaction() {
.expect("insert enabled account");
}
let revision = PgAdminAccountStore::new(database.pool.clone())
let revision = PgAdminAccountStore::new(database.pool.clone(), None)
.delete_accounts(
DeleteAccounts {
account_ids: vec![
@@ -804,7 +911,7 @@ async fn core_refresh_cas_updates_profile_and_credential_under_one_revision() {
provider_kind: "xai".to_owned(),
name: "before refresh".to_owned(),
email: Some("before@example.invalid".to_owned()),
upstream_user_id: "upstream-core-refresh".to_owned(),
upstream_user_id: Some("upstream-core-refresh".to_owned()),
upstream_account_id: None,
plan_type: Some("free".to_owned()),
authentication_kind: "oauth".to_owned(),
@@ -1193,6 +1300,7 @@ async fn xai_free_quota_exhaustion_state_is_persisted() {
repository
.apply_state_change(AccountStateChange {
message: None,
account_id: account_id.clone(),
expected_revision: CredentialRevision::new(1).expect("credential revision"),
availability: AccountAvailability::QuotaExhausted,
@@ -1231,6 +1339,7 @@ async fn xai_resettable_usage_limit_state_is_persisted() {
let observed_at = SystemTime::now();
repository
.apply_state_change(AccountStateChange {
message: None,
account_id: account_id.clone(),
expected_revision: CredentialRevision::new(1).expect("credential revision"),
availability: AccountAvailability::Ready,
@@ -1280,6 +1389,7 @@ async fn disabled_account_preserves_user_state_during_refresh_writes() {
repository
.apply_state_change(AccountStateChange {
message: None,
account_id: account_id.clone(),
expected_revision: CredentialRevision::new(1).expect("credential revision"),
availability: AccountAvailability::Ready,
@@ -1324,7 +1434,7 @@ fn account(id: &str, upstream_user_id: &str) -> NewProviderAccount {
provider_kind: "openai".to_owned(),
name: id.to_owned(),
email: Some(format!("{id}@example.invalid")),
upstream_user_id: upstream_user_id.to_owned(),
upstream_user_id: Some(upstream_user_id.to_owned()),
upstream_account_id: None,
plan_type: Some("pro".to_owned()),
authentication_kind: "oauth".to_owned(),
@@ -42,7 +42,7 @@ fn runtime_settings_reject_invalid_model_mapping() {
}
#[tokio::test]
async fn refresh_margin_change_should_reschedule_existing_refreshable_accounts() {
async fn refresh_margin_change_should_preserve_existing_account_refresh_facts() {
let Some(database) = TestDatabase::create("refresh_margin_reschedule").await else {
return;
};
@@ -55,6 +55,7 @@ async fn refresh_margin_change_should_reschedule_existing_refreshable_accounts()
)
.await;
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
let before = account_refresh_facts(&database.pool, "acct_refresh_margin_changed").await;
repository
.update_runtime_settings(settings_with_margin(1_800))
@@ -62,14 +63,14 @@ async fn refresh_margin_change_should_reschedule_existing_refreshable_accounts()
.expect("update refresh margin");
assert_eq!(
account_next_refresh_at(&database.pool, "acct_refresh_margin_changed").await,
expires_at - TimeDelta::seconds(1_800)
account_refresh_facts(&database.pool, "acct_refresh_margin_changed").await,
before
);
database.close().await;
}
#[tokio::test]
async fn unchanged_refresh_margin_should_preserve_existing_retry_schedule() {
async fn unchanged_refresh_margin_should_preserve_existing_account_refresh_facts() {
let Some(database) = TestDatabase::create("refresh_margin_unchanged").await else {
return;
};
@@ -83,6 +84,7 @@ async fn unchanged_refresh_margin_should_preserve_existing_retry_schedule() {
)
.await;
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
let before = account_refresh_facts(&database.pool, "acct_refresh_margin_unchanged").await;
repository
.update_runtime_settings(settings_with_margin(3_600))
@@ -90,8 +92,8 @@ async fn unchanged_refresh_margin_should_preserve_existing_retry_schedule() {
.expect("update unrelated runtime settings");
assert_eq!(
account_next_refresh_at(&database.pool, "acct_refresh_margin_unchanged").await,
retry_at
account_refresh_facts(&database.pool, "acct_refresh_margin_unchanged").await,
before
);
database.close().await;
}
@@ -118,12 +120,31 @@ async fn insert_refreshable_account(
.expect("insert refreshable account");
}
async fn account_next_refresh_at(pool: &sqlx::PgPool, account_id: &str) -> DateTime<Utc> {
sqlx::query_scalar("select next_refresh_at from provider_accounts where id = $1")
#[derive(Debug, PartialEq, Eq)]
struct AccountRefreshFacts {
access_token_expires_at: DateTime<Utc>,
next_refresh_at: DateTime<Utc>,
updated_at: DateTime<Utc>,
credential_revision: i64,
}
async fn account_refresh_facts(pool: &sqlx::PgPool, account_id: &str) -> AccountRefreshFacts {
let (access_token_expires_at, next_refresh_at, updated_at, credential_revision) =
sqlx::query_as(
"select access_token_expires_at, next_refresh_at, updated_at, credential_revision
from provider_accounts
where id = $1",
)
.bind(account_id)
.fetch_one(pool)
.await
.expect("load next refresh time")
.expect("load account refresh facts");
AccountRefreshFacts {
access_token_expires_at,
next_refresh_at,
updated_at,
credential_revision,
}
}
fn timestamp_micros(value: DateTime<Utc>) -> DateTime<Utc> {
@@ -39,7 +39,7 @@ async fn request_snapshots_should_survive_account_deletion() {
.await
.expect("delete provider account");
let repository = PgObservabilityRepository::new(database.pool.clone());
let repository = PgObservabilityRepository::new(database.pool.clone(), None);
let page = repository
.list_usage_records(usage_query(started_at, UsageRecordFilter::default()))
.await
@@ -154,7 +154,7 @@ async fn attempts_should_keep_their_own_account_snapshots() {
.await
.expect("delete provider accounts");
let repository = PgObservabilityRepository::new(database.pool.clone());
let repository = PgObservabilityRepository::new(database.pool.clone(), None);
let detail = repository
.usage_record_detail("req_snap_ab")
.await
@@ -276,7 +276,7 @@ async fn ops_errors_should_keep_request_and_event_snapshots_after_account_deleti
.await
.expect("delete provider accounts");
let repository = PgObservabilityRepository::new(database.pool.clone());
let repository = PgObservabilityRepository::new(database.pool.clone(), None);
let errors = repository
.list_ops_errors(OpsErrorQuery {
range: range_around(started_at),
@@ -376,7 +376,7 @@ async fn diagnostics_should_group_same_email_accounts_by_stable_ref() {
.await
.expect("delete provider accounts");
let repository = PgObservabilityRepository::new(database.pool.clone());
let repository = PgObservabilityRepository::new(database.pool.clone(), None);
let diagnostics = repository
.usage_diagnostics(
range_around(started_at),
@@ -445,7 +445,7 @@ async fn diagnostics_should_fallback_to_name_then_ref_for_missing_snapshots() {
.expect("insert legacy request without live account");
finalize_request(&database.pool, "req_snap_legacy", started_at).await;
let repository = PgObservabilityRepository::new(database.pool.clone());
let repository = PgObservabilityRepository::new(database.pool.clone(), None);
let diagnostics = repository
.usage_diagnostics(
range_around(started_at),
@@ -511,7 +511,7 @@ async fn failure_diagnostics_should_only_include_errored_requests() {
.await
.expect("mark request failed");
let repository = PgObservabilityRepository::new(database.pool.clone());
let repository = PgObservabilityRepository::new(database.pool.clone(), None);
let diagnostics = repository
.usage_diagnostics(
range_around(started_at),
@@ -566,7 +566,7 @@ async fn api_key_diagnostics_should_display_key_name_and_fallback_to_ref() {
.await
.expect("set request b latency");
let repository = PgObservabilityRepository::new(database.pool.clone());
let repository = PgObservabilityRepository::new(database.pool.clone(), None);
let diagnostics = repository
.usage_diagnostics(
range_around(started_at),
@@ -633,7 +633,7 @@ async fn renaming_account_should_not_rewrite_historical_snapshots() {
.await
.expect("rename provider account");
let repository = PgObservabilityRepository::new(database.pool.clone());
let repository = PgObservabilityRepository::new(database.pool.clone(), None);
let page = repository
.list_usage_records(usage_query(started_at, UsageRecordFilter::default()))
.await
+141
View File
@@ -0,0 +1,141 @@
use std::io;
use chrono::Utc;
use gateway_admin::model::backup::{BackupObjectMetadata, BackupStorageConfig};
use gateway_admin::ports::backup::{BackupObjectStorePort, UploadObjectRequest};
use gateway_core::engine::CancellationToken;
use gateway_store::backup::s3::S3ObjectStoreAdapter;
use secrecy::SecretString;
use tokio::io::{AsyncReadExt as _, AsyncWriteExt as _};
use tokio::net::{TcpListener, TcpStream};
const PART_SIZE_ABOVE_TOKIO_DEFAULT: usize = 2 * 1024 * 1024 + 1;
#[tokio::test]
async fn upload_file_sends_full_part_beyond_tokio_default_buffer_limit() {
let directory = tempfile::tempdir().expect("create temporary directory");
let source = directory.path().join("archive.dump");
let expected = vec![b'x'; PART_SIZE_ABOVE_TOKIO_DEFAULT];
tokio::fs::write(&source, &expected)
.await
.expect("write temporary archive");
let listener = TcpListener::bind("127.0.0.1:0")
.await
.expect("bind multipart probe server");
let endpoint = format!(
"http://{}",
listener.local_addr().expect("read listener address")
);
let server = tokio::spawn(serve_multipart_probe(listener));
let storage = BackupStorageConfig {
storage_revision: 1,
endpoint,
region: "auto".to_owned(),
bucket: "backup-bucket".to_owned(),
access_key_id: "test-access-key".to_owned(),
secret_access_key: SecretString::from("test-secret-key"),
prefix: "backups".to_owned(),
force_path_style: true,
};
let request = UploadObjectRequest {
object_key: "backups/archive.dump".to_owned(),
source,
metadata: BackupObjectMetadata::new(
"backup_test".to_owned(),
"a".repeat(64),
Utc::now(),
PART_SIZE_ABOVE_TOKIO_DEFAULT as u64,
),
cancellation: CancellationToken::new(),
};
S3ObjectStoreAdapter::new()
.upload_file(&storage, request)
.await
.expect("upload multipart archive");
let uploaded_len = server
.await
.expect("multipart probe task did not panic")
.expect("serve multipart probe");
assert_eq!(uploaded_len, PART_SIZE_ABOVE_TOKIO_DEFAULT);
}
async fn serve_multipart_probe(listener: TcpListener) -> io::Result<usize> {
let mut uploaded_len = None;
for _ in 0..3 {
let (mut stream, _) = listener.accept().await?;
let (request, body) = read_http_request(&mut stream).await?;
let response = if request.starts_with("POST ") && request.contains("uploads") {
xml_response(
"<InitiateMultipartUploadResult xmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\"><UploadId>probe-upload</UploadId></InitiateMultipartUploadResult>",
)
} else if request.starts_with("PUT ") && request.contains("partNumber=") {
uploaded_len = Some(body.len());
"HTTP/1.1 200 OK\r\ncontent-length: 0\r\nconnection: close\r\netag: \"probe-part\"\r\n\r\n".to_owned()
} else if request.starts_with("POST ") && request.contains("uploadId=") {
xml_response(
"<CompleteMultipartUploadResult xmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\"><ETag>\"probe-part\"</ETag></CompleteMultipartUploadResult>",
)
} else {
return Err(io::Error::other("unexpected multipart probe request"));
};
stream.write_all(response.as_bytes()).await?;
stream.shutdown().await?;
}
uploaded_len.ok_or_else(|| io::Error::other("multipart probe did not receive an upload part"))
}
async fn read_http_request(stream: &mut TcpStream) -> io::Result<(String, Vec<u8>)> {
let mut bytes = Vec::new();
let header_end = loop {
if let Some(position) = bytes.windows(4).position(|window| window == b"\r\n\r\n") {
break position + 4;
}
let mut chunk = [0_u8; 8 * 1024];
let read = stream.read(&mut chunk).await?;
if read == 0 {
return Err(io::Error::new(
io::ErrorKind::UnexpectedEof,
"multipart probe request ended before headers",
));
}
bytes.extend_from_slice(&chunk[..read]);
};
let headers = String::from_utf8_lossy(&bytes[..header_end]).into_owned();
let content_len = headers
.lines()
.filter_map(|line| line.split_once(':'))
.find(|(name, _)| name.eq_ignore_ascii_case("content-length"))
.and_then(|(_, value)| value.trim().parse::<usize>().ok())
.unwrap_or_default();
if headers
.to_ascii_lowercase()
.contains("expect: 100-continue")
{
stream.write_all(b"HTTP/1.1 100 Continue\r\n\r\n").await?;
}
while bytes.len() < header_end + content_len {
let mut chunk = [0_u8; 8 * 1024];
let read = stream.read(&mut chunk).await?;
if read == 0 {
return Err(io::Error::new(
io::ErrorKind::UnexpectedEof,
"multipart probe request ended before body",
));
}
bytes.extend_from_slice(&chunk[..read]);
}
Ok((
headers,
bytes[header_end..header_end + content_len].to_vec(),
))
}
fn xml_response(body: &str) -> String {
format!(
"HTTP/1.1 200 OK\r\ncontent-type: application/xml\r\ncontent-length: {}\r\nconnection: close\r\n\r\n{body}",
body.len()
)
}
+1 -1
View File
@@ -27,7 +27,6 @@ hex = "0.4.3"
# reqwest 0.12.28 + ring)。rustls 0.23.37 起 ClientHello 扩展顺序变化,JA3
# 会偏离官方指纹;升级前必须重新抓握手对比扩展顺序。
hyper-rustls = "=0.27.7"
jsonwebtoken = { version = "=11.0.0", default-features = false, features = ["aws_lc_rs", "use_pem"] }
# 与官方 Codex Desktop 一致:reqwest 不启用压缩特性,上游请求不带 accept-encoding。
reqwest = { version = "=0.12.28", default-features = false, features = ["json", "stream", "cookies", "rustls-tls-native-roots", "http2"] }
roxmltree = "0.21.1"
@@ -58,6 +57,7 @@ zstd = "0.13"
[dev-dependencies]
tempfile = "3.27.0"
tokio = { version = "1.52.3", default-features = false, features = ["rt-multi-thread", "test-util"] }
tracing-subscriber = { version = "0.3.23", features = ["json"] }
wiremock = "0.6.5"
[[test]]
+84 -112
View File
@@ -19,7 +19,7 @@ use gateway_admin::model::provider_credentials::{
PreparedCredentialCreate, PreparedCredentialImport, PreparedCredentialRotation,
PreparedCredentialRotationFacts, ProviderDocument, ProviderExport,
ProviderExportCredentialInput, ProviderModel, ProviderModels, ProviderQuota,
ProviderQuotaRequest, ProviderQuotaWindow,
ProviderQuotaRequest, ProviderQuotaWindow, QuotaLocalUsageAttribution,
};
use gateway_admin::model::{MutationActor, MutationContext, Revision};
use gateway_admin::ports::provider::{ProviderAdmin, ProviderAdminError, ProviderAdminErrorKind};
@@ -33,28 +33,24 @@ use gateway_core::error::StoreErrorKind;
use gateway_core::operation::{GenerateRequest, Operation, ProtocolPayload};
use gateway_core::provider_ports::{
NewOAuthPendingFlow, OAuthPendingBinding, OAuthPendingClaimOutcome, OAuthPendingConsumeOutcome,
OAuthPendingFlowPort, OAuthPendingPutOutcome, OAuthPendingReleaseOutcome,
ProviderRuntimePolicyPort, ProviderStoreError, ProviderStoreErrorKind,
OAuthPendingFlowPort, OAuthPendingPutOutcome, OAuthPendingReleaseOutcome, ProviderStoreError,
ProviderStoreErrorKind,
};
use gateway_core::routing::{ProviderKind, UpstreamModelId};
use secrecy::{ExposeSecret as _, SecretString};
use serde::Deserialize;
use serde_json::{Map, Number, Value};
use crate::credential::{
CodexAccountIdentityVerifier, CodexCredentialCodec, CodexIdentityExpectation, CodexOAuthSecret,
oauth_owner_ref,
};
use crate::credential::{
CodexAccountQuotaSnapshot, CodexCredentialAdmin, CodexCredentialAdminError,
CodexCredentialAdminService, CodexCredentialCatalogError, CodexCredentialCatalogService,
CodexCredentialQuotaError, CodexCredentialQuotaService, CodexOAuthAdmin, CodexOAuthAdminError,
CodexOAuthPendingClaimOutcome, CodexOAuthPendingStore, CodexOAuthPendingStoreError,
CodexPendingAuthorization, CodexQuotaWindowKind, CodexQuotaWindowRole,
CodexPendingAuthorization, CodexQuotaWindow, CodexQuotaWindowKind, CodexQuotaWindowRole,
CompleteCodexOAuthAuthorization, CompletedCodexOAuthCredential, ExportManagedCodexCredential,
RotateManagedCodexCredential, StartCodexOAuthAuthorization, StoredCodexPendingAuthorization,
refresh_time,
StartCodexOAuthAuthorization, StoredCodexPendingAuthorization,
};
use crate::credential::{CodexCredentialCodec, CodexOAuthSecret, oauth_owner_ref};
use crate::transport::CodexWebSocketPool;
use crate::transport::openai_billing_breakdown;
use crate::transport::profile::{
@@ -63,8 +59,6 @@ use crate::transport::profile::{
const PROVIDER_NAME: &str = "openai";
const PENDING_DOCUMENT_SCHEMA_VERSION: u64 = 3;
const MAX_ACCESS_TOKEN_BYTES: usize = 16 * 1024;
const MAX_REFRESH_TOKEN_BYTES: usize = 64 * 1024;
/// OpenAI 对终态 Admin port 的唯一实现。
pub(crate) struct OpenAiAdminProvider {
@@ -72,22 +66,18 @@ pub(crate) struct OpenAiAdminProvider {
profile: CodexWireProfileState,
accounts: Arc<dyn ProviderAccountStore>,
credentials: Arc<CodexCredentialAdminService>,
verifier: Arc<dyn CodexAccountIdentityVerifier>,
oauth: Arc<dyn CodexOAuthAdmin>,
quota: Arc<CodexCredentialQuotaService>,
catalog: Arc<CodexCredentialCatalogService>,
runtime_policy: Arc<dyn ProviderRuntimePolicyPort>,
websocket_pool: Arc<CodexWebSocketPool>,
desktop_release: CodexDesktopReleaseStatus,
}
pub(crate) struct OpenAiAdminServices {
pub(crate) credentials: Arc<CodexCredentialAdminService>,
pub(crate) verifier: Arc<dyn CodexAccountIdentityVerifier>,
pub(crate) oauth: Arc<dyn CodexOAuthAdmin>,
pub(crate) quota: Arc<CodexCredentialQuotaService>,
pub(crate) catalog: Arc<CodexCredentialCatalogService>,
pub(crate) runtime_policy: Arc<dyn ProviderRuntimePolicyPort>,
}
impl OpenAiAdminProvider {
@@ -105,11 +95,9 @@ impl OpenAiAdminProvider {
profile,
accounts,
credentials: services.credentials,
verifier: services.verifier,
oauth: services.oauth,
quota: services.quota,
catalog: services.catalog,
runtime_policy: services.runtime_policy,
websocket_pool,
desktop_release,
}
@@ -194,12 +182,13 @@ impl ProviderAdmin for OpenAiAdminProvider {
let profile = self.profile.snapshot();
let release = self.desktop_release.snapshot();
let user_agent = profile.user_agent();
let client_identity = format!("{}; {}", profile.originator, profile.desktop_version);
let release = dashboard_desktop_release(&profile, release);
Some(DashboardWireProfile {
provider: self.provider_kind.as_str().to_owned(),
product: profile.originator,
version: profile.desktop_version,
build: Some(profile.desktop_build),
version: profile.codex_version,
build: None,
target: DashboardWireTarget {
os_type: profile.os_type,
os_version: profile.os_version,
@@ -208,8 +197,8 @@ impl ProviderAdmin for OpenAiAdminProvider {
},
user_agent,
attributes: vec![DashboardWireAttribute {
label: "Codex Core".to_owned(),
value: profile.codex_version,
label: "客户端标识".to_owned(),
value: client_identity,
}],
verified_at: Some(profile.verified_at),
release: Some(release),
@@ -380,49 +369,9 @@ impl ProviderAdmin for OpenAiAdminProvider {
if !account_matches_record(&current.account, &command.account) {
return Err(provider_admin_error(ProviderAdminErrorKind::Conflict));
}
let runtime = CodexCredentialCodec::decode(&current.credential)
.map_err(|_| provider_admin_error(ProviderAdminErrorKind::Invalid))?;
let account_id = current
.account
.upstream_account_id()
.ok_or_else(|| provider_admin_error(ProviderAdminErrorKind::Invalid))?;
let principal = runtime
.principal
.ok_or_else(|| provider_admin_error(ProviderAdminErrorKind::Invalid))?;
let expectation = CodexIdentityExpectation::current(
principal.oauth_subject,
principal.poid,
account_id.to_owned(),
current.account.upstream_user_id().to_owned(),
runtime.installation_id,
)
.map_err(|_| provider_admin_error(ProviderAdminErrorKind::Invalid))?;
let secret = rotation_secret(command.provider_material)?;
let verified_account = self
.verifier
.verify(&secret, &expectation)
.await
.and_then(crate::credential::CodexIdentityVerification::into_complete)
.map_err(|_| provider_admin_error(ProviderAdminErrorKind::Invalid))?;
let policy = self
.runtime_policy
.load_refresh_policy()
.await
.map_err(map_provider_store_error)?;
let next_refresh_at = refresh_time(
policy,
current.account.id(),
verified_account.access_token_expires_at,
secret.refresh_token.is_some(),
)
.map_err(map_credential_admin_error)?;
let prepared = CodexCredentialAdmin
.prepare_rotation(RotateManagedCodexCredential {
current,
secret,
verified_account,
next_refresh_at,
})
.prepare_refreshed_oauth_rotation(current, secret, None, None)
.map_err(map_credential_admin_error)?;
prepared_rotation(prepared, command.account.provider_kind)
}
@@ -475,7 +424,14 @@ impl ProviderAdmin for OpenAiAdminProvider {
.await
.map_err(map_quota_error)?
};
Ok(project_quota(snapshot, &account))
let mut quota = project_quota(snapshot, &account);
quota.rate_limited_until = self
.quota
.rate_limited_until(account.id(), account.revision())
.await
.map_err(map_quota_error)?
.map(DateTime::<Utc>::from);
Ok(quota)
}
async fn models(
@@ -569,7 +525,7 @@ fn prepared_create(
provider_kind: account.provider().clone(),
name: account.name().to_owned(),
email: account.email().map(str::to_owned),
upstream_user_id: account.upstream_user_id().to_owned(),
upstream_user_id: account.upstream_user_id().map(str::to_owned),
upstream_account_id: account.upstream_account_id().map(str::to_owned),
plan_type: account.plan_type().map(str::to_owned),
authentication_kind: account.authentication_kind().to_owned(),
@@ -633,15 +589,6 @@ fn rotation_secret(document: ProviderDocument) -> Result<CodexOAuthSecret, Provi
let document: RotationDocument =
serde_json::from_value(Value::Object(document.into_provider_data().into_inner()))
.map_err(|_| provider_admin_error(ProviderAdminErrorKind::Invalid))?;
if document.access_token.is_empty()
|| document.access_token.len() > MAX_ACCESS_TOKEN_BYTES
|| document
.refresh_token
.as_deref()
.is_some_and(|token| token.is_empty() || token.len() > MAX_REFRESH_TOKEN_BYTES)
{
return Err(provider_admin_error(ProviderAdminErrorKind::Invalid));
}
Ok(CodexOAuthSecret {
access_token: SecretString::from(document.access_token),
refresh_token: document.refresh_token.map(SecretString::from),
@@ -729,7 +676,7 @@ fn dashboard_desktop_release(
fn account_matches_record(account: &ProviderAccount, record: &AccountRecord) -> bool {
account.id().as_str() == record.id
&& account.provider() == &record.provider_kind
&& account.upstream_user_id() == record.upstream_user_id
&& account.upstream_user_id() == record.upstream_user_id.as_deref()
&& account.upstream_account_id() == record.upstream_account_id.as_deref()
&& account.authentication_kind() == record.authentication_kind
}
@@ -740,6 +687,7 @@ fn empty_quota() -> ProviderQuota {
refresh_token_expires_at: None,
windows: Vec::new(),
limit_reached: false,
rate_limited_until: None,
provider_data: None,
}
}
@@ -774,6 +722,7 @@ fn project_quota_snapshot(snapshot: CodexAccountQuotaSnapshot) -> ProviderQuota
let windows: Vec<ProviderQuotaWindow> = snapshot
.windows()
.iter()
.filter(|window| should_project_quota_window(window))
.map(|window| {
let mut data = Map::new();
data.insert(
@@ -790,10 +739,16 @@ fn project_quota_snapshot(snapshot: CodexAccountQuotaSnapshot) -> ProviderQuota
label: codex_quota_window_label(
window.limit_name(),
window.kind(),
window.role(),
window.source(),
window.window_seconds(),
),
source: Some(window.source().to_owned()),
local_usage_attribution: if window.is_account_wide() {
QuotaLocalUsageAttribution::AccountWide
} else {
QuotaLocalUsageAttribution::Unavailable
},
window_seconds: window.window_seconds(),
used_percent: window.used_percent(),
reset_at: window.reset_at(),
@@ -811,12 +766,24 @@ fn project_quota_snapshot(snapshot: CodexAccountQuotaSnapshot) -> ProviderQuota
refresh_token_expires_at: None,
windows,
limit_reached,
rate_limited_until: None,
provider_data: Some(ProviderDocument::new(OpaqueProviderData::new(
provider_data,
))),
}
}
/// `secondary_window` 有时只是上游的空占位:没有时长、重置时间,也没有用量。
/// 它不能被可靠翻译成 5 小时或周额度,因此不应占用账号面板;带有实际事实的
/// 次级窗口(时长、重置、非零用量或触顶)仍完整保留。
fn should_project_quota_window(window: &CodexQuotaWindow) -> bool {
window.role() != CodexQuotaWindowRole::Secondary
|| window.window_seconds().is_some()
|| window.reset_at().is_some()
|| window.used_percent().is_some_and(|used| used > 0.0)
|| window.limit_reached()
}
fn quota_windows_limit_reached(windows: &[ProviderQuotaWindow]) -> bool {
windows.iter().any(|window| window.limit_reached)
}
@@ -840,6 +807,7 @@ fn force_confirmed_exhaustion_projection(quota: &mut ProviderQuota) {
group: "other".to_owned(),
label: "额度".to_owned(),
source: None,
local_usage_attribution: QuotaLocalUsageAttribution::Unavailable,
window_seconds: None,
used_percent: Some(100.0),
reset_at: None,
@@ -882,45 +850,66 @@ const fn quota_role(role: CodexQuotaWindowRole) -> &'static str {
}
}
/// 窗口展示名,优先使用上游 `limit_name`(原文,`_` 转空格);
/// 无 limit_name 时按窗口时长猜(±5% 容差,汉化输出),猜不中兜底「额度」。
/// 按窗口时长显示汉化额度名;非核心桶补上上游桶名称作为前缀。
fn codex_quota_window_label(
limit_name: Option<&str>,
kind: CodexQuotaWindowKind,
role: CodexQuotaWindowRole,
source: &str,
window_seconds: Option<u64>,
) -> String {
if let Some(limit_name) = limit_name {
return limit_name.replace('_', " ");
}
let base = match kind {
CodexQuotaWindowKind::Monthly => "月限额".to_owned(),
CodexQuotaWindowKind::Weekly => "周限额".to_owned(),
CodexQuotaWindowKind::Monthly => "月额度".to_owned(),
CodexQuotaWindowKind::Weekly => "周额度".to_owned(),
CodexQuotaWindowKind::ShortTerm => {
if window_seconds.is_some_and(|seconds| seconds > 86_400) {
"周限额".to_owned()
"周额度".to_owned()
} else {
"5小时限额".to_owned()
"5小时额度".to_owned()
}
}
CodexQuotaWindowKind::Other => custom_quota_window_label(window_seconds),
CodexQuotaWindowKind::Other => custom_quota_window_label(window_seconds, role),
};
if matches!(source, "core" | "codex" | "code_review" | "spend_control") {
return base;
if let Some(prefix) = codex_quota_window_prefix(limit_name, source) {
return format!("{prefix} · {base}");
}
format!("{source} · {base}")
base
}
fn custom_quota_window_label(window_seconds: Option<u64>) -> String {
fn codex_quota_window_prefix(limit_name: Option<&str>, source: &str) -> Option<String> {
let source_identifier = quota_identifier(source);
if matches!(source_identifier.as_str(), "core" | "codex") {
return None;
}
let label = limit_name.unwrap_or(source);
match quota_identifier(label).as_str() {
"premium" => Some("Premium".to_owned()),
"code_review" => Some("代码审查".to_owned()),
"spend_control" => Some("消费控制".to_owned()),
_ => Some(label.replace('_', " ")),
}
}
fn quota_identifier(value: &str) -> String {
value.trim().to_ascii_lowercase().replace(['-', ' '], "_")
}
fn custom_quota_window_label(window_seconds: Option<u64>, role: CodexQuotaWindowRole) -> String {
let Some(seconds) = window_seconds.filter(|seconds| *seconds > 0) else {
return "额度".to_owned();
// 与官方客户端一致:没有时长时不臆测为 5 小时或周额度;保留
// primary/secondary 语义,让用户知道这是上游未标明时长的独立窗口。
return match role {
CodexQuotaWindowRole::Primary => "主额度".to_owned(),
CodexQuotaWindowRole::Secondary => "次级额度".to_owned(),
CodexQuotaWindowRole::Monthly => "月额度".to_owned(),
};
};
if seconds % 86_400 == 0 {
format!("{}日限额", seconds / 86_400)
format!("{}日额度", seconds / 86_400)
} else if seconds % 3_600 == 0 {
format!("{}小时限额", seconds / 3_600)
format!("{}小时额度", seconds / 3_600)
} else {
format!("{}分钟限额", seconds.div_ceil(60))
format!("{}分钟额度", seconds.div_ceil(60))
}
}
@@ -1309,14 +1298,6 @@ fn map_pending_store_error(error: ProviderStoreError) -> CodexOAuthPendingStoreE
}
}
fn map_provider_store_error(error: ProviderStoreError) -> ProviderAdminError {
provider_admin_error(match error.kind() {
ProviderStoreErrorKind::InvalidData => ProviderAdminErrorKind::Invalid,
ProviderStoreErrorKind::Conflict => ProviderAdminErrorKind::Conflict,
ProviderStoreErrorKind::Unavailable => ProviderAdminErrorKind::Unavailable,
})
}
fn map_store_error(error: gateway_core::error::StoreError) -> ProviderAdminError {
provider_admin_error(match error.kind() {
StoreErrorKind::Conflict => ProviderAdminErrorKind::Conflict,
@@ -1333,25 +1314,20 @@ fn map_credential_admin_error(error: CodexCredentialAdminError) -> ProviderAdmin
provider_admin_error(match error {
Error::InvalidInput
| Error::InvalidCredential
| Error::IdentityMismatch
| Error::MissingRefreshToken
| Error::RefreshRejected
| Error::AccountBanned
| Error::IdentityRejected => ProviderAdminErrorKind::Invalid,
| Error::AccountBanned => ProviderAdminErrorKind::Invalid,
Error::NotFound => ProviderAdminErrorKind::NotFound,
Error::RefreshLeaseUnavailable | Error::RefreshAmbiguous => {
ProviderAdminErrorKind::Conflict
}
Error::RefreshUnavailable | Error::IdentityUnavailable => {
ProviderAdminErrorKind::Unavailable
}
Error::RefreshUnavailable => ProviderAdminErrorKind::Unavailable,
})
}
const fn credential_admin_error_code(error: CodexCredentialAdminError) -> &'static str {
match error {
CodexCredentialAdminError::InvalidInput => "invalid_input",
CodexCredentialAdminError::IdentityMismatch => "identity_mismatch",
CodexCredentialAdminError::InvalidCredential => "invalid_credential",
CodexCredentialAdminError::NotFound => "not_found",
CodexCredentialAdminError::MissingRefreshToken => "missing_refresh_token",
@@ -1360,8 +1336,6 @@ const fn credential_admin_error_code(error: CodexCredentialAdminError) -> &'stat
CodexCredentialAdminError::AccountBanned => "account_banned",
CodexCredentialAdminError::RefreshUnavailable => "refresh_unavailable",
CodexCredentialAdminError::RefreshAmbiguous => "refresh_ambiguous",
CodexCredentialAdminError::IdentityRejected => "identity_rejected",
CodexCredentialAdminError::IdentityUnavailable => "identity_unavailable",
}
}
@@ -1390,7 +1364,6 @@ fn map_oauth_error(error: CodexOAuthAdminError) -> ProviderAdminError {
Error::InvalidInput
| Error::CallbackRejected
| Error::TokenRejected
| Error::IdentityRejected
| Error::Credential => ProviderAdminErrorKind::Invalid,
Error::NotFound | Error::FlowExpired => ProviderAdminErrorKind::NotFound,
Error::Conflict | Error::Ambiguous => ProviderAdminErrorKind::Conflict,
@@ -1408,7 +1381,6 @@ const fn oauth_error_code(error: &CodexOAuthAdminError) -> &'static str {
CodexOAuthAdminError::FlowExpired => "flow_expired",
CodexOAuthAdminError::CallbackRejected => "callback_rejected",
CodexOAuthAdminError::TokenRejected => "token_rejected",
CodexOAuthAdminError::IdentityRejected => "identity_rejected",
CodexOAuthAdminError::UpstreamUnavailable => "upstream_unavailable",
CodexOAuthAdminError::Ambiguous => "ambiguous",
CodexOAuthAdminError::StorageUnavailable => "storage_unavailable",
@@ -43,10 +43,8 @@ impl OpenAiConfig {
self.quota.validate()?;
self.auth.validate()?;
self.wire_profile.validate()?;
self.identity_secret_path = source_dir
.parent()
.unwrap_or(source_dir)
.join(".runtime/data/identity_hmac_secret");
let project_root = source_dir.parent().unwrap_or(source_dir);
self.identity_secret_path = project_root.join(".runtime/data/identity_hmac_secret");
Ok(())
}
@@ -278,6 +276,7 @@ impl CodexAuthSettings {
#[serde(deny_unknown_fields)]
pub struct CodexWireProfileConfig {
pub originator: String,
/// npm `@openai/codex` 启动版本;运行时发布任务会更新此画像字段。
pub codex_version: String,
pub desktop_version: String,
pub desktop_build: String,
@@ -292,10 +291,10 @@ impl Default for CodexWireProfileConfig {
fn default() -> Self {
Self {
originator: "Codex Desktop".to_owned(),
codex_version: "0.145.0".to_owned(),
desktop_version: "26.721.31836".to_owned(),
desktop_build: "5828".to_owned(),
os_type: "macOS".to_owned(),
codex_version: "0.146.0".to_owned(),
desktop_version: "26.727.51351".to_owned(),
desktop_build: "6119".to_owned(),
os_type: "Mac OS".to_owned(),
os_version: "15.7.1".to_owned(),
arch: "arm64".to_owned(),
terminal: "unknown".to_owned(),
@@ -16,8 +16,7 @@ use gateway_core::engine::credential::{
};
use gateway_core::provider_ports::{
ProviderLeaseAcquisition, ProviderLeaseGuard, ProviderLeasePort, ProviderLeaseRequest,
ProviderRefreshCapacityRequest, ProviderRefreshLeaseRequest, ProviderRefreshPolicy,
ProviderRuntimePolicyPort,
ProviderRefreshCapacityRequest, ProviderRefreshLeaseRequest, ProviderRuntimePolicyPort,
};
use gateway_core::routing::ProviderKind;
use secrecy::{ExposeSecret, SecretString};
@@ -25,21 +24,21 @@ use serde::Serialize;
use serde_json::Value;
use thiserror::Error;
use super::identity::{
CodexAccountIdentityVerifier, CodexIdentityExpectation, CodexIdentityVerificationError,
};
use super::recovery_log::{CodexOAuthRecoveryOperation, record_oauth_recovery};
use super::security::CodexCredentialCodec;
use super::token_client::{RefreshFailure, TokenRefresher};
use super::types::{
CODEX_AUTHENTICATION_KIND_AGENT_IDENTITY, CODEX_AUTHENTICATION_KIND_OAUTH, CodexAccountProfile,
CodexAgentIdentityAuthMode, CodexAgentIdentityCredentialData, CodexCredentialData,
CodexOAuthSecret,
CodexCredentialPrincipal, CodexOAuthMetadata, CodexOAuthSecret, parse_access_token_expiration,
};
const PROVIDER_NAME: &str = "openai";
const CODEX_CREDENTIAL_SCHEMA_VERSION: u32 = 1;
const MAX_BATCH: usize = 200;
const MAX_IMPORT_DOCUMENT_BYTES: usize = 64 * 1024 * 1024;
const MAX_IMPORT_EMAIL_BYTES: usize = 512;
const MAX_IMPORT_PLAN_BYTES: usize = 128;
pub struct ImportCodexOAuthCredential {
pub account_id: String,
@@ -50,6 +49,35 @@ pub struct ImportCodexOAuthCredential {
pub enabled: bool,
}
/// OAuth credential 的最小创建输入。
///
/// metadata 来自 ID token 的本地 payload 解码,或导入文档的展示字段;不包含
/// 签名、issuer、audience、nonce 或 access/refresh token 验证。
pub(crate) struct UnresolvedCodexOAuthCredential {
pub(crate) account_id: String,
pub(crate) name: String,
pub(crate) secret: CodexOAuthSecret,
pub(crate) metadata: CodexOAuthMetadata,
pub(crate) access_token_expires_at: Option<DateTime<Utc>>,
pub(crate) next_refresh_at: Option<DateTime<Utc>>,
pub(crate) enabled: bool,
}
impl fmt::Debug for UnresolvedCodexOAuthCredential {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("UnresolvedCodexOAuthCredential")
.field("account_id", &self.account_id)
.field("name", &self.name)
.field("secret", &"<redacted>")
.field("metadata", &self.metadata)
.field("access_token_expires_at", &self.access_token_expires_at)
.field("next_refresh_at", &self.next_refresh_at)
.field("enabled", &self.enabled)
.finish()
}
}
impl std::fmt::Debug for ImportCodexOAuthCredential {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter
@@ -106,6 +134,7 @@ enum ParsedCodexAuthentication {
OAuth {
access_token: Option<String>,
refresh_token: Option<String>,
id_token: Option<String>,
},
AgentIdentity {
runtime_id: String,
@@ -120,6 +149,7 @@ impl fmt::Debug for ParsedCodexAuthentication {
Self::OAuth {
access_token,
refresh_token,
id_token,
} => formatter
.debug_struct("OAuth")
.field("access_token", &access_token.as_ref().map(|_| "<redacted>"))
@@ -127,6 +157,7 @@ impl fmt::Debug for ParsedCodexAuthentication {
"refresh_token",
&refresh_token.as_ref().map(|_| "<redacted>"),
)
.field("id_token", &id_token.as_ref().map(|_| "<redacted>"))
.finish(),
Self::AgentIdentity {
runtime_id: _,
@@ -347,8 +378,6 @@ impl fmt::Debug for PreparedCodexCredentialRotationGuard {
pub enum CodexCredentialAdminError {
#[error("Codex account input is invalid")]
InvalidInput,
#[error("Codex account identity does not match the existing account")]
IdentityMismatch,
#[error("Codex credential JSON is invalid")]
InvalidCredential,
#[error("Codex account was not found")]
@@ -365,10 +394,6 @@ pub enum CodexCredentialAdminError {
RefreshUnavailable,
#[error("Codex refresh send state is ambiguous")]
RefreshAmbiguous,
#[error("Codex refreshed identity was rejected")]
IdentityRejected,
#[error("Codex identity verification is unavailable")]
IdentityUnavailable,
}
/// 无状态的 Codex Admin command preparer。
@@ -387,8 +412,7 @@ impl CodexCredentialAdmin {
if input.name.trim().is_empty() {
return Err(CodexCredentialAdminError::InvalidInput);
}
let access_token_expires_at =
required_time(input.verified_account.access_token_expires_at)?;
let access_token_expires_at = optional_time(input.verified_account.access_token_expires_at);
let revision =
CredentialRevision::new(1).map_err(|_| CodexCredentialAdminError::InvalidCredential)?;
let upstream_user_id = input.verified_account.chatgpt_user_id.clone();
@@ -399,10 +423,10 @@ impl CodexCredentialAdmin {
account_id,
provider,
input.name,
upstream_user_id,
Some(upstream_user_id),
CODEX_AUTHENTICATION_KIND_OAUTH.to_owned(),
revision,
Some(access_token_expires_at),
access_token_expires_at,
)
.with_profile(
input.verified_account.email,
@@ -420,6 +444,59 @@ impl CodexCredentialAdmin {
})
}
/// 为已取得 OAuth access token、但资料尚未补全的账号创建最小记录。
///
/// ID token payload 只用于本地字段投影;创建路径绝不发起 usage/profile 请求。
pub(crate) fn prepare_unresolved_oauth(
&self,
input: UnresolvedCodexOAuthCredential,
) -> Result<NewProviderAccount, CodexCredentialAdminError> {
let account_id = ProviderAccountId::new(input.account_id)
.map_err(|_| CodexCredentialAdminError::InvalidInput)?;
let provider = ProviderKind::new(PROVIDER_NAME)
.map_err(|_| CodexCredentialAdminError::InvalidInput)?;
if input.name.trim().is_empty() {
return Err(CodexCredentialAdminError::InvalidInput);
}
let revision =
CredentialRevision::new(1).map_err(|_| CodexCredentialAdminError::InvalidCredential)?;
let CodexOAuthMetadata {
email,
chatgpt_plan_type: plan_type,
chatgpt_user_id: upstream_user_id,
chatgpt_account_id: upstream_account_id,
} = input.metadata;
let has_upstream_user_id = upstream_user_id.is_some();
let credential = CodexCredentialCodec::encode_unresolved(&input.secret, Vec::new())
.map_err(|_| CodexCredentialAdminError::InvalidCredential)?;
let account = ProviderAccount::new(
account_id,
provider,
input.name,
upstream_user_id,
CODEX_AUTHENTICATION_KIND_OAUTH.to_owned(),
revision,
optional_time(input.access_token_expires_at),
)
.with_profile(email, upstream_account_id, plan_type)
.with_runtime_state(
input.enabled,
if has_upstream_user_id {
AccountAvailability::Ready
} else {
AccountAvailability::Unknown
},
)
.with_refresh_schedule(
input.secret.refresh_token.is_some(),
optional_time(input.next_refresh_at),
);
Ok(NewProviderAccount {
account,
credential,
})
}
/// 严格输出可被 CPR 导入逻辑直接读取的 canonical 文档。
pub fn format_cpr_export(
&self,
@@ -444,7 +521,7 @@ impl CodexCredentialAdmin {
id: account.id().as_str().to_owned(),
email: account.email().map(str::to_owned),
account_id: account.upstream_account_id().map(str::to_owned),
user_id: Some(account.upstream_user_id().to_owned()),
user_id: account.upstream_user_id().map(str::to_owned),
label: Some(account.name().to_owned()),
plan_type: account.plan_type().map(str::to_owned),
status: cpr_status(&account),
@@ -499,12 +576,60 @@ impl CodexCredentialAdmin {
self.prepare_oauth_rotation(input, false)
}
/// 完整 OAuth 授权按目标数据 ID 替换凭据与上游账号身份。
pub(crate) fn prepare_reauthorization(
/// 构建一次成功 RT exchange 的 CAS 写入,保留已存账号身份资料。
///
/// Refresh endpoint 已经完成本次 token 交换的授权;这里不对新 access
/// token 重新执行身份验证。
pub(crate) fn prepare_refreshed_oauth_rotation(
&self,
input: RotateManagedCodexCredential,
current: LoadedCredential,
secret: CodexOAuthSecret,
access_token_expires_at: Option<DateTime<Utc>>,
next_refresh_at: Option<DateTime<Utc>>,
) -> Result<PreparedCodexCredentialRotation, CodexCredentialAdminError> {
self.prepare_oauth_rotation(input, true)
if current.account.provider().as_str() != PROVIDER_NAME
|| current.account.authentication_kind() != CODEX_AUTHENTICATION_KIND_OAUTH
{
return Err(CodexCredentialAdminError::InvalidCredential);
}
let mut data = CodexCredentialCodec::decode_complete(&current.credential)
.map_err(|_| CodexCredentialAdminError::InvalidCredential)?;
let oauth = data
.oauth_mut()
.ok_or(CodexCredentialAdminError::InvalidCredential)?;
oauth.access_token = secret.access_token.expose_secret().to_owned();
oauth.refresh_token = secret
.refresh_token
.as_ref()
.map(|value| value.expose_secret().to_owned());
oauth.id_token = secret
.id_token
.as_ref()
.map(|value| value.expose_secret().to_owned());
let credential = CodexCredentialCodec::encode_complete(data)
.map_err(|_| CodexCredentialAdminError::InvalidCredential)?;
let profile = ProviderAccountUpdate {
account_id: current.account.id().clone(),
name: current.account.name().to_owned(),
email: current.account.email().map(str::to_owned),
plan_type: current.account.plan_type().map(str::to_owned),
};
let credential = CredentialCasUpdate::new(
current.account.id().clone(),
current.account.revision(),
profile.clone(),
credential,
secret.refresh_token.is_some(),
optional_time(access_token_expires_at),
optional_time(next_refresh_at),
)
.map_err(|_| CodexCredentialAdminError::InvalidCredential)?;
Ok(PreparedCodexCredentialRotation {
profile,
credential,
replacement_identity: None,
refresh_guards: None,
})
}
fn prepare_oauth_rotation(
@@ -512,24 +637,16 @@ impl CodexCredentialAdmin {
input: RotateManagedCodexCredential,
replace_identity: bool,
) -> Result<PreparedCodexCredentialRotation, CodexCredentialAdminError> {
let access_token_expires_at =
required_time(input.verified_account.access_token_expires_at)?;
let access_token_expires_at = input.verified_account.access_token_expires_at;
let mut data = CodexCredentialCodec::decode_complete(&input.current.credential)
.map_err(|_| CodexCredentialAdminError::InvalidCredential)?;
let oauth = data
.oauth_mut()
.ok_or(CodexCredentialAdminError::IdentityMismatch)?;
.ok_or(CodexCredentialAdminError::InvalidCredential)?;
if input.current.account.provider().as_str() != PROVIDER_NAME
|| input.current.account.authentication_kind() != CODEX_AUTHENTICATION_KIND_OAUTH
|| (!replace_identity
&& (input.current.account.upstream_account_id()
!= Some(input.verified_account.chatgpt_account_id.as_str())
|| input.current.account.upstream_user_id()
!= input.verified_account.chatgpt_user_id
|| oauth.principal.oauth_subject != input.verified_account.oauth_subject
|| oauth.principal.poid != input.verified_account.poid))
{
return Err(CodexCredentialAdminError::IdentityMismatch);
return Err(CodexCredentialAdminError::InvalidCredential);
}
let replacement_identity = replace_identity.then(|| {
ProviderAccountIdentity::new(
@@ -538,14 +655,10 @@ impl CodexCredentialAdmin {
)
});
if replace_identity {
oauth
.principal
.oauth_subject
.clone_from(&input.verified_account.oauth_subject);
oauth
.principal
.poid
.clone_from(&input.verified_account.poid);
oauth.principal = Some(CodexCredentialPrincipal {
oauth_subject: input.verified_account.oauth_subject.clone(),
poid: input.verified_account.poid.clone(),
});
}
oauth.access_token = input.secret.access_token.expose_secret().to_owned();
oauth.refresh_token = input
@@ -572,7 +685,7 @@ impl CodexCredentialAdmin {
profile.clone(),
credential,
input.secret.refresh_token.is_some(),
Some(access_token_expires_at),
optional_time(access_token_expires_at),
optional_time(input.next_refresh_at),
)
.map_err(|_| CodexCredentialAdminError::InvalidCredential)?;
@@ -588,7 +701,6 @@ impl CodexCredentialAdmin {
/// 有状态的 Codex 手工刷新边界;消费调用方刚读取的当前 credential 并准备 CAS。
pub struct CodexCredentialAdminService {
refresher: Arc<dyn TokenRefresher>,
verifier: Arc<dyn CodexAccountIdentityVerifier>,
leases: Arc<dyn ProviderLeasePort>,
runtime_policy: Arc<dyn ProviderRuntimePolicyPort>,
}
@@ -598,7 +710,6 @@ impl fmt::Debug for CodexCredentialAdminService {
formatter
.debug_struct("CodexCredentialAdminService")
.field("refresher", &"TokenRefresher")
.field("verifier", &"CodexAccountIdentityVerifier")
.field("leases", &"ProviderLeasePort")
.field("runtime_policy", &"ProviderRuntimePolicyPort")
.finish()
@@ -608,19 +719,17 @@ impl fmt::Debug for CodexCredentialAdminService {
impl CodexCredentialAdminService {
pub fn new(
refresher: Arc<dyn TokenRefresher>,
verifier: Arc<dyn CodexAccountIdentityVerifier>,
leases: Arc<dyn ProviderLeasePort>,
runtime_policy: Arc<dyn ProviderRuntimePolicyPort>,
) -> Self {
Self {
refresher,
verifier,
leases,
runtime_policy,
}
}
/// 官方 RT exchange + AT 身份验证;结果由 App 在同一 revision/audit 事务中提交。
/// 官方 RT exchange;结果由 App 在同一 revision/audit 事务中提交。
pub async fn manual_refresh(
&self,
current: LoadedCredential,
@@ -632,7 +741,6 @@ impl CodexCredentialAdminService {
}
let runtime = CodexCredentialCodec::decode(&current.credential)
.map_err(|_| CodexCredentialAdminError::InvalidCredential)?;
let expectation = current_identity_expectation(&current.account, &runtime)?;
let oauth = runtime
.authentication
.oauth()
@@ -677,9 +785,6 @@ impl CodexCredentialAdminService {
.refresh(refresh_token.expose_secret())
.await
.map_err(map_refresh_failure)?;
if tokens.access_token.trim().is_empty() || tokens.expires_in.is_zero() {
return Err(CodexCredentialAdminError::InvalidCredential);
}
let secret = CodexOAuthSecret {
access_token: SecretString::from(tokens.access_token),
refresh_token: tokens
@@ -688,24 +793,24 @@ impl CodexCredentialAdminService {
.or_else(|| oauth.refresh_token.clone()),
id_token: oauth.id_token.clone(),
};
let verified_account = self
.verifier
.verify(&secret, &expectation)
.await
.and_then(super::identity::CodexIdentityVerification::into_complete)
.map_err(map_identity_error)?;
let next_refresh_at = refresh_time(
policy,
&account_id,
verified_account.access_token_expires_at,
secret.refresh_token.is_some(),
)?;
let mut prepared = CodexCredentialAdmin.prepare_rotation(RotateManagedCodexCredential {
Self::record_recovery_log(
CodexOAuthRecoveryOperation::ManualRefresh,
Some(account_id.as_str()),
&secret,
);
let access_token_expires_at = tokens
.expires_in
.and_then(|expires_in| SystemTime::now().checked_add(expires_in))
.map(DateTime::<Utc>::from);
// 正常预刷新由 worker 读取当时的 runtime policy 动态判断;这里仅清除
// 之前瞬态失败留下的 retry-not-before。
let next_refresh_at = None;
let mut prepared = CodexCredentialAdmin.prepare_refreshed_oauth_rotation(
current,
secret,
verified_account,
access_token_expires_at,
next_refresh_at,
})?;
)?;
prepared.refresh_guards = Some(ProviderRefreshGuards {
_capacity: capacity_guard,
_account: account_guard,
@@ -713,7 +818,10 @@ impl CodexCredentialAdminService {
Ok(prepared)
}
/// 从 OAuth token 对补全账号事实,并归一到唯一 `NewProviderAccount` 写入路径。
/// 归一导入 OAuth/agent-identity 凭据到唯一 `NewProviderAccount` 写入路径。
///
/// OAuth 导入只要取得 access token(直接提供或 RT exchange)即创建 Unknown
/// 账号;资格资料由后续 worker 异步补齐。
pub async fn prepare_import_document(
&self,
payload: Value,
@@ -732,56 +840,54 @@ impl CodexCredentialAdminService {
let mut account_ids = BTreeSet::new();
let mut upstream_identities = BTreeSet::new();
let mut accounts = Vec::with_capacity(candidates.len());
let mut policy = None;
for candidate in candidates {
let prepared = match &candidate.authentication {
ParsedCodexAuthentication::OAuth {
access_token,
refresh_token,
id_token,
} => {
let account_id = format!("acct_{}", uuid::Uuid::now_v7().simple());
if !account_ids.insert(account_id.clone()) {
return Err(CodexCredentialAdminError::InvalidInput);
}
let (secret, verified_account) = self
.verify_import_tokens(access_token.clone(), refresh_token.clone())
let typed_account_id = ProviderAccountId::new(account_id.clone())
.map_err(|_| CodexCredentialAdminError::InvalidInput)?;
let (secret, access_token_expires_at) = self
.resolve_import_tokens(
&typed_account_id,
access_token.clone(),
refresh_token.clone(),
id_token.clone(),
)
.await?;
if !upstream_identities.insert((
verified_account.chatgpt_user_id.clone(),
verified_account.chatgpt_account_id.clone(),
)) {
return Err(CodexCredentialAdminError::InvalidInput);
}
let refresh_policy = if let Some(policy) = policy {
policy
} else {
let loaded = self
.runtime_policy
.load_refresh_policy()
.await
.map_err(|_| CodexCredentialAdminError::RefreshUnavailable)?;
policy = Some(loaded);
loaded
};
let next_refresh_at = refresh_time(
refresh_policy,
&ProviderAccountId::new(account_id.clone())
.map_err(|_| CodexCredentialAdminError::InvalidInput)?,
verified_account.access_token_expires_at,
secret.refresh_token.is_some(),
)?;
CodexCredentialAdmin.prepare_import(ImportCodexOAuthCredential {
account_id,
name: verified_account
.email
.clone()
.filter(|name| !name.trim().is_empty())
.unwrap_or_else(|| "Codex OAuth".to_owned()),
secret,
verified_account,
next_refresh_at,
enabled: true,
})?
// 导入材料按管理员输入原样保存;ID token 仅尽力补展示字段,
// 解析失败不阻塞 AT/RT 入库。
let mut metadata = secret
.id_token
.as_ref()
.and_then(|token| {
super::types::parse_id_token_metadata(token.expose_secret()).ok()
})
.unwrap_or_default();
apply_import_metadata(&mut metadata, &candidate);
CodexCredentialAdmin.prepare_unresolved_oauth(
UnresolvedCodexOAuthCredential {
account_id,
name: candidate
.name
.clone()
.or_else(|| candidate.email.clone())
.or_else(|| metadata.email.clone())
.filter(|name| !name.trim().is_empty())
.unwrap_or_else(|| "Codex OAuth".to_owned()),
secret,
metadata,
access_token_expires_at,
next_refresh_at: None,
enabled: true,
},
)?
}
ParsedCodexAuthentication::AgentIdentity { .. } => {
let account_id = candidate
@@ -796,14 +902,17 @@ impl CodexCredentialAdminService {
import_runtime_state(candidate.status.as_deref())?;
let mut account =
self.prepare_agent_identity_import(&candidate, account_id, enabled)?;
if !upstream_identities.insert((
account.account.upstream_user_id().to_owned(),
account
.account
.upstream_account_id()
.ok_or(CodexCredentialAdminError::InvalidInput)?
.to_owned(),
)) {
let upstream_user_id = account
.account
.upstream_user_id()
.ok_or(CodexCredentialAdminError::InvalidInput)?
.to_owned();
let upstream_account_id = account
.account
.upstream_account_id()
.ok_or(CodexCredentialAdminError::InvalidInput)?
.to_owned();
if !upstream_identities.insert((upstream_user_id, upstream_account_id)) {
return Err(CodexCredentialAdminError::InvalidInput);
}
account.account = account.account.with_runtime_state(enabled, availability);
@@ -815,35 +924,27 @@ impl CodexCredentialAdminService {
Ok(PreparedCodexAccountImport { accounts })
}
async fn verify_import_tokens(
async fn resolve_import_tokens(
&self,
account_id: &ProviderAccountId,
access_token: Option<String>,
refresh_token: Option<String>,
) -> Result<(CodexOAuthSecret, CodexAccountProfile), CodexCredentialAdminError> {
let access_token = access_token
.as_deref()
.map(normalize_bearer)
.filter(|token| !token.is_empty());
let refresh_token = refresh_token
.as_deref()
.map(normalize_bearer)
.filter(|token| !token.is_empty());
id_token: Option<String>,
) -> Result<(CodexOAuthSecret, Option<DateTime<Utc>>), CodexCredentialAdminError> {
let id_token = id_token.map(SecretString::from);
if let Some(access_token) = access_token {
let access_token_expires_at = parse_access_token_expiration(&access_token);
let secret = CodexOAuthSecret {
access_token: SecretString::from(access_token),
refresh_token: refresh_token.clone().map(SecretString::from),
id_token: None,
refresh_token: refresh_token.map(SecretString::from),
id_token,
};
match self.verifier.verify_import(&secret).await {
Ok(profile) => return Ok((secret, profile)),
Err(CodexIdentityVerificationError::Unavailable) => {
return Err(CodexCredentialAdminError::IdentityUnavailable);
}
Err(CodexIdentityVerificationError::Rejected) if refresh_token.is_none() => {
return Err(CodexCredentialAdminError::IdentityRejected);
}
Err(CodexIdentityVerificationError::Rejected) => {}
}
Self::record_recovery_log(
CodexOAuthRecoveryOperation::ImportDirect,
Some(account_id.as_str()),
&secret,
);
return Ok((secret, access_token_expires_at));
}
let refresh_token = refresh_token.ok_or(CodexCredentialAdminError::InvalidCredential)?;
let tokens = self
@@ -851,9 +952,6 @@ impl CodexCredentialAdminService {
.refresh(&refresh_token)
.await
.map_err(map_refresh_failure)?;
if tokens.access_token.trim().is_empty() || tokens.expires_in.is_zero() {
return Err(CodexCredentialAdminError::InvalidCredential);
}
let secret = CodexOAuthSecret {
access_token: SecretString::from(tokens.access_token),
// RT 未轮换时仍保留导入提供的 RT,保证本次补全不会丢失后续刷新能力。
@@ -861,14 +959,34 @@ impl CodexCredentialAdminService {
.refresh_token
.map(SecretString::from)
.or_else(|| Some(SecretString::from(refresh_token))),
id_token: None,
id_token,
};
let profile = self
.verifier
.verify_import(&secret)
.await
.map_err(map_identity_error)?;
Ok((secret, profile))
Self::record_recovery_log(
CodexOAuthRecoveryOperation::ImportRefreshToken,
Some(account_id.as_str()),
&secret,
);
let access_token_expires_at = tokens
.expires_in
.and_then(|expires_in| SystemTime::now().checked_add(expires_in))
.map(DateTime::<Utc>::from);
Ok((secret, access_token_expires_at))
}
fn record_recovery_log(
operation: CodexOAuthRecoveryOperation,
account_id: Option<&str>,
secret: &CodexOAuthSecret,
) {
record_oauth_recovery(
operation,
account_id,
secret.access_token.expose_secret(),
secret
.refresh_token
.as_ref()
.map(ExposeSecret::expose_secret),
);
}
fn prepare_agent_identity_import(
@@ -920,7 +1038,7 @@ impl CodexCredentialAdminService {
account_id,
provider,
name,
upstream_user_id,
Some(upstream_user_id),
CODEX_AUTHENTICATION_KIND_AGENT_IDENTITY.to_owned(),
revision,
None,
@@ -939,56 +1057,10 @@ impl CodexCredentialAdminService {
}
}
fn current_identity_expectation(
account: &ProviderAccount,
credential: &super::security::CodexRuntimeCredential,
) -> Result<CodexIdentityExpectation, CodexCredentialAdminError> {
let account_id = account
.upstream_account_id()
.ok_or(CodexCredentialAdminError::InvalidCredential)?;
let principal = credential
.principal
.as_ref()
.ok_or(CodexCredentialAdminError::InvalidCredential)?;
CodexIdentityExpectation::current(
principal.oauth_subject.clone(),
principal.poid.clone(),
account_id.to_owned(),
account.upstream_user_id().to_owned(),
credential.installation_id.clone(),
)
.map_err(|_| CodexCredentialAdminError::InvalidCredential)
}
fn required_time(
value: Option<chrono::DateTime<chrono::Utc>>,
) -> Result<SystemTime, CodexCredentialAdminError> {
value
.map(SystemTime::from)
.ok_or(CodexCredentialAdminError::InvalidCredential)
}
fn optional_time(value: Option<chrono::DateTime<chrono::Utc>>) -> Option<SystemTime> {
value.map(SystemTime::from)
}
pub(crate) fn refresh_time(
policy: ProviderRefreshPolicy,
account_id: &ProviderAccountId,
access_token_expires_at: Option<DateTime<Utc>>,
has_refresh_token: bool,
) -> Result<Option<DateTime<Utc>>, CodexCredentialAdminError> {
if !has_refresh_token {
return Ok(None);
}
let expires_at = required_time(access_token_expires_at)?;
policy
.next_attempt_at(account_id, expires_at, SystemTime::now())
.map(DateTime::<Utc>::from)
.map(Some)
.map_err(|_| CodexCredentialAdminError::InvalidCredential)
}
fn cpr_status(account: &ProviderAccount) -> &'static str {
if !account.enabled() {
return "disabled";
@@ -1016,15 +1088,6 @@ const fn map_refresh_failure(error: RefreshFailure) -> CodexCredentialAdminError
}
}
const fn map_identity_error(error: CodexIdentityVerificationError) -> CodexCredentialAdminError {
match error {
CodexIdentityVerificationError::Rejected => CodexCredentialAdminError::IdentityRejected,
CodexIdentityVerificationError::Unavailable => {
CodexCredentialAdminError::IdentityUnavailable
}
}
}
fn parse_import_document(
payload: &Value,
) -> Result<Vec<ParsedCodexImportAccount>, CodexCredentialAdminError> {
@@ -1037,20 +1100,28 @@ fn parse_import_document(
if !is_openai_oauth_candidate(value) {
continue;
}
accounts.push(ParsedCodexImportAccount {
id: None,
name: None,
email: None,
plan_type: None,
chatgpt_account_id: None,
chatgpt_user_id: None,
authentication: parse_oauth_import_tokens(value)?,
status: None,
});
accounts.push(parse_oauth_import_account(value)?);
}
Ok(accounts)
}
fn parse_oauth_import_account(
value: &Value,
) -> Result<ParsedCodexImportAccount, CodexCredentialAdminError> {
let credentials = value.get("credentials").unwrap_or(value);
Ok(ParsedCodexImportAccount {
id: first_string(value, &["id"]),
name: first_string(value, &["name", "label"]),
email: first_string(value, &["email"]).or_else(|| first_string(credentials, &["email"])),
plan_type: first_string(value, &["plan_type", "planType"])
.or_else(|| first_string(credentials, &["plan_type", "planType"])),
chatgpt_account_id: None,
chatgpt_user_id: None,
authentication: parse_oauth_import_tokens(value)?,
status: first_string(value, &["status"]),
})
}
fn parse_agent_identity_account(
value: &Value,
) -> Result<ParsedCodexImportAccount, CodexCredentialAdminError> {
@@ -1126,6 +1197,7 @@ fn parse_oauth_import_tokens(
) -> Result<ParsedCodexAuthentication, CodexCredentialAdminError> {
let mut access_token = None;
let mut refresh_token = None;
let mut id_token = None;
let mut pending = vec![value];
while let Some(current) = pending.pop() {
match current {
@@ -1134,18 +1206,14 @@ fn parse_oauth_import_tokens(
let token = match key.as_str() {
"access_token" | "accessToken" | "at" | "token" => &mut access_token,
"refresh_token" | "refreshToken" | "rt" => &mut refresh_token,
"id_token" | "idToken" => &mut id_token,
_ => {
pending.push(value);
continue;
}
};
let value = value
.as_str()
.map(str::trim)
.filter(|value| !value.is_empty())
.ok_or(CodexCredentialAdminError::InvalidCredential)?;
if token.replace(value.to_owned()).is_some() {
return Err(CodexCredentialAdminError::InvalidInput);
if let Some(value) = value.as_str() {
*token = Some(value.to_owned());
}
}
}
@@ -1159,6 +1227,7 @@ fn parse_oauth_import_tokens(
Ok(ParsedCodexAuthentication::OAuth {
access_token,
refresh_token,
id_token,
})
}
@@ -1202,14 +1271,25 @@ fn first_string(value: &Value, keys: &[&str]) -> Option<String> {
.map(str::to_owned)
}
fn normalize_bearer(value: &str) -> String {
fn apply_import_metadata(metadata: &mut CodexOAuthMetadata, candidate: &ParsedCodexImportAccount) {
if metadata.email.is_none() {
metadata.email = candidate
.email
.as_deref()
.and_then(|value| normalize_import_text(value, MAX_IMPORT_EMAIL_BYTES));
}
if metadata.chatgpt_plan_type.is_none() {
metadata.chatgpt_plan_type = candidate
.plan_type
.as_deref()
.and_then(|value| normalize_import_text(value, MAX_IMPORT_PLAN_BYTES));
}
}
fn normalize_import_text(value: &str, max_bytes: usize) -> Option<String> {
let value = value.trim();
value
.strip_prefix("Bearer ")
.or_else(|| value.strip_prefix("bearer "))
.unwrap_or(value)
.trim()
.to_owned()
(!value.is_empty() && value.len() <= max_bytes && !value.chars().any(char::is_control))
.then(|| value.to_owned())
}
fn import_runtime_state(
@@ -10,19 +10,33 @@ use crate::transport::request::derive_conversation_anchor;
pub(crate) fn derive_codex_session_affinity_key(
request: &CodexResponsesRequest,
) -> Option<ProviderSessionAffinityKey> {
if let Some(conversation_id) = request
let session_key = if let Some(conversation_id) = request
.local_conversation_id
.as_deref()
.map(str::trim)
.filter(|value| !value.is_empty())
{
if let Ok(key) = ProviderSessionAffinityKey::try_new(conversation_id.to_owned()) {
return Some(key);
Some(key)
} else {
opaque_affinity_key("local-conversation", conversation_id)
}
return opaque_affinity_key("local-conversation", conversation_id);
}
let (domain, value) = derive_conversation_anchor(request)?;
opaque_affinity_key(domain, &value)
} else {
let (domain, value) = derive_conversation_anchor(request)?;
opaque_affinity_key(domain, &value)
};
let session_discriminator = request.subagent_kind();
session_key.and_then(|session_key| match session_discriminator.as_deref() {
None => Some(session_key),
Some(discriminator) => {
// 仅扩展原会话锚点的键空间;后续查找、绑定和调度仍使用同一套亲和逻辑。
opaque_affinity_key(
"subagent-session",
&format!("{discriminator}\0{}", session_key.expose_to_store()),
)
}
})
}
fn opaque_affinity_key(domain: &str, value: &str) -> Option<ProviderSessionAffinityKey> {
File diff suppressed because it is too large Load Diff
@@ -5,9 +5,9 @@ mod affinity;
mod agent_identity;
mod catalog;
mod cookie;
mod identity;
mod oauth;
mod quota;
mod recovery_log;
mod refresh;
mod repository;
mod security;
@@ -15,7 +15,6 @@ mod selector;
pub mod token_client;
mod types;
pub(crate) use admin::refresh_time;
pub(crate) use affinity::{
derive_codex_cyber_policy_session_key, derive_codex_session_affinity_key,
};
@@ -38,13 +37,6 @@ pub use catalog::{
CodexCredentialCatalogSnapshot, CodexPlanCatalog,
};
pub use cookie::{CodexCookiePolicy, CookiePolicyError};
pub use identity::{
CodexAccountIdentityService, CodexAccountIdentityVerifier, CodexAuthenticatedAccount,
CodexAuthenticatedAccountSource, CodexIdentityExpectation, CodexIdentityVerification,
CodexIdentityVerificationError, CodexJwksSource, CodexJwtIdentityVerifier, CodexSignedIdentity,
CodexSignedIdentityVerifier, OFFICIAL_OPENAI_API_AUDIENCE, OFFICIAL_OPENAI_ISSUER,
OFFICIAL_OPENAI_JWKS_URI, ReqwestCodexAuthenticatedAccountSource, ReqwestOpenAiJwksSource,
};
pub use oauth::{
CodexOAuthAdmin, CodexOAuthAdminError, CodexOAuthAdminService, CodexOAuthAuthorizationStarted,
CodexOAuthPendingClaimOutcome, CodexOAuthPendingStore, CodexOAuthPendingStoreError,
@@ -1,6 +1,10 @@
//! Codex Authorization Code + PKCE/OIDC 管理流。
use std::{fmt, sync::Arc, time::Duration};
use std::{
fmt,
sync::Arc,
time::{Duration, SystemTime},
};
use async_trait::async_trait;
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
@@ -13,9 +17,8 @@ use gateway_admin::model::{
},
};
use gateway_core::engine::credential::{
NewProviderAccount, ProviderAccountId, ProviderAccountStore,
LoadedCredential, NewProviderAccount, ProviderAccountId, ProviderAccountStore,
};
use gateway_core::provider_ports::ProviderRuntimePolicyPort;
use secrecy::{ExposeSecret, SecretString};
use sha2::{Digest, Sha256};
use subtle::ConstantTimeEq as _;
@@ -23,16 +26,16 @@ use url::Url;
use uuid::Uuid;
use super::admin::{
CodexCredentialAdmin, CodexCredentialAdminError, ImportCodexOAuthCredential,
PreparedCodexCredentialRotation, RotateManagedCodexCredential, refresh_time,
};
use super::identity::{
CodexAccountIdentityVerifier, CodexIdentityExpectation, CodexIdentityVerificationError,
CodexCredentialAdmin, CodexCredentialAdminError, PreparedCodexCredentialRotation,
UnresolvedCodexOAuthCredential,
};
use super::recovery_log::{CodexOAuthRecoveryOperation, record_oauth_recovery};
use super::security::CodexCredentialCodec;
use super::token_client::{
AuthorizationCodeExchangeError, AuthorizationCodeExchanger, AuthorizationCodeGrant,
OFFICIAL_CODEX_OAUTH_CLIENT_ID, OFFICIAL_CODEX_REDIRECT_URI,
};
use super::types::parse_id_token_metadata;
const AUTHORIZATION_ENDPOINT: &str = "https://auth.openai.com/oauth/authorize";
const AUTHORIZATION_SCOPE: &str = "openid profile email offline_access";
@@ -316,8 +319,6 @@ pub enum CodexOAuthAdminError {
CallbackRejected,
#[error("Codex OAuth token exchange was rejected")]
TokenRejected,
#[error("Codex OAuth identity verification was rejected")]
IdentityRejected,
#[error("Codex OAuth upstream is unavailable")]
UpstreamUnavailable,
#[error("Codex OAuth exchange send state is ambiguous")]
@@ -344,9 +345,7 @@ pub trait CodexOAuthAdmin: Send + Sync {
pub struct CodexOAuthAdminService {
pending: Arc<dyn CodexOAuthPendingStore>,
exchanger: Arc<dyn AuthorizationCodeExchanger>,
verifier: Arc<dyn CodexAccountIdentityVerifier>,
store: Arc<dyn ProviderAccountStore>,
runtime_policy: Arc<dyn ProviderRuntimePolicyPort>,
credentials: CodexCredentialAdmin,
oauth_client_id: String,
}
@@ -396,17 +395,13 @@ impl CodexOAuthAdminService {
pub fn new(
pending: Arc<dyn CodexOAuthPendingStore>,
exchanger: Arc<dyn AuthorizationCodeExchanger>,
verifier: Arc<dyn CodexAccountIdentityVerifier>,
store: Arc<dyn ProviderAccountStore>,
runtime_policy: Arc<dyn ProviderRuntimePolicyPort>,
credentials: CodexCredentialAdmin,
) -> Self {
Self {
pending,
exchanger,
verifier,
store,
runtime_policy,
credentials,
oauth_client_id: OFFICIAL_CODEX_OAUTH_CLIENT_ID.to_owned(),
}
@@ -487,8 +482,6 @@ impl CodexOAuthAdminService {
callback_url: &str,
) -> Result<(PendingAuthorizationMutation, CompletedCodexOAuthCredential), CodexOAuthAdminError>
{
let (mut secret, id_token) = self.exchange_pending(&pending, callback_url).await?;
let mutation = pending.mutation.clone();
let current = if let Some(target) = pending.reauthorization() {
let current = self
.store
@@ -499,56 +492,43 @@ impl CodexOAuthAdminService {
} else {
None
};
let expectation = CodexIdentityExpectation::default();
let profile = self
.verifier
.verify_authorization(&secret, &id_token, &pending.nonce, &expectation)
.await
.and_then(super::identity::CodexIdentityVerification::into_complete)
.map_err(map_identity_error)?;
let fallback_refresh_token = current
.as_ref()
.map(current_oauth_refresh_token)
.transpose()?
.flatten();
let (mut secret, id_token, expires_in) = self
.exchange_pending(&pending, callback_url, fallback_refresh_token)
.await?;
let mutation = pending.mutation.clone();
let access_token_expires_at = expires_in
.and_then(|expires_in| SystemTime::now().checked_add(expires_in))
.map(DateTime::<Utc>::from);
let metadata = parse_id_token_metadata(id_token.expose_secret())
.map_err(|_| CodexOAuthAdminError::TokenRejected)?;
secret.id_token = Some(id_token);
let policy = self
.runtime_policy
.load_refresh_policy()
.await
.map_err(|_| CodexOAuthAdminError::StorageUnavailable)?;
let credential = if let Some(current) = current {
let next_refresh_at = refresh_time(
policy,
current.account.id(),
profile.access_token_expires_at,
secret.refresh_token.is_some(),
)
.map_err(map_admin_error)?;
CompletedCodexOAuthCredential::Reauthorize(
self.credentials
.prepare_reauthorization(RotateManagedCodexCredential {
.prepare_refreshed_oauth_rotation(
current,
secret,
verified_account: profile,
next_refresh_at,
})
access_token_expires_at,
None,
)
.map_err(map_admin_error)?,
)
} else {
let account_id = format!("acct_{}", Uuid::now_v7().simple());
let typed_account_id = ProviderAccountId::new(account_id.clone())
.map_err(|_| CodexOAuthAdminError::Credential)?;
let next_refresh_at = refresh_time(
policy,
&typed_account_id,
profile.access_token_expires_at,
secret.refresh_token.is_some(),
)
.map_err(map_admin_error)?;
CompletedCodexOAuthCredential::Create(
self.credentials
.prepare_import(ImportCodexOAuthCredential {
.prepare_unresolved_oauth(UnresolvedCodexOAuthCredential {
account_id,
name: pending.name,
secret,
verified_account: profile,
next_refresh_at,
metadata,
access_token_expires_at,
next_refresh_at: None,
enabled: true,
})
.map_err(map_admin_error)?,
@@ -654,7 +634,15 @@ impl CodexOAuthAdminService {
&self,
pending: &CodexPendingAuthorization,
callback_url: &str,
) -> Result<(super::types::CodexOAuthSecret, SecretString), CodexOAuthAdminError> {
fallback_refresh_token: Option<SecretString>,
) -> Result<
(
super::types::CodexOAuthSecret,
SecretString,
Option<Duration>,
),
CodexOAuthAdminError,
> {
let (code, callback_state) = callback_parts(callback_url)?;
if !constant_time_equal(
pending.state.expose_secret().as_bytes(),
@@ -670,10 +658,36 @@ impl CodexOAuthAdminService {
})
.await
.map_err(map_exchange_error)?;
Ok((tokens.secret, tokens.id_token))
let mut secret = tokens.secret;
if secret.refresh_token.is_none() {
secret.refresh_token = fallback_refresh_token;
}
record_oauth_recovery(
CodexOAuthRecoveryOperation::AuthorizationCode,
pending
.reauthorization()
.map(|account_id| account_id.as_str()),
secret.access_token.expose_secret(),
secret
.refresh_token
.as_ref()
.map(ExposeSecret::expose_secret),
);
Ok((secret, tokens.id_token, tokens.expires_in))
}
}
fn current_oauth_refresh_token(
current: &LoadedCredential,
) -> Result<Option<SecretString>, CodexOAuthAdminError> {
let runtime = CodexCredentialCodec::decode(&current.credential)
.map_err(|_| CodexOAuthAdminError::Credential)?;
Ok(runtime
.authentication
.oauth()
.and_then(|oauth| oauth.refresh_token.clone()))
}
fn authorization_url(
pending: &CodexPendingAuthorization,
oauth_client_id: &str,
@@ -699,27 +713,19 @@ fn authorization_url(
fn callback_parts(value: &str) -> Result<(SecretString, SecretString), CodexOAuthAdminError> {
let url = Url::parse(value).map_err(|_| CodexOAuthAdminError::CallbackRejected)?;
if url.scheme() != "http"
|| url.host_str() != Some("localhost")
|| url.port() != Some(1455)
|| url.path() != "/auth/callback"
|| url.fragment().is_some()
{
return Err(CodexOAuthAdminError::CallbackRejected);
}
let mut code = None;
let mut state = None;
for (key, value) in url.query_pairs() {
match key.as_ref() {
"code" => set_unique_callback_parameter(&mut code, value.into_owned())?,
"state" => set_unique_callback_parameter(&mut state, value.into_owned())?,
// OAuth 服务可能附加 `scope`、`iss` 等标准参数。
// 固定回调地址与唯一的 `code`/`state` 才是安全边界,无关参数不参与校验。
// 回调地址只承载 query 参数;安全绑定由唯一的 code/state、
// 服务端保存的 state 和 PKCE token exchange 共同完成。
_ => {}
}
}
let code = code.filter(|value| valid_secret(value));
let state = state.filter(|value| valid_secret(value));
let code = code.filter(|value| !value.is_empty());
let state = state.filter(|value| !value.is_empty());
match (code, state) {
(Some(code), Some(state)) => Ok((SecretString::from(code), SecretString::from(state))),
_ => Err(CodexOAuthAdminError::CallbackRejected),
@@ -813,13 +819,6 @@ fn map_exchange_error(error: AuthorizationCodeExchangeError) -> CodexOAuthAdminE
}
}
fn map_identity_error(error: CodexIdentityVerificationError) -> CodexOAuthAdminError {
match error {
CodexIdentityVerificationError::Rejected => CodexOAuthAdminError::IdentityRejected,
CodexIdentityVerificationError::Unavailable => CodexOAuthAdminError::UpstreamUnavailable,
}
}
fn map_admin_error(_: CodexCredentialAdminError) -> CodexOAuthAdminError {
CodexOAuthAdminError::Credential
}
@@ -20,12 +20,13 @@ use gateway_core::engine::credential::{
AccountAvailability, AccountQuotaSignals, CredentialRevision, OpaqueProviderData,
ProviderAccount, ProviderAccountId, ProviderAccountStore, QuotaObservation, QuotaWriteOutcome,
};
use gateway_core::provider_ports::{ProviderCooldown, ProviderCooldownPort};
use gateway_protocol::openai::events::{
ParsedRateLimits, RateLimitDetails, RateLimitWindow, parse_rate_limit_headers,
};
use reqwest::Client;
use secrecy::ExposeSecret;
use serde_json::{Map, Number, Value};
use serde_json::{Map, Value};
use thiserror::Error;
use tokio::sync::Mutex;
use uuid::Uuid;
@@ -35,7 +36,10 @@ use crate::transport::{CodexBackendClient, CodexClientError, CodexRequestContext
use super::agent_identity::{CodexAgentIdentityTaskService, PreparedCodexRuntimeCredential};
use super::repository::{CodexCredentialRepository, CredentialRepositoryError};
use recovery::{QuotaRecoveryEvidence, quota_state_transition, quota_success_state};
use recovery::{
authoritative_quota_snapshot_state, quota_state_reason, quota_state_transition,
quota_success_state,
};
use snapshot::{
parse_account_quota_snapshot, quota_projection_ttl, quota_snapshot_from_observation,
scheduling_signals_from_snapshot,
@@ -45,6 +49,8 @@ const DEFAULT_RATE_LIMIT_COOLDOWN: Duration = Duration::from_secs(60);
pub(crate) const QUOTA_SCHEDULING_TTL: Duration = Duration::from_secs(10 * 60);
const QUOTA_HYDRATION_FAILURE_TTL: Duration = Duration::from_secs(5);
const QUOTA_PERIODIC_REFRESH_MIN_INTERVAL: Duration = Duration::from_secs(30 * 60);
/// 首次 OAuth 异步观察失败时,由既有 quota worker 兜底重试的单轮上限。
const INITIAL_QUOTA_SYNC_BATCH: usize = 100;
// 5xx 上游拒绝的短退避重试预算;指数退避 1s/2s,吞掉瞬时抖动。
const QUOTA_FETCH_5XX_MAX_RETRIES: u32 = 2;
const QUOTA_FETCH_5XX_BASE_DELAY: Duration = Duration::from_secs(1);
@@ -120,6 +126,7 @@ pub struct CodexCredentialQuotaService {
http: Client,
base_url: String,
agent_identity: Arc<CodexAgentIdentityTaskService>,
cooldowns: Arc<dyn ProviderCooldownPort>,
scheduling: CodexQuotaSchedulingProjection,
}
@@ -333,7 +340,7 @@ fn quota_refresh_candidate(
now: SystemTime,
limit_reached: bool,
) -> Option<ProviderAccount> {
// 定时器只兜底「已耗尽(402)」或「任一窗口触顶(limit_reached 死锁)」账号,
// 定时器只兜底已耗尽或任一窗口触顶的账号;
// 正常账号由真实请求的响应头被动同步。
(eligible_periodic_quota_refresh(&account, now)
&& (account.availability() == AccountAvailability::QuotaExhausted || limit_reached))
@@ -379,6 +386,7 @@ impl CodexCredentialQuotaService {
http: Client,
base_url: String,
agent_identity: Arc<CodexAgentIdentityTaskService>,
cooldowns: Arc<dyn ProviderCooldownPort>,
) -> Self {
Self {
store: Arc::clone(repository.store()),
@@ -387,11 +395,12 @@ impl CodexCredentialQuotaService {
http,
base_url,
agent_identity,
cooldowns,
scheduling: CodexQuotaSchedulingProjection::default(),
}
}
/// 写入新快照前读取旧确认 reset(402 恢复用「旧 reset 已到期」判定)。
/// 写入新快照前读取 core primary 已确认耗尽窗口的 reset,用于判断该窗口是否真的推进。
async fn previous_confirmed_reset(&self, account: &ProviderAccount) -> Option<SystemTime> {
let observations = self
.store
@@ -403,7 +412,59 @@ impl CodexCredentialQuotaService {
&& observation.expected_revision == account.revision()
})?;
let snapshot = quota_snapshot_from_observation(&observation)?;
snapshot.fact().resets_at().map(SystemTime::from)
snapshot
.core_primary_window()
.and_then(|window| window.reset_at())
.map(SystemTime::from)
}
/// 将已确认的额度拒绝投影为 100%,使展示、调度和恢复共用同一窗口事实。
///
/// `reset_at` 来自官方 `usage_limit_reached` 错误体;缺失时保留当前快照的 reset。
pub(crate) async fn record_confirmed_exhaustion(
&self,
account: &ProviderAccount,
reset_at: Option<i64>,
) -> Result<(), CodexCredentialQuotaError> {
let existing = self
.store
.get_quotas(std::slice::from_ref(account.id()))
.await?
.into_iter()
.find(|observation| {
observation.account_id == *account.id()
&& observation.expected_revision == account.revision()
})
.and_then(|observation| observation.quota)
.map(OpaqueProviderData::into_inner)
.unwrap_or_default();
let quota = confirmed_exhaustion_projection(
normalize_quota_window_placeholders(existing),
reset_at,
);
let observed_at = SystemTime::now();
let outcome = self
.store
.compare_and_swap_quota(QuotaObservation {
account_id: account.id().clone(),
expected_revision: account.revision(),
quota: Some(OpaqueProviderData::new(quota.clone())),
observed_at: Some(observed_at),
limit_reached: Some(true),
})
.await?;
if outcome == QuotaWriteOutcome::Conflict {
return Err(CodexCredentialQuotaError::RevisionConflict);
}
if let Ok(snapshot) = parse_account_quota_snapshot(
account.id().clone(),
account.revision(),
observed_at,
&Value::Object(quota),
) {
self.scheduling.observe(&snapshot);
}
Ok(())
}
/// 批量预热请求级额度投影;持久层或 Provider JSON 异常只退化为未知额度。
@@ -454,7 +515,17 @@ impl CodexCredentialQuotaService {
let accounts = self.repository.list_for_provider().await?;
let mut summary = CodexQuotaSyncSummary::default();
let now = SystemTime::now();
let accounts = self.scheduling.reserve_periodic_refreshes(accounts, now);
let initial = self.initial_quota_sync_accounts(&accounts, now).await?;
let periodic = self.scheduling.reserve_periodic_refreshes(accounts, now);
let accounts = initial
.into_iter()
.chain(periodic)
.fold(BTreeMap::new(), |mut unique, account| {
unique.insert(account.id().clone(), account);
unique
})
.into_values()
.collect::<Vec<_>>();
if accounts.is_empty() {
return Ok(summary);
}
@@ -495,23 +566,25 @@ impl CodexCredentialQuotaService {
// 402:真额度耗尽,写入 QuotaExhausted(worker 按 reset_at 自动恢复)。
Some(AccountAvailability::QuotaExhausted) => {
summary.exhausted += 1;
let _ = self
.repository
.apply_state(
&account,
AccountAvailability::QuotaExhausted,
observed_at,
)
.await;
self.persist_upstream_state(
&account,
AccountAvailability::QuotaExhausted,
&error,
observed_at,
)
.await;
}
Some(AccountAvailability::Banned) => {
summary.banned += 1;
let _ = self
.repository
.apply_state(&account, AccountAvailability::Banned, observed_at)
.await;
self.persist_upstream_state(
&account,
AccountAvailability::Banned,
&error,
observed_at,
)
.await;
}
// 429/503 等临时拒绝:不写 availability(限流不改变账号可用性),仅记失败待重试。
// usage 401/403 与 429/503 都不写 availability;前者等待 OAuth 或真实请求确认。
None | Some(_) => {
summary.transient += 1;
tracing::warn!(
@@ -527,6 +600,31 @@ impl CodexCredentialQuotaService {
Ok(summary)
}
/// `quota_observed_at` 为空代表首次异步观察尚未成功;不另建同步状态表。
async fn initial_quota_sync_accounts(
&self,
accounts: &[ProviderAccount],
now: SystemTime,
) -> Result<Vec<ProviderAccount>, CodexCredentialQuotaError> {
let account_ids = accounts
.iter()
.map(|account| account.id().clone())
.collect::<Vec<_>>();
let observed = self.store.get_quotas(&account_ids).await?;
let observed_ids = observed
.into_iter()
.filter_map(|observation| observation.observed_at.map(|_| observation.account_id))
.collect::<BTreeSet<_>>();
Ok(accounts
.iter()
.filter(|account| {
!observed_ids.contains(account.id()) && eligible_initial_quota_sync(account, now)
})
.take(INITIAL_QUOTA_SYNC_BATCH)
.cloned()
.collect())
}
/// 解析并 revision-fenced 落库单账号的 Provider quota JSON。
async fn apply_fetched_quota(
&self,
@@ -535,18 +633,41 @@ impl CodexCredentialQuotaService {
observed_at: SystemTime,
summary: &mut CodexQuotaSyncSummary,
) -> Result<(), CodexCredentialQuotaError> {
let snapshot = parse_account_quota_snapshot(
let refreshed_snapshot = parse_account_quota_snapshot(
account.id().clone(),
account.revision(),
observed_at,
value,
)?;
let fact = snapshot.fact();
let previous_reset_at = self.previous_confirmed_reset(account).await;
let object = value
.as_object()
.cloned()
.ok_or(CodexCredentialQuotaError::InvalidCredentialData)?;
let object = normalize_quota_window_placeholders(
value
.as_object()
.cloned()
.ok_or(CodexCredentialQuotaError::InvalidCredentialData)?,
);
let object = if account.availability() == AccountAvailability::QuotaExhausted
&& quota_success_state(
account.availability(),
&refreshed_snapshot,
previous_reset_at,
)
.is_none()
{
confirmed_exhaustion_projection(
object,
confirmed_exhaustion_reset_at(previous_reset_at, &refreshed_snapshot),
)
} else {
object
};
let snapshot = parse_account_quota_snapshot(
account.id().clone(),
account.revision(),
observed_at,
&Value::Object(object.clone()),
)?;
let fact = snapshot.fact();
let outcome = self
.store
.compare_and_swap_quota(QuotaObservation {
@@ -575,12 +696,10 @@ impl CodexCredentialQuotaService {
} else {
summary.updated += 1;
}
if let Some(availability) = quota_success_state(
if let Some(availability) = authoritative_quota_snapshot_state(
current.availability(),
fact,
&refreshed_snapshot,
previous_reset_at,
SystemTime::now(),
QuotaRecoveryEvidence::RefreshedSnapshot,
) {
let _ = self
.repository
@@ -599,6 +718,15 @@ impl CodexCredentialQuotaService {
let Some(rate_limits) = parse_rate_limit_headers(headers) else {
return Ok(false);
};
let has_quota_facts = rate_limits
.limits
.values()
.any(|details| passive_rate_limit_snapshot(details).is_some());
let previous_reset_at = if account.availability() == AccountAvailability::QuotaExhausted {
self.previous_confirmed_reset(account).await
} else {
None
};
let existing = self
.store
.get_quotas(std::slice::from_ref(account.id()))
@@ -607,12 +735,53 @@ impl CodexCredentialQuotaService {
.find(|observation| {
observation.account_id == *account.id()
&& observation.expected_revision == account.revision()
})
});
let existing_observed_at = existing
.as_ref()
.and_then(|observation| observation.observed_at);
let existing = existing
.and_then(|observation| observation.quota)
.map(OpaqueProviderData::into_inner)
.unwrap_or_default();
// 套餐、credits 等元数据可以更新,但没有额度窗口事实时必须保留旧观察时刻,
// 也不能借旧快照重新推导 quota state。
if !has_quota_facts {
let Some(observed_at) = existing_observed_at else {
return Ok(false);
};
let outcome = self
.store
.compare_and_swap_quota(QuotaObservation {
account_id: account.id().clone(),
expected_revision: account.revision(),
quota: Some(OpaqueProviderData::new(merge_passive_quota(
existing,
&rate_limits,
))),
observed_at: Some(observed_at),
limit_reached: None,
})
.await?;
return Ok(outcome != QuotaWriteOutcome::Conflict);
}
let quota = merge_passive_quota(existing, &rate_limits);
let observed_at = SystemTime::now();
let refreshed_snapshot = parse_account_quota_snapshot(
account.id().clone(),
account.revision(),
observed_at,
&Value::Object(quota.clone()),
)?;
let quota = if account.availability() == AccountAvailability::QuotaExhausted {
// 被动成功响应不能解除确认额度耗尽;继续展示 100%,直到权威 usage 快照
// 证明旧 reset 已跨过且新窗口已经推进。
confirmed_exhaustion_projection(
quota,
confirmed_exhaustion_reset_at(previous_reset_at, &refreshed_snapshot),
)
} else {
quota
};
let fact = parse_codex_quota_usage(&Value::Object(quota.clone()))?;
let snapshot = parse_account_quota_snapshot(
account.id().clone(),
@@ -627,123 +796,68 @@ impl CodexCredentialQuotaService {
expected_revision: account.revision(),
quota: Some(OpaqueProviderData::new(quota)),
observed_at: Some(observed_at),
limit_reached: Some(true),
limit_reached: Some(fact.exhausted()),
})
.await?;
if outcome == QuotaWriteOutcome::Conflict {
return Ok(false);
}
// 可用性写回必须基于最新账号状态:revision fence 只覆盖凭据轮换,
// 不覆盖其他路径的仅状态变化(Cooldown/QuotaExhausted 转换)。
let Some(current) = self.store.get_account(account.id()).await? else {
return Ok(false);
};
if current.revision() != account.revision() {
return Ok(false);
}
if let Ok(snapshot) = snapshot {
self.scheduling.observe(&snapshot);
}
if let Some(availability) = quota_success_state(
current.availability(),
fact,
None,
observed_at,
QuotaRecoveryEvidence::SuccessfulResponse,
) {
self.repository
.apply_state(&current, availability, observed_at)
.await?;
}
Ok(true)
}
/// 真实 429 的单一事实入口:合成/更新 quota 主窗口(`limit_reached=true`、
/// `used_percent=100`、`reset_at=retry-after`)。availability 不改变,
/// 窗口 reset 到期后由投影自动滚动恢复(429 只记录限流窗口)。
/// 已有更晚的 `reset_at` 不会被缩短(never-shrink)。
/// 真实 429 的单一事实入口:写入 Redis 临时限流冷却(`until = now + retry_after`)。
/// availability 与 quota 主窗口(额度重置时间)都不改变——临时限流是独立维度,
/// 到期由 Redis key 过期自动解除,不污染配额耗尽状态。
/// 已有更晚的冷却不会被缩短(put_if_later)。
pub async fn apply_rate_limit_429(
&self,
account: &ProviderAccount,
retry_after: Option<Duration>,
observed_at: SystemTime,
) -> Result<(), CodexCredentialQuotaError> {
let mut object = self
.store
.get_quotas(std::slice::from_ref(account.id()))
.await?
.into_iter()
.find(|observation| {
observation.account_id == *account.id()
&& observation.expected_revision == account.revision()
})
.and_then(|observation| observation.quota)
.map(OpaqueProviderData::into_inner)
.unwrap_or_default();
let new_reset_at = observed_at
let until = observed_at
.checked_add(retry_after.unwrap_or(DEFAULT_RATE_LIMIT_COOLDOWN))
.and_then(|reset| reset.duration_since(UNIX_EPOCH).ok())
.map(|duration| duration.as_secs() as i64);
let existing_reset_at = object
.get("rate_limit")
.and_then(Value::as_object)
.and_then(|rate_limit| rate_limit.get("primary_window"))
.and_then(Value::as_object)
.and_then(|window| window.get("reset_at"))
.and_then(Value::as_i64);
let reset_at = match (existing_reset_at, new_reset_at) {
(Some(existing), Some(next)) if existing > next => Some(existing),
(_, next) => next,
};
let mut rate_limit = object
.get("rate_limit")
.cloned()
.and_then(|value| value.as_object().cloned())
.unwrap_or_default();
rate_limit.insert("allowed".to_owned(), Value::Bool(false));
rate_limit.insert("limit_reached".to_owned(), Value::Bool(true));
let mut primary = rate_limit
.get("primary_window")
.cloned()
.and_then(|value| value.as_object().cloned())
.unwrap_or_default();
let used_percent = primary
.get("used_percent")
.and_then(Value::as_f64)
.unwrap_or(0.0)
.max(100.0);
primary.insert(
"used_percent".to_owned(),
Number::from_f64(used_percent)
.map(Value::Number)
.unwrap_or_else(|| Value::Number(Number::from(100))),
);
if let Some(reset_at) = reset_at {
primary.insert("reset_at".to_owned(), Value::from(reset_at));
}
rate_limit.insert("primary_window".to_owned(), Value::Object(primary));
object.insert("rate_limit".to_owned(), Value::Object(rate_limit));
if self
.store
.compare_and_swap_quota(QuotaObservation {
account_id: account.id().clone(),
expected_revision: account.revision(),
quota: Some(OpaqueProviderData::new(object)),
observed_at: Some(observed_at),
limit_reached: Some(true),
})
.await?
== QuotaWriteOutcome::Conflict
{
return Ok(());
}
// 429 合成的限流窗口要立即反映到调度信号(reset 到期由投影滚动解除)。
if let Ok(Some(snapshot)) = self.read_snapshot_for(account).await {
self.scheduling.observe(&snapshot);
}
.unwrap_or(observed_at);
self.cooldowns
.put_if_later(ProviderCooldown::new(
account.id().clone(),
account.revision(),
until,
))
.await
.map_err(|error| CodexCredentialQuotaError::Store {
detail: error.to_string(),
})?;
Ok(())
}
/// 读取账号当前是否处于临时限流(429)冷却,及到期时间。
pub async fn rate_limited_until(
&self,
account_id: &ProviderAccountId,
credential_revision: CredentialRevision,
) -> Result<Option<SystemTime>, CodexCredentialQuotaError> {
let Some(cooldown) = self.cooldowns.read(account_id).await.map_err(|error| {
CodexCredentialQuotaError::Store {
detail: error.to_string(),
}
})?
else {
return Ok(None);
};
if cooldown.credential_revision() != credential_revision {
return Ok(None);
}
let until = cooldown.until();
if until <= SystemTime::now() {
return Ok(None);
}
Ok(Some(until))
}
/// 读取单账号最后一次落库的 Provider quota,并由 Codex 域解析展示窗口。
pub async fn read_account(
&self,
@@ -843,11 +957,9 @@ impl CodexCredentialQuotaService {
});
}
Err(CodexQuotaFetchError::Upstream { account, error }) => {
// 402/deactivated → 写终态;429/503 等临时拒绝不写 availability(限流不改变账号可用性)。
// 402 与明确的停用错误写对应状态;usage 401/403 不写 availability。
if let Some(availability) = quota_state_transition(&error) {
let _ = self
.repository
.apply_state(&account, availability, observed_at)
self.persist_upstream_state(&account, availability, &error, observed_at)
.await;
}
return Err(CodexCredentialQuotaError::Upstream {
@@ -855,17 +967,40 @@ impl CodexCredentialQuotaService {
});
}
};
let snapshot = parse_account_quota_snapshot(
let refreshed_snapshot = parse_account_quota_snapshot(
account.id().clone(),
account.revision(),
observed_at,
&value,
)?;
let previous_reset_at = self.previous_confirmed_reset(&account).await;
let object = value
.as_object()
.cloned()
.ok_or(CodexCredentialQuotaError::InvalidCredentialData)?;
let object = normalize_quota_window_placeholders(
value
.as_object()
.cloned()
.ok_or(CodexCredentialQuotaError::InvalidCredentialData)?,
);
let object = if account.availability() == AccountAvailability::QuotaExhausted
&& quota_success_state(
account.availability(),
&refreshed_snapshot,
previous_reset_at,
)
.is_none()
{
confirmed_exhaustion_projection(
object,
confirmed_exhaustion_reset_at(previous_reset_at, &refreshed_snapshot),
)
} else {
object
};
let snapshot = parse_account_quota_snapshot(
account.id().clone(),
account.revision(),
observed_at,
&Value::Object(object.clone()),
)?;
if self
.store
.compare_and_swap_quota(QuotaObservation {
@@ -889,12 +1024,10 @@ impl CodexCredentialQuotaService {
return Err(CodexCredentialQuotaError::RevisionConflict);
}
self.scheduling.observe(&snapshot);
if let Some(availability) = quota_success_state(
if let Some(availability) = authoritative_quota_snapshot_state(
current.availability(),
snapshot.fact(),
&refreshed_snapshot,
previous_reset_at,
SystemTime::now(),
QuotaRecoveryEvidence::RefreshedSnapshot,
) {
self.repository
.apply_state(&current, availability, observed_at)
@@ -903,6 +1036,29 @@ impl CodexCredentialQuotaService {
Ok(snapshot)
}
async fn persist_upstream_state(
&self,
account: &ProviderAccount,
availability: AccountAvailability,
upstream_error: &CodexClientError,
observed_at: SystemTime,
) {
let reason = quota_state_reason(upstream_error).map(str::to_owned);
if let Err(error) = self
.repository
.apply_state_with_message(account, availability, observed_at, reason.clone())
.await
{
tracing::warn!(
account_id = %account.id(),
?availability,
reason = reason.as_deref().unwrap_or_default(),
error = %error,
"OpenAI quota upstream state write failed"
);
}
}
async fn fetch_usage_with_recovery(
&self,
client: &CodexBackendClient,
@@ -1004,6 +1160,15 @@ fn eligible_periodic_quota_refresh(account: &ProviderAccount, now: SystemTime) -
account.enabled() && access_token_is_current(account, now)
}
fn eligible_initial_quota_sync(account: &ProviderAccount, now: SystemTime) -> bool {
// 首次观察只兜底刚入库、尚无 quota 快照的账号。已耗尽等运行时状态必须由
// periodic 路径处理,才能保留同一账号的最小复核间隔。
matches!(
account.availability(),
AccountAvailability::Ready | AccountAvailability::Unknown
) && eligible_periodic_quota_refresh(account, now)
}
fn access_token_is_current(account: &ProviderAccount, now: SystemTime) -> bool {
account
.access_token_expires_at()
@@ -1095,6 +1260,108 @@ fn merge_passive_quota(
quota
}
/// 丢弃 core 限额中上游给出的无事实 `secondary_window` 占位。
///
/// 生产响应头可能携带 `used_percent=0`、零时长和空 reset 的占位,`/usage` 也会
/// 返回 `secondary_window: null`。正常被动同步保留该响应事实,Admin 展示会将其
/// 隐藏;但主动 `/usage` 刷新或 402 确认投影会把一个存在但无事实的字段写成
/// 100%,从而显示并不存在的“次级额度”。因此这些非被动写入口在落库前移除
/// 无事实值。带 reset、时长、正用量、触顶、未知或非法字段的次级窗口均完全按
/// 原有额度逻辑保留。
fn normalize_quota_window_placeholders(mut quota: Map<String, Value>) -> Map<String, Value> {
if let Some(rate_limit) = quota.get_mut("rate_limit").and_then(Value::as_object_mut) {
drop_secondary_window_placeholder(rate_limit);
}
quota
}
fn drop_secondary_window_placeholder(rate_limit: &mut Map<String, Value>) {
let placeholder = rate_limit.get("secondary_window").is_some_and(|window| {
window.is_null()
|| window
.as_object()
.is_some_and(secondary_window_is_placeholder)
});
if placeholder {
rate_limit.remove("secondary_window");
}
}
fn secondary_window_is_placeholder(window: &Map<String, Value>) -> bool {
window.iter().all(|(field, value)| match field.as_str() {
"used_percent" => value
.as_f64()
.is_some_and(|used_percent| used_percent.is_finite() && used_percent == 0.0),
"limit_reached" => value.as_bool() == Some(false),
_ => false,
})
}
/// 用已确认的上游额度拒绝覆盖 core 限额投影。
///
/// 上游 usage 快照可能在结算期间短暂滞后为 98/99% 或 `allowed=true`;保留其他
/// 原始 metadata,但把 core 窗口固定为 100%,直到权威快照证明进入下一窗口。
fn confirmed_exhaustion_projection(
mut quota: Map<String, Value>,
reset_at: Option<i64>,
) -> Map<String, Value> {
let mut rate_limit = quota
.remove("rate_limit")
.and_then(|value| value.as_object().cloned())
.unwrap_or_default();
rate_limit.insert("allowed".to_owned(), Value::Bool(false));
rate_limit.insert("limit_reached".to_owned(), Value::Bool(true));
for field in ["primary_window", "secondary_window"] {
let existed = rate_limit.contains_key(field);
let mut window = rate_limit
.remove(field)
.and_then(|value| value.as_object().cloned())
.unwrap_or_default();
if field == "primary_window" || existed {
window.insert("used_percent".to_owned(), Value::from(100_u64));
if field == "primary_window"
&& let Some(reset_at) = reset_at
{
window.insert("reset_at".to_owned(), Value::from(reset_at));
}
rate_limit.insert(field.to_owned(), Value::Object(window));
}
}
quota.insert("rate_limit".to_owned(), Value::Object(rate_limit));
quota
}
/// 选择应继续锁定的 reset:新鲜快照仍确认耗尽时可把锁推进到更晚窗口;
/// 否则保留已确认失败的旧窗口,避免 98/99% 快照反向解锁。
fn confirmed_exhaustion_reset_at(
previous_reset_at: Option<SystemTime>,
refreshed_snapshot: &CodexAccountQuotaSnapshot,
) -> Option<i64> {
let refreshed_reset_at = refreshed_snapshot
.core_primary_window()
.and_then(|window| window.reset_at())
.map(SystemTime::from);
let reset_at = match (previous_reset_at, refreshed_reset_at) {
(Some(previous), Some(refreshed))
if refreshed_snapshot.fact().exhausted() && refreshed > previous =>
{
refreshed
}
(Some(previous), _) => previous,
(None, refreshed) => refreshed?,
};
system_time_to_unix_seconds(reset_at)
}
fn system_time_to_unix_seconds(value: SystemTime) -> Option<i64> {
value
.duration_since(UNIX_EPOCH)
.ok()?
.as_secs()
.try_into()
.ok()
}
fn passive_rate_limit_snapshot(details: &RateLimitDetails) -> Option<Map<String, Value>> {
// 响应头的窗口属于同一次上游观测;跨响应合并会制造不存在的额度窗口。
let mut snapshot = Map::new();
@@ -1,7 +1,8 @@
//! 402 恢复证据与 availability 转换。
//! 权威额度快照的 availability 转换。
//!
//! `QuotaExhausted -> Ready` 只接受同一 credential revision 的真实成功响应、
//! 新鲜快照显式 `allowed=true` 且无 exhaustion、或已到期旧 reset 配合未耗尽新快照。
//! 已触顶的 core primary 会立即把可调度账号标为 `QuotaExhausted`;
//! `QuotaExhausted -> Ready` 则只能由同一 credential revision 的权威 usage 快照证明:
//! 新快照未耗尽、core primary reset 已推进到下一窗口,且该窗口用量低于 10%。
use std::time::SystemTime;
@@ -9,50 +10,55 @@ use gateway_core::engine::credential::AccountAvailability;
use reqwest::StatusCode;
use serde_json::Value;
use super::snapshot::CodexQuotaFact;
use super::snapshot::CodexAccountQuotaSnapshot;
use crate::transport::CodexClientError;
/// 402 恢复可接受的证据强度。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum QuotaRecoveryEvidence {
/// 一次真实成功推理响应(最强证据)。
SuccessfulResponse,
/// 周期 worker 刷新出的新快照(需旧 reset 已到期佐证)。
RefreshedSnapshot,
const RESET_RECOVERY_MAX_USED_PERCENT: f64 = 10.0;
/// 基于成功的权威 usage 快照收敛可调度账号的额度状态。
///
/// 只接受上游明确拒绝使用且 core primary 触顶的证据;secondary、附加桶、
/// credits 与 spend control 都不能单独把整个账号标成额度耗尽。
pub(crate) fn authoritative_quota_snapshot_state(
current: AccountAvailability,
snapshot: &CodexAccountQuotaSnapshot,
previous_reset_at: Option<SystemTime>,
) -> Option<AccountAvailability> {
match current {
AccountAvailability::Ready | AccountAvailability::Unknown
if snapshot.authoritative_core_primary_exhausted() =>
{
Some(AccountAvailability::QuotaExhausted)
}
AccountAvailability::QuotaExhausted => {
quota_success_state(current, snapshot, previous_reset_at)
}
AccountAvailability::Ready | AccountAvailability::Unknown => None,
AccountAvailability::Expired
| AccountAvailability::Banned
| AccountAvailability::Invalid => None,
}
}
/// 只处理 `QuotaExhausted` 的恢复——429/触顶不再进入该状态,由 quota 数据
/// 驱动调度排除(限流不改变账号可用性)。`Invalid/Expired/Banned` 等终态由 selector 的
/// 成功响应恢复路径处理。
/// 只处理 `QuotaExhausted` 的恢复。`allowed=true`、成功响应和百分比回落都不能单独覆盖
/// 一次已确认的额度拒绝;它们在 reset 未推进时可能只是上游滞后的观测。
///
/// `previous_reset_at` 是写入新快照前读取的旧确认 reset;`RefreshedSnapshot`
/// 恢复要求它已到期(窗口真实滚动后新快照的 reset 指向下一个未来周期,
/// 不能用它判定旧耗尽窗口是否已结束)。
/// `previous_reset_at` 是写入新快照前读取的 core primary 已确认耗尽窗口。上游可能只
/// 调整 reset 时间而尚未重置额度,因此还要求同一新鲜 primary 窗口的用量低于 10%。
pub(crate) fn quota_success_state(
current: AccountAvailability,
fact: CodexQuotaFact,
snapshot: &CodexAccountQuotaSnapshot,
previous_reset_at: Option<SystemTime>,
now: SystemTime,
recovery_evidence: QuotaRecoveryEvidence,
) -> Option<AccountAvailability> {
if current != AccountAvailability::QuotaExhausted || fact.exhausted() {
if current != AccountAvailability::QuotaExhausted || snapshot.fact().exhausted() {
return None;
}
// 新鲜快照显式 `allowed=true` 且无 exhaustion,直接恢复。
if fact.explicit_allowed() {
return Some(AccountAvailability::Ready);
}
// 一次真实成功响应是最强证据;周期刷新需要旧确认 reset 已到期
// (used_percent 是可滞后、会取整的观测值,其回落不能覆盖仍未到期的旧 reset)。
match recovery_evidence {
QuotaRecoveryEvidence::SuccessfulResponse => Some(AccountAvailability::Ready),
QuotaRecoveryEvidence::RefreshedSnapshot
if previous_reset_at.is_some_and(|reset_at| reset_at <= now) =>
{
Some(AccountAvailability::Ready)
}
QuotaRecoveryEvidence::RefreshedSnapshot => None,
}
let previous_reset_at = previous_reset_at?;
let primary_window = snapshot.core_primary_window()?;
let refreshed_reset_at = primary_window.reset_at().map(SystemTime::from)?;
let used_percent = primary_window.used_percent()?;
(refreshed_reset_at > previous_reset_at && used_percent < RESET_RECOVERY_MAX_USED_PERCENT)
.then_some(AccountAvailability::Ready)
}
pub(crate) fn quota_state_transition(error: &CodexClientError) -> Option<AccountAvailability> {
@@ -65,14 +71,28 @@ pub(crate) fn quota_state_transition(error: &CodexClientError) -> Option<Account
match *status {
// 402:真额度耗尽 → QuotaExhausted(worker 按 reset_at 自动恢复)。
StatusCode::PAYMENT_REQUIRED => Some(AccountAvailability::QuotaExhausted),
// 429/503:临时限流,不写 availability(限流不改变账号可用性),由 quota 数据驱动。
StatusCode::TOO_MANY_REQUESTS | StatusCode::SERVICE_UNAVAILABLE => None,
// quota/usage 接口只能产生额度域状态;鉴权拒绝不具备判定 RT
// 永久失效的证据,终态只能由 OAuth refresh/身份校验状态机写入。
// usage 401/403 不能单独判定账号不可用:保持现有状态,交给 OAuth 或真实请求确认。
// 429/503 同样不写 availability,仅记失败待重试。
StatusCode::UNAUTHORIZED
| StatusCode::FORBIDDEN
| StatusCode::TOO_MANY_REQUESTS
| StatusCode::SERVICE_UNAVAILABLE => None,
_ => None,
}
}
/// 配额接口将账号变为非 ready 状态时可安全持久化的上游原因码。
pub(crate) fn quota_state_reason(error: &CodexClientError) -> Option<&'static str> {
match quota_state_transition(error)? {
AccountAvailability::Banned => Some("deactivated_workspace"),
AccountAvailability::QuotaExhausted => Some("payment_required"),
AccountAvailability::Unknown
| AccountAvailability::Ready
| AccountAvailability::Expired
| AccountAvailability::Invalid => None,
}
}
fn is_deactivated_workspace(body: &str) -> bool {
serde_json::from_str::<Value>(body)
.ok()
@@ -13,13 +13,13 @@ use serde_json::{Map, Value};
use super::{CodexCredentialQuotaError, QUOTA_SCHEDULING_TTL};
const CORE_PRIMARY_WINDOW_KEY: &str = "core-primary";
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct CodexQuotaFact {
remaining_percent: Option<u8>,
resets_at: Option<DateTime<Utc>>,
exhausted: bool,
/// 上游显式 `allowed=true`(恢复证据:新鲜快照明确放行)。
explicit_allowed: bool,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
@@ -41,7 +41,8 @@ pub enum CodexQuotaWindowRole {
pub struct CodexQuotaWindow {
key: String,
source: String,
/// 上游提供的限流名称(`metered_feature`/`limit_name`/`limitId`);显示优先用它。
account_wide: bool,
/// 上游提供的限流桶名称(`metered_feature`/`limit_name`/`limitId`);用于非核心桶的展示前缀。
limit_name: Option<String>,
kind: CodexQuotaWindowKind,
role: CodexQuotaWindowRole,
@@ -62,6 +63,12 @@ impl CodexQuotaWindow {
&self.source
}
/// 该窗口是否覆盖账号全部 Codex 请求,可使用账号级本地用量聚合。
#[must_use]
pub const fn is_account_wide(&self) -> bool {
self.account_wide
}
#[must_use]
pub fn limit_name(&self) -> Option<&str> {
self.limit_name.as_deref()
@@ -105,6 +112,7 @@ pub struct CodexAccountQuotaSnapshot {
credential_revision: CredentialRevision,
observed_at: SystemTime,
fact: CodexQuotaFact,
authoritative_core_primary_exhausted: bool,
windows: Vec<CodexQuotaWindow>,
}
@@ -134,6 +142,19 @@ impl CodexAccountQuotaSnapshot {
&self.windows
}
#[must_use]
pub(crate) fn core_primary_window(&self) -> Option<&CodexQuotaWindow> {
self.windows
.iter()
.find(|window| window.key() == CORE_PRIMARY_WINDOW_KEY)
}
/// 上游成功 usage 明确拒绝请求,且 canonical core primary 已确认触顶。
#[must_use]
pub(crate) const fn authoritative_core_primary_exhausted(&self) -> bool {
self.authoritative_core_primary_exhausted
}
/// 投影滚动已过期窗口:`reset_at` 已过的窗口 `used_percent=0`、
/// `limit_reached=false`、`reset_at` 滚到下一个周期(reset 到期自动归零)。
/// 只作用于展示/调度投影,不写回持久化 raw JSON。
@@ -189,12 +210,6 @@ impl CodexQuotaFact {
pub const fn exhausted(&self) -> bool {
self.exhausted
}
/// 上游是否显式 `allowed=true`(新鲜快照的恢复证据)。
#[must_use]
pub const fn explicit_allowed(&self) -> bool {
self.explicit_allowed
}
}
pub fn parse_codex_quota_usage(usage: &Value) -> Result<CodexQuotaFact, CodexCredentialQuotaError> {
@@ -220,7 +235,6 @@ pub fn parse_codex_quota_usage(usage: &Value) -> Result<CodexQuotaFact, CodexCre
remaining_percent: aggregate.remaining_percent,
resets_at: aggregate.resets_at,
exhausted: aggregate.exhausted,
explicit_allowed: aggregate.explicit_allowed,
})
}
@@ -316,28 +330,90 @@ pub(crate) fn parse_account_quota_snapshot(
let object = usage
.as_object()
.ok_or(CodexCredentialQuotaError::InvalidCredentialData)?;
let authoritative_core_primary_exhausted = authoritative_core_primary_exhausted(object)?;
let mut windows = Vec::new();
for limit in canonical_rate_limits(object)? {
parse_rate_limit_windows(
&limit.key,
&limit.source,
limit.account_wide,
limit.limit_name.as_deref(),
limit.rate_limit,
&mut windows,
)?;
}
windows.sort_by(|left, right| left.key.cmp(&right.key));
windows.sort_by(|left, right| {
quota_source_order(left.source())
.cmp(&quota_source_order(right.source()))
.then_with(|| left.key.cmp(&right.key))
});
Ok(CodexAccountQuotaSnapshot {
account_id,
credential_revision,
observed_at,
fact,
authoritative_core_primary_exhausted,
windows,
})
}
/// 一个规范化后的限流桶:source 是调度/展示分组键,limit_name 是上游显示名。
fn quota_source_order(source: &str) -> u8 {
match source {
"codex" => 0,
"code_review" => 1,
_ => 2,
}
}
/// 账号状态只能由顶层 canonical Codex 额度桶的明确拒绝来改变。
///
/// `fact.exhausted()` 和窗口 `limit_reached` 都会聚合或继承 secondary、附加桶等事实,
/// 因此不能单独用来将整个账号标为 `QuotaExhausted`。
fn authoritative_core_primary_exhausted(
usage: &Map<String, Value>,
) -> Result<bool, CodexCredentialQuotaError> {
let Some(rate_limit) = usage.get("rate_limit") else {
return Ok(false);
};
if rate_limit.is_null() {
return Ok(false);
}
let rate_limit = rate_limit
.as_object()
.ok_or(CodexCredentialQuotaError::InvalidCredentialData)?;
if optional_bool(rate_limit, "allowed")? != Some(false) {
return Ok(false);
}
let Some(primary_window) = rate_limit.get("primary_window") else {
return Ok(false);
};
if primary_window.is_null() {
return Ok(false);
}
let primary_window = primary_window
.as_object()
.ok_or(CodexCredentialQuotaError::InvalidCredentialData)?;
let primary_window_reached = optional_bool(rate_limit, "limit_reached")?.unwrap_or(false)
|| optional_bool(primary_window, "limit_reached")?.unwrap_or(false)
|| primary_window
.get("used_percent")
.map(|value| {
value
.as_f64()
.filter(|value| value.is_finite() && *value >= 0.0)
.map(|value| value >= 100.0)
.ok_or(CodexCredentialQuotaError::InvalidCredentialData)
})
.transpose()?
.unwrap_or(false);
Ok(primary_window_reached)
}
/// 一个规范化后的限流桶:`key` 用于稳定区分窗口,`source` 用于展示分组。
struct CanonicalRateLimit<'a> {
key: String,
source: String,
account_wide: bool,
limit_name: Option<String>,
rate_limit: &'a Value,
}
@@ -346,9 +422,16 @@ fn canonical_rate_limits(
object: &Map<String, Value>,
) -> Result<Vec<CanonicalRateLimit<'_>>, CodexCredentialQuotaError> {
let mut limits = Vec::new();
let has_primary_codex_rate_limit = object
.get("rate_limit")
.is_some_and(|value| !value.is_null());
if let Some(rate_limit) = object.get("rate_limit") {
// 官方 Codex 将顶层 `rate_limit` 固定识别为 `codex`,而不把
// `active_limit` 投影成另一个展示桶。后者只是当前活动套餐的元数据。
limits.push(CanonicalRateLimit {
source: "core".to_owned(),
key: "core".to_owned(),
source: "codex".to_owned(),
account_wide: true,
limit_name: None,
rate_limit,
});
@@ -358,7 +441,9 @@ fn canonical_rate_limits(
.filter(|value| !value.is_null())
{
limits.push(CanonicalRateLimit {
key: "code-review".to_owned(),
source: "code_review".to_owned(),
account_wide: false,
limit_name: Some("code_review".to_owned()),
rate_limit,
});
@@ -379,23 +464,30 @@ fn canonical_rate_limits(
.as_object()
.ok_or(CodexCredentialQuotaError::InvalidCredentialData)?;
let (source, limit_name) = canonical_rate_limit_source(item, index);
// 部分上游响应会在 `additional_rate_limits` 中重复默认的 `codex`
// 桶。它没有独立的 limit id,不能在账号面板再次展示。
if has_primary_codex_rate_limit && is_codex_limit_id(&source) {
continue;
}
if let Some(rate_limit) = item.get("rate_limit") {
// additional 里的 codex 桶覆盖顶层 rate_limit(同源替换)。
if let Some(existing) = limits.iter_mut().find(|existing| existing.source == source) {
existing.rate_limit = rate_limit;
existing.limit_name = limit_name;
} else {
limits.push(CanonicalRateLimit {
source,
limit_name,
rate_limit,
});
}
// 非 Codex additional 桶保留为独立快照;额外用索引生成窗口键,
// 避免同名 additional 产生重复前端 key。
limits.push(CanonicalRateLimit {
key: format!("additional-{index}-{source}"),
account_wide: is_codex_limit_id(&source),
source,
limit_name,
rate_limit,
});
}
}
Ok(limits)
}
fn is_codex_limit_id(value: &str) -> bool {
value.trim().eq_ignore_ascii_case("codex")
}
fn canonical_rate_limit_source(
item: &Map<String, Value>,
index: usize,
@@ -417,15 +509,13 @@ fn canonical_rate_limit_source(
!value.is_empty() && value.len() <= 128 && !value.chars().any(char::is_control)
})
.map(str::to_owned);
if source.trim().to_ascii_lowercase().replace(['-', ' '], "_") == "codex" {
("core".to_owned(), limit_name)
} else {
(source, limit_name)
}
(source, limit_name)
}
fn parse_rate_limit_windows(
key_source: &str,
source: &str,
account_wide: bool,
limit_name: Option<&str>,
value: &Value,
output: &mut Vec<CodexQuotaWindow>,
@@ -482,8 +572,9 @@ fn parse_rate_limit_windows(
|| used_percent.is_some_and(|used| used >= 100.0);
let kind = quota_window_kind(window_seconds);
output.push(CodexQuotaWindow {
key: format!("{}-{}", quota_key(source), quota_role_name(role, kind)),
key: format!("{}-{}", quota_key(key_source), quota_role_name(role, kind)),
source: source.to_owned(),
account_wide,
limit_name: limit_name.map(str::to_owned),
kind,
role,
@@ -559,7 +650,6 @@ const fn quota_role_name(role: CodexQuotaWindowRole, kind: CodexQuotaWindowKind)
struct QuotaAggregate {
recognized: bool,
exhausted: bool,
explicit_allowed: bool,
remaining_percent: Option<u8>,
resets_at: Option<DateTime<Utc>>,
}
@@ -572,7 +662,6 @@ impl QuotaAggregate {
self.recognized = true;
self.exhausted |= optional_bool(object, "limit_reached")?.unwrap_or(false);
self.exhausted |= optional_bool(object, "allowed")?.is_some_and(|allowed| !allowed);
self.explicit_allowed |= optional_bool(object, "allowed")?.is_some_and(|allowed| allowed);
for key in ["primary_window", "secondary_window"] {
if let Some(window) = object.get(key) {
self.observe_window(window)?;
@@ -0,0 +1,60 @@
//! 成功取得 OAuth token 后写入独立恢复日志的记录。
//!
//! 此日志会刻意保存原始 AT/RT,避免后续数据库写入或校验失败时,已成功
//! 交换的账号无法恢复。记录独立落盘,但沿用普通结构化日志的滚动和清理策略。
const OAUTH_RECOVERY_LOG_TARGET: &str = "oauth_recovery";
const OAUTH_RECOVERY_PROVIDER: &str = "openai";
/// 产生可恢复 token 组的 OAuth 操作。
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum CodexOAuthRecoveryOperation {
/// 直接提供 AT/RT 的导入。
ImportDirect,
/// 仅提供 RT 的导入,先交换得到 AT。
ImportRefreshToken,
/// 管理员手动触发的 RT 交换。
ManualRefresh,
/// 调度器触发的 RT 交换。
ScheduledRefresh,
/// 授权码加 PKCE 的 token 交换。
AuthorizationCode,
}
impl CodexOAuthRecoveryOperation {
const fn as_str(self) -> &'static str {
match self {
Self::ImportDirect => "import_direct",
Self::ImportRefreshToken => "import_refresh_token",
Self::ManualRefresh => "manual_refresh",
Self::ScheduledRefresh => "scheduled_refresh",
Self::AuthorizationCode => "authorization_code",
}
}
}
/// 将已取得的 AT/RT 写入独立恢复结构化日志。
///
/// 调用方必须在任何可能拒绝响应的本地校验、画像补全或数据库写入之前调用。
/// 该调用不执行额外 I/O,也不返回业务错误。Host 负责恢复日志文件的分割、清理和
/// 输出;日志系统不可用时按其既有策略处理,不能阻断 OAuth 业务流程。
pub(crate) fn record_oauth_recovery(
operation: CodexOAuthRecoveryOperation,
account_id: Option<&str>,
access_token: &str,
refresh_token: Option<&str>,
) {
let has_refresh_token = refresh_token.is_some();
let refresh_token = refresh_token.unwrap_or_default();
tracing::info!(
target: OAUTH_RECOVERY_LOG_TARGET,
event = "oauth_recovery",
provider = OAUTH_RECOVERY_PROVIDER,
operation = operation.as_str(),
account_id = account_id.unwrap_or_default(),
access_token,
refresh_token,
has_refresh_token,
"OAuth recovery token record"
);
}
@@ -11,38 +11,123 @@ use gateway_core::provider_ports::{
ProviderCredentialStatePort, ProviderLeaseAcquisition, ProviderLeasePort, ProviderLeaseRequest,
ProviderRefreshCapacityRequest, ProviderRefreshLeaseRequest, ProviderRefreshPolicy,
ProviderRuntimePolicyPort, ProviderStoreError, provider_refresh_backoff_at,
provider_refresh_retry_at,
};
use secrecy::{ExposeSecret, SecretString};
use thiserror::Error;
use super::identity::{
CodexAccountIdentityVerifier, CodexIdentityExpectation, CodexIdentityVerification,
CodexIdentityVerificationError,
};
use super::recovery_log::{CodexOAuthRecoveryOperation, record_oauth_recovery};
use super::repository::{CodexCredentialRepository, CredentialRepositoryError};
use super::token_client::{RefreshFailure, TokenPair, TokenRefresher};
use super::types::{CodexCredentialPrincipal, CodexOAuthSecret, RotateCodexCredential};
use super::types::CodexOAuthSecret;
const PROVIDER_NAME: &str = "openai";
const MAX_REFRESH_BATCH: u32 = 1_000;
/// 连续失败计数窗口;每次瞬态失败刷新该 TTL,静默满窗后计数过期归零。
const REFRESH_BACKOFF_WINDOW: Duration = Duration::from_secs(30 * 60);
/// 窗口内先执行五次指数退避,再进入固定恢复周期。
const REFRESH_BACKOFF_MAX_ATTEMPTS: u32 = 5;
/// 耗尽指数退避后的 OAuth 恢复周期。
const REFRESH_RECOVERY_DELAY: Duration = Duration::from_secs(10 * 60);
/// 过期 AT 仍允许 RT 恢复的最长窗口。
const REFRESH_RECOVERY_WINDOW: Duration = Duration::from_secs(24 * 60 * 60);
fn refresh_due_at(account: &ProviderAccount, now: SystemTime) -> Option<SystemTime> {
if let Some(expires_at) = account.access_token_expires_at()
&& expires_at <= now
pub(crate) fn refresh_recovery_deadline(
access_token_expires_at: Option<SystemTime>,
) -> Option<SystemTime> {
access_token_expires_at.and_then(|expires_at| expires_at.checked_add(REFRESH_RECOVERY_WINDOW))
}
fn refresh_recovery_window_exhausted(
access_token_expires_at: Option<SystemTime>,
now: SystemTime,
) -> bool {
refresh_recovery_deadline(access_token_expires_at).is_some_and(|deadline| deadline <= now)
}
fn bounded_refresh_retry_at(
retry_at: SystemTime,
access_token_expires_at: Option<SystemTime>,
observed_at: SystemTime,
) -> SystemTime {
let mut bounded = retry_at;
if let Some(expires_at) = access_token_expires_at
&& expires_at > observed_at
&& bounded > expires_at
{
// AT 已过期时不能继续等待持久退避;后续额度/请求探测都必须先用 RT 续出可用 AT。
return Some(expires_at);
bounded = expires_at;
}
account.next_refresh_at().filter(|next| *next <= now)
if let Some(deadline) = refresh_recovery_deadline(access_token_expires_at)
&& bounded > deadline
{
bounded = deadline;
}
bounded
}
fn oauth_refresh_retry_at(
account_id: &ProviderAccountId,
access_token_expires_at: Option<SystemTime>,
observed_at: SystemTime,
attempt: u32,
reason: &'static str,
) -> Result<SystemTime, ProviderStoreError> {
let retry_at = if attempt <= REFRESH_BACKOFF_MAX_ATTEMPTS {
provider_refresh_backoff_at(account_id, observed_at, attempt, reason)?
} else {
provider_refresh_retry_at(account_id, observed_at, REFRESH_RECOVERY_DELAY, reason)?
};
Ok(bounded_refresh_retry_at(
retry_at,
access_token_expires_at,
observed_at,
))
}
fn log_refresh_deferred(
account_id: &ProviderAccountId,
access_token_expires_at: Option<SystemTime>,
attempt: u32,
reason: &'static str,
retry_at: SystemTime,
) {
tracing::warn!(
account_id = %account_id,
attempt,
reason,
retry_at = %DateTime::<Utc>::from(retry_at),
access_token_expires_at = ?access_token_expires_at.map(DateTime::<Utc>::from),
recovery_deadline = ?refresh_recovery_deadline(access_token_expires_at)
.map(DateTime::<Utc>::from),
"OpenAI OAuth refresh deferred"
);
}
fn refresh_due_at(
account: &ProviderAccount,
policy: ProviderRefreshPolicy,
now: SystemTime,
) -> Option<SystemTime> {
if refresh_recovery_window_exhausted(account.access_token_expires_at(), now) {
// 即使 retry-not-before 异常地落在 deadline 之后,也必须进入终态化路径。
return refresh_recovery_deadline(account.access_token_expires_at());
}
let access_token_expires_at = account.access_token_expires_at()?;
if !policy.is_refresh_due(access_token_expires_at, now) {
return None;
}
if account
.next_refresh_at()
.is_some_and(|retry_at| retry_at > now)
{
return None;
}
Some(access_token_expires_at)
}
pub struct DueCodexCredential {
pub account: ProviderAccount,
pub secret: CodexOAuthSecret,
principal: CodexCredentialPrincipal,
installation_id: String,
}
impl std::fmt::Debug for DueCodexCredential {
@@ -94,7 +179,6 @@ pub enum CodexCredentialRefreshError {
pub struct CodexCredentialRefreshService {
repository: CodexCredentialRepository,
refresher: Arc<dyn TokenRefresher>,
identity: Arc<dyn CodexAccountIdentityVerifier>,
leases: Arc<dyn ProviderLeasePort>,
credential_state: Arc<dyn ProviderCredentialStatePort>,
runtime_policy: Arc<dyn ProviderRuntimePolicyPort>,
@@ -104,7 +188,6 @@ impl CodexCredentialRefreshService {
pub fn new(
repository: CodexCredentialRepository,
refresher: Arc<dyn TokenRefresher>,
identity: Arc<dyn CodexAccountIdentityVerifier>,
leases: Arc<dyn ProviderLeasePort>,
credential_state: Arc<dyn ProviderCredentialStatePort>,
runtime_policy: Arc<dyn ProviderRuntimePolicyPort>,
@@ -112,7 +195,6 @@ impl CodexCredentialRefreshService {
Self {
repository,
refresher,
identity,
leases,
credential_state,
runtime_policy,
@@ -130,7 +212,7 @@ impl CodexCredentialRefreshService {
excluded: &BTreeSet<ProviderAccountId>,
) -> Result<Vec<CodexCredentialRefreshOutcome>, CodexCredentialRefreshError> {
let policy = self.runtime_policy.load_refresh_policy().await?;
let (due, mut outcomes) = self.list_due_refresh(excluded).await?;
let (due, mut outcomes) = self.list_due_refresh(excluded, policy).await?;
outcomes.reserve(due.len());
let refreshed = stream::iter(due.into_iter().map(|credential| async move {
let account_id = credential.account.id().to_string();
@@ -145,7 +227,14 @@ impl CodexCredentialRefreshService {
for (account_id, result) in refreshed {
match result {
Ok(outcome) => outcomes.push(outcome),
Err(_) => outcomes.push(CodexCredentialRefreshOutcome::Failed { account_id }),
Err(error) => {
tracing::warn!(
account_id = %account_id,
error = %error,
"OpenAI OAuth refresh attempt failed"
);
outcomes.push(CodexCredentialRefreshOutcome::Failed { account_id });
}
}
}
Ok(outcomes)
@@ -157,6 +246,19 @@ impl CodexCredentialRefreshService {
policy: ProviderRefreshPolicy,
) -> Result<CodexCredentialRefreshOutcome, CodexCredentialRefreshError> {
let account_id = due.account.id().to_string();
if refresh_recovery_window_exhausted(
due.account.access_token_expires_at(),
SystemTime::now(),
) {
return self
.persist_terminal(
&due.account,
AccountAvailability::Expired,
"refresh_recovery_window_exhausted",
CodexCredentialRefreshOutcome::Invalidated { account_id },
)
.await;
}
let capacity = self
.leases
.try_acquire(ProviderLeaseRequest::RefreshCapacity(
@@ -187,11 +289,12 @@ impl CodexCredentialRefreshService {
.as_ref()
.ok_or(CodexCredentialRefreshError::InvalidRefreshResponse)?;
match self.refresher.refresh(refresh_token.expose_secret()).await {
Ok(tokens) => self.persist_success(due, tokens, policy).await,
Ok(tokens) => self.persist_success(due, tokens).await,
Err(RefreshFailure::InvalidGrant) => {
self.persist_terminal(
&due.account,
AccountAvailability::Expired,
"refresh_invalid_grant",
CodexCredentialRefreshOutcome::Invalidated { account_id },
)
.await
@@ -200,6 +303,7 @@ impl CodexCredentialRefreshService {
self.persist_terminal(
&due.account,
AccountAvailability::Banned,
"account_banned",
CodexCredentialRefreshOutcome::Banned { account_id },
)
.await
@@ -232,6 +336,7 @@ impl CodexCredentialRefreshService {
async fn list_due_refresh(
&self,
excluded: &BTreeSet<ProviderAccountId>,
policy: ProviderRefreshPolicy,
) -> Result<
(Vec<DueCodexCredential>, Vec<CodexCredentialRefreshOutcome>),
CodexCredentialRefreshError,
@@ -254,9 +359,10 @@ impl CodexCredentialRefreshService {
| AccountAvailability::Banned
| AccountAvailability::Invalid
)
&& refresh_due_at(account, now).is_some()
&& refresh_due_at(account, policy, now).is_some()
});
accounts.sort_by_key(|account| (refresh_due_at(account, now), account.id().clone()));
accounts
.sort_by_key(|account| (refresh_due_at(account, policy, now), account.id().clone()));
accounts.truncate(MAX_REFRESH_BATCH as usize);
let mut due = Vec::with_capacity(accounts.len());
let mut failures = Vec::new();
@@ -272,15 +378,9 @@ impl CodexCredentialRefreshService {
let Some(secret) = runtime.authentication.oauth() else {
unreachable!("OAuth runtime authentication was checked above")
};
let Some(principal) = runtime.principal else {
failures.push(CodexCredentialRefreshOutcome::Failed { account_id });
continue;
};
due.push(DueCodexCredential {
account,
secret: secret.clone(),
principal,
installation_id: runtime.installation_id,
});
}
Ok(_) | Err(_) => {
@@ -295,115 +395,42 @@ impl CodexCredentialRefreshService {
&self,
due: DueCodexCredential,
tokens: TokenPair,
policy: ProviderRefreshPolicy,
) -> Result<CodexCredentialRefreshOutcome, CodexCredentialRefreshError> {
if tokens.access_token.is_empty() || tokens.expires_in.is_zero() {
return Err(CodexCredentialRefreshError::InvalidRefreshResponse);
}
let refresh_token = tokens
.refresh_token
let TokenPair {
access_token,
refresh_token: rotated_refresh_token,
expires_in,
} = tokens;
let refresh_token = rotated_refresh_token
.map(SecretString::from)
.or(due.secret.refresh_token);
let account_id = due
.account
.upstream_account_id()
.ok_or(CodexCredentialRefreshError::InvalidRefreshResponse)?;
let expectation = CodexIdentityExpectation::current(
due.principal.oauth_subject,
due.principal.poid,
account_id.to_owned(),
due.account.upstream_user_id().to_owned(),
due.installation_id,
)
.map_err(|_| CodexCredentialRefreshError::InvalidRefreshResponse)?;
let secret = CodexOAuthSecret {
access_token: SecretString::from(tokens.access_token),
access_token: SecretString::from(access_token),
refresh_token,
id_token: due.secret.id_token,
};
let verification = match self.identity.verify(&secret, &expectation).await {
Ok(verification) => verification,
Err(CodexIdentityVerificationError::Rejected) => {
return self
.persist_terminal(
&due.account,
AccountAvailability::Invalid,
CodexCredentialRefreshOutcome::Invalidated {
account_id: due.account.id().to_string(),
},
)
.await;
}
Err(CodexIdentityVerificationError::Unavailable) => {
// JWKS/签名边界短暂不可用属瞬态:保留现有凭据、推进退避重试,
// 不终态失效账号。
return if self
.defer_refresh(&due.account, "identity-unavailable")
.await?
{
Ok(CodexCredentialRefreshOutcome::Transient {
account_id: due.account.id().to_string(),
})
} else {
Ok(CodexCredentialRefreshOutcome::Stale {
account_id: due.account.id().to_string(),
})
};
}
};
let profile = match verification {
CodexIdentityVerification::Complete(profile) => profile,
CodexIdentityVerification::SignedOnly(signed) => {
let attempt = self
.credential_state
.record_refresh_backoff(due.account.id(), REFRESH_BACKOFF_WINDOW)
.await
.unwrap_or(1);
let retry_at = provider_refresh_backoff_at(
due.account.id(),
SystemTime::now(),
attempt,
"identity-completion",
)?;
match self
.repository
.rotate_signed_secret(&due.account, secret, &signed, retry_at)
.await
{
Ok(_) => {
return Ok(CodexCredentialRefreshOutcome::Transient {
account_id: due.account.id().to_string(),
});
}
Err(CredentialRepositoryError::RevisionConflict) => {
return Ok(CodexCredentialRefreshOutcome::Stale {
account_id: due.account.id().to_string(),
});
}
Err(error) => return Err(error.into()),
}
}
};
let next_refresh_at = policy
.next_attempt_at(
due.account.id(),
profile
.access_token_expires_at
.map(SystemTime::from)
.ok_or(CodexCredentialRefreshError::InvalidRefreshResponse)?,
SystemTime::now(),
)
.map(DateTime::<Utc>::from)
.map_err(CodexCredentialRefreshError::from)?;
record_oauth_recovery(
CodexOAuthRecoveryOperation::ScheduledRefresh,
Some(due.account.id().as_str()),
secret.access_token.expose_secret(),
secret
.refresh_token
.as_ref()
.map(ExposeSecret::expose_secret),
);
let observed_at = SystemTime::now();
let access_token_expires_at = expires_in.and_then(|value| observed_at.checked_add(value));
// 成功轮换清掉仅用于失败退避的 retry-not-before;正常预刷新窗口由
// worker 结合当前 runtime policy 动态判断,不写入账号时间字段。
let next_refresh_at = None;
let result = self
.repository
.rotate_oauth_secret(RotateCodexCredential {
account_id: due.account.id().to_string(),
expected_credential_revision: due.account.revision().get(),
.rotate_refreshed_oauth_secret(
&due.account,
secret,
verified_account: profile,
next_refresh_at: Some(next_refresh_at),
})
access_token_expires_at,
next_refresh_at,
)
.await;
match result {
Ok(revision) => {
@@ -412,6 +439,13 @@ impl CodexCredentialRefreshService {
.credential_state
.clear_refresh_backoff(due.account.id())
.await;
tracing::info!(
account_id = %due.account.id(),
credential_revision = revision.get(),
access_token_expires_at = ?access_token_expires_at.map(DateTime::<Utc>::from),
refresh_retry_not_before = ?next_refresh_at.map(DateTime::<Utc>::from),
"OpenAI OAuth refresh succeeded"
);
Ok(CodexCredentialRefreshOutcome::Refreshed {
account_id: due.account.id().to_string(),
credential_revision: revision.get(),
@@ -430,6 +464,7 @@ impl CodexCredentialRefreshService {
&self,
account: &ProviderAccount,
availability: AccountAvailability,
reason: &'static str,
outcome: CodexCredentialRefreshOutcome,
) -> Result<CodexCredentialRefreshOutcome, CodexCredentialRefreshError> {
match self.repository.load_runtime_credential(account).await {
@@ -443,10 +478,27 @@ impl CodexCredentialRefreshService {
}
match self
.repository
.apply_state(account, availability, SystemTime::now())
.apply_state_with_message(
account,
availability,
SystemTime::now(),
Some(reason.to_owned()),
)
.await
{
Ok(()) => Ok(outcome),
Ok(()) => {
tracing::warn!(
account_id = %account.id(),
?availability,
reason,
access_token_expires_at = ?account.access_token_expires_at()
.map(DateTime::<Utc>::from),
recovery_deadline = ?refresh_recovery_deadline(account.access_token_expires_at())
.map(DateTime::<Utc>::from),
"OpenAI OAuth refresh marked account terminal"
);
Ok(outcome)
}
Err(CredentialRepositoryError::RevisionConflict) => {
Ok(CodexCredentialRefreshOutcome::Stale {
account_id: account.id().to_string(),
@@ -466,10 +518,24 @@ impl CodexCredentialRefreshService {
.record_refresh_backoff(account.id(), REFRESH_BACKOFF_WINDOW)
.await
.unwrap_or(1);
let retry_at =
provider_refresh_backoff_at(account.id(), SystemTime::now(), attempt, reason)?;
let retry_at = oauth_refresh_retry_at(
account.id(),
account.access_token_expires_at(),
SystemTime::now(),
attempt,
reason,
)?;
match self.repository.defer_refresh(account, retry_at).await {
Ok(_) => Ok(true),
Ok(_) => {
log_refresh_deferred(
account.id(),
account.access_token_expires_at(),
attempt,
reason,
retry_at,
);
Ok(true)
}
Err(CredentialRepositoryError::RevisionConflict) => Ok(false),
Err(error) => Err(error.into()),
}
@@ -6,18 +6,15 @@ use std::time::SystemTime;
use chrono::{DateTime, Utc};
use gateway_core::engine::credential::{
AccountAvailability, AccountStateChange, CredentialCasOutcome, CredentialCasUpdate,
CredentialRevision, LoadedCredential, ProviderAccount, ProviderAccountId, ProviderAccountStore,
CredentialRevision, ProviderAccount, ProviderAccountId, ProviderAccountStore,
ProviderAccountUpdate,
};
use gateway_core::routing::ProviderKind;
use secrecy::ExposeSecret;
use thiserror::Error;
use super::identity::CodexSignedIdentity;
use super::security::{CodexCredentialCodec, CodexCredentialDataError, CodexRuntimeCredential};
use super::types::{
CodexAccountProfile, CodexCredentialData, CodexOAuthSecret, RotateCodexCredential,
};
use super::types::{CodexCredentialData, CodexOAuthSecret, RotateCodexCredential};
const PROVIDER_NAME: &str = "openai";
@@ -47,7 +44,6 @@ impl CodexCredentialRepository {
.map_err(|_| CredentialRepositoryError::InvalidInput("credential_revision"))?;
let current = self.store.load_credential(&account_id, expected).await?;
let mut data = CodexCredentialCodec::decode_complete(&current.credential)?;
verify_identity(&current, &data, &input.verified_account)?;
let oauth = data
.oauth_mut()
.ok_or(CredentialRepositoryError::InvalidCredentialData)?;
@@ -83,13 +79,16 @@ impl CodexCredentialRepository {
cas_revision(self.store.compare_and_swap_credential(update).await?)
}
/// usage 暂时不可用时,只在签名 principal 与现有绑定完全一致后保存已轮换 token。
pub async fn rotate_signed_secret(
/// 持久化成功 RT exchange 的 token,同时保留既有账号身份投影。
///
/// Refresh endpoint 已经是这次 AT/RT 轮换的授权边界;这里仅以 revision
/// CAS 保护并发写入,不重新验证新 access token 的身份声明。
pub async fn rotate_refreshed_oauth_secret(
&self,
account: &ProviderAccount,
secret: CodexOAuthSecret,
signed: &CodexSignedIdentity,
next_refresh_at: SystemTime,
access_token_expires_at: Option<SystemTime>,
next_refresh_at: Option<SystemTime>,
) -> Result<CredentialRevision, CredentialRepositoryError> {
let current = self
.store
@@ -102,17 +101,6 @@ impl CodexCredentialRepository {
let oauth = data
.oauth_mut()
.ok_or(CredentialRepositoryError::InvalidCredentialData)?;
if oauth.principal.oauth_subject != signed.oauth_subject()
|| oauth.principal.poid.as_deref() != signed.poid()
|| signed
.claimed_account_id()
.is_some_and(|value| account.upstream_account_id() != Some(value))
|| signed
.claimed_user_id()
.is_some_and(|value| account.upstream_user_id() != value)
{
return Err(CredentialRepositoryError::IdentityMismatch);
}
oauth.access_token = secret.access_token.expose_secret().to_owned();
oauth.refresh_token = secret
.refresh_token
@@ -129,8 +117,8 @@ impl CodexCredentialRepository {
unchanged_profile(account),
credential,
secret.refresh_token.is_some(),
Some(SystemTime::from(signed.access_token_expires_at())),
Some(next_refresh_at),
access_token_expires_at,
next_refresh_at,
)
.map_err(|_| CredentialRepositoryError::InvalidCredentialData)?;
cas_revision(self.store.compare_and_swap_credential(update).await?)
@@ -234,16 +222,33 @@ impl CodexCredentialRepository {
account: &ProviderAccount,
availability: AccountAvailability,
observed_at: SystemTime,
) -> Result<(), CredentialRepositoryError> {
self.apply_state_with_message(account, availability, observed_at, None)
.await
}
/// 带上游错误描述的状态写入;上游 message 优先,终态缺失 message 时写稳定兜底,
/// 恢复 `Ready` 时清空。
pub async fn apply_state_with_message(
&self,
account: &ProviderAccount,
availability: AccountAvailability,
observed_at: SystemTime,
message: Option<String>,
) -> Result<(), CredentialRepositoryError> {
if !account.enabled() {
return Ok(());
}
let message = message
.filter(|value| !value.trim().is_empty())
.or_else(|| availability.fallback_error_reason().map(str::to_owned));
self.store
.apply_state_change(AccountStateChange {
account_id: account.id().clone(),
expected_revision: account.revision(),
availability,
observed_at,
message,
})
.await?;
Ok(())
@@ -268,24 +273,6 @@ fn cas_revision(
}
}
fn verify_identity(
current: &LoadedCredential,
credential: &CodexCredentialData,
verified: &CodexAccountProfile,
) -> Result<(), CredentialRepositoryError> {
if current.account.provider().as_str() != PROVIDER_NAME
|| current.account.upstream_account_id() != Some(verified.chatgpt_account_id.as_str())
|| verified.chatgpt_user_id != current.account.upstream_user_id()
|| credential.oauth().is_none_or(|credential| {
verified.oauth_subject != credential.principal.oauth_subject
|| verified.poid != credential.principal.poid
})
{
return Err(CredentialRepositoryError::IdentityMismatch);
}
Ok(())
}
fn required_time(value: Option<DateTime<Utc>>) -> Result<SystemTime, CredentialRepositoryError> {
value
.map(SystemTime::from)
@@ -302,8 +289,6 @@ pub enum CredentialRepositoryError {
InvalidInput(&'static str),
#[error("Codex credential data is invalid")]
InvalidCredentialData,
#[error("Codex credential identity does not match")]
IdentityMismatch,
#[error("Codex credential revision conflict")]
RevisionConflict,
#[error("provider account store is unavailable")]
@@ -102,12 +102,32 @@ impl CodexCredentialCodec {
account: &CodexAccountProfile,
cookies: Vec<CodexCookie>,
) -> Result<PlaintextCredential, CodexCredentialDataError> {
Self::encode_complete(CodexCredentialData::OAuth(CodexOAuthCredentialData {
schema_version: CODEX_CREDENTIAL_SCHEMA_VERSION,
principal: CodexCredentialPrincipal {
Self::encode_oauth(
secret,
Some(CodexCredentialPrincipal {
oauth_subject: account.oauth_subject.clone(),
poid: account.poid.clone(),
},
}),
cookies,
)
}
/// 为尚未解析资料的 OAuth 账号编码凭据。
pub(crate) fn encode_unresolved(
secret: &CodexOAuthSecret,
cookies: Vec<CodexCookie>,
) -> Result<PlaintextCredential, CodexCredentialDataError> {
Self::encode_oauth(secret, None, cookies)
}
fn encode_oauth(
secret: &CodexOAuthSecret,
principal: Option<CodexCredentialPrincipal>,
cookies: Vec<CodexCookie>,
) -> Result<PlaintextCredential, CodexCredentialDataError> {
Self::encode_complete(CodexCredentialData::OAuth(CodexOAuthCredentialData {
schema_version: CODEX_CREDENTIAL_SCHEMA_VERSION,
principal,
installation_id: uuid::Uuid::new_v4().to_string(),
access_token: secret.access_token.expose_secret().to_owned(),
refresh_token: secret
@@ -171,7 +191,7 @@ impl CodexCredentialCodec {
refresh_token: data.refresh_token.map(SecretString::from),
id_token: data.id_token.map(SecretString::from),
}),
Some(data.principal),
data.principal,
data.installation_id,
data.cookies,
data.oauth_client_id,
@@ -232,9 +252,6 @@ impl CodexCredentialCodec {
let existing = Self::decode_complete(existing)?;
match (&mut incoming, existing) {
(CodexCredentialData::OAuth(incoming), CodexCredentialData::OAuth(existing)) => {
if incoming.principal != existing.principal {
return Err(CodexCredentialDataError::Invalid);
}
incoming.installation_id = existing.installation_id;
}
(
@@ -260,23 +277,7 @@ impl CodexCredentialCodec {
fn validate(data: &CodexCredentialData) -> Result<(), CodexCredentialDataError> {
let (installation_id, cookies) = match data {
CodexCredentialData::OAuth(data) => {
if data.schema_version != CODEX_CREDENTIAL_SCHEMA_VERSION
|| !valid_identity(&data.principal.oauth_subject)
|| data
.principal
.poid
.as_deref()
.is_some_and(|value| !valid_identity(value))
|| !valid_secret(&data.access_token)
|| data
.refresh_token
.as_deref()
.is_some_and(|value| !valid_secret(value))
|| data
.id_token
.as_deref()
.is_some_and(|value| !valid_secret(value))
{
if data.schema_version != CODEX_CREDENTIAL_SCHEMA_VERSION {
return Err(CodexCredentialDataError::Invalid);
}
(&data.installation_id, &data.cookies)
@@ -25,11 +25,12 @@ use super::agent_identity::CodexAgentIdentityTaskService;
use super::catalog::CodexCredentialCatalogService;
use super::cookie::CodexCookiePolicy;
use super::quota::CodexCredentialQuotaService;
use super::refresh::refresh_recovery_deadline;
use super::repository::{CodexCredentialRepository, CredentialRepositoryError};
use super::security::CodexRuntimeAuthentication;
use super::types::{
CODEX_AUTHENTICATION_KIND_AGENT_IDENTITY, CodexCookie, CodexCookieCaptureOutcome,
RuntimeCodexCookie,
CODEX_AUTHENTICATION_KIND_AGENT_IDENTITY, CODEX_AUTHENTICATION_KIND_OAUTH, CodexCookie,
CodexCookieCaptureOutcome, RuntimeCodexCookie,
};
const CLOUDFLARE_RECOVERY_STALE_AFTER: Duration = Duration::from_secs(60 * 60);
@@ -73,6 +74,21 @@ pub enum CodexAccountFailure {
CloudflarePathBlocked,
}
impl CodexAccountFailure {
/// 上游未提供可展示错误文本时,保留已分类的稳定原因。
const fn fallback_error_reason(self) -> Option<&'static str> {
match self {
Self::CredentialExpired => Some("upstream_credential_expired"),
Self::IdentityVerificationRequired => Some("upstream_identity_verification_required"),
Self::Banned => Some("upstream_account_banned"),
Self::QuotaExhausted => Some("upstream_quota_exhausted"),
Self::UsageLimitExhausted { .. } => Some("upstream_usage_limit_exhausted"),
Self::CloudflarePathBlocked => Some("upstream_cloudflare_path_blocked"),
Self::RateLimited { .. } | Self::CloudflareChallenge { .. } => None,
}
}
}
#[derive(Debug, Clone, Copy)]
struct RiskRecoveryState {
challenge_count: u32,
@@ -294,6 +310,24 @@ impl CodexCredentialSelector {
if !diagnostic {
self.quota.prepare_scheduling(&accounts).await;
}
// 429 临时限流(Redis 冷却)中的账号即使额度正常也不参与调度,
// 到期后冷却过期自动恢复。诊断请求(连接测试)绕过冷却,需要真实探测。
let accounts = if diagnostic {
accounts
} else {
let mut retained = Vec::with_capacity(accounts.len());
for account in accounts {
let cooling = self
.quota
.rate_limited_until(account.id(), account.revision())
.await
.unwrap_or(None);
if cooling.is_none() {
retained.push(account);
}
}
retained
};
let account_ids = accounts
.iter()
.map(|account| account.id().clone())
@@ -327,46 +361,6 @@ impl CodexCredentialSelector {
AccountCandidate { account, signals }
})
.collect::<Vec<_>>();
// skip_exhausted 语义:true 时排除任一窗口触顶(limit_reached)的账号,
// 仅由上游 429 兜底;false 时保留全部候选(含 QuotaExhausted 的放宽投影)。
// 保留原始账号供 revision-fenced 凭据加载,用影子候选承载资格放宽。
let scheduling_candidates = if self.skip_exhausted {
candidates
.iter()
.filter(|candidate| !candidate.signals.quota_limit_reached)
.cloned()
.collect::<Vec<_>>()
} else {
candidates
.iter()
.cloned()
.map(|mut candidate| {
candidate.account = quota_exhausted_scheduling_projection(&candidate.account);
candidate
})
.collect::<Vec<_>>()
};
let scheduling_candidates = scheduling_candidates.as_slice();
let mut affinity = if diagnostic {
AffinitySelection::default()
} else {
self.resolve_session_affinity(
request.session_affinity_key,
&candidates,
SystemTime::now(),
)
.await
};
let cyber_policy_scope = self
.prepare_cyber_policy_scope(cyber_policy_session_key)
.await;
let mut excluded = request.attempt.excluded_accounts().clone();
if let Some(state) = cyber_policy_scope
.as_ref()
.and_then(|scope| scope.state.as_ref())
{
excluded.extend(state.excluded_accounts().iter().cloned());
}
let continuation_account = match request.attempt.continuation_attempt() {
ContinuationAttempt::Native => request
.attempt
@@ -388,7 +382,74 @@ impl CodexCredentialSelector {
{
return Err(CredentialSelectionError::NoEligibleCredential);
}
let pinned_account = required_account.or(continuation_account);
let pinned_account = required_account.or_else(|| continuation_account.clone());
let preserve_native_continuation_state = matches!(
request.attempt.continuation_attempt(),
ContinuationAttempt::Native
) && continuation_account.is_some();
// skip_exhausted 语义:true 时排除任一窗口触顶(limit_reached)的账号,
// 仅由上游 429 兜底;false 时保留全部候选(含 QuotaExhausted 的放宽投影)。
// 保留原始账号供 revision-fenced 凭据加载,用影子候选承载资格放宽。
let scheduling_candidates = if diagnostic {
// 连接测试必须实际探测指定账号;额度快照只影响常规调度。
candidates.clone()
} else if self.skip_exhausted {
candidates
.iter()
.filter(|candidate| !candidate.signals.quota_limit_reached)
.cloned()
.collect::<Vec<_>>()
} else {
candidates
.iter()
.cloned()
.map(|mut candidate| {
// 已命中 previous_response_id 的 native continuation 先检查原账号。
// 即使常规调度允许探测额度耗尽账号,也必须保留原账号真实状态,
// 由 Coordinator 据此切换到可携带重放,而非伪装为 Ready。
if !preserve_native_continuation_state
|| pinned_account.as_ref() != Some(candidate.account.id())
{
candidate.account =
quota_exhausted_scheduling_projection(&candidate.account);
}
candidate
})
.collect::<Vec<_>>()
};
let scheduling_candidates = scheduling_candidates.as_slice();
let mut affinity = if diagnostic {
AffinitySelection::default()
} else {
self.resolve_session_affinity(
request.session_affinity_key,
&candidates,
SystemTime::now(),
)
.await
};
let cyber_policy_scope = self
.prepare_cyber_policy_scope(cyber_policy_session_key)
.await;
let mut excluded = request.attempt.excluded_accounts().clone();
if !diagnostic
&& preserve_native_continuation_state
&& let Some(pinned) = pinned_account.as_ref()
&& candidates.iter().any(|candidate| {
candidate.account.id() == pinned && candidate.signals.quota_limit_reached
})
{
// `skip_exhausted = false` 只允许普通新请求探测可疑账号;已绑定的
// native response handle 先将真实触顶状态交给 Coordinator,随后才可
// 通过 ReplayAny 按当前调度策略换号。
excluded.insert(pinned.clone());
}
if let Some(state) = cyber_policy_scope
.as_ref()
.and_then(|scope| scope.state.as_ref())
{
excluded.extend(state.excluded_accounts().iter().cloned());
}
if let Some(required) = pinned_account.as_ref() {
for candidate in &candidates {
if candidate.account.id() != required {
@@ -685,26 +746,53 @@ impl CodexCredentialSelector {
&self,
account: &ProviderAccount,
failure: CodexAccountFailure,
message: Option<String>,
) -> Result<(), CredentialSelectionError> {
let now = SystemTime::now();
let message = message
.filter(|value| !value.trim().is_empty())
.or_else(|| failure.fallback_error_reason().map(str::to_owned));
match failure {
CodexAccountFailure::CredentialExpired => {
self.apply_state(account, AccountAvailability::Expired, now)
if account.authentication_kind() == CODEX_AUTHENTICATION_KIND_OAUTH
&& account.has_refresh_token()
&& account
.access_token_expires_at()
.is_some_and(|expires_at| expires_at <= now)
&& refresh_recovery_deadline(account.access_token_expires_at())
.is_some_and(|deadline| deadline > now)
{
tracing::info!(
account_id = %account.id(),
access_token_expires_at = ?account.access_token_expires_at()
.map(chrono::DateTime::<chrono::Utc>::from),
recovery_deadline = ?refresh_recovery_deadline(account.access_token_expires_at())
.map(chrono::DateTime::<chrono::Utc>::from),
"OpenAI access token expired; retaining account for bounded OAuth refresh recovery"
);
return Ok(());
}
self.apply_state_with_message(account, AccountAvailability::Expired, now, message)
.await
}
CodexAccountFailure::IdentityVerificationRequired => {
self.apply_state(account, AccountAvailability::Invalid, now)
self.apply_state_with_message(account, AccountAvailability::Invalid, now, message)
.await
}
CodexAccountFailure::Banned => {
self.apply_state(account, AccountAvailability::Banned, now)
self.apply_state_with_message(account, AccountAvailability::Banned, now, message)
.await
}
// 402:真额度耗尽,由 quota worker 按窗口 reset 自动恢复。
CodexAccountFailure::QuotaExhausted
| CodexAccountFailure::UsageLimitExhausted { .. } => {
self.apply_state(account, AccountAvailability::QuotaExhausted, now)
.await
self.apply_state_with_message(
account,
AccountAvailability::QuotaExhausted,
now,
message,
)
.await
}
// 429:临时限流只写 quota,availability 不改变。
CodexAccountFailure::RateLimited { retry_after } => {
@@ -727,8 +815,13 @@ impl CodexCredentialSelector {
CodexAccountFailure::CloudflarePathBlocked => {
let blocked = self.record_cloudflare_path_block(account.id(), now);
if blocked >= CLOUDFLARE_PATH_BLOCK_THRESHOLD {
self.apply_state(account, AccountAvailability::Invalid, now)
.await?;
self.apply_state_with_message(
account,
AccountAvailability::Invalid,
now,
message,
)
.await?;
}
self.apply_cookie_recovery(account, CookieRecovery::Clear)
.await?;
@@ -737,14 +830,15 @@ impl CodexCredentialSelector {
}
}
async fn apply_state(
async fn apply_state_with_message(
&self,
account: &ProviderAccount,
availability: AccountAvailability,
observed_at: SystemTime,
message: Option<String>,
) -> Result<(), CredentialSelectionError> {
self.repository
.apply_state(account, availability, observed_at)
.apply_state_with_message(account, availability, observed_at, message)
.await?;
Ok(())
}
@@ -1108,8 +1202,7 @@ impl From<CredentialRepositoryError> for CredentialSelectionError {
fn from(error: CredentialRepositoryError) -> Self {
match error {
CredentialRepositoryError::InvalidInput(_)
| CredentialRepositoryError::InvalidCredentialData
| CredentialRepositoryError::IdentityMismatch => Self::InvalidCredential,
| CredentialRepositoryError::InvalidCredentialData => Self::InvalidCredential,
CredentialRepositoryError::RevisionConflict | CredentialRepositoryError::Store => {
Self::Store
}
@@ -7,7 +7,6 @@ use serde::Deserialize;
use std::fmt;
use std::time::Duration;
const MAX_TOKEN_LIFETIME_SECONDS: u64 = 366 * 24 * 60 * 60;
const MAX_OAUTH_RESPONSE_BYTES: usize = 64 * 1024;
const TOKEN_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
const TOKEN_CONNECT_TIMEOUT: Duration = Duration::from_secs(10);
@@ -24,7 +23,9 @@ pub const OFFICIAL_CODEX_REDIRECT_URI: &str = "http://localhost:1455/auth/callba
pub struct TokenPair {
pub access_token: String,
pub refresh_token: Option<String>,
pub expires_in: Duration,
/// OAuth endpoint 可以省略或返回零 expiry;这不影响已经成功的 token
/// exchange,只意味着调用方不能据此安排下一次刷新。
pub expires_in: Option<Duration>,
}
impl fmt::Debug for TokenPair {
@@ -76,10 +77,14 @@ impl fmt::Debug for AuthorizationCodeGrant {
}
}
/// 官方 token endpoint 返回、尚待 OIDC/JWKS 绑定校验的 token set。
/// 官方 token endpoint 返回的 token set。
///
/// 与官方首次 authorization-code exchange 一致:`id_token`、`access_token` 与
/// `refresh_token` 都是响应的必填字段。这里不检查 token 内容、签名或 claims。
pub struct AuthorizationTokenSet {
pub secret: crate::credential::CodexOAuthSecret,
pub id_token: SecretString,
pub expires_in: Option<Duration>,
}
impl fmt::Debug for AuthorizationTokenSet {
@@ -88,6 +93,7 @@ impl fmt::Debug for AuthorizationTokenSet {
.debug_struct("AuthorizationTokenSet")
.field("secret", &"[REDACTED]")
.field("id_token", &"[REDACTED]")
.field("expires_in", &self.expires_in)
.finish()
}
}
@@ -168,16 +174,15 @@ pub fn official_openai_token_client() -> Result<OpenAiTokenClient, TokenClientBu
struct RefreshTokenResponse {
access_token: String,
refresh_token: Option<String>,
expires_in: u64,
expires_in: Option<u64>,
}
#[derive(Deserialize)]
struct AuthorizationCodeResponse {
access_token: String,
refresh_token: Option<String>,
refresh_token: String,
id_token: String,
token_type: String,
expires_in: u64,
expires_in: Option<u64>,
}
#[async_trait]
@@ -227,13 +232,6 @@ impl AuthorizationCodeExchanger for OpenAiTokenClient {
AuthorizationCodeExchangeError::Ambiguous
}
})?;
let success_is_json = response.status().is_success()
&& response
.headers()
.get(reqwest::header::CONTENT_TYPE)
.and_then(|value| value.to_str().ok())
.and_then(|value| value.split(';').next())
.is_some_and(|value| value.trim().eq_ignore_ascii_case("application/json"));
let (status, body) = read_bounded_response(response)
.await
.map_err(|_| AuthorizationCodeExchangeError::Ambiguous)?;
@@ -243,30 +241,17 @@ impl AuthorizationCodeExchanger for OpenAiTokenClient {
_ => AuthorizationCodeExchangeError::Rejected,
});
}
if !success_is_json {
return Err(AuthorizationCodeExchangeError::Rejected);
}
let tokens = serde_json::from_slice::<AuthorizationCodeResponse>(&body)
.map_err(|_| AuthorizationCodeExchangeError::Rejected)?;
if tokens.access_token.is_empty()
|| tokens.id_token.is_empty()
|| !tokens.token_type.eq_ignore_ascii_case("bearer")
|| tokens.expires_in == 0
|| tokens.expires_in > MAX_TOKEN_LIFETIME_SECONDS
|| tokens
.refresh_token
.as_deref()
.is_some_and(|token| token.is_empty())
{
return Err(AuthorizationCodeExchangeError::Rejected);
}
let id_token = SecretString::from(tokens.id_token);
Ok(AuthorizationTokenSet {
secret: crate::credential::CodexOAuthSecret {
access_token: SecretString::from(tokens.access_token),
refresh_token: tokens.refresh_token.map(SecretString::from),
refresh_token: Some(SecretString::from(tokens.refresh_token)),
id_token: None,
},
id_token: SecretString::from(tokens.id_token),
id_token,
expires_in: optional_expiry(tokens.expires_in),
})
}
}
@@ -294,19 +279,20 @@ async fn read_bounded_response(
fn parse_token_pair(body: &[u8]) -> Result<TokenPair, ()> {
let tokens = serde_json::from_slice::<RefreshTokenResponse>(body).map_err(|_| ())?;
if tokens.access_token.trim().is_empty() {
return Err(());
}
if tokens.expires_in == 0 || tokens.expires_in > MAX_TOKEN_LIFETIME_SECONDS {
return Err(());
}
Ok(TokenPair {
access_token: tokens.access_token,
// OAuth 刷新响应可能省略未变更的 RT;缺失时由调用方保留当前值。
refresh_token: tokens.refresh_token,
expires_in: Duration::from_secs(tokens.expires_in),
expires_in: optional_expiry(tokens.expires_in),
})
}
fn optional_expiry(value: Option<u64>) -> Option<Duration> {
value
.filter(|seconds| *seconds > 0)
.map(Duration::from_secs)
}
fn classify_refresh_failure(status: StatusCode, body: &[u8]) -> RefreshFailure {
// 5xx 与 429 按状态码判为瞬态:其正文(CDN/网关页等)不是权威 OAuth 错误响应,
// 不据此终态失效账号。
@@ -2,9 +2,10 @@
use std::fmt;
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use chrono::{DateTime, Utc};
use secrecy::SecretString;
use serde::{Deserialize, Serialize};
use serde::{Deserialize, Serialize, de::DeserializeOwned};
use url::Url;
/// OAuth AT/RT/ID Token;`Debug` 永不输出明文。
@@ -29,7 +30,7 @@ impl fmt::Debug for CodexOAuthSecret {
}
}
/// 已由官方 JWT/OIDC 与认证 usage 响应共同确认的完整账号投影。
/// OAuth credential 持久化时使用的账号投影。
#[derive(Clone)]
pub struct CodexAccountProfile {
pub email: Option<String>,
@@ -41,6 +42,116 @@ pub struct CodexAccountProfile {
pub access_token_expires_at: Option<DateTime<Utc>>,
}
/// 从 OAuth ID token payload 尽力提取的展示资料。
///
/// 这只是与官方客户端一致的 base64 JSON 读取,不校验 JWT 签名或 claims。
#[derive(Clone, Default)]
pub(crate) struct CodexOAuthMetadata {
pub(crate) email: Option<String>,
pub(crate) chatgpt_plan_type: Option<String>,
pub(crate) chatgpt_user_id: Option<String>,
pub(crate) chatgpt_account_id: Option<String>,
}
impl fmt::Debug for CodexOAuthMetadata {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("CodexOAuthMetadata")
.field("email", &self.email.as_ref().map(|_| "<redacted>"))
.field(
"chatgpt_user_id",
&self.chatgpt_user_id.as_ref().map(|_| "<redacted>"),
)
.field(
"chatgpt_account_id",
&self.chatgpt_account_id.as_ref().map(|_| "<redacted>"),
)
.field("chatgpt_plan_type", &self.chatgpt_plan_type)
.finish()
}
}
/// 按官方 `token_data.rs` 的字段优先级读取 ID token payload。
///
/// 与官方一致,`id_token` 的 JWT 外形、payload base64 或 JSON 无法解析时返回错误;
/// 这不是签名或 claims 验证。各身份字段本身仍全部可缺失。
pub(crate) fn parse_id_token_metadata(jwt: &str) -> Result<CodexOAuthMetadata, ()> {
let claims = decode_jwt_payload::<IdTokenClaims>(jwt)?;
let email = claims
.email
.or_else(|| claims.profile.and_then(|profile| profile.email));
let Some(auth) = claims.auth else {
return Ok(CodexOAuthMetadata {
email,
..CodexOAuthMetadata::default()
});
};
Ok(CodexOAuthMetadata {
email,
chatgpt_plan_type: auth.chatgpt_plan_type,
chatgpt_user_id: auth.chatgpt_user_id.or(auth.user_id),
chatgpt_account_id: auth.chatgpt_account_id,
})
}
/// 按官方 Codex 的方式,从 access token JWT payload 读取调度所需的过期时刻。
///
/// 此处只投影未验证的 `exp`,不把它当作身份或签名验证结果。无法解析或缺少
/// `exp` 的直接导入仍可保留为没有已知过期时刻的 OAuth 凭据。
pub(crate) fn parse_access_token_expiration(jwt: &str) -> Option<DateTime<Utc>> {
let claims = decode_jwt_payload::<StandardJwtClaims>(jwt).ok()?;
claims
.exp
.and_then(|seconds| DateTime::<Utc>::from_timestamp(seconds, 0))
}
#[derive(Deserialize)]
struct IdTokenClaims {
#[serde(default)]
email: Option<String>,
#[serde(rename = "https://api.openai.com/profile", default)]
profile: Option<ProfileClaims>,
#[serde(rename = "https://api.openai.com/auth", default)]
auth: Option<AuthClaims>,
}
#[derive(Deserialize)]
struct StandardJwtClaims {
#[serde(default)]
exp: Option<i64>,
}
#[derive(Deserialize)]
struct ProfileClaims {
#[serde(default)]
email: Option<String>,
}
#[derive(Deserialize)]
struct AuthClaims {
#[serde(default)]
chatgpt_plan_type: Option<String>,
#[serde(default)]
chatgpt_user_id: Option<String>,
#[serde(default)]
user_id: Option<String>,
#[serde(default)]
chatgpt_account_id: Option<String>,
}
fn decode_jwt_payload<T: DeserializeOwned>(jwt: &str) -> Result<T, ()> {
let mut parts = jwt.split('.');
let (Some(header), Some(payload), Some(signature)) = (parts.next(), parts.next(), parts.next())
else {
return Err(());
};
if header.is_empty() || payload.is_empty() || signature.is_empty() {
return Err(());
}
let payload = URL_SAFE_NO_PAD.decode(payload).map_err(|_| ())?;
serde_json::from_slice(&payload).map_err(|_| ())
}
impl fmt::Debug for CodexAccountProfile {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
@@ -109,7 +220,8 @@ pub const CODEX_AUTHENTICATION_KIND_AGENT_IDENTITY: &str = "agent_identity";
#[serde(deny_unknown_fields)]
pub struct CodexOAuthCredentialData {
pub schema_version: u32,
pub principal: CodexCredentialPrincipal,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub principal: Option<CodexCredentialPrincipal>,
pub installation_id: String,
pub access_token: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
+6 -26
View File
@@ -16,13 +16,10 @@ use gateway_core::task::WorkerContribution;
use crate::admin::{OpenAiAdminProvider, OpenAiAdminServices, OpenAiOAuthPendingStore};
use crate::credential::token_client::{AuthorizationCodeExchanger, TokenRefresher};
use crate::credential::{
CodexAccountIdentityService, CodexAccountIdentityVerifier, CodexAgentIdentityTaskService,
CodexAuthenticatedAccountSource, CodexCookiePolicy, CodexCredentialAdmin,
CodexAgentIdentityTaskService, CodexCookiePolicy, CodexCredentialAdmin,
CodexCredentialAdminService, CodexCredentialCatalogService, CodexCredentialQuotaService,
CodexCredentialRefreshService, CodexCredentialRepository, CodexCredentialSelector,
CodexJwtIdentityVerifier, CodexOAuthAdmin, CodexOAuthAdminService, CodexSignedIdentityVerifier,
OfficialCodexAgentIdentityTaskRegistrar, ReqwestCodexAuthenticatedAccountSource,
ReqwestOpenAiJwksSource,
CodexOAuthAdmin, CodexOAuthAdminService, OfficialCodexAgentIdentityTaskRegistrar,
};
use crate::transport::profile::{
CodexCliReleaseService, CodexDesktopReleaseService, OfficialCodexCliReleaseTransport,
@@ -82,7 +79,7 @@ pub async fn initialize(
profile.clone(),
Arc::new(
OfficialCodexCliReleaseTransport::new()
.map_err(|_| OpenAiInitializeError::DesktopRelease)?,
.map_err(|_| OpenAiInitializeError::CliRelease)?,
),
));
let repository = CodexCredentialRepository::new(Arc::clone(&accounts));
@@ -111,6 +108,7 @@ pub async fn initialize(
http.clone(),
config.base_url().to_owned(),
Arc::clone(&agent_identity),
ports.cooldowns(),
));
let quota_skip_exhausted = config.quota_skip_exhausted();
let selector = Arc::new(CodexCredentialSelector::new(
@@ -141,34 +139,20 @@ pub async fn initialize(
.map_err(OpenAiInitializeError::Provider)?
.with_session_identity(session_identity),
);
let token_client = Arc::new(
credential::token_client::openai_token_client(config.token_client_config())
.map_err(|_| OpenAiInitializeError::TokenClient)?,
);
let jwks = ReqwestOpenAiJwksSource::new().map_err(|_| OpenAiInitializeError::Identity)?;
let signed: Arc<dyn CodexSignedIdentityVerifier> = Arc::new(
CodexJwtIdentityVerifier::new(Box::new(jwks))
.with_oauth_client_id(config.oauth_client_id()),
);
let account_source: Arc<dyn CodexAuthenticatedAccountSource> = Arc::new(
ReqwestCodexAuthenticatedAccountSource::new(profile.clone(), config.base_url().to_owned())
.map_err(|_| OpenAiInitializeError::Identity)?,
);
let identity: Arc<dyn CodexAccountIdentityVerifier> =
Arc::new(CodexAccountIdentityService::new(signed, account_source));
let refresher: Arc<dyn TokenRefresher> = token_client.clone();
let exchanger: Arc<dyn AuthorizationCodeExchanger> = token_client;
let credential_admin = Arc::new(CodexCredentialAdminService::new(
Arc::clone(&refresher),
Arc::clone(&identity),
Arc::clone(&leases),
Arc::clone(&runtime_policy),
));
let refresh = Arc::new(CodexCredentialRefreshService::new(
repository,
refresher,
Arc::clone(&identity),
Arc::clone(&leases),
credential_state,
Arc::clone(&runtime_policy),
@@ -181,9 +165,7 @@ pub async fn initialize(
CodexOAuthAdminService::new(
pending,
exchanger,
Arc::clone(&identity),
Arc::clone(&accounts),
Arc::clone(&runtime_policy),
CodexCredentialAdmin,
)
.with_oauth_client_id(config.oauth_client_id()),
@@ -194,11 +176,9 @@ pub async fn initialize(
accounts,
OpenAiAdminServices {
credentials: credential_admin,
verifier: identity,
oauth: oauth_admin,
quota: Arc::clone(&quota),
catalog: Arc::clone(&catalog),
runtime_policy,
},
websocket_pool,
desktop_release_status,
@@ -257,8 +237,6 @@ pub enum OpenAiInitializeError {
CookiePolicy,
#[error("OpenAI token client could not initialize")]
TokenClient,
#[error("OpenAI identity verifier could not initialize")]
Identity,
#[error("OpenAI Agent Identity service could not initialize")]
AgentIdentity,
#[error("OpenAI credential administration could not initialize")]
@@ -267,6 +245,8 @@ pub enum OpenAiInitializeError {
Refresh,
#[error("OpenAI Desktop release service could not initialize")]
DesktopRelease,
#[error("OpenAI CLI release service could not initialize")]
CliRelease,
#[error("OpenAI worker plan is invalid")]
Worker,
}
+48 -10
View File
@@ -273,6 +273,8 @@ impl Provider for CodexProvider {
.await
.map_err(map_selection_error)?;
let lease = Arc::new(lease);
// 首字计时的起点:账号选择完成之后、上游建立之前。
let output_started_at = Instant::now();
let provider_kind = ProviderKind::new(PROVIDER_NAME)
.map_err(|_| provider_error(ProviderErrorKind::Protocol, UpstreamSendState::NotSent))?;
let cross_account = context
@@ -373,6 +375,7 @@ impl Provider for CodexProvider {
catalog: Arc::clone(&self.catalog),
agent_identity: Arc::clone(&self.agent_identity),
lease: Arc::clone(&lease),
output_started_at,
session_affinity_key,
session_capture,
});
@@ -397,6 +400,7 @@ struct ColdResponse {
catalog: Arc<CodexCredentialCatalogService>,
agent_identity: Arc<CodexAgentIdentityTaskService>,
lease: Arc<CodexCredentialLease>,
output_started_at: Instant,
session_affinity_key: Option<ProviderSessionAffinityKey>,
session_capture: Option<OpenAiSessionCapture>,
}
@@ -555,6 +559,7 @@ fn cold_response_stream(response: ColdResponse) -> EventStream {
catalog,
agent_identity,
lease,
output_started_at,
session_affinity_key,
mut session_capture,
} = response;
@@ -800,7 +805,7 @@ fn cold_response_stream(response: ColdResponse) -> EventStream {
capture.turn_state = Some(turn_state);
}
let first_event_changed =
observation_state.observe_stream_chunk(&chunk, context.timing_started_at());
observation_state.observe_stream_chunk(&chunk, output_started_at);
let chunk_len = chunk.len();
let (mut events, canonical_failure) = match decoder.push(&chunk) {
CodexCanonicalOutcome::Events(events) => (events, None),
@@ -830,7 +835,7 @@ fn cold_response_stream(response: ColdResponse) -> EventStream {
let timing_signals = decoder.take_timing_signals();
let timing_changed = first_event_changed
|| observation_state
.observe_timing_signals(timing_signals, context.timing_started_at());
.observe_timing_signals(timing_signals, output_started_at);
let completed = events
.iter()
.flat_map(ProviderEvent::canonical_facts)
@@ -909,7 +914,7 @@ fn cold_response_stream(response: ColdResponse) -> EventStream {
let service_tier_changed = observation_state
.observe_upstream_service_tier(decoder.response_service_tier());
let timing_changed = observation_state
.observe_timing_signals(timing_signals, context.timing_started_at());
.observe_timing_signals(timing_signals, output_started_at);
let updates = take_rate_limit_updates(rate_limit_updates.as_ref()).await;
let rate_limits_changed = if updates.is_empty() {
false
@@ -1063,7 +1068,9 @@ impl OpenAiResponseObservationState {
started_at: Instant,
) -> bool {
let mut changed = false;
if signals.semantic_output {
// 首个非前导输出事件(结构帧也算)开启首字计时;
// 真实语义首字由 first_reasoning_ms / first_text_ms 单独观测。
if signals.output_start {
changed |= insert_first_timing(&mut self.timings.first_token_ms, started_at);
}
if signals.reasoning_output {
@@ -1697,9 +1704,13 @@ fn map_selection_error(error: CredentialSelectionError) -> ProviderError {
struct MappedProviderFailure {
error: ProviderError,
account_failure: Option<CodexAccountFailure>,
/// 原始上游错误描述,随 availability 状态一并持久化。
error_message: Option<String>,
cyber_policy_failure: bool,
set_cookie_headers: Vec<String>,
rate_limit_headers: Vec<(String, String)>,
/// 结构化 usage-limit 错误给出的窗口 reset(Unix 秒)。
confirmed_exhaustion_reset_at: Option<i64>,
observation: Option<ProviderResponseObservation>,
capture_response_cookies: bool,
}
@@ -1709,9 +1720,11 @@ impl MappedProviderFailure {
Self {
error,
account_failure: None,
error_message: None,
cyber_policy_failure: false,
set_cookie_headers: Vec::new(),
rate_limit_headers: Vec::new(),
confirmed_exhaustion_reset_at: None,
observation: None,
capture_response_cookies: false,
}
@@ -1869,6 +1882,18 @@ async fn apply_failure(
failure.account_failure,
Some(CodexAccountFailure::QuotaExhausted | CodexAccountFailure::UsageLimitExhausted { .. })
);
if needs_authoritative_quota_refresh
&& let Err(error) = context
.quota
.record_confirmed_exhaustion(account, failure.confirmed_exhaustion_reset_at)
.await
{
tracing::warn!(
account_id = %account.id(),
error = %error,
"Failed to project confirmed OpenAI quota exhaustion"
);
}
if failure.cyber_policy_failure {
context
.selector
@@ -1882,7 +1907,7 @@ async fn apply_failure(
.await;
if let Err(error) = context
.selector
.record_failure(account, account_failure)
.record_failure(account, account_failure, failure.error_message.clone())
.await
{
tracing::warn!(
@@ -2162,9 +2187,11 @@ fn map_upstream_failure(
MappedProviderFailure {
error,
account_failure: account_failure(category, failure.retry_after_seconds),
error_message: failure.client_message,
cyber_policy_failure,
set_cookie_headers: failure.set_cookie_headers,
rate_limit_headers: failure.rate_limit_headers,
confirmed_exhaustion_reset_at: failure.usage_limit_resets_at,
observation,
capture_response_cookies: !matches!(
category,
@@ -2415,11 +2442,10 @@ impl ScheduledTask for OpenAiOAuthRefreshTask {
if context.cancellation().is_cancelled() {
return Ok(());
}
let outcomes = self
.service
.refresh_due()
.await
.map_err(|_| WorkerTaskError::safe("OpenAI OAuth refresh failed"))?;
let outcomes = self.service.refresh_due().await.map_err(|error| {
tracing::error!(error = %error, "OpenAI OAuth refresh cycle failed");
WorkerTaskError::safe("OpenAI OAuth refresh failed")
})?;
let mut refreshed = 0_u64;
let mut invalidated = 0_u64;
let mut banned = 0_u64;
@@ -2448,6 +2474,18 @@ impl ScheduledTask for OpenAiOAuthRefreshTask {
}
}
}
if !outcomes.is_empty() {
tracing::info!(
refreshed,
invalidated,
banned,
transient,
lease_unavailable,
stale,
failed,
"OpenAI OAuth refresh cycle completed"
);
}
if transient > 0 || failed > 0 {
tracing::warn!(
refreshed,
@@ -270,6 +270,11 @@ impl CodexCanonicalDecoder {
.event
.as_deref()
.or_else(|| value.get("type").and_then(Value::as_str));
if event_type == Some("codex.rate_limits") {
// HTTP transport has already projected this control frame into local quota facts.
// It must not become client output or start first-output timing.
return Ok(());
}
self.observe_response_service_tier(&value);
let signals = response_event_signals(event_type, &value);
self.merge_timing_signals(signals);
@@ -310,6 +315,7 @@ impl CodexCanonicalDecoder {
fn merge_timing_signals(&mut self, signals: ResponseEventSignals) {
self.timing_signals.protocol_progress |= signals.protocol_progress;
self.timing_signals.output_start |= signals.output_start;
self.timing_signals.semantic_output |= signals.semantic_output;
self.timing_signals.reasoning_output |= signals.reasoning_output;
self.timing_signals.text_output |= signals.text_output;
@@ -410,7 +416,7 @@ impl CodexCanonicalDecoder {
self.complete(event_type, value, output)
}
"response.failed" | "error" => Err(protocol_error_marker()),
"response.rate_limits.updated" | "codex.rate_limits" | "response.metadata" => Ok(()),
"response.rate_limits.updated" | "response.metadata" => Ok(()),
_ => Ok(()),
}
}
@@ -87,8 +87,13 @@ impl CodexBackendClient {
let headers = self.request_headers_for_http_response(upstream_request, context)?;
let headers_started_at = Instant::now();
// 与官方 Codex app-server 一致:/v1/responses 请求体默认 zstd 压缩。
// Codex 上游只交付 SSE;即使下游请求 `stream: false`,也要上游流式执行,
// 再由 API 层收集 canonical events 并返回完整 JSON。不能把下游的传输偏好
// 直接透传给 Codex,否则上游会以 400 拒绝非流式请求。
let mut upstream_body = upstream_request.body().clone();
upstream_body.insert("stream".to_owned(), serde_json::Value::Bool(true));
let body =
serde_json::to_vec(&upstream_request).map_err(CodexClientError::RequestBodyEncode)?;
serde_json::to_vec(&upstream_body).map_err(CodexClientError::RequestBodyEncode)?;
let body = zstd::stream::encode_all(std::io::Cursor::new(body), 3)
.map_err(CodexClientError::RequestCompression)?;
let response = self
@@ -3,6 +3,7 @@
use super::client::CodexClientVisibleUpstreamResponse;
use super::protocol::responses::ResponsesSseFailure;
use base64::{Engine as _, engine::general_purpose::STANDARD};
use chrono::{DateTime, Utc};
use reqwest::{StatusCode, header::HeaderMap};
use serde_json::Value;
@@ -113,6 +114,8 @@ pub struct CodexUpstreamFailure {
pub(crate) client_error_type: Option<String>,
pub(crate) client_response: Option<Box<CodexClientVisibleUpstreamResponse>>,
pub(crate) identity_error_code: Option<String>,
/// 官方 `usage_limit_reached` 错误体携带的窗口重置时间(Unix 秒)。
pub(crate) usage_limit_resets_at: Option<i64>,
pub(crate) retry_after_seconds: Option<u64>,
pub(crate) request_id: Option<String>,
pub(crate) set_cookie_headers: Vec<String>,
@@ -139,6 +142,7 @@ impl CodexUpstreamFailure {
) -> Self {
let fields = ParsedUpstreamError::from_body(body);
let category = classify_upstream_failure(
UpstreamFailureSource::HttpResponse,
Some(status),
body,
&fields,
@@ -153,6 +157,9 @@ impl CodexUpstreamFailure {
client_error_type: fields.error_type,
client_response: client_response.cloned().map(Box::new),
identity_error_code: diagnostics.identity_error_code.clone(),
usage_limit_resets_at: (category == CodexFailureCategory::UsageLimitExhausted)
.then_some(fields.resets_at)
.flatten(),
retry_after_seconds,
request_id: diagnostics.request_id.clone(),
set_cookie_headers: set_cookie_headers.to_vec(),
@@ -174,11 +181,13 @@ impl CodexUpstreamFailure {
error_type: failure.upstream_type.clone(),
message: failure.message.clone(),
client_message: Some(failure.message.clone()),
resets_at: None,
};
let status = failure
.explicit_status_code
.and_then(|code| StatusCode::from_u16(code).ok());
let category = classify_upstream_failure(
UpstreamFailureSource::SseFailure,
status,
"",
&fields,
@@ -193,6 +202,7 @@ impl CodexUpstreamFailure {
client_error_type: fields.error_type,
client_response: None,
identity_error_code: diagnostics.identity_error_code.clone(),
usage_limit_resets_at: None,
retry_after_seconds: failure.retry_after_seconds,
request_id: diagnostics.request_id.clone(),
set_cookie_headers: set_cookie_headers.to_vec(),
@@ -246,6 +256,7 @@ struct ParsedUpstreamError {
error_type: Option<String>,
message: String,
client_message: Option<String>,
resets_at: Option<i64>,
}
impl ParsedUpstreamError {
@@ -256,6 +267,7 @@ impl ParsedUpstreamError {
error_type: None,
message: body.to_owned(),
client_message: None,
resets_at: None,
};
};
let error = value
@@ -282,15 +294,26 @@ impl ParsedUpstreamError {
.clone()
.or_else(|| error.as_str().and_then(non_empty_owned))
.unwrap_or_else(|| body.to_owned());
let resets_at = error
.get("resets_at")
.and_then(Value::as_i64)
.filter(|seconds| DateTime::<Utc>::from_timestamp(*seconds, 0).is_some());
Self {
code,
error_type,
message,
client_message,
resets_at,
}
}
}
#[derive(Clone, Copy)]
enum UpstreamFailureSource {
HttpResponse,
SseFailure,
}
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct CodexUpstreamDiagnostics {
pub status_code: Option<u16>,
@@ -363,6 +386,7 @@ fn decode_identity_error_code(encoded: &str) -> Option<String> {
}
fn classify_upstream_failure(
source: UpstreamFailureSource,
status: Option<StatusCode>,
body: &str,
fields: &ParsedUpstreamError,
@@ -376,6 +400,19 @@ fn classify_upstream_failure(
let message = fields.message.to_ascii_lowercase();
let body = body.to_ascii_lowercase();
// 与官方 Codex HTTP/WS 路径一致:429 只有结构化
// `error.type=usage_limit_reached` 才能确认额度窗口耗尽;其余 429 都是临时限流。
// SSE `response.failed` 的结构化字段形态不同,保留其 code/type 语义单独分类。
if matches!(source, UpstreamFailureSource::HttpResponse)
&& status == Some(StatusCode::TOO_MANY_REQUESTS)
{
return if error_type == "usage_limit_reached" {
CodexFailureCategory::UsageLimitExhausted
} else {
CodexFailureCategory::RateLimited
};
}
if [code.as_str(), message.as_str(), body.as_str()]
.into_iter()
.any(is_model_unsupported)
@@ -14,21 +14,22 @@ use url::Url;
pub const CODEX_DESKTOP_APPCAST_URL: &str =
"https://persistent.oaistatic.com/codex-app-prod/appcast.xml";
pub const CODEX_CLI_RELEASE_URL: &str = "https://registry.npmjs.org/@openai%2Fcodex/latest";
/// 官方制品画像检查周期。
/// 官方发布元数据检查周期。
pub const APPCAST_POLL_INTERVAL: Duration = Duration::from_secs(24 * 60 * 60);
const APPCAST_TIMEOUT: Duration = Duration::from_secs(30);
const MAX_APPCAST_BYTES: usize = 1024 * 1024;
const MAX_CLI_RELEASE_BYTES: usize = 64 * 1024;
/// Codex Desktop 上游请求身份。
///
/// 启动配置提供经源码审计的 Core、运行环境和初始 Desktop 版本。官方 appcast
/// 检查成功后,会仅同步 Desktop 版本和构建号。
/// 启动配置提供经源码审计的 Core、运行环境和 Desktop 启动版本。npm CLI metadata
/// 与官方 appcast 检查成功后,分别同步 Core 版本和 Desktop 版本及构建号。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CodexWireProfile {
/// `originator` 请求头及 User-Agent 产品名。
pub originator: String,
/// bundled Codex Core 版本;同时用于 `/codex/models?client_version=`。
/// npm `@openai/codex` 启动版本;用于 `/codex/models?client_version=` 与 UA。
pub codex_version: String,
/// Desktop 应用版本,用于 app-server `clientInfo.version` 对应的 UA 后缀。
pub desktop_version: String,
@@ -47,16 +48,18 @@ pub struct CodexWireProfile {
}
impl CodexWireProfile {
/// 按 Codex Core 的官方格式生成最终 User-Agent。
/// 按 bundled Core app-server 的官方格式生成最终 User-Agent。
pub fn user_agent(&self) -> String {
format!(
"{}/{} ({} {}; {}) {}",
"{}/{} ({} {}; {}) {} ({}; {})",
self.originator,
self.codex_version,
self.os_type,
self.os_version,
self.arch,
self.terminal,
self.originator,
self.desktop_version,
)
}
}
@@ -96,7 +99,7 @@ impl CodexWireProfileState {
profile.desktop_build = desktop_build.to_owned();
}
/// 使用官方 npm metadata 发布的 Codex CLI 版本更新运行时 Core 画像。
/// 使用官方 npm metadata 发布的 Codex CLI 版本更新运行时请求画像。
pub fn update_cli_release(&self, codex_version: &str) {
let mut profile = self
.profile
@@ -143,12 +146,12 @@ impl CodexCliReleaseTransport for OfficialCodexCliReleaseTransport {
}
if response
.content_length()
.is_some_and(|size| size > 64 * 1024)
.is_some_and(|size| size > MAX_CLI_RELEASE_BYTES as u64)
{
return Err(CodexCliReleaseError::ResponseTooLarge);
}
let bytes = response.bytes().await?;
if bytes.len() > 64 * 1024 {
if bytes.len() > MAX_CLI_RELEASE_BYTES {
return Err(CodexCliReleaseError::ResponseTooLarge);
}
let document: serde_json::Value = serde_json::from_slice(&bytes)
@@ -182,11 +182,13 @@ pub fn transport_requirement(request: &CodexResponsesRequest) -> TransportRequir
/// 单个 Responses 事件对计时系统提供的稳定语义信号。
///
/// `protocol_progress` 只说明上游仍在工作,不能替代首字;其余字段分别标记
/// 客户端可消费的输出、reasoning 输出与正文输出。
/// `protocol_progress` 只说明上游仍在工作,不能替代首字;`output_start` 标记首个会
/// 开启客户端输出的非前导事件(结构帧也算);其余字段分别标记客户端可消费的
/// 输出、reasoning 输出与正文输出。
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct ResponseEventSignals {
pub protocol_progress: bool,
pub output_start: bool,
pub semantic_output: bool,
pub reasoning_output: bool,
pub text_output: bool,
@@ -194,13 +196,22 @@ pub struct ResponseEventSignals {
/// 从已解析的 Responses 事件提取计时语义。
///
/// 生命周期帧和结构帧不会被当作输出。`response.output_item.done` 与终态帧只有
/// 实际携带文本、工具参数、推理或图片结果时才算语义输出。
/// `output_start` 在任意非前导、非失败事件上置位(含结构帧
/// `response.output_item.added`/`content_part.added`),用于开启首字计时;前导帧
/// (`response.created`/`response.in_progress`)与失败帧不置位。语义输出仍要求
/// 实际携带文本、工具参数、推理或图片结果,`response.output_item.done` 与终态帧
/// 只有携带语义内容才算。
pub fn response_event_signals(event_type: Option<&str>, value: &Value) -> ResponseEventSignals {
let mut signals = ResponseEventSignals {
protocol_progress: !matches!(event_type, Some("response.failed" | "error")),
..ResponseEventSignals::default()
};
signals.output_start = event_type.is_some_and(|event_type| {
!matches!(
event_type,
"response.created" | "response.in_progress" | "response.failed" | "error"
)
});
match event_type {
Some("response.output_text.delta") => {
signals.text_output = non_empty_string(value.get("delta"));
@@ -606,6 +617,26 @@ impl CodexResponsesRequest {
)
}
/// 返回会话亲和所用的子代理区分值。
///
/// Codex 原生请求在 turn metadata 中声明 `subagent_kind`;网关的
/// `/responses/review` 等入口则通过 `client_metadata.x-openai-subagent`
/// 传递给上游。该可选值仅参与会话亲和键派生。
pub fn subagent_kind(&self) -> Option<String> {
self.semantics()
.subagent_kind
.filter(|value| !value.trim().is_empty())
.or_else(|| {
self.client_metadata()
.and_then(Value::as_object)
.and_then(|metadata| metadata.get("x-openai-subagent"))
.and_then(Value::as_str)
.map(str::trim)
.filter(|value| !value.is_empty())
.map(str::to_owned)
})
}
/// 设置 / 合并 client metadata。
pub fn set_client_metadata(&mut self, client_metadata: Option<Value>) {
match client_metadata {
@@ -10,6 +10,8 @@ use sha2::{Digest, Sha256};
use crate::transport::protocol::responses::CodexResponsesRequest;
const PASSTHROUGH_HEADERS_CONTEXT_KEY: &str = "opaque_request_headers";
const THREAD_SPAWN_SUBAGENT_KIND: &str = "thread_spawn";
const THREAD_SPAWN_CONVERSATION_PREFIX: &str = "thread-spawn:";
const CROSS_ACCOUNT_IDENTITY_KEYS: &[&str] = &[
"authorization",
@@ -160,28 +162,52 @@ fn string_value(value: Option<&Value>) -> Option<String> {
pub(crate) fn derive_conversation_anchor(
request: &CodexResponsesRequest,
) -> Option<(&'static str, String)> {
thread_spawn_conversation_anchor(request).or_else(|| {
request
.client_session_id
.as_deref()
.map(|value| ("session", value.to_owned()))
.or_else(|| {
request
.client_conversation_id
.as_deref()
.map(|value| ("conversation", value.to_owned()))
})
.or_else(|| {
request
.client_thread_id
.as_deref()
.map(|value| ("thread", value.to_owned()))
})
.or_else(|| {
request
.prompt_cache_key()
.map(|value| ("prompt-cache", value.to_owned()))
})
.or_else(|| derive_stable_conversation_key(request).map(|value| ("request", value)))
})
}
/// A `thread_spawn` shares its parent session but is a separate child execution.
/// Its child thread must therefore own the affinity and WebSocket pool identity.
fn thread_spawn_conversation_anchor(
request: &CodexResponsesRequest,
) -> Option<(&'static str, String)> {
if request.subagent_kind().as_deref() != Some(THREAD_SPAWN_SUBAGENT_KIND) {
return None;
}
request
.client_session_id
.client_thread_id
.as_deref()
.map(|value| ("session", value.to_owned()))
.or_else(|| {
request
.client_conversation_id
.as_deref()
.map(|value| ("conversation", value.to_owned()))
.map(str::trim)
.filter(|thread_id| !thread_id.is_empty())
.map(|thread_id| {
(
"subagent-thread",
format!("{THREAD_SPAWN_CONVERSATION_PREFIX}{thread_id}"),
)
})
.or_else(|| {
request
.client_thread_id
.as_deref()
.map(|value| ("thread", value.to_owned()))
})
.or_else(|| {
request
.prompt_cache_key()
.map(|value| ("prompt-cache", value.to_owned()))
})
.or_else(|| derive_stable_conversation_key(request).map(|value| ("request", value)))
}
const LEADING_SYSTEM_REMINDER_OPEN: &str = "<system-reminder>";
@@ -3,7 +3,7 @@
use std::fmt;
use bytes::Bytes;
use gateway_protocol::openai::events::is_rate_limit_header_name;
use gateway_protocol::openai::events::{is_codex_quota_header_name, is_rate_limit_header_name};
use reqwest::header::{HeaderMap, SET_COOKIE};
use super::diagnostics::CodexUpstreamDiagnostics;
@@ -155,6 +155,9 @@ fn client_response_header_is_forwardable(name: &str, connection_options: &[Strin
{
return false;
}
if is_codex_quota_header_name(&name) {
return false;
}
!matches!(
name.as_str(),
+103 -10
View File
@@ -15,7 +15,7 @@ use gateway_admin::model::provider_credentials::{
AuthorizationMutationTarget, AuthorizationOwnerBinding, CompleteAuthorization,
PendingAuthorizationMutation, PrepareCredentialImport, PrepareCredentialRefresh,
PrepareCredentialRotation, ProviderDocument, ProviderExportCredentialInput,
ProviderQuotaRequest,
ProviderQuotaRequest, QuotaLocalUsageAttribution,
};
use gateway_admin::model::{MutationActor, MutationContext, Revision};
use gateway_admin::ports::provider::ProviderAdminErrorKind;
@@ -82,6 +82,22 @@ async fn openai_bundle_exposes_one_core_provider_and_drains_worker_contributions
panic!("Desktop release worker must be scheduled");
};
assert_eq!(schedule.interval(), APPCAST_POLL_INTERVAL);
let cli_release_worker = contributions
.iter()
.find_map(|contribution| match contribution {
WorkerContribution::Registration(registration)
if registration.id.owner() == "openai-cli-release" =>
{
Some(registration)
}
WorkerContribution::Registration(_) | WorkerContribution::Disabled { .. } => None,
})
.expect("CLI release worker");
assert_eq!(cli_release_worker.id.kind(), WorkerKind::QuotaCatalogHealth);
let WorkerRunnable::Scheduled { schedule, .. } = &cli_release_worker.runnable else {
panic!("CLI release worker must be scheduled");
};
assert_eq!(schedule.interval(), APPCAST_POLL_INTERVAL);
assert!(contributions.iter().any(|contribution| {
matches!(
contribution,
@@ -101,13 +117,21 @@ async fn openai_admin_provider_exposes_live_wire_profile_and_validated_billing()
.expect("OpenAI bundle");
let admin = bundle.admin_provider();
let profile = admin.dashboard_wire_profile().expect("wire profile");
assert_eq!(profile.version, "0.102.0");
assert_eq!(profile.build, None);
assert_eq!(profile.target.os_type, "Mac OS");
assert_eq!(profile.target.os_version, "15.5.0");
assert_eq!(
profile.user_agent,
"Codex Desktop/0.102.0 (Mac OS 15.5.0; arm64) xterm-256color (Codex Desktop; 1.2026.190)"
);
assert_eq!(
profile
.attributes
.iter()
.find(|attribute| attribute.label == "Codex Core")
.find(|attribute| attribute.label == "客户端标识")
.map(|attribute| attribute.value.as_str()),
Some("0.102.0")
Some("Codex Desktop; 1.2026.190")
);
assert_eq!(
profile.release.as_ref().map(|release| release.status),
@@ -428,7 +452,8 @@ async fn openai_admin_provider_projects_cached_quota_models_and_canonical_export
}
#[tokio::test]
async fn openai_admin_provider_projects_codex_additional_limit_as_the_primary_quota() {
async fn openai_admin_provider_projects_official_codex_quota_and_independent_buckets_with_chinese_labels()
{
let store = Arc::new(MemoryAccountStore::default());
store
.seed_oauth_credential(ImportCodexOAuthCredential {
@@ -444,11 +469,15 @@ async fn openai_admin_provider_projects_codex_additional_limit_as_the_primary_qu
.account("acct_admin_canonical_quota")
.expect("stored account");
let raw = json!({
"active_limit": "premium",
"rate_limit": {
"primary_window": {
"used_percent": 91,
"reset_at": 1_900_000_000,
"limit_window_seconds": 2_592_000
},
"secondary_window": {
"used_percent": 88
}
},
"additional_rate_limits": [{
@@ -459,6 +488,29 @@ async fn openai_admin_provider_projects_codex_additional_limit_as_the_primary_qu
"used_percent": 2,
"reset_at": 1_900_000_000,
"limit_window_seconds": 2_592_000
},
"secondary_window": {
"used_percent": 0
}
}
}, {
"limit_name": "code_review",
"metered_feature": "code_review",
"rate_limit": {
"primary_window": {
"used_percent": 12,
"reset_at": 1_900_000_000,
"limit_window_seconds": 604_800
}
}
}, {
"limit_name": "GPT-5.3-Codex-Spark",
"metered_feature": "codex_bengalfox",
"rate_limit": {
"primary_window": {
"used_percent": 0,
"reset_at": 1_900_000_000,
"limit_window_seconds": 604_800
}
}
}]
@@ -499,10 +551,48 @@ async fn openai_admin_provider_projects_codex_additional_limit_as_the_primary_qu
.collect::<Vec<_>>();
assert_eq!(monthly.len(), 1);
// limit_name 优先显示,不再猜成「月限额」。
assert_eq!(monthly[0].label, "custom codex label");
assert_eq!(monthly[0].source.as_deref(), Some("core"));
assert_eq!(monthly[0].used_percent, Some(2.0));
assert!(monthly.iter().any(|window| {
window.label == "月额度"
&& window.source.as_deref() == Some("codex")
&& window.used_percent == Some(91.0)
}));
assert!(
!quota
.windows
.iter()
.any(|window| window.key.starts_with("additional-0-codex")),
"the additional codex alias should not become a second display bucket"
);
let secondary = quota
.windows
.iter()
.find(|window| window.key == "core-secondary")
.expect("core secondary quota");
assert_eq!(secondary.label, "次级额度");
assert_eq!(secondary.used_percent, Some(88.0));
let review = quota
.windows
.iter()
.find(|window| window.source.as_deref() == Some("code_review"))
.expect("code review quota");
assert_eq!(review.label, "代码审查 · 周额度");
let spark = quota
.windows
.iter()
.find(|window| window.source.as_deref() == Some("codex_bengalfox"))
.expect("Spark quota");
assert_eq!(
(
monthly[0].local_usage_attribution,
review.local_usage_attribution,
spark.local_usage_attribution,
),
(
QuotaLocalUsageAttribution::AccountWide,
QuotaLocalUsageAttribution::Unavailable,
QuotaLocalUsageAttribution::Unavailable,
),
);
}
#[tokio::test]
@@ -544,6 +634,7 @@ async fn openai_admin_projects_confirmed_quota_exhaustion_as_full_without_mutati
.expect("persist raw quota");
store
.apply_state_change(AccountStateChange {
message: None,
account_id: account.id().clone(),
expected_revision: account.revision(),
availability: CoreAccountAvailability::QuotaExhausted,
@@ -584,6 +675,7 @@ async fn openai_admin_projects_confirmed_quota_exhaustion_as_full_without_mutati
store
.apply_state_change(AccountStateChange {
message: None,
account_id: account.id().clone(),
expected_revision: account.revision(),
availability: CoreAccountAvailability::Ready,
@@ -700,7 +792,7 @@ fn account_record(account: &ProviderAccount) -> AccountRecord {
provider_kind: account.provider().clone(),
name: account.name().to_owned(),
email: account.email().map(str::to_owned),
upstream_user_id: account.upstream_user_id().to_owned(),
upstream_user_id: account.upstream_user_id().map(str::to_owned),
upstream_account_id: account.upstream_account_id().map(str::to_owned),
plan_type: account.plan_type().map(str::to_owned),
authentication_kind: account.authentication_kind().to_owned(),
@@ -711,6 +803,7 @@ fn account_record(account: &ProviderAccount) -> AccountRecord {
enabled: account.enabled(),
availability: AdminAccountAvailability::Ready,
availability_observed_at: now,
last_error_message: None,
quota_observed_at: None,
created_at: now,
updated_at: now,
@@ -729,7 +822,7 @@ fn valid_config() -> TestOpenAiConfig {
codex_version: "0.102.0".to_owned(),
desktop_version: "1.2026.190".to_owned(),
desktop_build: "19012345678".to_owned(),
os_type: "macOS".to_owned(),
os_type: "Mac OS".to_owned(),
os_version: "15.5.0".to_owned(),
arch: "arm64".to_owned(),
terminal: "xterm-256color".to_owned(),
@@ -13,7 +13,7 @@ fn openai_config_builds_the_audited_wire_profile() {
assert_eq!(
profile.user_agent(),
"Codex Desktop/0.102.0 (macOS 15.5.0; arm64) xterm-256color"
"Codex Desktop/0.102.0 (Mac OS 15.5.0; arm64) xterm-256color (Codex Desktop; 1.2026.190)"
);
assert_eq!(profile.desktop_build, "19012345678");
}
@@ -124,6 +124,10 @@ fn openai_config_defaults_to_the_provider_owned_operating_values() {
"https://auth.openai.com/oauth/token",
)
);
assert_eq!(
config.wire_profile_state().snapshot().user_agent(),
"Codex Desktop/0.146.0 (Mac OS 15.7.1; arm64) unknown (Codex Desktop; 26.727.51351)"
);
}
fn valid_config() -> OpenAiConfig {
@@ -133,7 +137,7 @@ fn valid_config() -> OpenAiConfig {
codex_version: "0.102.0".to_owned(),
desktop_version: "1.2026.190".to_owned(),
desktop_build: "19012345678".to_owned(),
os_type: "macOS".to_owned(),
os_type: "Mac OS".to_owned(),
os_version: "15.5.0".to_owned(),
arch: "arm64".to_owned(),
terminal: "xterm-256color".to_owned(),
@@ -1,804 +1,97 @@
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::{Arc, Mutex};
use std::time::Duration;
use std::sync::Arc;
use async_trait::async_trait;
use base64::Engine as _;
use base64::engine::general_purpose::STANDARD;
use chrono::{TimeZone, Utc};
use ed25519_dalek::SigningKey;
use ed25519_dalek::pkcs8::EncodePrivateKey as _;
use futures::future::BoxFuture;
use gateway_core::engine::credential::{
AccountAvailability, CredentialRevision, LoadedCredential, PlaintextCredential,
ProviderAccount, ProviderAccountId, ProviderAccountStore,
};
use gateway_core::provider_ports::{
ProviderLeaseAcquisition, ProviderLeasePort, ProviderLeaseRequest, ProviderRefreshLeaseRequest,
ProviderStoreError,
};
use gateway_core::routing::ProviderKind;
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
use chrono::{TimeZone as _, Utc};
use provider_openai::credential::token_client::{RefreshFailure, TokenPair, TokenRefresher};
use provider_openai::credential::{
CODEX_AUTHENTICATION_KIND_AGENT_IDENTITY, CodexAccountIdentityVerifier,
CodexAgentIdentityAuthMode, CodexAgentIdentityCredentialData, CodexCredentialAdmin,
CodexCredentialAdminError, CodexCredentialAdminService, CodexCredentialCodec,
CodexIdentityExpectation, CodexIdentityVerification, CodexIdentityVerificationError,
ExportManagedCodexCredential, ImportCodexOAuthCredential, RotateManagedCodexCredential,
};
use secrecy::{ExposeSecret, SecretString};
use provider_openai::credential::{CodexCredentialAdminService, CodexCredentialCodec};
use secrecy::ExposeSecret as _;
use crate::support::{MemoryAccountStore, codex_account, profile, runtime_policy, secret};
use crate::support::{TestLeaseCoordinator, runtime_policy};
fn import(id: &str, token: &str) -> ImportCodexOAuthCredential {
ImportCodexOAuthCredential {
account_id: id.to_owned(),
name: format!("name-{id}"),
secret: secret(token),
verified_account: profile(&format!("chatgpt-{id}")),
next_refresh_at: Some(chrono::Utc::now() + chrono::Duration::minutes(30)),
enabled: true,
struct UnusedRefresher;
#[async_trait]
impl TokenRefresher for UnusedRefresher {
async fn refresh(&self, _refresh_token: &str) -> Result<TokenPair, RefreshFailure> {
panic!("a direct access-token import must not refresh its refresh token")
}
}
fn encoded_credential(account_id: &str, token: &str) -> PlaintextCredential {
CodexCredentialCodec::encode_new(&secret(token), &profile(account_id), Vec::new())
.expect("encode current credential")
}
#[test]
fn prepare_import_returns_core_account_with_plaintext_json() {
let prepared = CodexCredentialAdmin
.prepare_import(import("acct_import", "access-import"))
.expect("prepare import");
assert_eq!(prepared.account.id().as_str(), "acct_import");
assert_eq!(prepared.account.provider().as_str(), "openai");
assert_eq!(
prepared
.credential
.expose_to_provider()
.get("access_token")
.and_then(serde_json::Value::as_str),
Some("access-import")
);
}
#[test]
fn prepare_rotation_preserves_provider_owned_cookie_data() {
let account = codex_account("acct_rotate");
let mut credential = encoded_credential("chatgpt-acct_rotate", "old-access");
credential
.expose_to_provider()
.get("access_token")
.expect("access token");
let mut data = CodexCredentialCodec::decode_complete(&credential).expect("decode data");
data.oauth_mut().expect("OAuth data").oauth_client_id = Some("oauth-client".to_owned());
credential = CodexCredentialCodec::encode_complete(data).expect("encode complete data");
let prepared = CodexCredentialAdmin
.prepare_rotation(RotateManagedCodexCredential {
current: LoadedCredential {
account,
credential,
},
secret: secret("new-access"),
verified_account: profile("chatgpt-acct_rotate"),
next_refresh_at: Some(chrono::Utc::now() + chrono::Duration::minutes(30)),
})
.expect("prepare rotation");
let decoded = CodexCredentialCodec::decode(prepared.credential.credential())
.expect("decode prepared credential");
assert_eq!(
decoded
.authentication
.oauth()
.expect("OAuth credential")
.access_token
.expose_secret(),
"new-access"
);
assert_eq!(decoded.oauth_client_id.as_deref(), Some("oauth-client"));
}
#[test]
fn prepare_rotation_rejects_account_rebinding() {
let error = CodexCredentialAdmin
.prepare_rotation(RotateManagedCodexCredential {
current: LoadedCredential {
account: codex_account("acct_rotate"),
credential: encoded_credential("chatgpt-acct_rotate", "old-access"),
},
secret: secret("new-access"),
verified_account: profile("chatgpt-other"),
next_refresh_at: Some(chrono::Utc::now() + chrono::Duration::minutes(30)),
})
.expect_err("identity rebinding must fail");
assert_eq!(error, CodexCredentialAdminError::IdentityMismatch);
}
#[test]
fn prepare_rotation_rejects_principal_rebinding_without_full_authorization() {
let mut rebound = profile("chatgpt-acct_rotate");
rebound.oauth_subject = "different-subject".to_owned();
rebound.poid = None;
let error = CodexCredentialAdmin
.prepare_rotation(RotateManagedCodexCredential {
current: LoadedCredential {
account: codex_account("acct_rotate"),
credential: encoded_credential("chatgpt-acct_rotate", "old-access"),
},
secret: secret("new-access"),
verified_account: rebound,
next_refresh_at: Some(chrono::Utc::now() + chrono::Duration::minutes(30)),
})
.expect_err("non-authorization rotation cannot replace the principal");
assert_eq!(error, CodexCredentialAdminError::IdentityMismatch);
}
#[test]
fn prepared_commands_debug_never_prints_tokens() {
let prepared = CodexCredentialAdmin
.prepare_import(import("acct_debug", "debug-secret"))
.expect("prepare import");
assert!(!format!("{prepared:?}").contains("debug-secret"));
let _: PlaintextCredential = prepared.credential;
}
fn export_item(id: &str, token: &str) -> ExportManagedCodexCredential {
ExportManagedCodexCredential {
current: LoadedCredential {
account: codex_account(id),
credential: encoded_credential(&format!("chatgpt-{id}"), token),
},
added_at: Utc
.with_ymd_and_hms(2026, 7, 18, 2, 47, 1)
.single()
.expect("added at"),
updated_at: Utc
.with_ymd_and_hms(2026, 7, 19, 3, 0, 0)
.single()
.expect("updated at"),
}
}
fn agent_export_item(id: &str) -> (ExportManagedCodexCredential, String) {
let signing_key = SigningKey::from_bytes(&[7; 32]);
let private_key = STANDARD.encode(
signing_key
.to_pkcs8_der()
.expect("encode Agent Identity key")
.as_bytes(),
);
let credential =
CodexCredentialCodec::encode_agent_identity(CodexAgentIdentityCredentialData {
schema_version: 1,
auth_mode: CodexAgentIdentityAuthMode::AgentIdentity,
installation_id: "00000000-0000-4000-8000-000000000007".to_owned(),
agent_runtime_id: "runtime-export".to_owned(),
agent_private_key: private_key.clone(),
task_id: Some("task-export".to_owned()),
cookies: Vec::new(),
})
.expect("encode Agent Identity credential");
let account = ProviderAccount::new(
ProviderAccountId::new(id.to_owned()).expect("account id"),
ProviderKind::new("openai").expect("provider"),
format!("Agent Identity {id}"),
"agent-user-export".to_owned(),
CODEX_AUTHENTICATION_KIND_AGENT_IDENTITY.to_owned(),
CredentialRevision::new(1).expect("revision"),
None,
)
.with_profile(
Some("agent-export@example.com".to_owned()),
Some("agent-account-export".to_owned()),
Some("pro".to_owned()),
)
.with_runtime_state(true, AccountAvailability::Ready)
.with_refresh_schedule(false, None);
(
ExportManagedCodexCredential {
current: LoadedCredential {
account,
credential,
},
added_at: Utc
.with_ymd_and_hms(2026, 7, 18, 2, 47, 1)
.single()
.expect("added at"),
updated_at: Utc
.with_ymd_and_hms(2026, 7, 19, 3, 0, 0)
.single()
.expect("updated at"),
},
private_key,
)
}
#[test]
fn cpr_export_matches_the_canonical_real_document_shape() {
let document = CodexCredentialAdmin
.format_cpr_export(vec![export_item("acct_export", "export-secret")])
.expect("format export");
assert_eq!(document.len(), 1);
let value = document.into_json().expect("serialize export");
assert_eq!(value["sourceFormat"], "cpr");
let account = &value["accounts"][0];
let mut keys = account
.as_object()
.expect("account object")
.keys()
.map(String::as_str)
.collect::<Vec<_>>();
keys.sort_unstable();
assert_eq!(
keys,
[
"accessTokenExpiresAt",
"accountId",
"addedAt",
"email",
"id",
"label",
"planType",
"refreshToken",
"status",
"token",
"updatedAt",
"userId",
]
);
assert_eq!(account["id"], "acct_export");
assert_eq!(account["token"], "export-secret");
assert_eq!(account["refreshToken"], "rt-export-secret");
assert_eq!(account["status"], "active");
assert!(
account["addedAt"]
.as_str()
.is_some_and(|value| value.ends_with("+08:00"))
);
assert!(
account["accessTokenExpiresAt"]
.as_str()
.is_some_and(|value| value.ends_with("+00:00"))
);
}
#[test]
fn cpr_export_batch_validation_and_debug_are_secret_safe() {
assert_eq!(
CodexCredentialAdmin
.format_cpr_export(Vec::new())
.expect_err("empty export"),
CodexCredentialAdminError::InvalidInput
);
let duplicate = CodexCredentialAdmin
.format_cpr_export(vec![
export_item("acct_duplicate_export", "first-secret"),
export_item("acct_duplicate_export", "second-secret"),
])
.expect_err("duplicate export account");
assert_eq!(duplicate, CodexCredentialAdminError::InvalidInput);
let document = CodexCredentialAdmin
.format_cpr_export(vec![export_item("acct_debug_export", "never-print-me")])
.expect("export document");
let debug = format!("{document:?}");
assert!(!debug.contains("never-print-me"));
assert!(!debug.contains("rt-never-print-me"));
assert!(debug.contains("account_count: 1"));
}
#[test]
fn cpr_export_preserves_agent_identity_material_without_oauth_fields() {
let (item, private_key) = agent_export_item("acct_agent_export");
let value = CodexCredentialAdmin
.format_cpr_export(vec![item])
.expect("Agent Identity export")
.into_json()
.expect("serialize Agent Identity export");
let account = &value["accounts"][0];
assert_eq!(account["authMode"], "agentIdentity");
assert_eq!(account["agentRuntimeId"], "runtime-export");
assert!(
account["agentPrivateKey"]
.as_str()
.is_some_and(|value| value == private_key)
);
assert_eq!(account["taskId"], "task-export");
assert!(account.get("token").is_none());
assert!(account.get("refreshToken").is_none());
assert!(account.get("accessTokenExpiresAt").is_none());
}
#[tokio::test]
async fn cpr_export_and_import_support_mixed_oauth_and_agent_identity_accounts() {
let (agent, _) = agent_export_item("acct_mixed_agent");
let value = CodexCredentialAdmin
.format_cpr_export(vec![
export_item("acct_mixed_oauth", "token-acct_mixed_oauth"),
agent,
])
.expect("mixed export")
.into_json()
.expect("serialize mixed export");
let prepared = import_service(unused_import_refresher())
.prepare_import_document(value)
.await
.expect("mixed CPR import");
assert_eq!(prepared.accounts().len(), 2);
assert!(prepared.accounts().iter().any(|account| {
account.account.authentication_kind() == CODEX_AUTHENTICATION_KIND_AGENT_IDENTITY
}));
assert!(
prepared
.accounts()
.iter()
.any(|account| { account.account.authentication_kind() == "oauth" })
);
}
pub(super) struct ManualRefresher {
outcome: Mutex<Option<Result<TokenPair, RefreshFailure>>>,
seen: Mutex<Vec<String>>,
}
#[async_trait]
impl TokenRefresher for ManualRefresher {
async fn refresh(&self, refresh_token: &str) -> Result<TokenPair, RefreshFailure> {
self.seen
.lock()
.expect("seen refresh tokens")
.push(refresh_token.to_owned());
self.outcome
.lock()
.expect("refresh outcome")
.take()
.expect("one refresh outcome")
}
}
struct ManualVerifier;
#[async_trait]
impl CodexAccountIdentityVerifier for ManualVerifier {
async fn verify(
&self,
secret: &provider_openai::credential::CodexOAuthSecret,
_expectation: &CodexIdentityExpectation,
) -> Result<CodexIdentityVerification, CodexIdentityVerificationError> {
if secret.access_token.expose_secret() != "refreshed-access" {
return Err(CodexIdentityVerificationError::Rejected);
}
Ok(CodexIdentityVerification::Complete(profile(
"chatgpt-acct_manual_refresh",
)))
}
async fn verify_authorization(
&self,
_secret: &provider_openai::credential::CodexOAuthSecret,
_id_token: &SecretString,
_expected_nonce: &SecretString,
_expectation: &CodexIdentityExpectation,
) -> Result<CodexIdentityVerification, CodexIdentityVerificationError> {
Err(CodexIdentityVerificationError::Rejected)
}
}
struct DropCountGuard(Arc<AtomicUsize>);
impl Drop for DropCountGuard {
fn drop(&mut self) {
self.0.fetch_add(1, Ordering::SeqCst);
}
}
struct ManualLeases {
requests: Mutex<Vec<ProviderRefreshLeaseRequest>>,
drops: Arc<AtomicUsize>,
available: bool,
}
impl ProviderLeasePort for ManualLeases {
fn load_state<'a>(
&'a self,
_: &'a gateway_core::routing::ProviderKind,
_: &'a [ProviderAccountId],
) -> BoxFuture<
'a,
Result<gateway_core::provider_ports::ProviderSchedulingState, ProviderStoreError>,
> {
Box::pin(async {
Ok(gateway_core::provider_ports::ProviderSchedulingState::new(
Default::default(),
0,
))
})
}
fn try_acquire(
&self,
request: ProviderLeaseRequest,
) -> BoxFuture<'_, Result<ProviderLeaseAcquisition, ProviderStoreError>> {
Box::pin(async move {
match request {
ProviderLeaseRequest::RefreshCapacity(_) => {}
ProviderLeaseRequest::Refresh(request) => {
self.requests.lock().expect("lease requests").push(request);
}
ProviderLeaseRequest::Scheduling(_) => panic!("unexpected scheduling lease"),
}
Ok(if self.available {
ProviderLeaseAcquisition::Acquired(Box::new(DropCountGuard(Arc::clone(
&self.drops,
))))
} else {
ProviderLeaseAcquisition::Busy { retry_after: None }
})
})
}
}
async fn manual_refresh_fixture(
outcome: Result<TokenPair, RefreshFailure>,
available: bool,
) -> (
Arc<MemoryAccountStore>,
Arc<ManualRefresher>,
Arc<ManualLeases>,
CodexCredentialAdminService,
) {
let store = Arc::new(MemoryAccountStore::default());
store
.seed_oauth_credential(ImportCodexOAuthCredential {
account_id: "acct_manual_refresh".to_owned(),
name: "manual refresh".to_owned(),
secret: secret("old-access"),
verified_account: profile("chatgpt-acct_manual_refresh"),
next_refresh_at: Some(chrono::Utc::now() + chrono::Duration::minutes(30)),
enabled: true,
})
.await;
let refresher = Arc::new(ManualRefresher {
outcome: Mutex::new(Some(outcome)),
seen: Mutex::new(Vec::new()),
});
let leases = Arc::new(ManualLeases {
requests: Mutex::new(Vec::new()),
drops: Arc::new(AtomicUsize::new(0)),
available,
});
async fn direct_import_persists_opaque_tokens_without_profile_or_token_validation() {
let service = CodexCredentialAdminService::new(
refresher.clone(),
Arc::new(ManualVerifier),
leases.clone(),
Arc::new(UnusedRefresher),
Arc::new(TestLeaseCoordinator::default()),
runtime_policy(),
);
(store, refresher, leases, service)
}
fn refreshed_tokens() -> TokenPair {
TokenPair {
access_token: "refreshed-access".to_owned(),
refresh_token: Some("rotated-refresh".to_owned()),
expires_in: Duration::from_secs(3_600),
}
}
#[tokio::test]
async fn manual_refresh_prepares_revision_fenced_rotation_without_store_mutation() {
let (store, refresher, leases, service) =
manual_refresh_fixture(Ok(refreshed_tokens()), true).await;
let account_id = ProviderAccountId::new("acct_manual_refresh").expect("account id");
let current = store
.load_current_credential(&account_id)
.await
.expect("current credential");
let prepared = service
.manual_refresh(current)
.prepare_import_document(serde_json::json!({
"token": " ",
"refresh_token": "",
"id_token": "not.a.parseable.jwt"
}))
.await
.expect("manual refresh");
.expect("opaque direct import must be accepted");
assert_eq!(prepared.credential.account_id(), &account_id);
assert_eq!(prepared.credential.expected_revision().get(), 1);
let runtime = CodexCredentialCodec::decode(prepared.credential.credential())
.expect("prepared credential");
let account = prepared.accounts().first().expect("one prepared account");
let runtime = CodexCredentialCodec::decode(&account.credential).expect("stored credential");
let oauth = runtime.authentication.oauth().expect("OAuth credential");
assert_eq!(oauth.access_token.expose_secret(), " ");
assert_eq!(
runtime
.authentication
.oauth()
.expect("OAuth credential")
.access_token
.expose_secret(),
"refreshed-access"
);
assert_eq!(
runtime
.authentication
.oauth()
.expect("OAuth credential")
oauth
.refresh_token
.as_ref()
.expect("rotated refresh")
.expect("provided refresh token")
.expose_secret(),
"rotated-refresh"
""
);
assert_eq!(
store
.account("acct_manual_refresh")
.expect("account")
.revision()
.get(),
1
oauth
.id_token
.as_ref()
.expect("provided ID token")
.expose_secret(),
"not.a.parseable.jwt"
);
assert_eq!(
refresher.seen.lock().expect("seen tokens").as_slice(),
["rt-old-access"]
);
assert_eq!(leases.drops.load(Ordering::SeqCst), 0);
drop(prepared);
assert_eq!(leases.drops.load(Ordering::SeqCst), 2);
assert!(account.account.upstream_user_id().is_none());
assert!(account.account.access_token_expires_at().is_none());
assert!(account.account.next_refresh_at().is_none());
}
#[tokio::test]
async fn manual_refresh_missing_lease_fails_before_exchange() {
let (store, unavailable_refresher, _, unavailable_service) =
manual_refresh_fixture(Ok(refreshed_tokens()), false).await;
let account_id = ProviderAccountId::new("acct_manual_refresh").expect("account id");
let current = store
.load_current_credential(&account_id)
.await
.expect("current credential");
let unavailable = unavailable_service
.manual_refresh(current)
.await
.expect_err("lease unavailable");
assert_eq!(
unavailable,
CodexCredentialAdminError::RefreshLeaseUnavailable
async fn direct_import_projects_access_token_jwt_expiry_without_persisting_refresh_margin() {
let expires_at = Utc
.timestamp_opt(2_000_000_000, 0)
.single()
.expect("valid test timestamp");
let access_token = format!(
"unverified-header.{}.unverified-signature",
URL_SAFE_NO_PAD.encode(
serde_json::to_vec(&serde_json::json!({"exp": expires_at.timestamp()}))
.expect("test JWT payload"),
)
);
assert!(
unavailable_refresher
.seen
.lock()
.expect("seen tokens")
.is_empty()
);
}
struct ImportVerifier;
#[async_trait]
impl CodexAccountIdentityVerifier for ImportVerifier {
async fn verify(
&self,
secret: &provider_openai::credential::CodexOAuthSecret,
expectation: &CodexIdentityExpectation,
) -> Result<CodexIdentityVerification, CodexIdentityVerificationError> {
let suffix = secret
.access_token
.expose_secret()
.strip_prefix("token-")
.ok_or(CodexIdentityVerificationError::Rejected)?;
let mut profile = profile(&format!("chatgpt-{suffix}"));
if let Some(user) = suffix.strip_prefix("shared-") {
profile.chatgpt_account_id = "chatgpt-shared".to_owned();
profile.chatgpt_user_id = format!("user-{user}");
}
if expectation
.chatgpt_account_id()
.is_some_and(|expected| expected != profile.chatgpt_account_id)
|| expectation
.chatgpt_user_id()
.is_some_and(|expected| expected != profile.chatgpt_user_id)
{
return Err(CodexIdentityVerificationError::Rejected);
}
Ok(CodexIdentityVerification::Complete(profile))
}
async fn verify_authorization(
&self,
_secret: &provider_openai::credential::CodexOAuthSecret,
_id_token: &SecretString,
_expected_nonce: &SecretString,
_expectation: &CodexIdentityExpectation,
) -> Result<CodexIdentityVerification, CodexIdentityVerificationError> {
Err(CodexIdentityVerificationError::Rejected)
}
}
pub(super) fn import_service(
refresher: Arc<ManualRefresher>,
) -> provider_openai::credential::CodexCredentialAdminService {
CodexCredentialAdminService::new(
refresher,
Arc::new(ImportVerifier),
Arc::new(ManualLeases {
requests: Mutex::new(Vec::new()),
drops: Arc::new(AtomicUsize::new(0)),
available: true,
}),
let service = CodexCredentialAdminService::new(
Arc::new(UnusedRefresher),
Arc::new(TestLeaseCoordinator::default()),
runtime_policy(),
)
}
pub(super) fn unused_import_refresher() -> Arc<ManualRefresher> {
Arc::new(ManualRefresher {
outcome: Mutex::new(Some(Err(RefreshFailure::InvalidGrant))),
seen: Mutex::new(Vec::new()),
})
}
#[tokio::test]
async fn oauth_import_uses_only_tokens_and_verified_identity() {
let refresher = unused_import_refresher();
let prepared = import_service(refresher.clone())
);
let prepared = service
.prepare_import_document(serde_json::json!({
"sourceFormat": "cpr",
"accounts": [{
"id": "acct_cpr_import",
"email": "cpr@example.com",
"accountId": "chatgpt-cpr",
"userId": "user-chatgpt-cpr",
"label": "CPR account",
"planType": "pro",
"token": "Bearer token-cpr",
"refreshToken": "refresh-cpr",
"accessTokenExpiresAt": "2100-01-01T00:00:00+00:00",
"status": "disabled",
"addedAt": "2026-07-18T10:47:01+08:00",
"updatedAt": "2026-07-19T11:00:00+08:00",
"unrelated": { "value": true }
}]
"token": access_token,
"refresh_token": "refresh-token"
}))
.await
.expect("CPR import");
assert_eq!(prepared.accounts().len(), 1);
let account = &prepared.accounts()[0];
assert!(account.account.id().as_str().starts_with("acct_"));
assert!(account.account.enabled());
assert_eq!(account.account.upstream_account_id(), Some("chatgpt-cpr"));
let runtime = CodexCredentialCodec::decode(&account.credential).expect("credential");
.expect("direct JWT import must be accepted");
let account = prepared.accounts().first().expect("one prepared account");
assert_eq!(
runtime
.authentication
.oauth()
.expect("OAuth credential")
.access_token
.expose_secret(),
"token-cpr"
account
.account
.access_token_expires_at()
.map(chrono::DateTime::<Utc>::from),
Some(expires_at)
);
assert!(refresher.seen.lock().expect("seen tokens").is_empty());
}
#[tokio::test]
async fn cpr_batch_allows_distinct_users_in_the_same_workspace() {
let prepared = import_service(unused_import_refresher())
.prepare_import_document(serde_json::json!({
"sourceFormat": "cpr",
"accounts": [{
"id": "acct_shared_a",
"accountId": "chatgpt-shared",
"userId": "user-a",
"token": "token-shared-a"
}, {
"id": "acct_shared_b",
"accountId": "chatgpt-shared",
"userId": "user-b",
"token": "token-shared-b"
}]
}))
.await
.expect("distinct users sharing one workspace are separate credentials");
assert_eq!(prepared.accounts().len(), 2);
assert!(
prepared
.accounts()
.iter()
.all(|account| { account.account.upstream_account_id() == Some("chatgpt-shared") })
);
let duplicate = import_service(unused_import_refresher())
.prepare_import_document(serde_json::json!({
"sourceFormat": "cpr",
"accounts": [{
"id": "acct_duplicate_a",
"accountId": "chatgpt-shared",
"userId": "user-a",
"token": "token-shared-a"
}, {
"id": "acct_duplicate_b",
"accountId": "chatgpt-shared",
"userId": "user-a",
"token": "token-shared-a"
}]
}))
.await
.expect_err("the same upstream user and workspace must stay unique");
assert_eq!(duplicate, CodexCredentialAdminError::InvalidInput);
}
#[tokio::test]
async fn oauth_import_extracts_tokens_without_parsing_source_projections() {
let bundle = import_service(unused_import_refresher())
.prepare_import_document(serde_json::json!({
"exported_at": "2026-07-03T15:46:38.717Z",
"proxies": [],
"accounts": [{
"name": "bundle@example.com",
"platform": "openai",
"type": "oauth",
"credentials": {
"access_token": "token-bundle",
"refresh_token": "refresh-bundle",
"chatgpt_account_id": "chatgpt-bundle",
"chatgpt_user_id": "user-chatgpt-bundle"
},
"concurrency": 3,
"priority": 50
}]
}))
.await
.expect("credential bundle import");
let auth_document = import_service(unused_import_refresher())
.prepare_import_document(serde_json::json!({
"accounts": [{
"type": "openai",
"access_token": "token-auth-document",
"refresh_token": "refresh-auth-document",
"chatgpt_account_id": "chatgpt-auth-document",
"chatgpt_user_id": "user-chatgpt-auth-document",
"email": "auth-document@example.com",
"label": "Auth document"
}]
}))
.await
.expect("auth document import");
for (prepared, expected) in [
(&bundle, "chatgpt-bundle"),
(&auth_document, "chatgpt-auth-document"),
] {
assert_eq!(prepared.accounts().len(), 1);
assert_eq!(
prepared.accounts()[0].account.upstream_account_id(),
Some(expected)
);
assert!(
prepared.accounts()[0]
.account
.id()
.as_str()
.starts_with("acct_")
);
}
assert!(!format!("{bundle:?} {auth_document:?}").contains("refresh-bundle"));
assert!(!format!("{bundle:?} {auth_document:?}").contains("refresh-auth-document"));
}
#[tokio::test]
async fn cliproxyapi_codex_auth_file_is_recognized_as_an_openai_auth_document() {
let prepared = import_service(unused_import_refresher())
.prepare_import_document(serde_json::json!({
"type": "codex",
"access_token": "token-cpa",
"refresh_token": "refresh-cpa",
"id_token": "id-cpa",
"account_id": "chatgpt-cpa",
"email": "cpa@example.com",
"expired": "2100-01-01T00:00:00Z"
}))
.await
.expect("CLIProxyAPI Codex auth file import");
assert_eq!(prepared.accounts().len(), 1);
let account = &prepared.accounts()[0].account;
assert_eq!(account.upstream_account_id(), Some("chatgpt-cpa"));
assert_eq!(account.email(), Some("chatgpt-cpa@example.com"));
assert_eq!(account.authentication_kind(), "oauth");
assert!(account.account.next_refresh_at().is_none());
}
@@ -24,7 +24,6 @@ use reqwest::StatusCode;
use secrecy::ExposeSecret as _;
use serde_json::Value;
use super::admin::{import_service, unused_import_refresher};
use crate::support::MemoryAccountStore;
fn signing_fixture(seed: u8) -> (SigningKey, String) {
@@ -56,7 +55,7 @@ fn agent_account(
account_id,
provider,
"Agent Identity test".to_owned(),
"agent-user".to_owned(),
Some("agent-user".to_owned()),
CODEX_AUTHENTICATION_KIND_AGENT_IDENTITY.to_owned(),
CredentialRevision::new(1).expect("revision"),
None,
@@ -232,29 +231,3 @@ async fn task_registration_and_recovery_are_revision_fenced() {
assert_eq!(fenced.account.revision().get(), 3);
assert_eq!(registrar.calls.load(Ordering::SeqCst), 2);
}
#[tokio::test]
async fn agent_identity_import_accepts_the_minimal_document_shape() {
let prepared = import_service(unused_import_refresher())
.prepare_import_document(serde_json::json!({
"auth_mode": "agentIdentity",
"agent_identity": {
"agent_runtime_id": "runtime-minimal",
"agent_private_key": signing_fixture(9).1,
"account_id": "chatgpt-minimal",
"chatgpt_user_id": "user-minimal",
"email": "minimal@example.com",
"plan_type": "pro"
}
}))
.await
.expect("minimal Agent Identity import");
assert_eq!(prepared.accounts().len(), 1);
let account = &prepared.accounts()[0].account;
assert_eq!(
account.authentication_kind(),
CODEX_AUTHENTICATION_KIND_AGENT_IDENTITY
);
assert_eq!(account.access_token_expires_at(), None);
assert!(!format!("{prepared:?}").contains("agent_private_key"));
}
@@ -4,15 +4,21 @@ use std::sync::Arc;
use std::time::{Duration, SystemTime};
use futures::executor::block_on;
use gateway_core::engine::continuation::{
ContinuationBinding, NativeContinuationPin, PreviousResponseId,
};
use gateway_core::engine::credential::{
AccountAttemptFeedback, AccountAvailability, AccountFeedbackStats, AccountSelectionPolicy,
AccountStateChange, ProviderAccountId, ProviderAccountStore as _, RotationStrategy,
AccountStateChange, OpaqueProviderData, ProviderAccountId, ProviderAccountStore as _,
QuotaObservation, QuotaWriteOutcome, RotationStrategy,
};
use gateway_core::engine::{
AccountAttemptContext, AttemptContext, CancellationToken, ModelRequestId, RequestAttemptContext,
};
use gateway_core::policy::ClientApiKeyId;
use gateway_core::provider_ports::{ProviderSessionAffinityKey, ProviderSessionAffinityPort};
use gateway_core::provider_ports::{
ProviderCooldownPort, ProviderSessionAffinityKey, ProviderSessionAffinityPort,
};
use gateway_core::routing::ProviderKind;
use provider_openai::OFFICIAL_CODEX_BASE_URL;
use provider_openai::credential::{
@@ -22,11 +28,12 @@ use provider_openai::credential::{
};
use provider_openai::transport::profile::{CodexWireProfile, CodexWireProfileState};
use secrecy::ExposeSecret;
use serde_json::json;
use url::Url;
use crate::support::{
MemoryAccountStore, MemorySessionAffinity, MemorySessionExclusions, TestLeaseCoordinator,
account_policy, agent_identity_service, catalog_cache, profile, secret,
MemoryAccountStore, MemoryCooldownPort, MemorySessionAffinity, MemorySessionExclusions,
TestLeaseCoordinator, account_policy, agent_identity_service, catalog_cache, profile, secret,
};
fn create_account(store: &Arc<MemoryAccountStore>, id: &str, token: &str) {
@@ -99,6 +106,7 @@ fn selector_with_affinity(
session_affinity,
Arc::new(AccountFeedbackStats::default()),
true,
Arc::new(MemoryCooldownPort::new()),
)
}
@@ -108,6 +116,7 @@ fn selector_with_runtime(
session_affinity: Arc<MemorySessionAffinity>,
account_feedback: Arc<AccountFeedbackStats>,
skip_exhausted: bool,
cooldowns: Arc<dyn ProviderCooldownPort>,
) -> CodexCredentialSelector {
let profile = CodexWireProfileState::new(CodexWireProfile {
originator: "codex_cli_rs".to_owned(),
@@ -136,6 +145,7 @@ fn selector_with_runtime(
http,
OFFICIAL_CODEX_BASE_URL.to_owned(),
Arc::clone(&agent_identity),
cooldowns,
));
CodexCredentialSelector::new(
ProviderKind::new("openai").expect("provider"),
@@ -224,7 +234,7 @@ fn codec_reimport_preserves_existing_installation_id_for_the_same_principal() {
}
#[test]
fn codec_reimport_rejects_installation_reuse_across_principals() {
fn codec_reimport_preserves_installation_id_without_principal_validation() {
let existing = CodexCredentialCodec::encode_new(
&secret("existing-access-token"),
&profile("chatgpt-existing-principal"),
@@ -238,7 +248,19 @@ fn codec_reimport_rejects_installation_reuse_across_principals() {
)
.expect("incoming credential");
assert!(CodexCredentialCodec::preserve_installation_id(&incoming, &existing).is_err());
let existing_id = CodexCredentialCodec::decode_complete(&existing)
.expect("existing data")
.installation_id()
.to_owned();
let preserved = CodexCredentialCodec::preserve_installation_id(&incoming, &existing)
.expect("preserve installation ID");
let preserved = CodexCredentialCodec::decode_complete(&preserved).expect("preserved data");
assert_eq!(preserved.installation_id(), existing_id);
assert_eq!(
preserved.oauth().expect("OAuth data").access_token,
"incoming-access-token"
);
}
#[test]
@@ -464,6 +486,7 @@ async fn selector_should_keep_a_schedulable_affinity_account_despite_soft_health
affinity,
Arc::clone(&account_feedback),
true,
Arc::new(MemoryCooldownPort::new()),
);
for _ in 0..4 {
account_feedback.report(
@@ -507,6 +530,7 @@ async fn selector_should_escape_a_quota_exhausted_affinity_account() {
let first = store.account("acct_first").expect("first account");
store
.apply_state_change(AccountStateChange {
message: None,
account_id: first.id().clone(),
expected_revision: first.revision(),
availability: AccountAvailability::QuotaExhausted,
@@ -663,8 +687,12 @@ fn credential_expired_failure_marks_unified_account_expired() {
}),
)
.expect("select account");
block_on(selector.record_failure(lease.account(), CodexAccountFailure::CredentialExpired))
.expect("record credential expiry");
block_on(selector.record_failure(
lease.account(),
CodexAccountFailure::CredentialExpired,
None,
))
.expect("record credential expiry");
assert_eq!(
store
.account("acct_primary")
@@ -672,13 +700,58 @@ fn credential_expired_failure_marks_unified_account_expired() {
.availability(),
AccountAvailability::Expired
);
assert_eq!(
store.last_error_message("acct_primary").as_deref(),
Some("upstream_credential_expired")
);
}
#[test]
fn credential_expired_failure_keeps_expired_oauth_for_bounded_refresh_recovery() {
let store = Arc::new(MemoryAccountStore::default());
let mut expired_profile = profile("chatgpt-acct_primary");
expired_profile.access_token_expires_at =
Some(chrono::Utc::now() - chrono::Duration::minutes(1));
block_on(store.seed_oauth_credential(ImportCodexOAuthCredential {
account_id: "acct_primary".to_owned(),
name: "acct_primary".to_owned(),
secret: secret("at-primary"),
verified_account: expired_profile,
next_refresh_at: Some(chrono::Utc::now() + chrono::Duration::minutes(10)),
enabled: true,
}));
let account = store.account("acct_primary").expect("account");
let selector = selector(&store, Arc::new(TestLeaseCoordinator::default()));
block_on(selector.record_failure(
&account,
CodexAccountFailure::CredentialExpired,
Some("token_expired".to_owned()),
))
.expect("record credential expiry");
assert_eq!(
store
.account("acct_primary")
.expect("account retained for refresh")
.availability(),
AccountAvailability::Ready
);
}
#[test]
fn rate_limited_failure_records_quota_window_without_changing_availability() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_primary", "at-primary");
let selector = selector(&store, Arc::new(TestLeaseCoordinator::default()));
let cooldowns = Arc::new(MemoryCooldownPort::new());
let selector = selector_with_runtime(
&store,
Arc::new(TestLeaseCoordinator::default()),
Arc::new(MemorySessionAffinity::default()),
Arc::new(AccountFeedbackStats::default()),
true,
Arc::clone(&cooldowns) as Arc<dyn ProviderCooldownPort>,
);
let attempt = attempt(BTreeSet::new());
let lease =
block_on(
@@ -697,36 +770,22 @@ fn rate_limited_failure_records_quota_window_without_changing_availability() {
CodexAccountFailure::RateLimited {
retry_after: Some(Duration::from_secs(30)),
},
None,
))
.expect("record rate-limit failure");
// 429 不改变 availability(限流不改变账号可用性),只把限流窗口合成进 quota JSON。
// 429 不改变 availability(限流不改变账号可用性),也不污染 quota JSON——
// 临时限流写入 Redis 冷却(到期自动解除),额度重置时间保持不变。
let account = store.account("acct_primary").expect("account");
assert_eq!(account.availability(), AccountAvailability::Ready);
let quota = store.quota_json("acct_primary").expect("quota");
assert_eq!(
quota
.pointer("/rate_limit/limit_reached")
.and_then(serde_json::Value::as_bool),
Some(true)
);
assert_eq!(
quota
.pointer("/rate_limit/primary_window/used_percent")
.and_then(serde_json::Value::as_f64),
Some(100.0)
);
assert!(
quota
.pointer("/rate_limit/primary_window/reset_at")
.and_then(serde_json::Value::as_i64)
.is_some_and(|reset_at| {
reset_at
> SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.expect("clock")
.as_secs() as i64
})
store.quota_json("acct_primary").is_none(),
"429 must not synthesize quota window"
);
let cooldown = block_on(cooldowns.read(account.id())).expect("read cooldown");
assert!(
cooldown.is_some_and(|cooling| cooling.until() > SystemTime::now()),
"429 must record a Redis cooldown expiring in the future"
);
}
@@ -743,6 +802,7 @@ async fn usage_limit_exhaustion_marks_quota_exhausted_without_usage_probe() {
CodexAccountFailure::UsageLimitExhausted {
retry_after: Some(Duration::from_secs(30)),
},
None,
)
.await
.expect("record usage-limit exhaustion");
@@ -758,6 +818,7 @@ fn rate_limited_failure_does_not_downgrade_persisted_quota_exhaustion() {
create_account(&store, "acct_primary", "at-primary");
let account = store.account("acct_primary").expect("account");
block_on(store.apply_state_change(AccountStateChange {
message: None,
account_id: account.id().clone(),
expected_revision: account.revision(),
availability: AccountAvailability::QuotaExhausted,
@@ -771,6 +832,7 @@ fn rate_limited_failure_does_not_downgrade_persisted_quota_exhaustion() {
CodexAccountFailure::RateLimited {
retry_after: Some(Duration::from_secs(30)),
},
None,
))
.expect("record rate-limit failure");
@@ -789,13 +851,22 @@ fn rate_limited_failure_does_not_consult_stale_quota_snapshot() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_primary", "at-primary");
let account = store.account("acct_primary").expect("account");
let selector = selector(&store, Arc::new(TestLeaseCoordinator::default()));
let cooldowns = Arc::new(MemoryCooldownPort::new());
let selector = selector_with_runtime(
&store,
Arc::new(TestLeaseCoordinator::default()),
Arc::new(MemorySessionAffinity::default()),
Arc::new(AccountFeedbackStats::default()),
true,
Arc::clone(&cooldowns) as Arc<dyn ProviderCooldownPort>,
);
block_on(selector.record_failure(
&account,
CodexAccountFailure::RateLimited {
retry_after: Some(Duration::from_secs(30)),
},
None,
))
.expect("record rate-limit failure");
@@ -806,8 +877,13 @@ fn rate_limited_failure_does_not_consult_stale_quota_snapshot() {
.availability(),
AccountAvailability::Ready
);
// 429 合成限流窗口需要读写 quota JSON。
assert!(store.quota_reads() >= 1);
// 429 临时限流写入 Redis 冷却,不读写 quota JSON。
assert_eq!(store.quota_reads(), 0);
let cooldown = block_on(cooldowns.read(account.id())).expect("read cooldown");
assert!(
cooldown.is_some_and(|cooling| cooling.until() > SystemTime::now()),
"429 must record a Redis cooldown"
);
}
#[test]
@@ -816,6 +892,7 @@ fn rate_limited_failure_does_not_overwrite_stale_authentication_state() {
create_account(&store, "acct_primary", "at-primary");
let account = store.account("acct_primary").expect("account");
block_on(store.apply_state_change(AccountStateChange {
message: None,
account_id: account.id().clone(),
expected_revision: account.revision(),
availability: AccountAvailability::Invalid,
@@ -830,6 +907,7 @@ fn rate_limited_failure_does_not_overwrite_stale_authentication_state() {
CodexAccountFailure::RateLimited {
retry_after: Some(Duration::from_secs(30)),
},
None,
))
.expect("record rate-limit failure");
@@ -854,6 +932,7 @@ fn successful_upstream_response_recovers_non_quota_terminal_states() {
create_account(&store, "acct_primary", "at-primary");
let account = store.account("acct_primary").expect("account");
block_on(store.apply_state_change(AccountStateChange {
message: None,
account_id: account.id().clone(),
expected_revision: account.revision(),
availability: stale,
@@ -892,12 +971,14 @@ fn rate_limited_failures_for_distinct_accounts_do_not_conflict() {
CodexAccountFailure::RateLimited {
retry_after: Some(Duration::from_secs(30)),
},
None,
),
selector.record_failure(
&second,
CodexAccountFailure::RateLimited {
retry_after: Some(Duration::from_secs(30)),
},
None,
),
);
first_result.expect("record first account failure");
@@ -925,7 +1006,7 @@ fn selector_keeps_a_quota_exhausted_account_eligible_when_configured_not_to_skip
create_account(&store, "acct_primary", "at-primary");
let strict_selector = selector(&store, Arc::new(TestLeaseCoordinator::default()));
let account = store.account("acct_primary").expect("account");
block_on(strict_selector.record_failure(&account, CodexAccountFailure::QuotaExhausted))
block_on(strict_selector.record_failure(&account, CodexAccountFailure::QuotaExhausted, None))
.expect("mark account exhausted");
let selector = selector_with_runtime(
@@ -934,6 +1015,7 @@ fn selector_keeps_a_quota_exhausted_account_eligible_when_configured_not_to_skip
Arc::new(MemorySessionAffinity::default()),
Arc::new(AccountFeedbackStats::default()),
false,
Arc::new(MemoryCooldownPort::new()),
);
let request_url =
Url::parse("https://chatgpt.com/backend-api/codex/responses").expect("request URL");
@@ -957,6 +1039,131 @@ fn selector_keeps_a_quota_exhausted_account_eligible_when_configured_not_to_skip
);
}
#[test]
fn native_continuation_surfaces_the_original_accounts_quota_status_to_the_coordinator() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_original", "at-original");
create_account(&store, "acct_fallback", "at-fallback");
let strict_selector = selector(&store, Arc::new(TestLeaseCoordinator::default()));
let original = store.account("acct_original").expect("original account");
block_on(strict_selector.record_failure(&original, CodexAccountFailure::QuotaExhausted, None))
.expect("mark original account exhausted");
let selector = selector_with_runtime(
&store,
Arc::new(TestLeaseCoordinator::default()),
Arc::new(MemorySessionAffinity::default()),
Arc::new(AccountFeedbackStats::default()),
false,
Arc::new(MemoryCooldownPort::new()),
);
let continuation = NativeContinuationPin::new(
PreviousResponseId::new("previous-response"),
PreviousResponseId::new("upstream-response"),
ProviderKind::new("openai").expect("provider"),
original.id().clone(),
);
let attempt = AttemptContext::new(
RequestAttemptContext::new(
ModelRequestId::new("req_native_continuation").expect("request id"),
ClientApiKeyId::new("key_codex_contract").expect("client key id"),
),
NonZeroU32::new(1).expect("attempt"),
SystemTime::now() + Duration::from_secs(30),
account_policy(),
AccountAttemptContext::new(BTreeSet::new(), None, None),
Some(ContinuationBinding::Pinned(continuation)),
CancellationToken::new(),
);
let error =
block_on(
selector.select(&SelectCodexCredential {
upstream_model: "gpt-5.4",
request_url: &Url::parse("https://chatgpt.com/backend-api/codex/responses")
.expect("request URL"),
attempt: &attempt,
session_affinity_key: None,
}),
)
.expect_err("the selector must surface the unavailable native account");
assert!(matches!(
error,
CredentialSelectionError::NoEligibleCredential
));
}
#[test]
fn native_continuation_surfaces_the_original_accounts_quota_signal_to_the_coordinator() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_original_signal", "at-original-signal");
create_account(&store, "acct_fallback_signal", "at-fallback-signal");
let original = store
.account("acct_original_signal")
.expect("original account");
let quota = json!({
"rate_limit": {
"allowed": false,
"limit_reached": true,
"primary_window": {"used_percent": 98}
}
});
let outcome = block_on(store.compare_and_swap_quota(QuotaObservation {
account_id: original.id().clone(),
expected_revision: original.revision(),
quota: Some(OpaqueProviderData::new(
quota.as_object().expect("quota object").clone(),
)),
observed_at: Some(SystemTime::now()),
limit_reached: None,
}))
.expect("persist quota signal");
assert!(matches!(outcome, QuotaWriteOutcome::Updated));
let selector = selector_with_runtime(
&store,
Arc::new(TestLeaseCoordinator::default()),
Arc::new(MemorySessionAffinity::default()),
Arc::new(AccountFeedbackStats::default()),
false,
Arc::new(MemoryCooldownPort::new()),
);
let continuation = NativeContinuationPin::new(
PreviousResponseId::new("previous-response"),
PreviousResponseId::new("upstream-response"),
ProviderKind::new("openai").expect("provider"),
original.id().clone(),
);
let attempt = AttemptContext::new(
RequestAttemptContext::new(
ModelRequestId::new("req_native_continuation_signal").expect("request id"),
ClientApiKeyId::new("key_codex_contract").expect("client key id"),
),
NonZeroU32::new(1).expect("attempt"),
SystemTime::now() + Duration::from_secs(30),
account_policy(),
AccountAttemptContext::new(BTreeSet::new(), None, None),
Some(ContinuationBinding::Pinned(continuation)),
CancellationToken::new(),
);
let error =
block_on(
selector.select(&SelectCodexCredential {
upstream_model: "gpt-5.4",
request_url: &Url::parse("https://chatgpt.com/backend-api/codex/responses")
.expect("request URL"),
attempt: &attempt,
session_affinity_key: None,
}),
)
.expect_err("the selector must surface the quota-limited native account");
assert!(matches!(
error,
CredentialSelectionError::NoEligibleCredential
));
}
#[test]
fn identity_verification_failure_isolates_only_selected_account() {
let store = Arc::new(MemoryAccountStore::default());
@@ -979,6 +1186,7 @@ fn identity_verification_failure_isolates_only_selected_account() {
block_on(selector.record_failure(
lease.account(),
CodexAccountFailure::IdentityVerificationRequired,
None,
))
.expect("record identity verification failure");
@@ -1022,6 +1230,7 @@ fn cloudflare_challenge_does_not_change_availability() {
block_on(selector.record_failure(
lease.account(),
CodexAccountFailure::CloudflareChallenge { retry_after: None },
None,
))
.expect("record challenge");
@@ -1058,9 +1267,11 @@ fn repeated_cloudflare_path_block_marks_only_the_affected_account_invalid() {
.expect("select account");
for _ in 0..3 {
block_on(
selector.record_failure(lease.account(), CodexAccountFailure::CloudflarePathBlocked),
)
block_on(selector.record_failure(
lease.account(),
CodexAccountFailure::CloudflarePathBlocked,
None,
))
.expect("record path block");
}
@@ -1114,6 +1325,7 @@ fn cloudflare_challenge_expires_provider_owned_cookies_at_cooldown_boundary() {
block_on(selector.record_failure(
second.account(),
CodexAccountFailure::CloudflareChallenge { retry_after: None },
None,
))
.expect("record challenge");
@@ -1157,8 +1369,12 @@ fn cloudflare_path_block_deletes_provider_owned_cookies() {
session_affinity_key: None,
}))
.expect("select revised account");
block_on(selector.record_failure(second.account(), CodexAccountFailure::CloudflarePathBlocked))
.expect("record path block");
block_on(selector.record_failure(
second.account(),
CodexAccountFailure::CloudflarePathBlocked,
None,
))
.expect("record path block");
let account = store.account("acct_primary").expect("account");
let data = block_on(store.repository().load_complete_data(&account)).expect("credential data");
@@ -1194,7 +1410,7 @@ fn response_cookie_rotation_returns_a_current_account_for_later_fenced_writes()
assert_eq!(outcome.credential_revision, Some(current.revision().get()));
assert_ne!(current.revision(), lease.account().revision());
block_on(selector.record_failure(&current, CodexAccountFailure::QuotaExhausted))
block_on(selector.record_failure(&current, CodexAccountFailure::QuotaExhausted, None))
.expect("record failure with current revision");
assert_eq!(
store
@@ -1,629 +0,0 @@
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use async_trait::async_trait;
use jsonwebtoken::{Algorithm, EncodingKey, Header, encode, get_current_timestamp};
use provider_openai::credential::token_client::OFFICIAL_CODEX_OAUTH_CLIENT_ID;
use provider_openai::credential::{
CodexAccountIdentityService, CodexAccountIdentityVerifier, CodexAuthenticatedAccount,
CodexAuthenticatedAccountSource, CodexIdentityExpectation, CodexIdentityVerification,
CodexIdentityVerificationError, CodexJwksSource, CodexJwtIdentityVerifier, CodexOAuthSecret,
CodexSignedIdentityVerifier, OFFICIAL_OPENAI_API_AUDIENCE, OFFICIAL_OPENAI_ISSUER,
ReqwestOpenAiJwksSource,
};
use secrecy::{ExposeSecret, SecretString};
use serde::Deserialize;
use serde_json::{Value, json};
const KEY_ID: &str = "codex-test-key";
const RSA_MODULUS: &str = "p6GpjR8xbWJF2X63CJL02fTHBmfhfEGSqckUQaAT7KYubw95XhpwQ-V2IQcMm8K-ckx4hTd1CfqPjz0xgZzFRxns2BD_lUWYazDrsNISW23Y25rhg8GoC5ad9msdJlEARgyEFA2WQxqKg9KDpV8aZmVOSUkUjujF7VPijQ63ogNzSxAhBPbctSIzhcJApO6KV97j4OizhynWxJtCzmCEsxVvoLVNOXWLTJ0JW3ogs7dY8Unf3PTkci7QQhslJRSaCuy_6HWpP-Z94ayfKyo8nhz0PQsXvTl3y_8ARtwOT9sywmrd8nkTZ1obi5QC3DUZLHtenIk--rJLhvGn0klevw";
const RSA_EXPONENT: &str = "AQAB";
const PRIVATE_KEY: &str = r#"-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQCnoamNHzFtYkXZ
frcIkvTZ9McGZ+F8QZKpyRRBoBPspi5vD3leGnBD5XYhBwybwr5yTHiFN3UJ+o+P
PTGBnMVHGezYEP+VRZhrMOuw0hJbbdjbmuGDwagLlp32ax0mUQBGDIQUDZZDGoqD
0oOlXxpmZU5JSRSO6MXtU+KNDreiA3NLECEE9ty1IjOFwkCk7opX3uPg6LOHKdbE
m0LOYISzFW+gtU05dYtMnQlbeiCzt1jxSd/c9ORyLtBCGyUlFJoK7L/odak/5n3h
rJ8rKjyeHPQ9Cxe9OXfL/wBG3A5P2zLCat3yeRNnWhuLlALcNRkse16ciT76skuG
8afSSV6/AgMBAAECggEATnDEkUfWbjP9MYAtD/MMZm03MJUtm2JnydFRDEBm8mb0
9X1NMq62d98I/v14de6UBZIpF1VTDB9WTBW7c8YzPkkwXg7eOjB92TLkJzK16lil
kGWSP4BALWQGeMe4tTRUJ3M13thYxv/Fb1fzyE+PX/jpwDtVRmZAMAT/zzfi0AmB
ZoVcN0ipr4cd8xq6SEk3rLz4UqstX4qZ58U0Yqhi9q5YVzMrwV4GXHf80zaq1DYn
m5Unddq1Igs0HDjTbHtY0jyQW3IK1fNr0DLvmw52AeQD/xYWB83EWDPITrMFuNa4
GYEGlnawalLyteh1/mgJZ/F/gqNcWNefkN5FDqtXIQKBgQDR/nriVyk12QXzDR91
7CAJRrEMl3QlvRgfqdVYveiC7osvM8MHQzE+yEfZYTMvOhBs4J5QH0bIfcLiGgeK
nyXnDqJx+gJr4dYOX2Ma/DT1wghItPyxCFsbGvL0c0LDewaL3NcQyPIFip6O2f2Z
jjlrHnXThbtcByH2y9i/oVWJ0QKBgQDMW0bpcXNBIfllZDQOUl4PfPnxTS67qZjC
3QpqHqdKqNs7nNutJNLuOaJig14stjTm5EdliwZPvOPAnbjEOdP5vG1GxwD3GKRt
G6MdPT2TdKCHaPOv/gE6tU2nt2wk7OsmTE20aDd8Czwc3l8ok8/KAM7YXOjbsg8y
StacIrDzjwKBgGD89Uqm0RzSwKGyVO6FAYLnSIy1QurPPF4bdbNH+yTGAjjp4lDv
YsZJgI3RC+/rFH0E/XmqCGo/U4xUU1leCgZ+xt53hzjGlLQMsFxdUiST2nmiRfeK
EXIib0YiGhrpLmvYsuhItyeCD5uQ6UVL4A8ugWMRqChoDvLK0bAoVraRAoGBAKig
EO2EHfR60k4l1waSVOc22w+P/qX/lfwFZRiX3rcuimiwUFyY7CyDBkl/2O/QEesM
JxXVGAon9U71VockqJOENi+W+mbqwJL/oSL5a5wHOodBxQNN9zm4bTGdmvEFRiw9
/kiFTnNe2eWAYMkc3vLyo7vJPqQ6U2vOcTQ5NAZlAoGBAMBeXrXAO/+cAUt41oVE
Qoul7TG+QFTN4A650QVbHQZsSDsinLzgPQ8TLLFtb180OptXSvIR+OGfc0E9JcJe
xDWp3FkNixBAikmMzlBqq39+numIilltPwT4RLib4OP6x8gZZHUB/+3HIYmnxk/G
b9ZuffSh1GPmnXSyKb7rtg6M
-----END PRIVATE KEY-----"#;
struct StaticJwksSource {
body: Vec<u8>,
calls: Arc<AtomicUsize>,
error: Option<CodexIdentityVerificationError>,
}
#[async_trait]
impl CodexJwksSource for StaticJwksSource {
async fn fetch(&self) -> Result<Vec<u8>, CodexIdentityVerificationError> {
self.calls.fetch_add(1, Ordering::SeqCst);
if let Some(error) = self.error {
return Err(error);
}
Ok(self.body.clone())
}
}
struct StaticAccountSource {
result: Result<CodexAuthenticatedAccount, CodexIdentityVerificationError>,
}
#[async_trait]
impl CodexAuthenticatedAccountSource for StaticAccountSource {
async fn fetch(
&self,
_secret: &CodexOAuthSecret,
_expectation: &CodexIdentityExpectation,
) -> Result<CodexAuthenticatedAccount, CodexIdentityVerificationError> {
self.result.clone()
}
}
fn verifier() -> (CodexJwtIdentityVerifier, Arc<AtomicUsize>) {
let calls = Arc::new(AtomicUsize::new(0));
let body = serde_json::to_vec(&json!({
"keys": [{
"kty": "RSA",
"use": "sig",
"alg": "RS256",
"kid": KEY_ID,
"n": RSA_MODULUS,
"e": RSA_EXPONENT
}]
}))
.expect("serialize test JWKS");
let verifier = CodexJwtIdentityVerifier::new(Box::new(StaticJwksSource {
body,
calls: Arc::clone(&calls),
error: None,
}));
(verifier, calls)
}
fn claims(exp: u64) -> Value {
let now = get_current_timestamp();
json!({
"iss": OFFICIAL_OPENAI_ISSUER,
"aud": [OFFICIAL_OPENAI_API_AUDIENCE],
"iat": now.saturating_sub(1),
"nbf": now.saturating_sub(1),
"exp": exp,
"sub": "subject-signed",
"https://api.openai.com/auth": {
"chatgpt_account_id": "account-signed",
"chatgpt_user_id": "user-signed",
"user_id": "user-signed",
"chatgpt_plan_type": "plus"
},
"https://api.openai.com/profile": {
"email": "signed@example.invalid"
}
})
}
fn token(claims: &Value) -> String {
let mut header = Header::new(Algorithm::RS256);
header.kid = Some(KEY_ID.to_owned());
header.typ = Some("JWT".to_owned());
encode(
&header,
claims,
&EncodingKey::from_rsa_pem(PRIVATE_KEY.as_bytes()).expect("parse signing key"),
)
.expect("sign test access token")
}
fn secret(claims: &Value, refresh_token: Option<&str>) -> CodexOAuthSecret {
CodexOAuthSecret {
access_token: SecretString::new(token(claims).into()),
refresh_token: refresh_token.map(|value| SecretString::new(value.to_owned().into())),
id_token: None,
}
}
pub(crate) async fn signed_identity_fixture() -> provider_openai::credential::CodexSignedIdentity {
let (verifier, _) = verifier();
verifier
.verify_access(&secret(
&claims(get_current_timestamp() + 3_600),
Some("refresh-token"),
))
.await
.expect("signed identity fixture")
}
fn id_token_claims(nonce: &str, subject: &str) -> Value {
let now = get_current_timestamp();
json!({
"iss": OFFICIAL_OPENAI_ISSUER,
"aud": [OFFICIAL_CODEX_OAUTH_CLIENT_ID],
"iat": now.saturating_sub(1),
"nbf": now.saturating_sub(1),
"exp": now + 3_600,
"sub": subject,
"nonce": nonce
})
}
#[tokio::test]
async fn verifier_should_derive_identity_only_from_valid_signed_claims() {
let (verifier, calls) = verifier();
let profile = verifier
.verify_access(&secret(
&claims(get_current_timestamp() + 3_600),
Some("refresh-token"),
))
.await
.expect("verify signed Codex token");
assert_eq!(profile.oauth_subject(), "subject-signed");
assert_eq!(profile.poid(), None);
assert_eq!(profile.claimed_account_id(), Some("account-signed"));
assert_eq!(profile.claimed_user_id(), Some("user-signed"));
assert_eq!(calls.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn verifier_should_accept_current_signed_poid_account_identity() {
let (verifier, _) = verifier();
let mut current = claims(get_current_timestamp() + 3_600);
current["https://api.openai.com/auth"]
.as_object_mut()
.expect("auth claims")
.remove("chatgpt_account_id");
current["https://api.openai.com/auth"]["poid"] = json!("organization-signed");
let profile = verifier
.verify_access(&secret(&current, Some("refresh-token")))
.await
.expect("verify current signed Codex token");
assert_eq!(profile.oauth_subject(), "subject-signed");
assert_eq!(profile.poid(), Some("organization-signed"));
assert_eq!(profile.claimed_account_id(), None);
}
#[tokio::test]
async fn account_identity_requires_matching_authenticated_usage_facts() {
let (signed, _) = verifier();
let service = CodexAccountIdentityService::new(
Arc::new(signed),
Arc::new(StaticAccountSource {
result: Ok(CodexAuthenticatedAccount {
chatgpt_account_id: "account-signed".to_owned(),
chatgpt_user_id: "user-signed".to_owned(),
email: Some("usage@example.invalid".to_owned()),
plan_type: Some("pro".to_owned()),
}),
}),
);
let verification = service
.verify(
&secret(&claims(get_current_timestamp() + 3_600), None),
&CodexIdentityExpectation::imported(
Some("account-signed".to_owned()),
Some("user-signed".to_owned()),
)
.expect("import expectation"),
)
.await
.expect("complete authenticated identity");
let CodexIdentityVerification::Complete(profile) = verification else {
panic!("usage facts must complete the identity");
};
assert_eq!(profile.oauth_subject, "subject-signed");
assert_eq!(profile.chatgpt_account_id, "account-signed");
assert_eq!(profile.chatgpt_user_id, "user-signed");
assert_eq!(profile.email.as_deref(), Some("usage@example.invalid"));
}
#[tokio::test]
async fn account_identity_rejects_imported_workspace_conflict() {
let (signed, _) = verifier();
let service = CodexAccountIdentityService::new(
Arc::new(signed),
Arc::new(StaticAccountSource {
result: Ok(CodexAuthenticatedAccount {
chatgpt_account_id: "account-signed".to_owned(),
chatgpt_user_id: "user-signed".to_owned(),
email: None,
plan_type: None,
}),
}),
);
let error = service
.verify(
&secret(&claims(get_current_timestamp() + 3_600), None),
&CodexIdentityExpectation::imported(Some("other-account".to_owned()), None)
.expect("import expectation"),
)
.await
.expect_err("document workspace cannot override authenticated usage");
assert_eq!(error, CodexIdentityVerificationError::Rejected);
}
#[tokio::test]
async fn import_identity_uses_signed_claims_when_usage_rejects_the_request() {
let (signed, _) = verifier();
let service = CodexAccountIdentityService::new(
Arc::new(signed),
Arc::new(StaticAccountSource {
result: Err(CodexIdentityVerificationError::Rejected),
}),
);
let profile = service
.verify_import(&secret(
&claims(get_current_timestamp() + 3_600),
Some("refresh-token"),
))
.await
.expect("signed import identity");
assert_eq!(profile.chatgpt_account_id, "account-signed");
assert_eq!(profile.chatgpt_user_id, "user-signed");
}
#[tokio::test]
async fn import_identity_keeps_signed_workspace_when_usage_returns_a_different_workspace() {
let (signed, _) = verifier();
let service = CodexAccountIdentityService::new(
Arc::new(signed),
Arc::new(StaticAccountSource {
result: Ok(CodexAuthenticatedAccount {
chatgpt_account_id: "account-default".to_owned(),
chatgpt_user_id: "user-default".to_owned(),
email: None,
plan_type: None,
}),
}),
);
let profile = service
.verify_import(&secret(
&claims(get_current_timestamp() + 3_600),
Some("refresh-token"),
))
.await
.expect("signed import identity");
assert_eq!(profile.chatgpt_account_id, "account-signed");
assert_eq!(profile.chatgpt_user_id, "user-signed");
}
#[tokio::test]
async fn authorization_identity_uses_signed_workspace_when_usage_returns_default_workspace() {
let (signed, _) = verifier();
let service = CodexAccountIdentityService::new(
Arc::new(signed),
Arc::new(StaticAccountSource {
result: Ok(CodexAuthenticatedAccount {
chatgpt_account_id: "account-default".to_owned(),
chatgpt_user_id: "user-default".to_owned(),
email: None,
plan_type: None,
}),
}),
);
let nonce = SecretString::from("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA");
let profile = service
.verify_authorization(
&secret(
&claims(get_current_timestamp() + 3_600),
Some("refresh-token"),
),
&SecretString::from(token(&id_token_claims(
nonce.expose_secret(),
"subject-signed",
))),
&nonce,
&CodexIdentityExpectation::reauthorization(
"account-signed".to_owned(),
"user-signed".to_owned(),
"00000000-0000-4000-8000-000000000001".to_owned(),
)
.expect("reauthorization expectation"),
)
.await
.expect("signed authorization identity")
.into_complete()
.expect("complete signed profile");
assert_eq!(profile.chatgpt_account_id, "account-signed");
assert_eq!(profile.chatgpt_user_id, "user-signed");
}
#[tokio::test]
async fn authorization_identity_accepts_free_token_without_signed_account_id() {
let (signed, _) = verifier();
let service = CodexAccountIdentityService::new(
Arc::new(signed),
Arc::new(StaticAccountSource {
result: Ok(CodexAuthenticatedAccount {
chatgpt_account_id: "account-signed".to_owned(),
chatgpt_user_id: "user-signed".to_owned(),
email: Some("usage@example.invalid".to_owned()),
plan_type: Some("free".to_owned()),
}),
}),
);
let mut free_claims = claims(get_current_timestamp() + 3_600);
free_claims["https://api.openai.com/auth"]
.as_object_mut()
.expect("auth claims")
.remove("chatgpt_account_id");
free_claims["https://api.openai.com/auth"]["poid"] = json!("new-free-poid");
let nonce = SecretString::from("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA");
let profile = service
.verify_authorization(
&secret(&free_claims, Some("refresh-token")),
&SecretString::from(token(&id_token_claims(
nonce.expose_secret(),
"subject-signed",
))),
&nonce,
&CodexIdentityExpectation::reauthorization(
"account-signed".to_owned(),
"user-signed".to_owned(),
"00000000-0000-4000-8000-000000000001".to_owned(),
)
.expect("reauthorization expectation"),
)
.await
.expect("verified Free authorization identity")
.into_complete()
.expect("complete Free profile");
assert_eq!(profile.chatgpt_account_id, "account-signed");
assert_eq!(profile.chatgpt_user_id, "user-signed");
assert_eq!(profile.poid.as_deref(), Some("new-free-poid"));
}
#[tokio::test]
async fn authorization_identity_rejects_signed_workspace_rebinding() {
let (signed, _) = verifier();
let service = CodexAccountIdentityService::new(
Arc::new(signed),
Arc::new(StaticAccountSource {
result: Ok(CodexAuthenticatedAccount {
chatgpt_account_id: "account-default".to_owned(),
chatgpt_user_id: "user-default".to_owned(),
email: None,
plan_type: None,
}),
}),
);
let nonce = SecretString::from("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA");
let error = service
.verify_authorization(
&secret(
&claims(get_current_timestamp() + 3_600),
Some("refresh-token"),
),
&SecretString::from(token(&id_token_claims(
nonce.expose_secret(),
"subject-signed",
))),
&nonce,
&CodexIdentityExpectation::reauthorization(
"account-target".to_owned(),
"user-target".to_owned(),
"00000000-0000-4000-8000-000000000001".to_owned(),
)
.expect("reauthorization expectation"),
)
.await
.expect_err("signed identity cannot rebind the target account");
assert_eq!(error, CodexIdentityVerificationError::Rejected);
}
#[tokio::test]
async fn account_identity_keeps_usage_unavailable_out_of_ready_profile() {
let (signed, _) = verifier();
let service = CodexAccountIdentityService::new(
Arc::new(signed),
Arc::new(StaticAccountSource {
result: Err(CodexIdentityVerificationError::Unavailable),
}),
);
let verification = service
.verify(
&secret(&claims(get_current_timestamp() + 3_600), None),
&CodexIdentityExpectation::default(),
)
.await
.expect("signed identity remains explicit");
assert!(matches!(
&verification,
CodexIdentityVerification::SignedOnly(_)
));
assert_eq!(
verification.into_complete().expect_err("not Ready"),
CodexIdentityVerificationError::Unavailable
);
}
#[tokio::test]
async fn verifier_should_reject_expired_wrong_issuer_audience_or_unbound_tokens() {
let (verifier, _) = verifier();
let now = get_current_timestamp();
let mut cases = vec![
claims(now),
claims(now + 3_600),
claims(now + 3_600),
claims(now + 3_600),
];
cases[1]["iss"] = json!("https://attacker.invalid");
cases[2]["aud"] = json!(["https://attacker.invalid"]);
cases[3]["sub"] = json!("");
for claims in cases {
let error = verifier
.verify_access(&secret(&claims, None))
.await
.expect_err("invalid identity token must fail");
assert_eq!(error, CodexIdentityVerificationError::Rejected);
}
}
#[tokio::test]
async fn verifier_should_reject_user_rebinding_and_invalid_refresh_shape() {
let (verifier, _) = verifier();
let mut rebound = claims(get_current_timestamp() + 3_600);
rebound["https://api.openai.com/auth"]["user_id"] = json!("other-user");
let rebound_error = verifier
.verify_access(&secret(&rebound, None))
.await
.expect_err("signed aliases must not disagree");
let access = token(&claims(get_current_timestamp() + 3_600));
let reused = CodexOAuthSecret {
access_token: SecretString::new(access.clone().into()),
refresh_token: Some(SecretString::new(access.into())),
id_token: None,
};
let refresh_error = verifier
.verify_access(&reused)
.await
.expect_err("refresh token must have independent shape");
assert_eq!(rebound_error, CodexIdentityVerificationError::Rejected);
assert_eq!(refresh_error, CodexIdentityVerificationError::Rejected);
}
#[tokio::test]
async fn verifier_should_cache_jwks_and_fail_closed_when_source_is_unavailable() {
let (verifier, calls) = verifier();
let valid = secret(&claims(get_current_timestamp() + 3_600), None);
verifier
.verify_access(&valid)
.await
.expect("first verification");
verifier
.verify_access(&valid)
.await
.expect("cached verification");
assert_eq!(calls.load(Ordering::SeqCst), 1);
let unavailable = CodexJwtIdentityVerifier::new(Box::new(StaticJwksSource {
body: Vec::new(),
calls: Arc::new(AtomicUsize::new(0)),
error: Some(CodexIdentityVerificationError::Unavailable),
}));
let error = unavailable
.verify_access(&valid)
.await
.expect_err("unavailable JWKS must fail closed");
assert_eq!(error, CodexIdentityVerificationError::Unavailable);
}
#[tokio::test]
async fn authorization_verifier_should_bind_signed_id_token_subject_audience_and_nonce() {
let (verifier, _) = verifier();
let access = secret(
&claims(get_current_timestamp() + 3_600),
Some("refresh-token"),
);
let nonce = SecretString::from("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA");
let valid_id = SecretString::from(token(&id_token_claims(
nonce.expose_secret(),
"subject-signed",
)));
verifier
.verify_authorization(&access, &valid_id, &nonce)
.await
.expect("verify authorization token set");
let wrong_nonce = verifier
.verify_authorization(
&access,
&valid_id,
&SecretString::from("BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB"),
)
.await
.expect_err("nonce mismatch must fail");
let wrong_subject = verifier
.verify_authorization(
&access,
&SecretString::from(token(&id_token_claims(
nonce.expose_secret(),
"other-subject",
))),
&nonce,
)
.await
.expect_err("subject mismatch must fail");
let mut wrong_audience = id_token_claims(nonce.expose_secret(), "subject-signed");
wrong_audience["aud"] = json!(["attacker-client"]);
let wrong_audience = verifier
.verify_authorization(&access, &SecretString::from(token(&wrong_audience)), &nonce)
.await
.expect_err("ID token audience mismatch must fail");
assert_eq!(wrong_nonce, CodexIdentityVerificationError::Rejected);
assert_eq!(wrong_subject, CodexIdentityVerificationError::Rejected);
assert_eq!(wrong_audience, CodexIdentityVerificationError::Rejected);
}
#[derive(Deserialize)]
struct RealAccountExport {
accounts: Vec<RealAccountWire>,
}
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
struct RealAccountWire {
token: String,
refresh_token: Option<String>,
}
#[tokio::test]
async fn supplied_real_access_only_account_should_pass_official_signature_verification() {
let Ok(path) = std::env::var("CODEX_REAL_ACCOUNT_FILE") else {
return;
};
let bytes = std::fs::read(path).expect("read isolated real account fixture");
let mut document: RealAccountExport =
serde_json::from_slice(&bytes).expect("parse real account fixture shape");
let account = document.accounts.pop().expect("one real account fixture");
assert!(account.refresh_token.is_none());
let verifier = CodexJwtIdentityVerifier::new(Box::new(
ReqwestOpenAiJwksSource::new().expect("build official JWKS source"),
));
let profile = verifier
.verify_access(&CodexOAuthSecret {
access_token: SecretString::new(account.token.into()),
refresh_token: account
.refresh_token
.map(|token| SecretString::new(token.into())),
id_token: None,
})
.await
.expect("verify official access-only Codex account");
assert!(!profile.oauth_subject().is_empty());
assert!(profile.access_token_expires_at() > chrono::Utc::now());
}
@@ -3,7 +3,6 @@ mod agent_identity;
mod catalog;
mod contract;
mod cookie;
mod identity;
mod oauth;
mod quota;
mod refresh;
@@ -1,4 +1,5 @@
use std::sync::{Arc, Mutex};
use std::time::{Duration, SystemTime};
use async_trait::async_trait;
use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD};
@@ -6,751 +7,204 @@ use gateway_admin::model::provider_credentials::{
AuthorizationMutationTarget, AuthorizationOwnerBinding, PendingAuthorizationMutation,
};
use gateway_admin::model::{MutationActor, MutationContext};
use gateway_core::engine::credential::ProviderAccountId;
use gateway_core::routing::ProviderKind;
use provider_openai::credential::token_client::{
AuthorizationCodeExchangeError, AuthorizationCodeExchanger, AuthorizationCodeGrant,
AuthorizationTokenSet,
};
use provider_openai::credential::{
CodexAccountIdentityVerifier, CodexCredentialAdmin, CodexIdentityExpectation,
CodexIdentityVerification, CodexIdentityVerificationError, CodexOAuthAdmin,
CodexOAuthAdminError, CodexOAuthAdminService, CodexOAuthPendingClaimOutcome,
CodexOAuthPendingStore, CodexOAuthPendingStoreError, CodexOAuthSecret,
CodexPendingAuthorization, CompleteCodexOAuthAuthorization, CompletedCodexOAuthCredential,
ImportCodexOAuthCredential, StartCodexOAuthAuthorization, StoredCodexPendingAuthorization,
CodexCredentialAdmin, CodexOAuthAdmin, CodexOAuthAdminError, CodexOAuthAdminService,
CodexOAuthPendingClaimOutcome, CodexOAuthPendingStore, CodexOAuthPendingStoreError,
CodexOAuthSecret, CompleteCodexOAuthAuthorization, CompletedCodexOAuthCredential,
StartCodexOAuthAuthorization, StoredCodexPendingAuthorization,
};
use secrecy::{ExposeSecret, SecretString};
use sha2::{Digest as _, Sha256};
use secrecy::SecretString;
use url::Url;
use crate::support::{MemoryAccountStore, profile, runtime_policy, secret};
use crate::support::MemoryAccountStore;
#[derive(Default)]
struct PendingStore {
value: Mutex<Option<(StoredCodexPendingAuthorization, Option<String>)>>,
}
fn duplicate_pending(
pending: &StoredCodexPendingAuthorization,
) -> Result<CodexPendingAuthorization, CodexOAuthPendingStoreError> {
CodexPendingAuthorization::from_stored(StoredCodexPendingAuthorization {
flow_id: pending.flow_id.clone(),
owner_ref: pending.owner_ref.clone(),
started_request_ref: pending.started_request_ref.clone(),
name: pending.name.clone(),
expires_at: pending.expires_at,
state: pending.state.clone(),
nonce: pending.nonce.clone(),
code_verifier: pending.code_verifier.clone(),
reauthorization_account_id: pending.reauthorization_account_id.clone(),
mutation: pending.mutation.clone(),
})
pending: Mutex<Option<StoredCodexPendingAuthorization>>,
}
#[async_trait]
impl CodexOAuthPendingStore for PendingStore {
async fn create(
&self,
pending: &CodexPendingAuthorization,
pending: &provider_openai::credential::CodexPendingAuthorization,
) -> Result<(), CodexOAuthPendingStoreError> {
let mut value = self.value.lock().expect("pending lock");
if value.is_some() {
let mut stored = self.pending.lock().expect("pending lock");
if stored.is_some() {
return Err(CodexOAuthPendingStoreError::Conflict);
}
*value = Some((
StoredCodexPendingAuthorization {
flow_id: pending.flow_id().to_owned(),
owner_ref: pending.owner_ref().to_owned(),
started_request_ref: pending.started_request_ref().to_owned(),
name: pending.name().to_owned(),
expires_at: pending.expires_at(),
state: pending.state().clone(),
nonce: pending.nonce().clone(),
code_verifier: pending.code_verifier().clone(),
reauthorization_account_id: pending.reauthorization().map(ToString::to_string),
mutation: pending.mutation().clone(),
},
None,
));
*stored = Some(StoredCodexPendingAuthorization {
flow_id: pending.flow_id().to_owned(),
owner_ref: pending.owner_ref().to_owned(),
started_request_ref: pending.started_request_ref().to_owned(),
name: pending.name().to_owned(),
expires_at: pending.expires_at(),
state: pending.state().clone(),
nonce: pending.nonce().clone(),
code_verifier: pending.code_verifier().clone(),
reauthorization_account_id: pending.reauthorization().map(ToString::to_string),
mutation: pending.mutation().clone(),
});
Ok(())
}
async fn claim(
&self,
owner_ref: &str,
flow_id: &str,
claim_ref: &str,
_claim_ttl: std::time::Duration,
_owner_ref: &str,
_flow_id: &str,
_claim_ref: &str,
_claim_ttl: Duration,
) -> Result<CodexOAuthPendingClaimOutcome, CodexOAuthPendingStoreError> {
let mut value = self.value.lock().expect("pending lock");
let Some((pending, claim)) = value.as_mut() else {
let Some(pending) = self.pending.lock().expect("pending lock").take() else {
return Ok(CodexOAuthPendingClaimOutcome::NotFound);
};
if pending.owner_ref != owner_ref || pending.flow_id != flow_id {
return Ok(CodexOAuthPendingClaimOutcome::NotFound);
}
if claim.is_some() {
return Ok(CodexOAuthPendingClaimOutcome::InProgress);
}
*claim = Some(claim_ref.to_owned());
duplicate_pending(pending)
.map(Box::new)
.map(CodexOAuthPendingClaimOutcome::Claimed)
let pending = provider_openai::credential::CodexPendingAuthorization::from_stored(pending)
.map_err(|_| CodexOAuthPendingStoreError::InvalidValue)?;
Ok(CodexOAuthPendingClaimOutcome::Claimed(Box::new(pending)))
}
async fn release_claim(
&self,
owner_ref: &str,
flow_id: &str,
claim_ref: &str,
_owner_ref: &str,
_flow_id: &str,
_claim_ref: &str,
) -> Result<bool, CodexOAuthPendingStoreError> {
let mut value = self.value.lock().expect("pending lock");
let Some((pending, claim)) = value.as_mut() else {
return Ok(false);
};
if pending.owner_ref != owner_ref
|| pending.flow_id != flow_id
|| claim.as_deref() != Some(claim_ref)
{
return Ok(false);
}
*claim = None;
Ok(true)
}
async fn consume_claim(
&self,
owner_ref: &str,
flow_id: &str,
claim_ref: &str,
_owner_ref: &str,
_flow_id: &str,
_claim_ref: &str,
) -> Result<bool, CodexOAuthPendingStoreError> {
let mut value = self.value.lock().expect("pending lock");
let Some((pending, claim)) = value.as_ref() else {
return Ok(false);
};
if pending.owner_ref != owner_ref
|| pending.flow_id != flow_id
|| claim.as_deref() != Some(claim_ref)
{
return Ok(false);
}
*value = None;
self.pending.lock().expect("pending lock").take();
Ok(true)
}
}
struct Exchanger;
struct Exchanger {
id_token: String,
}
#[async_trait]
impl AuthorizationCodeExchanger for Exchanger {
async fn exchange_authorization_code(
&self,
grant: AuthorizationCodeGrant,
_grant: AuthorizationCodeGrant,
) -> Result<AuthorizationTokenSet, AuthorizationCodeExchangeError> {
if grant.code.expose_secret() != "authorization-code" {
return Err(AuthorizationCodeExchangeError::Rejected);
}
Ok(AuthorizationTokenSet {
secret: CodexOAuthSecret {
access_token: SecretString::from("oauth-access-token"),
refresh_token: Some(SecretString::from("oauth-refresh-token")),
access_token: SecretString::from("access-token-from-upstream"),
refresh_token: Some(SecretString::from("refresh-token-from-upstream")),
id_token: None,
},
id_token: SecretString::from("oauth-id-token"),
id_token: SecretString::from(self.id_token.clone()),
expires_in: Some(Duration::from_secs(60 * 60)),
})
}
}
struct Verifier;
#[async_trait]
impl CodexAccountIdentityVerifier for Verifier {
async fn verify(
&self,
_secret: &CodexOAuthSecret,
_expectation: &CodexIdentityExpectation,
) -> Result<CodexIdentityVerification, CodexIdentityVerificationError> {
Err(CodexIdentityVerificationError::Rejected)
}
async fn verify_authorization(
&self,
_secret: &CodexOAuthSecret,
id_token: &SecretString,
expected_nonce: &SecretString,
_expectation: &CodexIdentityExpectation,
) -> Result<CodexIdentityVerification, CodexIdentityVerificationError> {
if id_token.expose_secret() != "oauth-id-token" || expected_nonce.expose_secret().len() < 16
{
return Err(CodexIdentityVerificationError::Rejected);
}
Ok(CodexIdentityVerification::Complete(profile(
"chatgpt-oauth",
)))
}
}
fn service() -> CodexOAuthAdminService {
service_with_store(Arc::new(MemoryAccountStore::default()), Arc::new(Verifier))
}
fn service_with_store(
store: Arc<MemoryAccountStore>,
verifier: Arc<dyn CodexAccountIdentityVerifier>,
) -> CodexOAuthAdminService {
CodexOAuthAdminService::new(
Arc::new(PendingStore::default()),
Arc::new(Exchanger),
verifier,
store,
runtime_policy(),
CodexCredentialAdmin,
)
}
async fn started(
service: &CodexOAuthAdminService,
) -> provider_openai::credential::CodexOAuthAuthorizationStarted {
service
.start_authorization(StartCodexOAuthAuthorization {
mutation: create_mutation("request-one"),
})
.await
.expect("start OAuth")
}
fn owner_context(request_id: &str) -> MutationContext {
MutationContext {
actor: MutationActor::AdminSession {
admin_user_id: "owner-one".to_owned(),
},
request_id: request_id.to_owned(),
}
}
fn create_mutation(request_id: &str) -> PendingAuthorizationMutation {
let context = owner_context(request_id);
fn mutation() -> PendingAuthorizationMutation {
let context = MutationContext {
actor: MutationActor::System,
request_id: "oauth-alignment-test".to_owned(),
};
PendingAuthorizationMutation::new(
ProviderKind::new("openai").expect("provider"),
AuthorizationMutationTarget::Create {
name: "OAuth Account".to_owned(),
name: "new OAuth account".to_owned(),
},
AuthorizationOwnerBinding::from_context(&context),
)
}
fn reauthorization_mutation(request_id: &str, account_id: &str) -> PendingAuthorizationMutation {
let context = owner_context(request_id);
PendingAuthorizationMutation::new(
ProviderKind::new("openai").expect("provider"),
AuthorizationMutationTarget::Reauthorize {
account_id: ProviderAccountId::new(account_id).expect("account id"),
},
AuthorizationOwnerBinding::from_context(&context),
)
fn id_token(payload: serde_json::Value) -> String {
let payload = URL_SAFE_NO_PAD.encode(serde_json::to_vec(&payload).expect("payload JSON"));
// 官方逻辑只读取 payload;header/signature 不参与本地 metadata 解析。
format!("unverified-header.{payload}.unverified-signature")
}
fn owner_ref() -> String {
let mut digest = Sha256::new();
digest.update(b"admin-session\0");
digest.update(b"owner-one");
URL_SAFE_NO_PAD.encode(digest.finalize())
}
#[tokio::test]
async fn start_authorization_uses_pkce_nonce_and_fixed_official_redirect() {
let service = service();
let started = started(&service).await;
let url = Url::parse(&started.authorization_url).expect("authorization URL");
let query = url
.query_pairs()
.collect::<std::collections::BTreeMap<_, _>>();
assert_eq!(
url.as_str().split('?').next(),
Some("https://auth.openai.com/oauth/authorize")
async fn complete(id_token: String) -> Result<CompletedCodexOAuthCredential, CodexOAuthAdminError> {
let service = CodexOAuthAdminService::new(
Arc::new(PendingStore::default()),
Arc::new(Exchanger { id_token }),
Arc::new(MemoryAccountStore::default()),
CodexCredentialAdmin,
);
assert_eq!(
query.get("code_challenge_method").map(|v| v.as_ref()),
Some("S256")
);
assert_eq!(
query.get("redirect_uri").map(|v| v.as_ref()),
Some("http://localhost:1455/auth/callback")
);
assert!(query.get("state").is_some_and(|value| value.len() >= 16));
assert!(query.get("nonce").is_some_and(|value| value.len() >= 16));
}
#[tokio::test]
async fn completion_returns_verified_core_account_without_writing_store() {
let service = service();
let started = started(&service).await;
let url = Url::parse(&started.authorization_url).expect("authorization URL");
let state = url
.query_pairs()
.find_map(|(key, value)| (key == "state").then(|| value.into_owned()))
.expect("state");
let prepared = service
.complete_authorization(CompleteCodexOAuthAuthorization {
owner_ref: owner_ref(),
flow_id: started.flow_id,
callback_url: SecretString::from(format!(
"http://localhost:1455/auth/callback?code=authorization-code&state={state}"
)),
})
.await
.expect("complete OAuth");
let CompletedCodexOAuthCredential::Create(prepared) = prepared.credential else {
panic!("expected prepared create");
};
assert_eq!(prepared.account.provider().as_str(), "openai");
assert!(prepared.account.id().as_str().starts_with("acct_"));
assert_eq!(
prepared
.credential
.expose_to_provider()
.get("access_token")
.and_then(serde_json::Value::as_str),
Some("oauth-access-token")
);
assert_eq!(
prepared
.credential
.expose_to_provider()
.get("id_token")
.and_then(serde_json::Value::as_str),
Some("oauth-id-token")
);
}
#[tokio::test]
async fn completion_accepts_unrelated_oauth_callback_parameters() {
let service = service();
let started = started(&service).await;
let state = Url::parse(&started.authorization_url)
.expect("authorization URL")
.query_pairs()
.find_map(|(key, value)| (key == "state").then(|| value.into_owned()))
.expect("state");
let prepared = service
.complete_authorization(CompleteCodexOAuthAuthorization {
owner_ref: owner_ref(),
flow_id: started.flow_id,
callback_url: SecretString::from(format!(
"http://localhost:1455/auth/callback?code=authorization-code&state={state}&scope=openid%20profile&iss=https%3A%2F%2Fauth.openai.com"
)),
})
.await
.expect("complete OAuth with standard callback parameters");
assert!(matches!(
prepared.credential,
CompletedCodexOAuthCredential::Create(_)
));
}
#[tokio::test]
async fn completion_rejects_repeated_code_callback_parameter() {
let service = service();
let started = started(&service).await;
let state = Url::parse(&started.authorization_url)
.expect("authorization URL")
.query_pairs()
.find_map(|(key, value)| (key == "state").then(|| value.into_owned()))
.expect("state");
let error = service
.complete_authorization(CompleteCodexOAuthAuthorization {
owner_ref: owner_ref(),
flow_id: started.flow_id,
callback_url: SecretString::from(format!(
"http://localhost:1455/auth/callback?code=authorization-code&code=another-code&state={state}"
)),
})
.await
.expect_err("repeated code");
assert_eq!(error, CodexOAuthAdminError::CallbackRejected);
}
#[tokio::test]
async fn completion_rejects_repeated_state_callback_parameter() {
let service = service();
let started = started(&service).await;
let state = Url::parse(&started.authorization_url)
.expect("authorization URL")
.query_pairs()
.find_map(|(key, value)| (key == "state").then(|| value.into_owned()))
.expect("state");
let error = service
.complete_authorization(CompleteCodexOAuthAuthorization {
owner_ref: owner_ref(),
flow_id: started.flow_id,
callback_url: SecretString::from(format!(
"http://localhost:1455/auth/callback?code=authorization-code&state={state}&state=another-state-value"
)),
})
.await
.expect_err("repeated state");
assert_eq!(error, CodexOAuthAdminError::CallbackRejected);
}
struct UnavailableVerifier;
#[async_trait]
impl CodexAccountIdentityVerifier for UnavailableVerifier {
async fn verify(
&self,
_: &CodexOAuthSecret,
_: &CodexIdentityExpectation,
) -> Result<CodexIdentityVerification, CodexIdentityVerificationError> {
Err(CodexIdentityVerificationError::Unavailable)
}
async fn verify_authorization(
&self,
_: &CodexOAuthSecret,
_: &SecretString,
_: &SecretString,
_: &CodexIdentityExpectation,
) -> Result<CodexIdentityVerification, CodexIdentityVerificationError> {
Err(CodexIdentityVerificationError::Unavailable)
}
}
#[tokio::test]
async fn completion_classifies_unavailable_identity_verification_as_upstream_unavailable() {
let store = Arc::new(MemoryAccountStore::default());
let service = service_with_store(store, Arc::new(UnavailableVerifier));
let started = started(&service).await;
let state = Url::parse(&started.authorization_url)
.expect("authorization URL")
.query_pairs()
.find_map(|(key, value)| (key == "state").then(|| value.into_owned()))
.expect("state");
let error = service
.complete_authorization(CompleteCodexOAuthAuthorization {
owner_ref: owner_ref(),
flow_id: started.flow_id,
callback_url: SecretString::from(format!(
"http://localhost:1455/auth/callback?code=authorization-code&state={state}"
)),
})
.await
.expect_err("identity verification unavailable");
assert_eq!(error, CodexOAuthAdminError::UpstreamUnavailable);
}
#[tokio::test]
async fn callback_state_mismatch_releases_flow_for_retry() {
let service = service();
let started = started(&service).await;
let flow_id = started.flow_id;
let command = || CompleteCodexOAuthAuthorization {
owner_ref: owner_ref(),
flow_id: flow_id.clone(),
callback_url: SecretString::from(
"http://localhost:1455/auth/callback?code=authorization-code&state=wrong-state-value",
),
};
assert_eq!(
service
.complete_authorization(command())
.await
.expect_err("bad state"),
CodexOAuthAdminError::CallbackRejected
);
assert_eq!(
service
.complete_authorization(command())
.await
.expect_err("retryable callback rejection"),
CodexOAuthAdminError::CallbackRejected
);
}
#[tokio::test]
async fn token_exchange_rejection_releases_flow_for_retry() {
let service = service();
let started = started(&service).await;
let flow_id = started.flow_id;
let state = Url::parse(&started.authorization_url)
.expect("authorization URL")
.query_pairs()
.find_map(|(key, value)| (key == "state").then(|| value.into_owned()))
.expect("state");
let command = || CompleteCodexOAuthAuthorization {
owner_ref: owner_ref(),
flow_id: flow_id.clone(),
callback_url: SecretString::from(format!(
"http://localhost:1455/auth/callback?code=rejected-authorization-code&state={state}"
)),
};
assert_eq!(
service
.complete_authorization(command())
.await
.expect_err("token exchange rejection"),
CodexOAuthAdminError::TokenRejected
);
assert_eq!(
service
.complete_authorization(command())
.await
.expect_err("retryable token exchange rejection"),
CodexOAuthAdminError::TokenRejected
);
}
#[test]
fn oauth_command_debug_redacts_owner_flow_and_callback() {
let command = CompleteCodexOAuthAuthorization {
owner_ref: "owner-private".to_owned(),
flow_id: "flow-private".to_owned(),
callback_url: SecretString::from("callback-private-value"),
};
let debug = format!("{command:?}");
for secret in ["owner-private", "flow-private", "callback-private-value"] {
assert!(!debug.contains(secret));
}
}
#[tokio::test]
async fn reauthorization_uses_revision_advanced_after_start_for_prepared_rotation() {
let store = Arc::new(MemoryAccountStore::default());
store
.seed_oauth_credential(ImportCodexOAuthCredential {
account_id: "acct_oauth_reauth".to_owned(),
name: "reauthorize".to_owned(),
secret: secret("old-oauth-access"),
verified_account: profile("chatgpt-oauth"),
next_refresh_at: Some(chrono::Utc::now() + chrono::Duration::minutes(30)),
enabled: true,
})
.await;
let original_account = store
.account("acct_oauth_reauth")
.expect("original account");
let original_installation_id = store
.repository()
.load_runtime_credential(&original_account)
.await
.expect("original credential")
.installation_id;
let service = service_with_store(store.clone(), Arc::new(Verifier));
let started = service
.start_authorization(StartCodexOAuthAuthorization {
mutation: reauthorization_mutation("request-reauth", "acct_oauth_reauth"),
mutation: mutation(),
})
.await
.expect("start reauthorization");
let advanced_revision = store.advance_credential_revision("acct_oauth_reauth").await;
assert_eq!(advanced_revision.get(), 2);
.expect("start OAuth authorization");
let state = Url::parse(&started.authorization_url)
.expect("authorization URL")
.query_pairs()
.find_map(|(key, value)| (key == "state").then(|| value.into_owned()))
.expect("state");
let prepared = service
.complete_authorization(CompleteCodexOAuthAuthorization {
owner_ref: owner_ref(),
flow_id: started.flow_id,
callback_url: SecretString::from(format!(
"http://localhost:1455/auth/callback?code=authorization-code&state={state}"
)),
})
.await
.expect("complete reauthorization");
let CompletedCodexOAuthCredential::Reauthorize(prepared) = prepared.credential else {
panic!("expected prepared reauthorization");
};
assert_eq!(
prepared.credential.account_id().as_str(),
"acct_oauth_reauth"
);
assert_eq!(prepared.credential.expected_revision().get(), 2);
let runtime =
provider_openai::credential::CodexCredentialCodec::decode(prepared.credential.credential())
.expect("prepared credential");
assert_eq!(
runtime
.authentication
.oauth()
.expect("OAuth credential")
.access_token
.expose_secret(),
"oauth-access-token"
);
assert_eq!(runtime.installation_id, original_installation_id);
assert_eq!(
store
.account("acct_oauth_reauth")
.expect("unchanged account")
.revision()
.get(),
2
);
}
struct ReplacementIdentityVerifier;
#[async_trait]
impl CodexAccountIdentityVerifier for ReplacementIdentityVerifier {
async fn verify(
&self,
_: &CodexOAuthSecret,
_: &CodexIdentityExpectation,
) -> Result<CodexIdentityVerification, CodexIdentityVerificationError> {
Err(CodexIdentityVerificationError::Rejected)
}
async fn verify_authorization(
&self,
_: &CodexOAuthSecret,
_: &SecretString,
_: &SecretString,
expectation: &CodexIdentityExpectation,
) -> Result<CodexIdentityVerification, CodexIdentityVerificationError> {
assert_eq!(expectation, &CodexIdentityExpectation::default());
Ok(CodexIdentityVerification::Complete(profile(
"chatgpt-different-owner",
)))
}
}
struct RotatedPrincipalVerifier;
#[async_trait]
impl CodexAccountIdentityVerifier for RotatedPrincipalVerifier {
async fn verify(
&self,
_: &CodexOAuthSecret,
_: &CodexIdentityExpectation,
) -> Result<CodexIdentityVerification, CodexIdentityVerificationError> {
Err(CodexIdentityVerificationError::Rejected)
}
async fn verify_authorization(
&self,
_: &CodexOAuthSecret,
_: &SecretString,
_: &SecretString,
_: &CodexIdentityExpectation,
) -> Result<CodexIdentityVerification, CodexIdentityVerificationError> {
let mut verified = profile("chatgpt-oauth");
verified.oauth_subject = "rotated-oauth-subject".to_owned();
verified.poid = None;
Ok(CodexIdentityVerification::Complete(verified))
}
}
#[tokio::test]
async fn reauthorization_updates_changed_free_principal_for_same_account() {
let store = Arc::new(MemoryAccountStore::default());
store
.seed_oauth_credential(ImportCodexOAuthCredential {
account_id: "acct_oauth_principal".to_owned(),
name: "reauthorize".to_owned(),
secret: secret("old-oauth-access"),
verified_account: profile("chatgpt-oauth"),
next_refresh_at: Some(chrono::Utc::now() + chrono::Duration::minutes(30)),
enabled: true,
})
.await;
let service = service_with_store(store, Arc::new(RotatedPrincipalVerifier));
let started = service
.start_authorization(StartCodexOAuthAuthorization {
mutation: reauthorization_mutation("request-rotated-principal", "acct_oauth_principal"),
})
.await
.expect("start reauthorization");
let state = Url::parse(&started.authorization_url)
.expect("authorization URL")
.query_pairs()
.find_map(|(key, value)| (key == "state").then(|| value.into_owned()))
.expect("state");
let prepared = service
.complete_authorization(CompleteCodexOAuthAuthorization {
owner_ref: owner_ref(),
flow_id: started.flow_id,
callback_url: SecretString::from(format!(
"http://localhost:1455/auth/callback?code=authorization-code&state={state}"
)),
})
.await
.expect("complete reauthorization");
let CompletedCodexOAuthCredential::Reauthorize(prepared) = prepared.credential else {
panic!("expected prepared reauthorization");
};
let runtime =
provider_openai::credential::CodexCredentialCodec::decode(prepared.credential.credential())
.expect("prepared credential");
let principal = runtime.principal.expect("OAuth principal");
assert_eq!(principal.oauth_subject, "rotated-oauth-subject");
assert_eq!(principal.poid, None);
}
#[tokio::test]
async fn reauthorization_rebinds_target_record_to_authorized_identity() {
let store = Arc::new(MemoryAccountStore::default());
store
.seed_oauth_credential(ImportCodexOAuthCredential {
account_id: "acct_oauth_rebind".to_owned(),
name: "reauthorize".to_owned(),
secret: secret("old-oauth-access"),
verified_account: profile("chatgpt-oauth"),
next_refresh_at: Some(chrono::Utc::now() + chrono::Duration::minutes(30)),
enabled: true,
})
.await;
let service = service_with_store(store, Arc::new(ReplacementIdentityVerifier));
let started = service
.start_authorization(StartCodexOAuthAuthorization {
mutation: reauthorization_mutation("request-rebind", "acct_oauth_rebind"),
})
.await
.expect("start reauthorization");
let state = Url::parse(&started.authorization_url)
.expect("authorization URL")
.query_pairs()
.find_map(|(key, value)| (key == "state").then(|| value.into_owned()))
.expect("state");
.expect("state parameter");
let completed = service
.complete_authorization(CompleteCodexOAuthAuthorization {
owner_ref: owner_ref(),
owner_ref: "test-owner".to_owned(),
flow_id: started.flow_id,
callback_url: SecretString::from(format!(
"http://localhost:1455/auth/callback?code=authorization-code&state={state}"
"http://localhost:1455/auth/callback?code=code-from-browser&state={state}"
)),
})
.await
.expect("complete identity replacement");
let CompletedCodexOAuthCredential::Reauthorize(prepared) = completed.credential else {
panic!("expected prepared reauthorization");
.await?;
Ok(completed.credential)
}
#[tokio::test]
async fn first_exchange_uses_official_id_token_claim_mapping_without_signature_validation() {
let credential = complete(id_token(serde_json::json!({
"email": "top@example.com",
"https://api.openai.com/profile": { "email": "fallback@example.com" },
"https://api.openai.com/auth": {
"chatgpt_plan_type": "pro",
"chatgpt_user_id": "chatgpt-user",
"user_id": "fallback-user",
"chatgpt_account_id": "workspace-id"
}
})))
.await
.expect("completed OAuth exchange");
let CompletedCodexOAuthCredential::Create(account) = credential else {
panic!("expected account creation");
};
assert_eq!(
prepared.credential.account_id().as_str(),
"acct_oauth_rebind"
);
let replacement = prepared
.replacement_identity
.as_ref()
.expect("replacement identity");
assert_eq!(
replacement.upstream_account_id(),
Some("chatgpt-different-owner")
);
assert_eq!(
replacement.upstream_user_id(),
"user-chatgpt-different-owner"
assert_eq!(account.account.email(), Some("top@example.com"));
assert_eq!(account.account.upstream_user_id(), Some("chatgpt-user"));
assert_eq!(account.account.upstream_account_id(), Some("workspace-id"));
assert_eq!(account.account.plan_type(), Some("pro"));
assert!(account.account.is_schedulable(SystemTime::now()));
assert!(
account.account.next_refresh_at().is_none(),
"normal OAuth creation must not persist a margin-derived refresh time"
);
}
#[tokio::test]
async fn first_exchange_permits_missing_identity_claims_but_requires_a_parseable_id_token() {
let credential = complete(id_token(serde_json::json!({
"https://api.openai.com/profile": { "email": "profile@example.com" }
})))
.await
.expect("missing optional claims are allowed");
let CompletedCodexOAuthCredential::Create(account) = credential else {
panic!("expected account creation");
};
assert_eq!(account.account.email(), Some("profile@example.com"));
assert_eq!(account.account.upstream_user_id(), None);
assert!(!account.account.is_schedulable(SystemTime::now()));
let error = complete("header.not-base64.signature".to_owned())
.await
.expect_err("official local payload parsing rejects malformed id tokens");
assert_eq!(error, CodexOAuthAdminError::TokenRejected);
}
File diff suppressed because it is too large Load Diff
@@ -1,6 +1,6 @@
//! 402 恢复回归。
//!
//! 覆盖 `QuotaExhausted -> Ready` 的证据规则:旧 reset 已到期 + 新快照未耗尽。
//! 覆盖 `QuotaExhausted -> Ready` 的证据规则:新 core primary reset 后移 + 新快照用量低于 10%。
use super::*;
@@ -12,6 +12,7 @@ async fn manual_quota_refresh_keeps_confirmed_exhaustion_until_the_recorded_rese
let account = store.account(account_id).expect("created account");
store
.apply_state_change(AccountStateChange {
message: None,
account_id: account.id().clone(),
expected_revision: account.revision(),
availability: AccountAvailability::Expired,
@@ -126,14 +127,13 @@ async fn manual_quota_refresh_keeps_confirmed_exhaustion_until_the_recorded_rese
}
#[tokio::test]
async fn expired_old_reset_with_fresh_allowed_snapshot_recovers_ready() {
// 旧快照 reset_at < now,新快照 allowed=true、
// limit_reached=false、reset_at > now;最终必须恢复 Ready。
async fn advanced_reset_with_primary_usage_below_ten_percent_recovers_ready() {
// 旧 reset 仍在未来;同一主窗口 reset 后移且用量低于 10% 时才恢复 Ready。
let store = Arc::new(MemoryAccountStore::default());
let account_id = "acct_cross_window_recovery";
create_account(&store, account_id).await;
let account = store.account(account_id).expect("created account");
let old_reset = Utc::now().timestamp() - 60;
let old_reset = Utc::now().timestamp() + 3_600;
store
.compare_and_swap_quota(QuotaObservation {
account_id: account.id().clone(),
@@ -161,6 +161,7 @@ async fn expired_old_reset_with_fresh_allowed_snapshot_recovers_ready() {
.expect("seed old exhausted snapshot");
store
.apply_state_change(AccountStateChange {
message: None,
account_id: account.id().clone(),
expected_revision: account.revision(),
availability: AccountAvailability::QuotaExhausted,
@@ -169,7 +170,7 @@ async fn expired_old_reset_with_fresh_allowed_snapshot_recovers_ready() {
.await
.expect("mark account exhausted");
let server = MockServer::start().await;
let future_reset = Utc::now().timestamp() + 3600;
let future_reset = old_reset + 3_600;
Mock::given(method("GET"))
.and(path("/api/codex/usage"))
.and(header(
@@ -180,7 +181,7 @@ async fn expired_old_reset_with_fresh_allowed_snapshot_recovers_ready() {
"rate_limit": {
"allowed": true,
"limit_reached": false,
"primary_window": {"used_percent": 5, "reset_at": future_reset}
"primary_window": {"used_percent": 9.9, "reset_at": future_reset}
}
})))
.mount(&server)
@@ -206,6 +207,134 @@ async fn expired_old_reset_with_fresh_allowed_snapshot_recovers_ready() {
.expect("account after cross-window recovery")
.availability(),
AccountAvailability::Ready,
"old reset elapsed + fresh allowed snapshot must recover QuotaExhausted"
"an advanced reset with a nearly unused primary window must recover QuotaExhausted"
);
}
#[tokio::test]
async fn advanced_reset_with_primary_usage_at_ten_percent_keeps_confirmed_exhaustion() {
let store = Arc::new(MemoryAccountStore::default());
let account_id = "acct_cross_window_ten_percent";
create_account(&store, account_id).await;
let account = store.account(account_id).expect("created account");
let old_reset = Utc::now().timestamp() + 3_600;
store
.compare_and_swap_quota(QuotaObservation {
account_id: account.id().clone(),
expected_revision: account.revision(),
quota: Some(OpaqueProviderData::new(
json!({
"rate_limit": {
"allowed": false,
"limit_reached": true,
"primary_window": {
"used_percent": 100,
"reset_at": old_reset,
"limit_window_seconds": 3600
}
}
})
.as_object()
.expect("quota object")
.clone(),
)),
observed_at: Some(SystemTime::now()),
limit_reached: None,
})
.await
.expect("seed old exhausted snapshot");
store
.apply_state_change(AccountStateChange {
message: None,
account_id: account.id().clone(),
expected_revision: account.revision(),
availability: AccountAvailability::QuotaExhausted,
observed_at: SystemTime::now(),
})
.await
.expect("mark account exhausted");
let server = MockServer::start().await;
let future_reset = old_reset + 3_600;
Mock::given(method("GET"))
.and(path("/api/codex/usage"))
.and(header(
"authorization",
format!("Bearer token-{account_id}"),
))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"rate_limit": {
"allowed": true,
"limit_reached": false,
"primary_window": {"used_percent": 10.0, "reset_at": future_reset}
}
})))
.mount(&server)
.await;
let service = quota_service_with_base_url(
&store,
reqwest::Client::builder()
.no_proxy()
.build()
.expect("client"),
server.uri(),
);
let held = service
.refresh_account(account.id())
.await
.expect("refresh non-empty new window quota");
assert!(held.fact().exhausted());
assert_eq!(
store
.account(account_id)
.expect("account after ten-percent snapshot")
.availability(),
AccountAvailability::QuotaExhausted,
"a reset advance at exactly 10% must not release confirmed exhaustion"
);
}
#[tokio::test]
async fn deactivated_workspace_quota_response_persists_the_upstream_reason() {
let store = Arc::new(MemoryAccountStore::default());
let account_id = "acct_deactivated_workspace";
create_account(&store, account_id).await;
let account = store.account(account_id).expect("created account");
let server = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/api/codex/usage"))
.and(header(
"authorization",
format!("Bearer token-{account_id}"),
))
.respond_with(ResponseTemplate::new(402).set_body_json(json!({
"detail": {
"code": "deactivated_workspace",
"message": "workspace disabled"
}
})))
.mount(&server)
.await;
let service = quota_service_with_base_url(
&store,
reqwest::Client::builder()
.no_proxy()
.build()
.expect("client"),
server.uri(),
);
assert!(service.refresh_account(account.id()).await.is_err());
assert_eq!(
store
.account(account_id)
.expect("account after deactivation")
.availability(),
AccountAvailability::Banned
);
assert_eq!(
store.last_error_message(account_id).as_deref(),
Some("deactivated_workspace")
);
}
@@ -305,7 +305,7 @@ async fn scheduling_signal_survives_limit_reached_without_percent_or_reset() {
}
#[tokio::test]
async fn persisted_codex_additional_limit_replaces_the_top_level_rate_limit() {
async fn persisted_codex_alias_keeps_the_top_level_rate_limit_canonical() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_canonical_codex_limit").await;
let account = store
@@ -323,7 +323,7 @@ async fn persisted_codex_additional_limit_replaces_the_top_level_rate_limit() {
"metered_feature": "codex",
"rate_limit": {
"primary_window": {
"used_percent": 2,
"used_percent": 98,
"reset_at": 1_900_000_000,
"limit_window_seconds": 2_592_000
}
@@ -355,10 +355,65 @@ async fn persisted_codex_additional_limit_replaces_the_top_level_rate_limit() {
.filter(|window| window.kind() == CodexQuotaWindowKind::Monthly)
.collect::<Vec<_>>();
assert_eq!(snapshot.fact().remaining_percent(), Some(98));
assert_eq!(snapshot.fact().remaining_percent(), Some(9));
assert_eq!(monthly.len(), 1);
assert_eq!(monthly[0].source(), "core");
assert_eq!(monthly[0].used_percent(), Some(2.0));
assert!(
monthly
.iter()
.any(|window| { window.source() == "codex" && window.used_percent() == Some(91.0) })
);
}
#[tokio::test]
async fn persisted_quota_orders_core_window_before_additional_limit() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_quota_order").await;
let account = store.account("acct_quota_order").expect("account");
let raw = json!({
"rate_limit": {
"secondary_window": {
"used_percent": 42,
"reset_at": 1_900_604_800,
"limit_window_seconds": 604_800
}
},
"additional_rate_limits": [{
"limit_name": "GPT-5.3-Codex-Spark",
"metered_feature": "gpt-5.3-codex-spark",
"rate_limit": {
"primary_window": {
"used_percent": 17,
"reset_at": 1_900_604_800,
"limit_window_seconds": 604_800
}
}
}]
});
store
.compare_and_swap_quota(QuotaObservation {
account_id: account.id().clone(),
expected_revision: account.revision(),
quota: Some(OpaqueProviderData::new(
raw.as_object().expect("quota object").clone(),
)),
observed_at: Some(SystemTime::now()),
limit_reached: None,
})
.await
.expect("persist quota");
let snapshot = quota_service(&store)
.read_account(account.id())
.await
.expect("read quota")
.expect("quota snapshot");
let sources = snapshot
.windows()
.iter()
.map(|window| window.source())
.collect::<Vec<_>>();
assert_eq!(sources, ["codex", "gpt-5.3-codex-spark"]);
}
#[tokio::test]
@@ -419,16 +474,21 @@ async fn code_review_limit_projects_as_independent_window_with_limit_name() {
.expect("read quota")
.expect("quota snapshot");
// spend_control 不再生成窗口(只作 exhaustion 信号),
// 顶层 code_review_rate_limit 与 additional 的 code_review 桶同源替换后保留一个。
// spend_control 不再生成窗口(只作 exhaustion 信号);顶层和 additional
// 的 code_review 桶分别保留,避免一个快照覆盖另一个。
let review = snapshot
.windows()
.iter()
.filter(|window| window.source() == "code_review")
.collect::<Vec<_>>();
assert_eq!(review.len(), 1);
assert_eq!(review[0].limit_name(), Some("code review"));
assert_eq!(review[0].used_percent(), Some(55.0));
assert_eq!(snapshot.windows().len(), 2);
assert_eq!(review.len(), 2);
assert!(review.iter().any(|window| {
window.limit_name() == Some("code_review") && window.used_percent() == Some(80.0)
}));
assert!(review.iter().any(|window| {
window.limit_name() == Some("code review") && window.used_percent() == Some(55.0)
}));
assert_ne!(review[0].key(), review[1].key());
assert_eq!(snapshot.windows().len(), 3);
assert!(!snapshot.fact().exhausted());
}
File diff suppressed because it is too large Load Diff
@@ -72,7 +72,29 @@ async fn bounded_oauth_response_should_parse_lifetime_and_rotated_token() {
assert_eq!(tokens.access_token, "access-rotated");
assert_eq!(tokens.refresh_token.as_deref(), Some("refresh-rotated"));
assert_eq!(tokens.expires_in, Duration::from_secs(3600));
assert_eq!(tokens.expires_in, Some(Duration::from_secs(3600)));
}
#[tokio::test]
async fn refresh_response_keeps_the_upstream_rotated_token_verbatim() {
let server = MockServer::start().await;
Mock::given(method("POST"))
.and(path("/oauth/token"))
.respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({
"access_token": "access-rotated",
"refresh_token": " ",
"expires_in": 3600
})))
.expect(1)
.mount(&server)
.await;
let tokens = client(&server)
.refresh("refresh-initial")
.await
.expect("upstream response is accepted");
assert_eq!(tokens.refresh_token.as_deref(), Some(" "));
}
#[tokio::test]
@@ -140,7 +162,41 @@ async fn authorization_code_exchange_should_require_bounded_oidc_token_set_and_p
}
#[tokio::test]
async fn authorization_code_exchange_should_reject_missing_nonce_carrier_or_non_json_success() {
async fn authorization_code_exchange_keeps_upstream_access_and_refresh_tokens_verbatim() {
let server = MockServer::start().await;
Mock::given(method("POST"))
.and(path("/oauth/token"))
.respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({
"access_token": " ",
"refresh_token": "",
"id_token": "header.payload.signature"
})))
.expect(1)
.mount(&server)
.await;
let tokens = client(&server)
.exchange_authorization_code(AuthorizationCodeGrant {
code: SecretString::from("authorization-code"),
code_verifier: SecretString::from("pkce-verifier"),
})
.await
.expect("token endpoint response is trusted structurally");
assert_eq!(tokens.secret.access_token.expose_secret(), " ");
assert_eq!(
tokens
.secret
.refresh_token
.as_ref()
.expect("required response field")
.expose_secret(),
""
);
}
#[tokio::test]
async fn authorization_code_exchange_requires_id_token_and_json_response() {
let missing_id = MockServer::start().await;
Mock::given(method("POST"))
.and(path("/oauth/token"))
@@ -40,10 +40,10 @@ fn account_profile_debug_redacts_identity_fields() {
fn plaintext_provider_schema_round_trips_dynamic_cookie_data() {
let data = CodexCredentialData::OAuth(CodexOAuthCredentialData {
schema_version: 1,
principal: CodexCredentialPrincipal {
principal: Some(CodexCredentialPrincipal {
oauth_subject: "subject-private".to_owned(),
poid: Some("poid-private".to_owned()),
},
}),
installation_id: "00000000-0000-4000-8000-000000000001".to_owned(),
access_token: "at".to_owned(),
refresh_token: Some("rt".to_owned()),

Some files were not shown because too many files have changed in this diff Show More