Compare commits

...
18 Commits
Author SHA1 Message Date
zyycnandCodex 80f7d53085 chore(release): v3.35.0
Co-authored-by: Codex <noreply@openai.com>
2026-10-29 00:14:10 +08:00
2f2c1787be fix(responses): align heartbeat and websocket connection lifecycles (#548)
Use passive upstream heartbeats and independent downstream liveness checks.
Reclaim unbound sockets after five idle minutes and release client-owned
resources on disconnect, with safe continuation ownership transfer.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-29 00:08:40 +08:00
ZyyandCodex d679d5e708 fix(i18n): default to Simplified Chinese and simplify login labels
Co-authored-by: Codex <noreply@openai.com>
2026-10-28 21:50:52 +08:00
ZyyandCodex 0f214c3e25 docs: align guides and workflows with current behavior
Co-authored-by: Codex <noreply@openai.com>
2026-10-28 21:50:51 +08:00
736eca6593 feat(accounts): support configurable error protection rules (#547)
* feat(accounts): support configurable error protection rules

Replace capacity-specific freezing with bounded, configurable account error rules and shared attempt feedback.

Drop legacy settings without conversion and expose rule editing and freeze reasons in the admin UI.

Co-authored-by: Codex <noreply@openai.com>

* fix(accounts): preserve error protection scheduling boundaries

Keep xAI accounts excluded while recovery probes are pending, and cover
selection before and after recovery.

Separate scheduling cooldown reads from admin display details, share the
atomic Redis migration and selection logic, and use one policy revision.
Remove the unused adaptive concurrency reduction path and its tests.

Validation: 393 targeted backend tests, workspace Clippy with -D warnings,
Rustfmt, frontend type checking and build, and local service health checks.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-28 21:50:51 +08:00
ZyyandCodex be5b6d199e fix(api): enforce version gates for official Codex clients
Recognize official TUI, exec, IDE, SDK and App Server/MCP product names using the Core version in the User-Agent. Keep Desktop app version handling and unknown-client compatibility intact.

Compare SemVer precedence without build metadata, cover HTTP/SSE and real WebSocket admission, and align API documentation and localized settings help.

Validated 54 targeted and architecture tests, strict Clippy, Rust formatting, frontend lint/build, and localized component previews.

Co-authored-by: Codex <noreply@openai.com>
2026-10-28 19:31:20 +08:00
ZyyandCodex 53e1e9cfb8 fix(ui): show loading failures in notifications
Remove inline loading failure placeholders and dedicated retry actions from
settings, accounts, plugins and usage views. Route foreground failures through
existing notifications while preserving disabled controls and available data.
Clean up obsolete error props, state and translations.

Co-authored-by: Codex <noreply@openai.com>
2026-10-28 19:11:55 +08:00
蓝芷怡andCodex e03851e23f fix(dashboard): show notes below API key accounts (#545)
* fix(dashboard): show notes below API key accounts

Co-authored-by: Codex <noreply@openai.com>

* fix(frontend): complete dashboard theme preview account notes

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
2026-10-28 18:55:57 +08:00
7cd17aff42 feat(usage): add request concurrency trends (#537)
* feat(usage): add request concurrency trends

* perf(usage)!: unify load insights and optimize concurrency

Aggregate concurrency on a single event timeline and reuse lifecycle
indexes instead of joining every request against every time bucket.

Align load summaries and trend points with the four chart metrics,
combine concurrency and scheduling in one view, and right-align the
statistics card selectors. Update the API contract and regression tests.

BREAKING CHANGE: Usage insights concurrency and scheduling P95 fields
move under performance.load and performance.points[].load. Unused
scheduling P50, capacity, and performance coverage fields are removed.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-28 17:22:28 +08:00
005466bcb0 feat(api): support Chat Completions through Responses (#542)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-28 15:15:31 +08:00
ZyyandCodex a6031310f1 fix(openai): exclude free accounts from live selection
Centralize account authentication, plan and model eligibility for Images and Live. Reuse the same checks before leases, affinity and retries, while preserving unknown plans and Free text/search access.

Co-authored-by: Codex <noreply@openai.com>
2026-10-28 13:12:01 +08:00
niceSprite 1ac2a4eac3 fix(logging): preserve warnings after disabled target probes (#539)
* fix(logging): preserve warnings after disabled target probes

* docs: keep logging fix focused on runtime behavior
2026-10-28 12:52:25 +08:00
ZyyandCodex 790a643020 fix(frontend): hide general quota heading in account details
Co-authored-by: Codex <noreply@openai.com>
2026-10-28 11:22:30 +08:00
ZyyandCodex 1fa37ac92f fix(openai): observe reasoning effort updates across continuations
Resolve reasoning effort from configuration updates and preserve overrides in response-scoped session metadata without rewriting upstream requests. Cover HTTP and WebSocket continuations, new-chain resets, and baseline changes.

Co-authored-by: Codex <noreply@openai.com>
2026-10-28 11:04:29 +08:00
ZyyandCodex 0ceac908f4 fix(ui): route Codex voice control through configured gateway
Include the realtime WebSocket base URL in generated Codex configuration and document voice setup. Simplify configuration comments.

Track key reveal actions separately so opening client configuration and copying a key show loading only on the initiating button. Preserve duplicate request protection and stale request cancellation.

Co-authored-by: Codex <noreply@openai.com>
2026-10-28 00:19:13 +08:00
ZyyandCodex 86a27bed94 fix(ui): include model catalog URL in Codex config
Enable remote model discovery for custom Codex providers so upstream catalog updates appear in the model picker.

Co-authored-by: Codex <noreply@openai.com>
2026-10-27 18:22:53 +08:00
ZyyandCodex 60f8f1ee80 refactor(plugins)!: remove model catalog v1 support
Keep direct aliases in v2 and update the SDK builder, runtime compatibility declaration, tests and migration guidance.

BREAKING CHANGE: model_catalog plugins must declare version 2, move their alias list to aliases and return models for adapted models.

Co-authored-by: Codex <noreply@openai.com>
2026-10-27 17:40:13 +08:00
ZyyandCodex 7b6a204889 docs(release): restore account routing advisory
Co-authored-by: Codex <noreply@openai.com>
2026-10-27 17:13:54 +08:00
294 changed files with 11230 additions and 4165 deletions
+2 -2
View File
@@ -20,7 +20,7 @@ description: Codex Proxy RS 开发、修复、排障、代码审查和文档修
| 模块或调用链变更 | [Workspace 边界](../../../docs/architecture.md#3-workspace-边界)及所属模块章节,不通读整份架构 |
| 新增、调整、清理或审查测试 | 下方[测试维护](#测试维护),按当前行为、职责归属与已有覆盖判断保留、合并或删除 |
| 修改部署行为、`deploy/` 配置或数据库迁移 | 按范围读取[部署](../../../deploy/README.md)、[迁移](../../../backend/migrations/README.md)对应章节 |
| 修改或审查文档 | [文档检查](references/documentation.md)及目标章节,不默认读取代码开发、页面验收或发布流程 |
| 修改或审查文档;实现或审查用法、公开合同、职责、约束或运维操作的变化 | [文档检查](references/documentation.md)及相关章节,尚无文档差异时也检查是否漏更;纯文档任务不默认读取代码开发、页面验收或发布流程 |
| 代码审查,包括 PR 审查 | [审查标准](../../../CONTRIBUTING.md#审查标准),叠加被审查行为所属领域的规范;审查默认只读 |
| PR / Issue / 发版 | 按当前意图选 [PR](../cpr-github-pr/SKILL.md)、[Issue](../cpr-github-issue/SKILL.md)或[发版](../cpr-release/SKILL.md),普通提交不创建 PR |
@@ -36,7 +36,7 @@ description: Codex Proxy RS 开发、修复、排障、代码审查和文档修
1. 遵守已提供的 [AGENTS.md](../../../AGENTS.md),按其 CodeGraph 约定定位代码;确认工作区、任务范围与已有改动,找到事实归属、同类实现和调用链,补齐上表命中规范及[项目约定](../../../CONTRIBUTING.md#项目约定)对应条目后再修改
2. 以最终差异重新匹配上表,补齐新增领域的规范并检查实际实现,再处理本次引入的职责越界、重复规则、冗余状态与无依据分支;删除或替换功能时一并清理失效的配置、类型、调用、测试与文档,已部署数据和公开合同仍遵守所属迁移及兼容规则;仅审查时报告,不自行修复
3. 常规文档只写当前状态;有文档差异时执行[文档检查](references/documentation.md),没有时只确认现有说明是否失真,普通开发不自动修改 `release/notes.md`
3. 按最终行为执行[文档检查](references/documentation.md),核对所属正文及相关概述、示例和引用,在本任务内完成必要同步;不以尚无文档差异跳过检查。内部重构等未改变文档所述事实时无需改文档;常规文档只写当前状态,普通开发不自动修改 `release/notes.md`
4. 按变更范围执行[验证](../../../CONTRIBUTING.md#验证)及其停止条件,后端命令见[验证命令](../../../docs/architecture.md#验证命令);沿用本轮有效证据与用户已确定的验收分工,区分通过、失败、跳过与未执行;提交时补读[提交约定](../../../CONTRIBUTING.md#项目约定)
5. 交付实际结果与验证缺口;前端按[前端流程](references/frontend.md)核对最终差异和运行证据,后端按上表检查生命周期,不把构建通过当作行为验收
@@ -1,14 +1,22 @@
# 文档检查
修改或审查文档时完整读取本流程,并读取 [文档职责与更新条件](../../../../CONTRIBUTING.md#文档职责与更新条件)的完整章节,覆盖内容归属与层次行文
修改或审查文档,以及实现或审查涉及文档更新条件的行为变化时,完整读取本流程和 [文档职责与更新条件](../../../../CONTRIBUTING.md#文档职责与更新条件),覆盖内容归属与层次行文
## 核实与同步范围
- 以当前目标分支及任务内工作区的实现、配置和有效运行证据核实行为;提交记录用于发现待核实的主题,不能直接当作现行事实或按提交顺序组织正文。按 [CodeGraph 约定](../../../../AGENTS.md#codegraph)同步索引后定位,证据不足时报告缺口,不把计划或推测写成已支持能力
- 开发任务按最终行为检查哪些用法、合同、职责、约束或运维操作需要同步,即使尚无 Markdown 差异也要检查;独立文档更新按用户范围核对现行说明,不等到发版时再补普通开发所需的文档
- 沿同一事实查找所属正文及相关概述、示例和引用,重点核对“支持/不支持”等能力声明、默认值、配置模板和命令。概述保留必要摘要并链接详细合同;不能只更新详细章节,留下 README 或示例中的矛盾说明,也不扩展为无关全仓改写
## 写之前
先确定读者、必须说明的当前事实与所属章节,再决定是否需要修改及详略;文档职责与更新条件按上节核对。涉及合同或职责说明时,补读所属架构、API 或 SDK 的对应条款,核实事实;不因文档提及实现而执行无关的开发与构建流程
在所属章节就地替换失效说明,先组织读者现在需要的操作、结果和限制,不从变更摘要改写正文。仅介绍被替代实现、修复动机或修改经过的段落直接删除;删掉“新增”“此前”“不再”等词后仍在回答变更经过的内容,也不保留
## 逐段验收
检查工作区和暂存区中本任务的最终文档差异,按内容用途处理,不只搜索关键词:
检查工作区和暂存区中本任务的最终文档差异,并连同受影响章节及相关概述、示例通读,按内容用途处理,不只搜索关键词:
| 段落回答的问题 | 处理 |
| --- | --- |
@@ -26,6 +34,6 @@
- 完成上述逐段验收及[行文检查](../../../../CONTRIBUTING.md#层次与行文),发现问题先处理,未完成内容审查不能称文档已检查
- 检查相对链接、章节锚点、表格与命令示例,`git diff --check` 或构建通过不能代替内容审查
- 报告实际覆盖和缺口即可,不把检查记录写进常规文档
- 交付时简述已同步的文档范围;涉及更新条件却未改文档时,说明现有内容为何仍准确或还有什么缺口。仅报告实际核对范围,不把链接检查通过称为事实核实,也不把检查记录写进常规文档
只有明确的发布说明任务才进入 `cpr-release`;发布说明与常规文档的边界按[内容归属](../../../../CONTRIBUTING.md#内容归属)执行
+1
View File
@@ -34,6 +34,7 @@ Thumbs.db
# Local tools and worktrees
/.codegraph/
/.playwright-cli/
/.codex-ws-audit/
/.reasonix/
/.claude
+1
View File
@@ -28,6 +28,7 @@ Issue 和 PR 回复统一按 [GitHub 回复口吻](CONTRIBUTING.md#github-回复
In repositories indexed by CodeGraph (a `.codegraph/` directory exists at the repo root), reach for it BEFORE grep/find or reading files when you need to understand or locate code:
- **Index freshness**: Run `codegraph sync` from the repository root before the first code query in a task. After pulling updates, switching branches, or changing code, run it again before the next query so symbols and call paths match the working tree. If synchronization fails, report the failure and verify against current source rather than treating the index as current.
- **MCP tools** (when available): `codegraph_explore` answers most code questions in one call — the relevant symbols' verbatim source plus the call paths between them. `codegraph_node` returns one symbol's source + callers, or reads a whole file with line numbers. If the tools are listed but deferred, load them by name via tool search.
- **Shell** (always works): `codegraph explore "<symbol names or question>"` and `codegraph node <symbol-or-file>` print the same output.
+2 -2
View File
@@ -17,7 +17,7 @@
</div>
> [!NOTE]
> 本项目提供 Responses API,不支持 `/v1/chat/completions`,接入前请确认客户端支持 Responses 协议
> 本项目提供 Responses API 和 Chat Completions API;Chat Completions 请求通过 Responses 链路执行,支持范围见 [API 参考](docs/api.md#chat-completions-请求)
## 快速预览
@@ -97,7 +97,7 @@ API Key 持有者可在登录页切换身份,查看自己的用量与额度,
完整步骤、生图配置与排障见[客户端配置](deploy/README.md#客户端配置)
**其他 Responses API 客户端**:填写以下信息:
**其他 Responses / Chat Completions API 客户端**:填写以下信息:
| 配置 | 值 |
| --- | --- |
@@ -1,9 +1,4 @@
//! 容量熔断冻结的恢复编排:到期探测、失败顺延与自适应并发下调
//!
//! 冻结事实保存在可丢失的 Redis 冷却中,本服务只组合 Admin 端口:
//! 观测(`AccountRuntimeStore`)、探测(`AccountsService`)与写回
//! (原子降低并发 / 按冻结代次结束探测)
//! 旧观测不会覆盖管理员操作
//! 账号错误保护的到期探测与按冻结快照恢复
use std::collections::BTreeMap;
use std::sync::Arc;
@@ -11,7 +6,7 @@ use std::time::{Duration, SystemTime};
use chrono::{DateTime, Utc};
use futures::StreamExt as _;
use gateway_core::account::{AccountConcurrencyLimit, ProviderAccountId};
use gateway_core::account::{AccountErrorProtection, ProviderAccountId};
use gateway_core::routing::UpstreamModelId;
use gateway_core::task::{ScheduledTask, WorkerCycleContext, WorkerTaskError};
use tracing::warn;
@@ -19,14 +14,9 @@ use tracing::warn;
use crate::model::accounts::{
AccountConnectionTestEvent, AccountFreeze, AccountPageItem, AccountRuntimeSnapshot,
};
use crate::model::{MutationActor, MutationContext};
use crate::ports::store::{AccountRuntimeStore, AccountStore, SettingsStore};
use crate::use_case::accounts::AccountsService;
/// 自适应并发下调保留系数:下调到观测在途峰值的 80%
const ADAPTIVE_CONCURRENCY_FACTOR: f64 = 0.8;
/// 自适应并发下限:过低的并发让账号几乎不可用,宁可保持冻结
const ADAPTIVE_CONCURRENCY_FLOOR: u32 = 2;
/// worker 周期;冷却观测与探测都依赖该节奏
pub const FREEZE_RECOVERY_INTERVAL: Duration = Duration::from_secs(30);
pub const FREEZE_RECOVERY_WORKER_OWNER: &str = "account-freeze-recovery";
@@ -34,9 +24,6 @@ pub const WORKER_INITIAL_BACKOFF: Duration = Duration::from_secs(1);
pub const WORKER_MAXIMUM_BACKOFF: Duration = Duration::from_secs(60);
pub const WORKER_LEASE_TTL: Duration = Duration::from_secs(15 * 60);
pub const WORKER_LEASE_RENEWAL: Duration = Duration::from_secs(5 * 60);
/// 系统审计上下文的固定请求 ID;账号并发更新审计由此溯源到自动冻结编排
const SYSTEM_REQUEST_ID: &str = "account-freeze-recovery";
/// 冻结恢复所需的管理端口组合
#[derive(Clone)]
pub struct FreezeRecoveryDeps {
@@ -55,26 +42,12 @@ impl FreezeRecoveryTask {
Self { deps }
}
fn system_context() -> MutationContext {
MutationContext {
actor: MutationActor::System,
request_id: SYSTEM_REQUEST_ID.to_owned(),
}
}
/// 读取冻结策略;读取或校验失败跳过本轮,不能把未知配置当作允许解冻
async fn freeze_policy(&self) -> Option<gateway_core::provider_ports::ProviderFreezePolicy> {
/// 无法读取或校验配置时跳过本轮,不能把未知状态当作允许解冻
async fn freeze_policy(&self) -> Option<AccountErrorProtection> {
let settings = self.deps.settings.load_runtime_settings().await.ok()?;
gateway_core::provider_ports::ProviderFreezePolicy::try_new(
settings.values.account_auto_freeze_enabled,
settings.values.account_auto_freeze_threshold,
settings.values.account_auto_freeze_window_seconds,
settings.values.account_auto_freeze_duration_seconds,
settings.values.account_auto_freeze_probe_enabled,
settings.values.account_auto_freeze_probe_model,
settings.values.account_auto_freeze_adaptive_concurrency,
)
.ok()
let policy = settings.values.account_error_protection;
policy.validate().ok()?;
Some(policy)
}
async fn active_freezes(&self) -> BTreeMap<String, AccountFreeze> {
@@ -89,55 +62,9 @@ impl FreezeRecoveryTask {
if freezes.is_empty() {
return;
}
if policy.enabled() && policy.adaptive_concurrency() {
self.adapt_concurrency_limits(&freezes, &policy).await;
}
self.recover_due_freezes(&freezes, &policy).await;
}
/// 对冻结中的账号执行自适应并发下调:目标为观测在途峰值的 80%(下限 2),
/// 只降不升;未观测到峰值证据的账号保持现状
async fn adapt_concurrency_limits(
&self,
freezes: &BTreeMap<String, AccountFreeze>,
policy: &gateway_core::provider_ports::ProviderFreezePolicy,
) {
let account_ids = freezes.keys().cloned().collect::<Vec<_>>();
let Ok(peaks) = self.deps.runtime.capacity_peaks(&account_ids).await else {
return;
};
for (account_id, peak) in peaks {
let Some(target) = adaptive_target(peak) else {
continue;
};
let Ok(account) = ProviderAccountId::new(account_id.clone()) else {
continue;
};
let Some(limit) = AccountConcurrencyLimit::new(target) else {
continue;
};
match self
.deps
.accounts
.lower_concurrency_limit(&Self::system_context(), account, limit)
.await
{
Ok(Some(_)) => {
tracing::info!(
account_id,
adapted_limit = target,
freeze_seconds = policy.freeze_duration().as_secs(),
"容量熔断:按观测在途峰值下调账号并发上限",
);
}
Ok(None) => {}
Err(error) => {
warn!(account_id, error = %error, "自适应并发下调失败");
}
}
}
}
async fn load_account(&self, account_id: &str) -> Option<Option<AccountPageItem>> {
let account_id = ProviderAccountId::new(account_id.to_owned()).ok()?;
self.deps
@@ -151,13 +78,13 @@ impl FreezeRecoveryTask {
async fn recover_due_freezes(
&self,
freezes: &BTreeMap<String, AccountFreeze>,
policy: &gateway_core::provider_ports::ProviderFreezePolicy,
policy: &AccountErrorProtection,
) {
for (account_id, freeze) in freezes {
if freeze.until > Utc::now() {
continue;
}
if policy.enabled() && policy.probe_enabled() && freeze.requires_probe {
if policy.enabled && policy.probe_enabled && freeze.requires_probe {
self.probe_and_recover(account_id, freeze, policy).await;
} else if let Err(error) = self
.deps
@@ -174,11 +101,19 @@ impl FreezeRecoveryTask {
&self,
account_id: &str,
freeze: &AccountFreeze,
policy: &gateway_core::provider_ports::ProviderFreezePolicy,
policy: &AccountErrorProtection,
) {
let Some(Some(item)) = self.load_account(account_id).await else {
return;
};
if item.account.credential_revision != freeze.credential_revision {
let _ = self
.deps
.runtime
.finish_freeze(account_id, freeze, None)
.await;
return;
}
if !item.account.enabled {
// 停用账号没有自动恢复意义;解冻交给管理员手动恢复
return;
@@ -186,7 +121,7 @@ impl FreezeRecoveryTask {
let Ok(account) = ProviderAccountId::new(account_id.to_owned()) else {
return;
};
let Some(model) = self.resolve_probe_model(&account, policy).await else {
let Some(model) = self.resolve_probe_model(&account, freeze, policy).await else {
// 没有可用探测模型时按失败处理,顺延冻结等待下一轮
self.postpone(&account, freeze, policy).await;
return;
@@ -224,9 +159,23 @@ impl FreezeRecoveryTask {
&self,
account: &ProviderAccountId,
freeze: &AccountFreeze,
policy: &gateway_core::provider_ports::ProviderFreezePolicy,
policy: &AccountErrorProtection,
) {
let Some(until) = SystemTime::now().checked_add(policy.freeze_duration()) else {
// 探测期间关闭保护或探测时,不再把已经到期的冻结向后延长
let Some(current) = self.freeze_policy().await else {
return;
};
if !current.enabled || !current.probe_enabled {
let _ = self
.deps
.runtime
.finish_freeze(account.as_str(), freeze, None)
.await;
return;
}
let Some(until) = SystemTime::now().checked_add(Duration::from_secs(
freeze.duration_seconds.unwrap_or(policy.duration_seconds),
)) else {
return;
};
let until = DateTime::<Utc>::from(until);
@@ -239,7 +188,10 @@ impl FreezeRecoveryTask {
Ok(true) => {
tracing::info!(
account_id = account.as_str(),
postpone_seconds = policy.freeze_duration().as_secs(),
postpone_seconds = Duration::from_secs(
freeze.duration_seconds.unwrap_or(policy.duration_seconds)
)
.as_secs(),
"冻结恢复探测失败:账号冻结顺延",
);
}
@@ -257,9 +209,14 @@ impl FreezeRecoveryTask {
async fn resolve_probe_model(
&self,
account: &ProviderAccountId,
policy: &gateway_core::provider_ports::ProviderFreezePolicy,
freeze: &AccountFreeze,
policy: &AccountErrorProtection,
) -> Option<UpstreamModelId> {
if let Some(model) = policy.probe_model() {
if let Some(model) = if freeze.duration_seconds.is_some() {
freeze.probe_model.as_deref()
} else {
policy.probe_model.as_deref()
} {
return UpstreamModelId::new(model.to_owned()).ok();
}
let models = self.deps.accounts.models(account, false).await.ok()?;
@@ -282,15 +239,6 @@ impl ScheduledTask for FreezeRecoveryTask {
}
}
/// 自适应目标并发:观测峰值 × 0.8 向下取整,下限 2;无证据(峰值为 0)不调整
fn adaptive_target(peak_in_flight: u32) -> Option<u32> {
if peak_in_flight == 0 {
return None;
}
let scaled = (f64::from(peak_in_flight) * ADAPTIVE_CONCURRENCY_FACTOR).floor() as u32;
Some(scaled.max(ADAPTIVE_CONCURRENCY_FLOOR))
}
/// 消耗连接测试事件流并返回是否以 `Completed` 终止
async fn drain_probe(mut events: crate::model::accounts::AccountConnectionTestEventStream) -> bool {
let mut completed = false;
@@ -81,6 +81,7 @@ pub struct AccountListQuery {
/// Admin query service 从运行态存储取得的当前账号冷却快照
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct AccountRuntimeSnapshot {
pub protection_rules: BTreeMap<String, Vec<gateway_core::account::AccountProtectionMatch>>,
pub cooldown: BTreeMap<String, gateway_core::account::AccountCooldown>,
/// `None` 表示实时 lease 存储不可用;`Some` 中未出现的账号当前使用量为零
pub in_flight: Option<BTreeMap<String, u64>>,
@@ -93,6 +94,9 @@ pub struct AccountFreeze {
pub until: DateTime<Utc>,
pub generation: String,
pub requires_probe: bool,
pub duration_seconds: Option<u64>,
pub probe_model: Option<String>,
pub protection_rules: Vec<gateway_core::account::AccountProtectionMatch>,
}
/// 可选的账号分组成员筛选条件
@@ -203,6 +207,7 @@ pub struct AccountPage {
/// 同一状态快照下的账号事实与唯一状态投影
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct AccountPageItem {
pub protection_rules: Vec<gateway_core::account::AccountProtectionMatch>,
pub account: AccountRecord,
pub projection: AccountStatusProjection,
pub capacity: AccountCapacity,
@@ -2,7 +2,9 @@
use chrono::{DateTime, Utc};
use super::observability::{DecimalAmount, RequestMetricPoint, UsageOverview};
use super::observability::{
DecimalAmount, RequestMetricPoint, UsageConcurrencyPoint, UsageOverview,
};
/// 数据可见性状态,不以缺失投影冒充零统计
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
@@ -57,9 +59,10 @@ pub struct UsageProjectedCostPoint {
}
/// 同一数据库快照内的完整洞察输入
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq)]
pub struct UsageInsightsObservation {
pub overview: UsageOverview,
pub trend: Vec<RequestMetricPoint>,
pub costs: Vec<UsageProjectedCostPoint>,
pub concurrency: Vec<UsageConcurrencyPoint>,
}
@@ -23,7 +23,6 @@ pub enum MutationAuditOperation {
BackupDelete,
BackupConnectionTest,
BackupDownloadUrl,
ProviderAccountAdaptConcurrency,
ProviderAccountRecover,
ProviderAccountBatchUpdate,
ProviderAccountDelete,
@@ -84,7 +83,6 @@ impl MutationAuditOperation {
Self::BackupDelete => ("backup.delete_requested", "backup"),
Self::BackupConnectionTest => ("backup.s3_connection_tested", "backup_settings"),
Self::BackupDownloadUrl => ("backup.download_url_created", "backup"),
Self::ProviderAccountAdaptConcurrency => ("adapt_concurrency", "provider_account"),
Self::ProviderAccountRecover => ("recover", "provider_account"),
Self::ProviderAccountBatchUpdate => ("batch_update", "provider_account"),
Self::ProviderAccountDelete => ("delete", "provider_account"),
@@ -653,6 +653,7 @@ pub struct DashboardAccountUsage {
pub authentication_kind: String,
pub name: String,
pub email: Option<String>,
pub notes: Option<String>,
pub plan_type: Option<String>,
/// Admin 按 Provider 补齐的套餐展示名称
pub plan_type_display: Option<String>,
@@ -1260,38 +1261,44 @@ pub struct UsageInsightsHealth {
pub points: Vec<UsageInsightsHealthPoint>,
}
/// 一个时间桶内按重叠时长加权的逻辑请求并发
#[derive(Debug, Clone, PartialEq)]
pub struct UsageConcurrencyPoint {
pub bucket_start: DateTime<Utc>,
pub average_concurrency: f64,
pub peak_concurrency: u64,
}
/// 请求并发与调度等待的负载指标
#[derive(Debug, Clone, PartialEq)]
pub struct UsageInsightsLoad {
pub average_concurrency: f64,
pub peak_concurrency: u64,
pub admission_decision_p95_ms: Option<PercentileMilliseconds>,
pub account_selection_wait_p95_ms: Option<PercentileMilliseconds>,
}
/// 洞察性能时间点
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq)]
pub struct UsageInsightsPerformancePoint {
pub bucket_start: DateTime<Utc>,
pub latency_percentiles: LatencyPercentiles,
pub first_token_latency_percentiles: LatencyPercentiles,
pub admission_decision_percentiles: LatencyPercentiles,
pub account_selection_wait_percentiles: LatencyPercentiles,
pub output_throughput_p10: Option<u64>,
pub output_throughput_p50: Option<u64>,
pub output_throughput_p90: Option<u64>,
pub capacity_utilization_avg_basis_points: Option<u64>,
pub capacity_utilization_p95_basis_points: Option<u64>,
pub load: UsageInsightsLoad,
}
/// 洞察性能汇总与可观测覆盖率
/// 洞察性能汇总
#[derive(Debug, Clone, PartialEq)]
pub struct UsageInsightsPerformance {
pub latency_percentiles: LatencyPercentiles,
pub first_token_latency_percentiles: LatencyPercentiles,
pub latency_coverage: f64,
pub first_token_coverage: f64,
pub admission_decision_percentiles: LatencyPercentiles,
pub account_selection_wait_percentiles: LatencyPercentiles,
pub admission_decision_coverage: f64,
pub account_selection_wait_coverage: f64,
pub output_throughput_p10: Option<u64>,
pub output_throughput_p50: Option<u64>,
pub output_throughput_p90: Option<u64>,
pub capacity_utilization_avg_basis_points: Option<u64>,
pub capacity_utilization_p95_basis_points: Option<u64>,
pub capacity_coverage: f64,
pub load: UsageInsightsLoad,
pub points: Vec<UsageInsightsPerformancePoint>,
}
@@ -1246,6 +1246,7 @@ impl fmt::Debug for ProviderExport {
/// 统一账号目录的一行完整结果
#[derive(Debug, Clone, PartialEq)]
pub struct AccountDirectoryItem {
pub protection_rules: Vec<gateway_core::account::AccountProtectionMatch>,
pub account: AccountRecord,
pub capacity: super::accounts::AccountCapacity,
pub capabilities: super::accounts::ProviderAccountCapabilities,
@@ -65,13 +65,7 @@ pub struct RuntimeSettingsValues {
pub usage_retention_days: u32,
pub ops_event_retention_days: u32,
pub audit_retention_days: u32,
pub account_auto_freeze_enabled: bool,
pub account_auto_freeze_threshold: u32,
pub account_auto_freeze_window_seconds: u64,
pub account_auto_freeze_duration_seconds: u64,
pub account_auto_freeze_probe_enabled: bool,
pub account_auto_freeze_probe_model: Option<String>,
pub account_auto_freeze_adaptive_concurrency: bool,
pub account_error_protection: gateway_core::account::AccountErrorProtection,
pub account_warmup_enabled: bool,
pub account_warmup_schedule_time: String,
pub account_warmup_model: Option<String>,
@@ -232,12 +226,7 @@ impl RuntimeSettingsValues {
self.ops_event_retention_days,
self.audit_retention_days,
)?;
gateway_core::provider_ports::ProviderFreezePolicy::validate_values(
self.account_auto_freeze_threshold,
self.account_auto_freeze_window_seconds,
self.account_auto_freeze_duration_seconds,
self.account_auto_freeze_probe_model.as_deref(),
)?;
self.account_error_protection.validate()?;
gateway_core::provider_ports::ProviderWarmupPolicy::validate_values(
self.account_warmup_enabled,
&self.account_warmup_schedule_time,
@@ -209,14 +209,6 @@ pub trait AccountStore:
context: &MutationContext,
) -> AdminStoreResult<AccountUpdateResult>;
/// 在事务内按最新启用状态、账号上限和全局默认值判断,只降低并发上限
async fn lower_concurrency_limit(
&self,
account_id: &gateway_core::account::ProviderAccountId,
limit: gateway_core::account::AccountConcurrencyLimit,
context: &MutationContext,
) -> AdminStoreResult<Option<AccountUpdateResult>>;
async fn recover_account(
&self,
account_id: &gateway_core::account::ProviderAccountId,
@@ -259,17 +251,11 @@ pub trait AccountRuntimeStore: Send + Sync {
account_ids: &[String],
) -> AdminStoreResult<AccountRuntimeSnapshot>;
/// 容量熔断自动冻结中的账号与其冻结截止时间;429 临时限流不包含在内
/// 账号错误保护中的账号与其冻结快照;临时限流不包含在内
async fn active_freezes(
&self,
) -> AdminStoreResult<BTreeMap<String, crate::model::accounts::AccountFreeze>>;
/// 读取容量失败窗口内观测到的在途并发峰值(自适应并发下调的证据)
async fn capacity_peaks(
&self,
account_ids: &[String],
) -> AdminStoreResult<BTreeMap<String, u32>>;
/// 仅当冻结快照仍匹配时解除或顺延;旧探测不得覆盖手动恢复或新一轮冻结
async fn finish_freeze(
&self,
@@ -88,13 +88,6 @@ pub trait AccountsService: Send + Sync {
command: UpdateAccount,
) -> Result<AccountUpdateResult, AdminError>;
async fn lower_concurrency_limit(
&self,
context: &MutationContext,
account_id: ProviderAccountId,
limit: gateway_core::account::AccountConcurrencyLimit,
) -> Result<Option<AccountUpdateResult>, AdminError>;
async fn batch_update(
&self,
context: &MutationContext,
@@ -474,6 +467,7 @@ impl DefaultAccountsService {
.and_then(|(window, _)| window.local_usage.clone())
});
Ok(AccountDirectoryItem {
protection_rules: stored.protection_rules,
capacity: stored.capacity,
capabilities: provider
.account_capabilities(account_id, &stored.account.authentication_kind),
@@ -583,6 +577,7 @@ impl AccountsService for DefaultAccountsService {
.and_then(|(window, _)| window.local_usage.clone())
});
Ok(AccountDirectoryItem {
protection_rules: item.protection_rules,
capacity: crate::model::accounts::AccountCapacity {
used_slots: in_flight
.as_ref()
@@ -766,27 +761,6 @@ impl AccountsService for DefaultAccountsService {
Ok(result)
}
async fn lower_concurrency_limit(
&self,
context: &MutationContext,
account_id: ProviderAccountId,
limit: gateway_core::account::AccountConcurrencyLimit,
) -> Result<Option<AccountUpdateResult>, AdminError> {
let (_, provider) = self.provider_for_account(&account_id).await?;
let result = self
.accounts
.lower_concurrency_limit(&account_id, limit, context)
.await
.map_err(|error| map_store_error(error, "provider account"))?;
if let Some(result) = &result {
provider
.account_facts_changed(std::slice::from_ref(&account_id))
.await;
publish_committed(self.snapshot.as_ref(), result.config_revision).await?;
}
Ok(result)
}
async fn batch_update(
&self,
context: &MutationContext,
@@ -20,9 +20,10 @@ use crate::{
HealthTimeline, HealthTimelinePoint, OpsErrorPage, OpsErrorQuery, ProviderBillingInput,
RequestMetricPoint, RequestMetrics, TimeRange, Trend, TrendKind, TrendPoint,
TrendSummary, UsageBilling, UsageBillingProjection, UsageCalculatedBillingFact,
UsageDetail, UsageFilter, UsageInsights, UsageInsightsCost, UsageInsightsCostPoint,
UsageInsightsHealth, UsageInsightsHealthPoint, UsageInsightsPerformance,
UsageInsightsPerformancePoint, UsageOverview, UsagePage, UsageQuery, UsageSummary,
UsageConcurrencyPoint, UsageDetail, UsageFilter, UsageInsights, UsageInsightsCost,
UsageInsightsCostPoint, UsageInsightsHealth, UsageInsightsHealthPoint,
UsageInsightsLoad, UsageInsightsPerformance, UsageInsightsPerformancePoint,
UsageOverview, UsagePage, UsageQuery, UsageSummary,
},
provider_credentials::ProviderQuotaRequest,
},
@@ -304,7 +305,7 @@ impl ObservabilityService for DefaultObservabilityService {
}
let overview = observation.overview;
let trend = observation.trend;
build_usage_insights(overview, trend, standard_costs)
build_usage_insights(overview, trend, standard_costs, observation.concurrency)
}
async fn diagnostics(
@@ -385,6 +386,7 @@ fn build_usage_insights(
overview: UsageOverview,
trend: Vec<RequestMetricPoint>,
standard_costs: UsageCostScenarios,
concurrency: Vec<UsageConcurrencyPoint>,
) -> Result<UsageInsights, AdminError> {
let granularity = trend.first().map_or(
crate::model::observability::Granularity::FifteenMinutes,
@@ -432,49 +434,69 @@ fn build_usage_insights(
})
.collect(),
};
let concurrency_by_bucket = concurrency
.iter()
.map(|point| (point.bucket_start, point))
.collect::<BTreeMap<_, _>>();
let mut weighted_concurrency_micros = 0.0;
let mut effective_micros = 0_i64;
for (index, point) in concurrency.iter().enumerate() {
let start = point.bucket_start.max(overview.range.start);
let end = concurrency
.get(index + 1)
.map_or(overview.range.end, |next| next.bucket_start)
.min(overview.range.end);
let micros = (end - start).num_microseconds().unwrap_or(0).max(0);
effective_micros = effective_micros.saturating_add(micros);
weighted_concurrency_micros += point.average_concurrency * micros as f64;
}
let average_concurrency = if effective_micros > 0 {
weighted_concurrency_micros / effective_micros as f64
} else {
0.0
};
let peak_concurrency = concurrency
.iter()
.map(|point| point.peak_concurrency)
.max()
.unwrap_or(0);
let performance = UsageInsightsPerformance {
latency_percentiles: requests.latency_percentiles,
first_token_latency_percentiles: requests.first_token_latency_percentiles,
latency_coverage: rate_or_zero(requests.latency_count, requests.request_count),
first_token_coverage: rate_or_zero(
requests.first_token_latency_count,
requests.request_count,
),
admission_decision_percentiles: requests.admission_decision_percentiles,
account_selection_wait_percentiles: requests.account_selection_wait_percentiles,
admission_decision_coverage: rate_or_zero(
requests.admission_decision_count,
requests.request_count,
),
account_selection_wait_coverage: rate_or_zero(
requests.account_selection_wait_count,
requests.request_count,
),
output_throughput_p10: requests.output_throughput_p10,
output_throughput_p50: requests.output_throughput_p50,
output_throughput_p90: requests.output_throughput_p90,
capacity_utilization_avg_basis_points: requests.capacity_utilization_avg_basis_points,
capacity_utilization_p95_basis_points: requests.capacity_utilization_p95_basis_points,
capacity_coverage: rate_or_zero(requests.capacity_sample_count, requests.request_count),
load: UsageInsightsLoad {
average_concurrency,
peak_concurrency,
admission_decision_p95_ms: requests.admission_decision_percentiles.p95_ms,
account_selection_wait_p95_ms: requests.account_selection_wait_percentiles.p95_ms,
},
points: trend
.iter()
.map(|point| UsageInsightsPerformancePoint {
bucket_start: point.bucket_start,
latency_percentiles: point.metrics.latency_percentiles,
first_token_latency_percentiles: point.metrics.first_token_latency_percentiles,
admission_decision_percentiles: point.metrics.admission_decision_percentiles,
account_selection_wait_percentiles: point
.metrics
.account_selection_wait_percentiles,
output_throughput_p10: point.metrics.output_throughput_p10,
output_throughput_p50: point.metrics.output_throughput_p50,
output_throughput_p90: point.metrics.output_throughput_p90,
capacity_utilization_avg_basis_points: point
.metrics
.capacity_utilization_avg_basis_points,
capacity_utilization_p95_basis_points: point
.metrics
.capacity_utilization_p95_basis_points,
.map(|point| {
let concurrency = concurrency_by_bucket.get(&point.bucket_start);
UsageInsightsPerformancePoint {
bucket_start: point.bucket_start,
latency_percentiles: point.metrics.latency_percentiles,
first_token_latency_percentiles: point.metrics.first_token_latency_percentiles,
output_throughput_p10: point.metrics.output_throughput_p10,
output_throughput_p50: point.metrics.output_throughput_p50,
output_throughput_p90: point.metrics.output_throughput_p90,
load: UsageInsightsLoad {
average_concurrency: concurrency
.map_or(0.0, |point| point.average_concurrency),
peak_concurrency: concurrency.map_or(0, |point| point.peak_concurrency),
admission_decision_p95_ms: point
.metrics
.admission_decision_percentiles
.p95_ms,
account_selection_wait_p95_ms: point
.metrics
.account_selection_wait_percentiles
.p95_ms,
},
}
})
.collect(),
};
@@ -315,6 +315,7 @@ impl AccountRuntimeStore for FakeRuntimeStore {
) -> AdminStoreResult<AccountRuntimeSnapshot> {
*self.requested_accounts.lock().expect("requested accounts") = account_ids.to_vec();
Ok(AccountRuntimeSnapshot {
protection_rules: std::collections::BTreeMap::new(),
cooldown: BTreeMap::from([(
"acct_limited".to_owned(),
std::time::SystemTime::from(Utc::now() + Duration::minutes(5)).into(),
@@ -329,13 +330,6 @@ impl AccountRuntimeStore for FakeRuntimeStore {
Ok(BTreeMap::new())
}
async fn capacity_peaks(
&self,
_account_ids: &[String],
) -> AdminStoreResult<BTreeMap<String, u32>> {
Ok(BTreeMap::new())
}
async fn finish_freeze(
&self,
_account_id: &str,
@@ -617,7 +617,6 @@ pub(super) struct FakeAccountStore {
automations: Mutex<
BTreeMap<String, gateway_admin::model::reset_credit_automation::ResetCreditAutomation>,
>,
pub(super) lowered_limits: Mutex<Vec<(String, u32)>>,
}
impl FakeAccountStore {
@@ -638,7 +637,6 @@ impl FakeAccountStore {
quota_window_queries: Mutex::new(Vec::new()),
quota_forecast_history: Mutex::new(QuotaForecastHistory::default()),
update_commands: Mutex::new(Vec::new()),
lowered_limits: Mutex::new(Vec::new()),
automations: Mutex::default(),
})
}
@@ -720,6 +718,7 @@ impl FakeAccountStore {
last_error_message: account.last_error_message.clone(),
};
AccountPageItem {
protection_rules: Vec::new(),
capacity: gateway_admin::model::accounts::AccountCapacity {
used_slots: None,
total_slots: account
@@ -1036,35 +1035,6 @@ impl AccountStore for FakeAccountStore {
})
}
async fn lower_concurrency_limit(
&self,
account_id: &ProviderAccountId,
limit: gateway_core::account::AccountConcurrencyLimit,
_: &MutationContext,
) -> AdminStoreResult<Option<AccountUpdateResult>> {
let mut accounts = self.accounts.lock().expect("accounts");
let Some(account) = accounts
.iter_mut()
.find(|account| account.id == account_id.as_str())
else {
return Ok(None);
};
if !account.enabled
|| account.concurrency_limit.map_or(5, |value| value.get()) <= limit.get()
{
return Ok(None);
}
account.concurrency_limit = Some(limit);
self.lowered_limits
.lock()
.expect("lowered limits")
.push((account.id.clone(), limit.get()));
Ok(Some(AccountUpdateResult {
config_revision: revision(2),
account_id: account_id.clone(),
}))
}
async fn recover_account(
&self,
account_id: &ProviderAccountId,
@@ -1189,13 +1159,7 @@ impl SettingsStore for StaticSettingsStore {
usage_retention_days: 30,
ops_event_retention_days: 30,
audit_retention_days: 30,
account_auto_freeze_enabled: true,
account_auto_freeze_threshold: 12,
account_auto_freeze_window_seconds: 600,
account_auto_freeze_duration_seconds: 7_200,
account_auto_freeze_probe_enabled: true,
account_auto_freeze_probe_model: None,
account_auto_freeze_adaptive_concurrency: true,
account_error_protection: gateway_core::account::AccountErrorProtection::default(),
account_warmup_enabled: false,
account_warmup_schedule_time: "08:00".to_owned(),
account_warmup_model: None,
@@ -3680,6 +3644,7 @@ impl AccountRuntimeStore for CapacityRuntime {
return Err(store_unavailable());
}
Ok(AccountRuntimeSnapshot {
protection_rules: std::collections::BTreeMap::new(),
in_flight: self.counts.clone(),
..Default::default()
})
@@ -3691,10 +3656,6 @@ impl AccountRuntimeStore for CapacityRuntime {
Ok(BTreeMap::new())
}
async fn capacity_peaks(&self, _: &[String]) -> AdminStoreResult<BTreeMap<String, u32>> {
Ok(BTreeMap::new())
}
async fn finish_freeze(
&self,
_: &str,
@@ -1,4 +1,4 @@
//! 容量熔断冻结恢复编排回归:到期探测、失败顺延与自适应并发下调
//! 账号保护恢复编排:到期探测、快照参数与失败顺延
use std::collections::BTreeMap;
use std::sync::{Arc, Mutex};
@@ -24,7 +24,7 @@ use gateway_core::task::{
use super::AdminHarness;
use super::accounts::{FakeAccountStore, FakeProviderAdmin, account_record, events, revision};
fn runtime_settings(enabled: bool, probe_enabled: bool, adaptive: bool) -> RuntimeSettings {
fn runtime_settings(enabled: bool, probe_enabled: bool) -> RuntimeSettings {
RuntimeSettings {
request_profiles: Default::default(),
config_revision: revision(1),
@@ -52,13 +52,22 @@ fn runtime_settings(enabled: bool, probe_enabled: bool, adaptive: bool) -> Runti
usage_retention_days: 31,
ops_event_retention_days: 30,
audit_retention_days: 90,
account_auto_freeze_enabled: enabled,
account_auto_freeze_threshold: 12,
account_auto_freeze_window_seconds: 600,
account_auto_freeze_duration_seconds: 7_200,
account_auto_freeze_probe_enabled: probe_enabled,
account_auto_freeze_probe_model: Some("gpt-5.5".to_owned()),
account_auto_freeze_adaptive_concurrency: adaptive,
account_error_protection: gateway_core::account::AccountErrorProtection {
enabled,
probe_enabled,
probe_model: Some("gpt-5.5".to_owned()),
rules: vec![gateway_core::account::AccountErrorRule {
id: "sample".into(),
name: "示例".into(),
conditions: vec![gateway_core::account::AccountErrorCondition {
field: gateway_core::account::AccountErrorField::Code,
operator: gateway_core::account::AccountErrorOperator::Equals,
values: vec![Some("custom_failure".into())],
ignore_case: false,
}],
}],
..Default::default()
},
account_warmup_enabled: false,
account_warmup_schedule_time: "08:00".to_owned(),
account_warmup_model: None,
@@ -131,19 +140,17 @@ fn store_unavailable() -> AdminStoreError {
)
}
/// 记录解冻/顺延调用的运行态 fake;冻结与峰值证据由测试预置
/// 记录解冻与顺延调用,冻结快照由测试预置
struct FreezeRuntimeStore {
freezes: BTreeMap<String, AccountFreeze>,
peaks: BTreeMap<String, u32>,
cleared: Mutex<Vec<String>>,
extended: Mutex<Vec<(String, DateTime<Utc>)>>,
}
impl FreezeRuntimeStore {
fn new(freezes: BTreeMap<String, AccountFreeze>, peaks: BTreeMap<String, u32>) -> Arc<Self> {
fn new(freezes: BTreeMap<String, AccountFreeze>) -> Arc<Self> {
Arc::new(Self {
freezes,
peaks,
cleared: Mutex::new(Vec::new()),
extended: Mutex::new(Vec::new()),
})
@@ -188,16 +195,6 @@ impl AccountRuntimeStore for FreezeRuntimeStore {
Ok(self.freezes.clone())
}
async fn capacity_peaks(
&self,
account_ids: &[String],
) -> AdminStoreResult<BTreeMap<String, u32>> {
Ok(account_ids
.iter()
.filter_map(|id| self.peaks.get(id).map(|peak| (id.clone(), *peak)))
.collect())
}
async fn finish_freeze(
&self,
account_id: &str,
@@ -241,9 +238,10 @@ struct FailingProbe;
impl AccountProbe for FailingProbe {
fn probe(
&self,
_: AccountProbeRequest,
request: AccountProbeRequest,
_: Option<Arc<gateway_core::routing::RuntimeSnapshot>>,
) -> futures::future::BoxFuture<'_, Result<AccountProbeResult, AccountProbeError>> {
assert_eq!(request.upstream_model.as_str(), "snapshot-model");
Box::pin(async {
Err(AccountProbeError::new(
gateway_core::error::GatewayError::new(
@@ -317,6 +315,9 @@ fn freeze_until(until: DateTime<Utc>) -> BTreeMap<String, AccountFreeze> {
BTreeMap::from([(
"acct_test".to_owned(),
AccountFreeze {
duration_seconds: None,
probe_model: None,
protection_rules: Vec::new(),
credential_revision: revision(1),
until,
generation: "freeze-generation".to_owned(),
@@ -331,9 +332,9 @@ fn freeze_due() -> BTreeMap<String, AccountFreeze> {
#[tokio::test]
async fn probe_success_clears_freeze() {
let runtime = FreezeRuntimeStore::new(freeze_due(), BTreeMap::new());
let runtime = FreezeRuntimeStore::new(freeze_due());
let (task, _store) = recovery_task(
runtime_settings(true, true, false),
runtime_settings(true, true),
Arc::clone(&runtime),
Arc::new(SuccessfulProbe),
)
@@ -346,10 +347,13 @@ async fn probe_success_clears_freeze() {
}
#[tokio::test]
async fn probe_failure_postpones_freeze() {
let runtime = FreezeRuntimeStore::new(freeze_due(), BTreeMap::new());
async fn probe_failure_postpones_using_the_freeze_snapshot() {
let mut freezes = freeze_due();
freezes.get_mut("acct_test").unwrap().duration_seconds = Some(300);
freezes.get_mut("acct_test").unwrap().probe_model = Some("snapshot-model".into());
let runtime = FreezeRuntimeStore::new(freezes);
let (task, _store) = recovery_task(
runtime_settings(true, true, false),
runtime_settings(true, true),
Arc::clone(&runtime),
Arc::new(FailingProbe),
)
@@ -364,16 +368,16 @@ async fn probe_failure_postpones_freeze() {
assert_eq!(account_id, "acct_test");
let remaining = until.timestamp_millis() - Utc::now().timestamp_millis();
assert!(
remaining > 7_000_000 && remaining <= 7_200_000,
"postponed freeze should be about 2 hours, got {remaining}ms"
remaining > 290_000 && remaining <= 300_000,
"postponed freeze should use the frozen 5-minute duration, got {remaining}ms"
);
}
#[tokio::test]
async fn disabled_policy_releases_due_freeze_without_probing() {
let runtime = FreezeRuntimeStore::new(freeze_due(), BTreeMap::new());
let runtime = FreezeRuntimeStore::new(freeze_due());
let (task, _store) = recovery_task(
runtime_settings(false, true, false),
runtime_settings(false, true),
Arc::clone(&runtime),
Arc::new(SuccessfulProbe),
)
@@ -385,59 +389,12 @@ async fn disabled_policy_releases_due_freeze_without_probing() {
assert!(runtime.extended().is_empty());
}
#[tokio::test]
async fn adaptive_concurrency_lowers_limit_to_observed_peak() {
let runtime = FreezeRuntimeStore::new(
freeze_until(Utc::now() + TimeDelta::hours(1)),
BTreeMap::from([("acct_test".to_owned(), 4_u32)]),
);
let (task, store) = recovery_task(
runtime_settings(true, false, true),
Arc::clone(&runtime),
Arc::new(SuccessfulProbe),
)
.await;
run_cycle(task.as_ref()).await;
// 峰值 4 × 0.8 = 3(不低于下限 2),低于全局默认 5,应下调到 3
assert_eq!(
*store.lowered_limits.lock().expect("lowered limits"),
vec![("acct_test".to_owned(), 3)]
);
assert!(store.update_commands().is_empty());
assert!(runtime.extended().is_empty());
assert!(runtime.cleared().is_empty());
}
#[tokio::test]
async fn adaptive_concurrency_never_raises_limit() {
let runtime = FreezeRuntimeStore::new(
freeze_until(Utc::now() + TimeDelta::hours(1)),
BTreeMap::from([("acct_test".to_owned(), 40_u32)]),
);
let (task, store) = recovery_task(
runtime_settings(true, false, true),
Arc::clone(&runtime),
Arc::new(SuccessfulProbe),
)
.await;
run_cycle(task.as_ref()).await;
// 峰值 40 的目标 32 高于全局默认 5;只降不升,不下发任何更新
assert!(store.update_commands().is_empty());
}
/// 到期时间还早(超过探测提前量)的冻结不发起探测
#[tokio::test]
async fn probe_skips_freezes_far_from_expiry() {
let runtime = FreezeRuntimeStore::new(
freeze_until(Utc::now() + TimeDelta::hours(1)),
BTreeMap::new(),
);
let runtime = FreezeRuntimeStore::new(freeze_until(Utc::now() + TimeDelta::hours(1)));
let (task, _store) = recovery_task(
runtime_settings(true, true, false),
runtime_settings(true, true),
Arc::clone(&runtime),
Arc::new(SuccessfulProbe),
)
@@ -449,13 +406,10 @@ async fn probe_skips_freezes_far_from_expiry() {
assert!(runtime.extended().is_empty());
}
/// 停用账号不参与探测恢复;并发下调同样跳过
/// 停用账号不参与探测恢复
#[tokio::test]
async fn disabled_accounts_are_skipped() {
let runtime = FreezeRuntimeStore::new(
freeze_due(),
BTreeMap::from([("acct_test".to_owned(), 4_u32)]),
);
let runtime = FreezeRuntimeStore::new(freeze_due());
let mut record = account_record("openai");
record.enabled = false;
let store = FakeAccountStore::with_account(record, events());
@@ -463,7 +417,7 @@ async fn disabled_accounts_are_skipped() {
.accounts(Arc::clone(&store) as Arc<dyn AccountStore>)
.account_runtime(Arc::clone(&runtime) as Arc<dyn AccountRuntimeStore>)
.settings(Arc::new(FreezeSettingsStore {
settings: runtime_settings(true, true, true),
settings: runtime_settings(true, true),
}))
.provider(FakeProviderAdmin::new("openai", events()))
.probe(Arc::new(SuccessfulProbe))
@@ -614,15 +614,6 @@ impl AccountStore for UnavailableStore {
Err(unavailable("account enabled"))
}
async fn lower_concurrency_limit(
&self,
_: &gateway_core::account::ProviderAccountId,
_: gateway_core::account::AccountConcurrencyLimit,
_: &MutationContext,
) -> AdminStoreResult<Option<gateway_admin::model::accounts::AccountUpdateResult>> {
Ok(None)
}
async fn recover_account(
&self,
_: &ProviderAccountId,
@@ -685,13 +676,6 @@ impl AccountRuntimeStore for UnavailableStore {
Ok(BTreeMap::new())
}
async fn capacity_peaks(
&self,
_account_ids: &[String],
) -> AdminStoreResult<BTreeMap<String, u32>> {
Ok(BTreeMap::new())
}
async fn finish_freeze(
&self,
_account_id: &str,
@@ -22,8 +22,9 @@ use gateway_admin::{
DiagnosticDimension, DiagnosticObservation, DiagnosticsObservation, Granularity,
HealthStatus, LatencyPercentiles, ObservabilityPageSize, OpsErrorPage, OpsErrorQuery,
PercentileMilliseconds, RequestMetricPoint, RequestMetrics, TimeRange, TrendKind,
UsageBilling, UsageDetail, UsageFilter, UsageInsightsObservation, UsageListRecord,
UsageOverview, UsagePage, UsageProjectedCostPoint, UsageQuery, UsageSummaryObservation,
UsageBilling, UsageConcurrencyPoint, UsageDetail, UsageFilter,
UsageInsightsObservation, UsageListRecord, UsageOverview, UsagePage,
UsageProjectedCostPoint, UsageQuery, UsageSummaryObservation,
},
settings::{
AdminApiKey, AdminApiKeyMutation, ReplaceRuntimeSettings, RotationStrategy,
@@ -678,10 +679,26 @@ async fn observability_services_should_calculate_usage_insights_and_diagnostic_s
assert_eq!(insights.health.success_rate, 0.75);
assert_eq!(insights.health.completion_rate, 1.0);
assert_eq!(insights.granularity, Granularity::FifteenMinutes);
assert_eq!(insights.performance.latency_coverage, 0.5);
assert_eq!(insights.performance.admission_decision_coverage, 1.0);
assert_eq!(insights.performance.account_selection_wait_coverage, 0.8);
assert_eq!(insights.performance.capacity_coverage, 0.8);
assert_eq!(
insights
.performance
.load
.admission_decision_p95_ms
.map(|value| value.as_f64()),
Some(5.0)
);
assert_eq!(
insights
.performance
.load
.account_selection_wait_p95_ms
.map(|value| value.as_f64()),
Some(40.0)
);
assert_eq!(
insights.performance.points[0].load,
insights.performance.load
);
assert_eq!(insights.performance.output_throughput_p50, Some(50));
assert_eq!(insights.cost.tokens_per_request, 100.0);
assert_eq!(
@@ -742,6 +759,52 @@ async fn observability_services_should_calculate_usage_insights_and_diagnostic_s
assert_eq!(diagnostics.items[1].request_share, 0.25);
}
#[tokio::test]
async fn usage_insights_should_weight_concurrency_by_effective_bucket_duration() {
let bucket_start = DateTime::parse_from_rfc3339("2026-09-01T10:00:00Z")
.unwrap()
.with_timezone(&Utc);
let range = TimeRange::new(
bucket_start + Duration::minutes(5),
bucket_start + Duration::minutes(20),
)
.unwrap();
let store = Arc::new(FixtureObservabilityStore::new(range));
store.replace_trend(vec![
health_metric_point(bucket_start, RequestMetrics::default()),
health_metric_point(
bucket_start + Duration::minutes(15),
RequestMetrics::default(),
),
]);
*store.concurrency.lock().unwrap() = vec![
UsageConcurrencyPoint {
bucket_start,
average_concurrency: 1.0,
peak_concurrency: 1,
},
UsageConcurrencyPoint {
bucket_start: bucket_start + Duration::minutes(15),
average_concurrency: 3.0,
peak_concurrency: 4,
},
];
let insights = observability_services(store)
.await
.observability()
.usage_insights(range, UsageFilter::default())
.await
.unwrap();
assert!((insights.performance.load.average_concurrency - 5.0 / 3.0).abs() < 1e-10);
assert_eq!(insights.performance.load.peak_concurrency, 4);
assert_eq!(insights.performance.points[0].load.average_concurrency, 1.0);
assert_eq!(insights.performance.points[0].load.peak_concurrency, 1);
assert_eq!(insights.performance.points[1].load.average_concurrency, 3.0);
assert_eq!(insights.performance.points[1].load.peak_concurrency, 4);
}
#[tokio::test]
async fn diagnostics_should_preserve_request_order_when_a_small_sample_fails() {
let now = Utc::now();
@@ -937,6 +1000,7 @@ async fn dashboard_should_preserve_provider_plan_names() {
authentication_kind: "oauth".to_owned(),
name: (*plan).to_owned(),
email: None,
notes: None,
plan_type: Some((*plan).to_owned()),
plan_type_display: None,
request_count: 0,
@@ -987,6 +1051,7 @@ struct FixtureObservabilityStore {
trend: Mutex<Vec<RequestMetricPoint>>,
overview: Mutex<UsageOverview>,
projected_costs: Mutex<Vec<UsageProjectedCostPoint>>,
concurrency: Mutex<Vec<UsageConcurrencyPoint>>,
insights_fail: AtomicBool,
summary_fail: AtomicBool,
diagnostics: Mutex<DiagnosticsObservation>,
@@ -1014,6 +1079,7 @@ impl FixtureObservabilityStore {
providers: Vec::new(),
}),
projected_costs: Mutex::new(Vec::new()),
concurrency: Mutex::new(Vec::new()),
insights_fail: AtomicBool::new(false),
summary_fail: AtomicBool::new(false),
diagnostics: Mutex::new(DiagnosticsObservation::default()),
@@ -1144,6 +1210,7 @@ impl ObservabilityStore for FixtureObservabilityStore {
.lock()
.expect("projected costs")
.clone(),
concurrency: self.concurrency.lock().expect("concurrency").clone(),
})
}
@@ -1253,13 +1320,7 @@ impl SettingsStore for FixtureSettingsStore {
usage_retention_days: 31,
ops_event_retention_days: 30,
audit_retention_days: 30,
account_auto_freeze_enabled: true,
account_auto_freeze_threshold: 12,
account_auto_freeze_window_seconds: 600,
account_auto_freeze_duration_seconds: 7_200,
account_auto_freeze_probe_enabled: true,
account_auto_freeze_probe_model: None,
account_auto_freeze_adaptive_concurrency: true,
account_error_protection: gateway_core::account::AccountErrorProtection::default(),
account_warmup_enabled: false,
account_warmup_schedule_time: "08:00".to_owned(),
account_warmup_model: None,
@@ -213,6 +213,16 @@ fn sdk_settings_contract_matches_host_declarations() {
let mut types = Vec::new();
let mut replacement_fields = Vec::new();
for (path, names) in [
(
"../gateway-core/src/account/error_protection.rs",
&[
"AccountErrorProtection",
"AccountErrorRule",
"AccountErrorCondition",
"AccountErrorField",
"AccountErrorOperator",
][..],
),
(
"../gateway-core/src/settings/privacy.rs",
&[
@@ -345,13 +345,8 @@ async fn settings_should_reject_zero_refresh_margin_before_store_call() {
usage_retention_days: 31,
ops_event_retention_days: 30,
audit_retention_days: 30,
account_auto_freeze_enabled: true,
account_auto_freeze_threshold: 12,
account_auto_freeze_window_seconds: 600,
account_auto_freeze_duration_seconds: 7_200,
account_auto_freeze_probe_enabled: true,
account_auto_freeze_probe_model: None,
account_auto_freeze_adaptive_concurrency: true,
account_error_protection:
gateway_core::account::AccountErrorProtection::default(),
account_warmup_enabled: false,
account_warmup_schedule_time: "08:00".to_owned(),
account_warmup_model: None,
@@ -72,6 +72,7 @@ pub(super) fn account_view(
projection,
usage,
quota,
protection_rules,
} = item;
let status = projection.status.as_str().to_owned();
let cooldown = projection.cooldown;
@@ -86,7 +87,8 @@ pub(super) fn account_view(
if account.authentication_kind == "api_key" {
usage.window_kind = AccountUsageWindowKind::General;
}
let (quota, refresh_token_expires_at) = account_quota_view(quota, cooldown, now, time);
let (mut quota, refresh_token_expires_at) = account_quota_view(quota, cooldown, now, time);
quota.protection_rules = protection_rules;
AccountView {
capabilities: capabilities.into(),
id: account.id.clone(),
@@ -232,8 +234,8 @@ pub(super) fn account_quota_view(
}
});
let rate_limit_reason = cooldown.map(|value| {
if value.kind.is_capacity_freeze() {
"capacity_freeze".to_owned()
if value.kind.is_error_protection() {
"error_protection".to_owned()
} else {
"upstream_rate_limit".to_owned()
}
@@ -251,6 +253,7 @@ pub(super) fn account_quota_view(
rate_limit_recovery_display,
rate_limit_reason,
recovery_probe_required,
protection_rules: Vec::new(),
windows,
credits: quota.credits.map(|credits| AccountQuotaCreditsView {
has_credits: credits.has_credits,
@@ -394,6 +394,7 @@ pub struct AccountQuotaView {
pub rate_limit_recovery_display: Option<String>,
pub rate_limit_reason: Option<String>,
pub recovery_probe_required: bool,
pub protection_rules: Vec<gateway_core::account::AccountProtectionMatch>,
pub windows: Vec<AccountQuotaWindowView>,
pub credits: Option<AccountQuotaCreditsView>,
}
@@ -41,10 +41,6 @@ pub(crate) fn display_rate(value: f64) -> String {
}
}
fn basis_points_rate(value: u64) -> f64 {
value as f64 / 10_000.0
}
pub(crate) fn outcome_name(outcome: &domain::RequestOutcome) -> &str {
outcome.as_str()
}
@@ -930,6 +926,7 @@ pub(crate) fn dashboard_view(
.email
.clone()
.unwrap_or_else(|| credential.name.clone()),
notes: credential.notes,
plan_type: credential.plan_type.clone(),
plan_type_display: credential
.plan_type_display
@@ -1050,6 +1047,17 @@ pub(crate) fn usage_summary_view(
}
}
fn overview_load_view(load: &domain::UsageInsightsLoad) -> OverviewLoadView {
OverviewLoadView {
average_concurrency: load.average_concurrency,
peak_concurrency: load.peak_concurrency,
admission_decision_p95_ms: load.admission_decision_p95_ms.map(|value| value.as_f64()),
account_selection_wait_p95_ms: load
.account_selection_wait_p95_ms
.map(|value| value.as_f64()),
}
}
pub(crate) fn usage_insights_view(
insights: domain::UsageInsights,
time: crate::time::TimePresenter,
@@ -1092,31 +1100,10 @@ pub(crate) fn usage_insights_view(
.first_token_latency_percentiles
.p99_ms
.map(|value| value.as_f64()),
admission_decision_p50_ms: point
.admission_decision_percentiles
.p50_ms
.map(|value| value.as_f64()),
admission_decision_p95_ms: point
.admission_decision_percentiles
.p95_ms
.map(|value| value.as_f64()),
account_selection_wait_p50_ms: point
.account_selection_wait_percentiles
.p50_ms
.map(|value| value.as_f64()),
account_selection_wait_p95_ms: point
.account_selection_wait_percentiles
.p95_ms
.map(|value| value.as_f64()),
output_throughput_p10: point.output_throughput_p10,
output_throughput_p50: point.output_throughput_p50,
output_throughput_p90: point.output_throughput_p90,
capacity_utilization: point
.capacity_utilization_avg_basis_points
.map(basis_points_rate),
capacity_utilization_p95: point
.capacity_utilization_p95_basis_points
.map(basis_points_rate),
load: overview_load_view(&point.load),
})
.collect();
let cost_points = insights
@@ -1185,42 +1172,10 @@ pub(crate) fn usage_insights_view(
.first_token_latency_percentiles
.p99_ms
.map(|value| value.as_f64()),
admission_decision_p50_ms: insights
.performance
.admission_decision_percentiles
.p50_ms
.map(|value| value.as_f64()),
admission_decision_p95_ms: insights
.performance
.admission_decision_percentiles
.p95_ms
.map(|value| value.as_f64()),
account_selection_wait_p50_ms: insights
.performance
.account_selection_wait_percentiles
.p50_ms
.map(|value| value.as_f64()),
account_selection_wait_p95_ms: insights
.performance
.account_selection_wait_percentiles
.p95_ms
.map(|value| value.as_f64()),
output_throughput_p10: insights.performance.output_throughput_p10,
output_throughput_p50: insights.performance.output_throughput_p50,
output_throughput_p90: insights.performance.output_throughput_p90,
capacity_utilization: insights
.performance
.capacity_utilization_avg_basis_points
.map(basis_points_rate),
capacity_utilization_p95: insights
.performance
.capacity_utilization_p95_basis_points
.map(basis_points_rate),
latency_coverage: insights.performance.latency_coverage,
first_token_coverage: insights.performance.first_token_coverage,
admission_decision_coverage: insights.performance.admission_decision_coverage,
account_selection_wait_coverage: insights.performance.account_selection_wait_coverage,
capacity_coverage: insights.performance.capacity_coverage,
load: overview_load_view(&insights.performance.load),
points: performance_points,
},
cost: OverviewCostView {
@@ -406,6 +406,7 @@ pub struct DashboardAccountUsageView {
pub provider: String,
pub authentication_kind: String,
pub email: String,
pub notes: Option<String>,
pub plan_type: Option<String>,
pub plan_type_display: String,
pub tokens: String,
@@ -648,6 +649,16 @@ pub struct OverviewHealthView {
pub points: Vec<OverviewHealthPointView>,
}
/// 并发与调度等待的负载指标
#[derive(Debug, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct OverviewLoadView {
pub average_concurrency: f64,
pub peak_concurrency: u64,
pub admission_decision_p95_ms: Option<f64>,
pub account_selection_wait_p95_ms: Option<f64>,
}
/// 洞察性能趋势点
#[derive(Debug, Serialize)]
#[serde(rename_all = "camelCase")]
@@ -660,15 +671,10 @@ pub struct OverviewPerformancePointView {
pub first_token_p50_ms: Option<f64>,
pub first_token_p95_ms: Option<f64>,
pub first_token_p99_ms: Option<f64>,
pub admission_decision_p50_ms: Option<f64>,
pub admission_decision_p95_ms: Option<f64>,
pub account_selection_wait_p50_ms: Option<f64>,
pub account_selection_wait_p95_ms: Option<f64>,
pub output_throughput_p10: Option<u64>,
pub output_throughput_p50: Option<u64>,
pub output_throughput_p90: Option<u64>,
pub capacity_utilization: Option<f64>,
pub capacity_utilization_p95: Option<f64>,
pub load: OverviewLoadView,
}
/// 洞察性能摘要
@@ -681,20 +687,10 @@ pub struct OverviewPerformanceView {
pub first_token_p50_ms: Option<f64>,
pub first_token_p95_ms: Option<f64>,
pub first_token_p99_ms: Option<f64>,
pub admission_decision_p50_ms: Option<f64>,
pub admission_decision_p95_ms: Option<f64>,
pub account_selection_wait_p50_ms: Option<f64>,
pub account_selection_wait_p95_ms: Option<f64>,
pub output_throughput_p10: Option<u64>,
pub output_throughput_p50: Option<u64>,
pub output_throughput_p90: Option<u64>,
pub capacity_utilization: Option<f64>,
pub capacity_utilization_p95: Option<f64>,
pub latency_coverage: f64,
pub first_token_coverage: f64,
pub admission_decision_coverage: f64,
pub account_selection_wait_coverage: f64,
pub capacity_coverage: f64,
pub load: OverviewLoadView,
pub points: Vec<OverviewPerformancePointView>,
}
@@ -56,13 +56,7 @@ pub struct RuntimeSettingsFields {
pub usage_retention_days: u64,
pub ops_event_retention_days: u64,
pub audit_retention_days: u64,
pub account_auto_freeze_enabled: bool,
pub account_auto_freeze_threshold: u64,
pub account_auto_freeze_window_seconds: u64,
pub account_auto_freeze_duration_seconds: u64,
pub account_auto_freeze_probe_enabled: bool,
pub account_auto_freeze_probe_model: Option<String>,
pub account_auto_freeze_adaptive_concurrency: bool,
pub account_error_protection: gateway_core::account::AccountErrorProtection,
pub account_warmup_enabled: bool,
pub account_warmup_schedule_time: String,
pub account_warmup_model: Option<String>,
@@ -593,11 +587,6 @@ fn map_wire_error(error: WireValidationError) -> AdminError {
"settingsOpsRetentionOverflow" => "opsEventRetentionDays 不合法".to_owned(),
"settingsAuditRetentionOverflow" => "auditRetentionDays 不合法".to_owned(),
"requestLocation" => "请求位置不合法,请检查国家代码、地区和城市".to_owned(),
"settingsFreezeThresholdOverflow" => "accountAutoFreezeThreshold 不合法".to_owned(),
"accountAutoFreezeThreshold" => "账号自动冻结阈值应为 2~1000 的整数".to_owned(),
"accountAutoFreezeWindowSeconds" => "账号自动冻结统计窗口应为 60~3600 秒".to_owned(),
"accountAutoFreezeDurationSeconds" => "账号自动冻结时长应为 300~604800 秒".to_owned(),
"accountAutoFreezeProbeModel" => "探测模型格式不合法".to_owned(),
"providerRequestProfiles" => "Provider 请求画像格式不合法或字段冲突".to_owned(),
"minCodexDesktopVersion" => "Codex Desktop 最低版本格式不合法".to_owned(),
"minCodexCliVersion" => "Codex CLI 最低版本格式不合法".to_owned(),
@@ -776,14 +765,7 @@ impl RuntimeSettingsFields {
.map_err(|_| WireValidationError::new("settingsOpsRetentionOverflow"))?,
audit_retention_days: u32::try_from(self.audit_retention_days)
.map_err(|_| WireValidationError::new("settingsAuditRetentionOverflow"))?,
account_auto_freeze_enabled: self.account_auto_freeze_enabled,
account_auto_freeze_threshold: u32::try_from(self.account_auto_freeze_threshold)
.map_err(|_| WireValidationError::new("settingsFreezeThresholdOverflow"))?,
account_auto_freeze_window_seconds: self.account_auto_freeze_window_seconds,
account_auto_freeze_duration_seconds: self.account_auto_freeze_duration_seconds,
account_auto_freeze_probe_enabled: self.account_auto_freeze_probe_enabled,
account_auto_freeze_probe_model: self.account_auto_freeze_probe_model,
account_auto_freeze_adaptive_concurrency: self.account_auto_freeze_adaptive_concurrency,
account_error_protection: self.account_error_protection,
account_warmup_enabled: self.account_warmup_enabled,
account_warmup_schedule_time: self.account_warmup_schedule_time,
account_warmup_model: self.account_warmup_model,
@@ -819,14 +801,7 @@ impl From<gateway_admin::model::settings::RuntimeSettingsValues> for RuntimeSett
usage_retention_days: u64::from(settings.usage_retention_days),
ops_event_retention_days: u64::from(settings.ops_event_retention_days),
audit_retention_days: u64::from(settings.audit_retention_days),
account_auto_freeze_enabled: settings.account_auto_freeze_enabled,
account_auto_freeze_threshold: u64::from(settings.account_auto_freeze_threshold),
account_auto_freeze_window_seconds: settings.account_auto_freeze_window_seconds,
account_auto_freeze_duration_seconds: settings.account_auto_freeze_duration_seconds,
account_auto_freeze_probe_enabled: settings.account_auto_freeze_probe_enabled,
account_auto_freeze_probe_model: settings.account_auto_freeze_probe_model,
account_auto_freeze_adaptive_concurrency: settings
.account_auto_freeze_adaptive_concurrency,
account_error_protection: settings.account_error_protection,
account_warmup_enabled: settings.account_warmup_enabled,
account_warmup_schedule_time: settings.account_warmup_schedule_time,
account_warmup_model: settings.account_warmup_model,
@@ -839,13 +814,7 @@ impl From<gateway_admin::model::settings::RuntimeSettingsValues> for RuntimeSett
fn settings_validation_error(field: &'static str) -> WireValidationError {
WireValidationError::new(match field {
"account_auto_freeze_adaptive_concurrency" => "accountAutoFreezeAdaptiveConcurrency",
"account_auto_freeze_duration_seconds" => "accountAutoFreezeDurationSeconds",
"account_auto_freeze_enabled" => "accountAutoFreezeEnabled",
"account_auto_freeze_probe_enabled" => "accountAutoFreezeProbeEnabled",
"account_auto_freeze_probe_model" => "accountAutoFreezeProbeModel",
"account_auto_freeze_threshold" => "accountAutoFreezeThreshold",
"account_auto_freeze_window_seconds" => "accountAutoFreezeWindowSeconds",
"account_error_protection" => "accountErrorProtection",
"account_warmup_enabled" => "accountWarmupEnabled",
"account_warmup_model" => "accountWarmupModel",
"reset_credit_lead_minutes" => "resetCreditLeadMinutes",
+24 -16
View File
@@ -153,6 +153,19 @@ pub(crate) fn client_access_error_response(error: ClientAccessError) -> Response
#[must_use]
pub fn identify_codex_client(headers: &HeaderMap) -> Option<IdentifiedCodexClient> {
const MAXIMUM_HEADER_LENGTH: usize = 4096;
// 官方 CLI、IDE、SDK 与 App Server 的 UA 产品版本均来自所运行的 Codex Core
const CORE_CLIENT_PRODUCTS: &[&str] = &[
"codex_cli_rs",
"codex-cli",
"codex-tui",
"codex_exec",
"codex_vscode",
"codex_sdk_ts",
"codex_python_sdk",
"codex-app-server",
"codex-app-server-sdk",
"codex_mcp_server",
];
let originator = bounded_ascii_header(headers, "originator", MAXIMUM_HEADER_LENGTH);
let user_agent = bounded_ascii_header(headers, "user-agent", MAXIMUM_HEADER_LENGTH);
@@ -186,22 +199,17 @@ pub fn identify_codex_client(headers: &HeaderMap) -> Option<IdentifiedCodexClien
for product in user_agent.split(|character: char| {
character.is_ascii_whitespace() || matches!(character, '(' | ')' | ';' | ',' | '[' | ']')
}) {
for prefix in ["codex_cli_rs", "codex-cli"] {
if product.eq_ignore_ascii_case(prefix) {
return Some(IdentifiedCodexClient {
kind: CodexClientKind::Cli,
version: None,
});
}
let Some((name, version)) = product.split_once('/') else {
continue;
};
if name.eq_ignore_ascii_case(prefix) {
return Some(IdentifiedCodexClient {
kind: CodexClientKind::Cli,
version: CodexClientVersion::parse(version).ok(),
});
}
let (name, version) = product
.split_once('/')
.map_or((product, None), |(name, version)| (name, Some(version)));
if CORE_CLIENT_PRODUCTS
.iter()
.any(|known| name.eq_ignore_ascii_case(known))
{
return Some(IdentifiedCodexClient {
kind: CodexClientKind::Cli,
version: version.and_then(|value| CodexClientVersion::parse(value).ok()),
});
}
}
None
@@ -0,0 +1,319 @@
//! Chat 出口持有协议投影缓冲 提交与资源回收始终委托原受管正文
use std::collections::VecDeque;
use axum::body::to_bytes;
use bytes::Bytes;
use futures::future::BoxFuture;
use gateway_core::{
engine::middleware::{
MiddlewareBody, MiddlewareError, MiddlewareFrame, MiddlewareFraming, MiddlewareHeader,
MiddlewareResponse,
},
error::{GatewayError, GatewayErrorKind},
};
use gateway_protocol::openai::sse::{DONE_SSE_FRAME, SseEvent, SseEventDecoder, sse_frame_is_done};
use serde_json::Value;
use crate::openai::{middleware::error_response, responses::ProtocolErrorBody};
use super::response::{ChatIdentity, ChatStreamEncoder, project_response};
pub(super) fn project_middleware_response(
response: MiddlewareResponse,
identity: ChatIdentity,
streaming: bool,
include_usage: bool,
) -> MiddlewareResponse {
if !(200..300).contains(&response.status_code()) {
return response;
}
let (protocol, status, mut headers, body, envelope) = response.into_parts();
// 投影改变正文长度和表示形式 不继承 Responses 的实体长度或编码
headers.retain(|header| {
!["content-length", "content-type", "content-encoding"]
.iter()
.any(|name| header.name().eq_ignore_ascii_case(name))
});
headers.push(MiddlewareHeader::new(
"content-type",
Bytes::from_static(if streaming {
b"text/event-stream"
} else {
b"application/json"
}),
));
let projection = if streaming {
Projection::Stream(Box::new(StreamProjection {
encoder: ChatStreamEncoder::new(identity, include_usage),
decoder: SseEventDecoder::default(),
done_received: false,
}))
} else {
Projection::Json(identity)
};
let mut response = MiddlewareResponse::new(
protocol,
status,
headers,
Box::new(ChatBody {
inner: Some(body),
projection,
pending: VecDeque::new(),
committed: false,
finished: false,
}),
);
if let Some(envelope) = envelope {
response = response.with_envelope(envelope);
}
response
}
pub(super) fn chat_error_frame(data: &Value) -> Bytes {
let error = data
.pointer("/response/error")
.or_else(|| data.get("error"));
Bytes::from(format!("data: {}\n\n", serde_json::json!({"error": error})))
}
enum Projection {
Json(ChatIdentity),
Stream(Box<StreamProjection>),
}
impl Projection {
fn terminal_seen(&self) -> bool {
matches!(self, Self::Stream(stream) if stream.terminal_seen())
}
}
struct StreamProjection {
encoder: ChatStreamEncoder,
decoder: SseEventDecoder,
done_received: bool,
}
impl StreamProjection {
fn terminal_seen(&self) -> bool {
self.encoder.is_completed() || self.encoder.has_wire_failure()
}
fn project(&mut self, events: Vec<SseEvent>) -> Result<Vec<Bytes>, MiddlewareError> {
let mut output = Vec::new();
for event in events {
if self.done_received {
return Err(invalid_response());
}
let data = serde_json::from_str(&event.data).map_err(|_| invalid_response())?;
// codec 区分辅助事件与终态后的业务输出 交付层不再维护一份事件分类
output.extend(
self.encoder
.push_event(event.event.as_deref(), &data)
.map_err(projection_error)?,
);
}
Ok(output)
}
}
struct ChatBody {
inner: Option<Box<dyn MiddlewareBody>>,
projection: Projection,
pending: VecDeque<MiddlewareFrame>,
committed: bool,
finished: bool,
}
impl ChatBody {
async fn read_one(&mut self) -> Result<(), MiddlewareError> {
let inner = self.inner.as_mut().ok_or(MiddlewareError::InvalidState)?;
let Some(frame) = inner.next_frame().await? else {
if let Projection::Stream(stream) = &mut self.projection {
let events = stream.decoder.finish().map_err(|_| invalid_response())?;
let frames = stream.project(events)?;
if !frames.is_empty() || !stream.done_received || !inner.is_finalized() {
return Err(invalid_response());
}
self.pending.push_back(done_frame());
}
self.finished = true;
return Ok(());
};
let transformed = frame.transformed();
let (bytes, framing, terminal, envelope) = frame.into_parts();
let output = match &mut self.projection {
Projection::Json(identity) => {
if framing != MiddlewareFraming::JsonDocument || !terminal {
return Err(invalid_response());
}
let value = serde_json::from_slice(&bytes).map_err(|_| invalid_response())?;
let value = project_response(&value, identity).map_err(projection_error)?;
vec![MiddlewareFrame::new(
Bytes::from(serde_json::to_vec(&value).map_err(|_| invalid_response())?),
MiddlewareFraming::JsonDocument,
true,
)]
}
Projection::Stream(stream) => {
if framing != MiddlewareFraming::SseEvent {
return Err(invalid_response());
}
let events = stream
.decoder
.push(&bytes)
.map_err(|_| invalid_response())?;
let output = stream.project(events)?;
// 共用 SSE 解码器只返回业务事件 DONE 由受管正文的控制帧单独确认
if std::str::from_utf8(&bytes).is_ok_and(sse_frame_is_done) {
if stream.done_received || !stream.terminal_seen() {
return Err(invalid_response());
}
stream.done_received = true;
}
if terminal && !stream.done_received {
return Err(invalid_response());
}
output
.into_iter()
.map(|bytes| MiddlewareFrame::new(bytes, MiddlewareFraming::SseEvent, false))
.collect()
}
};
let mut output: VecDeque<_> = output
.into_iter()
.map(|frame| frame.with_transformed(transformed))
.collect();
if let Some(envelope) = envelope
&& let Some(last) = output.pop_back()
{
output.push_back(last.with_envelope(envelope));
}
self.pending.extend(output);
Ok(())
}
async fn verify_terminal(&mut self) -> Result<(), MiddlewareError> {
while !self.finished {
self.read_one().await?;
}
Ok(())
}
async fn fail_stream(&mut self, error: MiddlewareError) {
self.pending.clear();
if let Some(inner) = self.inner.take() {
inner.close().await;
}
let response = error_response(error);
let bytes = to_bytes(response.into_body(), usize::MAX)
.await
.unwrap_or_default();
let value = serde_json::from_slice(&bytes).unwrap_or_else(|_| {
serde_json::json!({"error": {
"type": "server_error", "code": "invalid_upstream_response",
"message": "The gateway could not deliver the Chat response."
}})
});
self.pending.push_back(MiddlewareFrame::new(
chat_error_frame(&value),
MiddlewareFraming::SseEvent,
false,
));
self.pending.push_back(done_frame());
self.finished = true;
}
}
impl MiddlewareBody for ChatBody {
fn next_frame(&mut self) -> BoxFuture<'_, Result<Option<MiddlewareFrame>, MiddlewareError>> {
Box::pin(async move {
while self.pending.is_empty() && !self.finished {
let result = match self.read_one().await {
Ok(()) if self.committed && self.projection.terminal_seen() => {
self.verify_terminal().await
}
result => result,
};
if let Err(error) = result {
if !self.committed {
return Err(error);
}
self.fail_stream(error).await;
}
}
Ok(self.pending.pop_front())
})
}
fn commit_downstream(
&mut self,
client_status_code: Option<u16>,
) -> BoxFuture<'_, Result<(), MiddlewareError>> {
Box::pin(async move {
self.inner
.as_mut()
.ok_or(MiddlewareError::InvalidState)?
.commit_downstream(client_status_code)
.await?;
self.committed = true;
// HTTP 尚未发送首帧 即使首批已经包含 finish 或 usage 也先确认原会话完成结算
if self.projection.terminal_seen()
&& let Err(error) = self.verify_terminal().await
{
// 原会话已提交 不能把已记录的 200 改成另一 HTTP 状态
// 首个 Chat 帧为 role 或 error 成功终态仍在 pending 内可安全撤回
self.fail_stream(error).await;
}
Ok(())
})
}
fn record_client_status(&mut self, status: u16) -> BoxFuture<'_, Result<(), MiddlewareError>> {
match self.inner.as_mut() {
Some(inner) => inner.record_client_status(status),
None => Box::pin(async { Ok(()) }),
}
}
fn is_finalized(&self) -> bool {
self.inner.as_ref().is_none_or(|inner| inner.is_finalized())
}
fn close(mut self: Box<Self>) -> BoxFuture<'static, ()> {
let inner = self.inner.take();
Box::pin(async move {
if let Some(inner) = inner {
inner.close().await;
}
})
}
}
fn projection_error(error: ProtocolErrorBody) -> MiddlewareError {
MiddlewareError::Gateway(
GatewayError::new(
GatewayErrorKind::UpstreamUnavailable,
"The gateway could not project the upstream response to Chat Completions.",
)
.with_client_code(error.error.code),
)
}
fn invalid_response() -> MiddlewareError {
MiddlewareError::Gateway(
GatewayError::new(
GatewayErrorKind::UpstreamUnavailable,
"The gateway could not deliver the Chat response.",
)
.with_client_code("invalid_upstream_response"),
)
}
fn done_frame() -> MiddlewareFrame {
MiddlewareFrame::new(
Bytes::from_static(DONE_SSE_FRAME.as_bytes()),
MiddlewareFraming::SseEvent,
true,
)
}
@@ -0,0 +1,100 @@
//! Chat HTTP 入口只规范化协议 共用 Responses 的执行与交付链路
use std::{net::SocketAddr, time::SystemTime};
use axum::{
body::Bytes,
extract::{Extension, State, connect_info::ConnectInfo},
http::{HeaderMap, StatusCode, header::CONTENT_ENCODING},
response::Response,
};
use crate::ApiState;
use crate::openai::{
auth::{authenticate_client, client_access_error_response},
error::{gateway_error_response, protocol_error_response},
responses::{
DecodedResponsesRequest, HttpResponseAdapter, RequestDecodeError, ResponsesHttpRequest,
decode_request_with_headers, decompress_request_body, execute_prepared_response,
},
router::CHAT_COMPLETIONS_PATH,
};
use super::{
delivery::{chat_error_frame, project_middleware_response},
request::convert_request,
response::ChatIdentity,
};
pub(in crate::openai) async fn chat_completions(
State(state): State<ApiState>,
connect_info: Option<Extension<ConnectInfo<SocketAddr>>>,
ingress_id: Option<Extension<tower_http::request_id::RequestId>>,
headers: HeaderMap,
body: Bytes,
) -> Response {
let service = state.openai().clone();
let client = match authenticate_client(&service, &headers).await {
Ok(client) => client,
Err(error) => return client_access_error_response(error),
};
let (decoded, middleware_body, include_usage) = match decode_request(
&body,
&headers,
client.snapshot().responses_max_decompressed_body_bytes(),
) {
Ok(request) => request,
Err(error) => {
return protocol_error_response(StatusCode::BAD_REQUEST, error.protocol_body());
}
};
let prepared = match service.execution().prepare_execution(client).await {
Ok(prepared) => prepared,
Err(error) => return gateway_error_response(&error),
};
let identity = ChatIdentity {
id: format!("chatcmpl-{}", prepared.request_id().as_str()),
created: SystemTime::now()
.duration_since(SystemTime::UNIX_EPOCH)
.unwrap_or_default()
.as_secs(),
model: decoded.metadata().requested_model().to_owned(),
};
let streaming = decoded.metadata().stream();
let adapter = HttpResponseAdapter {
endpoint: CHAT_COMPLETIONS_PATH,
wrap: Box::new(move |response| {
project_middleware_response(response, identity, streaming, include_usage)
}),
stream_error: chat_error_frame,
};
execute_prepared_response(
service,
prepared,
ResponsesHttpRequest {
peer_address: connect_info.map(|Extension(ConnectInfo(address))| address),
ingress_id,
headers,
decoded,
middleware_body,
},
None,
Some(adapter),
)
.await
}
fn decode_request(
body: &[u8],
headers: &HeaderMap,
maximum_body_bytes: usize,
) -> Result<(DecodedResponsesRequest, Bytes, bool), RequestDecodeError> {
let body = decompress_request_body(body, headers, maximum_body_bytes)?;
let converted = convert_request(&body)?;
let body =
serde_json::to_vec(&converted.body).map_err(|_| RequestDecodeError::MalformedJson)?;
let mut decoded_headers = headers.clone();
decoded_headers.remove(CONTENT_ENCODING);
let decoded = decode_request_with_headers(&body, &decoded_headers, maximum_body_bytes)?;
Ok((decoded, Bytes::from(body), converted.include_usage))
}
@@ -0,0 +1,8 @@
//! Chat Completions 与 Responses 之间的 API 协议适配
mod delivery;
mod http;
mod request;
mod response;
pub(super) use http::chat_completions;
@@ -0,0 +1,550 @@
//! 将 Chat 请求转换为 Responses 正文,协议选项留在 API,业务 JSON 与上游兼容规则保持原有归属
use serde_json::{Map, Value, json};
use crate::openai::responses::RequestDecodeError;
pub(in crate::openai) struct ConvertedChatRequest {
pub(in crate::openai) body: Map<String, Value>,
pub(in crate::openai) include_usage: bool,
}
pub(in crate::openai) fn convert_request(
body: &[u8],
) -> Result<ConvertedChatRequest, RequestDecodeError> {
let value = serde_json::from_slice(body).map_err(|_| RequestDecodeError::MalformedJson)?;
let Value::Object(mut chat) = value else {
return Err(RequestDecodeError::ExpectedObject);
};
validate_options(&chat)?;
let model = required_string(&mut chat, "model", "model", false)?;
let stream = optional_bool(&mut chat, "stream", "stream")?.unwrap_or(false);
let include_usage = stream_options(optional(&mut chat, "stream_options"), stream)?;
let input = messages(required(&mut chat, "messages", "messages")?)?;
let mut body = Map::from_iter([
("model".into(), model.into()),
("input".into(), input.into()),
("stream".into(), stream.into()),
("store".into(), false.into()),
]);
if let Some(value) = optional(&mut chat, "tools") {
body.insert("tools".into(), function_tools(value)?.into());
}
if let Some(value) = optional(&mut chat, "tool_choice") {
let choice = tool_choice(value, body.get("tools"))?;
body.insert("tool_choice".into(), choice);
}
if let Some(value) = optional_bool(&mut chat, "parallel_tool_calls", "parallel_tool_calls")? {
body.insert("parallel_tool_calls".into(), value.into());
}
let mut text = Map::new();
if let Some(value) = optional(&mut chat, "response_format") {
text.insert("format".into(), response_format(value)?);
}
if let Some(value) = optional(&mut chat, "verbosity") {
string(&value, "verbosity", false)?;
text.insert("verbosity".into(), value);
}
if !text.is_empty() {
body.insert("text".into(), text.into());
}
if let Some(value) = optional(&mut chat, "reasoning_effort") {
string(&value, "reasoning_effort", false)?;
body.insert("reasoning".into(), json!({"effort": value}));
}
for field in ["max_tokens", "max_completion_tokens"] {
if let Some(value) = optional(&mut chat, field) {
if !value.as_u64().is_some_and(|tokens| tokens > 0) {
return Err(invalid(field));
}
// 新字段优先,具体上游是否接受长度控制仍由现有 Provider 决定
body.insert("max_output_tokens".into(), value);
}
}
for (field, maximum) in [("temperature", 2.0), ("top_p", 1.0)] {
if let Some(value) = optional(&mut chat, field) {
if !value
.as_f64()
.is_some_and(|number| (0.0..=maximum).contains(&number))
{
return Err(invalid(field));
}
body.insert(field.into(), value);
}
}
for field in [
"user",
"safety_identifier",
"service_tier",
"prompt_cache_key",
"prompt_cache_retention",
] {
if let Some(value) = optional(&mut chat, field) {
string(&value, field, true)?;
body.insert(field.into(), value);
}
}
for field in ["metadata", "prompt_cache_options", "moderation"] {
if let Some(value) = optional(&mut chat, field) {
let object = object(value, field)?;
body.insert(field.into(), object.into());
}
}
// 未识别的客户端顶层扩展不等于 Responses 参数,不进入上游正文
Ok(ConvertedChatRequest {
body,
include_usage,
})
}
fn validate_options(chat: &Map<String, Value>) -> Result<(), RequestDecodeError> {
for (field, value) in chat.iter().filter(|(_, value)| !value.is_null()) {
let supported = match field.as_str() {
"n" => value.as_u64() == Some(1),
"store" | "logprobs" => value == false,
"frequency_penalty" | "presence_penalty" => value.as_f64() == Some(0.0),
"top_logprobs" => value.as_u64() == Some(0),
"logit_bias" => value.as_object().is_some_and(Map::is_empty),
"functions" | "stop" => value.as_array().is_some_and(Vec::is_empty),
"modalities" => value == &json!(["text"]),
"audio"
| "prediction"
| "seed"
| "function_call"
| "web_search_options"
| "reasoning_content"
| "input"
| "previous_response_id"
| "conversation"
| "instructions"
| "reasoning"
| "text"
| "include"
| "background"
| "max_output_tokens" => false,
_ => true,
};
if !supported {
return Err(unsupported(field));
}
}
Ok(())
}
fn stream_options(value: Option<Value>, stream: bool) -> Result<bool, RequestDecodeError> {
let Some(value) = value else {
return Ok(false);
};
if !stream {
return Err(invalid("stream_options"));
}
let mut options = object(value, "stream_options")?;
let include_usage = optional_bool(
&mut options,
"include_usage",
"stream_options.include_usage",
)?
.unwrap_or(false);
if optional_bool(
&mut options,
"include_obfuscation",
"stream_options.include_obfuscation",
)? == Some(true)
{
// Chat 重编码会改变帧长度,不能声称继承上游流的长度混淆保证
return Err(unsupported("stream_options.include_obfuscation"));
}
Ok(include_usage)
}
fn messages(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
let messages = array(value, "messages")?;
if messages.is_empty() {
return Err(RequestDecodeError::EmptyField {
field: "messages".into(),
});
}
let mut input = Vec::with_capacity(messages.len());
for (index, value) in messages.into_iter().enumerate() {
let path = format!("messages[{index}]");
let mut message = object(value, &path)?;
let role = required_string(&mut message, "role", &format!("{path}.role"), false)?;
if !matches!(
role.as_str(),
"system" | "developer" | "user" | "assistant" | "tool"
) {
return Err(unsupported(&format!("{path}.role")));
}
if let Some(name) = optional(&mut message, "name")
&& !string(&name, &format!("{path}.name"), true)?.is_empty()
{
return Err(unsupported(&format!("{path}.name")));
}
for field in ["function_call", "reasoning_content", "reasoning", "audio"] {
if optional(&mut message, field).is_some() {
return Err(unsupported(&format!("{path}.{field}")));
}
}
for (field, allowed_role) in [
("tool_calls", "assistant"),
("refusal", "assistant"),
("tool_call_id", "tool"),
] {
if role != allowed_role && message.get(field).is_some_and(|value| !value.is_null()) {
return Err(invalid(&format!("{path}.{field}")));
}
}
if role == "tool" {
let call_id = required_string(
&mut message,
"tool_call_id",
&format!("{path}.tool_call_id"),
false,
)?;
let content_path = format!("{path}.content");
let content = required(&mut message, "content", &content_path)?;
let output = if content.is_string() {
content
} else {
content_parts(content, &role, &content_path, false)?.into()
};
input.push(
json!({"type": "function_call_output", "call_id": call_id, "output": output}),
);
continue;
}
let calls = optional(&mut message, "tool_calls")
.map(|value| tool_calls(value, &format!("{path}.tool_calls")))
.transpose()?
.unwrap_or_default();
let refusal = optional(&mut message, "refusal");
if let Some(value) = &refusal {
string(value, &format!("{path}.refusal"), true)?;
}
let content = optional(&mut message, "content");
if content.is_none() && refusal.is_none() && calls.is_empty() {
return Err(invalid(&format!("{path}.content")));
}
let is_assistant = role == "assistant";
let mut parts = content
.map(|value| content_parts(value, &role, &format!("{path}.content"), is_assistant))
.transpose()?;
if let Some(refusal) = refusal {
parts
.get_or_insert_with(Vec::new)
.push(json!({"type": "refusal", "refusal": refusal}));
}
if let Some(parts) = parts {
let mut item = json!({"type": "message", "role": role, "content": parts});
if is_assistant {
// assistant 历史采用输出消息合同,普通文本和 refusal 都不能编码成 input_text
item["id"] = format!("msg_chat_{index}").into();
item["status"] = "completed".into();
}
input.push(item);
}
input.extend(calls);
}
Ok(input)
}
fn content_parts(
value: Value,
role: &str,
path: &str,
is_assistant: bool,
) -> Result<Vec<Value>, RequestDecodeError> {
let parts = match value {
Value::String(text) => return Ok(vec![text_part(text, is_assistant)]),
Value::Array(parts) => parts,
_ => return Err(invalid_type(path, "a string or an array")),
};
let mut result = Vec::with_capacity(parts.len());
for (index, value) in parts.into_iter().enumerate() {
let path = format!("{path}[{index}]");
let mut part = object(value, &path)?;
let kind = required_string(&mut part, "type", &format!("{path}.type"), false)?;
let mut converted = match kind.as_str() {
"text" => text_part(
required_string(&mut part, "text", &format!("{path}.text"), true)?,
is_assistant,
),
"image_url" if role == "user" => image_part(
required(&mut part, "image_url", &format!("{path}.image_url"))?,
&format!("{path}.image_url"),
)?,
"refusal" if role == "assistant" => json!({
"type": "refusal",
"refusal": required_string(&mut part, "refusal", &format!("{path}.refusal"), true)?,
}),
_ => return Err(unsupported(&format!("{path}.type"))),
};
if let Some(value) = optional(&mut part, "prompt_cache_breakpoint") {
if is_assistant {
return Err(unsupported(&format!("{path}.prompt_cache_breakpoint")));
}
converted["prompt_cache_breakpoint"] =
object(value, &format!("{path}.prompt_cache_breakpoint"))?.into();
}
result.push(converted);
}
Ok(result)
}
fn text_part(text: String, is_assistant: bool) -> Value {
if is_assistant {
json!({"type": "output_text", "text": text, "annotations": []})
} else {
json!({"type": "input_text", "text": text})
}
}
fn image_part(value: Value, path: &str) -> Result<Value, RequestDecodeError> {
let mut image = object(value, path)?;
let url = required_string(&mut image, "url", &format!("{path}.url"), false)?;
let valid = url::Url::parse(&url).is_ok_and(|parsed| {
matches!(parsed.scheme(), "http" | "https") && parsed.host_str().is_some()
|| parsed.scheme() == "data"
&& url.starts_with("data:image/")
&& url
.split_once(";base64,")
.is_some_and(|(_, data)| !data.trim().is_empty())
});
if !valid {
return Err(invalid(&format!("{path}.url")));
}
let mut converted = json!({"type": "input_image", "image_url": url});
if let Some(detail) = optional(&mut image, "detail") {
string(&detail, &format!("{path}.detail"), false)?;
converted["detail"] = detail;
}
Ok(converted)
}
fn tool_calls(value: Value, path: &str) -> Result<Vec<Value>, RequestDecodeError> {
let calls = array(value, path)?;
let mut result = Vec::with_capacity(calls.len());
for (index, value) in calls.into_iter().enumerate() {
let path = format!("{path}[{index}]");
let mut call = object(value, &path)?;
let id = required_string(&mut call, "id", &format!("{path}.id"), false)?;
if required_string(&mut call, "type", &format!("{path}.type"), false)? != "function" {
return Err(unsupported(&format!("{path}.type")));
}
let function_path = format!("{path}.function");
let mut function = object(
required(&mut call, "function", &function_path)?,
&function_path,
)?;
let name = required_string(
&mut function,
"name",
&format!("{function_path}.name"),
false,
)?;
let arguments = required_string(
&mut function,
"arguments",
&format!("{function_path}.arguments"),
true,
)?;
result.push(
json!({"type": "function_call", "call_id": id, "name": name, "arguments": arguments}),
);
}
Ok(result)
}
fn function_tools(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
array(value, "tools")?
.into_iter()
.enumerate()
.map(|(index, value)| {
let path = format!("tools[{index}]");
let mut tool = object(value, &path)?;
if required_string(&mut tool, "type", &format!("{path}.type"), false)? != "function" {
return Err(unsupported(&format!("{path}.type")));
}
let function_path = format!("{path}.function");
let mut function = object(
required(&mut tool, "function", &function_path)?,
&function_path,
)?;
let name = required_string(
&mut function,
"name",
&format!("{function_path}.name"),
false,
)?;
// Chat 的缺省是宽松函数 schema,不能让 Responses 的缺省收紧原始参数合同
let strict =
optional_bool(&mut function, "strict", &format!("{function_path}.strict"))?
.unwrap_or(false);
let mut converted = json!({"type": "function", "name": name, "strict": strict});
if let Some(description) = optional(&mut function, "description") {
string(&description, &format!("{function_path}.description"), true)?;
converted["description"] = description;
}
if let Some(parameters) = optional(&mut function, "parameters") {
converted["parameters"] = parameters;
}
Ok(converted)
})
.collect()
}
fn tool_choice(value: Value, tools: Option<&Value>) -> Result<Value, RequestDecodeError> {
let tools = tools.and_then(Value::as_array);
if let Value::String(choice) = &value {
return match choice.as_str() {
"auto" | "none" => Ok(value),
"required" if tools.is_some_and(|tools| !tools.is_empty()) => Ok(value),
"required" => Err(invalid("tool_choice")),
_ => Err(unsupported("tool_choice")),
};
}
let mut choice = object(value, "tool_choice")?;
if required_string(&mut choice, "type", "tool_choice.type", false)? != "function" {
return Err(unsupported("tool_choice.type"));
}
let mut function = object(
required(&mut choice, "function", "tool_choice.function")?,
"tool_choice.function",
)?;
let name = required_string(&mut function, "name", "tool_choice.function.name", false)?;
if !tools.is_some_and(|tools| {
tools
.iter()
.any(|tool| tool.get("name").and_then(Value::as_str) == Some(name.as_str()))
}) {
return Err(invalid("tool_choice.function.name"));
}
Ok(json!({"type": "function", "name": name}))
}
fn response_format(value: Value) -> Result<Value, RequestDecodeError> {
let mut format = object(value, "response_format")?;
let kind = required_string(&mut format, "type", "response_format.type", false)?;
match kind.as_str() {
"text" | "json_object" => Ok(json!({"type": kind})),
"json_schema" => {
let mut schema = object(
required(&mut format, "json_schema", "response_format.json_schema")?,
"response_format.json_schema",
)?;
let name = required_string(
&mut schema,
"name",
"response_format.json_schema.name",
false,
)?;
let definition = required(&mut schema, "schema", "response_format.json_schema.schema")?;
if definition.is_null() {
return Err(invalid("response_format.json_schema.schema"));
}
let mut converted = json!({"type": "json_schema", "name": name, "schema": definition});
if let Some(strict) =
optional_bool(&mut schema, "strict", "response_format.json_schema.strict")?
{
converted["strict"] = strict.into();
}
if let Some(description) = optional(&mut schema, "description") {
string(
&description,
"response_format.json_schema.description",
true,
)?;
converted["description"] = description;
}
Ok(converted)
}
_ => Err(unsupported("response_format.type")),
}
}
fn optional(object: &mut Map<String, Value>, key: &str) -> Option<Value> {
object.remove(key).filter(|value| !value.is_null())
}
fn required(
object: &mut Map<String, Value>,
key: &str,
path: &str,
) -> Result<Value, RequestDecodeError> {
object
.remove(key)
.ok_or_else(|| RequestDecodeError::MissingField { field: path.into() })
}
fn object(value: Value, path: &str) -> Result<Map<String, Value>, RequestDecodeError> {
match value {
Value::Object(object) => Ok(object),
_ => Err(invalid_type(path, "an object")),
}
}
fn array(value: Value, path: &str) -> Result<Vec<Value>, RequestDecodeError> {
match value {
Value::Array(array) => Ok(array),
_ => Err(invalid_type(path, "an array")),
}
}
fn string<'a>(
value: &'a Value,
path: &str,
allow_empty: bool,
) -> Result<&'a str, RequestDecodeError> {
let value = value
.as_str()
.ok_or_else(|| invalid_type(path, "a string"))?;
if !allow_empty && value.trim().is_empty() {
return Err(RequestDecodeError::EmptyField { field: path.into() });
}
Ok(value)
}
fn required_string(
object: &mut Map<String, Value>,
key: &str,
path: &str,
allow_empty: bool,
) -> Result<String, RequestDecodeError> {
let value = required(object, key, path)?;
string(&value, path, allow_empty)?;
match value {
Value::String(value) => Ok(value),
_ => Err(invalid_type(path, "a string")),
}
}
fn optional_bool(
object: &mut Map<String, Value>,
key: &str,
path: &str,
) -> Result<Option<bool>, RequestDecodeError> {
optional(object, key)
.map(|value| {
value
.as_bool()
.ok_or_else(|| invalid_type(path, "a boolean"))
})
.transpose()
}
fn invalid_type(path: &str, expected: &'static str) -> RequestDecodeError {
RequestDecodeError::InvalidType {
field: path.into(),
expected,
}
}
fn invalid(path: &str) -> RequestDecodeError {
RequestDecodeError::InvalidValue { field: path.into() }
}
fn unsupported(path: &str) -> RequestDecodeError {
RequestDecodeError::UnsupportedField { field: path.into() }
}
@@ -0,0 +1,664 @@
//! 将实际 Responses wire 投影为 Chat JSON 和 SSE,不拥有执行、计费或终结生命周期
use std::collections::{BTreeMap, BTreeSet};
use bytes::Bytes;
use gateway_protocol::openai::events::{billable_usage_is_complete, extract_usage};
use gateway_protocol::openai::sse::encode_sse_event;
use serde::Serialize;
use serde_json::{Value, json};
use crate::openai::responses::{ProtocolError, ProtocolErrorBody};
#[derive(Debug, Clone, Serialize)]
pub(in crate::openai) struct ChatIdentity {
pub id: String,
pub created: u64,
pub model: String,
}
#[derive(Serialize)]
struct Envelope<'a> {
#[serde(flatten)]
identity: &'a ChatIdentity,
object: &'static str,
choices: Value,
#[serde(skip_serializing_if = "Option::is_none")]
usage: Option<Value>,
#[serde(skip_serializing_if = "Option::is_none")]
metadata: Option<&'a Value>,
#[serde(skip_serializing_if = "Option::is_none")]
service_tier: Option<&'a Value>,
#[serde(skip_serializing_if = "Option::is_none")]
moderation: Option<Value>,
}
/// 输入已由现有 Responses collector 聚合,投影不再恢复或重建缺失的 output
pub(in crate::openai) fn project_response(
response: &Value,
identity: &ChatIdentity,
) -> Result<Value, ProtocolErrorBody> {
let output = response
.get("output")
.and_then(Value::as_array)
.ok_or_else(invalid_response)?;
validate_output_identities(output)?;
let mut content = None;
let mut refusal = None;
let mut tools = Vec::new();
for item in output {
match required_str(item, "type")? {
"message" => {
for part in message_parts(item)? {
let (kind, text) = part_text(part)?;
let target = match kind {
PartKind::Text => &mut content,
PartKind::Refusal => &mut refusal,
};
target.get_or_insert_with(String::new).push_str(text);
}
}
"function_call" => {
let (id, name, arguments) = function_fields(item)?;
tools.push(json!({
"id": id, "type": "function",
"function": {"name": name, "arguments": arguments}
}));
}
"reasoning" => {}
_ => return Err(invalid_response()),
}
}
let finish_reason = finish_reason(response, !tools.is_empty())?;
let mut message = json!({"role": "assistant", "content": content, "refusal": refusal});
if !tools.is_empty() {
message["tool_calls"] = Value::Array(tools);
}
serde_json::to_value(Envelope {
identity,
object: "chat.completion",
choices: json!([{"index": 0, "message": message, "finish_reason": finish_reason, "logprobs": null}]),
usage: chat_usage(response),
metadata: response.get("metadata"),
service_tier: response.get("service_tier"),
moderation: project_moderation(response)?,
})
.map_err(|_| invalid_response())
}
#[derive(Clone, Copy, PartialEq, Eq)]
enum PartKind {
Text,
Refusal,
}
impl PartKind {
const fn field(self) -> &'static str {
match self {
Self::Text => "content",
Self::Refusal => "refusal",
}
}
}
struct Part {
kind: PartKind,
text: String,
}
struct Tool {
index: usize,
call_id: String,
name: String,
arguments: String,
}
enum Output {
Message(BTreeMap<u64, Part>),
Function(Tool),
Reasoning,
}
struct OutputItem {
id: Option<String>,
output: Output,
}
#[derive(Clone, Copy, PartialEq, Eq)]
enum Terminal {
Completed,
Failed,
}
/// 只保存身份关联与快照去重所需内容;终态帧由外层受管 body 决定何时交付
pub(in crate::openai) struct ChatStreamEncoder {
identity: ChatIdentity,
include_usage: bool,
role_sent: bool,
output: BTreeMap<u64, OutputItem>,
terminal: Option<Terminal>,
}
impl ChatStreamEncoder {
pub(in crate::openai) fn new(identity: ChatIdentity, include_usage: bool) -> Self {
Self {
identity,
include_usage,
role_sent: false,
output: BTreeMap::new(),
terminal: None,
}
}
pub(in crate::openai) fn is_completed(&self) -> bool {
self.terminal == Some(Terminal::Completed)
}
pub(in crate::openai) fn has_wire_failure(&self) -> bool {
self.terminal == Some(Terminal::Failed)
}
pub(in crate::openai) fn push_event(
&mut self,
event_type: Option<&str>,
data: &Value,
) -> Result<Vec<Bytes>, ProtocolErrorBody> {
let event_type = event_type.or_else(|| data.get("type").and_then(Value::as_str));
if self.has_wire_failure() {
return Ok(Vec::new());
}
let mut frames = Vec::new();
match event_type {
Some("response.created") => {
self.require_open()?;
if !self.role_sent {
frames.push(self.delta(json!({"role": "assistant"}))?);
self.role_sent = true;
}
}
Some("response.output_text.delta" | "response.refusal.delta") => {
self.require_open()?;
let kind = if event_type == Some("response.refusal.delta") {
PartKind::Refusal
} else {
PartKind::Text
};
let delta = required_str(data, "delta")?;
let part = self.event_part(data, kind)?;
part.text.push_str(delta);
if !delta.is_empty() {
frames.push(self.delta(json!({kind.field(): delta}))?);
}
}
Some("response.output_text.done" | "response.refusal.done") => {
self.require_open()?;
let (kind, field) = if event_type == Some("response.refusal.done") {
(PartKind::Refusal, "refusal")
} else {
(PartKind::Text, "text")
};
let text = required_str(data, field)?;
let part = self.event_part(data, kind)?;
let suffix = append_snapshot(&mut part.text, text)?;
if !suffix.is_empty() {
frames.push(self.delta(json!({kind.field(): suffix}))?);
}
}
Some("response.content_part.added" | "response.content_part.done") => {
self.require_open()?;
let part = data.get("part").ok_or_else(invalid_response)?;
let (kind, text) = part_text(part)?;
let part = self.event_part(data, kind)?;
let suffix = append_snapshot(&mut part.text, text)?;
if !suffix.is_empty() {
frames.push(self.delta(json!({kind.field(): suffix}))?);
}
}
Some("response.output_item.added" | "response.output_item.done") => {
self.require_open()?;
self.item_snapshot(
required_index(data, "output_index")?,
data.get("item").ok_or_else(invalid_response)?,
false,
&mut frames,
)?;
}
Some(
"response.function_call_arguments.delta" | "response.function_call_arguments.done",
) => {
self.require_open()?;
let index = required_index(data, "output_index")?;
let item = self.output.get_mut(&index).ok_or_else(invalid_response)?;
bind_id(item, optional_id(data, "item_id")?)?;
let Output::Function(tool) = &mut item.output else {
return Err(invalid_response());
};
let delta = if event_type == Some("response.function_call_arguments.delta") {
let delta = required_str(data, "delta")?;
tool.arguments.push_str(delta);
delta
} else {
append_snapshot(&mut tool.arguments, required_str(data, "arguments")?)?
};
let tool_index = tool.index;
if !delta.is_empty() {
frames.push(self.delta(json!({"tool_calls": [{"index": tool_index, "function": {"arguments": delta}}]}))?);
}
}
Some("response.completed" | "response.incomplete") => {
self.require_open()?;
let response = data.get("response").ok_or_else(invalid_response)?;
let status = required_str(response, "status")?;
if (event_type == Some("response.completed") && status != "completed")
|| (event_type == Some("response.incomplete") && status != "incomplete")
{
return Err(invalid_response());
}
if let Some(output) = response.get("output") {
let output = output.as_array().ok_or_else(invalid_response)?;
validate_output_identities(output)?;
for (index, item) in output.iter().enumerate() {
self.item_snapshot(index as u64, item, true, &mut frames)?;
}
} else if self.output.is_empty() {
return Err(invalid_response());
}
let finish_reason = finish_reason(response, self.tool_count() > 0)?;
frames.push(self.chunk(
json!([{"index": 0, "delta": {}, "finish_reason": finish_reason, "logprobs": null}]),
self.include_usage.then_some(Value::Null),
Some(response),
)?);
if self.include_usage
&& let Some(usage) = chat_usage(response)
{
frames.push(self.chunk(json!([]), Some(usage), None)?);
}
self.terminal = Some(Terminal::Completed);
}
Some("response.failed" | "error") => {
self.require_open()?;
let error = data
.pointer("/response/error")
.or_else(|| data.get("error"))
.unwrap_or(data);
required_str(error, "message")?;
// 错误恢复信号和扩展由既有出口拥有,Chat 仅转换外层包装
let mut error = error.clone();
if error
.get("type")
.and_then(Value::as_str)
.is_none_or(|kind| kind == "error")
{
error["type"] = json!("server_error");
}
frames.push(sse_frame(&json!({"error": error})));
self.terminal = Some(Terminal::Failed);
}
// 辅助事件可随上游演进;实际 output item 和 content part 仍逐项检查可表达性
_ => {}
}
// done-only 流也要提供完整 Chat 消息身份,辅助事件或私有推理不触发角色帧
if !self.role_sent && !frames.is_empty() && !self.has_wire_failure() {
frames.insert(0, self.delta(json!({"role": "assistant"}))?);
self.role_sent = true;
}
Ok(frames)
}
fn require_open(&self) -> Result<(), ProtocolErrorBody> {
if self.terminal.is_some() {
Err(invalid_response())
} else {
Ok(())
}
}
fn tool_count(&self) -> usize {
self.output
.values()
.filter(|item| matches!(item.output, Output::Function(_)))
.count()
}
fn event_part(&mut self, data: &Value, kind: PartKind) -> Result<&mut Part, ProtocolErrorBody> {
let index = required_index(data, "output_index")?;
let content_index = required_index(data, "content_index")?;
let id = optional_id(data, "item_id")?;
let item = self.output.entry(index).or_insert_with(|| OutputItem {
id: None,
output: Output::Message(BTreeMap::new()),
});
bind_id(item, id)?;
message_part(item, content_index, kind)
}
fn item_snapshot(
&mut self,
index: u64,
snapshot: &Value,
terminal: bool,
frames: &mut Vec<Bytes>,
) -> Result<(), ProtocolErrorBody> {
let id = optional_id(snapshot, "id")?;
let kind = required_str(snapshot, "type")?;
let call_id = if kind == "function_call" {
Some(required_nonempty_str(snapshot, "call_id")?)
} else {
None
};
let known_index = self.find_item(id, call_id)?;
let index = match known_index {
Some(known) if terminal || known == index => known,
Some(_) => return Err(invalid_response()),
None => index,
};
if !self.output.contains_key(&index) {
let output = match kind {
"message" => Output::Message(BTreeMap::new()),
"reasoning" => Output::Reasoning,
"function_call" => {
let (call_id, name, _) = function_fields(snapshot)?;
let tool_index = self.tool_count();
frames.push(self.delta(json!({"tool_calls": [{"index": tool_index, "id": call_id, "type": "function", "function": {"name": name, "arguments": ""}}]}))?);
Output::Function(Tool {
index: tool_index,
call_id: call_id.to_owned(),
name: name.to_owned(),
arguments: String::new(),
})
}
_ => return Err(invalid_response()),
};
self.output.insert(index, OutputItem { id: None, output });
}
let item = self.output.get_mut(&index).ok_or_else(invalid_response)?;
bind_id(item, id)?;
match (&mut item.output, kind) {
(Output::Message(parts), "message") => {
for (index, part) in message_parts(snapshot)?.iter().enumerate() {
let (kind, text) = part_text(part)?;
let part = part_entry(parts, index as u64, kind)?;
let suffix = append_snapshot(&mut part.text, text)?;
if !suffix.is_empty() {
frames.push(delta_frame(
&self.identity,
self.include_usage,
json!({kind.field(): suffix}),
)?);
}
}
}
(Output::Function(tool), "function_call") => {
let (id, name, arguments) = function_fields(snapshot)?;
if tool.call_id != id || tool.name != name {
return Err(invalid_response());
}
let suffix = append_snapshot(&mut tool.arguments, arguments)?;
if !suffix.is_empty() {
frames.push(delta_frame(&self.identity, self.include_usage, json!({"tool_calls": [{"index": tool.index, "function": {"arguments": suffix}}]}))?);
}
}
(Output::Reasoning, "reasoning") => {}
_ => return Err(invalid_response()),
}
Ok(())
}
fn find_item(
&self,
id: Option<&str>,
call_id: Option<&str>,
) -> Result<Option<u64>, ProtocolErrorBody> {
let mut found = None;
for (&index, item) in &self.output {
let same_id = id.is_some() && item.id.as_deref() == id;
let same_call = matches!(&item.output, Output::Function(tool) if call_id == Some(tool.call_id.as_str()));
if same_id || same_call {
if found.is_some_and(|previous| previous != index) {
return Err(invalid_response());
}
found = Some(index);
}
}
Ok(found)
}
fn delta(&self, delta: Value) -> Result<Bytes, ProtocolErrorBody> {
delta_frame(&self.identity, self.include_usage, delta)
}
fn chunk(
&self,
choices: Value,
usage: Option<Value>,
response: Option<&Value>,
) -> Result<Bytes, ProtocolErrorBody> {
chunk_frame(&self.identity, choices, usage, response)
}
}
fn message_part(
item: &mut OutputItem,
index: u64,
kind: PartKind,
) -> Result<&mut Part, ProtocolErrorBody> {
let Output::Message(parts) = &mut item.output else {
return Err(invalid_response());
};
part_entry(parts, index, kind)
}
fn part_entry(
parts: &mut BTreeMap<u64, Part>,
index: u64,
kind: PartKind,
) -> Result<&mut Part, ProtocolErrorBody> {
let part = parts.entry(index).or_insert_with(|| Part {
kind,
text: String::new(),
});
if part.kind != kind {
return Err(invalid_response());
}
Ok(part)
}
fn bind_id(item: &mut OutputItem, id: Option<&str>) -> Result<(), ProtocolErrorBody> {
if let Some(id) = id {
if item.id.as_deref().is_some_and(|previous| previous != id) {
return Err(invalid_response());
}
if item.id.is_none() {
item.id = Some(id.to_owned());
}
}
Ok(())
}
fn append_snapshot<'a>(text: &mut String, snapshot: &'a str) -> Result<&'a str, ProtocolErrorBody> {
let suffix = snapshot
.strip_prefix(text.as_str())
.ok_or_else(invalid_response)?;
text.push_str(suffix);
Ok(suffix)
}
fn message_parts(item: &Value) -> Result<&[Value], ProtocolErrorBody> {
if required_str(item, "role")? != "assistant" {
return Err(invalid_response());
}
item.get("content")
.and_then(Value::as_array)
.map(Vec::as_slice)
.ok_or_else(invalid_response)
}
fn part_text(part: &Value) -> Result<(PartKind, &str), ProtocolErrorBody> {
match required_str(part, "type")? {
"output_text" => Ok((PartKind::Text, required_str(part, "text")?)),
"refusal" => Ok((PartKind::Refusal, required_str(part, "refusal")?)),
_ => Err(invalid_response()),
}
}
fn function_fields(item: &Value) -> Result<(&str, &str, &str), ProtocolErrorBody> {
Ok((
required_nonempty_str(item, "call_id")?,
required_nonempty_str(item, "name")?,
required_str(item, "arguments")?,
))
}
fn validate_output_identities(output: &[Value]) -> Result<(), ProtocolErrorBody> {
let mut item_ids = BTreeSet::new();
let mut call_ids = BTreeSet::new();
for item in output {
if let Some(id) = optional_id(item, "id")?
&& !item_ids.insert(id)
{
return Err(invalid_response());
}
if item.get("type").and_then(Value::as_str) == Some("function_call")
&& !call_ids.insert(required_nonempty_str(item, "call_id")?)
{
return Err(invalid_response());
}
}
Ok(())
}
fn finish_reason(response: &Value, has_tools: bool) -> Result<&'static str, ProtocolErrorBody> {
if response.get("error").is_some_and(|value| !value.is_null()) {
return Err(invalid_response());
}
match required_str(response, "status")? {
"completed" => Ok(if has_tools { "tool_calls" } else { "stop" }),
"incomplete" => match response
.pointer("/incomplete_details/reason")
.and_then(Value::as_str)
{
Some("max_output_tokens") => Ok("length"),
Some("content_filter") => Ok("content_filter"),
_ => Err(invalid_response()),
},
_ => Err(invalid_response()),
}
}
fn chat_usage(response: &Value) -> Option<Value> {
let usage = extract_usage(response)?;
if !billable_usage_is_complete(response, usage) {
return None;
}
let mut result = json!({"prompt_tokens": usage.input_tokens, "completion_tokens": usage.output_tokens, "total_tokens": usage.total_tokens});
if response
.pointer("/usage/input_tokens_details/cached_tokens")
.is_some()
{
result["prompt_tokens_details"] = json!({"cached_tokens": usage.cached_input_tokens});
}
if response
.pointer("/usage/output_tokens_details/reasoning_tokens")
.and_then(Value::as_u64)
.is_some_and(|tokens| tokens <= usage.output_tokens)
{
result["completion_tokens_details"] =
json!({"reasoning_tokens": usage.reasoning_output_tokens});
}
Some(result)
}
fn project_moderation(response: &Value) -> Result<Option<Value>, ProtocolErrorBody> {
let Some(moderation) = response.get("moderation").filter(|value| !value.is_null()) else {
return Ok(None);
};
let mut result = json!({});
for side in ["input", "output"] {
let value = moderation.get(side).ok_or_else(invalid_response)?;
result[side] = match required_str(value, "type")? {
"moderation_result" => {
json!({"type": "moderation_results", "model": required_str(value, "model")?, "results": [value]})
}
"error" => value.clone(),
_ => return Err(invalid_response()),
};
}
Ok(Some(result))
}
fn delta_frame(
identity: &ChatIdentity,
include_usage: bool,
delta: Value,
) -> Result<Bytes, ProtocolErrorBody> {
chunk_frame(
identity,
json!([{"index": 0, "delta": delta, "finish_reason": null, "logprobs": null}]),
include_usage.then_some(Value::Null),
None,
)
}
fn chunk_frame(
identity: &ChatIdentity,
choices: Value,
usage: Option<Value>,
response: Option<&Value>,
) -> Result<Bytes, ProtocolErrorBody> {
let value = serde_json::to_value(Envelope {
identity,
object: "chat.completion.chunk",
choices,
usage,
metadata: None,
service_tier: response.and_then(|value| value.get("service_tier")),
moderation: response.map(project_moderation).transpose()?.flatten(),
})
.map_err(|_| invalid_response())?;
Ok(sse_frame(&value))
}
fn sse_frame(value: &Value) -> Bytes {
Bytes::from(encode_sse_event("", &value.to_string()))
}
fn required_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, ProtocolErrorBody> {
value
.get(field)
.and_then(Value::as_str)
.ok_or_else(invalid_response)
}
fn required_nonempty_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, ProtocolErrorBody> {
let value = required_str(value, field)?;
if value.is_empty() {
return Err(invalid_response());
}
Ok(value)
}
fn optional_id<'a>(value: &'a Value, field: &str) -> Result<Option<&'a str>, ProtocolErrorBody> {
value
.get(field)
.map(|_| required_nonempty_str(value, field))
.transpose()
}
fn required_index(value: &Value, field: &str) -> Result<u64, ProtocolErrorBody> {
value
.get(field)
.and_then(Value::as_u64)
.ok_or_else(invalid_response)
}
fn invalid_response() -> ProtocolErrorBody {
ProtocolErrorBody {
error: ProtocolError {
kind: "server_error",
code: "invalid_upstream_response",
message: "The upstream response cannot be represented as a Chat completion.".to_owned(),
param: None,
},
}
}
@@ -1,6 +1,7 @@
//! OpenAI 客户端协议 adapter
pub mod auth;
mod chat_completions;
mod endpoint;
pub mod error;
pub mod images;
@@ -31,6 +31,7 @@ use gateway_core::event::{ProviderEvent, ProviderResponseHeader};
use gateway_core::lifecycle::{CancellationToken, ConnectionGuard};
use gateway_core::operation::OperationKind;
use gateway_protocol::openai::sse::{DONE_SSE_FRAME, response_failed_sse_event_with_id};
use serde_json::Value;
use tokio::time::Instant;
use crate::ApiState;
@@ -120,6 +121,23 @@ pub(crate) async fn execute_prepared_responses(
prepared: PreparedRootExecution,
request: ResponsesHttpRequest,
authorization: Option<Arc<dyn ResponseAuthorization>>,
) -> Response {
execute_prepared_response(service, prepared, request, authorization, None).await
}
/// 下游协议只投影出口字节 执行会话和计量仍归原 owner
pub(in crate::openai) struct HttpResponseAdapter {
pub endpoint: &'static str,
pub wrap: Box<dyn FnOnce(MiddlewareResponse) -> MiddlewareResponse + Send>,
pub stream_error: fn(&Value) -> Bytes,
}
pub(in crate::openai) async fn execute_prepared_response(
service: OpenAiService,
prepared: PreparedRootExecution,
request: ResponsesHttpRequest,
authorization: Option<Arc<dyn ResponseAuthorization>>,
adapter: Option<HttpResponseAdapter>,
) -> Response {
let ResponsesHttpRequest {
peer_address,
@@ -148,8 +166,14 @@ pub(crate) async fn execute_prepared_responses(
// 中间件正文已经由入口按上限解压;传输编码和旧长度不能与改写后的正文混用
middleware_headers.remove(CONTENT_ENCODING);
middleware_headers.remove(CONTENT_LENGTH);
let endpoint = adapter
.as_ref()
.map_or(crate::openai::router::RESPONSES_PATH, |adapter| {
adapter.endpoint
});
let stream_error = adapter.as_ref().map(|adapter| adapter.stream_error);
let input = RequestInput {
endpoint: crate::openai::router::RESPONSES_PATH.to_owned(),
endpoint: endpoint.to_owned(),
protocol: OPENAI_PROTOCOL.to_owned(),
operation: Some(OperationKind::Generate),
transport,
@@ -194,12 +218,7 @@ pub(crate) async fn execute_prepared_responses(
None
};
let started = terminal_service
.start_prepared_response(
prepared,
decoded,
transport,
crate::openai::router::RESPONSES_PATH,
)
.start_prepared_response(prepared, decoded, transport, endpoint)
.await
.map_err(MiddlewareError::Gateway)?;
let _ = terminal_request_id.set(started.request_id.clone());
@@ -242,17 +261,18 @@ pub(crate) async fn execute_prepared_responses(
} else {
response
};
into_http_response(
validated_middleware_response(response, validation, stream),
expected,
)
.await
let response = validated_middleware_response(response, validation, stream);
let response = match adapter {
Some(adapter) => (adapter.wrap)(response),
None => response,
};
into_http_response(response, expected).await
}
Err(error) => error_response(error),
}
};
let response = if stream {
super::keepalive::response(response, keepalive).await
super::keepalive::response_with_encoder(response, keepalive, stream_error).await
} else {
response.await
};
@@ -21,6 +21,14 @@ use crate::openai::middleware::{SSE_KEEPALIVE_FRAME, SSE_KEEPALIVE_INTERVAL};
pub(super) async fn response(
response: impl Future<Output = Response> + Send + 'static,
admitted: oneshot::Receiver<ResponseCommit>,
) -> Response {
response_with_encoder(response, admitted, None).await
}
pub(super) async fn response_with_encoder(
response: impl Future<Output = Response> + Send + 'static,
admitted: oneshot::Receiver<ResponseCommit>,
error_encoder: Option<fn(&Value) -> Bytes>,
) -> Response {
let mut response = Box::pin(response);
// 鉴权、模型门禁和准入失败仍保留原 HTTP 状态,不提前建立 SSE
@@ -40,7 +48,7 @@ pub(super) async fn response(
if !commit.record(Some(StatusCode::OK.as_u16())) {
return response.await;
}
let pending = PendingResponse::Waiting(response);
let pending = PendingResponse::Waiting(response, error_encoder);
let stream = stream::once(async { Ok::<_, axum::Error>(SSE_KEEPALIVE_FRAME) })
.chain(stream::try_unfold(pending, PendingResponse::next));
let mut response = Response::new(Body::from_stream(stream));
@@ -58,7 +66,7 @@ pub(super) async fn response(
}
enum PendingResponse {
Waiting(BoxFuture<'static, Response>),
Waiting(BoxFuture<'static, Response>, Option<fn(&Value) -> Bytes>),
Streaming(BodyDataStream),
}
@@ -66,7 +74,7 @@ impl PendingResponse {
async fn next(mut self) -> Result<Option<(Bytes, Self)>, axum::Error> {
loop {
match &mut self {
Self::Waiting(response) => {
Self::Waiting(response, error_encoder) => {
// 心跳只轮询同一个 future,不取消重建首帧等待或插件调用
let response = tokio::select! {
biased;
@@ -85,7 +93,7 @@ impl PendingResponse {
let body = if sse {
response.into_body()
} else {
stream_failure(response).await
stream_failure(response, *error_encoder).await
};
// 正式响应的正文已经带有交付期心跳,后续只转交,不叠加定时器
self = Self::Streaming(body.into_data_stream());
@@ -102,7 +110,7 @@ impl PendingResponse {
}
}
async fn stream_failure(response: Response) -> Body {
async fn stream_failure(response: Response, error_encoder: Option<fn(&Value) -> Bytes>) -> Body {
let status = response.status();
let retry_after = response
.headers()
@@ -145,7 +153,10 @@ async fn stream_failure(response: Response) -> Body {
.entry("retry-after")
.or_insert(Value::String(retry_after));
}
let mut event = encode_sse_event("response.failed", &data.to_string());
event.push_str(DONE_SSE_FRAME);
let mut event = match error_encoder {
Some(encode) => encode(&data).to_vec(),
None => encode_sse_event("response.failed", &data.to_string()).into_bytes(),
};
event.extend_from_slice(DONE_SSE_FRAME.as_bytes());
Body::from(event)
}
@@ -10,10 +10,12 @@ pub mod websocket;
pub(crate) use crate::request_context::request_client_context;
pub use error::{ProtocolError, ProtocolErrorBody, RequestDecodeError, ResponseEncodeError};
pub(in crate::openai) use http::{HttpResponseAdapter, execute_prepared_response};
pub(crate) use http::{
ResponseAuthorization, ResponsesHttpRequest, execute_prepared_responses, responses,
};
pub use http::{collect_execution_response, stream_execution_response};
pub(in crate::openai) use request::decompress_request_body;
pub use request::{
ContinuationIntent, DecodedResponsesRequest, OpenAiRequestHeaders, ResponsesRequestMetadata,
decode_request_with_headers,
@@ -371,7 +371,7 @@ const MAX_ZSTD_WINDOW_LOG: u32 = 26;
/// 未压缩与 `identity` 借用原始正文;压缩正文在读取过程中限制展开大小,
/// 避免先完整分配再检查
/// 只接受单一编码,重复头和叠加编码不能只解释第一项
fn decompress_request_body<'a>(
pub(in crate::openai) fn decompress_request_body<'a>(
body: &'a [u8],
headers: &HeaderMap,
max_decompressed_bytes: usize,
@@ -10,6 +10,8 @@ pub(super) const DOWNSTREAM_WRITE_TIMEOUT: Duration = Duration::from_secs(5 * 60
pub(super) const DOWNSTREAM_CLOSE_TIMEOUT: Duration = Duration::from_secs(1);
pub(super) const CONNECTION_MAX_AGE: Duration = Duration::from_secs(60 * 60);
pub(super) const DOWNSTREAM_PING_INTERVAL: Duration = Duration::from_secs(25);
pub(super) const DOWNSTREAM_PONG_TIMEOUT: Duration = Duration::from_secs(30);
pub(super) const DOWNSTREAM_PING_WRITE_TIMEOUT: Duration = Duration::from_secs(5);
/// WebSocket pump 的写入与生命周期预算
#[derive(Clone, Copy)]
@@ -124,6 +126,7 @@ pub enum PumpExitReason {
ReadError,
WriteError,
WriteTimeout,
PongTimeout,
LifecycleShutdown,
ConnectionMaxAge,
CoordinatorDropped,
@@ -141,6 +144,7 @@ impl PumpExitReason {
Self::ReadError => "read_error",
Self::WriteError => "write_error",
Self::WriteTimeout => "write_timeout",
Self::PongTimeout => "pong_timeout",
Self::LifecycleShutdown => "lifecycle_shutdown",
Self::ConnectionMaxAge => "connection_max_age",
Self::CoordinatorDropped => "coordinator_dropped",
@@ -30,6 +30,7 @@ use tokio::{
pub struct ResponsesWebSocketConnection {
connection_id: Arc<str>,
opened_at: Instant,
cancellation: CancellationToken,
expired: Arc<AtomicBool>,
commands: Option<mpsc::Sender<ConnectionCommand>>,
incoming: mpsc::Receiver<PendingConnectionEvent>,
@@ -266,7 +267,9 @@ impl ResponsesWebSocketConnection {
}
fn terminate(&mut self, reason: PumpExitReason) {
let reason = self.exited.try_recv().unwrap_or(reason);
self.exit_reason.get_or_insert(reason);
self.cancellation.cancel();
self.commands.take();
if let Some(task) = self.pump_task.take() {
task.abort();
@@ -306,6 +309,7 @@ impl ResponsesWebSocketConnection {
impl Drop for ResponsesWebSocketConnection {
fn drop(&mut self) {
self.cancellation.cancel();
self.commands.take();
if let Some(task) = self.pump_task.take() {
task.abort();
@@ -314,6 +318,7 @@ impl Drop for ResponsesWebSocketConnection {
}
/// 为 WebSocket transport 启动包含冻结中间件计划的单 owner pump
/// cancellation 必须属于本连接,连接退出或句柄释放时会取消它
pub fn spawn_connection<S, E>(
socket: S,
connection_id: Arc<str>,
@@ -341,7 +346,7 @@ where
incoming_tx,
exit_tx,
Arc::clone(&connection_id),
cancellation,
cancellation.clone(),
opened_at,
Arc::clone(&expired),
Arc::clone(&stats),
@@ -353,6 +358,7 @@ where
ResponsesWebSocketConnection {
connection_id,
opened_at,
cancellation,
expired,
commands: Some(command_tx),
incoming: incoming_rx,
@@ -19,7 +19,7 @@ use std::{
},
};
use tokio::{
sync::{Semaphore, mpsc, oneshot},
sync::{Semaphore, mpsc, oneshot, watch},
time::Instant,
};
@@ -42,6 +42,7 @@ pub(super) async fn run_pump<S, E>(
S: Stream<Item = Result<Message, E>> + Sink<Message, Error = E> + Unpin + Send + 'static,
E: fmt::Display + Send + 'static,
{
let _connection_lifetime = middleware::CancelOnDrop(Some(cancellation.clone()));
let context = Arc::new(PumpContext {
extensions,
connection_id,
@@ -57,11 +58,13 @@ pub(super) async fn run_pump<S, E>(
let (writer, mut reader) = socket.split();
let writer = Arc::new(middleware::Writer::new(writer, context.clone()));
let sender: Arc<dyn contract::Sender> = writer.clone();
// 同一个受管任务同时驱动两侧;等待写入不能阻塞入站控制、关闭或取消
let (activity, received_activity) = watch::channel(());
// 心跳独立于业务写入等待,但共享唯一 writer,避免长写入或排队阻止失联检测
let reason = tokio::select! {
biased;
reason = read_connection(&mut reader, &incoming, sender.clone(), &context) => reason,
reason = read_connection(&mut reader, &incoming, sender.clone(), &activity, &context) => reason,
reason = write_connection(writer.clone(), commands, &context) => reason,
reason = heartbeat_connection(writer.clone(), received_activity, &context) => reason,
};
if matches!(reason, PumpExitReason::ClientClose) {
if let Err(error) = writer.finish_peer_close(&cancellation).await {
@@ -78,6 +81,7 @@ async fn read_connection<S, E>(
socket: &mut S,
incoming: &mpsc::Sender<PendingConnectionEvent>,
sender: Arc<dyn contract::Sender>,
activity: &watch::Sender<()>,
context: &PumpContext,
) -> PumpExitReason
where
@@ -103,6 +107,10 @@ where
}
message = socket.next() => {
if matches!(&message, Some(Ok(_))) { context.stats.record_read(context.opened_at); }
if matches!(&message, Some(Ok(message)) if !matches!(message, Message::Close(_))) {
// 链路活动以真实入站为准,插件丢弃消息不应抹掉对端已经响应的事实
activity.send_replace(());
}
let message = match message {
Some(Ok(message)) => match middleware::transform(message, contract::Direction::Incoming, sender.clone(), context).await {
Ok(Some(message)) => Some(Ok(message)),
@@ -145,40 +153,22 @@ where
S: Sink<Message, Error = E> + Unpin + Send + 'static,
E: fmt::Display + Send + 'static,
{
let mut heartbeat = tokio::time::interval_at(
context.opened_at + DOWNSTREAM_PING_INTERVAL,
DOWNSTREAM_PING_INTERVAL,
);
heartbeat.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
let mut ping_sequence = 0_u64;
loop {
let message = tokio::select! {
let command = tokio::select! {
biased;
() = context.cancellation.cancelled() => return PumpExitReason::LifecycleShutdown,
command = commands.recv() => {
let Some(command) = command else { return PumpExitReason::CoordinatorDropped; };
let ConnectionCommand { message, context: write_context, acknowledged } = command;
let closing = matches!(message, Message::Close(_));
let result = match middleware::transform(message, contract::Direction::Outgoing, writer.clone(), context).await {
Ok(Some(message)) => writer.write(message).await.map(|()| WriteOutcome::Written),
Ok(None) => Ok(WriteOutcome::Suppressed),
Err(error) => Err(ConnectionWriteError::Transport { message: error.to_string() }),
};
let reason = result.as_ref().err().map(|error| write_exit_reason(error, context));
if let Err(error) = &result {
tracing::debug!(websocket_connection_id = %context.connection_id, request_id = write_context.request_id.as_deref().unwrap_or(""), frame_phase = write_context.phase.as_str(), error = %error, "Responses WebSocket pump transport write failed");
}
let _ = acknowledged.send(result);
if let Some(reason) = reason { return reason; }
if closing { return PumpExitReason::ServerClose; }
continue;
}
_ = heartbeat.tick() => {
ping_sequence = ping_sequence.wrapping_add(1);
Message::Ping(ping_sequence.to_be_bytes().to_vec().into())
}
command = commands.recv() => command,
};
let message = match middleware::transform(
let Some(command) = command else {
return PumpExitReason::CoordinatorDropped;
};
let ConnectionCommand {
message,
context: write_context,
acknowledged,
} = command;
let closing = matches!(message, Message::Close(_));
let result = match middleware::transform(
message,
contract::Direction::Outgoing,
writer.clone(),
@@ -186,21 +176,125 @@ where
)
.await
{
Ok(Some(message)) => message,
Ok(None) => continue,
Err(_) => return PumpExitReason::MiddlewareFailed,
Ok(Some(message)) => writer.write(message).await.map(|()| WriteOutcome::Written),
Ok(None) => Ok(WriteOutcome::Suppressed),
Err(error) => Err(ConnectionWriteError::Transport {
message: error.to_string(),
}),
};
if let Err(error) = writer.write(message).await {
tracing::info!(websocket_connection_id = %context.connection_id, error = %error, "Responses WebSocket control write failed");
return write_exit_reason(&error, context);
let reason = result
.as_ref()
.err()
.map(|error| write_exit_reason(error, context));
if let Err(error) = &result {
tracing::debug!(websocket_connection_id = %context.connection_id, request_id = write_context.request_id.as_deref().unwrap_or(""), frame_phase = write_context.phase.as_str(), error = %error, "Responses WebSocket pump transport write failed");
}
let _ = acknowledged.send(result);
if let Some(reason) = reason {
return reason;
}
if closing {
return PumpExitReason::ServerClose;
}
context
.stats
.ping_written_count
.fetch_add(1, Ordering::Relaxed);
}
}
async fn heartbeat_connection<S, E>(
writer: Arc<middleware::Writer<S>>,
mut activity: watch::Receiver<()>,
context: &PumpContext,
) -> PumpExitReason
where
S: Sink<Message, Error = E> + Unpin + Send + 'static,
E: fmt::Display + Send + 'static,
{
let mut heartbeat = tokio::time::interval_at(
context.opened_at + DOWNSTREAM_PING_INTERVAL,
DOWNSTREAM_PING_INTERVAL,
);
heartbeat.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
let mut ping_sequence = 0_u64;
let pong_deadline = tokio::time::sleep(DOWNSTREAM_PONG_TIMEOUT);
tokio::pin!(pong_deadline);
let mut pending_ping = false;
loop {
tokio::select! {
biased;
() = context.cancellation.cancelled() => return PumpExitReason::LifecycleShutdown,
changed = activity.changed() => {
if changed.is_err() { return PumpExitReason::PumpStopped; }
pending_ping = false;
continue;
}
() = &mut pong_deadline, if pending_ping => return PumpExitReason::PongTimeout,
_ = heartbeat.tick() => {}
}
ping_sequence = ping_sequence.wrapping_add(1);
let write = tokio::time::timeout(
DOWNSTREAM_PING_WRITE_TIMEOUT,
write_heartbeat(&writer, ping_sequence, context),
);
tokio::pin!(write);
let mut received_during_write = false;
loop {
tokio::select! {
biased;
() = context.cancellation.cancelled() => return PumpExitReason::LifecycleShutdown,
changed = activity.changed() => {
if changed.is_err() { return PumpExitReason::PumpStopped; }
pending_ping = false;
received_during_write = true;
}
() = &mut pong_deadline, if pending_ping => return PumpExitReason::PongTimeout,
result = &mut write => {
match result {
Ok(Ok(true)) => {
context.stats.ping_written_count.fetch_add(1, Ordering::Relaxed);
// 重复 Ping 不顺延截止,写入完成前已经收到的响应也不能丢失
if !pending_ping && !received_during_write {
pong_deadline.as_mut().reset(Instant::now() + DOWNSTREAM_PONG_TIMEOUT);
pending_ping = true;
}
}
Ok(Ok(false)) => {}
Ok(Err(reason)) => return reason,
Err(_) => return PumpExitReason::WriteTimeout,
}
break;
}
}
}
}
}
async fn write_heartbeat<S, E>(
writer: &Arc<middleware::Writer<S>>,
sequence: u64,
context: &PumpContext,
) -> Result<bool, PumpExitReason>
where
S: Sink<Message, Error = E> + Unpin + Send + 'static,
E: fmt::Display + Send + 'static,
{
let message = middleware::transform(
Message::Ping(sequence.to_be_bytes().to_vec().into()),
contract::Direction::Outgoing,
writer.clone(),
context,
)
.await
.map_err(|_| PumpExitReason::MiddlewareFailed)?;
let Some(message) = message else {
return Ok(false);
};
let is_ping = matches!(message, Message::Ping(_));
writer.write(message).await.map_err(|error| {
tracing::info!(websocket_connection_id = %context.connection_id, error = %error, "Responses WebSocket control write failed");
write_exit_reason(&error, context)
})?;
Ok(is_ping)
}
fn write_exit_reason(error: &ConnectionWriteError, context: &PumpContext) -> PumpExitReason {
match error {
ConnectionWriteError::Timeout { .. } => PumpExitReason::WriteTimeout,
@@ -181,13 +181,15 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
"Responses WebSocket connected"
);
let _connection_guard = connection_guard;
let cancellation = lifecycle.cancellation();
let cancellation = lifecycle.cancellation().child_token();
let _connection_cancellation =
crate::middleware::websocket::CancelOnDrop(Some(cancellation.clone()));
let request_headers =
request_headers.with_downstream_websocket_connection_id(connection_id.clone());
let mut connection = ResponsesWebSocketConnection::new(
socket,
connection_id,
cancellation,
cancellation.clone(),
middleware,
encode_headers(&raw_headers).into(),
);
@@ -299,7 +301,11 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
}
execution.prepare_execution(client).await
};
let prepared = match preparation.await {
let prepared = match tokio::select! {
biased;
_ = connection.wait_for_exit() => break,
prepared = preparation => prepared,
} {
Ok(prepared) => prepared,
Err(error) => {
trace_rejected_request(
@@ -317,6 +323,7 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
continue;
}
};
let prepared = prepared.with_connection_cancellation(cancellation.clone());
let request_id = Arc::<str>::from(prepared.request_id().to_string());
response_control = prepared.response_control();
let capture = new_replay_capture();
@@ -7,6 +7,7 @@ use axum::{
};
use super::{
chat_completions::chat_completions,
images::{image_edits, image_generations},
live,
models::{model_detail, models},
@@ -19,6 +20,7 @@ use crate::ApiState;
// 公开协议路径由 API 层统一定义;Core 使用操作类型,不根据这些路径分派业务
pub(super) const RESPONSES_PATH: &str = "/v1/responses";
pub(super) const CHAT_COMPLETIONS_PATH: &str = "/v1/chat/completions";
pub(super) const IMAGE_GENERATIONS_PATH: &str = "/v1/images/generations";
pub(super) const IMAGE_EDITS_PATH: &str = "/v1/images/edits";
pub(super) const SEARCH_PATH: &str = "/v1/alpha/search";
@@ -33,6 +35,7 @@ pub(crate) fn router() -> Router<ApiState> {
.route(IMAGE_EDITS_PATH, post(image_edits))
.route(SEARCH_PATH, post(standalone_search))
.route(RESPONSES_PATH, get(responses_websocket).post(responses))
.route(CHAT_COMPLETIONS_PATH, post(chat_completions))
.route(MODELS_PATH, get(models))
// 官方 OpenAI 模型详情合同使用 path ID;它不属于 Admin API 约束
.route(MODEL_DETAIL_PATH, get(model_detail))
@@ -91,17 +91,17 @@ async fn account_cooldown_returns_an_instant_and_server_formatted_recovery() {
),
(
Some(7200),
AccountCooldownKind::CapacityFreeze,
AccountCooldownKind::ErrorProtection,
Some("剩余 2 小时"),
),
(
Some(7500),
AccountCooldownKind::CapacityFreezeProbe,
AccountCooldownKind::ErrorProtectionProbe,
Some("剩余 2 小时 5 分"),
),
(
Some(-60),
AccountCooldownKind::CapacityFreezeProbe,
AccountCooldownKind::ErrorProtectionProbe,
Some("等待探测成功"),
),
(Some(20), AccountCooldownKind::RateLimit, None),
@@ -121,7 +121,14 @@ async fn account_cooldown_returns_an_instant_and_server_formatted_recovery() {
last_error_reason: None,
last_error_message: None,
};
*fixture.account.lock().unwrap() = Some(account_page_item(facts, now, "oauth"));
let mut item = account_page_item(facts, now, "oauth");
if kind.is_error_protection() {
item.protection_rules = vec![gateway_core::account::AccountProtectionMatch {
id: "removed-rule".to_owned(),
name: "触发时的规则名称".to_owned(),
}];
}
*fixture.account.lock().unwrap() = Some(item);
let response = admin::accounts::router::<AdminTestState>()
.with_state(fixture.state())
.oneshot(
@@ -138,6 +145,17 @@ async fn account_cooldown_returns_an_instant_and_server_formatted_recovery() {
let body = to_bytes(response.into_body(), 32768).await.unwrap();
let value: serde_json::Value = serde_json::from_slice(&body).unwrap();
let quota = &value["data"]["items"][0]["quota"];
if kind.is_error_protection() {
assert_eq!(quota["rateLimitReason"], "error_protection");
assert_eq!(
quota["protectionRules"],
serde_json::json!([
{"id":"removed-rule", "name":"触发时的规则名称"}
])
);
} else {
assert_eq!(quota["protectionRules"], serde_json::json!([]));
}
match until {
Some(until) => {
assert_eq!(
@@ -256,6 +274,7 @@ fn account_page_item(
use gateway_core::account::resolve_account_status;
AccountPageItem {
protection_rules: Vec::new(),
account: AccountRecord {
outbound_proxy_id: None,
outbound_proxy_enabled: true,
@@ -506,6 +506,7 @@ mod provider {
last_error_message: None,
};
AccountPageItem {
protection_rules: Vec::new(),
capacity: gateway_admin::model::accounts::AccountCapacity {
used_slots: None,
total_slots: None,
+6 -30
View File
@@ -647,13 +647,7 @@ impl SettingsStore for MemorySettingsStore {
usage_retention_days: command.values.usage_retention_days,
ops_event_retention_days: command.values.ops_event_retention_days,
audit_retention_days: command.values.audit_retention_days,
account_auto_freeze_enabled: true,
account_auto_freeze_threshold: 12,
account_auto_freeze_window_seconds: 600,
account_auto_freeze_duration_seconds: 7_200,
account_auto_freeze_probe_enabled: true,
account_auto_freeze_probe_model: None,
account_auto_freeze_adaptive_concurrency: true,
account_error_protection: command.values.account_error_protection,
account_warmup_enabled: false,
account_warmup_schedule_time: "08:00".to_owned(),
account_warmup_model: None,
@@ -697,6 +691,7 @@ pub(super) struct MemoryClientKeyStore(Arc<Mutex<Option<ClientKeyRecord>>>);
pub(super) struct MemoryObservations {
pub summary: Option<UsageOverview>,
pub trend: Option<Vec<RequestMetricPoint>>,
pub concurrency: Vec<gateway_admin::model::observability::UsageConcurrencyPoint>,
pub summaries: Vec<(TimeRange, UsageFilter)>,
pub trends: Vec<(TimeRange, UsageFilter)>,
pub records: Vec<UsageQuery>,
@@ -1326,15 +1321,6 @@ impl AccountStore for UnusedStore {
Err(unavailable("account enabled"))
}
async fn lower_concurrency_limit(
&self,
_: &gateway_core::account::ProviderAccountId,
_: gateway_core::account::AccountConcurrencyLimit,
_: &MutationContext,
) -> AdminStoreResult<Option<gateway_admin::model::accounts::AccountUpdateResult>> {
Ok(None)
}
async fn recover_account(
&self,
_: &ProviderAccountId,
@@ -1399,6 +1385,7 @@ impl AccountRuntimeStore for UnusedStore {
async fn active_rate_limits(&self) -> AdminStoreResult<AccountRuntimeSnapshot> {
Ok(AccountRuntimeSnapshot {
cooldown: BTreeMap::new(),
protection_rules: BTreeMap::new(),
in_flight: Some(BTreeMap::new()),
})
}
@@ -1406,6 +1393,7 @@ impl AccountRuntimeStore for UnusedStore {
async fn account_runtime(&self, _: &[String]) -> AdminStoreResult<AccountRuntimeSnapshot> {
Ok(AccountRuntimeSnapshot {
cooldown: BTreeMap::new(),
protection_rules: BTreeMap::new(),
in_flight: Some(BTreeMap::new()),
})
}
@@ -1416,13 +1404,6 @@ impl AccountRuntimeStore for UnusedStore {
Ok(BTreeMap::new())
}
async fn capacity_peaks(
&self,
_account_ids: &[String],
) -> AdminStoreResult<BTreeMap<String, u32>> {
Ok(BTreeMap::new())
}
async fn finish_freeze(
&self,
_account_id: &str,
@@ -1526,6 +1507,7 @@ impl ObservabilityStore for UnusedStore {
.clone()
.ok_or_else(|| unavailable("usage trend"))?,
costs: Vec::new(),
concurrency: data.concurrency.clone(),
},
)
}
@@ -1841,13 +1823,7 @@ fn test_runtime_settings() -> RuntimeSettings {
usage_retention_days: 31,
ops_event_retention_days: 30,
audit_retention_days: 90,
account_auto_freeze_enabled: true,
account_auto_freeze_threshold: 12,
account_auto_freeze_window_seconds: 600,
account_auto_freeze_duration_seconds: 7_200,
account_auto_freeze_probe_enabled: true,
account_auto_freeze_probe_model: None,
account_auto_freeze_adaptive_concurrency: true,
account_error_protection: Default::default(),
account_warmup_enabled: false,
account_warmup_schedule_time: "08:00".to_owned(),
account_warmup_model: None,
@@ -1286,6 +1286,102 @@ async fn calendar_query_rejects_an_anchor_outside_the_local_calendar() {
assert!(fixture.dashboard_summary_range.lock().unwrap().is_none());
}
#[tokio::test]
async fn usage_insights_maps_load_summary_and_points_to_camel_case() {
use crate::admin::{AdminTestFixture, AdminTestState};
use axum::{
body::{Body, to_bytes},
http::{Request, StatusCode, header},
};
use chrono::Duration;
use gateway_admin::model::observability::{
Granularity, LatencyPercentiles, PercentileMilliseconds, RequestMetricPoint,
RequestMetrics, TimeRange, UsageConcurrencyPoint, UsageOverview,
};
use gateway_api::admin::observability;
use tower::ServiceExt as _;
let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session");
let now = Utc::now();
let range = TimeRange::new(now - Duration::minutes(15), now).unwrap();
let bucket = range.start;
{
let mut observations = fixture.observations.lock().unwrap();
observations.summary = Some(UsageOverview {
analytics: Default::default(),
range,
requests: RequestMetrics {
admission_decision_percentiles: LatencyPercentiles {
p95_ms: Some(PercentileMilliseconds::new(12.5).unwrap()),
..Default::default()
},
account_selection_wait_percentiles: LatencyPercentiles {
p95_ms: Some(PercentileMilliseconds::new(128.75).unwrap()),
..Default::default()
},
..Default::default()
},
attempts: Default::default(),
providers: Vec::new(),
});
observations.trend = Some(vec![RequestMetricPoint {
bucket_start: bucket,
granularity: Granularity::FifteenMinutes,
metrics: RequestMetrics {
admission_decision_percentiles: LatencyPercentiles {
p95_ms: Some(PercentileMilliseconds::new(2.5).unwrap()),
..Default::default()
},
..Default::default()
},
cost_coverage: Default::default(),
costs: Vec::new(),
}]);
observations.concurrency = vec![UsageConcurrencyPoint {
bucket_start: bucket,
average_concurrency: 1.25,
peak_concurrency: 3,
}];
}
let response = observability::router::<AdminTestState>()
.with_state(fixture.state())
.oneshot(
Request::builder()
.uri("/api/admin/usage/insights/overview")
.header(header::COOKIE, "cpr_session=valid-session")
.header("x-request-id", "req_usage_insights_load")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
let status = response.status();
let body = to_bytes(response.into_body(), 1024 * 1024).await.unwrap();
let value: serde_json::Value = serde_json::from_slice(&body).unwrap();
assert_eq!(status, StatusCode::OK, "{value}");
let performance = &value["data"]["performance"];
assert_eq!(
performance["load"],
json!({
"averageConcurrency": 1.25,
"peakConcurrency": 3,
"admissionDecisionP95Ms": 12.5,
"accountSelectionWaitP95Ms": 128.75,
})
);
assert_eq!(
performance["points"][0]["load"],
json!({
"averageConcurrency": 1.25,
"peakConcurrency": 3,
"admissionDecisionP95Ms": 2.5,
"accountSelectionWaitP95Ms": null,
})
);
}
#[tokio::test]
async fn observability_queries_reject_group_filters_and_analytics_search() {
use crate::admin::{AdminTestFixture, AdminTestState};
@@ -1511,6 +1607,7 @@ async fn dashboard_summary_should_identify_metrics_without_localized_labels() {
authentication_kind: "oauth".to_owned(),
name: account_id.clone(),
email: None,
notes: None,
plan_type: None,
plan_type_display: None,
request_count: 10,
@@ -1709,6 +1806,7 @@ async fn dashboard_relative_time_preserves_server_anchor_boundaries_and_deployme
authentication_kind: "oauth".to_owned(),
name: "Relative time fixture".to_owned(),
email: None,
notes: None,
plan_type: None,
plan_type_display: None,
request_count: 1,
@@ -65,13 +65,7 @@ fn update_body() -> Value {
"usageRetentionDays": 31,
"opsEventRetentionDays": 31,
"auditRetentionDays": 91,
"accountAutoFreezeEnabled": true,
"accountAutoFreezeThreshold": 12,
"accountAutoFreezeWindowSeconds": 600,
"accountAutoFreezeDurationSeconds": 7200,
"accountAutoFreezeProbeEnabled": true,
"accountAutoFreezeProbeModel": null,
"accountAutoFreezeAdaptiveConcurrency": true,
"accountErrorProtection": gateway_core::account::AccountErrorProtection::default(),
"accountWarmupEnabled": false,
"accountWarmupScheduleTime": "08:00",
"accountWarmupModel": null,
@@ -185,6 +179,87 @@ fn privacy_policy_is_required_and_rejects_unknown_contract_fields() {
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(unknown).is_err());
}
#[tokio::test]
async fn error_protection_rules_round_trip_and_invalid_policy_does_not_replace_them() {
let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session");
let mut body = update_body();
let policy = json!({
"enabled": true,
"rules": [{
"id": "custom-rejection", "name": "自定义上游拒绝",
"conditions": [
{"field": "provider", "operator": "equals", "values": ["openai", "xai"], "ignoreCase": false},
{"field": "code", "operator": "equals", "values": ["custom_vendor_rejection", null], "ignoreCase": true},
{"field": "message", "operator": "contains", "values": ["please pause"], "ignoreCase": false}
]
}],
"threshold": 3, "windowSeconds": 120, "durationSeconds": 300,
"probeEnabled": false, "probeModel": null
});
body["accountErrorProtection"] = policy.clone();
let response = app(fixture.state())
.oneshot(request(
Method::POST,
"/api/admin/settings/update",
Some(body.clone()),
))
.await
.unwrap();
assert_eq!(response.status(), StatusCode::OK);
let saved = response_json(response).await;
assert_eq!(saved["data"]["accountErrorProtection"], policy);
body["configRevision"] = saved["data"]["configRevision"].clone();
let mut invalid_policies = Vec::new();
let mut empty = policy.clone();
empty["rules"] = json!([]);
invalid_policies.push(empty);
let mut provider_only = policy.clone();
provider_only["rules"][0]["conditions"] = json!([
{"field": "provider", "operator": "equals", "values": ["openai"], "ignoreCase": false}
]);
invalid_policies.push(provider_only);
let mut empty_contains = policy.clone();
empty_contains["rules"][0]["conditions"][2]["values"] = json!([" "]);
invalid_policies.push(empty_contains);
for invalid in invalid_policies {
body["accountErrorProtection"] = invalid;
let response = app(fixture.state())
.oneshot(request(
Method::POST,
"/api/admin/settings/update",
Some(body.clone()),
))
.await
.unwrap();
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
}
let response = app(fixture.state())
.oneshot(request(Method::GET, "/api/admin/settings", None))
.await
.unwrap();
let loaded = response_json(response).await;
assert_eq!(loaded["data"]["accountErrorProtection"], policy);
assert_eq!(
loaded["data"]["configRevision"],
saved["data"]["configRevision"]
);
}
#[test]
fn error_protection_requires_an_explicit_policy_in_settings_replacement() {
let mut missing = update_body();
missing
.as_object_mut()
.unwrap()
.remove("accountErrorProtection");
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(missing).is_err());
let mut unknown = update_body();
unknown["accountErrorProtection"]["unknownField"] = json!(true);
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(unknown).is_err());
}
#[tokio::test]
async fn smart_settings_round_trip_and_invalid_updates_leave_the_saved_value_intact() {
let fixture = AdminTestFixture::new().await;
@@ -400,13 +475,7 @@ fn settings_response_should_cover_the_full_runtime_settings_contract() {
usage_retention_days: 31,
ops_event_retention_days: 31,
audit_retention_days: 91,
account_auto_freeze_enabled: true,
account_auto_freeze_threshold: 12,
account_auto_freeze_window_seconds: 600,
account_auto_freeze_duration_seconds: 7_200,
account_auto_freeze_probe_enabled: true,
account_auto_freeze_probe_model: None,
account_auto_freeze_adaptive_concurrency: true,
account_error_protection: Default::default(),
account_warmup_enabled: false,
account_warmup_schedule_time: "08:00".to_owned(),
account_warmup_model: None,
@@ -453,13 +522,7 @@ fn settings_response_should_cover_the_full_runtime_settings_contract() {
"usageRetentionDays": 31,
"opsEventRetentionDays": 31,
"auditRetentionDays": 91,
"accountAutoFreezeEnabled": true,
"accountAutoFreezeThreshold": 12,
"accountAutoFreezeWindowSeconds": 600,
"accountAutoFreezeDurationSeconds": 7200,
"accountAutoFreezeProbeEnabled": true,
"accountAutoFreezeProbeModel": null,
"accountAutoFreezeAdaptiveConcurrency": true,
"accountErrorProtection": gateway_core::account::AccountErrorProtection::default(),
"accountWarmupEnabled": false,
"accountWarmupScheduleTime": "08:00",
"accountWarmupModel": null,
+60 -22
View File
@@ -229,29 +229,67 @@ fn desktop_headers_should_take_precedence_over_embedded_cli_marker() {
}
#[test]
fn cli_user_agent_should_expose_semver_or_recognized_missing_version() {
let mut headers = HeaderMap::new();
headers.insert(
"user-agent",
HeaderValue::from_static("codex_cli_rs/0.39.0 (Linux; x86_64)"),
);
let identified = identify_codex_client(&headers).expect("recognized CLI");
assert_eq!(identified.kind(), CodexClientKind::Cli);
assert_eq!(
identified.version().map(ToString::to_string).as_deref(),
Some("0.39.0")
);
fn official_core_clients_should_use_the_core_version_instead_of_the_app_suffix() {
for product in [
"codex_cli_rs",
"codex-cli",
"codex-tui",
"codex_exec",
"codex_vscode",
"codex_sdk_ts",
"codex_python_sdk",
"codex-app-server",
"codex-app-server-sdk",
"codex_mcp_server",
"CoDeX-TuI",
] {
for (version, expected) in [
(Some("0.158.0"), Some("0.158.0")),
(
Some("0.165.0-alpha.1+build.2"),
Some("0.165.0-alpha.1+build.2"),
),
(Some("not-a-version"), None),
(Some(""), None),
(None, None),
] {
let prefix = version.map_or_else(|| product.to_owned(), |v| format!("{product}/{v}"));
let mut headers = HeaderMap::new();
headers.insert(
"user-agent",
HeaderValue::from_str(&format!(
"{prefix} (Mac OS 26.6.2; arm64) Apple_Terminal ({product}; 0.165.0)"
))
.unwrap(),
);
let identified = identify_codex_client(&headers).expect("recognized official client");
assert_eq!(identified.kind(), CodexClientKind::Cli, "{prefix}");
assert_eq!(
identified.version().map(ToString::to_string).as_deref(),
expected,
"{prefix}"
);
}
}
}
headers.insert(
"user-agent",
HeaderValue::from_static("codex-cli/not-a-version"),
);
assert!(
identify_codex_client(&headers)
.expect("recognized invalid CLI")
.version()
.is_none()
);
#[test]
fn unknown_codex_like_products_should_remain_unrestricted() {
for product in [
"custom-codex-tui",
"codex-tui-custom",
"codex_cli_rs_custom",
"codex_python_sdk_custom",
"my_app",
"deepseek-harness",
] {
let mut headers = HeaderMap::new();
headers.insert(
"user-agent",
HeaderValue::from_str(&format!("{product}/0.158.0 (Linux; x86_64)")).unwrap(),
);
assert_eq!(identify_codex_client(&headers), None, "{product}");
}
}
#[test]
@@ -0,0 +1,624 @@
//! Chat HTTP 夹具和共享执行、鉴权、插件及终结边界
use std::{
collections::VecDeque,
io::Write as _,
sync::{
Arc, Mutex,
atomic::{AtomicBool, AtomicUsize, Ordering},
},
time::Duration,
};
use axum::{
body::{Body, to_bytes},
http::{Request, StatusCode},
response::Response,
};
use bytes::Bytes;
use futures::future::BoxFuture;
use gateway_core::{
engine::{
CommitRequirement, CoordinatedEvent, EngineError, ModelRequestId,
execution::{
AuthenticatedClient, ClientAuthenticationError, ExecutionService, ExecutionSession,
PreparedRootExecution, ResponseCommit, StartExecution, StartProviderExecution,
StartedExecution,
},
middleware::{
FrozenMiddlewarePlan, MiddlewareContext, MiddlewareError, MiddlewareNext,
MiddlewarePlan, MiddlewareRequest, MiddlewareResponse,
},
},
error::GatewayError,
event::{ProviderEvent, ProviderResponseHeader},
operation::Operation,
routing::{
PublicModelId,
extensions::{ExtensionSetId, ExtensionSetLease, ExtensionSetReference},
},
};
use serde_json::{Value, json};
use tower::ServiceExt as _;
use super::{chat_request, completed, created, message};
use crate::openai::{
api_router, authenticated_client, authenticated_client_for_provider_with_limit,
middleware::{RequestMiddleware, ResponseFrameAction},
};
pub(super) const PATH: &str = "/v1/chat/completions";
const KEY: &str = "sk_chat_fixture";
#[derive(Default)]
pub(super) struct Trace {
pub commits: AtomicUsize,
pub discarded: AtomicUsize,
pub cancelled: AtomicBool,
pub finalized: AtomicBool,
pub detached: AtomicUsize,
response_commit: ResponseCommit,
}
#[derive(Default)]
pub(super) enum Finish {
#[default]
Success,
Error,
Pending,
}
pub(super) struct Session {
events: VecDeque<Vec<ProviderEvent>>,
pub trace: Arc<Trace>,
pub finish: Finish,
pub delay: Option<Duration>,
pub fail_commit: bool,
pub response_headers: Vec<ProviderResponseHeader>,
committed: bool,
buffered: bool,
}
impl Session {
pub fn new(events: Vec<ProviderEvent>) -> Self {
Self {
events: events.into_iter().map(|event| vec![event]).collect(),
trace: Arc::default(),
finish: Finish::Success,
delay: None,
fail_commit: false,
response_headers: Vec::new(),
committed: false,
buffered: false,
}
}
pub fn with_batch(mut self, events: Vec<ProviderEvent>) -> Self {
self.events.push_back(events);
self
}
}
impl ExecutionSession for Session {
fn response_commit(&self) -> ResponseCommit {
self.trace.response_commit.clone()
}
fn next_event(&mut self) -> BoxFuture<'_, Result<Option<CoordinatedEvent>, EngineError>> {
Box::pin(async move {
if let Some(delay) = self.delay.take() {
tokio::time::sleep(delay).await;
}
if let Some(event) = self.events.pop_front() {
let requirement = if self.committed {
CommitRequirement::AlreadyCommitted
} else {
CommitRequirement::CommitBeforeDelivery
};
return CoordinatedEvent::try_batch(event, requirement).map(Some);
}
match self.finish {
Finish::Success => {
self.trace.finalized.store(true, Ordering::SeqCst);
Ok(None)
}
Finish::Error => {
self.trace.finalized.store(true, Ordering::SeqCst);
Err(EngineError::ProviderMetadataMismatch)
}
Finish::Pending => futures::future::pending().await,
}
})
}
fn collect_uncommitted(&mut self) -> BoxFuture<'_, Result<Vec<ProviderEvent>, EngineError>> {
self.buffered = true;
Box::pin(async move { Ok(self.events.drain(..).flatten().collect()) })
}
fn response_headers(&self) -> &[ProviderResponseHeader] {
&self.response_headers
}
fn discard_pending_delivery(&mut self) -> Result<(), EngineError> {
self.trace.discarded.fetch_add(1, Ordering::SeqCst);
Ok(())
}
fn commit_downstream(&mut self, status: Option<u16>) -> BoxFuture<'_, Result<(), EngineError>> {
Box::pin(async move {
self.trace.commits.fetch_add(1, Ordering::SeqCst);
if self.fail_commit {
return Err(EngineError::ProviderMetadataMismatch);
}
self.trace.response_commit.record(status);
self.committed = true;
if self.buffered {
self.trace.finalized.store(true, Ordering::SeqCst);
}
Ok(())
})
}
fn record_client_status(&mut self, status: u16) -> BoxFuture<'_, Result<(), EngineError>> {
Box::pin(async move {
self.trace.response_commit.record(Some(status));
Ok(())
})
}
fn is_finalized(&self) -> bool {
self.trace.finalized.load(Ordering::SeqCst)
}
fn cancel(&self) {
self.trace.cancelled.store(true, Ordering::SeqCst);
}
fn detach_finalize(self: Box<Self>) -> BoxFuture<'static, ()> {
Box::pin(async move {
self.trace.detached.fetch_add(1, Ordering::SeqCst);
self.trace.finalized.store(true, Ordering::SeqCst);
})
}
}
#[derive(Clone, Debug)]
pub(super) struct Captured {
pub body: Value,
pub context: Value,
pub endpoint: String,
}
pub(super) struct Execution {
pub captured: Mutex<Option<Captured>>,
pub client: AuthenticatedClient,
pub middleware: Option<FrozenMiddlewarePlan>,
session: Mutex<Option<Session>>,
}
impl Execution {
pub fn new(session: Session) -> Self {
Self {
captured: Mutex::default(),
client: authenticated_client(KEY),
middleware: None,
session: Mutex::new(Some(session)),
}
}
}
impl ExecutionService for Execution {
fn authenticate(&self, key: &str) -> Result<AuthenticatedClient, ClientAuthenticationError> {
if key == KEY {
Ok(self.client.clone())
} else {
Err(ClientAuthenticationError::InvalidKey)
}
}
fn public_models(&self, _: &AuthenticatedClient) -> Vec<PublicModelId> {
Vec::new()
}
fn contains_public_model(&self, _: &AuthenticatedClient, _: &PublicModelId) -> bool {
true
}
fn middleware_plan(&self, _: &PreparedRootExecution) -> Option<FrozenMiddlewarePlan> {
self.middleware.clone()
}
fn start(
&self,
request: StartExecution,
) -> BoxFuture<'_, Result<StartedExecution, GatewayError>> {
Box::pin(async move {
let Operation::Generate(generation) = request.operation else {
panic!("Chat must generate")
};
*self.captured.lock().unwrap() = Some(Captured {
body: Value::Object(generation.protocol_payload().body().clone()),
context: Value::Object(generation.protocol_payload().context().clone()),
endpoint: request.metadata.endpoint,
});
Ok(StartedExecution {
request_id: ModelRequestId::new("req_chat_fixture").unwrap(),
created_at: std::time::SystemTime::now(),
stream: request.metadata.stream,
session: Box::new(self.session.lock().unwrap().take().expect("one request")),
})
})
}
fn start_provider_endpoint(
&self,
_: StartProviderExecution,
) -> BoxFuture<'_, Result<StartedExecution, GatewayError>> {
Box::pin(async { panic!("Chat must use model execution") })
}
}
pub(super) struct Fixture {
pub router: axum::Router,
pub execution: Arc<Execution>,
pub trace: Arc<Trace>,
}
impl Fixture {
pub async fn new(session: Session) -> Self {
Self::with_execution(Execution::new(session)).await
}
pub async fn with_execution(execution: Execution) -> Self {
let trace = execution
.session
.lock()
.unwrap()
.as_ref()
.unwrap()
.trace
.clone();
let execution = Arc::new(execution);
Self {
router: api_router(execution.clone()).await,
execution,
trace,
}
}
pub async fn post(&self, body: Value) -> Response {
self.router.clone().oneshot(request(body)).await.unwrap()
}
pub fn captured(&self) -> Captured {
self.execution
.captured
.lock()
.unwrap()
.clone()
.expect("started request")
}
}
pub(super) fn request(body: Value) -> Request<Body> {
Request::post(PATH)
.header("authorization", format!("Bearer {KEY}"))
.header("content-type", "application/json")
.body(Body::from(body.to_string()))
.unwrap()
}
pub(super) async fn json_body(response: Response) -> Value {
serde_json::from_slice(&to_bytes(response.into_body(), usize::MAX).await.unwrap()).unwrap()
}
#[derive(Debug, Default)]
pub(super) struct InspectMiddleware {
pub inner: RequestMiddleware,
pub requests: Mutex<Vec<Value>>,
pub update: Option<fn(&mut Value)>,
}
impl InspectMiddleware {
pub fn frozen(self: &Arc<Self>) -> FrozenMiddlewarePlan {
FrozenMiddlewarePlan::new(
self.clone(),
ExtensionSetReference::new(
ExtensionSetId::new("chat-inspector".to_owned()).unwrap(),
Arc::new(ReadyLease),
),
)
}
}
struct ReadyLease;
impl ExtensionSetLease for ReadyLease {
fn is_ready(&self) -> bool {
true
}
}
impl MiddlewarePlan for InspectMiddleware {
fn handle(
&self,
context: MiddlewareContext,
mut request: MiddlewareRequest,
next: MiddlewareNext,
) -> BoxFuture<'static, Result<MiddlewareResponse, MiddlewareError>> {
let mut body: Value = serde_json::from_slice(request.body()).unwrap();
self.requests.lock().unwrap().push(body.clone());
if let Some(update) = self.update {
update(&mut body);
let headers = request.headers().to_vec();
request = request
.replace_parts(
"openai".to_owned(),
headers,
Bytes::from(body.to_string()),
None,
)
.unwrap();
}
self.inner.handle(context, request, next)
}
}
#[tokio::test]
async fn rejects_missing_authentication_before_decoding_or_execution() {
let fixture = Fixture::new(Session::new(Vec::new())).await;
for authorization in [None, Some("Bearer sk_invalid")] {
let mut request = Request::post(PATH);
if let Some(authorization) = authorization {
request = request.header("authorization", authorization);
}
let response = fixture
.router
.clone()
.oneshot(request.body(Body::from("broken json")).unwrap())
.await
.unwrap();
assert_eq!(response.status(), StatusCode::UNAUTHORIZED);
}
assert!(fixture.execution.captured.lock().unwrap().is_none());
}
#[tokio::test]
async fn request_middleware_sees_responses_and_preserves_the_chat_endpoint() {
let middleware = Arc::new(InspectMiddleware {
inner: RequestMiddleware {
expected_operation: Some(gateway_core::operation::OperationKind::Generate),
..RequestMiddleware::default()
},
update: Some(|body| {
body["input"] = json!("rewritten by request middleware");
}),
..InspectMiddleware::default()
});
let mut execution = Execution::new(Session::new(vec![completed(json!([message("ok")]))]));
execution.middleware = Some(middleware.frozen());
let fixture = Fixture::with_execution(execution).await;
let response = fixture.post(chat_request(false)).await;
assert_eq!(response.status(), StatusCode::OK);
let downstream = json_body(response).await;
assert_eq!(downstream["object"], "chat.completion");
let observed = middleware.requests.lock().unwrap();
assert!(observed[0].get("messages").is_none());
assert_eq!(observed[0]["input"][0]["role"], "user");
assert_eq!(
middleware.inner.endpoints.lock().unwrap().as_slice(),
[PATH]
);
let captured = fixture.captured();
assert_eq!(captured.endpoint, PATH);
assert_eq!(captured.body["input"], "rewritten by request middleware");
assert_eq!(fixture.trace.commits.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn dropping_chat_body_cancels_and_finalizes_the_original_execution() {
let mut session = Session::new(vec![created()]);
session.finish = Finish::Pending;
let fixture = Fixture::new(session).await;
let response = fixture.post(chat_request(true)).await;
assert_eq!(response.status(), StatusCode::OK);
drop(response);
for _ in 0..8 {
tokio::task::yield_now().await;
}
assert!(fixture.trace.cancelled.load(Ordering::SeqCst));
assert_eq!(fixture.trace.detached.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn gzip_and_headers_reuse_the_responses_ingress_contract() {
let mut session = Session::new(vec![completed(json!([message("ok")]))]);
session.response_headers = vec![
ProviderResponseHeader::new("x-future-multi", Bytes::from_static(b"first")),
ProviderResponseHeader::new("x-future-multi", Bytes::from_static(b"second")),
ProviderResponseHeader::new("connection", Bytes::from_static(b"x-hop-only")),
ProviderResponseHeader::new("x-hop-only", Bytes::from_static(b"removed")),
];
let fixture = Fixture::new(session).await;
let body = chat_request(false).to_string();
let mut encoder = flate2::write::GzEncoder::new(Vec::new(), flate2::Compression::fast());
encoder.write_all(body.as_bytes()).unwrap();
let compressed = encoder.finish().unwrap();
let request = Request::post(PATH)
.header("authorization", format!("Bearer {KEY}"))
.header("content-type", "application/json")
.header("content-encoding", "gzip")
.header("content-length", compressed.len())
.header("session_id", "chat-session")
.header("x-future-input", "first")
.header("x-future-input", "second")
.body(Body::from(compressed))
.unwrap();
let response = fixture.router.clone().oneshot(request).await.unwrap();
assert_eq!(response.status(), StatusCode::OK);
assert_eq!(
response
.headers()
.get_all("x-future-multi")
.iter()
.map(|value| value.to_str().unwrap())
.collect::<Vec<_>>(),
["first", "second"]
);
assert!(!response.headers().contains_key("x-hop-only"));
assert!(!response.headers().contains_key("content-encoding"));
assert_eq!(
json_body(response).await["choices"][0]["message"]["content"],
"ok"
);
let captured = fixture.captured();
assert_eq!(captured.context["session_id"], "chat-session");
let headers = captured.context["opaque_request_headers"]
.as_array()
.unwrap();
assert_eq!(
headers
.iter()
.filter(|header| header[0] == "x-future-input")
.count(),
2
);
assert!(!headers.iter().any(|header| matches!(
header[0].as_str(),
Some("content-encoding" | "content-length" | "authorization")
)));
}
#[tokio::test]
async fn corrupt_or_oversized_compressed_chat_never_starts_execution() {
let mut encoder = flate2::write::GzEncoder::new(Vec::new(), flate2::Compression::fast());
encoder
.write_all(chat_request(false).to_string().as_bytes())
.unwrap();
let valid_compressed = encoder.finish().unwrap();
for bytes in [b"broken gzip".to_vec(), valid_compressed] {
let mut execution = Execution::new(Session::new(Vec::new()));
execution.client = authenticated_client_for_provider_with_limit(KEY, "openai", 16);
let fixture = Fixture::with_execution(execution).await;
let request = Request::post(PATH)
.header("authorization", format!("Bearer {KEY}"))
.header("content-encoding", "gzip")
.body(Body::from(bytes))
.unwrap();
let response = fixture.router.clone().oneshot(request).await.unwrap();
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
assert!(fixture.execution.captured.lock().unwrap().is_none());
}
}
#[tokio::test]
async fn response_middleware_is_validated_as_responses_before_chat_projection() {
for matching_identity in [true, false] {
let replacement = json!({"id":if matching_identity {"resp_chat_test"} else {"resp_other"},
"status":"completed","output":[message("plugin text")]});
let middleware = Arc::new(
RequestMiddleware {
expected_operation: Some(gateway_core::operation::OperationKind::Generate),
..RequestMiddleware::default()
}
.with_response_actions(vec![ResponseFrameAction::Replace(Bytes::from(
replacement.to_string(),
))]),
);
let mut execution =
Execution::new(Session::new(vec![completed(json!([message("original")]))]));
execution.middleware = Some(middleware.frozen());
let fixture = Fixture::with_execution(execution).await;
let response = fixture.post(chat_request(false)).await;
if matching_identity {
assert_eq!(response.status(), StatusCode::OK);
assert_eq!(
json_body(response).await["choices"][0]["message"]["content"],
"plugin text"
);
} else {
assert_eq!(response.status(), StatusCode::BAD_GATEWAY);
assert_eq!(
json_body(response).await["error"]["code"],
"middleware_failed"
);
assert_eq!(fixture.trace.commits.load(Ordering::SeqCst), 0);
}
}
}
#[tokio::test]
async fn commit_failure_does_not_deliver_buffered_or_first_terminal_chat_success() {
for streaming in [false, true] {
let mut session = Session::new(vec![completed(json!([message("not delivered")]))]);
session.fail_commit = true;
let fixture = Fixture::new(session).await;
let response = fixture.post(chat_request(streaming)).await;
assert!(response.status().is_server_error());
let body = json_body(response).await;
assert!(body.get("error").is_some());
assert!(body.get("choices").is_none());
assert!(fixture.trace.cancelled.load(Ordering::SeqCst));
}
}
#[tokio::test]
async fn the_same_execution_fixture_finishes_native_responses_with_one_commit() {
for already_started in [false, true] {
let mut events = Vec::new();
if already_started {
events.push(created());
}
events.push(completed(json!([message("native baseline")])));
let fixture = Fixture::new(Session::new(events)).await;
let mut request = request(json!({"model":"model-a","input":"hello","stream":true}));
*request.uri_mut() = "/v1/responses".parse().unwrap();
let response = fixture.router.clone().oneshot(request).await.unwrap();
assert_eq!(response.status(), StatusCode::OK);
let bytes = to_bytes(response.into_body(), usize::MAX).await.unwrap();
let text = String::from_utf8(bytes.to_vec()).unwrap();
assert!(text.contains("event: response.completed"), "{text}");
assert!(text.contains("native baseline"), "{text}");
assert!(!text.contains("response.failed"), "{text}");
assert_eq!(text.matches("data: [DONE]").count(), 1, "{text}");
assert!(text.ends_with("data: [DONE]\n\n"), "{text}");
assert!(fixture.trace.finalized.load(Ordering::SeqCst));
assert_eq!(fixture.trace.commits.load(Ordering::SeqCst), 1);
assert_eq!(fixture.trace.response_commit.status_code(), Some(200));
}
}
#[tokio::test]
async fn finalization_failure_never_leaks_success_finish_or_usage() {
for already_started in [false, true] {
let mut events = Vec::new();
if already_started {
events.push(created());
}
events.push(completed(json!([])));
let mut session = Session::new(events);
session.finish = Finish::Error;
let fixture = Fixture::new(session).await;
let mut body = chat_request(true);
body["stream_options"] = json!({"include_usage":true});
let response = fixture.post(body).await;
// 原会话已经接受 200 提交,后续终结错误只能作为流内错误交付
assert_eq!(response.status(), StatusCode::OK);
let (_, chunks) = super::response::sse_body(response).await;
assert!(chunks.iter().any(|chunk| chunk.get("error").is_some()));
assert!(
!chunks
.iter()
.any(|chunk| chunk["choices"][0]["finish_reason"].is_string())
);
assert!(!chunks.iter().any(|chunk| chunk["choices"] == json!([])));
assert!(fixture.trace.finalized.load(Ordering::SeqCst));
assert_eq!(fixture.trace.commits.load(Ordering::SeqCst), 1);
assert_eq!(fixture.trace.response_commit.status_code(), Some(200));
}
}
#[tokio::test(start_paused = true)]
async fn error_after_admission_keepalive_uses_chat_error_encoding() {
let mut session = Session::new(Vec::new());
session.delay = Some(Duration::from_secs(31));
session.finish = Finish::Error;
let fixture = Fixture::new(session).await;
let response = fixture.post(chat_request(true)).await;
let (body, chunks) = super::response::sse_body(response).await;
assert!(body.starts_with(": keep-alive\n\n"));
assert!(chunks.iter().any(|chunk| chunk.get("error").is_some()));
assert!(!body.contains("response.failed"));
assert_eq!(fixture.trace.commits.load(Ordering::SeqCst), 0);
assert_eq!(fixture.trace.response_commit.status_code(), Some(200));
}
@@ -0,0 +1,45 @@
//! Chat Completions 的请求转换、输出投影与 HTTP 生命周期测试
mod http;
mod request;
mod response;
use gateway_core::event::{ProtocolWireEvent, ProviderEvent};
use serde_json::{Value, json};
fn chat_request(stream: bool) -> Value {
json!({"model":"model-a","messages":[{"role":"user","content":"hello"}],"stream":stream})
}
fn wire(kind: &str, mut value: Value) -> ProviderEvent {
value["type"] = json!(kind);
ProviderEvent::wire(ProtocolWireEvent::json("openai", Some(kind.to_owned()), value).unwrap())
}
fn completed(output: Value) -> ProviderEvent {
wire(
"response.completed",
json!({"response":{
"id":"resp_chat_test","object":"response","model":"upstream-model",
"status":"completed","output":output,
"usage":{"input_tokens":11,"output_tokens":7,"total_tokens":18,
"input_tokens_details":{"cached_tokens":3},
"output_tokens_details":{"reasoning_tokens":2}}
}}),
)
}
fn message(text: &str) -> Value {
json!({"id":"msg_chat_test","type":"message","role":"assistant","status":"completed",
"content":[{"type":"output_text","text":text,"annotations":[]}]})
}
fn created() -> ProviderEvent {
wire(
"response.created",
json!({"response":{
"id":"resp_chat_test","object":"response","model":"upstream-model",
"status":"in_progress","output":[]
}}),
)
}
@@ -0,0 +1,243 @@
//! Chat 请求映射保留消息、工具和不透明业务数据的语义
use axum::http::StatusCode;
use serde_json::{Value, json};
use super::{
chat_request, completed,
http::{Fixture, Session, json_body},
message,
};
async fn convert(body: Value) -> Value {
let fixture = Fixture::new(Session::new(vec![completed(json!([message("ok")]))])).await;
let response = fixture.post(body).await;
let status = response.status();
let result = json_body(response).await;
assert_eq!(status, StatusCode::OK, "{result}");
fixture.captured().body
}
#[tokio::test]
async fn preserves_ordered_roles_images_and_modern_tool_round_trip() {
let arguments = "{ \"city\" : \"北京\", \"store\":true }";
let converted = convert(json!({"model":"model-a","messages":[
{"role":"system","content":"system text"},
{"role":"developer","content":"developer text"},
{"role":"user","content":[{"type":"text","text":"describe"},
{"type":"image_url","image_url":{"url":"data:image/png;base64,aGVsbG8=","detail":"high"}}]},
{"role":"assistant","content":"checking","tool_calls":[
{"id":"call-weather","type":"function","function":{"name":"weather","arguments":arguments}}]},
{"role":"tool","tool_call_id":"call-weather","content":"{\"temperature\":25}"},
{"role":"user","content":"continue"}
]})).await;
let input = converted["input"].as_array().unwrap();
assert_eq!(
input
.iter()
.filter_map(|item| item["role"].as_str())
.collect::<Vec<_>>(),
["system", "developer", "user", "assistant", "user"]
);
assert_eq!(
input[2]["content"][1],
json!({"type":"input_image","image_url":"data:image/png;base64,aGVsbG8=","detail":"high"})
);
assert_eq!(
input[3]["content"],
json!([{"type":"output_text","text":"checking","annotations":[]}])
);
assert_eq!(
input[4],
json!({"type":"function_call","call_id":"call-weather","name":"weather","arguments":arguments})
);
assert_eq!(
input[5],
json!({"type":"function_call_output","call_id":"call-weather","output":"{\"temperature\":25}"})
);
assert!(converted.get("previous_response_id").is_none());
}
#[tokio::test]
async fn assistant_history_uses_output_content_for_text_and_refusals() {
for content in [
json!("previous answer"),
json!([{"type":"text","text":"previous answer"}]),
json!([{"type":"text","text":"previous answer"},{"type":"refusal","refusal":"declined"}]),
] {
let converted = convert(json!({"model":"model-a","messages":[
{"role":"user","content":"first turn"},
{"role":"assistant","content":content},
{"role":"user","content":"continue from the previous answer"}
]}))
.await;
let history = &converted["input"][1];
assert_eq!(history["role"], "assistant");
assert_eq!(history["status"], "completed");
assert!(!history["id"].as_str().unwrap().is_empty());
assert_eq!(
history["content"][0],
json!({"type":"output_text","text":"previous answer","annotations":[]})
);
if content.is_array() && content.as_array().unwrap().len() == 2 {
assert_eq!(
history["content"][1],
json!({"type":"refusal","refusal":"declined"})
);
}
assert_eq!(converted["input"][2]["content"][0]["type"], "input_text");
}
}
#[tokio::test]
async fn preserves_nested_schema_metadata_and_future_opaque_values() {
let schema = json!({"type":"object","properties":{
"messages":{"type":"string"},"store":{"type":"boolean"},
"function_call":{"type":"string"},"future":{"x-extension":[1,2]}},
"required":["messages"],"additionalProperties":true});
let mut body = chat_request(false);
body["model"] = json!("future-model-opaque:β");
body["tools"] = json!([{"type":"function","function":{"name":"inspect","parameters":schema}}]);
body["tool_choice"] = json!({"type":"function","function":{"name":"inspect"}});
body["response_format"] = json!({"type":"json_schema","json_schema":{
"name":"result","description":"preserve","strict":true,"schema":schema}});
body["metadata"] = json!({"messages":"business data","future":"kept"});
body["reasoning_effort"] = json!("future-effort");
body["verbosity"] = json!("future-verbosity");
body["service_tier"] = json!("future-tier");
body["future_client_extension"] = json!({"opaque":true});
let converted = convert(body).await;
assert_eq!(converted["model"], "future-model-opaque:β");
assert_eq!(converted["tools"][0]["parameters"], schema);
assert_eq!(converted["tools"][0]["strict"], false);
assert_eq!(
converted["tool_choice"],
json!({"type":"function","name":"inspect"})
);
assert_eq!(converted["text"]["format"]["schema"], schema);
assert_eq!(converted["text"]["format"]["strict"], true);
assert_eq!(
converted["metadata"],
json!({"messages":"business data","future":"kept"})
);
assert_eq!(converted["reasoning"]["effort"], "future-effort");
assert_eq!(converted["text"]["verbosity"], "future-verbosity");
assert_eq!(converted["service_tier"], "future-tier");
assert!(converted.get("future_client_extension").is_none());
}
#[tokio::test]
async fn normalizes_neutral_defaults_without_dropping_shared_provider_parameters() {
let mut body = chat_request(false);
for name in [
"stream",
"store",
"n",
"functions",
"function_call",
"logprobs",
"seed",
] {
body[name] = Value::Null;
}
body["frequency_penalty"] = json!(0);
body["presence_penalty"] = json!(0);
body["stop"] = json!([]);
body["max_tokens"] = json!(12);
body["max_completion_tokens"] = json!(24);
body["temperature"] = json!(0.2);
body["top_p"] = json!(0.9);
body["prompt_cache_retention"] = json!("24h");
let converted = convert(body).await;
assert_eq!(converted["stream"], false);
assert_eq!(converted["store"], false);
assert_eq!(converted["max_output_tokens"], 24);
assert_eq!(converted["temperature"], 0.2);
assert_eq!(converted["top_p"], 0.9);
assert_eq!(converted["prompt_cache_retention"], "24h");
}
#[tokio::test]
async fn refuses_known_unrepresentable_features_before_starting_execution() {
for (name, value) in [
("n", json!(2)),
("store", json!(true)),
("stop", json!(["stop"])),
("seed", json!(7)),
("frequency_penalty", json!(0.3)),
("logprobs", json!(true)),
("functions", json!([{"name":"old"}])),
("function_call", json!("auto")),
("input", json!("not chat")),
("previous_response_id", json!("resp_previous")),
("audio", json!({"voice":"alloy","format":"wav"})),
(
"prediction",
json!({"type":"content","content":"predicted"}),
),
] {
let fixture = Fixture::new(Session::new(Vec::new())).await;
let mut body = chat_request(false);
body[name] = value;
let response = fixture.post(body).await;
assert_eq!(response.status(), StatusCode::BAD_REQUEST, "{name}");
let body = json_body(response).await;
assert_eq!(body["error"]["param"], name, "{body}");
assert!(
fixture.execution.captured.lock().unwrap().is_none(),
"{name}"
);
}
}
#[tokio::test]
async fn rejects_unmapped_message_content_and_identity_fields() {
for message in [
json!({"role":"user","name":"speaker","content":"hello"}),
json!({"role":"user","content":[{"type":"file","file":{"file_id":"file-example"}}]}),
json!({"role":"assistant","content":"hello","reasoning_content":"private"}),
json!({"role":"assistant","content":[{"type":"text","text":"history","prompt_cache_breakpoint":{"type":"default"}}]}),
json!({"role":"tool","content":"unassociated output"}),
] {
let fixture = Fixture::new(Session::new(Vec::new())).await;
let response = fixture
.post(json!({"model":"model-a","messages":[message]}))
.await;
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
let body = json_body(response).await;
assert!(
body["error"]["param"]
.as_str()
.unwrap()
.starts_with("messages")
);
assert!(fixture.execution.captured.lock().unwrap().is_none());
}
}
#[tokio::test]
async fn assistant_tool_history_keeps_arguments_opaque_and_cache_annotations_structural() {
let cache = json!({"type":"default","future_setting":{"keep":true}});
let arguments = "not valid JSON {\n \"raw\" : 🦀";
let converted = convert(json!({"model":"model-a","messages":[
{"role":"user","content":[{"type":"text","text":"cached","prompt_cache_breakpoint":cache}]},
{"role":"assistant","content":null,"tool_calls":[{"id":"call-raw","type":"function",
"function":{"name":"raw","arguments":arguments}}]},
{"role":"tool","tool_call_id":"call-raw","content":[{"type":"text","text":"result"}]}
],"prompt_cache_options":{"future":{"keep":true}},"moderation":{"future":{"keep":true}}}))
.await;
assert_eq!(
converted["input"][0]["content"][0]["prompt_cache_breakpoint"],
cache
);
assert_eq!(converted["input"][1]["arguments"], arguments);
assert_eq!(
converted["input"][2]["output"],
json!([{"type":"input_text","text":"result"}])
);
assert_eq!(
converted["prompt_cache_options"],
json!({"future":{"keep":true}})
);
assert_eq!(converted["moderation"], json!({"future":{"keep":true}}));
}
@@ -0,0 +1,596 @@
//! Chat JSON 与 SSE 的真实 wire 投影、去重和协议扩展边界
use axum::{body::to_bytes, http::StatusCode, response::Response};
use serde_json::{Value, json};
use std::collections::BTreeMap;
use super::{
chat_request, completed, created,
http::{Fixture, Session, json_body},
message, wire,
};
pub(super) async fn sse_body(response: Response) -> (String, Vec<Value>) {
assert_eq!(response.status(), StatusCode::OK);
assert_eq!(response.headers()["content-type"], "text/event-stream");
let bytes = to_bytes(response.into_body(), usize::MAX).await.unwrap();
let text = String::from_utf8(bytes.to_vec()).unwrap();
assert_eq!(text.matches("data: [DONE]").count(), 1, "{text}");
assert!(text.ends_with("data: [DONE]\n\n"), "{text}");
assert!(!text.contains("event: response."), "{text}");
let chunks = text
.lines()
.filter_map(|line| line.strip_prefix("data: "))
.filter(|data| *data != "[DONE]")
.map(|data| serde_json::from_str(data).unwrap())
.collect();
(text, chunks)
}
fn combined(chunks: &[Value], name: &str) -> String {
chunks
.iter()
.filter_map(|value| value["choices"][0]["delta"][name].as_str())
.collect()
}
fn tool(id: &str, call_id: &str, name: &str, arguments: &str) -> Value {
json!({"type":"function_call","id":id,"call_id":call_id,"name":name,"arguments":arguments,"status":"completed"})
}
#[tokio::test]
async fn json_projects_public_identity_text_and_actual_usage() {
let fixture = Fixture::new(Session::new(vec![completed(json!([message("你好 🌍")]))])).await;
let response = fixture.post(chat_request(false)).await;
assert_eq!(response.status(), StatusCode::OK);
assert_eq!(
response.headers()["x-gateway-request-id"],
"req_chat_fixture"
);
let body = json_body(response).await;
assert_eq!(body["object"], "chat.completion");
assert!(body["id"].as_str().unwrap().starts_with("chatcmpl-"));
assert!(body["created"].as_u64().is_some());
assert_eq!(body["model"], "model-a");
assert_eq!(body["choices"][0]["message"]["content"], "你好 🌍");
assert_eq!(body["choices"][0]["finish_reason"], "stop");
assert_eq!(
body["usage"],
json!({"prompt_tokens":11,"completion_tokens":7,"total_tokens":18,
"prompt_tokens_details":{"cached_tokens":3},"completion_tokens_details":{"reasoning_tokens":2}})
);
}
#[tokio::test]
async fn json_tool_arguments_and_refusal_do_not_become_assistant_text() {
for output in [
json!([tool("fc1", "call1", "weather", "{ \"city\" : \"北京\" }")]),
json!([{"type":"reasoning","id":"rs1","summary":[{"type":"summary_text","text":"private"}]},
{"type":"message","id":"refusal1","role":"assistant","status":"completed",
"content":[{"type":"refusal","refusal":"cannot help"}]}]),
] {
let fixture = Fixture::new(Session::new(vec![completed(output.clone())])).await;
let response = fixture.post(chat_request(false)).await;
assert_eq!(response.status(), StatusCode::OK);
let body = json_body(response).await;
let choice = &body["choices"][0];
assert_eq!(choice["message"]["content"], Value::Null);
assert!(!body.to_string().contains("private"));
if output[0]["type"] == "function_call" {
assert_eq!(choice["finish_reason"], "tool_calls");
assert_eq!(
choice["message"]["tool_calls"][0]["function"]["arguments"],
"{ \"city\" : \"北京\" }"
);
assert_eq!(choice["message"]["tool_calls"][0]["id"], "call1");
} else {
assert_eq!(choice["message"]["refusal"], "cannot help");
}
}
}
#[tokio::test]
async fn sse_deduplicates_text_snapshots_and_emits_usage_only_when_requested() {
for include_usage in [false, true] {
let fixture = Fixture::new(Session::new(vec![created(),
wire("response.output_text.delta",json!({"item_id":"msg_chat_test","output_index":0,"content_index":0,"delta":"你好 "})),
wire("response.output_text.done",json!({"item_id":"msg_chat_test","output_index":0,"content_index":0,"text":"你好 🌍"})),
wire("response.content_part.done",json!({"item_id":"msg_chat_test","output_index":0,"content_index":0,"part":{"type":"output_text","text":"你好 🌍"}})),
wire("response.output_item.done",json!({"output_index":0,"item":message("你好 🌍")})),
completed(json!([message("你好 🌍")]))])).await;
let mut body = chat_request(true);
body["stream_options"] = json!({"include_usage":include_usage});
let (_, chunks) = sse_body(fixture.post(body).await).await;
assert_eq!(combined(&chunks, "content"), "你好 🌍");
assert_eq!(
chunks
.iter()
.filter(|chunk| chunk["choices"][0]["delta"]["role"] == "assistant")
.count(),
1
);
assert_eq!(
chunks
.iter()
.filter(|chunk| chunk["choices"][0]["finish_reason"] == "stop")
.count(),
1
);
for chunk in &chunks {
assert_eq!(chunk["object"], "chat.completion.chunk");
assert_eq!(chunk["id"], chunks[0]["id"]);
assert_eq!(chunk["created"], chunks[0]["created"]);
assert_eq!(chunk["model"], "model-a");
}
let usage: Vec<_> = chunks
.iter()
.filter(|chunk| chunk["choices"] == json!([]))
.collect();
assert_eq!(usage.len(), usize::from(include_usage));
if include_usage {
assert_eq!(usage[0]["usage"]["total_tokens"], 18);
}
let captured = fixture.captured();
assert!(captured.body.get("stream_options").is_none());
}
}
#[tokio::test]
async fn sse_preserves_interleaved_tools_and_reordered_terminal_snapshots() {
let first = tool(
"fc_first",
"call_first",
"weather",
"{ \"city\": \"北京\" }",
);
let second = tool("fc_second", "call_second", "search", "{\"query\":\"Rust\"}");
let fixture = Fixture::new(Session::new(vec![
created(),
wire(
"response.output_item.added",
json!({"output_index":2,"item":tool("fc_first","call_first","weather","")}),
),
wire(
"response.output_item.added",
json!({"output_index":5,"item":tool("fc_second","call_second","search","")}),
),
wire(
"response.function_call_arguments.delta",
json!({"output_index":2,"item_id":"fc_first","delta":"{ \"city\": "}),
),
wire(
"response.function_call_arguments.delta",
json!({"output_index":5,"item_id":"fc_second","delta":"{\"query\":"}),
),
wire(
"response.function_call_arguments.delta",
json!({"output_index":2,"item_id":"fc_first","delta":"\"北京\" }"}),
),
wire(
"response.function_call_arguments.done",
json!({"output_index":5,"item_id":"fc_second","arguments":second["arguments"]}),
),
wire(
"response.output_item.done",
json!({"output_index":2,"item":first}),
),
completed(json!([second, first])),
]))
.await;
let (_, chunks) = sse_body(fixture.post(chat_request(true)).await).await;
let mut tools = BTreeMap::<u64, (String, String, String)>::new();
for chunk in &chunks {
if let Some(calls) = chunk["choices"][0]["delta"]["tool_calls"].as_array() {
for call in calls {
let entry = tools.entry(call["index"].as_u64().unwrap()).or_default();
if let Some(id) = call["id"].as_str() {
entry.0.push_str(id);
}
if let Some(name) = call["function"]["name"].as_str() {
entry.1.push_str(name);
}
if let Some(arguments) = call["function"]["arguments"].as_str() {
entry.2.push_str(arguments);
}
}
}
}
assert_eq!(
tools,
BTreeMap::from([
(
0,
(
"call_first".into(),
"weather".into(),
"{ \"city\": \"北京\" }".into()
)
),
(
1,
(
"call_second".into(),
"search".into(),
"{\"query\":\"Rust\"}".into()
)
),
])
);
assert_eq!(
chunks.last().unwrap()["choices"][0]["finish_reason"],
"tool_calls"
);
}
#[tokio::test]
async fn extra_attributes_and_auxiliary_events_preserve_existing_chat_results() {
let mut text = message("preserved");
text["future_optional"] = json!({"opaque":true});
text["content"][0]["future_attribute"] = json!([1, 2, 3]);
let fixture = Fixture::new(Session::new(vec![
wire("response.future_auxiliary", json!({"future":true})),
wire(
"response.reasoning_summary_text.delta",
json!({"output_index":0,"summary_index":0,"delta":"private"}),
),
created(),
completed(json!([text])),
]))
.await;
let (_, chunks) = sse_body(fixture.post(chat_request(true)).await).await;
assert_eq!(combined(&chunks, "content"), "preserved");
assert!(
!chunks
.iter()
.any(|chunk| chunk.to_string().contains("private"))
);
assert_eq!(
fixture
.trace
.discarded
.load(std::sync::atomic::Ordering::SeqCst),
2
);
}
#[tokio::test]
async fn incomplete_projects_only_supported_finish_reasons() {
for (reason, expected) in [
("max_output_tokens", "length"),
("content_filter", "content_filter"),
] {
for streaming in [false, true] {
let event = wire(
"response.incomplete",
json!({"response":{
"id":"resp_chat_test","status":"incomplete","output":[message("partial")],
"incomplete_details":{"reason":reason}}}),
);
let fixture = Fixture::new(Session::new(vec![created(), event])).await;
let response = fixture.post(chat_request(streaming)).await;
if streaming {
let (_, chunks) = sse_body(response).await;
assert_eq!(
chunks.last().unwrap()["choices"][0]["finish_reason"],
expected
);
} else {
assert_eq!(response.status(), StatusCode::OK);
assert_eq!(
json_body(response).await["choices"][0]["finish_reason"],
expected
);
}
}
}
}
#[tokio::test]
async fn encrypted_reasoning_stays_private_while_public_text_and_usage_are_projected() {
let reasoning = json!({
"id":"rs_private","type":"reasoning",
"summary":[{"type":"summary_text","text":"PRIVATE_SUMMARY_MARKER"}],
"encrypted_content":"OPAQUE_ENCRYPTED_REASONING_MARKER"
});
for streaming in [false, true] {
let fixture = Fixture::new(Session::new(vec![
created(),
wire(
"response.output_item.added",
json!({"output_index":0,"item":{"id":"rs_private","type":"reasoning","summary":[]}}),
),
wire(
"response.reasoning_summary_text.delta",
json!({"item_id":"rs_private","output_index":0,"summary_index":0,"delta":"PRIVATE_SUMMARY_MARKER"}),
),
wire(
"response.output_item.done",
json!({"output_index":0,"item":reasoning}),
),
wire(
"response.output_item.done",
json!({"output_index":1,"item":message("public answer")}),
),
completed(json!([reasoning, message("public answer")])),
]))
.await;
let mut request = chat_request(streaming);
if streaming {
request["stream_options"] = json!({"include_usage":true});
}
let response = fixture.post(request).await;
let (body, content, usage) = if streaming {
let (body, chunks) = sse_body(response).await;
let usage = chunks
.iter()
.find(|chunk| chunk["choices"] == json!([]))
.expect("terminal usage chunk")["usage"]
.clone();
(body, combined(&chunks, "content"), usage)
} else {
assert_eq!(response.status(), StatusCode::OK);
let body = json_body(response).await;
(
body.to_string(),
body["choices"][0]["message"]["content"]
.as_str()
.unwrap()
.to_owned(),
body["usage"].clone(),
)
};
assert_eq!(content, "public answer");
assert!(!body.contains("PRIVATE_SUMMARY_MARKER"), "{body}");
assert!(
!body.contains("OPAQUE_ENCRYPTED_REASONING_MARKER"),
"{body}"
);
assert_eq!(usage["completion_tokens_details"]["reasoning_tokens"], 2);
}
}
#[tokio::test]
async fn unrepresentable_business_output_fails_without_successful_chat_completion() {
for output in [
json!({"id":"future_output","type":"future_business_output","payload":"unrepresentable"}),
json!({"id":"ig_native","type":"image_generation_call","status":"completed","result":"OPAQUE_IMAGE_MARKER"}),
] {
for streaming in [false, true] {
let fixture =
Fixture::new(Session::new(vec![created(), completed(json!([output]))])).await;
let mut request = chat_request(streaming);
if streaming {
request["stream_options"] = json!({"include_usage":true});
}
let response = fixture.post(request).await;
if streaming {
let (body, chunks) = sse_body(response).await;
assert!(chunks.iter().any(|chunk| chunk.get("error").is_some()));
assert!(
!chunks
.iter()
.any(|chunk| chunk["choices"][0]["finish_reason"].is_string())
);
assert!(!chunks.iter().any(|chunk| chunk["choices"] == json!([])));
assert!(!body.contains("OPAQUE_IMAGE_MARKER"));
} else {
assert!(response.status().is_server_error());
let body = json_body(response).await;
assert!(body.get("error").is_some());
assert!(body.get("choices").is_none());
assert!(!body.to_string().contains("OPAQUE_IMAGE_MARKER"));
}
}
}
}
#[tokio::test]
async fn stream_without_terminal_does_not_emit_a_successful_finish_or_usage() {
let fixture = Fixture::new(Session::new(vec![
created(),
wire(
"response.output_text.delta",
json!({"item_id":"msg_chat_test","output_index":0,"content_index":0,"delta":"partial"}),
),
]))
.await;
let mut body = chat_request(true);
body["stream_options"] = json!({"include_usage":true});
let (_, chunks) = sse_body(fixture.post(body).await).await;
assert!(chunks.iter().any(|chunk| chunk.get("error").is_some()));
assert!(
!chunks
.iter()
.any(|chunk| chunk["choices"][0]["finish_reason"].is_string())
);
assert!(!chunks.iter().any(|chunk| chunk["choices"] == json!([])));
}
#[tokio::test]
async fn first_batch_terminal_can_finish_without_creating_usage_facts() {
let fixture = Fixture::new(Session::new(vec![wire(
"response.completed",
json!({"response":{
"id":"resp_chat_test","status":"completed","output":[message("done only")]
}}),
)]))
.await;
let mut body = chat_request(true);
body["stream_options"] = json!({"include_usage":true});
let (_, chunks) = sse_body(fixture.post(body).await).await;
assert_eq!(combined(&chunks, "content"), "done only");
assert_eq!(
chunks
.iter()
.filter(|chunk| chunk["choices"][0]["finish_reason"] == "stop")
.count(),
1
);
assert!(!chunks.iter().any(|chunk| chunk["usage"].is_object()));
assert!(
fixture
.trace
.finalized
.load(std::sync::atomic::Ordering::SeqCst)
);
}
#[tokio::test]
async fn native_sse_byte_fragments_and_multiple_events_preserve_unicode() {
let first = gateway_protocol::openai::sse::encode_sse_event("response.output_text.delta",&json!({
"type":"response.output_text.delta","item_id":"msg_chat_test","output_index":0,"content_index":0,"delta":"你"
}).to_string());
let second = gateway_protocol::openai::sse::encode_sse_event("response.output_text.delta",&json!({
"type":"response.output_text.delta","item_id":"msg_chat_test","output_index":0,"content_index":0,"delta":"好"
}).to_string());
let split = first.find('你').unwrap() + 1;
let mut remainder = first.as_bytes()[split..].to_vec();
remainder.extend_from_slice(second.as_bytes());
let raw = |bytes| {
gateway_core::event::ProviderEvent::wire(
gateway_core::event::ProtocolWireEvent::raw_sse("openai", bytes).unwrap(),
)
};
let fixture = Fixture::new(Session::new(vec![
created(),
raw(bytes::Bytes::copy_from_slice(&first.as_bytes()[..split])),
raw(bytes::Bytes::from(remainder)),
completed(json!([message("你好")])),
]))
.await;
let (_, chunks) = sse_body(fixture.post(chat_request(true)).await).await;
assert_eq!(combined(&chunks, "content"), "你好");
assert_eq!(
chunks.last().unwrap()["choices"][0]["finish_reason"],
"stop"
);
}
#[tokio::test]
async fn refusal_deltas_and_done_snapshots_produce_only_standard_refusal() {
let output = json!({"id":"msg_refusal","type":"message","role":"assistant","status":"completed",
"content":[{"type":"refusal","refusal":"cannot help"}]});
let fixture = Fixture::new(Session::new(vec![created(),
wire("response.refusal.delta",json!({"item_id":"msg_refusal","output_index":0,"content_index":0,"delta":"cannot "})),
wire("response.refusal.done",json!({"item_id":"msg_refusal","output_index":0,"content_index":0,"refusal":"cannot help"})),
completed(json!([output]))])).await;
let (_, chunks) = sse_body(fixture.post(chat_request(true)).await).await;
assert_eq!(combined(&chunks, "refusal"), "cannot help");
assert_eq!(combined(&chunks, "content"), "");
assert_eq!(
chunks.last().unwrap()["choices"][0]["finish_reason"],
"stop"
);
}
#[tokio::test]
async fn auxiliary_events_after_completed_preserve_exactly_one_finish_and_usage() {
for with_created in [false, true] {
// 终态与辅助事件同属一批 Core 交付 不构造已终结会话继续产出事件的另一种异常
let session = Session::new(if with_created {
vec![created()]
} else {
Vec::new()
})
.with_batch(vec![
completed(json!([message("completed text")])),
wire(
"response.future_metadata",
json!({"response_id":"resp_chat_test","future":{"kept":true}}),
),
]);
let fixture = Fixture::new(session).await;
let mut body = chat_request(true);
body["stream_options"] = json!({"include_usage":true});
let (_, chunks) = sse_body(fixture.post(body).await).await;
assert_eq!(combined(&chunks, "content"), "completed text");
assert_eq!(
chunks
.iter()
.filter(|chunk| chunk["choices"][0]["finish_reason"] == "stop")
.count(),
1
);
let usage = chunks
.iter()
.filter(|chunk| chunk["choices"] == json!([]))
.collect::<Vec<_>>();
assert_eq!(usage.len(), 1);
assert_eq!(usage[0]["usage"]["total_tokens"], 18);
assert!(!chunks.iter().any(|chunk| chunk.get("error").is_some()));
assert_eq!(
fixture
.trace
.commits
.load(std::sync::atomic::Ordering::SeqCst),
1
);
}
}
#[tokio::test]
async fn wire_failures_preserve_existing_recovery_fields_without_success_chunks() {
for shape in ["response.failed", "error", "bare_error"] {
let mut expected = json!({
"type":"rate_limit_error","code":"rate_limit_exceeded","param":null,
"message":"retry later","headers":{"retry-after":"2"},
"recovery":{"action":"retry","future":[1,2]},
"future_error_property":{"enabled":true}
});
let failure = match shape {
"response.failed" => wire(
"response.failed",
json!({"response":{
"id":"resp_chat_test","object":"response","model":"upstream-model",
"status":"failed","output":[],"error":expected.clone()
}}),
),
"error" => wire("error", json!({"error":expected.clone()})),
_ => {
// 裸事件的 type 是 wire 名称,仅该包装身份归一化为 Chat 错误类型
let failure = wire("error", expected.clone());
expected["type"] = json!("server_error");
failure
}
};
let fixture = Fixture::new(Session::new(vec![created(), failure])).await;
let mut body = chat_request(true);
body["stream_options"] = json!({"include_usage":true});
let (_, chunks) = sse_body(fixture.post(body).await).await;
let errors = chunks
.iter()
.filter_map(|chunk| chunk.get("error"))
.collect::<Vec<_>>();
assert_eq!(errors, vec![&expected], "{shape}");
assert!(
!chunks
.iter()
.any(|chunk| chunk["choices"][0]["finish_reason"].is_string()),
"{shape}"
);
assert!(!chunks.iter().any(|chunk| chunk["choices"] == json!([])));
assert!(!chunks.iter().any(|chunk| chunk["usage"].is_object()));
assert_eq!(
fixture
.trace
.commits
.load(std::sync::atomic::Ordering::SeqCst),
1
);
}
}
#[tokio::test]
async fn conflicting_text_snapshot_fails_instead_of_appending_replacement_content() {
let fixture = Fixture::new(Session::new(vec![created(),
wire("response.output_text.delta",json!({"item_id":"msg_chat_test","output_index":0,"content_index":0,"delta":"already sent"})),
completed(json!([message("different result")]))])).await;
let (_, chunks) = sse_body(fixture.post(chat_request(true)).await).await;
assert_eq!(combined(&chunks, "content"), "already sent");
assert!(chunks.iter().any(|chunk| chunk.get("error").is_some()));
assert!(
!chunks
.iter()
.any(|chunk| chunk["choices"][0]["finish_reason"].is_string())
);
}
@@ -1,6 +1,7 @@
//! OpenAI 协议 HTTP 与 WebSocket 接口的测试入口
mod auth;
mod chat_completions;
mod endpoint;
mod error;
mod image_policy;
@@ -78,9 +78,9 @@ impl ModelsExecution {
})
}
fn with_cli_min() -> Arc<Self> {
pub(super) fn with_cli_min(minimum: &str) -> Arc<Self> {
Arc::new(Self {
client: authenticated_client_with_min_versions("sk_models_test", None, Some("0.40.0")),
client: authenticated_client_with_min_versions("sk_models_test", None, Some(minimum)),
presentation: None,
native: None,
requested_versions: Mutex::default(),
@@ -440,7 +440,7 @@ async fn models_should_reject_recognized_cli_below_configured_min() {
.header("user-agent", "codex_cli_rs/0.39.0 (Linux; x86_64)")
.body(Body::empty())
.expect("build models request");
let response = api_router(ModelsExecution::with_cli_min())
let response = api_router(ModelsExecution::with_cli_min("0.40.0"))
.await
.oneshot(request)
.await
@@ -464,7 +464,7 @@ async fn models_should_reject_recognized_cli_with_invalid_version() {
.header("user-agent", "codex-cli/not-a-version")
.body(Body::empty())
.expect("build models request");
let response = api_router(ModelsExecution::with_cli_min())
let response = api_router(ModelsExecution::with_cli_min("0.40.0"))
.await
.oneshot(request)
.await
@@ -513,7 +513,7 @@ async fn models_should_allow_cli_at_its_configured_min() {
.header("user-agent", "codex_cli_rs/0.40.0 (Linux; x86_64)")
.body(Body::empty())
.expect("build models request");
let response = api_router(ModelsExecution::with_cli_min())
let response = api_router(ModelsExecution::with_cli_min("0.40.0"))
.await
.oneshot(request)
.await
@@ -529,7 +529,7 @@ async fn models_should_leave_unknown_clients_unrestricted() {
.header("user-agent", "curl/8.14.1")
.body(Body::empty())
.expect("build models request");
let response = api_router(ModelsExecution::with_cli_min())
let response = api_router(ModelsExecution::with_cli_min("0.40.0"))
.await
.oneshot(request)
.await
@@ -685,7 +685,7 @@ async fn model_detail_should_hide_unknown_models() {
#[tokio::test]
async fn models_should_authenticate_before_applying_the_version_gate() {
let response = api_router(ModelsExecution::with_cli_min())
let response = api_router(ModelsExecution::with_cli_min("0.40.0"))
.await
.oneshot(
Request::get("/v1/models")
@@ -2550,6 +2550,110 @@ async fn buffered_response_collects_completed_output_items_without_rewriting_str
assert!(!trace.is_cancelled());
}
#[tokio::test]
async fn native_responses_route_preserves_images_and_encrypted_reasoning_in_json_and_sse() {
let admin = crate::admin::AdminTestFixture::new().await.services;
let reasoning = json!({
"id":"rs_native","type":"reasoning","encrypted_content":"opaque-reasoning-for-replay",
"summary":[{"type":"summary_text","text":"native summary"}],
"future_reasoning_field":{"keep":true}
});
let image = json!({
"id":"ig_native","type":"image_generation_call","status":"completed",
"result":"b3BhcXVlLWltYWdl","output_format":"png",
"future_image_field":{"keep":true}
});
let terminal = json!({
"id":"resp_test","object":"response","model":"public-model","status":"completed",
"output":[reasoning, image],"future_terminal_field":{"keep":true}
});
for streaming in [false, true] {
let trace = Arc::new(Trace::default());
let mut expected_sse = Vec::new();
let mut events = Vec::new();
for (index, (kind, mut data, facts)) in [
(
"response.created",
json!({"response":{"id":"resp_test","model":"public-model","status":"in_progress","output":[]}}),
vec![started()],
),
(
"response.output_item.done",
json!({"output_index":0,"item":reasoning}),
vec![],
),
(
"response.image_generation_call.partial_image",
json!({"item_id":"ig_native","output_index":1,"partial_image_index":0,"partial_image_b64":"b3BhcXVlLXBhcnRpYWw="}),
vec![],
),
(
"response.output_item.done",
json!({"output_index":1,"item":image}),
vec![],
),
(
"response.completed",
json!({"response":terminal}),
vec![completed()],
),
]
.into_iter()
.enumerate()
{
data["type"] = json!(kind);
let id = format!("native-boundary-{index}");
// 保留行结束符、空白和 SSE 元数据,不能仅比较 JSON 后掩盖重编码
let raw = Bytes::from(format!(
"id: {id}\r\nevent: {kind}\r\nretry: 1234\r\ndata: {data} \r\n\r\n"
));
expected_sse.extend_from_slice(&raw);
events.push(ProviderEvent::canonical_with_wire(
facts,
ProtocolWireEvent::json_with_raw_sse_metadata(
"openai",
Some(kind.to_owned()),
data,
raw,
Some(id),
Some(1234),
)
.unwrap(),
));
}
let session = if streaming {
FakeSession::streaming(
Arc::clone(&trace),
vec![
NextStep::Event(
CoordinatedEvent::try_batch(
events,
CommitRequirement::CommitBeforeDelivery,
)
.unwrap(),
),
NextStep::FinalizeSuccess,
],
)
} else {
FakeSession::buffered_provider(Arc::clone(&trace), events)
};
// 使用真实 /v1/responses 入口,覆盖共享执行函数的可选适配器为空分支
let response = response(&admin, "openai", session, streaming).await;
assert_eq!(response.status(), StatusCode::OK);
let body = to_bytes(response.into_body(), usize::MAX).await.unwrap();
if streaming {
expected_sse.extend_from_slice(b"data: [DONE]\n\n");
assert_eq!(body.as_ref(), expected_sse.as_slice());
assert_eq!(trace.snapshot(), ["next_event", "commit", "next_end"]);
} else {
assert_eq!(serde_json::from_slice::<Value>(&body).unwrap(), terminal);
assert_eq!(trace.snapshot(), ["collect", "commit"]);
}
assert!(!trace.is_cancelled());
}
}
#[tokio::test]
async fn buffered_response_keeps_authoritative_terminal_and_supports_omitted_output() {
let item = json!({"id": "rs_test", "type": "reasoning", "encrypted_content": "opaque"});
@@ -2844,6 +2948,69 @@ struct SessionExecution {
middleware: Option<FrozenMiddlewarePlan>,
}
#[tokio::test]
async fn official_core_clients_should_be_version_checked_before_starting_responses() {
let admin = crate::admin::AdminTestFixture::new().await.services;
for product in [
"codex-tui",
"codex_exec",
"codex_vscode",
"codex_sdk_ts",
"codex_python_sdk",
] {
for (version, allowed) in [("0.158.0", false), ("0.162.0", true)] {
let trace = Arc::new(Trace::default());
let execution = Arc::new(SessionExecution {
client: authenticated_client_with_min_versions(
"sk_version_test",
None,
Some("0.162.0"),
),
session: Mutex::new(Some(Box::new(FakeSession::streaming(
Arc::clone(&trace),
vec![
NextStep::Event(delivery(
started(),
CommitRequirement::CommitBeforeDelivery,
)),
NextStep::Event(delivery(completed(), CommitRequirement::AlreadyCommitted)),
NextStep::FinalizeSuccess,
],
)))),
middleware: None,
});
let request = Request::post("/v1/responses")
.header(AUTHORIZATION, "Bearer sk_version_test")
.header("user-agent", format!(
"{product}/{version} (Mac OS 26.6.2; arm64) Apple_Terminal ({product}; 0.165.0)"
))
.body(Body::from(json!({"model": "model-a", "input": "synthetic", "stream": true}).to_string()))
.unwrap();
let response = crate::openai::api_router_with_admin_and_execution(
admin.clone(),
execution.clone(),
)
.oneshot(request)
.await
.unwrap();
let status = response.status();
let body = to_bytes(response.into_body(), usize::MAX).await.unwrap();
if allowed {
assert_eq!(status, StatusCode::OK, "{product}/{version}");
assert!(String::from_utf8_lossy(&body).contains("event: response.completed\n"));
assert!(trace.snapshot().contains(&"commit"));
} else {
assert_eq!(status, StatusCode::UPGRADE_REQUIRED, "{product}/{version}");
let value: Value = serde_json::from_slice(&body).unwrap();
assert_eq!(value["error"]["code"], "client_version_too_old");
assert_eq!(value["error"]["current_version"], version);
assert_eq!(value["error"]["min_version"], "0.162.0");
assert!(execution.session.lock().unwrap().is_some());
}
}
}
}
#[tokio::test]
async fn chatgpt_remote_responses_should_only_skip_missing_desktop_versions() {
let admin = crate::admin::AdminTestFixture::new().await.services;
@@ -557,7 +557,7 @@ async fn outbound_commands_are_written_in_acknowledged_order() {
}
#[tokio::test(start_paused = true)]
async fn stalled_write_times_out_at_three_hundred_seconds_and_aborts_the_pump() {
async fn stalled_business_write_cannot_block_the_heartbeat_write_deadline() {
let PumpHarness {
mut connection,
incoming: _incoming,
@@ -566,6 +566,7 @@ async fn stalled_write_times_out_at_three_hundred_seconds_and_aborts_the_pump()
..
} = test_connection(true);
let request_id = Arc::<str>::from("req_test");
let started = tokio::time::Instant::now();
let error = connection
.send_text(
@@ -573,12 +574,14 @@ async fn stalled_write_times_out_at_three_hundred_seconds_and_aborts_the_pump()
WriteContext::request(&request_id, FramePhase::First),
)
.await
.expect_err("stalled write must time out");
.expect_err("heartbeat deadline must interrupt the stalled business write");
tokio::task::yield_now().await;
assert!(
matches!(error, ConnectionWriteError::Timeout { timeout } if timeout == Duration::from_secs(300))
&& dropped.load(Ordering::Acquire)
assert!(matches!(error, ConnectionWriteError::Closed) && dropped.load(Ordering::Acquire));
assert_eq!(started.elapsed(), Duration::from_secs(25 + 5));
assert_eq!(
connection.wait_for_exit().await,
PumpExitReason::WriteTimeout
);
}
@@ -586,10 +589,19 @@ async fn stalled_write_times_out_at_three_hundred_seconds_and_aborts_the_pump()
async fn pump_marks_the_connection_expired_at_sixty_minutes() {
let PumpHarness {
mut connection,
incoming: _incoming,
written: _written,
incoming,
mut written,
..
} = test_connection(false);
let peer = tokio::spawn(async move {
while let Some(message) = written.recv().await {
if let Message::Ping(payload) = message {
incoming
.send(Ok(Message::Pong(payload)))
.expect("reply to Ping");
}
}
});
let event = connection
.next_event()
@@ -597,6 +609,7 @@ async fn pump_marks_the_connection_expired_at_sixty_minutes() {
.expect("deadline must emit an event");
assert!(matches!(event, ConnectionEvent::Expired) && connection.is_expired());
peer.abort();
}
#[tokio::test]
@@ -606,6 +619,7 @@ async fn dropping_the_connection_aborts_and_drops_the_socket_owner() {
incoming: _incoming,
written: _written,
dropped,
cancellation,
..
} = test_connection(false);
@@ -613,6 +627,7 @@ async fn dropping_the_connection_aborts_and_drops_the_socket_owner() {
tokio::task::yield_now().await;
assert!(dropped.load(Ordering::Acquire));
assert!(cancellation.is_cancelled());
}
#[tokio::test(start_paused = true)]
@@ -1122,7 +1137,7 @@ async fn client_close_should_cancel_active_execution_without_starting_queued_req
async fn idle_pump_sends_heartbeat_without_bursting_after_a_delay() {
let PumpHarness {
connection,
incoming: _incoming,
incoming,
mut written,
..
} = test_connection(false);
@@ -1132,6 +1147,10 @@ async fn idle_pump_sends_heartbeat_without_bursting_after_a_delay() {
tokio::time::advance(Duration::from_secs(25)).await;
let first = written.recv().await.expect("first heartbeat");
assert!(matches!(first, Message::Ping(_)));
incoming
.send(Ok(Message::Pong(vec![1].into())))
.expect("reply before scheduler delay");
tokio::task::yield_now().await;
tokio::time::advance(Duration::from_secs(120)).await;
let second = written.recv().await.expect("heartbeat after delay");
@@ -1143,7 +1162,35 @@ async fn idle_pump_sends_heartbeat_without_bursting_after_a_delay() {
}
#[tokio::test(start_paused = true)]
async fn missing_pongs_do_not_cancel_the_connection_or_block_business_frames() {
async fn missing_responses_expire_from_the_first_ping_without_extending_on_repeated_pings() {
let PumpHarness {
mut connection,
incoming,
mut written,
dropped,
cancellation,
..
} = test_connection(false);
let started = tokio::time::Instant::now();
for _ in 0..32 {
incoming
.send(Ok(Message::Text("queued request".into())))
.expect("fill the business queue before probing");
}
assert_eq!(
connection.wait_for_exit().await,
PumpExitReason::PongTimeout
);
assert_eq!(started.elapsed(), Duration::from_secs(25 + 30));
assert!(matches!(written.try_recv(), Ok(Message::Ping(_))));
assert!(matches!(written.try_recv(), Ok(Message::Ping(_))));
assert!(dropped.load(Ordering::Acquire));
assert!(cancellation.is_cancelled());
}
#[tokio::test(start_paused = true)]
async fn business_frames_keep_the_connection_healthy_without_pongs() {
let PumpHarness {
mut connection,
incoming,
@@ -1153,15 +1200,14 @@ async fn missing_pongs_do_not_cancel_the_connection_or_block_business_frames() {
for _ in 0..13 {
tokio::time::advance(Duration::from_secs(25)).await;
assert!(matches!(written.recv().await, Some(Message::Ping(_))));
incoming
.send(Ok(Message::Text("still connected".into())))
.expect("send business frame instead of Pong");
assert!(matches!(
connection.next_event().await,
Some(ConnectionEvent::Text(text)) if text == "still connected"
));
}
incoming
.send(Ok(Message::Text("still connected".into())))
.expect("send business frame after 325 seconds without Pong");
assert!(matches!(
connection.next_event().await,
Some(ConnectionEvent::Text(text)) if text == "still connected"
));
incoming.send(Ok(Message::Close(None))).expect("close peer");
assert_eq!(
connection.wait_for_exit().await,
@@ -1192,7 +1238,7 @@ async fn stalled_heartbeat_can_be_cancelled_without_waiting_for_write_timeout()
}
#[tokio::test(start_paused = true)]
async fn stalled_heartbeat_obeys_the_existing_write_timeout() {
async fn stalled_heartbeat_uses_a_short_write_timeout() {
let PumpHarness {
mut connection,
incoming: _incoming,
@@ -1205,7 +1251,7 @@ async fn stalled_heartbeat_obeys_the_existing_write_timeout() {
connection.wait_for_exit().await,
PumpExitReason::WriteTimeout
);
assert_eq!(started.elapsed(), Duration::from_secs(25 + 300));
assert_eq!(started.elapsed(), Duration::from_secs(25 + 5));
assert!(dropped.load(Ordering::Acquire));
}
@@ -1208,6 +1208,50 @@ fn upgrade_request(authorization: &str) -> Request<Body> {
.expect("build WebSocket upgrade request")
}
#[tokio::test]
async fn official_core_version_gate_should_run_before_websocket_upgrade() {
let app = api_router(ModelsExecution::with_cli_min("0.162.0")).await;
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let address = listener.local_addr().unwrap();
let server = tokio::spawn(async move { axum::serve(listener, app).await.unwrap() });
for version in ["0.158.0", "0.162.0"] {
let mut request = format!("ws://{address}/v1/responses")
.into_client_request()
.unwrap();
request.headers_mut().insert(
AUTHORIZATION,
HeaderValue::from_static("Bearer sk_models_test"),
);
request.headers_mut().insert(
"user-agent",
HeaderValue::from_str(&format!(
"codex-tui/{version} (Mac OS 26.6.2; arm64) Apple_Terminal (codex-tui; 0.165.0)"
))
.unwrap(),
);
let result = connect_async(request).await;
if version == "0.162.0" {
let (mut socket, response) = result.expect("current Core upgrades successfully");
assert_eq!(response.status(), StatusCode::SWITCHING_PROTOCOLS);
socket.close(None).await.unwrap();
} else {
let tokio_tungstenite::tungstenite::Error::Http(response) =
result.expect_err("old Core must not upgrade")
else {
panic!("expected an HTTP version rejection");
};
assert_eq!(response.status(), StatusCode::UPGRADE_REQUIRED);
let value: Value = serde_json::from_slice(response.body().as_deref().unwrap()).unwrap();
assert_eq!(value["error"]["code"], "client_version_too_old");
assert_eq!(value["error"]["current_version"], "0.158.0");
assert_eq!(value["error"]["min_version"], "0.162.0");
}
}
server.abort();
let _ = server.await;
}
#[tokio::test]
async fn get_responses_should_route_to_the_websocket_upgrade_boundary() {
let response = api_router(ModelsExecution::new())
@@ -0,0 +1,327 @@
//! 账号错误保护的可编辑规则、匹配事实与有界恢复快照
use sha2::{Digest as _, Sha256};
use std::collections::BTreeSet;
/// 规则只观察已解析的上游事实,不扫描原始响应或改写协议分类
#[derive(Debug, Clone, Copy)]
pub struct AccountErrorFacts<'a> {
pub provider: &'a str,
pub status: Option<u16>,
pub code: Option<&'a str>,
pub error_type: Option<&'a str>,
pub message: Option<&'a str>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum AccountErrorField {
Provider,
Status,
Code,
Type,
Message,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum AccountErrorOperator {
Equals,
Contains,
}
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct AccountErrorCondition {
pub field: AccountErrorField,
pub operator: AccountErrorOperator,
pub values: Vec<Option<String>>,
#[serde(default)]
pub ignore_case: bool,
}
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct AccountErrorRule {
pub id: String,
pub name: String,
pub conditions: Vec<AccountErrorCondition>,
}
/// 持久状态仅保存有界规则标识,不保存上游错误正文
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct AccountProtectionMatch {
pub id: String,
pub name: String,
}
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct AccountErrorProtection {
pub enabled: bool,
pub rules: Vec<AccountErrorRule>,
pub threshold: u32,
pub window_seconds: u64,
pub duration_seconds: u64,
pub probe_enabled: bool,
pub probe_model: Option<String>,
}
impl Default for AccountErrorProtection {
fn default() -> Self {
Self {
enabled: false,
rules: Vec::new(),
threshold: 12,
window_seconds: 600,
duration_seconds: 7200,
probe_enabled: true,
probe_model: None,
}
}
}
impl AccountErrorProtection {
pub fn validate(&self) -> Result<(), &'static str> {
if !(2..=1000).contains(&self.threshold)
|| !(60..=3600).contains(&self.window_seconds)
|| !(300..=604800).contains(&self.duration_seconds)
|| self
.probe_model
.as_deref()
.is_some_and(|v| !valid_label(v, 128))
|| self.rules.len() > 32
|| (self.enabled && self.rules.is_empty())
{
return Err("account_error_protection");
}
let mut text_bytes = self.probe_model.as_ref().map_or(0, String::len);
let mut ids = BTreeSet::new();
for rule in &self.rules {
text_bytes += rule.id.len() + rule.name.len();
if text_bytes > 65536 {
return Err("account_error_protection");
}
if !valid_label(&rule.id, 128)
|| !valid_label(&rule.name, 128)
|| !ids.insert(&rule.id)
|| rule.conditions.is_empty()
|| rule.conditions.len() > 8
|| !rule
.conditions
.iter()
.any(|c| c.field != AccountErrorField::Provider)
{
return Err("account_error_protection");
}
for condition in &rule.conditions {
let max = if condition.field == AccountErrorField::Message {
1024
} else {
256
};
if condition.values.is_empty()
|| condition.values.len() > 16
|| (condition.operator == AccountErrorOperator::Contains
&& condition.field != AccountErrorField::Message)
{
return Err("account_error_protection");
}
for value in &condition.values {
text_bytes += value.as_ref().map_or(0, String::len);
if text_bytes > 65536 {
return Err("account_error_protection");
}
if value.as_deref().is_some_and(|v| v.chars().count() > max)
|| (condition.operator == AccountErrorOperator::Contains
&& value.as_deref().is_none_or(|v| v.trim().is_empty()))
|| (condition.field == AccountErrorField::Status
&& value.as_deref().is_some_and(|v| {
v.len() != 3
|| v.parse::<u16>()
.map_or(true, |status| !(100..=599).contains(&status))
}))
{
return Err("account_error_protection");
}
}
}
}
Ok(())
}
#[must_use]
pub fn compile(&self) -> CompiledAccountErrorProtection {
let mut normalized = self.clone();
let mut ignore_case = [false; 5];
for rule in &mut normalized.rules {
for condition in &mut rule.conditions {
ignore_case[condition.field as usize] |=
condition.ignore_case && condition.operator == AccountErrorOperator::Contains;
for candidate in condition.values.iter_mut().flatten() {
*candidate = if condition.ignore_case {
candidate.trim().to_ascii_lowercase()
} else {
candidate.trim().to_owned()
};
}
}
}
CompiledAccountErrorProtection {
policy: self.clone(),
fingerprint: self.fingerprint(),
normalized,
ignore_case,
}
}
#[must_use]
pub fn matching_rules(&self, facts: AccountErrorFacts<'_>) -> Vec<AccountProtectionMatch> {
self.compile().matching_rules(facts)
}
/// 长度前缀避免分隔符碰撞,内容指纹用于保留无关设置更新前的计数
#[must_use]
pub fn fingerprint(&self) -> String {
let mut hash = Sha256::new();
fn value(hash: &mut Sha256, value: &[u8]) {
hash.update((value.len() as u64).to_le_bytes());
hash.update(value);
}
value(
&mut hash,
&[u8::from(self.enabled), u8::from(self.probe_enabled)],
);
value(&mut hash, &self.threshold.to_le_bytes());
value(&mut hash, &self.window_seconds.to_le_bytes());
value(&mut hash, &self.duration_seconds.to_le_bytes());
value(
&mut hash,
self.probe_model.as_deref().unwrap_or_default().as_bytes(),
);
for rule in &self.rules {
value(&mut hash, rule.id.as_bytes());
value(&mut hash, rule.name.as_bytes());
value(&mut hash, &(rule.conditions.len() as u64).to_le_bytes());
for c in &rule.conditions {
value(
&mut hash,
&[c.field as u8, c.operator as u8, u8::from(c.ignore_case)],
);
value(&mut hash, &(c.values.len() as u64).to_le_bytes());
for v in &c.values {
value(&mut hash, &[u8::from(v.is_some())]);
value(&mut hash, v.as_deref().unwrap_or_default().as_bytes());
}
}
}
hash.finalize()
.iter()
.map(|byte| format!("{byte:02x}"))
.collect()
}
}
fn valid_label(value: &str, max: usize) -> bool {
!value.is_empty()
&& value == value.trim()
&& value.chars().count() <= max
&& !value.chars().any(char::is_control)
}
/// 发布时归一化候选并缓存指纹,请求只归一化每个实际输入一次
#[derive(Debug, Clone)]
pub struct CompiledAccountErrorProtection {
policy: AccountErrorProtection,
fingerprint: String,
normalized: AccountErrorProtection,
ignore_case: [bool; 5],
}
impl CompiledAccountErrorProtection {
#[must_use]
pub fn policy(&self) -> &AccountErrorProtection {
&self.policy
}
#[must_use]
pub fn fingerprint(&self) -> &str {
&self.fingerprint
}
#[must_use]
pub fn matching_rules(&self, facts: AccountErrorFacts<'_>) -> Vec<AccountProtectionMatch> {
if !self.policy.enabled {
return Vec::new();
}
let status = facts.status.map(|value| value.to_string());
let raw = [
Some(facts.provider),
status.as_deref(),
facts.code,
facts.error_type,
facts.message,
];
// 保留实际存在性;超长输入不能冒充缺失,也不能用截断前缀做整串相等
let full: [Option<&str>; 5] = raw.map(|value| value.map(str::trim));
let limits = [1024, 3, 1024, 1024, 8192];
let observed: [Option<&str>; 5] =
std::array::from_fn(|index| raw[index].map(|text| prefix(text, limits[index]).trim()));
let folded: [Option<String>; 5] = std::array::from_fn(|index| {
if self.ignore_case[index] {
observed[index].map(str::to_ascii_lowercase)
} else {
None
}
});
self.normalized
.rules
.iter()
.filter(|rule| {
!rule.conditions.is_empty()
&& rule.conditions.iter().all(|condition| {
let index = condition.field as usize;
let actual = if condition.ignore_case
&& condition.operator == AccountErrorOperator::Contains
{
folded[index].as_deref()
} else {
observed[index]
};
condition.values.iter().any(|candidate| {
match (actual, candidate.as_deref()) {
(None, None) => condition.operator == AccountErrorOperator::Equals,
(Some(actual), Some(candidate)) => match condition.operator {
AccountErrorOperator::Equals => {
full[index].is_some_and(|value| {
if condition.ignore_case {
value.eq_ignore_ascii_case(candidate)
} else {
value == candidate
}
})
}
AccountErrorOperator::Contains => {
!candidate.is_empty() && actual.contains(candidate)
}
},
_ => false,
}
})
})
})
.map(|rule| AccountProtectionMatch {
id: rule.id.clone(),
name: rule.name.clone(),
})
.collect()
}
}
fn prefix(value: &str, maximum: usize) -> &str {
let mut end = value.len().min(maximum);
while !value.is_char_boundary(end) {
end -= 1;
}
&value[..end]
}
@@ -1,6 +1,8 @@
//! Provider 账号领域、持久化端口与同一 target 内的账号选择
mod error;
mod error_protection;
pub use error_protection::*;
mod fast_mode;
pub use fast_mode::FastMode;
mod location;
@@ -623,6 +623,10 @@ pub enum AccountCooldownKind {
CapacityFreeze,
/// 到期后仍阻止调度,必须由恢复探测成功或管理员恢复解除
CapacityFreezeProbe,
/// 自定义错误规则触发的账号保护
ErrorProtection,
/// 账号保护到期后需要探测确认恢复
ErrorProtectionProbe,
}
impl AccountCooldownKind {
@@ -632,17 +636,25 @@ impl AccountCooldownKind {
Self::RateLimit => "rate_limit",
Self::CapacityFreeze => "capacity_freeze",
Self::CapacityFreezeProbe => "capacity_freeze_probe",
Self::ErrorProtection => "error_protection",
Self::ErrorProtectionProbe => "error_protection_probe",
}
}
#[must_use]
pub const fn is_capacity_freeze(self) -> bool {
matches!(self, Self::CapacityFreeze | Self::CapacityFreezeProbe)
pub const fn is_error_protection(self) -> bool {
matches!(
self,
Self::CapacityFreeze
| Self::CapacityFreezeProbe
| Self::ErrorProtection
| Self::ErrorProtectionProbe
)
}
#[must_use]
pub const fn requires_probe(self) -> bool {
matches!(self, Self::CapacityFreezeProbe)
matches!(self, Self::CapacityFreezeProbe | Self::ErrorProtectionProbe)
}
#[must_use]
@@ -651,6 +663,8 @@ impl AccountCooldownKind {
"rate_limit" => Some(Self::RateLimit),
"capacity_freeze" => Some(Self::CapacityFreeze),
"capacity_freeze_probe" => Some(Self::CapacityFreezeProbe),
"error_protection" => Some(Self::ErrorProtection),
"error_protection_probe" => Some(Self::ErrorProtectionProbe),
_ => None,
}
}
@@ -0,0 +1,137 @@
//! 单次上游尝试的通用保护反馈;副作用有界等待且不改变原始执行结果
use std::{sync::Arc, time::Duration};
use futures::{FutureExt, future::BoxFuture, pin_mut, select_biased};
use futures_timer::Delay;
use super::provider::ProviderCallMetadata;
use crate::{
account::{AccountErrorFacts, CompiledAccountErrorProtection},
error::ProviderError,
event::ProviderEvent,
provider_ports::{AccountProtectionGuard, ProviderCooldownPort},
routing::ConfigRevision,
};
const FEEDBACK_TIMEOUT: Duration = Duration::from_millis(100);
#[derive(Clone)]
pub(super) struct ProtectionContext {
pub(super) store: Arc<dyn ProviderCooldownPort>,
pub(super) policy: Arc<CompiledAccountErrorProtection>,
pub(super) revision: u64,
}
impl std::fmt::Debug for ProtectionContext {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter
.debug_struct("ProtectionContext")
.field("revision", &self.revision)
.finish_non_exhaustive()
}
}
pub(super) struct AttemptProtection {
store: Arc<dyn ProviderCooldownPort>,
guard: AccountProtectionGuard,
policy: Arc<CompiledAccountErrorProtection>,
provider: crate::identity::ProviderKind,
completed: bool,
}
impl AttemptProtection {
pub(super) async fn begin(
store: Arc<dyn ProviderCooldownPort>,
policy: Arc<CompiledAccountErrorProtection>,
policy_revision: u64,
metadata: &ProviderCallMetadata,
) -> Option<Self> {
if !policy.policy().enabled {
return None;
}
let revision = metadata.credential_revision()?;
let policy_revision = ConfigRevision::new(policy_revision).ok()?;
let guard = match bounded(store.begin_protection(
metadata.provider_account_id(),
revision,
policy.fingerprint(),
policy_revision,
))
.await
{
Ok(Ok(guard)) => guard?,
_ => {
tracing::warn!("account protection feedback could not start");
return None;
}
};
Some(Self {
store,
guard,
policy,
provider: metadata.provider().clone(),
completed: false,
})
}
pub(super) fn observe(&mut self, event: &ProviderEvent) {
self.completed |= event
.canonical_facts()
.iter()
.any(|fact| matches!(fact, crate::event::GatewayEvent::Completed(_)));
}
pub(super) const fn is_completed(&self) -> bool {
self.completed
}
pub(super) fn complete(
self,
result: Option<Result<ProviderEvent, ProviderError>>,
) -> BoxFuture<'static, Option<Result<ProviderEvent, ProviderError>>> {
Box::pin(async move {
let feedback = match &result {
Some(Err(error)) => {
let Some(fields) = error.upstream_failure_fields() else {
return result;
};
let matched = self.policy.matching_rules(AccountErrorFacts {
provider: self.provider.as_str(),
status: fields.status,
code: fields.code.as_deref(),
error_type: fields.error_type.as_deref(),
message: fields.message.as_deref(),
});
if matched.is_empty() {
return result;
}
bounded(self.store.record_protection_failure(
&self.guard,
self.policy.policy(),
&matched,
))
.await
}
None | Some(Ok(_)) if self.completed => {
bounded(self.store.record_protection_success(&self.guard)).await
}
_ => return result,
};
if !matches!(feedback, Ok(Ok(_))) {
tracing::warn!("account protection feedback was not persisted");
}
result
})
}
}
async fn bounded<T>(future: impl std::future::Future<Output = T>) -> Result<T, ()> {
let future = future.fuse();
let timeout = Delay::new(FEEDBACK_TIMEOUT).fuse();
pin_mut!(future, timeout);
select_biased! {
value = future => Ok(value),
_ = timeout => Err(()),
}
}
@@ -1150,6 +1150,17 @@ where
}
let context = AttemptContext::new(
RequestAttemptContext::new(self.request_id.clone(), self.client_api_key_ref.clone())
.with_account_error_protection(
(!self.account_selection.is_diagnostic()
&& self.plan.account_error_protection().policy().enabled)
.then(|| self.engine.account_error_protection.as_ref())
.flatten()
.map(|store| super::account_protection::ProtectionContext {
store: Arc::clone(store),
policy: self.plan.account_error_protection(),
revision: self.plan.account_error_protection_revision(),
}),
)
.with_lease_metadata(Arc::clone(&self.lease_metadata))
.with_response_control(self.response_control.clone())
.with_request_profile(self.request_profiles.get(candidate.provider()).cloned())
@@ -211,6 +211,15 @@ impl PreparedRootExecution {
self.response_control.clone()
}
/// 连接退出同时撤销跨轮次保留的上游资源,不改变单次执行的取消范围
#[must_use]
pub fn with_connection_cancellation(mut self, cancellation: CancellationToken) -> Self {
self.response_control = self
.response_control
.with_connection_cancellation(cancellation);
self
}
pub(super) fn new(client: AuthenticatedClient) -> Result<Self, GatewayError> {
let started_at = SystemTime::now();
let deadline_at = Deadline::from_timeout(started_at, client.execution_timeout())
@@ -92,6 +92,7 @@ pub struct DefaultExecutionService {
frontend_authentication: Option<FrontendAuthenticationExtensionIndex>,
session_affinity: Option<Arc<dyn ProviderSessionAffinityPort>>,
client_retry: Option<Arc<dyn crate::engine::client_retry::ClientRetryStore>>,
account_error_protection: Option<Arc<dyn crate::provider_ports::ProviderCooldownPort>>,
snapshot_refresh: Option<Arc<RuntimeSnapshotPublisher>>,
active_requests: ActiveRequestRegistry,
}
@@ -123,6 +124,7 @@ impl DefaultExecutionService {
frontend_authentication: None,
session_affinity: None,
client_retry: None,
account_error_protection: None,
snapshot_refresh: None,
active_requests: Arc::default(),
}
@@ -134,6 +136,15 @@ impl DefaultExecutionService {
self
}
#[must_use]
pub fn with_account_error_protection(
mut self,
store: Arc<dyn crate::provider_ports::ProviderCooldownPort>,
) -> Self {
self.account_error_protection = Some(store);
self
}
#[must_use]
pub fn with_client_retry_store(
mut self,
@@ -850,6 +861,9 @@ impl DefaultExecutionService {
deadline_at: authorization.deadline_at,
};
let mut engine = GatewayEngine::new(self.observations.clone(), providers);
if let Some(store) = &self.account_error_protection {
engine = engine.with_account_error_protection(Arc::clone(store));
}
// 插件内部调用不共享外部客户端的失败轮次,探测另走 transient engine
if authorization.nested.is_none()
&& authorization.bound.is_none()
@@ -1,5 +1,6 @@
//! 模型请求生命周期、单行持久化 port 与 commit/send/cancellation 边界
mod account_protection;
pub mod admission;
pub mod authentication;
pub mod budget;
@@ -286,6 +287,7 @@ impl ContinuationAttempt {
/// Provider 每次执行可见的 request-local context
#[derive(Debug, Clone)]
pub struct RequestAttemptContext {
account_error_protection: Option<account_protection::ProtectionContext>,
request_body_limit: Option<usize>,
lease_metadata: Option<Arc<crate::provider_ports::ProviderLeaseMetadata>>,
response_control: Option<response_control::ResponseControl>,
@@ -312,6 +314,14 @@ pub struct RequestAttemptContext {
}
impl RequestAttemptContext {
fn with_account_error_protection(
mut self,
protection: Option<account_protection::ProtectionContext>,
) -> Self {
self.account_error_protection = protection;
self
}
#[must_use]
pub const fn with_request_body_limit(mut self, bytes: usize) -> Self {
self.request_body_limit = Some(bytes);
@@ -393,6 +403,7 @@ impl RequestAttemptContext {
#[must_use]
pub fn new(request_id: ModelRequestId, client_api_key_ref: ClientApiKeyId) -> Self {
Self {
account_error_protection: None,
response_control: None,
request_body_limit: None,
lease_metadata: None,
@@ -777,6 +788,24 @@ impl AttemptContext {
} else {
terminal
};
// 保护观察真正的 Provider terminal,先于 attempt 中间件的响应改写
let terminal: provider::ProviderMiddlewareTerminal =
if let Some(protection) = self.request.account_error_protection.clone() {
Box::new(move |operation, headers| {
Box::pin(async move {
let stream = terminal(operation, headers).await?;
Ok(stream
.with_account_error_protection(
protection.store,
protection.policy,
protection.revision,
)
.await)
})
})
} else {
terminal
};
let context = self.middleware_context(
middleware::MiddlewareMount::Attempt,
Some(operation.kind()),
@@ -1245,6 +1274,7 @@ pub struct GatewayEngine<S: ?Sized> {
store: Arc<S>,
providers: provider::ProviderRegistry,
client_retry: Option<Arc<dyn client_retry::ClientRetryStore>>,
account_error_protection: Option<Arc<dyn crate::provider_ports::ProviderCooldownPort>>,
}
impl<S: ?Sized> GatewayEngine<S> {
@@ -1254,9 +1284,19 @@ impl<S: ?Sized> GatewayEngine<S> {
store,
providers,
client_retry: None,
account_error_protection: None,
}
}
#[must_use]
pub fn with_account_error_protection(
mut self,
store: Arc<dyn crate::provider_ports::ProviderCooldownPort>,
) -> Self {
self.account_error_protection = Some(store);
self
}
#[must_use]
pub fn with_client_retry_store(
mut self,
@@ -50,6 +50,7 @@ pub struct ProviderCallMetadata {
transport: UpstreamTransport,
selection_observation: Option<ProviderSelectionObservation>,
account_snapshot: Option<AccountHistorySnapshot>,
credential_revision: Option<crate::account::CredentialRevision>,
}
/// Provider 账号选择阶段输出的中立运行压力事实
@@ -99,6 +100,7 @@ impl ProviderCallMetadata {
transport,
selection_observation: None,
account_snapshot: None,
credential_revision: None,
}
}
@@ -117,9 +119,25 @@ impl ProviderCallMetadata {
transport,
selection_observation: None,
account_snapshot: None,
credential_revision: None,
}
}
/// 账号实际选定时冻结的凭据版本,不在反馈时重新读取当前账号
#[must_use]
pub const fn with_credential_revision(
mut self,
revision: crate::account::CredentialRevision,
) -> Self {
self.credential_revision = Some(revision);
self
}
#[must_use]
pub const fn credential_revision(&self) -> Option<crate::account::CredentialRevision> {
self.credential_revision
}
#[must_use]
pub fn with_account_snapshot(mut self, snapshot: Option<AccountHistorySnapshot>) -> Self {
self.account_snapshot = snapshot;
@@ -236,6 +254,9 @@ pub struct ProviderStream {
_lease: Box<dyn ResourceLease>,
native_response_translator: Option<Box<dyn NativeResponseTranslator>>,
account_feedback: Option<ProviderStreamAccountFeedback>,
account_protection: Option<super::account_protection::AttemptProtection>,
protection_feedback:
Option<futures::future::BoxFuture<'static, Option<Result<ProviderEvent, ProviderError>>>>,
validator: Result<EventSequenceValidator, EventSequenceError>,
strict_canonical_seen: bool,
terminated: bool,
@@ -343,12 +364,32 @@ impl ProviderStream {
_lease: Box::new(lease),
native_response_translator: None,
account_feedback: None,
account_protection: None,
protection_feedback: None,
validator: Ok(EventSequenceValidator::new()),
strict_canonical_seen: false,
terminated: false,
}
}
/// 在第一次发送前捕获反馈栅栏,诊断调用由协调器排除
#[must_use]
pub async fn with_account_error_protection(
mut self,
store: Arc<dyn crate::provider_ports::ProviderCooldownPort>,
policy: Arc<crate::account::CompiledAccountErrorProtection>,
policy_revision: u64,
) -> Self {
self.account_protection = super::account_protection::AttemptProtection::begin(
store,
policy,
policy_revision,
&self.metadata,
)
.await;
self
}
/// 让公共 stream 边界统一回灌账号成功率与首个有效输出延迟
#[must_use]
pub fn with_account_feedback(mut self, stats: Arc<AccountFeedbackStats>) -> Self {
@@ -928,6 +969,40 @@ impl Stream for ProviderStream {
fn poll_next(self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let this = self.get_mut();
if let Some(feedback) = this.protection_feedback.as_mut() {
return match feedback.as_mut().poll(context) {
Poll::Pending => Poll::Pending,
Poll::Ready(result) => {
this.protection_feedback = None;
Poll::Ready(result)
}
};
}
let result = this.poll_event(context);
// 客户端可在成功终态后停止拉取,不能依赖之后的 EOF 才清除保护证据
let protection_terminal = matches!(&result, Poll::Ready(Some(Err(_)) | None))
|| (matches!(&result, Poll::Ready(Some(Ok(_))))
&& this
.account_protection
.as_ref()
.is_some_and(super::account_protection::AttemptProtection::is_completed));
if protection_terminal
&& let Some(protection) = this.account_protection.take()
&& let Poll::Ready(result) = result
{
this.protection_feedback = Some(protection.complete(result));
return Pin::new(this).poll_next(context);
}
result
}
}
impl ProviderStream {
fn poll_event(
&mut self,
context: &mut Context<'_>,
) -> Poll<Option<Result<ProviderEvent, ProviderError>>> {
let this = self;
if this.terminated {
return Poll::Ready(None);
}
@@ -961,6 +1036,9 @@ impl Stream for ProviderStream {
if let Some(feedback) = this.account_feedback.as_mut() {
feedback.observe(&event);
}
if let Some(protection) = this.account_protection.as_mut() {
protection.observe(&event);
}
Poll::Ready(Some(Ok(event)))
}
Poll::Ready(Some(Err(error))) => {
@@ -1015,7 +1093,7 @@ pub struct ProviderRequest {
pub struct ProviderRequestObservation {
/// 独立 Provider 端点声明的请求模型,仅用于观测,不参与文本模型目录路由
pub requested_model: Option<PublicModelId>,
/// 客户端原始请求中的推理强度
/// Provider 从客户端请求与续接配置中解析的推理强度
pub reasoning_effort: Option<String>,
pub reasoning_preset: Option<String>,
pub request_kind: Option<String>,
@@ -3,6 +3,7 @@
use std::fmt;
use std::sync::{Arc, Mutex, Weak};
use crate::lifecycle::CancellationToken;
use async_trait::async_trait;
/// Provider 实现的连接控制通道,消息内容由上游协议解释
@@ -22,6 +23,7 @@ pub struct ResponseControlUnavailable;
#[derive(Clone, Default)]
pub struct ResponseControl {
target: Arc<Mutex<Option<Weak<dyn ResponseControlTransport>>>>,
connection_cancellation: Option<CancellationToken>,
}
/// 某次已交付响应所属的弱连接快照,重绑后不能把旧消息发送到新连接
@@ -50,6 +52,18 @@ impl fmt::Debug for ResponseControl {
}
impl ResponseControl {
/// 下游连接跨多轮存续,不能使用单次响应的取消信号代替
#[must_use]
pub fn with_connection_cancellation(mut self, cancellation: CancellationToken) -> Self {
self.connection_cancellation = Some(cancellation);
self
}
#[must_use]
pub fn connection_cancellation(&self) -> Option<&CancellationToken> {
self.connection_cancellation.as_ref()
}
/// 仅保存弱引用,不延长旧 attempt 的连接生命周期
#[must_use]
pub fn binding(&self) -> Option<ResponseControlBinding> {
@@ -496,6 +496,7 @@ pub struct ProviderError {
#[derive(Clone, Default)]
struct ProviderErrorUpstreamValues {
failure_fields: Option<crate::upstream::UpstreamFailureFields>,
code: Option<OpaqueUpstreamValue>,
request_id: Option<OpaqueUpstreamValue>,
}
@@ -546,6 +547,21 @@ impl ProviderError {
self
}
/// 明确上游来源;本地错误不得使用这个入口构造匹配事实
#[must_use]
pub fn with_upstream_failure_fields(
mut self,
fields: crate::upstream::UpstreamFailureFields,
) -> Self {
self.upstream_values_mut().failure_fields = Some(fields);
self
}
#[must_use]
pub fn upstream_failure_fields(&self) -> Option<&crate::upstream::UpstreamFailureFields> {
self.upstream_values.as_ref()?.failure_fields.as_ref()
}
/// 保留原始上游错误 code,不参与普通格式化或替代稳定分类
#[must_use]
pub fn with_upstream_code(mut self, code: OpaqueUpstreamValue) -> Self {
+20 -5
View File
@@ -64,6 +64,7 @@ pub struct CoreStorePorts {
budget: Option<Arc<dyn engine::budget::ClientBudgetPort>>,
session_affinity: Option<Arc<dyn ProviderSessionAffinityPort>>,
client_retry: Option<Arc<dyn engine::client_retry::ClientRetryStore>>,
account_error_protection: Option<Arc<dyn crate::provider_ports::ProviderCooldownPort>>,
}
impl CoreStorePorts {
@@ -94,6 +95,7 @@ impl CoreStorePorts {
budget: None,
session_affinity: None,
client_retry: None,
account_error_protection: None,
}
}
@@ -103,6 +105,15 @@ impl CoreStorePorts {
self
}
#[must_use]
pub fn with_account_error_protection(
mut self,
store: Arc<dyn crate::provider_ports::ProviderCooldownPort>,
) -> Self {
self.account_error_protection = Some(store);
self
}
#[must_use]
pub fn with_client_retry_store(
mut self,
@@ -224,6 +235,9 @@ fn build_execution_service(
if let Some(budget) = &ports.budget {
service = service.with_budget(Arc::clone(budget));
}
if let Some(store) = &ports.account_error_protection {
service = service.with_account_error_protection(Arc::clone(store));
}
if let Some(store) = &ports.client_retry {
service = service.with_client_retry_store(Arc::clone(store));
}
@@ -515,11 +529,12 @@ pub fn prepare_control_plane(
}
let compiler = Arc::new(compiler);
let snapshots = RuntimeSnapshotHandle::default();
let publisher = Arc::new(RuntimeSnapshotPublisher::new(
compiler,
snapshots.clone(),
ports.snapshot_subscriptions,
));
let mut publisher =
RuntimeSnapshotPublisher::new(compiler, snapshots.clone(), ports.snapshot_subscriptions);
if let Some(store) = ports.account_error_protection {
publisher = publisher.with_account_error_protection(store);
}
let publisher = Arc::new(publisher);
CoreControlPlaneStartup {
snapshots,
publisher,
@@ -100,7 +100,8 @@ impl CodexClientMinVersions {
min: min.clone(),
});
};
if current < min {
// 最低版本按 SemVer 优先级比较,构建元数据不影响准入
if current.0.cmp_precedence(&min.0).is_lt() {
return Err(ClientVersionRejection::TooOld {
kind,
current: current.clone(),
@@ -1,4 +1,4 @@
//! Provider 冷却、容量窗口和运行状态端口
//! Provider 冷却与账号错误保护的运行状态端口
use super::ProviderStoreError;
/// 账号级 cooldown 的来源类别;调度状态统一按 `rate_limited` 处理
@@ -8,7 +8,7 @@ use crate::{
routing::UpstreamModelId,
};
use futures::future::BoxFuture;
use std::time::{Duration, SystemTime};
use std::time::SystemTime;
/// 可丢失的账号冷却事实,不进入持久状态;探测冻结到期后仍需确认恢复
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -152,7 +152,63 @@ impl ProviderScopedCooldown {
}
}
/// 单次实际上游尝试开始时捕获的反馈栅栏
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct AccountProtectionGuard {
pub account_id: ProviderAccountId,
pub credential_revision: CredentialRevision,
pub policy_generation: String,
pub reset_generation: String,
pub evidence_version: u64,
}
pub trait ProviderCooldownPort: Send + Sync {
/// 由配置发布边界调用,只有保护配置变化才推进策略代次
fn publish_protection_policy<'a>(
&'a self,
_policy: &'a crate::account::AccountErrorProtection,
_revision: crate::routing::ConfigRevision,
) -> BoxFuture<'a, Result<(), ProviderStoreError>> {
Box::pin(async { Ok(()) })
}
fn begin_protection<'a>(
&'a self,
_account_id: &'a ProviderAccountId,
_revision: CredentialRevision,
_fingerprint: &'a str,
_snapshot_revision: crate::routing::ConfigRevision,
) -> BoxFuture<'a, Result<Option<AccountProtectionGuard>, ProviderStoreError>> {
Box::pin(async { Ok(None) })
}
/// 凭据提交后只使旧凭据保护反馈失效,保留账号与模型限流
fn advance_protection_revision<'a>(
&'a self,
_account_id: &'a ProviderAccountId,
_revision: CredentialRevision,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async { Ok(false) })
}
/// 同一原子操作内递增窗口计数并冻结,活动冻结不被迟到反馈顺延
fn record_protection_failure<'a>(
&'a self,
_guard: &'a AccountProtectionGuard,
_policy: &'a crate::account::AccountErrorProtection,
_matched: &'a [crate::account::AccountProtectionMatch],
) -> BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async { Ok(false) })
}
/// 只清除开始时所见的未冻结证据,不解除冻结或清除较新的失败
fn record_protection_success<'a>(
&'a self,
_guard: &'a AccountProtectionGuard,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async { Ok(false) })
}
fn put_if_later(
&self,
cooldown: ProviderCooldown,
@@ -194,26 +250,10 @@ pub trait ProviderCooldownPort: Send + Sync {
account_id: &'a ProviderAccountId,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>>;
/// 记录一次容量类失败并返回滑动窗口内的累计次数,同时把观测到的账号
/// 在途并发并入窗口峰值(`in_flight` 为 0 表示本次未观测,跳过峰值更新)
/// 只服务容量熔断触发器;调用频率受失败频率约束,不需要批量接口
fn record_capacity_failure<'a>(
&'a self,
account_id: &'a ProviderAccountId,
window: Duration,
in_flight: u32,
) -> BoxFuture<'a, Result<u32, ProviderStoreError>>;
/// 普通请求成功后原子清除临时限流及失败证据;必须保留任何容量冻结
/// 普通请求成功只清除临时限流,保护证据由带栅栏的专用反馈负责
fn clear_after_success<'a>(
&'a self,
account_id: &'a ProviderAccountId,
through_revision: CredentialRevision,
) -> BoxFuture<'a, Result<(), ProviderStoreError>>;
/// 读取窗口内观测到的在途并发峰值;无证据时返回 `None`
fn capacity_peak_in_flight<'a>(
&'a self,
account_id: &'a ProviderAccountId,
) -> BoxFuture<'a, Result<Option<u32>, ProviderStoreError>>;
}
@@ -22,15 +22,14 @@ mod credential;
pub use credential::{ProviderCredentialState, ProviderCredentialStatePort};
mod cooldown;
pub use cooldown::{
ProviderCooldown, ProviderCooldownKind, ProviderCooldownPort, ProviderCooldownScope,
ProviderScopedCooldown,
AccountProtectionGuard, ProviderCooldown, ProviderCooldownKind, ProviderCooldownPort,
ProviderCooldownScope, ProviderScopedCooldown,
};
mod refresh;
pub use refresh::{ProviderRefreshPolicy, provider_refresh_backoff_at, provider_refresh_retry_at};
mod runtime_policy;
pub use runtime_policy::{
ProviderFreezePolicy, ProviderRuntimePolicyPort, ProviderWarmupPolicy,
valid_warmup_schedule_time,
ProviderRuntimePolicyPort, ProviderWarmupPolicy, valid_warmup_schedule_time,
};
mod oauth;
pub use oauth::{
@@ -3,7 +3,6 @@
use super::{ProviderRefreshPolicy, ProviderStoreError, ProviderStoreErrorKind};
use crate::{account::OpaqueProviderData, identity::ProviderKind, routing::ConfigRevision};
use futures::future::BoxFuture;
use std::time::Duration;
pub trait ProviderRuntimePolicyPort: Send + Sync {
/// 原子推进预热执行游标,重启或时钟回拨后不重复领取已消费的时刻
@@ -45,13 +44,6 @@ pub trait ProviderRuntimePolicyPort: Send + Sync {
&self,
) -> BoxFuture<'_, Result<ProviderRefreshPolicy, ProviderStoreError>>;
/// 读取账号容量熔断策略;默认关闭,只有实现运行时设置的存储需要覆盖
fn load_freeze_policy(
&self,
) -> BoxFuture<'_, Result<ProviderFreezePolicy, ProviderStoreError>> {
Box::pin(async move { Ok(ProviderFreezePolicy::disabled()) })
}
/// 读取账号模型预激活策略;默认关闭,只有实现运行时设置的存储需要覆盖
fn load_warmup_policy(
&self,
@@ -60,141 +52,6 @@ pub trait ProviderRuntimePolicyPort: Send + Sync {
}
}
/// 账号容量熔断(自动冻结)策略;来源于 `runtime_settings`,
/// 由 Provider 触发路径与恢复 worker 共享同一份配置事实
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ProviderFreezePolicy {
enabled: bool,
threshold: u32,
window: Duration,
freeze_duration: Duration,
probe_enabled: bool,
probe_model: Option<String>,
adaptive_concurrency: bool,
}
impl ProviderFreezePolicy {
/// 管理写入与运行策略构造共用的冻结约束
pub fn validate_values(
threshold: u32,
window_seconds: u64,
duration_seconds: u64,
probe_model: Option<&str>,
) -> Result<(), &'static str> {
for (valid, field) in [
(
(2..=1_000).contains(&threshold),
"account_auto_freeze_threshold",
),
(
(60..=3_600).contains(&window_seconds),
"account_auto_freeze_window_seconds",
),
(
(300..=604_800).contains(&duration_seconds),
"account_auto_freeze_duration_seconds",
),
(
valid_optional_probe_model(probe_model),
"account_auto_freeze_probe_model",
),
] {
if !valid {
return Err(field);
}
}
Ok(())
}
/// 边界与迁移 `0010_account_auto_freeze.sql` 的 check 约束一致;
/// store 层写入前已校验,这里兜底防御越界配置
pub fn try_new(
enabled: bool,
threshold: u32,
window_seconds: u64,
freeze_duration_seconds: u64,
probe_enabled: bool,
probe_model: Option<String>,
adaptive_concurrency: bool,
) -> Result<Self, ProviderStoreError> {
if Self::validate_values(
threshold,
window_seconds,
freeze_duration_seconds,
probe_model.as_deref(),
)
.is_err()
{
return Err(ProviderStoreError::new(
ProviderStoreErrorKind::InvalidData,
"validate freeze policy",
));
}
Ok(Self {
enabled,
threshold,
window: Duration::from_secs(window_seconds),
freeze_duration: Duration::from_secs(freeze_duration_seconds),
probe_enabled,
probe_model,
adaptive_concurrency,
})
}
/// 功能关闭时的全零策略;触发路径与 worker 都以此短路
#[must_use]
pub const fn disabled() -> Self {
Self {
enabled: false,
threshold: 2,
window: Duration::from_secs(60),
freeze_duration: Duration::from_secs(300),
probe_enabled: false,
probe_model: None,
adaptive_concurrency: false,
}
}
#[must_use]
pub const fn enabled(&self) -> bool {
self.enabled
}
/// 窗口内触发冻结的请求级失败次数阈值
#[must_use]
pub const fn threshold(&self) -> u32 {
self.threshold
}
/// 失败计数滑动窗口;每次失败都会顺延窗口
#[must_use]
pub const fn window(&self) -> Duration {
self.window
}
/// 冻结时长;探测失败后的顺延也使用该值
#[must_use]
pub const fn freeze_duration(&self) -> Duration {
self.freeze_duration
}
#[must_use]
pub const fn probe_enabled(&self) -> bool {
self.probe_enabled
}
/// 探测模型;`None` 表示由 worker 选择账号可用的第一个模型
#[must_use]
pub fn probe_model(&self) -> Option<&str> {
self.probe_model.as_deref()
}
#[must_use]
pub const fn adaptive_concurrency(&self) -> bool {
self.adaptive_concurrency
}
}
/// 校验每日预激活时间格式,如 "08:00" 或 "08:00,13:00"
#[must_use]
pub fn valid_warmup_schedule_time(value: &str) -> bool {
@@ -561,6 +561,8 @@ impl ProviderCandidate {
/// 一次请求冻结的 Provider 尝试顺序
#[derive(Debug, Clone)]
pub struct RoutingPlan {
account_error_protection: Arc<crate::account::CompiledAccountErrorProtection>,
account_error_protection_revision: u64,
request_body_limit: usize,
pricing: Arc<crate::metering::PricingOverrides>,
request_location: Option<crate::account::RequestLocation>,
@@ -574,6 +576,16 @@ pub struct RoutingPlan {
}
impl RoutingPlan {
#[must_use]
pub const fn account_error_protection_revision(&self) -> u64 {
self.account_error_protection_revision
}
#[must_use]
pub fn account_error_protection(&self) -> Arc<crate::account::CompiledAccountErrorProtection> {
Arc::clone(&self.account_error_protection)
}
#[must_use]
pub const fn request_body_limit(&self) -> usize {
self.request_body_limit
@@ -1349,6 +1349,11 @@ impl RuntimeSnapshot {
pricing: Arc::clone(&self.settings.values.pricing),
request_location: self.settings.request_location.clone(),
privacy: self.settings.privacy.clone(),
account_error_protection: Arc::clone(&self.settings.account_error_protection),
account_error_protection_revision: self
.settings
.values
.account_error_protection_revision,
account_selection_policy: self.settings.account_selection_policy,
operation: operation.kind(),
max_attempts: NonZeroU32::new(super::MAX_REQUEST_ATTEMPTS)
@@ -1431,6 +1436,11 @@ impl RuntimeSnapshot {
pricing: Arc::clone(&self.settings.values.pricing),
request_location: self.settings.request_location.clone(),
privacy: self.settings.privacy.clone(),
account_error_protection: Arc::clone(&self.settings.account_error_protection),
account_error_protection_revision: self
.settings
.values
.account_error_protection_revision,
account_selection_policy: self.settings.account_selection_policy,
operation: operation.kind(),
max_attempts: NonZeroU32::new(super::MAX_REQUEST_ATTEMPTS)
@@ -141,6 +141,7 @@ pub struct RuntimeSnapshotPublisher {
compiler: Arc<RuntimeSnapshotCompiler>,
snapshots: RuntimeSnapshotHandle,
subscriptions: Arc<dyn SnapshotSubscriptionPort>,
account_error_protection: Option<Arc<dyn crate::provider_ports::ProviderCooldownPort>>,
refresh_lock: Arc<Mutex<()>>,
refresh_priority: Arc<SyncMutex<RefreshPriorityState>>,
}
@@ -162,11 +163,21 @@ impl RuntimeSnapshotPublisher {
compiler,
snapshots,
subscriptions,
account_error_protection: None,
refresh_lock: Arc::new(Mutex::new(())),
refresh_priority: Arc::new(SyncMutex::new(RefreshPriorityState::default())),
}
}
#[must_use]
pub fn with_account_error_protection(
mut self,
store: Arc<dyn crate::provider_ports::ProviderCooldownPort>,
) -> Self {
self.account_error_protection = Some(store);
self
}
/// 串行重编译并替换本进程快照;无法确认配置时暂停新请求
pub async fn refresh(&self) -> Result<ConfigRevision, RuntimeSnapshotCompileError> {
self.refresh_with_mode(RefreshMode::Required).await
@@ -217,11 +228,13 @@ impl RuntimeSnapshotPublisher {
);
if !changed
&& !self.provider_catalogs_need_refresh()
&& self
.snapshots
.acquire()
.is_ok_and(|snapshot| snapshot.extensions().is_none_or(|set| set.is_ready()))
&& let Ok(snapshot) = self.snapshots.acquire()
&& snapshot.extensions().is_none_or(|set| set.is_ready())
{
// 周期对账恢复可能丢失的 Redis gate,不因此重新编译目录或暂停现有配置
if !self.publish_protection_policy(&snapshot).await {
tracing::warn!("account protection policy reconciliation is unavailable");
}
return Ok(persisted_revision);
}
changed
@@ -254,10 +267,47 @@ impl RuntimeSnapshotPublisher {
}
};
let revision = snapshot.revision();
let protection_changed = self
.snapshots
.snapshot_for_diagnostics()
.is_none_or(|previous| {
previous.settings().account_error_protection_revision()
!= snapshot.settings().account_error_protection_revision()
});
if (protection_changed || matches!(mode, RefreshMode::Reconcile))
&& !self.publish_protection_policy(&snapshot).await
{
if protection_changed {
// 真正的策略替换必须先撤销旧反馈资格,继续沿用已有 pending 发布语义
self.snapshots.suspend();
return Err(RuntimeSnapshotCompileError::StoreUnavailable);
}
tracing::warn!("account protection policy reconciliation is unavailable");
}
self.snapshots.publish(snapshot);
Ok(revision)
}
async fn publish_protection_policy(&self, snapshot: &crate::routing::RuntimeSnapshot) -> bool {
let Some(store) = &self.account_error_protection else {
return true;
};
let Ok(revision) =
ConfigRevision::new(snapshot.settings().account_error_protection_revision())
else {
return false;
};
let publish = store
.publish_protection_policy(snapshot.settings().account_error_protection(), revision)
.fuse();
let timeout = Delay::new(Duration::from_secs(1)).fuse();
pin_mut!(publish, timeout);
select_biased! {
result = publish => result.is_ok(),
_ = timeout => false,
}
}
#[must_use]
pub fn published_revision(&self) -> Option<ConfigRevision> {
self.snapshots.revision()
@@ -13,6 +13,8 @@ use crate::{
#[derive(Debug, Clone)]
pub(crate) struct CompiledSettings {
pub(crate) values: SettingsValues,
pub(crate) account_error_protection:
std::sync::Arc<crate::account::CompiledAccountErrorProtection>,
pub(crate) privacy: Option<std::sync::Arc<dyn super::privacy::CompiledPrivacyPolicy>>,
pub(crate) account_selection_policy: AccountSelectionPolicy,
pub(crate) client_queue_policy: ConcurrencyQueuePolicy,
@@ -23,10 +25,29 @@ pub(crate) struct CompiledSettings {
impl CompiledSettings {
pub(crate) fn new(
settings: SettingsValues,
mut settings: SettingsValues,
compiler: Option<&dyn super::privacy::PrivacyPolicyCompiler>,
previous: Option<&Self>,
) -> Result<Self, InvalidSettings> {
if let Some(previous) = previous {
settings.account_error_protection_revision =
previous.values.account_error_protection_revision;
}
if settings.account_error_protection_revision == 0 {
return Err(InvalidSettings);
}
settings
.account_error_protection
.validate()
.map_err(|_| InvalidSettings)?;
let account_error_protection = previous
.filter(|previous| {
previous.values.account_error_protection == settings.account_error_protection
})
.map_or_else(
|| std::sync::Arc::new(settings.account_error_protection.compile()),
|previous| previous.account_error_protection.clone(),
);
let privacy = if let Some(previous) = previous.filter(|previous| {
previous.values.codex_privacy_policy == settings.codex_privacy_policy
}) {
@@ -100,6 +121,7 @@ impl CompiledSettings {
}
}
Ok(Self {
account_error_protection,
account_selection_policy,
client_queue_policy,
responses_max_decompressed_body_bytes,
@@ -8,6 +8,10 @@ use crate::identity::ProviderKind;
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SettingsValues {
#[serde(default)]
pub(crate) account_error_protection: crate::account::AccountErrorProtection,
#[serde(skip, default = "initial_protection_revision")]
pub(crate) account_error_protection_revision: u64,
pub(crate) pricing: Arc<crate::metering::PricingOverrides>,
pub(crate) request_profiles: Arc<BTreeMap<ProviderKind, crate::account::OpaqueProviderData>>,
pub(crate) codex_privacy_policy: super::privacy::CodexPrivacyPolicy,
@@ -29,7 +33,36 @@ pub struct SettingsValues {
pub(crate) min_codex_cli_version: Option<String>,
}
fn initial_protection_revision() -> u64 {
1
}
impl SettingsValues {
#[must_use]
pub const fn with_account_error_protection_revision(mut self, revision: u64) -> Self {
self.account_error_protection_revision = revision;
self
}
#[must_use]
pub const fn account_error_protection_revision(&self) -> u64 {
self.account_error_protection_revision
}
#[must_use]
pub fn with_account_error_protection(
mut self,
policy: crate::account::AccountErrorProtection,
) -> Self {
self.account_error_protection = policy;
self
}
#[must_use]
pub const fn account_error_protection(&self) -> &crate::account::AccountErrorProtection {
&self.account_error_protection
}
#[must_use]
pub fn with_codex_privacy_policy(mut self, policy: super::privacy::CodexPrivacyPolicy) -> Self {
self.codex_privacy_policy = policy;
@@ -134,6 +167,8 @@ impl SettingsValues {
min_codex_cli_version: Option<String>,
) -> Self {
Self {
account_error_protection: Default::default(),
account_error_protection_revision: 1,
request_profiles: Arc::default(),
pricing: Arc::default(),
codex_privacy_policy: Default::default(),
@@ -85,3 +85,22 @@ pub enum AttemptTransport {
/// 使用 Provider 定义的备用传输
Fallback,
}
/// Provider 从实际失败响应或错误事件提取的原始结构化字段
/// 不承载本地错误、状态推导、客户端文案改写或非结构化正文
#[derive(Clone, PartialEq, Eq)]
pub struct UpstreamFailureFields {
pub status: Option<u16>,
pub code: Option<String>,
pub error_type: Option<String>,
pub message: Option<String>,
}
impl fmt::Debug for UpstreamFailureFields {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("UpstreamFailureFields")
.field("status", &self.status)
.finish_non_exhaustive()
}
}
@@ -0,0 +1,181 @@
//! 账号保护的任意错误规则、缺失语义与匹配资源上限
use gateway_core::account::*;
fn sample_policy() -> AccountErrorProtection {
serde_json::from_value(serde_json::json!({
"enabled":true,"threshold":2,"windowSeconds":60,"durationSeconds":300,
"probeEnabled":true,"probeModel":null,
"rules":[{"id":"custom-code","name":"自定义拒绝","conditions":[
{"field":"provider","operator":"equals","values":["vendor"],"ignoreCase":false},
{"field":"status","operator":"equals","values":["400","409"],"ignoreCase":false},
{"field":"code","operator":"equals","values":["plain_custom_error"],"ignoreCase":true}
]}]
}))
.unwrap()
}
fn facts() -> AccountErrorFacts<'static> {
AccountErrorFacts {
provider: "vendor",
status: Some(400),
code: Some(" PLAIN_CUSTOM_ERROR "),
error_type: None,
message: Some("anything"),
}
}
#[test]
fn rules_share_or_candidates_and_require_every_condition() {
let mut policy = sample_policy();
policy.rules.push(AccountErrorRule {
id: "message".into(),
name: "消息片段".into(),
conditions: vec![AccountErrorCondition {
field: AccountErrorField::Message,
operator: AccountErrorOperator::Contains,
values: vec![Some("THING".into())],
ignore_case: true,
}],
});
assert!(policy.validate().is_ok());
let compiled = policy.compile();
assert_eq!(compiled.matching_rules(facts()).len(), 2);
assert_eq!(
compiled
.matching_rules(AccountErrorFacts {
status: Some(401),
..facts()
})
.len(),
1
);
assert_eq!(
compiled
.matching_rules(AccountErrorFacts {
provider: "elsewhere",
message: None,
..facts()
})
.len(),
0
);
}
#[test]
fn absent_empty_and_case_are_explicit() {
let mut policy = sample_policy();
policy.rules[0].conditions = vec![AccountErrorCondition {
field: AccountErrorField::Type,
operator: AccountErrorOperator::Equals,
values: vec![None],
ignore_case: false,
}];
assert_eq!(policy.matching_rules(facts()).len(), 1);
assert!(
policy
.matching_rules(AccountErrorFacts {
error_type: Some(""),
..facts()
})
.is_empty()
);
policy.rules[0].conditions[0].values = vec![Some("".into())];
assert!(policy.matching_rules(facts()).is_empty());
assert_eq!(
policy
.matching_rules(AccountErrorFacts {
error_type: Some(" "),
..facts()
})
.len(),
1
);
}
#[test]
fn message_contains_is_bounded_but_equals_uses_complete_trimmed_value() {
let mut policy = sample_policy();
policy.rules[0].conditions = vec![AccountErrorCondition {
field: AccountErrorField::Message,
operator: AccountErrorOperator::Contains,
values: vec![Some("tail".into())],
ignore_case: false,
}];
let long = format!("{}tail", "x".repeat(8192));
assert!(
policy
.matching_rules(AccountErrorFacts {
message: Some(&long),
..facts()
})
.is_empty()
);
let unicode = format!("{}tail", "界".repeat(2730));
assert!(
policy
.matching_rules(AccountErrorFacts {
message: Some(&unicode),
..facts()
})
.is_empty()
);
policy.rules[0].conditions[0].operator = AccountErrorOperator::Equals;
let padded = format!("{}tail", " ".repeat(9000));
assert_eq!(
policy
.matching_rules(AccountErrorFacts {
message: Some(&padded),
..facts()
})
.len(),
1
);
let full = format!("tail{}", "x".repeat(8192));
assert!(
policy
.matching_rules(AccountErrorFacts {
message: Some(&full),
..facts()
})
.is_empty()
);
}
#[test]
fn validation_rejects_broad_rules_and_excessive_text() {
let mut policy = sample_policy();
policy.rules[0].conditions.truncate(1);
assert!(policy.validate().is_err());
let mut policy = sample_policy();
policy.rules[0].conditions = vec![AccountErrorCondition {
field: AccountErrorField::Message,
operator: AccountErrorOperator::Contains,
values: vec![None],
ignore_case: false,
}];
assert!(policy.validate().is_err());
policy.rules[0].conditions[0].values = vec![Some(" ".into())];
assert!(policy.validate().is_err());
policy.rules[0].conditions[0].values = vec![Some("x".repeat(1000)); 16];
policy.rules[0].conditions = vec![policy.rules[0].conditions[0].clone(); 4];
assert!(policy.validate().is_ok());
let condition = policy.rules[0].conditions[0].clone();
policy.rules[0].conditions.push(condition);
assert!(policy.validate().is_err());
}
#[test]
fn policy_fingerprint_tracks_every_semantic_setting() {
let policy = sample_policy();
assert_eq!(policy.fingerprint(), policy.compile().fingerprint());
let mut changed = policy.clone();
changed.enabled = false;
assert_ne!(policy.fingerprint(), changed.fingerprint());
let mut changed = policy.clone();
changed.rules[0].conditions[0].ignore_case = true;
assert_ne!(policy.fingerprint(), changed.fingerprint());
let mut changed = policy.clone();
changed.probe_model = Some("probe-model".into());
assert_ne!(policy.fingerprint(), changed.fingerprint());
}
@@ -7,6 +7,8 @@ mod selection;
mod selection_diagnostics;
mod smart_scheduling;
mod error_protection;
use std::collections::{BTreeMap, BTreeSet};
use std::num::NonZeroU32;
use std::sync::Arc;
@@ -0,0 +1,373 @@
//! 验证单次尝试的保护反馈、原始错误保留及慢存储与取消边界
use futures::{StreamExt, future::BoxFuture, stream};
use gateway_core::{
account::{
AccountErrorCondition, AccountErrorField, AccountErrorOperator, AccountErrorProtection,
AccountErrorRule, AccountProtectionMatch, CredentialRevision, ProviderAccountId,
},
engine::provider::{ProviderCallMetadata, ProviderStream},
error::{ProviderError, ProviderErrorKind},
provider_ports::{
AccountProtectionGuard, ProviderCooldown, ProviderCooldownPort, ProviderCooldownScope,
ProviderScopedCooldown, ProviderStoreError,
},
routing::{ConfigRevision, ProviderKind, UpstreamModelId},
upstream::{UpstreamFailureFields, UpstreamSendState, UpstreamTransport},
};
use std::{
sync::{
Arc, Mutex,
atomic::{AtomicBool, AtomicUsize, Ordering},
},
time::{Duration, Instant},
};
#[derive(Default)]
pub(crate) struct Feedback {
pub(crate) publications: Mutex<Vec<u64>>,
pub(crate) publish_fails: AtomicBool,
pub(super) begin: AtomicUsize,
pub(super) successes: AtomicUsize,
pub(super) failures: Mutex<Vec<(AccountProtectionGuard, Vec<AccountProtectionMatch>)>>,
stalled_begin: bool,
stalled_feedback: bool,
stale: bool,
}
impl ProviderCooldownPort for Feedback {
fn publish_protection_policy<'a>(
&'a self,
_: &'a AccountErrorProtection,
revision: ConfigRevision,
) -> BoxFuture<'a, Result<(), ProviderStoreError>> {
Box::pin(async move {
self.publications.lock().unwrap().push(revision.get());
if self.publish_fails.load(Ordering::SeqCst) {
Err(ProviderStoreError::new(
gateway_core::provider_ports::ProviderStoreErrorKind::Unavailable,
"test_publish",
))
} else {
Ok(())
}
})
}
fn begin_protection<'a>(
&'a self,
account_id: &'a ProviderAccountId,
revision: CredentialRevision,
fingerprint: &'a str,
_: ConfigRevision,
) -> BoxFuture<'a, Result<Option<AccountProtectionGuard>, ProviderStoreError>> {
Box::pin(async move {
self.begin.fetch_add(1, Ordering::SeqCst);
if self.stalled_begin {
return futures::future::pending().await;
}
Ok((!self.stale).then(|| AccountProtectionGuard {
account_id: account_id.clone(),
credential_revision: revision,
policy_generation: fingerprint.to_owned(),
reset_generation: "epoch".into(),
evidence_version: 0,
}))
})
}
fn record_protection_failure<'a>(
&'a self,
guard: &'a AccountProtectionGuard,
_: &'a AccountErrorProtection,
matched: &'a [AccountProtectionMatch],
) -> BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async move {
if self.stalled_feedback {
return futures::future::pending().await;
}
self.failures
.lock()
.unwrap()
.push((guard.clone(), matched.to_vec()));
Ok(true)
})
}
fn record_protection_success<'a>(
&'a self,
_: &'a AccountProtectionGuard,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async move {
self.successes.fetch_add(1, Ordering::SeqCst);
Ok(true)
})
}
fn put_if_later(&self, _: ProviderCooldown) -> BoxFuture<'_, Result<bool, ProviderStoreError>> {
Box::pin(async { Ok(false) })
}
fn read<'a>(
&'a self,
_: &'a ProviderAccountId,
) -> BoxFuture<'a, Result<Option<ProviderCooldown>, ProviderStoreError>> {
Box::pin(async { Ok(None) })
}
fn clear<'a>(
&'a self,
_: &'a ProviderAccountId,
_: CredentialRevision,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async { Ok(false) })
}
fn put_scoped_if_later(
&self,
_: ProviderScopedCooldown,
) -> BoxFuture<'_, Result<bool, ProviderStoreError>> {
Box::pin(async { Ok(false) })
}
fn read_scoped<'a>(
&'a self,
_: &'a ProviderAccountId,
_: &'a ProviderCooldownScope,
) -> BoxFuture<'a, Result<Option<ProviderScopedCooldown>, ProviderStoreError>> {
Box::pin(async { Ok(None) })
}
fn clear_scoped<'a>(
&'a self,
_: &'a ProviderAccountId,
_: &'a ProviderCooldownScope,
_: CredentialRevision,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async { Ok(false) })
}
fn clear_all<'a>(
&'a self,
_: &'a ProviderAccountId,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async { Ok(false) })
}
fn clear_after_success<'a>(
&'a self,
_: &'a ProviderAccountId,
_: CredentialRevision,
) -> BoxFuture<'a, Result<(), ProviderStoreError>> {
Box::pin(async { Ok(()) })
}
}
pub(super) fn policy() -> Arc<AccountErrorProtection> {
Arc::new(AccountErrorProtection {
enabled: true,
rules: vec![AccountErrorRule {
id: "prompt".into(),
name: "自定义错误".into(),
conditions: vec![AccountErrorCondition {
field: AccountErrorField::Code,
operator: AccountErrorOperator::Equals,
values: vec![Some("invalid_prompt".into())],
ignore_case: false,
}],
}],
..AccountErrorProtection::default()
})
}
fn metadata() -> ProviderCallMetadata {
ProviderCallMetadata::new(
ProviderKind::new("openai").unwrap(),
UpstreamModelId::new("model").unwrap(),
ProviderAccountId::new("acct_protection").unwrap(),
UpstreamTransport::new("http_sse").unwrap(),
)
.with_credential_revision(CredentialRevision::new(7).unwrap())
}
fn failure(code: &str) -> ProviderError {
ProviderError::new(ProviderErrorKind::InvalidRequest, UpstreamSendState::Sent)
.with_retry_prohibited()
.with_upstream_failure_fields(UpstreamFailureFields {
status: None,
code: Some(code.into()),
error_type: None,
message: Some("original".into()),
})
}
#[test]
fn custom_rule_records_one_failure_without_changing_error_or_replay() {
futures::executor::block_on(async {
let store = Arc::new(Feedback::default());
let mut policy = policy().as_ref().clone();
let mut other = policy.rules[0].clone();
other.id = "second".into();
policy.rules.push(other);
let mut stream = ProviderStream::new(
metadata(),
stream::iter([Err(failure("invalid_prompt"))]),
(),
)
.with_account_error_protection(store.clone(), Arc::new(policy.compile()), 5)
.await;
let error = stream.next().await.unwrap().unwrap_err();
assert_eq!(error.kind(), ProviderErrorKind::InvalidRequest);
assert!(error.retry_is_prohibited());
assert_eq!(error.upstream_failure_fields().unwrap().status, None);
assert!(stream.next().await.is_none());
assert!(stream.next().await.is_none());
let records = store.failures.lock().unwrap();
assert_eq!(records.len(), 1);
assert_eq!(records[0].1.len(), 2);
assert_eq!(
records[0].0.credential_revision,
CredentialRevision::new(7).unwrap()
);
assert_eq!(store.successes.load(Ordering::SeqCst), 0);
});
}
#[test]
fn local_unmatched_and_stale_attempt_errors_do_not_count() {
futures::executor::block_on(async {
for (error, stale) in [
(
ProviderError::new(ProviderErrorKind::InvalidRequest, UpstreamSendState::Sent)
.with_status(400),
false,
),
(failure("other"), false),
(failure("invalid_prompt"), true),
] {
let store = Arc::new(Feedback {
stale,
..Feedback::default()
});
let mut stream = ProviderStream::new(metadata(), stream::iter([Err(error)]), ())
.with_account_error_protection(store.clone(), Arc::new(policy().compile()), 5)
.await;
assert!(stream.next().await.unwrap().is_err());
assert!(store.failures.lock().unwrap().is_empty());
assert_eq!(store.successes.load(Ordering::SeqCst), 0);
}
});
}
#[test]
fn disabled_protection_performs_no_store_io() {
futures::executor::block_on(async {
let store = Arc::new(Feedback::default());
let mut stream = ProviderStream::new(
metadata(),
stream::iter([Err(failure("invalid_prompt"))]),
(),
)
.with_account_error_protection(
store.clone(),
Arc::new(AccountErrorProtection::default().compile()),
5,
)
.await;
assert!(stream.next().await.unwrap().is_err());
assert_eq!(store.begin.load(Ordering::SeqCst), 0);
assert!(store.failures.lock().unwrap().is_empty());
});
}
#[test]
fn complete_success_is_reported_once_but_unfinished_drop_is_not_success() {
futures::executor::block_on(async {
let store = Arc::new(Feedback::default());
let unfinished = ProviderStream::new(metadata(), stream::pending(), ())
.with_account_error_protection(store.clone(), Arc::new(policy().compile()), 5)
.await;
drop(unfinished);
assert_eq!(store.successes.load(Ordering::SeqCst), 0);
let response = gateway_core::event::ResponseMeta::new("response", "model");
let events = stream::iter([
Ok(gateway_core::event::GatewayEvent::Started(response.clone()).into()),
Ok(gateway_core::event::GatewayEvent::Completed(response).into()),
]);
let mut completed = ProviderStream::new(metadata(), events, ())
.with_account_error_protection(store.clone(), Arc::new(policy().compile()), 5)
.await;
assert!(completed.next().await.unwrap().is_ok());
assert_eq!(store.successes.load(Ordering::SeqCst), 0);
assert!(completed.next().await.unwrap().is_ok());
assert_eq!(store.successes.load(Ordering::SeqCst), 1);
assert!(completed.next().await.is_none());
assert!(completed.next().await.is_none());
assert_eq!(store.successes.load(Ordering::SeqCst), 1);
});
}
struct Lease(Arc<AtomicUsize>);
impl Drop for Lease {
fn drop(&mut self) {
self.0.fetch_add(1, Ordering::SeqCst);
}
}
#[test]
fn stalled_begin_and_feedback_are_bounded_and_preserve_original_failure() {
futures::executor::block_on(async {
for stalled_begin in [false, true] {
let store = Arc::new(Feedback {
stalled_begin,
stalled_feedback: true,
..Feedback::default()
});
let released = Arc::new(AtomicUsize::new(0));
let started = Instant::now();
let mut stream = ProviderStream::new(
metadata(),
stream::iter([Err(failure("invalid_prompt"))]),
Lease(released.clone()),
)
.with_account_error_protection(store, Arc::new(policy().compile()), 5)
.await;
let error = timeout(Duration::from_secs(1), stream.next())
.await
.unwrap()
.unwrap()
.unwrap_err();
assert_eq!(error.kind(), ProviderErrorKind::InvalidRequest);
assert!(started.elapsed() < Duration::from_millis(500));
drop(stream);
assert_eq!(released.load(Ordering::SeqCst), 1);
}
});
}
#[test]
fn cancelling_pending_feedback_releases_lease_and_does_not_report_success() {
futures::executor::block_on(async {
let store = Arc::new(Feedback {
stalled_feedback: true,
..Feedback::default()
});
let released = Arc::new(AtomicUsize::new(0));
let mut stream = ProviderStream::new(
metadata(),
stream::iter([Err(failure("invalid_prompt"))]),
Lease(released.clone()),
)
.with_account_error_protection(store.clone(), Arc::new(policy().compile()), 5)
.await;
assert!(
timeout(Duration::from_millis(10), stream.next())
.await
.is_err()
);
drop(stream);
assert_eq!(released.load(Ordering::SeqCst), 1);
assert_eq!(store.successes.load(Ordering::SeqCst), 0);
});
}
async fn timeout<T>(
duration: Duration,
future: impl std::future::Future<Output = T>,
) -> Result<T, ()> {
let future = Box::pin(future);
match futures::future::select(future, futures_timer::Delay::new(duration)).await {
futures::future::Either::Left((value, _)) => Ok(value),
futures::future::Either::Right(_) => Err(()),
}
}
@@ -718,6 +718,10 @@ impl ExecutionStore for FakeStore {
}
enum Script {
ProtectedStream {
account_id: &'static str,
items: Vec<Result<GatewayEvent, ProviderError>>,
},
Stream {
account_id: &'static str,
items: Vec<Result<GatewayEvent, ProviderError>>,
@@ -823,7 +827,10 @@ impl Provider for ScriptedProvider {
.lock()
.expect("operations lock")
.push(request.operation().clone());
self.contexts.lock().expect("contexts lock").push(context);
self.contexts
.lock()
.expect("contexts lock")
.push(context.clone());
// 模拟官方发布在每次上游尝试开始后推进,重试应继续使用首次解析版本
self.profile_generation.fetch_add(1, Ordering::SeqCst);
let script = self
@@ -832,10 +839,12 @@ impl Provider for ScriptedProvider {
.expect("scripts lock")
.pop_front()
.expect("one script per provider call");
let protected = matches!(script, Script::ProtectedStream { .. });
match script {
Script::Error(error) => Err(error),
Script::Pending => futures::future::pending().await,
Script::Stream { account_id, items } => {
Script::Stream { account_id, items }
| Script::ProtectedStream { account_id, items } => {
let candidate = request.candidate();
let metadata = ProviderCallMetadata::new(
candidate.provider().clone(),
@@ -847,13 +856,29 @@ impl Provider for ScriptedProvider {
UpstreamTransport::new("http_sse").expect("transport"),
)
.with_upstream_request_id(OpaqueUpstreamValue::new("upstream-request"));
Ok(ProviderStream::new(
let metadata = metadata.with_credential_revision(
gateway_core::account::CredentialRevision::new(7).unwrap(),
);
let stream = ProviderStream::new(
metadata,
Box::pin(futures::stream::iter(
items.into_iter().map(canonical_provider_event),
)),
TrackedLease(Arc::clone(&self.released_leases)),
))
);
if protected {
context
.execute_middleware(
request.operation().clone(),
candidate.provider().clone(),
Some("gpt-5".into()),
ProviderAccountId::new(account_id).unwrap(),
Box::new(move |_, _| Box::pin(async move { Ok(stream) })),
)
.await
} else {
Ok(stream)
}
}
Script::HangingStream { account_id, items } => {
let candidate = request.candidate();
@@ -1141,6 +1166,24 @@ fn plan_with_profiles(
request_location: gateway_core::account::RequestLocation,
fast_mode: FastMode,
profiles: BTreeMap<ProviderKind, gateway_core::account::OpaqueProviderData>,
) -> RoutingPlan {
plan_with_protection(
operation,
account_selection_policy,
request_location,
fast_mode,
profiles,
Default::default(),
)
}
fn plan_with_protection(
operation: &Operation,
account_selection_policy: AccountSelectionPolicy,
request_location: gateway_core::account::RequestLocation,
fast_mode: FastMode,
profiles: BTreeMap<ProviderKind, gateway_core::account::OpaqueProviderData>,
protection: gateway_core::account::AccountErrorProtection,
) -> RoutingPlan {
let provider = ProviderKind::new("openai").expect("provider");
let public_model = PublicModelId::new("gpt-5").expect("public model");
@@ -1193,7 +1236,8 @@ fn plan_with_profiles(
)
.unwrap(),
)
.with_request_location(request_location, true),
.with_request_location(request_location, true)
.with_account_error_protection(protection),
vec![provider.clone()],
vec![ProviderModel::new(
provider.clone(),
@@ -7128,3 +7172,87 @@ fn lease_cancellation_is_preserved_in_the_single_request_terminal() {
);
});
}
#[test]
fn error_protection_tracks_real_retry_attempts_and_skips_diagnostic_runs() {
use super::account_protection::{Feedback, policy};
for diagnostic in [false, true] {
let operation = generate_operation();
let route_plan = plan_with_protection(
&operation,
AccountSelectionPolicy::new(RotationStrategy::Smart, NonZeroU32::new(1).unwrap()),
Default::default(),
FastMode::Default,
BTreeMap::new(),
policy().as_ref().clone(),
);
let feedback = Arc::new(Feedback::default());
let failure = ProviderError::new(ProviderErrorKind::Unavailable, UpstreamSendState::Sent)
.with_replay_safe()
.with_upstream_failure_fields(gateway_core::upstream::UpstreamFailureFields {
status: None,
code: Some("invalid_prompt".into()),
error_type: None,
message: None,
});
let provider = Arc::new(ScriptedProvider::new(vec![
Script::ProtectedStream {
account_id: "acct_first",
items: vec![Err(failure)],
},
Script::ProtectedStream {
account_id: "acct_second",
items: complete_stream(None),
},
]));
let mut registry = ProviderRegistry::builder();
registry.register(provider.clone()).unwrap();
let coordinator = AttemptCoordinator::new(
GatewayEngine::new(Arc::new(FakeStore::default()), registry.build())
.with_account_error_protection(feedback.clone()),
);
let request = model_request(&operation, SystemTime::now() + Duration::from_secs(30));
let mut session = block_on(async {
if diagnostic {
coordinator
.start_diagnostic(
request,
operation,
route_plan,
ProviderAccountId::new("acct_first").unwrap(),
None,
CancellationToken::new(),
)
.await
} else {
coordinator
.start(
request,
operation,
route_plan,
None,
None,
CancellationToken::new(),
)
.await
}
})
.unwrap();
if diagnostic {
assert!(block_on(session.next_event()).is_err());
assert_eq!(feedback.begin.load(Ordering::SeqCst), 0);
} else {
block_on(session.collect_uncommitted()).unwrap();
assert_eq!(feedback.begin.load(Ordering::SeqCst), 2);
assert_eq!(feedback.failures.lock().unwrap().len(), 1);
assert_eq!(feedback.successes.load(Ordering::SeqCst), 1);
let observed = feedback.failures.lock().unwrap();
assert_eq!(observed[0].0.account_id.as_str(), "acct_first");
}
drop(session);
assert_eq!(
provider.released_leases.load(Ordering::SeqCst),
if diagnostic { 1 } else { 2 }
);
}
}
@@ -1,5 +1,6 @@
//! 执行引擎各阶段与端口合同的测试入口
pub(crate) mod account_protection;
mod admission;
mod connection;
mod continuation;
+25 -13
View File
@@ -119,17 +119,29 @@ fn min_client_versions_should_reject_missing_and_old_recognized_clients() {
#[test]
fn min_client_versions_should_follow_semver_precedence() {
let min = CodexClientVersion::parse("1.0.0").expect("min version");
let policy = CodexClientMinVersions::new(None, Some(min));
let prerelease = CodexClientVersion::parse("1.0.0-rc.1").expect("prerelease");
let release = CodexClientVersion::parse("1.0.0").expect("release");
let newer = CodexClientVersion::parse("1.0.1").expect("newer release");
assert!(
policy
.enforce(CodexClientKind::Cli, Some(&prerelease))
.is_err()
);
assert_eq!(policy.enforce(CodexClientKind::Cli, Some(&release)), Ok(()));
assert_eq!(policy.enforce(CodexClientKind::Cli, Some(&newer)), Ok(()));
for (minimum, current, allowed) in [
("1.0.0", "1.0.0-rc.1", false),
("1.0.0", "1.0.0", true),
("1.0.0", "1.0.1", true),
("1.0.0-rc.2", "1.0.0-rc.1", false),
("1.0.0-rc.2", "1.0.0-rc.10", true),
("1.0.0+build.2", "1.0.0", true),
("1.0.0", "1.0.0+build.2", true),
("1.0.0+build.2", "1.0.0+build.1", true),
("1.0.0-rc.2+build.2", "1.0.0-rc.2+build.1", true),
("1.0.0+build.1", "1.0.0-rc.1+build.9", false),
("1.0.1+build.1", "1.0.0+build.9", false),
] {
let minimum_version = CodexClientVersion::parse(minimum).unwrap();
let current_version = CodexClientVersion::parse(current).unwrap();
let policy =
CodexClientMinVersions::new(Some(minimum_version.clone()), Some(minimum_version));
for kind in [CodexClientKind::Cli, CodexClientKind::Desktop] {
assert_eq!(
policy.enforce(kind, Some(&current_version)).is_ok(),
allowed,
"{kind:?}: {current} >= {minimum}"
);
}
}
}
@@ -1,6 +1,7 @@
//! 验证运行时快照发布、版本对账与并发更新行为
use std::collections::BTreeMap;
mod protection;
mod publications;
use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering};
use std::sync::{Arc, Mutex};
@@ -0,0 +1,51 @@
//! 验证保护策略发布门控、辅助存储失败与周期对账恢复
use super::*;
use crate::engine::account_protection::Feedback;
fn protection_facts(config: u64, protection_revision: u64) -> SnapshotFacts {
SnapshotFacts::new(
revision(config),
revision(config),
SettingsValues::new(1, "smart", BTreeMap::new(), None, None)
.with_account_error_protection_revision(protection_revision),
Vec::new(),
Vec::new(),
Vec::new(),
Vec::new(),
)
}
#[test]
fn protection_publication_reuses_generation_and_requires_changed_gate() {
block_on(async {
let store = Arc::new(TestSnapshotStore::new(Ok(protection_facts(1, 1))));
let handle = RuntimeSnapshotHandle::default();
let protection = Arc::new(Feedback::default());
let publisher = RuntimeSnapshotPublisher::new(
Arc::new(compiler(store.clone())),
handle.clone(),
Arc::new(TestSnapshotSubscriptions::default()),
)
.with_account_error_protection(protection.clone());
publisher.refresh().await.unwrap();
assert_eq!(*protection.publications.lock().unwrap(), vec![1]);
publisher.refresh().await.unwrap();
assert_eq!(*protection.publications.lock().unwrap(), vec![1]);
protection.publish_fails.store(true, Ordering::SeqCst);
let (task, context) = reconciliation_task(&publisher);
task.run_cycle(context).await.unwrap();
assert!(handle.acquire().is_ok());
assert_eq!(*protection.publications.lock().unwrap(), vec![1, 1]);
*store.facts.lock().unwrap() = Ok(protection_facts(2, 2));
*store.current_revision.lock().unwrap() = Ok(revision(2));
publisher.publish_committed(revision(2)).await;
assert!(handle.acquire().is_err());
assert!(handle.snapshot_for_diagnostics().is_none());
protection.publish_fails.store(false, Ordering::SeqCst);
let (task, context) = reconciliation_task(&publisher);
task.run_cycle(context).await.unwrap();
assert_eq!(handle.acquire().unwrap().revision(), revision(2));
assert_eq!(*protection.publications.lock().unwrap(), vec![1, 1, 2, 2]);
});
}
+13 -4
View File
@@ -166,11 +166,20 @@ pub fn initialize_logging(
.with_span_list(true)
.with_filter(request_dump_file_filter)
});
// 空 layer 会让禁用 target 的 enabled! 探测遗留过滤状态,影响后续事件
let layers: Vec<
Box<dyn tracing_subscriber::Layer<tracing_subscriber::Registry> + Send + Sync>,
> = [
stdout_layer.map(|layer| layer.boxed()),
file_layer.map(|layer| layer.boxed()),
recovery_file_layer.map(|layer| layer.boxed()),
request_dump_file_layer.map(|layer| layer.boxed()),
]
.into_iter()
.flatten()
.collect();
tracing_subscriber::registry()
.with(stdout_layer)
.with(file_layer)
.with(recovery_file_layer)
.with(request_dump_file_layer)
.with(layers)
.try_init()
.map_err(|_| LogError::AlreadyInitialized)?;
Ok(LogGuard {
@@ -265,6 +265,57 @@ fn oauth_recovery_switch_controls_only_its_dedicated_file() {
}
}
#[test]
fn disabled_target_queries_do_not_drop_application_warnings() {
if env::var_os(CHILD_PROCESS_ENV).is_some() {
let mut config = logging_config(
PathBuf::from(env::var_os(LOG_DIRECTORY_ENV).unwrap()),
false,
);
config.logging.level = "info".to_owned();
config.logging.oauth_recovery = false;
with_logging(config, || {
fn warning(index: u64) {
tracing::warn!(target: APPLICATION_LOG_TARGET, disabled_target_probe_index = index);
}
warning(0);
let dump_enabled =
tracing::enabled!(target: REQUEST_DUMP_LOG_TARGET, tracing::Level::INFO);
warning(1);
let recovery_enabled =
tracing::enabled!(target: OAUTH_RECOVERY_LOG_TARGET, tracing::Level::INFO);
warning(2);
assert!(!dump_enabled);
assert!(!recovery_enabled);
});
return;
}
let directory = tempfile::tempdir().unwrap();
let output = Command::new(env::current_exe().unwrap())
.args([
"--exact",
"logging::disabled_target_queries_do_not_drop_application_warnings",
"--nocapture",
])
.env(CHILD_PROCESS_ENV, "1")
.env(LOG_DIRECTORY_ENV, directory.path())
.env("RUST_LOG", "info")
.output()
.unwrap();
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let log = read_log_file_set(directory.path(), APPLICATION_LOG_FILE_PREFIX);
let indices = log
.lines()
.map(|line| serde_json::from_str::<serde_json::Value>(line).unwrap())
.filter_map(|row| row["fields"]["disabled_target_probe_index"].as_u64())
.collect::<std::collections::BTreeSet<_>>();
assert_eq!(indices, std::collections::BTreeSet::from([0, 1, 2]));
}
fn write_sensitive_logs(directory: PathBuf, request_dump: bool) {
with_file_logging(directory, request_dump, || {
tracing::info!(
@@ -6,7 +6,7 @@
{ "capability": "frontend_authentication", "versions": [2] },
{ "capability": "scheduler", "versions": [3] },
{ "capability": "model_router", "versions": [2] },
{ "capability": "model_catalog", "versions": [1, 2] },
{ "capability": "model_catalog", "versions": [2] },
{ "capability": "retry_policy", "versions": [2] },
{ "capability": "middleware", "versions": [6] },
{ "capability": "upstream_adapter", "versions": [5] },
@@ -14,10 +14,7 @@ use gateway_core::{
};
use gateway_plugin_sdk::{
Capability, Manifest, Stage,
call::{
catalog::{ModelCatalogRegistration, ModelCatalogRegistrationV2},
middleware::RequestFeature,
},
call::{catalog::ModelCatalogRegistrationV2, middleware::RequestFeature},
};
use crate::RpcSession;
@@ -52,15 +49,8 @@ pub(crate) async fn prepare(
if reply.payload.len() > 64 * 1024 {
return Err(AdminError::invalid("插件模型目录信封无效或过大"));
}
let (aliases, models) = if manifest.contributes[&Capability::ModelCatalog].version == 2 {
let registration: ModelCatalogRegistrationV2 = serde_json::from_slice(&reply.payload)
.map_err(|_| AdminError::invalid("插件模型目录声明无效"))?;
(registration.aliases, registration.models)
} else {
let registration: ModelCatalogRegistration = serde_json::from_slice(&reply.payload)
.map_err(|_| AdminError::invalid("插件模型目录声明无效"))?;
(registration.models, Vec::new())
};
let ModelCatalogRegistrationV2 { aliases, models } = serde_json::from_slice(&reply.payload)
.map_err(|_| AdminError::invalid("插件模型目录声明无效"))?;
if aliases.len().saturating_add(models.len()) > 256 {
return Err(AdminError::invalid("单个插件最多贡献 256 个模型条目"));
}
@@ -57,6 +57,10 @@ pub(super) fn failure_error(
) -> Result<ProviderError, ProviderError> {
if failure.message.len() > 64 * 1024
|| failure.code.as_ref().is_some_and(|code| code.len() > 256)
|| failure
.error_type
.as_ref()
.is_some_and(|value| value.len() > 256)
|| failure
.status
.is_some_and(|status| !(400..=599).contains(&status))
@@ -81,6 +85,12 @@ pub(super) fn failure_error(
let raw =
serde_json::to_string(&failure).map_err(|source| invalid(sent).with_source(source))?;
let mut error = ProviderError::new(kind, sent)
.with_upstream_failure_fields(gateway_core::upstream::UpstreamFailureFields {
status: failure.status,
code: failure.code.clone(),
error_type: failure.error_type.clone(),
message: Some(failure.message.clone()),
})
.with_raw_upstream_error(RawUpstreamError::new(raw))
.with_diagnostic(
ProviderDiagnostic::new(format!(
@@ -96,7 +106,7 @@ pub(super) fn failure_error(
error = error.with_client_visible_upstream_error(ClientVisibleUpstreamError::new(
failure.message,
failure.code,
None,
failure.error_type,
));
if let Some(status) = failure.status {
error = error.with_status(status);
@@ -12,13 +12,12 @@ use gateway_plugin_sdk::Stage;
#[tokio::test]
async fn adapted_catalog_freezes_adapter_accounts_and_preserves_the_active_generation() {
use gateway_admin::model::plugins::instances::{PluginCapabilityBinding, PluginFailurePolicy};
let mut catalog = crate::support::contribution(
let catalog = crate::support::contribution(
Capability::ModelCatalog,
vec![Stage::Registration],
vec![],
vec![],
);
catalog.1.version = 2;
let adapter = crate::support::contribution(
Capability::UpstreamAdapter,
vec![Stage::Upstream],
@@ -46,7 +45,7 @@ async fn adapted_catalog_freezes_adapter_accounts_and_preserves_the_active_gener
"paths":[{"path":"responses","purpose":"inference"}], "authentication_kinds":["oauth"],
"account_ids":["acct_b"], "models":["shared-model"], "transport":"http_sse","protocol":"openai"
}]},
"model_catalog":{"aliases":[],"models":[{
"model_catalog":{"aliases":[{"id":"native-alias","provider":"openai","model":"native-model"}],"models":[{
"id":"shared-model","adapter_id":"example","features":["tools"],"presentation":{"display_name":"Shared"}
}]}
});
@@ -59,7 +58,8 @@ async fn adapted_catalog_freezes_adapter_accounts_and_preserves_the_active_gener
active.models()[0].accounts.iter().next().unwrap().as_str(),
"acct_b"
);
assert!(active.model_aliases().is_empty());
assert_eq!(active.model_aliases().len(), 1);
assert_eq!(active.model_aliases()[0].id.as_str(), "native-alias");
for scenario in ["accounts", "model", "binding", "key", "group"] {
let mut invalid = snapshot.clone();
invalid.config_revision = Revision::new(2).unwrap();
@@ -168,13 +168,12 @@ async fn adapted_catalog_freezes_adapter_accounts_and_preserves_the_active_gener
#[tokio::test]
async fn adapted_catalog_requires_its_registered_adapter() {
let mut catalog = crate::support::contribution(
let catalog = crate::support::contribution(
Capability::ModelCatalog,
vec![Stage::Registration],
vec![],
vec![],
);
catalog.1.version = 2;
let (cache, store, runtime) =
super::super::setup_with_contributions(Contributions::from([catalog])).await;
let mut snapshot = store.snapshot.lock().unwrap().clone();
@@ -214,7 +213,7 @@ async fn initial_registration_http_uses_the_prepared_settings_without_a_publicat
.await;
store.snapshot.lock().unwrap().instances[0].configuration = serde_json::json!({
"registration_http": {"params": {"method":"POST", "url":server.uri(), "headers":[]}, "body":"x".repeat(64)},
"model_catalog":{"models":[{"id":"fixture-alias","provider":"openai","model":"fixture-upstream"}]}
"model_catalog":{"aliases":[{"id":"fixture-alias","provider":"openai","model":"fixture-upstream"}],"models":[]}
});
let prepared = ExtensionPreparationPort::prepare_with_request_body_limit(
&runtime,
@@ -242,7 +241,7 @@ async fn model_catalog_is_frozen_without_entering_the_request_chain() {
]))
.await;
store.snapshot.lock().unwrap().instances[0].configuration = serde_json::json!({
"model_catalog": {"models": [{"id": "my-model", "provider": "openai", "model": "upstream-model"}]}
"model_catalog": {"aliases": [{"id": "my-model", "provider": "openai", "model": "upstream-model"}], "models": []}
});
let generation = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
.await
@@ -297,7 +296,7 @@ async fn model_catalog_is_frozen_without_entering_the_request_chain() {
#[tokio::test]
async fn model_catalog_prepares_plugin_aliases_without_semantic_validation() {
for models in [
for aliases in [
serde_json::json!([{"id":"alias","provider":"custom","model":"target"}]),
serde_json::json!([{"id":"alias","provider":"openai","model":"alias"}]),
serde_json::json!([
@@ -318,8 +317,8 @@ async fn model_catalog_prepares_plugin_aliases_without_semantic_validation() {
.await;
let mut snapshot = store.snapshot.lock().unwrap().clone();
snapshot.instances[0].configuration =
serde_json::json!({"model_catalog":{"models":models}});
let expected = models.as_array().unwrap().len();
serde_json::json!({"model_catalog":{"aliases":aliases,"models":[]}});
let expected = aliases.as_array().unwrap().len();
let prepared = PluginPreparation::prepare(&runtime, snapshot)
.await
.unwrap();
@@ -340,7 +339,7 @@ async fn model_catalog_prepares_plugin_aliases_without_semantic_validation() {
.await;
let mut snapshot = store.snapshot.lock().unwrap().clone();
snapshot.instances[0].configuration = serde_json::json!({
"model_catalog":{"models":[{"id":"alias","provider":"openai","model":"target"}]}
"model_catalog":{"aliases":[{"id":"alias","provider":"openai","model":"target"}],"models":[]}
});
let mut other = snapshot.instances[0].clone();
other.id = "instance-two".into();
@@ -81,13 +81,20 @@ async fn retired_contracts_remain_loadable_with_compatibility_warnings() {
vec!["openai".into()],
vec!["openai".into()],
);
let mut catalog = crate::support::contribution(
Capability::ModelCatalog,
vec![Stage::Registration],
vec![],
vec![],
);
if legacy {
middleware.1.version = 3;
upstream.1.version = 1;
catalog.1.version = 1;
}
let archive = crate::support::package_with_contributions(
b"not-an-executable",
Contributions::from([middleware, upstream]),
Contributions::from([middleware, upstream, catalog]),
);
let inspector =
PackageInspector::new(PackageLimits::default(), semver::Version::new(1, 0, 0));
@@ -102,6 +109,7 @@ async fn retired_contracts_remain_loadable_with_compatibility_warnings() {
let warning = warning.expect("retired contracts must remain visible");
assert!(warning.contains("middleware v3"));
assert!(warning.contains("upstream_adapter v1"));
assert!(warning.contains("model_catalog v1"));
} else {
assert!(warning.is_none());
}
+2 -4
View File
@@ -180,10 +180,8 @@ RPC v7 的调度控制消息没有载荷:`Suspend { id, epoch }` 归还执行
结果接收本身不恢复父任务、不释放驻留额度;父调用终结统一回收剩余回调。
取消、恢复授权和结果可能交错,终结后的消息不得重新打开调用
宿主只接受 RPC v7,不提供旧协议兼容或降级。使用旧 SDK 的插件须更新 SDK、重新构建并由 CLI 生成安装包后安装,
只修改清单中的版本号不能满足任务挂起、恢复和回调回收合同
插件须校验 `Hello` 的协议版本。一次 `Call` 的结果、错误和流帧按 ID 关联;实例、代次、阶段、账号或资源 ID 用于关联调用和管理资源生命周期
宿主只接受 RPC v7,插件须校验 `Hello` 的协议版本并遵守上述挂起、恢复和回调回收合同。
一次 `Call` 的结果、错误和流帧按 ID 关联;实例、代次、阶段、账号或资源 ID 用于关联调用和管理资源生命周期
## 本地验证
@@ -61,7 +61,7 @@ Provider 固定为宿主内置的 OpenAI 与 xAI。宿主开放以下 11 类扩
| `middleware` | `PluginBuilder::middleware` / `middleware.handle` |
| `upstream_adapter` | `methods::UPSTREAM_ADAPTER_REGISTER`、`methods::UPSTREAM_ADAPTER_EXECUTE`,见[受管上游](upstream-adapters.md) |
| `model_router` | `methods::ROUTE_MODEL` |
| `model_catalog` | `PluginBuilder::model_catalog` / `methods::MODEL_CATALOG_REGISTER` |
| `model_catalog` | `PluginBuilder::model_catalog` / `methods::MODEL_CATALOG_REGISTER_V2` |
| `retry_policy` | `methods::RETRY_DECISION` |
| `scheduler` | `methods::SCHEDULE_ACCOUNT` |
| `observer` | `methods::OBSERVE` / `observer.observe`,按事件类型订阅 |
@@ -344,6 +344,10 @@ let status = host.service::<SetResetCreditAutomation>((account_id, true)).await?
插件改写后用于后续认证、版本检查、正文解压及执行;不新增设置业务的中间件入口。`timeout_ms` 使用宿主 HTTP 超时基线,
插件可改写或用 `None` 清除;终端按有效值执行,不恢复宿主默认值。宿主快照不可用时 `settings` 为 `null`
`POST /v1/chat/completions` 在 HTTP 挂载中保留 Chat 请求与 Chat JSON/SSE 响应视图,
转换为 Responses 发生在内层模型入口。自行按 URI 筛选、改写或分派模型请求的 HTTP 插件,
需要将该路径包含在自身匹配范围内;只匹配 `/v1/responses` 不会覆盖 Chat
`HttpCall.settings_sources` 返回 `config_revision`、宿主 `host` 基线、运行设置的 `overrides`、Key 作用域的 `execution` 与 `http_timeout`。
每条改写记录含 `instance_id`、`order` 和 `value`;来源由宿主记录,修改查询结果不会写入设置。
宿主比较提交前后的值,仅记录实际改变的设置项;同值赋值不新增改写记录。每个顶层设置项整体替换,`false`、`0`、
@@ -468,6 +472,13 @@ Responses 终端重新解码插件正文时读取改写后的正文上限;atte
更早的改写使用上述 HTTP 请求的 `settings`。`request.head.settings_sources` 使用相同的来源结构,`execution.input` 表示进入当前 Key 作用域时的有效值。
原生 Responses WebSocket 每轮请求读取当前宿主快照,重新认证并解析 Key 策略,同时保留握手阶段的显式改写;已开始的执行保持自己的冻结值
Chat Completions 在 request 与 attempt 挂载中使用规范化的 Responses 请求和响应正文,
`request.head.endpoint` 仍为 `/v1/chat/completions`,`protocol` 仍为 `openai`。
request 改写按 Responses 合同重新解码,响应先经过该合同的复核,再由 API 投影成 Chat 交给外层 HTTP 挂载。
Chat 的下游流模式及 `include_usage` 由 API 持有,request 阶段不能切换已确定的下游流模式。
同一边界的原样保留不要求 Chat 与 Responses 字节相同;需要读写 `messages` 或 `choices` 的插件应使用 HTTP 挂载,
字段范围见 [Chat 合同](../../../../../docs/api.md#chat-completions-请求)
中间件可读取和改写完整请求、响应 headers、正文与响应帧,也不按字段名称过滤认证、Cookie、会话或连接头。
该能力遵循[完整信任](manifest.md#完整信任)边界;类型、HTTP 格式、大小及资源生命周期仍需符合合同
@@ -530,26 +541,30 @@ restore_and_validate_response(response).await
### 模型目录
`model_catalog` v1 在 registration 阶段一次提交 `ModelCatalogRegistration { models: Vec<ModelAlias> }`,不需要请求级 binding。
每条 `ModelAlias { id, provider, model }` 定义公开 ID、Provider 及直接上游目标,条目正确性由插件负责。
每实例最多 256 条,整个二进制 JSON 最多 64 KiB;条目重复、目标不存在或 Provider 未注册不阻止宿主快照发布。
`model_catalog` 仅支持 v2,在 registration 阶段通过 `model_catalog.register` 一次提交
`ModelCatalogRegistrationV2 { aliases, models }`,不接受请求级 binding。
返回值必须同时包含 `aliases` 和 `models`,没有对应条目时使用空数组。
`aliases` 的每条 `ModelAlias { id, provider, model }` 定义公开 ID、Provider 及直接上游目标,条目正确性由插件负责。
别名条目重复、目标不存在或 Provider 未注册不阻止宿主快照发布。
请求按发布顺序取首个匹配条目,插件别名优先于静态映射,目标直接用于上游选路,不递归展开别名或目标上的静态映射
```rust,ignore
use gateway_plugin_sdk::call::catalog::{ModelAlias, ModelCatalogRegistration};
use gateway_plugin_sdk::call::catalog::{ModelAlias, ModelCatalogRegistrationV2};
let plugin = PluginBuilder::from_json(include_bytes!("../plugin.json"))?
.model_catalog(ModelCatalogRegistration { models: vec![ModelAlias {
id: "team-default".into(), provider: "openai".into(), model: "gpt-target".into(),
}] })?
.model_catalog(ModelCatalogRegistrationV2 {
aliases: vec![ModelAlias {
id: "team-default".into(), provider: "openai".into(), model: "gpt-target".into(),
}],
models: vec![],
})?
.build()?;
```
有效目录、路由和目标能力随同一个不可变快照发布,`/v1/models`、详情、原生目录和 `host.models.list` 共用该事实及现有 Key/账号模型范围。
目标存在时,别名复用目标的可公开对象;禁用后新请求不再使用该别名,在途请求持有旧代次。
v1 是直接别名,动态选择可组合 `model_router`;实际路由仍须通过宿主的权限和能力校验,不提供多目标虚拟模型注册合同
别名仅表达直接映射,动态选择可组合 `model_router`;实际路由仍须通过宿主的权限和能力校验,不提供多目标虚拟模型注册合同
`model_catalog` v2 使用同一 RPC 方法,返回 `ModelCatalogRegistrationV2 { aliases, models }`。
`aliases` 沿用 v1 直接别名语义;`models` 的每条 `AdaptedModel { id, adapter_id, features, max_output_tokens, presentation }`
`models` 的每条 `AdaptedModel { id, adapter_id, features, max_output_tokens, presentation }`
声明一个受管适配器提供的公开模型,不要求内置 Provider 的原生目录存在同名条目
- `adapter_id` 必须指向同实例已绑定的 `upstream_adapter`,Provider 与非空账号集合取自该适配器声明
@@ -557,10 +572,10 @@ v1 是直接别名,动态选择可组合 `model_router`;实际路由仍须
- `features` 使用 `RequestFeature`,仅为生成操作声明工具、图片输入、推理或 JSON Schema 支持;没有声明的能力不能借用同名原生模型的支持事实
- `presentation` 是独立通用画像,字段见 `call::catalog::ModelPresentation`;省略的布尔能力为 false,未声明推理档位不补猜
- 同名条目可关联不同账号,但 Provider、能力及画像须一致,账号集合不能重叠,也不能与插件直接别名同名,以免展示与实际选择产生歧义
- v2 仍受每实例合计 256 条、64 KiB 信封限制;目录与适配器随同一代次发布,账号最终资格取当前 Key、分组及账号模型政策的交集
- 别名与适配模型每实例合计最多 256 条,注册响应载荷最多 64 KiB;目录与适配器随同一代次发布,账号最终资格取当前 Key、分组及账号模型政策的交集
使用 `PluginBuilder::on(methods::MODEL_CATALOG_REGISTER_V2, handler)` 返回 `TypedReply<ModelCatalogRegistrationV2>`,
并在清单声明 `model_catalog.version = 2`。原有 `.model_catalog(...)` 作者入口继续使用 v1 合同
清单须显式声明 `model_catalog.version = 2`,可使用上述 `.model_catalog(...)` 固定目录,
或通过 `PluginBuilder::on(methods::MODEL_CATALOG_REGISTER_V2, handler)` 返回 `TypedReply<ModelCatalogRegistrationV2>`。
原生 Codex 目录成功返回后,宿主保留其中的完整对象,只追加可见且尚未出现的适配模型画像;原生读取错误仍向调用方返回错误。
适配模型的 wire 模型名、协议编解码及摘要由插件负责,宿主只使用公开 ID、能力和账号范围参与现有选号,不生成请求正文或转换历史
@@ -17,8 +17,8 @@
| `package` | CLI 生成的协议版本、单一目标平台与文件摘要,不需作者手写 |
作者清单不能包含 `package`。使用 `Manifest::from_author_slice()` 校验并规范化;直接反序列化的 `Manifest`
不会执行作者清单的 ID 与固定阶段推导,声明的可选字段仍使用其默认值。打包后,`main` 和所有资源都必须进入 `package.files`;宿主用
`package_for(host, os, architecture)` 检查版本与平台。运行模式为 `trustedProcess`,信任边界见[完整信任](#完整信任)
不会执行作者清单的 ID 与固定阶段推导,声明的可选字段仍使用其默认值。打包后,`main` 和所有资源都必须进入 `package.files`,
宿主校验包结构、目标平台与文件完整性,宿主范围和能力版本另行进行兼容检查。运行模式为 `trustedProcess`,信任边界见[完整信任](#完整信任)
宿主实际开放的合同由 [plugin-host-compatibility.json](../../runtime/plugin-host-compatibility.json)
声明,SDK 的 `Capability::contract_versions()` 只表示 SDK 能描述的行为版本,不能代替宿主支持检查
@@ -57,7 +57,7 @@ RPC v7 的帧布局、编码与会话生命周期见[通信边界](../README.md#
| `scheduler` | `3` |
| `upstream_adapter` | `5` |
| `frontend_authentication`、`model_router`、`retry_policy`、`observer`、`management` | `2` |
| `model_catalog` | `1`、`2` |
| `model_catalog` | `2` |
| `command_line`、`maintenance` | `1` |
例如,仅声明管理页面、命令行和模型请求中间件:
@@ -93,6 +93,7 @@ let (message, ()) = tokio::try_join!(
执行返回 `TypedReply<Empty>` 的有界 `ResponseStream`。每个 chunk 使用 `UpstreamAdapterEvent::encode()`,
包含标准事实与可选原始 wire;事件需符合开始、内容、工具、用量和唯一结束的顺序。
原始 JSON / SSE / WebSocket 字节与标准事实分开传递,错误使用 `UpstreamFailure`;宿主网络观测决定发送状态,插件不能将其降为未发送
`status`、`code`、可选 `error_type` 和 `message` 保留实际上游错误字段,缺失类型保持空值;流内失败不能使用成功握手状态或从业务码推导状态
宿主只对一次受管 `inference` HTTP 调用明确返回的 429 提供有限重放证明:插件必须报告
`rate_limited` 和 `status: 429`,且此前及失败事件中均无业务事实、wire 或续接材料。
@@ -2,12 +2,6 @@
use serde::{Deserialize, Serialize};
/// 一次实例准备产生的完整目录,随实例配置原子发布
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct ModelCatalogRegistration {
pub models: Vec<ModelAlias>,
}
/// 指向内置 Provider 上游模型的直接别名,不创建新的 Provider 执行器
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct ModelAlias {

Some files were not shown because too many files have changed in this diff Show More