mirror of
https://github.com/zyycn/codex-proxy-rs.git
synced 2026-11-09 16:06:27 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5ae38dc0c5 | ||
|
|
503e47fac2 | ||
|
|
b4aea0ea09 | ||
|
|
63036a3694 | ||
|
|
a3e24198cd | ||
|
|
fdd124e72f | ||
|
|
3a2ccbc5e6 | ||
|
|
aa0ce52034 | ||
|
|
7e7acad6a3 | ||
|
|
b3dd9b5337 | ||
|
|
0e1c609cd9 | ||
|
|
12771aca87 | ||
|
|
23b85172b5 | ||
|
|
d89db84a77 | ||
|
|
6504761ce4 | ||
|
|
5877b0cb2f | ||
|
|
958e13c370 | ||
|
|
80f7d53085 | ||
|
|
2f2c1787be | ||
|
|
d679d5e708 | ||
|
|
0f214c3e25 | ||
|
|
736eca6593 | ||
|
|
be5b6d199e | ||
|
|
53e1e9cfb8 | ||
|
|
e03851e23f | ||
|
|
7cd17aff42 | ||
|
|
005466bcb0 | ||
|
|
a6031310f1 | ||
|
|
1ac2a4eac3 | ||
|
|
790a643020 | ||
|
|
1fa37ac92f | ||
|
|
0ceac908f4 | ||
|
|
86a27bed94 | ||
|
|
60f8f1ee80 | ||
|
|
7b6a204889 |
@@ -17,10 +17,11 @@ description: Codex Proxy RS 开发、修复、排障、代码审查和文档修
|
||||
| 开发、排查或审查插件接入、宿主 SDK、Runtime、扩展接口或独立网关插件;目标在 `backend/crates/gateway-plugin/`、`modules/plugins/` | 完整读取[插件职责边界](references/plugin-boundaries.md)及所用 [SDK](../../../backend/crates/gateway-plugin/sdk/README.md) 类型与能力合同;宿主维护自身合同、资源与稳定性,插件业务错误在插件侧修复,不在宿主、SDK 或 Runtime 中补偿;独立插件开发、排查、审查另用 [cpr-plugin-dev](../cpr-plugin-dev/SKILL.md) |
|
||||
| 开发、排查或审查服务端接口、路由、调度、账号与用量规则、持久化、后台任务或协议行为;目标是 `backend/` 中的实现、配置、迁移,或 Rust 源码 | [Workspace 边界](../../../docs/architecture.md#3-workspace-边界)的模块与依赖规则、所属模块及[后端自审](../../../docs/architecture.md#后端自审);Rust 使用 `$rust-best-practices` 和下方后端注释要求,宿主 Rust 同时读取 [Rust 模块组织约定](../../../docs/architecture.md#32-rust-模块组织约定) |
|
||||
| 修改页面接口调用,或开发、排查、审查面向管理端或模型客户端的 HTTP/WS API 合同 | [API](../../../docs/api.md)对应章节;优先复用已有接口,核对调用方 → 服务端解析 → 实际查询或执行,不能只凭前端类型认定参数生效;按实际改动叠加前后端规范 |
|
||||
| 新增、修改或审查客户端可见错误、失败提示、后台任务状态文案 | 下方[错误文案](#错误文案),并按实际出口核对[错误与诊断三层边界](../../../docs/architecture.md#错误与诊断三层边界) |
|
||||
| 模块或调用链变更 | [Workspace 边界](../../../docs/architecture.md#3-workspace-边界)及所属模块章节,不通读整份架构 |
|
||||
| 新增、调整、清理或审查测试 | 下方[测试维护](#测试维护),按当前行为、职责归属与已有覆盖判断保留、合并或删除 |
|
||||
| 修改部署行为、`deploy/` 配置或数据库迁移 | 按范围读取[部署](../../../deploy/README.md)、[迁移](../../../backend/migrations/README.md)对应章节 |
|
||||
| 修改或审查文档 | [文档检查](references/documentation.md)及目标章节,不默认读取代码开发、页面验收或发布流程 |
|
||||
| 修改或审查文档;实现或审查用法、公开合同、职责、约束或运维操作的变化 | [文档检查](references/documentation.md)及相关章节,尚无文档差异时也检查是否漏更;纯文档任务不默认读取代码开发、页面验收或发布流程 |
|
||||
| 代码审查,包括 PR 审查 | [审查标准](../../../CONTRIBUTING.md#审查标准),叠加被审查行为所属领域的规范;审查默认只读 |
|
||||
| PR / Issue / 发版 | 按当前意图选 [PR](../cpr-github-pr/SKILL.md)、[Issue](../cpr-github-issue/SKILL.md)或[发版](../cpr-release/SKILL.md),普通提交不创建 PR |
|
||||
|
||||
@@ -36,7 +37,7 @@ description: Codex Proxy RS 开发、修复、排障、代码审查和文档修
|
||||
|
||||
1. 遵守已提供的 [AGENTS.md](../../../AGENTS.md),按其 CodeGraph 约定定位代码;确认工作区、任务范围与已有改动,找到事实归属、同类实现和调用链,补齐上表命中规范及[项目约定](../../../CONTRIBUTING.md#项目约定)对应条目后再修改
|
||||
2. 以最终差异重新匹配上表,补齐新增领域的规范并检查实际实现,再处理本次引入的职责越界、重复规则、冗余状态与无依据分支;删除或替换功能时一并清理失效的配置、类型、调用、测试与文档,已部署数据和公开合同仍遵守所属迁移及兼容规则;仅审查时报告,不自行修复
|
||||
3. 常规文档只写当前状态;有文档差异时执行[文档检查](references/documentation.md),没有时只确认现有说明是否失真,普通开发不自动修改 `release/notes.md`
|
||||
3. 按最终行为执行[文档检查](references/documentation.md),核对所属正文及相关概述、示例和引用,在本任务内完成必要同步;不以尚无文档差异跳过检查。内部重构等未改变文档所述事实时无需改文档;常规文档只写当前状态,普通开发不自动修改 `release/notes.md`
|
||||
4. 按变更范围执行[验证](../../../CONTRIBUTING.md#验证)及其停止条件,后端命令见[验证命令](../../../docs/architecture.md#验证命令);沿用本轮有效证据与用户已确定的验收分工,区分通过、失败、跳过与未执行;提交时补读[提交约定](../../../CONTRIBUTING.md#项目约定)
|
||||
5. 交付实际结果与验证缺口;前端按[前端流程](references/frontend.md)核对最终差异和运行证据,后端按上表检查生命周期,不把构建通过当作行为验收
|
||||
|
||||
@@ -55,3 +56,23 @@ description: Codex Proxy RS 开发、修复、排障、代码审查和文档修
|
||||
- 每个 Rust 文件顶层使用简短的中文 `//!` 注释说明文件职责,覆盖生产源码、测试与构建脚本;测试文件说明测试范围或辅助用途,生成文件同步维护生成器
|
||||
- 注释不使用中文或英文句号,多句说明按语义分行;保留 URL、版本号、标识符和代码示例中有语义的点号
|
||||
- 文件说明聚焦当前功能与职责边界,局部注释解释原因和约束,避免复述代码或记录修改经过
|
||||
|
||||
## 错误文案
|
||||
|
||||
适用于项目自定义的客户端提示,包括 HTTP、WS、SSE 和任务失败状态;原始上游错误与内部诊断遵守[三层边界](../../../docs/architecture.md#错误与诊断三层边界),不为统一外观翻译、截短或改写
|
||||
|
||||
- 管理端自定义提示统一中文,模型 API 自定义提示统一英文;英文句首大写、句末使用句号,API、Codex 等缩写及专有名词保持正确大小写,中文短提示不加句末句号
|
||||
- 默认一句说明具体问题,确有必要才补第二句处理办法,不设机械字数限制;明确字段、对象或限制,避免只说「请求不合法」或堆叠内部术语
|
||||
- 处理办法必须明确且适用;不把「请稍后重试」「联系管理员」作为统一后缀,不罗列尚未确认的网络、权限、磁盘等可能原因,不对执行结果未知的操作笼统建议重试
|
||||
- 机器码、状态与结构化字段负责程序判断,文案供人阅读;只改措辞时保留分类、重试和账号处置语义,先核对现有调用方是否依赖消息文本
|
||||
- 在拥有错误事实的模块提供安全提示,保留独立的内部原因链;不在响应出口批量首字母大写、翻译或裁剪任意消息,以免改写上游原文
|
||||
|
||||
| 场景 | 文案示例 |
|
||||
| --- | --- |
|
||||
| 日费用额度耗尽 | `Daily spending limit reached.` |
|
||||
| 请求过大 | `Request too large. Reduce the input size.` |
|
||||
| 客户端过旧 | `Codex CLI is outdated. Upgrade to 0.152.0 or later.` |
|
||||
| 名称为空 | `请输入 API Key 名称` |
|
||||
| 更新包校验失败 | `更新包校验失败,请重新下载` |
|
||||
|
||||
审查时沿实际出口检查语言、大小写、标点、长度和处理建议;复用现有行为测试核对错误码及敏感信息边界,不为逐条文案建立重复的快照或字符串清单测试
|
||||
|
||||
@@ -1,14 +1,22 @@
|
||||
# 文档检查
|
||||
|
||||
修改或审查文档时完整读取本流程,并读取 [文档职责与更新条件](../../../../CONTRIBUTING.md#文档职责与更新条件)的完整章节,覆盖内容归属与层次行文
|
||||
修改或审查文档,以及实现或审查涉及文档更新条件的行为变化时,完整读取本流程和 [文档职责与更新条件](../../../../CONTRIBUTING.md#文档职责与更新条件),覆盖内容归属与层次行文
|
||||
|
||||
## 核实与同步范围
|
||||
|
||||
- 以当前目标分支及任务内工作区的实现、配置和有效运行证据核实行为;提交记录用于发现待核实的主题,不能直接当作现行事实或按提交顺序组织正文。按 [CodeGraph 约定](../../../../AGENTS.md#codegraph)同步索引后定位,证据不足时报告缺口,不把计划或推测写成已支持能力
|
||||
- 开发任务按最终行为检查哪些用法、合同、职责、约束或运维操作需要同步,即使尚无 Markdown 差异也要检查;独立文档更新按用户范围核对现行说明,不等到发版时再补普通开发所需的文档
|
||||
- 沿同一事实查找所属正文及相关概述、示例和引用,重点核对“支持/不支持”等能力声明、默认值、配置模板和命令。概述保留必要摘要并链接详细合同;不能只更新详细章节,留下 README 或示例中的矛盾说明,也不扩展为无关全仓改写
|
||||
|
||||
## 写之前
|
||||
|
||||
先确定读者、必须说明的当前事实与所属章节,再决定是否需要修改及详略;文档职责与更新条件按上节核对。涉及合同或职责说明时,补读所属架构、API 或 SDK 的对应条款,核实事实;不因文档提及实现而执行无关的开发与构建流程
|
||||
|
||||
在所属章节就地替换失效说明,先组织读者现在需要的操作、结果和限制,不从变更摘要改写正文。仅介绍被替代实现、修复动机或修改经过的段落直接删除;删掉“新增”“此前”“不再”等词后仍在回答变更经过的内容,也不保留
|
||||
|
||||
## 逐段验收
|
||||
|
||||
检查工作区和暂存区中本任务的最终文档差异,按内容用途处理,不只搜索关键词:
|
||||
检查工作区和暂存区中本任务的最终文档差异,并连同受影响章节及相关概述、示例通读,按内容用途处理,不只搜索关键词:
|
||||
|
||||
| 段落回答的问题 | 处理 |
|
||||
| --- | --- |
|
||||
@@ -26,6 +34,6 @@
|
||||
|
||||
- 完成上述逐段验收及[行文检查](../../../../CONTRIBUTING.md#层次与行文),发现问题先处理,未完成内容审查不能称文档已检查
|
||||
- 检查相对链接、章节锚点、表格与命令示例,`git diff --check` 或构建通过不能代替内容审查
|
||||
- 报告实际覆盖和缺口即可,不把检查记录写进常规文档
|
||||
- 交付时简述已同步的文档范围;涉及更新条件却未改文档时,说明现有内容为何仍准确或还有什么缺口。仅报告实际核对范围,不把链接检查通过称为事实核实,也不把检查记录写进常规文档
|
||||
|
||||
只有明确的发布说明任务才进入 `cpr-release`;发布说明与常规文档的边界按[内容归属](../../../../CONTRIBUTING.md#内容归属)执行
|
||||
|
||||
@@ -34,6 +34,7 @@ Thumbs.db
|
||||
|
||||
# Local tools and worktrees
|
||||
/.codegraph/
|
||||
/.playwright-cli/
|
||||
/.codex-ws-audit/
|
||||
/.reasonix/
|
||||
/.claude
|
||||
|
||||
@@ -28,6 +28,7 @@ Issue 和 PR 回复统一按 [GitHub 回复口吻](CONTRIBUTING.md#github-回复
|
||||
|
||||
In repositories indexed by CodeGraph (a `.codegraph/` directory exists at the repo root), reach for it BEFORE grep/find or reading files when you need to understand or locate code:
|
||||
|
||||
- **Index freshness**: Run `codegraph sync` from the repository root before the first code query in a task. After pulling updates, switching branches, or changing code, run it again before the next query so symbols and call paths match the working tree. If synchronization fails, report the failure and verify against current source rather than treating the index as current.
|
||||
- **MCP tools** (when available): `codegraph_explore` answers most code questions in one call — the relevant symbols' verbatim source plus the call paths between them. `codegraph_node` returns one symbol's source + callers, or reads a whole file with line numbers. If the tools are listed but deferred, load them by name via tool search.
|
||||
- **Shell** (always works): `codegraph explore "<symbol names or question>"` and `codegraph node <symbol-or-file>` print the same output.
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@
|
||||
</div>
|
||||
|
||||
> [!NOTE]
|
||||
> 本项目提供 Responses API,不支持 `/v1/chat/completions`,接入前请确认客户端支持 Responses 协议
|
||||
> 本项目提供 Responses API 和 Chat Completions API;Chat Completions 请求通过 Responses 链路执行,支持范围见 [API 参考](docs/api.md#chat-completions-请求)
|
||||
|
||||
## 快速预览
|
||||
|
||||
@@ -97,7 +97,7 @@ API Key 持有者可在登录页切换身份,查看自己的用量与额度,
|
||||
|
||||
完整步骤、生图配置与排障见[客户端配置](deploy/README.md#客户端配置)
|
||||
|
||||
**其他 Responses API 客户端**:填写以下信息:
|
||||
**其他 Responses / Chat Completions API 客户端**:填写以下信息:
|
||||
|
||||
| 配置 | 值 |
|
||||
| --- | --- |
|
||||
|
||||
Generated
+2
@@ -1743,6 +1743,8 @@ dependencies = [
|
||||
"http",
|
||||
"http-body",
|
||||
"http-body-util",
|
||||
"hyper",
|
||||
"hyper-util",
|
||||
"regex",
|
||||
"secrecy",
|
||||
"serde",
|
||||
|
||||
@@ -126,30 +126,32 @@ pub struct RetentionDecision {
|
||||
/// 自动保留规则:
|
||||
/// - `retentionDays = 0` 时不按天数清理;`retentionCount = 0` 时不按份数清理
|
||||
/// - 自动计划备份满足任一启用阈值即可进入删除
|
||||
/// - 无论阈值如何,至少保留最近一个成功的计划备份(倒序下标 0)
|
||||
/// - 无论阈值如何,至少保留最近一个成功的计划备份(全局倒序下标 0)
|
||||
/// - 手动备份不进入本规则;手工/计划记录各自的 `expires_at` 由到期扫描处理
|
||||
/// - `offset` 是当前页在全部 completed 计划备份中的起始位置
|
||||
#[must_use]
|
||||
pub fn decide_retention(
|
||||
retention_days: u32,
|
||||
retention_count: u32,
|
||||
now: DateTime<Utc>,
|
||||
offset: u64,
|
||||
scheduled_completed_desc: &[BackupRecord],
|
||||
) -> Vec<RetentionDecision> {
|
||||
let mut decisions = Vec::new();
|
||||
for (index, record) in scheduled_completed_desc.iter().enumerate() {
|
||||
let position = offset.saturating_add(index as u64);
|
||||
if record.status != BackupStatus::Completed {
|
||||
continue;
|
||||
}
|
||||
// 最近一个成功的计划备份永远保留
|
||||
if index == 0 {
|
||||
if position == 0 {
|
||||
continue;
|
||||
}
|
||||
let expired_by_days = retention_days > 0
|
||||
&& record.completed_at.is_some_and(|completed_at| {
|
||||
completed_at <= now - Duration::days(i64::from(retention_days))
|
||||
});
|
||||
let exceeded_count =
|
||||
retention_count > 0 && index >= usize::try_from(retention_count).unwrap_or(usize::MAX);
|
||||
let exceeded_count = retention_count > 0 && position >= u64::from(retention_count);
|
||||
if expired_by_days || exceeded_count {
|
||||
decisions.push(RetentionDecision {
|
||||
record_id: record.id.clone(),
|
||||
|
||||
@@ -38,8 +38,8 @@ use super::policy::{BackupSchedule, decide_retention};
|
||||
const BACKUP_LOOP_INTERVAL: Duration = Duration::from_secs(30);
|
||||
/// 每个循环最多执行的保留删除数量
|
||||
const RETENTION_BATCH_SIZE: usize = 10;
|
||||
/// 保留扫描一次读取的 completed 计划备份上限
|
||||
const RETENTION_SCAN_LIMIT: u32 = 1000;
|
||||
/// 保留扫描每页读取的 completed 计划备份数量
|
||||
const RETENTION_PAGE_SIZE: u32 = 1000;
|
||||
/// 每个循环最多完成的待删除记录数量
|
||||
const PENDING_DELETION_BATCH: u32 = 20;
|
||||
|
||||
@@ -112,7 +112,7 @@ impl BackupTask {
|
||||
self.recover_intermediate(cancellation).await?;
|
||||
self.finish_pending_deletions().await?;
|
||||
self.process_next_task(cancellation).await?;
|
||||
self.run_retention_batch(Utc::now()).await?;
|
||||
self.run_retention_batch(Utc::now(), cancellation).await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -637,7 +637,11 @@ impl BackupTask {
|
||||
}
|
||||
|
||||
/// 执行一小批到期保留清理:先处理 `expires_at` 已到期的记录,再按天数/份数清理计划备份
|
||||
async fn run_retention_batch(&self, now: DateTime<Utc>) -> Result<(), WorkerTaskError> {
|
||||
async fn run_retention_batch(
|
||||
&self,
|
||||
now: DateTime<Utc>,
|
||||
cancellation: &CancellationToken,
|
||||
) -> Result<(), WorkerTaskError> {
|
||||
let settings = self.repository.load_settings().await.map_err(repo_error)?;
|
||||
|
||||
// 1. expires_at 已到期的记录(手动或计划)无条件进入删除
|
||||
@@ -659,18 +663,41 @@ impl BackupTask {
|
||||
}
|
||||
|
||||
// 2. 计划备份按 retentionDays / retentionCount 清理
|
||||
let records = self
|
||||
.repository
|
||||
.list_scheduled_completed_desc(RETENTION_SCAN_LIMIT)
|
||||
.await
|
||||
.map_err(repo_error)?;
|
||||
let decisions = decide_retention(
|
||||
settings.retention_days,
|
||||
settings.retention_count,
|
||||
now,
|
||||
&records,
|
||||
);
|
||||
for decision in decisions.into_iter().take(RETENTION_BATCH_SIZE) {
|
||||
if settings.retention_days == 0 && settings.retention_count == 0 {
|
||||
return Ok(());
|
||||
}
|
||||
let mut offset = 0;
|
||||
let mut decisions = Vec::new();
|
||||
loop {
|
||||
if cancellation.is_cancelled() {
|
||||
return Ok(());
|
||||
}
|
||||
let records = self
|
||||
.repository
|
||||
.list_scheduled_completed_desc(offset, RETENTION_PAGE_SIZE)
|
||||
.await
|
||||
.map_err(repo_error)?;
|
||||
let remaining = RETENTION_BATCH_SIZE - decisions.len();
|
||||
decisions.extend(
|
||||
decide_retention(
|
||||
settings.retention_days,
|
||||
settings.retention_count,
|
||||
now,
|
||||
offset,
|
||||
&records,
|
||||
)
|
||||
.into_iter()
|
||||
.take(remaining),
|
||||
);
|
||||
if decisions.len() == RETENTION_BATCH_SIZE
|
||||
|| records.len() < RETENTION_PAGE_SIZE as usize
|
||||
{
|
||||
break;
|
||||
}
|
||||
offset += records.len() as u64;
|
||||
}
|
||||
// 收齐本批候选后再删除,避免删除前页记录导致后续分页跳过数据
|
||||
for decision in decisions {
|
||||
let Some(record) = self
|
||||
.repository
|
||||
.transition_to_deleting(&decision.record_id, now, None)
|
||||
|
||||
@@ -1,9 +1,4 @@
|
||||
//! 容量熔断冻结的恢复编排:到期探测、失败顺延与自适应并发下调
|
||||
//!
|
||||
//! 冻结事实保存在可丢失的 Redis 冷却中,本服务只组合 Admin 端口:
|
||||
//! 观测(`AccountRuntimeStore`)、探测(`AccountsService`)与写回
|
||||
//! (原子降低并发 / 按冻结代次结束探测)
|
||||
//! 旧观测不会覆盖管理员操作
|
||||
//! 账号错误保护的到期探测与按冻结快照恢复
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::sync::Arc;
|
||||
@@ -11,7 +6,7 @@ use std::time::{Duration, SystemTime};
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
use futures::StreamExt as _;
|
||||
use gateway_core::account::{AccountConcurrencyLimit, ProviderAccountId};
|
||||
use gateway_core::account::{AccountErrorProtection, ProviderAccountId};
|
||||
use gateway_core::routing::UpstreamModelId;
|
||||
use gateway_core::task::{ScheduledTask, WorkerCycleContext, WorkerTaskError};
|
||||
use tracing::warn;
|
||||
@@ -19,14 +14,9 @@ use tracing::warn;
|
||||
use crate::model::accounts::{
|
||||
AccountConnectionTestEvent, AccountFreeze, AccountPageItem, AccountRuntimeSnapshot,
|
||||
};
|
||||
use crate::model::{MutationActor, MutationContext};
|
||||
use crate::ports::store::{AccountRuntimeStore, AccountStore, SettingsStore};
|
||||
use crate::use_case::accounts::AccountsService;
|
||||
|
||||
/// 自适应并发下调保留系数:下调到观测在途峰值的 80%
|
||||
const ADAPTIVE_CONCURRENCY_FACTOR: f64 = 0.8;
|
||||
/// 自适应并发下限:过低的并发让账号几乎不可用,宁可保持冻结
|
||||
const ADAPTIVE_CONCURRENCY_FLOOR: u32 = 2;
|
||||
/// worker 周期;冷却观测与探测都依赖该节奏
|
||||
pub const FREEZE_RECOVERY_INTERVAL: Duration = Duration::from_secs(30);
|
||||
pub const FREEZE_RECOVERY_WORKER_OWNER: &str = "account-freeze-recovery";
|
||||
@@ -34,9 +24,6 @@ pub const WORKER_INITIAL_BACKOFF: Duration = Duration::from_secs(1);
|
||||
pub const WORKER_MAXIMUM_BACKOFF: Duration = Duration::from_secs(60);
|
||||
pub const WORKER_LEASE_TTL: Duration = Duration::from_secs(15 * 60);
|
||||
pub const WORKER_LEASE_RENEWAL: Duration = Duration::from_secs(5 * 60);
|
||||
/// 系统审计上下文的固定请求 ID;账号并发更新审计由此溯源到自动冻结编排
|
||||
const SYSTEM_REQUEST_ID: &str = "account-freeze-recovery";
|
||||
|
||||
/// 冻结恢复所需的管理端口组合
|
||||
#[derive(Clone)]
|
||||
pub struct FreezeRecoveryDeps {
|
||||
@@ -55,26 +42,12 @@ impl FreezeRecoveryTask {
|
||||
Self { deps }
|
||||
}
|
||||
|
||||
fn system_context() -> MutationContext {
|
||||
MutationContext {
|
||||
actor: MutationActor::System,
|
||||
request_id: SYSTEM_REQUEST_ID.to_owned(),
|
||||
}
|
||||
}
|
||||
|
||||
/// 读取冻结策略;读取或校验失败跳过本轮,不能把未知配置当作允许解冻
|
||||
async fn freeze_policy(&self) -> Option<gateway_core::provider_ports::ProviderFreezePolicy> {
|
||||
/// 无法读取或校验配置时跳过本轮,不能把未知状态当作允许解冻
|
||||
async fn freeze_policy(&self) -> Option<AccountErrorProtection> {
|
||||
let settings = self.deps.settings.load_runtime_settings().await.ok()?;
|
||||
gateway_core::provider_ports::ProviderFreezePolicy::try_new(
|
||||
settings.values.account_auto_freeze_enabled,
|
||||
settings.values.account_auto_freeze_threshold,
|
||||
settings.values.account_auto_freeze_window_seconds,
|
||||
settings.values.account_auto_freeze_duration_seconds,
|
||||
settings.values.account_auto_freeze_probe_enabled,
|
||||
settings.values.account_auto_freeze_probe_model,
|
||||
settings.values.account_auto_freeze_adaptive_concurrency,
|
||||
)
|
||||
.ok()
|
||||
let policy = settings.values.account_error_protection;
|
||||
policy.validate().ok()?;
|
||||
Some(policy)
|
||||
}
|
||||
|
||||
async fn active_freezes(&self) -> BTreeMap<String, AccountFreeze> {
|
||||
@@ -89,55 +62,9 @@ impl FreezeRecoveryTask {
|
||||
if freezes.is_empty() {
|
||||
return;
|
||||
}
|
||||
if policy.enabled() && policy.adaptive_concurrency() {
|
||||
self.adapt_concurrency_limits(&freezes, &policy).await;
|
||||
}
|
||||
self.recover_due_freezes(&freezes, &policy).await;
|
||||
}
|
||||
|
||||
/// 对冻结中的账号执行自适应并发下调:目标为观测在途峰值的 80%(下限 2),
|
||||
/// 只降不升;未观测到峰值证据的账号保持现状
|
||||
async fn adapt_concurrency_limits(
|
||||
&self,
|
||||
freezes: &BTreeMap<String, AccountFreeze>,
|
||||
policy: &gateway_core::provider_ports::ProviderFreezePolicy,
|
||||
) {
|
||||
let account_ids = freezes.keys().cloned().collect::<Vec<_>>();
|
||||
let Ok(peaks) = self.deps.runtime.capacity_peaks(&account_ids).await else {
|
||||
return;
|
||||
};
|
||||
for (account_id, peak) in peaks {
|
||||
let Some(target) = adaptive_target(peak) else {
|
||||
continue;
|
||||
};
|
||||
let Ok(account) = ProviderAccountId::new(account_id.clone()) else {
|
||||
continue;
|
||||
};
|
||||
let Some(limit) = AccountConcurrencyLimit::new(target) else {
|
||||
continue;
|
||||
};
|
||||
match self
|
||||
.deps
|
||||
.accounts
|
||||
.lower_concurrency_limit(&Self::system_context(), account, limit)
|
||||
.await
|
||||
{
|
||||
Ok(Some(_)) => {
|
||||
tracing::info!(
|
||||
account_id,
|
||||
adapted_limit = target,
|
||||
freeze_seconds = policy.freeze_duration().as_secs(),
|
||||
"容量熔断:按观测在途峰值下调账号并发上限",
|
||||
);
|
||||
}
|
||||
Ok(None) => {}
|
||||
Err(error) => {
|
||||
warn!(account_id, error = %error, "自适应并发下调失败");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn load_account(&self, account_id: &str) -> Option<Option<AccountPageItem>> {
|
||||
let account_id = ProviderAccountId::new(account_id.to_owned()).ok()?;
|
||||
self.deps
|
||||
@@ -151,13 +78,13 @@ impl FreezeRecoveryTask {
|
||||
async fn recover_due_freezes(
|
||||
&self,
|
||||
freezes: &BTreeMap<String, AccountFreeze>,
|
||||
policy: &gateway_core::provider_ports::ProviderFreezePolicy,
|
||||
policy: &AccountErrorProtection,
|
||||
) {
|
||||
for (account_id, freeze) in freezes {
|
||||
if freeze.until > Utc::now() {
|
||||
continue;
|
||||
}
|
||||
if policy.enabled() && policy.probe_enabled() && freeze.requires_probe {
|
||||
if policy.enabled && policy.probe_enabled && freeze.requires_probe {
|
||||
self.probe_and_recover(account_id, freeze, policy).await;
|
||||
} else if let Err(error) = self
|
||||
.deps
|
||||
@@ -174,11 +101,19 @@ impl FreezeRecoveryTask {
|
||||
&self,
|
||||
account_id: &str,
|
||||
freeze: &AccountFreeze,
|
||||
policy: &gateway_core::provider_ports::ProviderFreezePolicy,
|
||||
policy: &AccountErrorProtection,
|
||||
) {
|
||||
let Some(Some(item)) = self.load_account(account_id).await else {
|
||||
return;
|
||||
};
|
||||
if item.account.credential_revision != freeze.credential_revision {
|
||||
let _ = self
|
||||
.deps
|
||||
.runtime
|
||||
.finish_freeze(account_id, freeze, None)
|
||||
.await;
|
||||
return;
|
||||
}
|
||||
if !item.account.enabled {
|
||||
// 停用账号没有自动恢复意义;解冻交给管理员手动恢复
|
||||
return;
|
||||
@@ -186,7 +121,7 @@ impl FreezeRecoveryTask {
|
||||
let Ok(account) = ProviderAccountId::new(account_id.to_owned()) else {
|
||||
return;
|
||||
};
|
||||
let Some(model) = self.resolve_probe_model(&account, policy).await else {
|
||||
let Some(model) = self.resolve_probe_model(&account, freeze, policy).await else {
|
||||
// 没有可用探测模型时按失败处理,顺延冻结等待下一轮
|
||||
self.postpone(&account, freeze, policy).await;
|
||||
return;
|
||||
@@ -224,9 +159,23 @@ impl FreezeRecoveryTask {
|
||||
&self,
|
||||
account: &ProviderAccountId,
|
||||
freeze: &AccountFreeze,
|
||||
policy: &gateway_core::provider_ports::ProviderFreezePolicy,
|
||||
policy: &AccountErrorProtection,
|
||||
) {
|
||||
let Some(until) = SystemTime::now().checked_add(policy.freeze_duration()) else {
|
||||
// 探测期间关闭保护或探测时,不再把已经到期的冻结向后延长
|
||||
let Some(current) = self.freeze_policy().await else {
|
||||
return;
|
||||
};
|
||||
if !current.enabled || !current.probe_enabled {
|
||||
let _ = self
|
||||
.deps
|
||||
.runtime
|
||||
.finish_freeze(account.as_str(), freeze, None)
|
||||
.await;
|
||||
return;
|
||||
}
|
||||
let Some(until) = SystemTime::now().checked_add(Duration::from_secs(
|
||||
freeze.duration_seconds.unwrap_or(policy.duration_seconds),
|
||||
)) else {
|
||||
return;
|
||||
};
|
||||
let until = DateTime::<Utc>::from(until);
|
||||
@@ -239,7 +188,10 @@ impl FreezeRecoveryTask {
|
||||
Ok(true) => {
|
||||
tracing::info!(
|
||||
account_id = account.as_str(),
|
||||
postpone_seconds = policy.freeze_duration().as_secs(),
|
||||
postpone_seconds = Duration::from_secs(
|
||||
freeze.duration_seconds.unwrap_or(policy.duration_seconds)
|
||||
)
|
||||
.as_secs(),
|
||||
"冻结恢复探测失败:账号冻结顺延",
|
||||
);
|
||||
}
|
||||
@@ -257,9 +209,14 @@ impl FreezeRecoveryTask {
|
||||
async fn resolve_probe_model(
|
||||
&self,
|
||||
account: &ProviderAccountId,
|
||||
policy: &gateway_core::provider_ports::ProviderFreezePolicy,
|
||||
freeze: &AccountFreeze,
|
||||
policy: &AccountErrorProtection,
|
||||
) -> Option<UpstreamModelId> {
|
||||
if let Some(model) = policy.probe_model() {
|
||||
if let Some(model) = if freeze.duration_seconds.is_some() {
|
||||
freeze.probe_model.as_deref()
|
||||
} else {
|
||||
policy.probe_model.as_deref()
|
||||
} {
|
||||
return UpstreamModelId::new(model.to_owned()).ok();
|
||||
}
|
||||
let models = self.deps.accounts.models(account, false).await.ok()?;
|
||||
@@ -282,15 +239,6 @@ impl ScheduledTask for FreezeRecoveryTask {
|
||||
}
|
||||
}
|
||||
|
||||
/// 自适应目标并发:观测峰值 × 0.8 向下取整,下限 2;无证据(峰值为 0)不调整
|
||||
fn adaptive_target(peak_in_flight: u32) -> Option<u32> {
|
||||
if peak_in_flight == 0 {
|
||||
return None;
|
||||
}
|
||||
let scaled = (f64::from(peak_in_flight) * ADAPTIVE_CONCURRENCY_FACTOR).floor() as u32;
|
||||
Some(scaled.max(ADAPTIVE_CONCURRENCY_FLOOR))
|
||||
}
|
||||
|
||||
/// 消耗连接测试事件流并返回是否以 `Completed` 终止
|
||||
async fn drain_probe(mut events: crate::model::accounts::AccountConnectionTestEventStream) -> bool {
|
||||
let mut completed = false;
|
||||
|
||||
@@ -81,6 +81,7 @@ pub struct AccountListQuery {
|
||||
/// Admin query service 从运行态存储取得的当前账号冷却快照
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq)]
|
||||
pub struct AccountRuntimeSnapshot {
|
||||
pub protection_rules: BTreeMap<String, Vec<gateway_core::account::AccountProtectionMatch>>,
|
||||
pub cooldown: BTreeMap<String, gateway_core::account::AccountCooldown>,
|
||||
/// `None` 表示实时 lease 存储不可用;`Some` 中未出现的账号当前使用量为零
|
||||
pub in_flight: Option<BTreeMap<String, u64>>,
|
||||
@@ -93,6 +94,9 @@ pub struct AccountFreeze {
|
||||
pub until: DateTime<Utc>,
|
||||
pub generation: String,
|
||||
pub requires_probe: bool,
|
||||
pub duration_seconds: Option<u64>,
|
||||
pub probe_model: Option<String>,
|
||||
pub protection_rules: Vec<gateway_core::account::AccountProtectionMatch>,
|
||||
}
|
||||
|
||||
/// 可选的账号分组成员筛选条件
|
||||
@@ -203,6 +207,7 @@ pub struct AccountPage {
|
||||
/// 同一状态快照下的账号事实与唯一状态投影
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct AccountPageItem {
|
||||
pub protection_rules: Vec<gateway_core::account::AccountProtectionMatch>,
|
||||
pub account: AccountRecord,
|
||||
pub projection: AccountStatusProjection,
|
||||
pub capacity: AccountCapacity,
|
||||
|
||||
@@ -2,7 +2,9 @@
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
|
||||
use super::observability::{DecimalAmount, RequestMetricPoint, UsageOverview};
|
||||
use super::observability::{
|
||||
DecimalAmount, RequestMetricPoint, UsageConcurrencyPoint, UsageOverview,
|
||||
};
|
||||
|
||||
/// 数据可见性状态,不以缺失投影冒充零统计
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
@@ -57,9 +59,10 @@ pub struct UsageProjectedCostPoint {
|
||||
}
|
||||
|
||||
/// 同一数据库快照内的完整洞察输入
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct UsageInsightsObservation {
|
||||
pub overview: UsageOverview,
|
||||
pub trend: Vec<RequestMetricPoint>,
|
||||
pub costs: Vec<UsageProjectedCostPoint>,
|
||||
pub concurrency: Vec<UsageConcurrencyPoint>,
|
||||
}
|
||||
|
||||
@@ -23,7 +23,6 @@ pub enum MutationAuditOperation {
|
||||
BackupDelete,
|
||||
BackupConnectionTest,
|
||||
BackupDownloadUrl,
|
||||
ProviderAccountAdaptConcurrency,
|
||||
ProviderAccountRecover,
|
||||
ProviderAccountBatchUpdate,
|
||||
ProviderAccountDelete,
|
||||
@@ -84,7 +83,6 @@ impl MutationAuditOperation {
|
||||
Self::BackupDelete => ("backup.delete_requested", "backup"),
|
||||
Self::BackupConnectionTest => ("backup.s3_connection_tested", "backup_settings"),
|
||||
Self::BackupDownloadUrl => ("backup.download_url_created", "backup"),
|
||||
Self::ProviderAccountAdaptConcurrency => ("adapt_concurrency", "provider_account"),
|
||||
Self::ProviderAccountRecover => ("recover", "provider_account"),
|
||||
Self::ProviderAccountBatchUpdate => ("batch_update", "provider_account"),
|
||||
Self::ProviderAccountDelete => ("delete", "provider_account"),
|
||||
|
||||
@@ -68,6 +68,7 @@ pub struct AdminError {
|
||||
kind: AdminErrorKind,
|
||||
message: String,
|
||||
source: Option<gateway_core::error::ErrorSource>,
|
||||
sensitive_context_redacted: bool,
|
||||
}
|
||||
|
||||
impl AdminError {
|
||||
@@ -77,6 +78,7 @@ impl AdminError {
|
||||
kind,
|
||||
message: message.into(),
|
||||
source: None,
|
||||
sensitive_context_redacted: false,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -86,9 +88,19 @@ impl AdminError {
|
||||
self
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn redact_sensitive_context(mut self) -> Self {
|
||||
self.sensitive_context_redacted = true;
|
||||
self
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn error_details(&self) -> Option<gateway_core::error::ErrorDetails> {
|
||||
gateway_core::error::ErrorDetails::capture(self.source.as_ref(), None, false)
|
||||
gateway_core::error::ErrorDetails::capture(
|
||||
self.source.as_ref(),
|
||||
None,
|
||||
self.sensitive_context_redacted,
|
||||
)
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
|
||||
@@ -653,6 +653,7 @@ pub struct DashboardAccountUsage {
|
||||
pub authentication_kind: String,
|
||||
pub name: String,
|
||||
pub email: Option<String>,
|
||||
pub notes: Option<String>,
|
||||
pub plan_type: Option<String>,
|
||||
/// Admin 按 Provider 补齐的套餐展示名称
|
||||
pub plan_type_display: Option<String>,
|
||||
@@ -1260,38 +1261,44 @@ pub struct UsageInsightsHealth {
|
||||
pub points: Vec<UsageInsightsHealthPoint>,
|
||||
}
|
||||
|
||||
/// 一个时间桶内按重叠时长加权的逻辑请求并发
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct UsageConcurrencyPoint {
|
||||
pub bucket_start: DateTime<Utc>,
|
||||
pub average_concurrency: f64,
|
||||
pub peak_concurrency: u64,
|
||||
}
|
||||
|
||||
/// 请求并发与调度等待的负载指标
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct UsageInsightsLoad {
|
||||
pub average_concurrency: f64,
|
||||
pub peak_concurrency: u64,
|
||||
pub admission_decision_p95_ms: Option<PercentileMilliseconds>,
|
||||
pub account_selection_wait_p95_ms: Option<PercentileMilliseconds>,
|
||||
}
|
||||
|
||||
/// 洞察性能时间点
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct UsageInsightsPerformancePoint {
|
||||
pub bucket_start: DateTime<Utc>,
|
||||
pub latency_percentiles: LatencyPercentiles,
|
||||
pub first_token_latency_percentiles: LatencyPercentiles,
|
||||
pub admission_decision_percentiles: LatencyPercentiles,
|
||||
pub account_selection_wait_percentiles: LatencyPercentiles,
|
||||
pub output_throughput_p10: Option<u64>,
|
||||
pub output_throughput_p50: Option<u64>,
|
||||
pub output_throughput_p90: Option<u64>,
|
||||
pub capacity_utilization_avg_basis_points: Option<u64>,
|
||||
pub capacity_utilization_p95_basis_points: Option<u64>,
|
||||
pub load: UsageInsightsLoad,
|
||||
}
|
||||
|
||||
/// 洞察性能汇总与可观测覆盖率
|
||||
/// 洞察性能汇总
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct UsageInsightsPerformance {
|
||||
pub latency_percentiles: LatencyPercentiles,
|
||||
pub first_token_latency_percentiles: LatencyPercentiles,
|
||||
pub latency_coverage: f64,
|
||||
pub first_token_coverage: f64,
|
||||
pub admission_decision_percentiles: LatencyPercentiles,
|
||||
pub account_selection_wait_percentiles: LatencyPercentiles,
|
||||
pub admission_decision_coverage: f64,
|
||||
pub account_selection_wait_coverage: f64,
|
||||
pub output_throughput_p10: Option<u64>,
|
||||
pub output_throughput_p50: Option<u64>,
|
||||
pub output_throughput_p90: Option<u64>,
|
||||
pub capacity_utilization_avg_basis_points: Option<u64>,
|
||||
pub capacity_utilization_p95_basis_points: Option<u64>,
|
||||
pub capacity_coverage: f64,
|
||||
pub load: UsageInsightsLoad,
|
||||
pub points: Vec<UsageInsightsPerformancePoint>,
|
||||
}
|
||||
|
||||
@@ -1345,6 +1352,84 @@ pub struct UsageInsights {
|
||||
pub providers: Vec<ProviderObservation>,
|
||||
}
|
||||
|
||||
/// 单次洞察查询所需的指标,持久化层据此选择计数、分布与费用状态
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
|
||||
pub enum UsageInsightMetric {
|
||||
Health,
|
||||
Latency,
|
||||
FirstToken,
|
||||
Throughput,
|
||||
Load,
|
||||
Cost,
|
||||
Savings,
|
||||
Cache,
|
||||
}
|
||||
|
||||
/// 单个指标的时间点,活动计数与指标共用读取快照
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct UsageMetricPoint<T> {
|
||||
pub bucket_start: DateTime<Utc>,
|
||||
pub request_count: u64,
|
||||
pub value: T,
|
||||
}
|
||||
|
||||
/// 各指标复用时间序列形状,汇总与时间点保留各自的类型
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct UsageMetricSeries<S, P = S> {
|
||||
pub summary: S,
|
||||
pub points: Vec<UsageMetricPoint<P>>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct UsageThroughputPercentiles {
|
||||
pub p10: Option<u64>,
|
||||
pub p50: Option<u64>,
|
||||
pub p90: Option<u64>,
|
||||
}
|
||||
|
||||
/// 成本卡片各视图共同显示的汇总
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct UsageCostSummary {
|
||||
pub estimated_cost: Option<DecimalAmount>,
|
||||
pub cache_savings: Option<DecimalAmount>,
|
||||
pub tier_premium: Option<DecimalAmount>,
|
||||
pub cost_per_successful_request: Option<DecimalAmount>,
|
||||
pub coverage: CostCoverage,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct UsageCostComparison {
|
||||
pub estimated_cost: Option<DecimalAmount>,
|
||||
pub no_cache_cost: Option<DecimalAmount>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct UsageCacheEfficiency {
|
||||
pub input_tokens: u64,
|
||||
pub cached_token_rate: f64,
|
||||
pub cache_hit_request_rate: Option<f64>,
|
||||
}
|
||||
|
||||
/// 只发布本次请求的指标,未查询的值不伪装成零
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub enum UsageInsightData {
|
||||
Health(UsageInsightsHealth),
|
||||
Latency(UsageMetricSeries<LatencyPercentiles>),
|
||||
FirstToken(UsageMetricSeries<LatencyPercentiles>),
|
||||
Throughput(UsageMetricSeries<UsageThroughputPercentiles>),
|
||||
Load(UsageMetricSeries<UsageInsightsLoad>),
|
||||
Cost(UsageMetricSeries<UsageCostSummary, UsageCostComparison>),
|
||||
Savings(UsageMetricSeries<UsageCostSummary, Option<DecimalAmount>>),
|
||||
Cache(UsageMetricSeries<UsageCostSummary, UsageCacheEfficiency>),
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct UsageInsight {
|
||||
pub analytics: AnalyticsMetadata,
|
||||
pub granularity: Granularity,
|
||||
pub data: UsageInsightData,
|
||||
}
|
||||
|
||||
/// 已完成分母计算的诊断项
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct DiagnosticsItem {
|
||||
|
||||
@@ -1246,6 +1246,7 @@ impl fmt::Debug for ProviderExport {
|
||||
/// 统一账号目录的一行完整结果
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct AccountDirectoryItem {
|
||||
pub protection_rules: Vec<gateway_core::account::AccountProtectionMatch>,
|
||||
pub account: AccountRecord,
|
||||
pub capacity: super::accounts::AccountCapacity,
|
||||
pub capabilities: super::accounts::ProviderAccountCapabilities,
|
||||
|
||||
@@ -72,7 +72,7 @@ pub fn account_quota_forecasts(
|
||||
},
|
||||
extrapolated: false,
|
||||
source: None,
|
||||
unavailable_reason: Some("没有可统计的周/月额度窗口,请先刷新账号额度。"),
|
||||
unavailable_reason: Some("缺少周/月额度窗口,请刷新账号额度"),
|
||||
low_sample: false,
|
||||
incomplete_cost: false,
|
||||
incomplete_tokens: false,
|
||||
@@ -138,30 +138,27 @@ impl AccountQuotaForecast {
|
||||
self.incomplete_tokens = usage.is_some_and(|usage| usage.missing_token_count > 0);
|
||||
let method = sample.map_or(QuotaForecastMethod::Cumulative, |sample| sample.method);
|
||||
let Some(start) = start.filter(|start| *start <= now && now < reset_at) else {
|
||||
self.unavailable_reason = Some("额度窗口已过期或边界无效,请刷新账号额度后重试。");
|
||||
self.unavailable_reason = Some("额度窗口已过期或无效,请刷新账号额度");
|
||||
return;
|
||||
};
|
||||
if !observed_at.is_some_and(|observed| start <= observed && observed <= now) {
|
||||
self.unavailable_reason = Some("缺少本周期的额度快照,请先刷新账号额度。");
|
||||
self.unavailable_reason = Some("缺少本周期的额度快照,请刷新账号额度");
|
||||
return;
|
||||
}
|
||||
if sample.is_some_and(|sample| sample.discontinuous) {
|
||||
self.unavailable_reason =
|
||||
Some("额度观测出现回落或累计记录不连续,正在重新积累配对样本。");
|
||||
self.unavailable_reason = Some("额度回落或记录不连续,正在重新积累配对样本");
|
||||
return;
|
||||
}
|
||||
if account_added_at > start && method == QuotaForecastMethod::Cumulative {
|
||||
self.unavailable_reason = Some(
|
||||
"本周期开始时的记录不完整,正在积累至少 5 个百分点的配对观测,无需等待下次重置。",
|
||||
);
|
||||
self.unavailable_reason = Some("周期初记录不完整,正在积累至少 5 个百分点的配对观测");
|
||||
return;
|
||||
}
|
||||
let Some(percent) = percent else {
|
||||
self.unavailable_reason = Some("已用比例未知,请刷新额度后查看预测。");
|
||||
self.unavailable_reason = Some("已用比例未知,请刷新账号额度");
|
||||
return;
|
||||
};
|
||||
let Some(usage) = usage.filter(|usage| usage.request_count > 0) else {
|
||||
self.unavailable_reason = Some("本周期没有网关用量记录,暂时无法预测额度。");
|
||||
self.unavailable_reason = Some("本周期没有网关用量记录,无法预测额度");
|
||||
return;
|
||||
};
|
||||
let Some(sample) = sample.filter(|sample| {
|
||||
@@ -172,8 +169,7 @@ impl AccountQuotaForecast {
|
||||
&& sample.sampled_percent >= MIN_USED_PERCENT
|
||||
&& sample.sampled_percent <= percent
|
||||
}) else {
|
||||
self.unavailable_reason =
|
||||
Some("有效额度进度不足 5 个百分点或采样边界无效,请继续积累用量。");
|
||||
self.unavailable_reason = Some("有效额度进度不足 5 个百分点或采样边界无效");
|
||||
return;
|
||||
};
|
||||
self.low_sample = sample.sampled_percent < LOW_SAMPLE_PERCENT
|
||||
@@ -196,7 +192,7 @@ impl AccountQuotaForecast {
|
||||
.and_then(|(used, remaining)| estimate(used + remaining, factor));
|
||||
self.unavailable_reason = if self.estimated_tokens.is_none() && self.estimated_usd.is_none()
|
||||
{
|
||||
Some("本周期暂无可用于估算的 Token 或费用数据,请积累用量后重试。")
|
||||
Some("本周期缺少可用于估算的 Token 或费用数据")
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
@@ -65,13 +65,7 @@ pub struct RuntimeSettingsValues {
|
||||
pub usage_retention_days: u32,
|
||||
pub ops_event_retention_days: u32,
|
||||
pub audit_retention_days: u32,
|
||||
pub account_auto_freeze_enabled: bool,
|
||||
pub account_auto_freeze_threshold: u32,
|
||||
pub account_auto_freeze_window_seconds: u64,
|
||||
pub account_auto_freeze_duration_seconds: u64,
|
||||
pub account_auto_freeze_probe_enabled: bool,
|
||||
pub account_auto_freeze_probe_model: Option<String>,
|
||||
pub account_auto_freeze_adaptive_concurrency: bool,
|
||||
pub account_error_protection: gateway_core::account::AccountErrorProtection,
|
||||
pub account_warmup_enabled: bool,
|
||||
pub account_warmup_schedule_time: String,
|
||||
pub account_warmup_model: Option<String>,
|
||||
@@ -232,12 +226,7 @@ impl RuntimeSettingsValues {
|
||||
self.ops_event_retention_days,
|
||||
self.audit_retention_days,
|
||||
)?;
|
||||
gateway_core::provider_ports::ProviderFreezePolicy::validate_values(
|
||||
self.account_auto_freeze_threshold,
|
||||
self.account_auto_freeze_window_seconds,
|
||||
self.account_auto_freeze_duration_seconds,
|
||||
self.account_auto_freeze_probe_model.as_deref(),
|
||||
)?;
|
||||
self.account_error_protection.validate()?;
|
||||
gateway_core::provider_ports::ProviderWarmupPolicy::validate_values(
|
||||
self.account_warmup_enabled,
|
||||
&self.account_warmup_schedule_time,
|
||||
|
||||
@@ -136,9 +136,10 @@ pub trait BackupRepository: Send + Sync {
|
||||
timezone: &str,
|
||||
) -> AdminStoreResult<bool>;
|
||||
|
||||
/// 按完成时间倒序读取 completed 计划备份,供保留策略扫描(最多 `limit` 条)
|
||||
/// 按完成时间及 ID 倒序分页读取 completed 计划备份,offset 相对完整集合
|
||||
async fn list_scheduled_completed_desc(
|
||||
&self,
|
||||
offset: u64,
|
||||
limit: u32,
|
||||
) -> AdminStoreResult<Vec<BackupRecord>>;
|
||||
}
|
||||
@@ -388,6 +389,7 @@ impl BackupRepository for UnavailableBackupRepository {
|
||||
}
|
||||
async fn list_scheduled_completed_desc(
|
||||
&self,
|
||||
_offset: u64,
|
||||
_limit: u32,
|
||||
) -> AdminStoreResult<Vec<BackupRecord>> {
|
||||
Err(disabled_store())
|
||||
|
||||
@@ -148,7 +148,7 @@ pub trait ProviderAdmin: Send + Sync {
|
||||
> {
|
||||
Err(gateway_core::settings::privacy::PrivacyError {
|
||||
rule_index: 0,
|
||||
reason: "Provider 不支持隐私策略",
|
||||
reason: "该服务商不支持隐私策略",
|
||||
})
|
||||
}
|
||||
|
||||
@@ -161,7 +161,7 @@ pub trait ProviderAdmin: Send + Sync {
|
||||
> {
|
||||
Err(gateway_core::settings::privacy::PrivacyError {
|
||||
rule_index: 0,
|
||||
reason: "Provider 不支持隐私策略",
|
||||
reason: "该服务商不支持隐私策略",
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -30,7 +30,8 @@ use crate::model::{
|
||||
DashboardObservation, DashboardQuery, DiagnosticDimension, DiagnosticsObservation,
|
||||
Granularity, OpsErrorPage, OpsErrorQuery, RequestMetricPoint, TimeRange,
|
||||
UsageBillingProjection, UsageCalculatedBillingFact, UsageDetail, UsageFilter,
|
||||
UsageInsightsObservation, UsagePage, UsageQuery, UsageSummaryObservation,
|
||||
UsageInsightMetric, UsageInsightsObservation, UsagePage, UsageQuery,
|
||||
UsageSummaryObservation,
|
||||
},
|
||||
provider_credentials::{
|
||||
AuthorizationCommit, CredentialDetails, CredentialImportCommit, CredentialImportResult,
|
||||
@@ -209,14 +210,6 @@ pub trait AccountStore:
|
||||
context: &MutationContext,
|
||||
) -> AdminStoreResult<AccountUpdateResult>;
|
||||
|
||||
/// 在事务内按最新启用状态、账号上限和全局默认值判断,只降低并发上限
|
||||
async fn lower_concurrency_limit(
|
||||
&self,
|
||||
account_id: &gateway_core::account::ProviderAccountId,
|
||||
limit: gateway_core::account::AccountConcurrencyLimit,
|
||||
context: &MutationContext,
|
||||
) -> AdminStoreResult<Option<AccountUpdateResult>>;
|
||||
|
||||
async fn recover_account(
|
||||
&self,
|
||||
account_id: &gateway_core::account::ProviderAccountId,
|
||||
@@ -259,17 +252,11 @@ pub trait AccountRuntimeStore: Send + Sync {
|
||||
account_ids: &[String],
|
||||
) -> AdminStoreResult<AccountRuntimeSnapshot>;
|
||||
|
||||
/// 容量熔断自动冻结中的账号与其冻结截止时间;429 临时限流不包含在内
|
||||
/// 账号错误保护中的账号与其冻结快照;临时限流不包含在内
|
||||
async fn active_freezes(
|
||||
&self,
|
||||
) -> AdminStoreResult<BTreeMap<String, crate::model::accounts::AccountFreeze>>;
|
||||
|
||||
/// 读取容量失败窗口内观测到的在途并发峰值(自适应并发下调的证据)
|
||||
async fn capacity_peaks(
|
||||
&self,
|
||||
account_ids: &[String],
|
||||
) -> AdminStoreResult<BTreeMap<String, u32>>;
|
||||
|
||||
/// 仅当冻结快照仍匹配时解除或顺延;旧探测不得覆盖手动恢复或新一轮冻结
|
||||
async fn finish_freeze(
|
||||
&self,
|
||||
@@ -470,11 +457,22 @@ pub trait ObservabilityStore: Send + Sync {
|
||||
) -> AdminStoreResult<Vec<RequestMetricPoint>>;
|
||||
|
||||
/// 同一读取快照中的汇总、趋势和已投影情景费用
|
||||
/// 不需要并发时关闭 include_concurrency,返回空并发序列并跳过明细计算
|
||||
async fn usage_insights_observation(
|
||||
&self,
|
||||
range: TimeRange,
|
||||
filter: UsageFilter,
|
||||
granularity: Granularity,
|
||||
include_concurrency: bool,
|
||||
) -> AdminStoreResult<UsageInsightsObservation>;
|
||||
|
||||
/// 只读取所选指标及其可见汇总依赖,未选择的字段不供业务层消费
|
||||
async fn usage_metric_observation(
|
||||
&self,
|
||||
range: TimeRange,
|
||||
filter: UsageFilter,
|
||||
granularity: Granularity,
|
||||
metric: UsageInsightMetric,
|
||||
) -> AdminStoreResult<UsageInsightsObservation>;
|
||||
|
||||
async fn list_usage_records(&self, query: UsageQuery) -> AdminStoreResult<UsagePage>;
|
||||
|
||||
@@ -38,11 +38,13 @@ pub enum SystemOperationErrorKind {
|
||||
}
|
||||
|
||||
/// 不泄漏路径、命令行或发布凭据的系统操作错误
|
||||
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
|
||||
#[error("system operation failed: {message}")]
|
||||
#[derive(Debug, Clone, thiserror::Error)]
|
||||
#[error("{message}")]
|
||||
pub struct SystemOperationError {
|
||||
kind: SystemOperationErrorKind,
|
||||
message: String,
|
||||
source: Option<gateway_core::error::ErrorSource>,
|
||||
sensitive_context_redacted: bool,
|
||||
}
|
||||
|
||||
impl SystemOperationError {
|
||||
@@ -51,9 +53,37 @@ impl SystemOperationError {
|
||||
Self {
|
||||
kind,
|
||||
message: message.into(),
|
||||
source: None,
|
||||
sensitive_context_redacted: false,
|
||||
}
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn with_source(mut self, source: impl Into<gateway_core::error::ErrorSource>) -> Self {
|
||||
self.source = Some(source.into());
|
||||
self
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn redact_sensitive_context(mut self) -> Self {
|
||||
self.sensitive_context_redacted = true;
|
||||
self
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub const fn sensitive_context_redacted(&self) -> bool {
|
||||
self.sensitive_context_redacted
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn error_details(&self) -> Option<gateway_core::error::ErrorDetails> {
|
||||
gateway_core::error::ErrorDetails::capture(
|
||||
self.source.as_ref(),
|
||||
None,
|
||||
self.sensitive_context_redacted,
|
||||
)
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub const fn kind(&self) -> SystemOperationErrorKind {
|
||||
self.kind
|
||||
|
||||
@@ -88,13 +88,6 @@ pub trait AccountsService: Send + Sync {
|
||||
command: UpdateAccount,
|
||||
) -> Result<AccountUpdateResult, AdminError>;
|
||||
|
||||
async fn lower_concurrency_limit(
|
||||
&self,
|
||||
context: &MutationContext,
|
||||
account_id: ProviderAccountId,
|
||||
limit: gateway_core::account::AccountConcurrencyLimit,
|
||||
) -> Result<Option<AccountUpdateResult>, AdminError>;
|
||||
|
||||
async fn batch_update(
|
||||
&self,
|
||||
context: &MutationContext,
|
||||
@@ -123,14 +116,14 @@ pub trait AccountsService: Send + Sync {
|
||||
&self,
|
||||
_account_id: &ProviderAccountId,
|
||||
) -> Result<AccountPersonalInfo, AdminError> {
|
||||
Err(AdminError::invalid("当前 Provider 不支持个人信息"))
|
||||
Err(AdminError::invalid("该服务商不支持个人信息查询"))
|
||||
}
|
||||
|
||||
async fn profile_avatar(
|
||||
&self,
|
||||
_account_id: &ProviderAccountId,
|
||||
) -> Result<ProviderProfileAvatar, AdminError> {
|
||||
Err(AdminError::invalid("当前 Provider 不支持账号头像"))
|
||||
Err(AdminError::invalid("该服务商不支持账号头像"))
|
||||
}
|
||||
|
||||
async fn reset_credit_automation_status(
|
||||
@@ -161,7 +154,7 @@ pub trait AccountsService: Send + Sync {
|
||||
_context: &MutationContext,
|
||||
_account_id: ProviderAccountId,
|
||||
) -> Result<ProviderResetCredits, AdminError> {
|
||||
Err(AdminError::invalid("当前 Provider 不支持重置额度"))
|
||||
Err(AdminError::invalid("该服务商不支持额度重置"))
|
||||
}
|
||||
|
||||
async fn consume_reset_credit(
|
||||
@@ -169,7 +162,7 @@ pub trait AccountsService: Send + Sync {
|
||||
_context: &MutationContext,
|
||||
_command: ConsumeProviderResetCredit,
|
||||
) -> Result<ProviderResetCreditResult, AdminError> {
|
||||
Err(AdminError::invalid("当前 Provider 不支持重置额度"))
|
||||
Err(AdminError::invalid("该服务商不支持额度重置"))
|
||||
}
|
||||
|
||||
async fn models(
|
||||
@@ -322,7 +315,7 @@ impl DefaultAccountsService {
|
||||
.load_account(account_id.as_str(), runtime)
|
||||
.await
|
||||
.map_err(|error| map_store_error(error, "provider account"))?
|
||||
.ok_or_else(|| AdminError::not_found("Provider 账号不存在"))
|
||||
.ok_or_else(|| AdminError::not_found("服务商账号不存在"))
|
||||
}
|
||||
|
||||
async fn provider_for_account(
|
||||
@@ -474,6 +467,7 @@ impl DefaultAccountsService {
|
||||
.and_then(|(window, _)| window.local_usage.clone())
|
||||
});
|
||||
Ok(AccountDirectoryItem {
|
||||
protection_rules: stored.protection_rules,
|
||||
capacity: stored.capacity,
|
||||
capabilities: provider
|
||||
.account_capabilities(account_id, &stored.account.authentication_kind),
|
||||
@@ -570,7 +564,7 @@ impl AccountsService for DefaultAccountsService {
|
||||
.zip(quotas)
|
||||
.map(|(mut item, quota)| {
|
||||
let id = ProviderAccountId::new(item.account.id.clone())
|
||||
.map_err(|_| AdminError::invalid("Provider 账号 ID 不合法"))?;
|
||||
.map_err(|_| AdminError::invalid("服务商账号 ID 不合法"))?;
|
||||
let capabilities = providers
|
||||
.require(&item.account.provider_kind)
|
||||
.map(|provider| {
|
||||
@@ -583,6 +577,7 @@ impl AccountsService for DefaultAccountsService {
|
||||
.and_then(|(window, _)| window.local_usage.clone())
|
||||
});
|
||||
Ok(AccountDirectoryItem {
|
||||
protection_rules: item.protection_rules,
|
||||
capacity: crate::model::accounts::AccountCapacity {
|
||||
used_slots: in_flight
|
||||
.as_ref()
|
||||
@@ -748,7 +743,7 @@ impl AccountsService for DefaultAccountsService {
|
||||
command: UpdateAccount,
|
||||
) -> Result<AccountUpdateResult, AdminError> {
|
||||
let account_id = ProviderAccountId::new(command.account_id.clone())
|
||||
.map_err(|_| AdminError::invalid("Provider 账号 ID 不合法"))?;
|
||||
.map_err(|_| AdminError::invalid("服务商账号 ID 不合法"))?;
|
||||
let (_, provider) = self.provider_for_account(&account_id).await?;
|
||||
let enabled = command.enabled;
|
||||
let result = self
|
||||
@@ -766,27 +761,6 @@ impl AccountsService for DefaultAccountsService {
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
async fn lower_concurrency_limit(
|
||||
&self,
|
||||
context: &MutationContext,
|
||||
account_id: ProviderAccountId,
|
||||
limit: gateway_core::account::AccountConcurrencyLimit,
|
||||
) -> Result<Option<AccountUpdateResult>, AdminError> {
|
||||
let (_, provider) = self.provider_for_account(&account_id).await?;
|
||||
let result = self
|
||||
.accounts
|
||||
.lower_concurrency_limit(&account_id, limit, context)
|
||||
.await
|
||||
.map_err(|error| map_store_error(error, "provider account"))?;
|
||||
if let Some(result) = &result {
|
||||
provider
|
||||
.account_facts_changed(std::slice::from_ref(&account_id))
|
||||
.await;
|
||||
publish_committed(self.snapshot.as_ref(), result.config_revision).await?;
|
||||
}
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
async fn batch_update(
|
||||
&self,
|
||||
context: &MutationContext,
|
||||
@@ -797,7 +771,7 @@ impl AccountsService for DefaultAccountsService {
|
||||
.iter()
|
||||
.map(|id| {
|
||||
ProviderAccountId::new(id.clone())
|
||||
.map_err(|_| AdminError::invalid("Provider 账号 ID 不合法"))
|
||||
.map_err(|_| AdminError::invalid("服务商账号 ID 不合法"))
|
||||
})
|
||||
.collect::<Result<Vec<_>, _>>()?;
|
||||
let mut providers = BTreeMap::<
|
||||
|
||||
@@ -326,17 +326,15 @@ impl DefaultBackupService {
|
||||
/// 校验存储配置;空串视为非法,Secret 缺省表示保留旧值
|
||||
fn validate_storage(command: &UpdateBackupStorageCommand) -> Result<(), AdminError> {
|
||||
validate_endpoint(&command.endpoint)?;
|
||||
validate_nonempty("region", &command.region)?;
|
||||
validate_nonempty("bucket", &command.bucket)?;
|
||||
validate_nonempty("accessKeyId", &command.access_key_id)?;
|
||||
validate_nonempty("存储区域", &command.region)?;
|
||||
validate_nonempty("存储桶名称", &command.bucket)?;
|
||||
validate_nonempty("访问密钥 ID", &command.access_key_id)?;
|
||||
if let Some(secret) = command.secret_access_key.as_ref() {
|
||||
let value = secret.expose_secret();
|
||||
if value.is_empty() {
|
||||
return Err(AdminError::bad_request(
|
||||
"secretAccessKey 不能为空字符串;缺省表示保留旧值",
|
||||
));
|
||||
return Err(AdminError::bad_request("访问密钥不能为空"));
|
||||
}
|
||||
validate_length("secretAccessKey", value, 512)?;
|
||||
validate_length("访问密钥", value, 512)?;
|
||||
}
|
||||
crate::model::backup::normalize_prefix(&command.prefix).map_err(map_backup_error)?;
|
||||
Ok(())
|
||||
@@ -344,17 +342,15 @@ fn validate_storage(command: &UpdateBackupStorageCommand) -> Result<(), AdminErr
|
||||
|
||||
fn validate_endpoint(value: &str) -> Result<(), AdminError> {
|
||||
if value.is_empty() {
|
||||
return Err(AdminError::bad_request("endpoint 不能为空"));
|
||||
return Err(AdminError::bad_request("备份存储地址不能为空"));
|
||||
}
|
||||
if !value.starts_with("https://") {
|
||||
return Err(AdminError::bad_request(
|
||||
"endpoint 必须使用 HTTPS;仅测试或显式开发配置允许 HTTP",
|
||||
));
|
||||
return Err(AdminError::bad_request("备份存储地址必须使用 HTTPS"));
|
||||
}
|
||||
if value.contains('@') || value.bytes().any(|byte| byte.is_ascii_control()) {
|
||||
return Err(AdminError::bad_request("endpoint 格式非法"));
|
||||
return Err(AdminError::bad_request("备份存储地址格式不合法"));
|
||||
}
|
||||
validate_length("endpoint", value, 2048)
|
||||
validate_length("备份存储地址", value, 2048)
|
||||
}
|
||||
|
||||
fn validate_nonempty(field: &str, value: &str) -> Result<(), AdminError> {
|
||||
|
||||
@@ -94,7 +94,7 @@ impl ClientKeyService for DefaultClientKeyService {
|
||||
.get_client_key(id)
|
||||
.await
|
||||
.map_err(|error| map_store_error(error, "client API key"))?
|
||||
.ok_or_else(|| AdminError::not_found("Client API Key 不存在"))
|
||||
.ok_or_else(|| AdminError::not_found("API Key 不存在"))
|
||||
}
|
||||
|
||||
async fn budget(&self, id: &ClientApiKeyId) -> Result<ClientBudgetStatus, AdminError> {
|
||||
@@ -149,7 +149,7 @@ impl ClientKeyService for DefaultClientKeyService {
|
||||
.reveal_client_key(id)
|
||||
.await
|
||||
.map_err(|error| map_store_error(error, "client API key"))?
|
||||
.ok_or_else(|| AdminError::not_found("Client API Key 不存在"))
|
||||
.ok_or_else(|| AdminError::not_found("API Key 不存在"))
|
||||
}
|
||||
|
||||
async fn create(
|
||||
@@ -158,7 +158,7 @@ impl ClientKeyService for DefaultClientKeyService {
|
||||
command: CreateClientKey,
|
||||
) -> Result<CreatedClientKey, AdminError> {
|
||||
crate::model::client_keys::validate_group_ids(&command.group_ids)
|
||||
.map_err(|_| AdminError::invalid("Client API Key 分组不合法"))?;
|
||||
.map_err(|_| AdminError::invalid("API Key 分组不合法"))?;
|
||||
for (provider, profile) in &command.request_profile_overrides {
|
||||
self.providers
|
||||
.require(provider)
|
||||
@@ -202,7 +202,7 @@ impl ClientKeyService for DefaultClientKeyService {
|
||||
command: UpdateClientKey,
|
||||
) -> Result<ClientKeyMutation, AdminError> {
|
||||
crate::model::client_keys::validate_group_ids(&command.group_ids)
|
||||
.map_err(|_| AdminError::invalid("Client API Key 分组不合法"))?;
|
||||
.map_err(|_| AdminError::invalid("API Key 分组不合法"))?;
|
||||
for (provider, profile) in &command.request_profile_override_updates {
|
||||
if let Some(profile) = profile {
|
||||
self.providers
|
||||
@@ -277,9 +277,7 @@ fn validate_cursor(query: &ClientKeyListQuery) -> Result<(), AdminError> {
|
||||
return Ok(());
|
||||
};
|
||||
if cursor.sort != query.sort {
|
||||
return Err(AdminError::invalid(
|
||||
"Client API Key 游标排序与查询条件不一致",
|
||||
));
|
||||
return Err(AdminError::invalid("分页游标与排序条件不一致"));
|
||||
}
|
||||
let matches = matches!(
|
||||
(cursor.sort.field, &cursor.value),
|
||||
@@ -300,7 +298,7 @@ fn validate_cursor(query: &ClientKeyListQuery) -> Result<(), AdminError> {
|
||||
if matches {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(AdminError::invalid("Client API Key 游标不合法"))
|
||||
Err(AdminError::invalid("API Key 分页游标不合法"))
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -65,7 +65,7 @@ pub(super) async fn required_credential(
|
||||
.credential_details(provider_kind, account_id)
|
||||
.await
|
||||
.map_err(|error| map_store_error(error, resource))?
|
||||
.ok_or_else(|| AdminError::not_found("Provider 凭据不存在"))
|
||||
.ok_or_else(|| AdminError::not_found("服务商凭据不存在"))
|
||||
}
|
||||
|
||||
pub(super) async fn required_plugin_credential(
|
||||
@@ -77,7 +77,7 @@ pub(super) async fn required_plugin_credential(
|
||||
.credential_details_by_id(account_id)
|
||||
.await
|
||||
.map_err(|error| map_store_error(error, resource))?
|
||||
.ok_or_else(|| AdminError::not_found("Provider 凭据不存在"))
|
||||
.ok_or_else(|| AdminError::not_found("服务商凭据不存在"))
|
||||
}
|
||||
|
||||
pub(super) async fn import_proxy_binding(
|
||||
@@ -156,7 +156,7 @@ pub(super) fn validate_prepared_import(
|
||||
.iter()
|
||||
.any(|credential| credential.provider_kind != *provider_kind)
|
||||
{
|
||||
return Err(AdminError::conflict("Provider 准备结果与请求范围不一致"));
|
||||
return Err(AdminError::conflict("服务商返回的凭据与请求范围不符"));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
@@ -176,7 +176,7 @@ pub(super) fn validate_prepared_rotation_facts(
|
||||
if facts.account_id.as_str() != account.id.as_str()
|
||||
|| facts.provider_kind != account.provider_kind
|
||||
{
|
||||
return Err(AdminError::conflict("Provider 准备结果与当前凭据不一致"));
|
||||
return Err(AdminError::conflict("凭据处理结果与当前凭据不符"));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -42,15 +42,17 @@ pub(super) fn map_provider_error(
|
||||
ProviderAdminErrorKind::BadGateway => AdminErrorKind::BadGateway,
|
||||
ProviderAdminErrorKind::Internal => AdminErrorKind::Internal,
|
||||
};
|
||||
let message = match kind {
|
||||
AdminErrorKind::Invalid => "Provider 请求不合法",
|
||||
AdminErrorKind::NotFound => "Provider 资源不存在",
|
||||
AdminErrorKind::Conflict => "Provider 资源状态冲突,请刷新后重试",
|
||||
AdminErrorKind::BadGateway => "上游服务请求失败",
|
||||
AdminErrorKind::UpstreamResultUnknown => "上游执行结果未知,请刷新状态后再决定是否重试",
|
||||
AdminErrorKind::Unavailable => "Provider 服务暂不可用",
|
||||
AdminErrorKind::Internal => "服务内部错误",
|
||||
_ => "Provider 操作失败",
|
||||
let message = match error.kind() {
|
||||
ProviderAdminErrorKind::Invalid => "服务商请求参数不合法",
|
||||
ProviderAdminErrorKind::Unsupported => "该服务商不支持此操作",
|
||||
ProviderAdminErrorKind::NotFound => "服务商资源不存在",
|
||||
ProviderAdminErrorKind::Conflict => "服务商资源状态冲突,请刷新后重试",
|
||||
ProviderAdminErrorKind::BadGateway => "上游服务请求失败",
|
||||
ProviderAdminErrorKind::Ambiguous => "上游执行结果未知,请刷新状态后再决定是否重试",
|
||||
ProviderAdminErrorKind::Unavailable | ProviderAdminErrorKind::CredentialRefreshRequired => {
|
||||
"服务商暂不可用"
|
||||
}
|
||||
ProviderAdminErrorKind::Internal => "服务内部错误",
|
||||
};
|
||||
AdminError::new(kind, error.public_message().unwrap_or(message)).with_source(OperationFailure {
|
||||
operation: resource,
|
||||
|
||||
@@ -194,7 +194,8 @@ impl KeyUsageService for DefaultKeyUsageService {
|
||||
self.observations.usage_insights_observation(
|
||||
query.range,
|
||||
filter,
|
||||
Granularity::for_range(query.range)
|
||||
Granularity::for_range(query.range),
|
||||
false,
|
||||
),
|
||||
self.observations.usage_trend(
|
||||
today,
|
||||
|
||||
@@ -19,10 +19,14 @@ use crate::{
|
||||
DiagnosticDimension, DiagnosticsItem, DiagnosticsResult, Granularity, HealthStatus,
|
||||
HealthTimeline, HealthTimelinePoint, OpsErrorPage, OpsErrorQuery, ProviderBillingInput,
|
||||
RequestMetricPoint, RequestMetrics, TimeRange, Trend, TrendKind, TrendPoint,
|
||||
TrendSummary, UsageBilling, UsageBillingProjection, UsageCalculatedBillingFact,
|
||||
UsageDetail, UsageFilter, UsageInsights, UsageInsightsCost, UsageInsightsCostPoint,
|
||||
UsageInsightsHealth, UsageInsightsHealthPoint, UsageInsightsPerformance,
|
||||
UsageInsightsPerformancePoint, UsageOverview, UsagePage, UsageQuery, UsageSummary,
|
||||
TrendSummary, UsageBilling, UsageBillingProjection, UsageCacheEfficiency,
|
||||
UsageCalculatedBillingFact, UsageConcurrencyPoint, UsageCostComparison,
|
||||
UsageCostSummary, UsageDetail, UsageFilter, UsageInsight, UsageInsightData,
|
||||
UsageInsightMetric, UsageInsights, UsageInsightsCost, UsageInsightsCostPoint,
|
||||
UsageInsightsHealth, UsageInsightsHealthPoint, UsageInsightsLoad,
|
||||
UsageInsightsObservation, UsageInsightsPerformance, UsageInsightsPerformancePoint,
|
||||
UsageMetricPoint, UsageMetricSeries, UsageOverview, UsagePage, UsageQuery,
|
||||
UsageSummary, UsageThroughputPercentiles,
|
||||
},
|
||||
provider_credentials::ProviderQuotaRequest,
|
||||
},
|
||||
@@ -64,6 +68,12 @@ pub trait ObservabilityService: Send + Sync {
|
||||
range: TimeRange,
|
||||
filter: UsageFilter,
|
||||
) -> Result<UsageInsights, AdminError>;
|
||||
async fn usage_metric(
|
||||
&self,
|
||||
range: TimeRange,
|
||||
filter: UsageFilter,
|
||||
metric: UsageInsightMetric,
|
||||
) -> Result<UsageInsight, AdminError>;
|
||||
async fn diagnostics(
|
||||
&self,
|
||||
range: TimeRange,
|
||||
@@ -233,6 +243,7 @@ impl ObservabilityService for DefaultObservabilityService {
|
||||
range,
|
||||
UsageFilter::default(),
|
||||
Granularity::for_range(range),
|
||||
false,
|
||||
)
|
||||
.await
|
||||
.map_err(|error| map_store_error(error, "dashboard trend"))?;
|
||||
@@ -290,21 +301,25 @@ impl ObservabilityService for DefaultObservabilityService {
|
||||
let granularity = Granularity::for_range(range);
|
||||
let observation = self
|
||||
.store
|
||||
.usage_insights_observation(range, filter, granularity)
|
||||
.usage_insights_observation(range, filter, granularity, true)
|
||||
.await
|
||||
.map_err(|error| map_store_error(error, "usage insights"))?;
|
||||
let mut standard_costs = UsageCostScenarios::default();
|
||||
for point in observation.costs {
|
||||
if let Some(amount) = point.standard_amount {
|
||||
standard_costs.standard.add(point.bucket_start, amount)?;
|
||||
}
|
||||
if let Some(amount) = point.no_cache_amount {
|
||||
standard_costs.no_cache.add(point.bucket_start, amount)?;
|
||||
}
|
||||
}
|
||||
let overview = observation.overview;
|
||||
let trend = observation.trend;
|
||||
build_usage_insights(overview, trend, standard_costs)
|
||||
insights_from_observation(observation)
|
||||
}
|
||||
|
||||
async fn usage_metric(
|
||||
&self,
|
||||
range: TimeRange,
|
||||
filter: UsageFilter,
|
||||
metric: UsageInsightMetric,
|
||||
) -> Result<UsageInsight, AdminError> {
|
||||
let observation = self
|
||||
.store
|
||||
.usage_metric_observation(range, filter, Granularity::for_range(range), metric)
|
||||
.await
|
||||
.map_err(|error| map_store_error(error, "usage metric"))?;
|
||||
let insights = insights_from_observation(observation)?;
|
||||
Ok(select_usage_metric(insights, metric))
|
||||
}
|
||||
|
||||
async fn diagnostics(
|
||||
@@ -381,10 +396,153 @@ impl ObservabilityService for DefaultObservabilityService {
|
||||
}
|
||||
}
|
||||
|
||||
fn insights_from_observation(
|
||||
observation: UsageInsightsObservation,
|
||||
) -> Result<UsageInsights, AdminError> {
|
||||
let mut standard_costs = UsageCostScenarios::default();
|
||||
for point in observation.costs {
|
||||
if let Some(amount) = point.standard_amount {
|
||||
standard_costs.standard.add(point.bucket_start, amount)?;
|
||||
}
|
||||
if let Some(amount) = point.no_cache_amount {
|
||||
standard_costs.no_cache.add(point.bucket_start, amount)?;
|
||||
}
|
||||
}
|
||||
let overview = observation.overview;
|
||||
let trend = observation.trend;
|
||||
build_usage_insights(overview, trend, standard_costs, observation.concurrency)
|
||||
}
|
||||
|
||||
fn select_usage_metric(insights: UsageInsights, metric: UsageInsightMetric) -> UsageInsight {
|
||||
let activity = insights
|
||||
.health
|
||||
.points
|
||||
.iter()
|
||||
.map(|point| (point.bucket_start, point.total_requests))
|
||||
.collect::<BTreeMap<_, _>>();
|
||||
let data = match metric {
|
||||
UsageInsightMetric::Health => UsageInsightData::Health(insights.health),
|
||||
UsageInsightMetric::Latency => UsageInsightData::Latency(metric_series(
|
||||
insights.performance.latency_percentiles,
|
||||
insights
|
||||
.performance
|
||||
.points
|
||||
.into_iter()
|
||||
.map(|point| (point.bucket_start, point.latency_percentiles)),
|
||||
&activity,
|
||||
)),
|
||||
UsageInsightMetric::FirstToken => UsageInsightData::FirstToken(metric_series(
|
||||
insights.performance.first_token_latency_percentiles,
|
||||
insights
|
||||
.performance
|
||||
.points
|
||||
.into_iter()
|
||||
.map(|point| (point.bucket_start, point.first_token_latency_percentiles)),
|
||||
&activity,
|
||||
)),
|
||||
UsageInsightMetric::Throughput => UsageInsightData::Throughput(metric_series(
|
||||
UsageThroughputPercentiles {
|
||||
p10: insights.performance.output_throughput_p10,
|
||||
p50: insights.performance.output_throughput_p50,
|
||||
p90: insights.performance.output_throughput_p90,
|
||||
},
|
||||
insights.performance.points.into_iter().map(|point| {
|
||||
(
|
||||
point.bucket_start,
|
||||
UsageThroughputPercentiles {
|
||||
p10: point.output_throughput_p10,
|
||||
p50: point.output_throughput_p50,
|
||||
p90: point.output_throughput_p90,
|
||||
},
|
||||
)
|
||||
}),
|
||||
&activity,
|
||||
)),
|
||||
UsageInsightMetric::Load => UsageInsightData::Load(metric_series(
|
||||
insights.performance.load,
|
||||
insights
|
||||
.performance
|
||||
.points
|
||||
.into_iter()
|
||||
.map(|point| (point.bucket_start, point.load)),
|
||||
&activity,
|
||||
)),
|
||||
UsageInsightMetric::Cost => UsageInsightData::Cost(metric_series(
|
||||
cost_summary(&insights.cost),
|
||||
insights.cost.points.into_iter().map(|point| {
|
||||
(
|
||||
point.bucket_start,
|
||||
UsageCostComparison {
|
||||
estimated_cost: point.estimated_cost,
|
||||
no_cache_cost: point.no_cache_cost,
|
||||
},
|
||||
)
|
||||
}),
|
||||
&activity,
|
||||
)),
|
||||
UsageInsightMetric::Savings => UsageInsightData::Savings(metric_series(
|
||||
cost_summary(&insights.cost),
|
||||
insights
|
||||
.cost
|
||||
.points
|
||||
.into_iter()
|
||||
.map(|point| (point.bucket_start, point.cache_savings)),
|
||||
&activity,
|
||||
)),
|
||||
UsageInsightMetric::Cache => UsageInsightData::Cache(metric_series(
|
||||
cost_summary(&insights.cost),
|
||||
insights.cost.points.into_iter().map(|point| {
|
||||
(
|
||||
point.bucket_start,
|
||||
UsageCacheEfficiency {
|
||||
input_tokens: point.input_tokens,
|
||||
cached_token_rate: point.cached_token_rate,
|
||||
cache_hit_request_rate: point.cache_hit_request_rate,
|
||||
},
|
||||
)
|
||||
}),
|
||||
&activity,
|
||||
)),
|
||||
};
|
||||
UsageInsight {
|
||||
analytics: insights.analytics,
|
||||
granularity: insights.granularity,
|
||||
data,
|
||||
}
|
||||
}
|
||||
|
||||
fn cost_summary(cost: &UsageInsightsCost) -> UsageCostSummary {
|
||||
UsageCostSummary {
|
||||
estimated_cost: cost.estimated_cost.clone(),
|
||||
cache_savings: cost.cache_savings.clone(),
|
||||
tier_premium: cost.tier_premium.clone(),
|
||||
cost_per_successful_request: cost.cost_per_successful_request.clone(),
|
||||
coverage: cost.coverage.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
fn metric_series<S, P>(
|
||||
summary: S,
|
||||
points: impl Iterator<Item = (DateTime<Utc>, P)>,
|
||||
activity: &BTreeMap<DateTime<Utc>, u64>,
|
||||
) -> UsageMetricSeries<S, P> {
|
||||
UsageMetricSeries {
|
||||
summary,
|
||||
points: points
|
||||
.map(|(bucket_start, value)| UsageMetricPoint {
|
||||
bucket_start,
|
||||
request_count: activity.get(&bucket_start).copied().unwrap_or_default(),
|
||||
value,
|
||||
})
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
|
||||
fn build_usage_insights(
|
||||
overview: UsageOverview,
|
||||
trend: Vec<RequestMetricPoint>,
|
||||
standard_costs: UsageCostScenarios,
|
||||
concurrency: Vec<UsageConcurrencyPoint>,
|
||||
) -> Result<UsageInsights, AdminError> {
|
||||
let granularity = trend.first().map_or(
|
||||
crate::model::observability::Granularity::FifteenMinutes,
|
||||
@@ -432,49 +590,69 @@ fn build_usage_insights(
|
||||
})
|
||||
.collect(),
|
||||
};
|
||||
let concurrency_by_bucket = concurrency
|
||||
.iter()
|
||||
.map(|point| (point.bucket_start, point))
|
||||
.collect::<BTreeMap<_, _>>();
|
||||
let mut weighted_concurrency_micros = 0.0;
|
||||
let mut effective_micros = 0_i64;
|
||||
for (index, point) in concurrency.iter().enumerate() {
|
||||
let start = point.bucket_start.max(overview.range.start);
|
||||
let end = concurrency
|
||||
.get(index + 1)
|
||||
.map_or(overview.range.end, |next| next.bucket_start)
|
||||
.min(overview.range.end);
|
||||
let micros = (end - start).num_microseconds().unwrap_or(0).max(0);
|
||||
effective_micros = effective_micros.saturating_add(micros);
|
||||
weighted_concurrency_micros += point.average_concurrency * micros as f64;
|
||||
}
|
||||
let average_concurrency = if effective_micros > 0 {
|
||||
weighted_concurrency_micros / effective_micros as f64
|
||||
} else {
|
||||
0.0
|
||||
};
|
||||
let peak_concurrency = concurrency
|
||||
.iter()
|
||||
.map(|point| point.peak_concurrency)
|
||||
.max()
|
||||
.unwrap_or(0);
|
||||
let performance = UsageInsightsPerformance {
|
||||
latency_percentiles: requests.latency_percentiles,
|
||||
first_token_latency_percentiles: requests.first_token_latency_percentiles,
|
||||
latency_coverage: rate_or_zero(requests.latency_count, requests.request_count),
|
||||
first_token_coverage: rate_or_zero(
|
||||
requests.first_token_latency_count,
|
||||
requests.request_count,
|
||||
),
|
||||
admission_decision_percentiles: requests.admission_decision_percentiles,
|
||||
account_selection_wait_percentiles: requests.account_selection_wait_percentiles,
|
||||
admission_decision_coverage: rate_or_zero(
|
||||
requests.admission_decision_count,
|
||||
requests.request_count,
|
||||
),
|
||||
account_selection_wait_coverage: rate_or_zero(
|
||||
requests.account_selection_wait_count,
|
||||
requests.request_count,
|
||||
),
|
||||
output_throughput_p10: requests.output_throughput_p10,
|
||||
output_throughput_p50: requests.output_throughput_p50,
|
||||
output_throughput_p90: requests.output_throughput_p90,
|
||||
capacity_utilization_avg_basis_points: requests.capacity_utilization_avg_basis_points,
|
||||
capacity_utilization_p95_basis_points: requests.capacity_utilization_p95_basis_points,
|
||||
capacity_coverage: rate_or_zero(requests.capacity_sample_count, requests.request_count),
|
||||
load: UsageInsightsLoad {
|
||||
average_concurrency,
|
||||
peak_concurrency,
|
||||
admission_decision_p95_ms: requests.admission_decision_percentiles.p95_ms,
|
||||
account_selection_wait_p95_ms: requests.account_selection_wait_percentiles.p95_ms,
|
||||
},
|
||||
points: trend
|
||||
.iter()
|
||||
.map(|point| UsageInsightsPerformancePoint {
|
||||
bucket_start: point.bucket_start,
|
||||
latency_percentiles: point.metrics.latency_percentiles,
|
||||
first_token_latency_percentiles: point.metrics.first_token_latency_percentiles,
|
||||
admission_decision_percentiles: point.metrics.admission_decision_percentiles,
|
||||
account_selection_wait_percentiles: point
|
||||
.metrics
|
||||
.account_selection_wait_percentiles,
|
||||
output_throughput_p10: point.metrics.output_throughput_p10,
|
||||
output_throughput_p50: point.metrics.output_throughput_p50,
|
||||
output_throughput_p90: point.metrics.output_throughput_p90,
|
||||
capacity_utilization_avg_basis_points: point
|
||||
.metrics
|
||||
.capacity_utilization_avg_basis_points,
|
||||
capacity_utilization_p95_basis_points: point
|
||||
.metrics
|
||||
.capacity_utilization_p95_basis_points,
|
||||
.map(|point| {
|
||||
let concurrency = concurrency_by_bucket.get(&point.bucket_start);
|
||||
UsageInsightsPerformancePoint {
|
||||
bucket_start: point.bucket_start,
|
||||
latency_percentiles: point.metrics.latency_percentiles,
|
||||
first_token_latency_percentiles: point.metrics.first_token_latency_percentiles,
|
||||
output_throughput_p10: point.metrics.output_throughput_p10,
|
||||
output_throughput_p50: point.metrics.output_throughput_p50,
|
||||
output_throughput_p90: point.metrics.output_throughput_p90,
|
||||
load: UsageInsightsLoad {
|
||||
average_concurrency: concurrency
|
||||
.map_or(0.0, |point| point.average_concurrency),
|
||||
peak_concurrency: concurrency.map_or(0, |point| point.peak_concurrency),
|
||||
admission_decision_p95_ms: point
|
||||
.metrics
|
||||
.admission_decision_percentiles
|
||||
.p95_ms,
|
||||
account_selection_wait_p95_ms: point
|
||||
.metrics
|
||||
.account_selection_wait_percentiles
|
||||
.p95_ms,
|
||||
},
|
||||
}
|
||||
})
|
||||
.collect(),
|
||||
};
|
||||
|
||||
@@ -178,7 +178,8 @@ pub(crate) struct CachedObservabilityStore {
|
||||
inner: Arc<dyn ObservabilityStore>,
|
||||
dashboard: Cache<DashboardQuery, DashboardObservation>,
|
||||
summaries: Cache<UsageKey, UsageSummaryObservation>,
|
||||
insights: Cache<(UsageKey, Granularity), UsageInsightsObservation>,
|
||||
insights: Cache<(UsageKey, Granularity, bool), UsageInsightsObservation>,
|
||||
metrics: Cache<(UsageKey, Granularity, UsageInsightMetric), UsageInsightsObservation>,
|
||||
diagnostics: Cache<(UsageKey, DiagnosticDimension, u16), DiagnosticsObservation>,
|
||||
}
|
||||
impl CachedObservabilityStore {
|
||||
@@ -188,6 +189,7 @@ impl CachedObservabilityStore {
|
||||
dashboard: Cache::default(),
|
||||
summaries: Cache::default(),
|
||||
insights: Cache::default(),
|
||||
metrics: Cache::default(),
|
||||
diagnostics: Cache::default(),
|
||||
}
|
||||
}
|
||||
@@ -237,19 +239,44 @@ impl ObservabilityStore for CachedObservabilityStore {
|
||||
range: TimeRange,
|
||||
filter: UsageFilter,
|
||||
granularity: Granularity,
|
||||
include_concurrency: bool,
|
||||
) -> AdminStoreResult<UsageInsightsObservation> {
|
||||
self.insights
|
||||
.get(
|
||||
((range, filter.clone()), granularity),
|
||||
((range, filter.clone()), granularity, include_concurrency),
|
||||
self.inner.as_ref(),
|
||||
TTL,
|
||||
|| {
|
||||
self.inner
|
||||
.usage_insights_observation(range, filter, granularity)
|
||||
self.inner.usage_insights_observation(
|
||||
range,
|
||||
filter,
|
||||
granularity,
|
||||
include_concurrency,
|
||||
)
|
||||
},
|
||||
)
|
||||
.await
|
||||
}
|
||||
async fn usage_metric_observation(
|
||||
&self,
|
||||
range: TimeRange,
|
||||
filter: UsageFilter,
|
||||
granularity: Granularity,
|
||||
metric: UsageInsightMetric,
|
||||
) -> AdminStoreResult<UsageInsightsObservation> {
|
||||
self.metrics
|
||||
.get(
|
||||
((range, filter.clone()), granularity, metric),
|
||||
self.inner.as_ref(),
|
||||
TTL,
|
||||
|| {
|
||||
self.inner
|
||||
.usage_metric_observation(range, filter, granularity, metric)
|
||||
},
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
async fn usage_summary(
|
||||
&self,
|
||||
range: TimeRange,
|
||||
|
||||
@@ -104,7 +104,7 @@ impl PluginAccountAccess for DefaultPluginAccountAccess {
|
||||
.load_credential_for_plugin(account_id)
|
||||
.await
|
||||
.map_err(|error| map_store_error(error, "plugin credential"))?
|
||||
.ok_or_else(|| AdminError::not_found("Provider 凭据不存在"))?;
|
||||
.ok_or_else(|| AdminError::not_found("服务商凭据不存在"))?;
|
||||
Ok(PluginAccountCredential {
|
||||
account: credential.account,
|
||||
provider_material: credential.provider_material,
|
||||
|
||||
@@ -71,7 +71,7 @@ impl PluginsService {
|
||||
validate_update_policy(&binding)?;
|
||||
let PluginUpdateSource::Github { repository } = &binding.source else {
|
||||
return Err(AdminError::invalid(
|
||||
"该来源不提供 Release 更新查询,请显式选择并安装插件包",
|
||||
"该来源不支持更新查询,请手动安装插件包",
|
||||
));
|
||||
};
|
||||
let (tag, allow_prerelease) = match &binding.policy {
|
||||
@@ -356,9 +356,7 @@ fn validate_update_policy(binding: &PluginSourceBinding) -> Result<(), AdminErro
|
||||
if !matches!(binding.source, PluginUpdateSource::Github { .. })
|
||||
&& !matches!(binding.policy, PluginUpdatePolicy::Manual {})
|
||||
{
|
||||
return Err(AdminError::invalid(
|
||||
"只有 GitHub 来源支持 Release 更新检查策略",
|
||||
));
|
||||
return Err(AdminError::invalid("仅 GitHub 来源支持发行版本更新检查"));
|
||||
}
|
||||
if let PluginUpdatePolicy::Pinned { tag, .. } = &binding.policy
|
||||
&& (tag.is_empty()
|
||||
@@ -366,7 +364,7 @@ fn validate_update_policy(binding: &PluginSourceBinding) -> Result<(), AdminErro
|
||||
|| tag.chars().any(char::is_control)
|
||||
|| tag.trim() != tag)
|
||||
{
|
||||
return Err(AdminError::invalid("固定 Release tag 不合法"));
|
||||
return Err(AdminError::invalid("固定版本标签不合法"));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -47,14 +47,14 @@ pub trait SettingsService: Send + Sync {
|
||||
&self,
|
||||
_provider: &str,
|
||||
) -> Result<gateway_core::account::OpaqueProviderData, AdminError> {
|
||||
Err(AdminError::invalid("当前 Provider 不支持客户端身份配置"))
|
||||
Err(AdminError::invalid("该服务商不支持客户端身份配置"))
|
||||
}
|
||||
async fn preview_client_profile(
|
||||
&self,
|
||||
_provider: &str,
|
||||
_configuration: Option<&gateway_core::account::OpaqueProviderData>,
|
||||
) -> Result<gateway_core::account::OpaqueProviderData, AdminError> {
|
||||
Err(AdminError::invalid("当前 Provider 不支持客户端身份配置"))
|
||||
Err(AdminError::invalid("该服务商不支持客户端身份配置"))
|
||||
}
|
||||
async fn load(&self) -> Result<RuntimeSettings, AdminError>;
|
||||
async fn replace(
|
||||
@@ -86,7 +86,7 @@ impl DefaultSettingsService {
|
||||
provider: &str,
|
||||
) -> Result<std::sync::Arc<dyn crate::ports::provider::ProviderAdmin>, AdminError> {
|
||||
let kind = gateway_core::routing::ProviderKind::new(provider)
|
||||
.map_err(|_| AdminError::invalid("Provider 不合法"))?;
|
||||
.map_err(|_| AdminError::invalid("服务商无效"))?;
|
||||
self.providers
|
||||
.require(&kind)
|
||||
.map_err(|error| super::map_provider_error(error, "client profile"))
|
||||
@@ -170,7 +170,7 @@ impl SettingsService for DefaultSettingsService {
|
||||
.keys()
|
||||
.any(|provider| !providers.contains_key(provider))
|
||||
{
|
||||
return Err(AdminError::invalid("Provider 不支持价目管理"));
|
||||
return Err(AdminError::invalid("该服务商不支持价目管理"));
|
||||
}
|
||||
let mut current = self.pricing_source.fetch().await?;
|
||||
if current != command.preview {
|
||||
@@ -242,7 +242,7 @@ impl SettingsService for DefaultSettingsService {
|
||||
|| model.chars().any(char::is_control)
|
||||
})
|
||||
{
|
||||
return Err(AdminError::invalid("Provider、模型 ID 或批量数量不合法"));
|
||||
return Err(AdminError::invalid("服务商、模型 ID 或批量数量不合法"));
|
||||
}
|
||||
if command.change == PricingChange::Delete
|
||||
&& command.models.iter().any(|model| {
|
||||
|
||||
@@ -146,12 +146,11 @@ fn map_system_error(error: SystemOperationError) -> AdminError {
|
||||
SystemOperationErrorKind::Upstream => AdminErrorKind::BadGateway,
|
||||
SystemOperationErrorKind::Internal => AdminErrorKind::Internal,
|
||||
};
|
||||
let message = match kind {
|
||||
AdminErrorKind::Invalid => "系统操作请求不合法",
|
||||
AdminErrorKind::Conflict => error.message(),
|
||||
AdminErrorKind::BadGateway => "系统更新服务请求失败",
|
||||
AdminErrorKind::Internal => "系统操作失败",
|
||||
_ => "系统操作失败",
|
||||
let failure = AdminError::new(kind, error.message());
|
||||
let failure = if error.sensitive_context_redacted() {
|
||||
failure.redact_sensitive_context()
|
||||
} else {
|
||||
failure
|
||||
};
|
||||
AdminError::new(kind, message)
|
||||
failure.with_source(error)
|
||||
}
|
||||
|
||||
@@ -24,9 +24,9 @@ impl DefaultUsageDataService {
|
||||
reason: reason.into(),
|
||||
};
|
||||
let kind = ProviderKind::new(input.provider.clone())
|
||||
.map_err(|_| UsageDataError::invalid("Provider 无效"))?;
|
||||
.map_err(|_| UsageDataError::invalid("服务商无效"))?;
|
||||
let Ok(provider) = self.providers.require(&kind) else {
|
||||
return Ok(unavailable("Provider 没有可用的计价规则"));
|
||||
return Ok(unavailable("服务商没有可用的计价规则"));
|
||||
};
|
||||
let tokens = &input.tokens;
|
||||
let price = match &input.price {
|
||||
@@ -61,9 +61,9 @@ impl DefaultUsageDataService {
|
||||
ProviderAdminErrorKind::Unsupported | ProviderAdminErrorKind::Invalid
|
||||
) =>
|
||||
{
|
||||
return Ok(unavailable("Provider 不支持该计价输入"));
|
||||
return Ok(unavailable("服务商不支持该计价输入"));
|
||||
}
|
||||
Err(_) => return Err(UsageDataError::unavailable("Provider 计价失败")),
|
||||
Err(_) => return Err(UsageDataError::unavailable("服务商计价失败")),
|
||||
};
|
||||
Ok(UsagePricingResult::Estimated {
|
||||
id: input.id,
|
||||
|
||||
@@ -106,7 +106,7 @@ fn retention_count_keeps_newest_n() {
|
||||
Some(base),
|
||||
),
|
||||
];
|
||||
let decisions = decide_retention(0, 2, now, &records);
|
||||
let decisions = decide_retention(0, 2, now, 0, &records);
|
||||
let ids: Vec<_> = decisions.iter().map(|d| d.record_id.as_str()).collect();
|
||||
assert_eq!(ids, vec!["oldest"]);
|
||||
assert_eq!(decisions[0].reason, RetentionReason::Count);
|
||||
@@ -131,7 +131,7 @@ fn retention_days_deletes_old_scheduled_but_keeps_newest() {
|
||||
Some(now - Duration::days(40)),
|
||||
),
|
||||
];
|
||||
let decisions = decide_retention(30, 0, now, &records);
|
||||
let decisions = decide_retention(30, 0, now, 0, &records);
|
||||
let ids: Vec<_> = decisions.iter().map(|d| d.record_id.as_str()).collect();
|
||||
assert_eq!(ids, vec!["ancient"]);
|
||||
}
|
||||
@@ -146,7 +146,7 @@ fn newest_scheduled_never_auto_deleted_even_when_ancient() {
|
||||
now - Duration::days(400),
|
||||
Some(now - Duration::days(400)),
|
||||
)];
|
||||
assert!(decide_retention(30, 1, now, &records).is_empty());
|
||||
assert!(decide_retention(30, 1, now, 0, &records).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -168,7 +168,7 @@ fn disabled_thresholds_delete_nothing() {
|
||||
Some(now - Duration::days(90)),
|
||||
),
|
||||
];
|
||||
assert!(decide_retention(0, 0, now, &records).is_empty());
|
||||
assert!(decide_retention(0, 0, now, 0, &records).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -470,7 +470,8 @@ impl BackupRepository for FakeBackupRepository {
|
||||
|
||||
async fn list_scheduled_completed_desc(
|
||||
&self,
|
||||
_limit: u32,
|
||||
offset: u64,
|
||||
limit: u32,
|
||||
) -> AdminStoreResult<Vec<BackupRecord>> {
|
||||
let mut records = self
|
||||
.records
|
||||
@@ -488,7 +489,11 @@ impl BackupRepository for FakeBackupRepository {
|
||||
.cmp(&a.completed_at)
|
||||
.then_with(|| b.id.cmp(&a.id))
|
||||
});
|
||||
Ok(records)
|
||||
Ok(records
|
||||
.into_iter()
|
||||
.skip(usize::try_from(offset).unwrap_or(usize::MAX))
|
||||
.take(limit as usize)
|
||||
.collect())
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -239,6 +239,75 @@ async fn retention_cleans_expired_scheduled_backups() {
|
||||
assert!(object_store.object(&new_seed.object_key).is_some());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn retention_scans_all_pages_and_deletes_only_one_batch_per_cycle() {
|
||||
for (retention_days, retention_count, retained) in [
|
||||
(0, 999, 999),
|
||||
(0, 1000, 1000),
|
||||
(0, 1001, 1001),
|
||||
(30, 0, 1000),
|
||||
] {
|
||||
let mut settings = configured_settings();
|
||||
settings.retention_days = retention_days;
|
||||
settings.retention_count = retention_count;
|
||||
let repository = Arc::new(FakeBackupRepository::new(settings));
|
||||
let object_store = Arc::new(FakeObjectStore::new());
|
||||
let now = Utc::now();
|
||||
let mut seeds = Vec::new();
|
||||
for index in 0..retained + 15 {
|
||||
let completed_at = if retention_days > 0 && index >= retained {
|
||||
now - Duration::days(40) - Duration::minutes(index)
|
||||
} else {
|
||||
now - Duration::minutes(index)
|
||||
};
|
||||
let seed = BackupRecordSeed {
|
||||
id: backup_id(&format!("retention-{index}")),
|
||||
trigger_kind: BackupTriggerKind::Scheduled,
|
||||
scheduled_at: Some(completed_at),
|
||||
object_key: format!("codex/retention-{index}.dump"),
|
||||
expires_at: None,
|
||||
};
|
||||
repository
|
||||
.insert_backup_record(seed.clone(), &system_context())
|
||||
.await
|
||||
.expect("insert scheduled backup");
|
||||
repository.set_completed(&seed.id, completed_at);
|
||||
object_store
|
||||
.objects
|
||||
.lock()
|
||||
.expect("objects")
|
||||
.insert(seed.object_key.clone(), DUMP_CONTENT.to_vec());
|
||||
seeds.push(seed);
|
||||
}
|
||||
let task = BackupTask::new(
|
||||
repository.clone(),
|
||||
Arc::new(FakeDumpPort::new()),
|
||||
object_store.clone(),
|
||||
);
|
||||
|
||||
for deleted in [10, 15] {
|
||||
task.run_cycle(&CancellationToken::new())
|
||||
.await
|
||||
.expect("run retention cycle");
|
||||
let records = repository.all_records();
|
||||
assert_eq!(records.len(), seeds.len() - deleted);
|
||||
for (index, seed) in seeds.iter().enumerate() {
|
||||
let retained = usize::try_from(retained).unwrap();
|
||||
let should_remain = index < retained || index >= retained + deleted;
|
||||
assert_eq!(
|
||||
records.iter().any(|record| record.id == seed.id),
|
||||
should_remain,
|
||||
"record {index}, retention days {retention_days}, count {retention_count}"
|
||||
);
|
||||
assert_eq!(
|
||||
object_store.object(&seed.object_key).is_some(),
|
||||
should_remain
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn timezone_change_rebases_future_cursor_without_running_old_due_backup() {
|
||||
let mut settings = configured_settings();
|
||||
|
||||
@@ -315,6 +315,7 @@ impl AccountRuntimeStore for FakeRuntimeStore {
|
||||
) -> AdminStoreResult<AccountRuntimeSnapshot> {
|
||||
*self.requested_accounts.lock().expect("requested accounts") = account_ids.to_vec();
|
||||
Ok(AccountRuntimeSnapshot {
|
||||
protection_rules: std::collections::BTreeMap::new(),
|
||||
cooldown: BTreeMap::from([(
|
||||
"acct_limited".to_owned(),
|
||||
std::time::SystemTime::from(Utc::now() + Duration::minutes(5)).into(),
|
||||
@@ -329,13 +330,6 @@ impl AccountRuntimeStore for FakeRuntimeStore {
|
||||
Ok(BTreeMap::new())
|
||||
}
|
||||
|
||||
async fn capacity_peaks(
|
||||
&self,
|
||||
_account_ids: &[String],
|
||||
) -> AdminStoreResult<BTreeMap<String, u32>> {
|
||||
Ok(BTreeMap::new())
|
||||
}
|
||||
|
||||
async fn finish_freeze(
|
||||
&self,
|
||||
_account_id: &str,
|
||||
|
||||
@@ -617,7 +617,6 @@ pub(super) struct FakeAccountStore {
|
||||
automations: Mutex<
|
||||
BTreeMap<String, gateway_admin::model::reset_credit_automation::ResetCreditAutomation>,
|
||||
>,
|
||||
pub(super) lowered_limits: Mutex<Vec<(String, u32)>>,
|
||||
}
|
||||
|
||||
impl FakeAccountStore {
|
||||
@@ -638,7 +637,6 @@ impl FakeAccountStore {
|
||||
quota_window_queries: Mutex::new(Vec::new()),
|
||||
quota_forecast_history: Mutex::new(QuotaForecastHistory::default()),
|
||||
update_commands: Mutex::new(Vec::new()),
|
||||
lowered_limits: Mutex::new(Vec::new()),
|
||||
automations: Mutex::default(),
|
||||
})
|
||||
}
|
||||
@@ -720,6 +718,7 @@ impl FakeAccountStore {
|
||||
last_error_message: account.last_error_message.clone(),
|
||||
};
|
||||
AccountPageItem {
|
||||
protection_rules: Vec::new(),
|
||||
capacity: gateway_admin::model::accounts::AccountCapacity {
|
||||
used_slots: None,
|
||||
total_slots: account
|
||||
@@ -1036,35 +1035,6 @@ impl AccountStore for FakeAccountStore {
|
||||
})
|
||||
}
|
||||
|
||||
async fn lower_concurrency_limit(
|
||||
&self,
|
||||
account_id: &ProviderAccountId,
|
||||
limit: gateway_core::account::AccountConcurrencyLimit,
|
||||
_: &MutationContext,
|
||||
) -> AdminStoreResult<Option<AccountUpdateResult>> {
|
||||
let mut accounts = self.accounts.lock().expect("accounts");
|
||||
let Some(account) = accounts
|
||||
.iter_mut()
|
||||
.find(|account| account.id == account_id.as_str())
|
||||
else {
|
||||
return Ok(None);
|
||||
};
|
||||
if !account.enabled
|
||||
|| account.concurrency_limit.map_or(5, |value| value.get()) <= limit.get()
|
||||
{
|
||||
return Ok(None);
|
||||
}
|
||||
account.concurrency_limit = Some(limit);
|
||||
self.lowered_limits
|
||||
.lock()
|
||||
.expect("lowered limits")
|
||||
.push((account.id.clone(), limit.get()));
|
||||
Ok(Some(AccountUpdateResult {
|
||||
config_revision: revision(2),
|
||||
account_id: account_id.clone(),
|
||||
}))
|
||||
}
|
||||
|
||||
async fn recover_account(
|
||||
&self,
|
||||
account_id: &ProviderAccountId,
|
||||
@@ -1189,13 +1159,7 @@ impl SettingsStore for StaticSettingsStore {
|
||||
usage_retention_days: 30,
|
||||
ops_event_retention_days: 30,
|
||||
audit_retention_days: 30,
|
||||
account_auto_freeze_enabled: true,
|
||||
account_auto_freeze_threshold: 12,
|
||||
account_auto_freeze_window_seconds: 600,
|
||||
account_auto_freeze_duration_seconds: 7_200,
|
||||
account_auto_freeze_probe_enabled: true,
|
||||
account_auto_freeze_probe_model: None,
|
||||
account_auto_freeze_adaptive_concurrency: true,
|
||||
account_error_protection: gateway_core::account::AccountErrorProtection::default(),
|
||||
account_warmup_enabled: false,
|
||||
account_warmup_schedule_time: "08:00".to_owned(),
|
||||
account_warmup_model: None,
|
||||
@@ -3124,7 +3088,7 @@ async fn accounts_refresh_should_not_expose_the_provider_failure_message() {
|
||||
.expect_err("Provider preparation must fail");
|
||||
|
||||
assert_eq!(error.kind(), gateway_admin::model::AdminErrorKind::Conflict);
|
||||
assert_eq!(error.to_string(), "Provider 资源状态冲突,请刷新后重试");
|
||||
assert_eq!(error.to_string(), "服务商资源状态冲突,请刷新后重试");
|
||||
assert!(!error.to_string().contains(upstream_message));
|
||||
assert_eq!(
|
||||
recorded(&events),
|
||||
@@ -3680,6 +3644,7 @@ impl AccountRuntimeStore for CapacityRuntime {
|
||||
return Err(store_unavailable());
|
||||
}
|
||||
Ok(AccountRuntimeSnapshot {
|
||||
protection_rules: std::collections::BTreeMap::new(),
|
||||
in_flight: self.counts.clone(),
|
||||
..Default::default()
|
||||
})
|
||||
@@ -3691,10 +3656,6 @@ impl AccountRuntimeStore for CapacityRuntime {
|
||||
Ok(BTreeMap::new())
|
||||
}
|
||||
|
||||
async fn capacity_peaks(&self, _: &[String]) -> AdminStoreResult<BTreeMap<String, u32>> {
|
||||
Ok(BTreeMap::new())
|
||||
}
|
||||
|
||||
async fn finish_freeze(
|
||||
&self,
|
||||
_: &str,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
//! 容量熔断冻结恢复编排回归:到期探测、失败顺延与自适应并发下调
|
||||
//! 账号保护恢复编排:到期探测、快照参数与失败顺延
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::sync::{Arc, Mutex};
|
||||
@@ -24,7 +24,7 @@ use gateway_core::task::{
|
||||
use super::AdminHarness;
|
||||
use super::accounts::{FakeAccountStore, FakeProviderAdmin, account_record, events, revision};
|
||||
|
||||
fn runtime_settings(enabled: bool, probe_enabled: bool, adaptive: bool) -> RuntimeSettings {
|
||||
fn runtime_settings(enabled: bool, probe_enabled: bool) -> RuntimeSettings {
|
||||
RuntimeSettings {
|
||||
request_profiles: Default::default(),
|
||||
config_revision: revision(1),
|
||||
@@ -52,13 +52,22 @@ fn runtime_settings(enabled: bool, probe_enabled: bool, adaptive: bool) -> Runti
|
||||
usage_retention_days: 31,
|
||||
ops_event_retention_days: 30,
|
||||
audit_retention_days: 90,
|
||||
account_auto_freeze_enabled: enabled,
|
||||
account_auto_freeze_threshold: 12,
|
||||
account_auto_freeze_window_seconds: 600,
|
||||
account_auto_freeze_duration_seconds: 7_200,
|
||||
account_auto_freeze_probe_enabled: probe_enabled,
|
||||
account_auto_freeze_probe_model: Some("gpt-5.5".to_owned()),
|
||||
account_auto_freeze_adaptive_concurrency: adaptive,
|
||||
account_error_protection: gateway_core::account::AccountErrorProtection {
|
||||
enabled,
|
||||
probe_enabled,
|
||||
probe_model: Some("gpt-5.5".to_owned()),
|
||||
rules: vec![gateway_core::account::AccountErrorRule {
|
||||
id: "sample".into(),
|
||||
name: "示例".into(),
|
||||
conditions: vec![gateway_core::account::AccountErrorCondition {
|
||||
field: gateway_core::account::AccountErrorField::Code,
|
||||
operator: gateway_core::account::AccountErrorOperator::Equals,
|
||||
values: vec![Some("custom_failure".into())],
|
||||
ignore_case: false,
|
||||
}],
|
||||
}],
|
||||
..Default::default()
|
||||
},
|
||||
account_warmup_enabled: false,
|
||||
account_warmup_schedule_time: "08:00".to_owned(),
|
||||
account_warmup_model: None,
|
||||
@@ -131,19 +140,17 @@ fn store_unavailable() -> AdminStoreError {
|
||||
)
|
||||
}
|
||||
|
||||
/// 记录解冻/顺延调用的运行态 fake;冻结与峰值证据由测试预置
|
||||
/// 记录解冻与顺延调用,冻结快照由测试预置
|
||||
struct FreezeRuntimeStore {
|
||||
freezes: BTreeMap<String, AccountFreeze>,
|
||||
peaks: BTreeMap<String, u32>,
|
||||
cleared: Mutex<Vec<String>>,
|
||||
extended: Mutex<Vec<(String, DateTime<Utc>)>>,
|
||||
}
|
||||
|
||||
impl FreezeRuntimeStore {
|
||||
fn new(freezes: BTreeMap<String, AccountFreeze>, peaks: BTreeMap<String, u32>) -> Arc<Self> {
|
||||
fn new(freezes: BTreeMap<String, AccountFreeze>) -> Arc<Self> {
|
||||
Arc::new(Self {
|
||||
freezes,
|
||||
peaks,
|
||||
cleared: Mutex::new(Vec::new()),
|
||||
extended: Mutex::new(Vec::new()),
|
||||
})
|
||||
@@ -188,16 +195,6 @@ impl AccountRuntimeStore for FreezeRuntimeStore {
|
||||
Ok(self.freezes.clone())
|
||||
}
|
||||
|
||||
async fn capacity_peaks(
|
||||
&self,
|
||||
account_ids: &[String],
|
||||
) -> AdminStoreResult<BTreeMap<String, u32>> {
|
||||
Ok(account_ids
|
||||
.iter()
|
||||
.filter_map(|id| self.peaks.get(id).map(|peak| (id.clone(), *peak)))
|
||||
.collect())
|
||||
}
|
||||
|
||||
async fn finish_freeze(
|
||||
&self,
|
||||
account_id: &str,
|
||||
@@ -241,9 +238,10 @@ struct FailingProbe;
|
||||
impl AccountProbe for FailingProbe {
|
||||
fn probe(
|
||||
&self,
|
||||
_: AccountProbeRequest,
|
||||
request: AccountProbeRequest,
|
||||
_: Option<Arc<gateway_core::routing::RuntimeSnapshot>>,
|
||||
) -> futures::future::BoxFuture<'_, Result<AccountProbeResult, AccountProbeError>> {
|
||||
assert_eq!(request.upstream_model.as_str(), "snapshot-model");
|
||||
Box::pin(async {
|
||||
Err(AccountProbeError::new(
|
||||
gateway_core::error::GatewayError::new(
|
||||
@@ -317,6 +315,9 @@ fn freeze_until(until: DateTime<Utc>) -> BTreeMap<String, AccountFreeze> {
|
||||
BTreeMap::from([(
|
||||
"acct_test".to_owned(),
|
||||
AccountFreeze {
|
||||
duration_seconds: None,
|
||||
probe_model: None,
|
||||
protection_rules: Vec::new(),
|
||||
credential_revision: revision(1),
|
||||
until,
|
||||
generation: "freeze-generation".to_owned(),
|
||||
@@ -331,9 +332,9 @@ fn freeze_due() -> BTreeMap<String, AccountFreeze> {
|
||||
|
||||
#[tokio::test]
|
||||
async fn probe_success_clears_freeze() {
|
||||
let runtime = FreezeRuntimeStore::new(freeze_due(), BTreeMap::new());
|
||||
let runtime = FreezeRuntimeStore::new(freeze_due());
|
||||
let (task, _store) = recovery_task(
|
||||
runtime_settings(true, true, false),
|
||||
runtime_settings(true, true),
|
||||
Arc::clone(&runtime),
|
||||
Arc::new(SuccessfulProbe),
|
||||
)
|
||||
@@ -346,10 +347,13 @@ async fn probe_success_clears_freeze() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn probe_failure_postpones_freeze() {
|
||||
let runtime = FreezeRuntimeStore::new(freeze_due(), BTreeMap::new());
|
||||
async fn probe_failure_postpones_using_the_freeze_snapshot() {
|
||||
let mut freezes = freeze_due();
|
||||
freezes.get_mut("acct_test").unwrap().duration_seconds = Some(300);
|
||||
freezes.get_mut("acct_test").unwrap().probe_model = Some("snapshot-model".into());
|
||||
let runtime = FreezeRuntimeStore::new(freezes);
|
||||
let (task, _store) = recovery_task(
|
||||
runtime_settings(true, true, false),
|
||||
runtime_settings(true, true),
|
||||
Arc::clone(&runtime),
|
||||
Arc::new(FailingProbe),
|
||||
)
|
||||
@@ -364,16 +368,16 @@ async fn probe_failure_postpones_freeze() {
|
||||
assert_eq!(account_id, "acct_test");
|
||||
let remaining = until.timestamp_millis() - Utc::now().timestamp_millis();
|
||||
assert!(
|
||||
remaining > 7_000_000 && remaining <= 7_200_000,
|
||||
"postponed freeze should be about 2 hours, got {remaining}ms"
|
||||
remaining > 290_000 && remaining <= 300_000,
|
||||
"postponed freeze should use the frozen 5-minute duration, got {remaining}ms"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn disabled_policy_releases_due_freeze_without_probing() {
|
||||
let runtime = FreezeRuntimeStore::new(freeze_due(), BTreeMap::new());
|
||||
let runtime = FreezeRuntimeStore::new(freeze_due());
|
||||
let (task, _store) = recovery_task(
|
||||
runtime_settings(false, true, false),
|
||||
runtime_settings(false, true),
|
||||
Arc::clone(&runtime),
|
||||
Arc::new(SuccessfulProbe),
|
||||
)
|
||||
@@ -385,59 +389,12 @@ async fn disabled_policy_releases_due_freeze_without_probing() {
|
||||
assert!(runtime.extended().is_empty());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn adaptive_concurrency_lowers_limit_to_observed_peak() {
|
||||
let runtime = FreezeRuntimeStore::new(
|
||||
freeze_until(Utc::now() + TimeDelta::hours(1)),
|
||||
BTreeMap::from([("acct_test".to_owned(), 4_u32)]),
|
||||
);
|
||||
let (task, store) = recovery_task(
|
||||
runtime_settings(true, false, true),
|
||||
Arc::clone(&runtime),
|
||||
Arc::new(SuccessfulProbe),
|
||||
)
|
||||
.await;
|
||||
|
||||
run_cycle(task.as_ref()).await;
|
||||
|
||||
// 峰值 4 × 0.8 = 3(不低于下限 2),低于全局默认 5,应下调到 3
|
||||
assert_eq!(
|
||||
*store.lowered_limits.lock().expect("lowered limits"),
|
||||
vec![("acct_test".to_owned(), 3)]
|
||||
);
|
||||
assert!(store.update_commands().is_empty());
|
||||
assert!(runtime.extended().is_empty());
|
||||
assert!(runtime.cleared().is_empty());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn adaptive_concurrency_never_raises_limit() {
|
||||
let runtime = FreezeRuntimeStore::new(
|
||||
freeze_until(Utc::now() + TimeDelta::hours(1)),
|
||||
BTreeMap::from([("acct_test".to_owned(), 40_u32)]),
|
||||
);
|
||||
let (task, store) = recovery_task(
|
||||
runtime_settings(true, false, true),
|
||||
Arc::clone(&runtime),
|
||||
Arc::new(SuccessfulProbe),
|
||||
)
|
||||
.await;
|
||||
|
||||
run_cycle(task.as_ref()).await;
|
||||
|
||||
// 峰值 40 的目标 32 高于全局默认 5;只降不升,不下发任何更新
|
||||
assert!(store.update_commands().is_empty());
|
||||
}
|
||||
|
||||
/// 到期时间还早(超过探测提前量)的冻结不发起探测
|
||||
#[tokio::test]
|
||||
async fn probe_skips_freezes_far_from_expiry() {
|
||||
let runtime = FreezeRuntimeStore::new(
|
||||
freeze_until(Utc::now() + TimeDelta::hours(1)),
|
||||
BTreeMap::new(),
|
||||
);
|
||||
let runtime = FreezeRuntimeStore::new(freeze_until(Utc::now() + TimeDelta::hours(1)));
|
||||
let (task, _store) = recovery_task(
|
||||
runtime_settings(true, true, false),
|
||||
runtime_settings(true, true),
|
||||
Arc::clone(&runtime),
|
||||
Arc::new(SuccessfulProbe),
|
||||
)
|
||||
@@ -449,13 +406,10 @@ async fn probe_skips_freezes_far_from_expiry() {
|
||||
assert!(runtime.extended().is_empty());
|
||||
}
|
||||
|
||||
/// 停用账号不参与探测恢复;并发下调同样跳过
|
||||
/// 停用账号不参与探测恢复
|
||||
#[tokio::test]
|
||||
async fn disabled_accounts_are_skipped() {
|
||||
let runtime = FreezeRuntimeStore::new(
|
||||
freeze_due(),
|
||||
BTreeMap::from([("acct_test".to_owned(), 4_u32)]),
|
||||
);
|
||||
let runtime = FreezeRuntimeStore::new(freeze_due());
|
||||
let mut record = account_record("openai");
|
||||
record.enabled = false;
|
||||
let store = FakeAccountStore::with_account(record, events());
|
||||
@@ -463,7 +417,7 @@ async fn disabled_accounts_are_skipped() {
|
||||
.accounts(Arc::clone(&store) as Arc<dyn AccountStore>)
|
||||
.account_runtime(Arc::clone(&runtime) as Arc<dyn AccountRuntimeStore>)
|
||||
.settings(Arc::new(FreezeSettingsStore {
|
||||
settings: runtime_settings(true, true, true),
|
||||
settings: runtime_settings(true, true),
|
||||
}))
|
||||
.provider(FakeProviderAdmin::new("openai", events()))
|
||||
.probe(Arc::new(SuccessfulProbe))
|
||||
|
||||
@@ -614,15 +614,6 @@ impl AccountStore for UnavailableStore {
|
||||
Err(unavailable("account enabled"))
|
||||
}
|
||||
|
||||
async fn lower_concurrency_limit(
|
||||
&self,
|
||||
_: &gateway_core::account::ProviderAccountId,
|
||||
_: gateway_core::account::AccountConcurrencyLimit,
|
||||
_: &MutationContext,
|
||||
) -> AdminStoreResult<Option<gateway_admin::model::accounts::AccountUpdateResult>> {
|
||||
Ok(None)
|
||||
}
|
||||
|
||||
async fn recover_account(
|
||||
&self,
|
||||
_: &ProviderAccountId,
|
||||
@@ -685,13 +676,6 @@ impl AccountRuntimeStore for UnavailableStore {
|
||||
Ok(BTreeMap::new())
|
||||
}
|
||||
|
||||
async fn capacity_peaks(
|
||||
&self,
|
||||
_account_ids: &[String],
|
||||
) -> AdminStoreResult<BTreeMap<String, u32>> {
|
||||
Ok(BTreeMap::new())
|
||||
}
|
||||
|
||||
async fn finish_freeze(
|
||||
&self,
|
||||
_account_id: &str,
|
||||
@@ -821,10 +805,27 @@ impl ObservabilityStore for UnavailableStore {
|
||||
_: TimeRange,
|
||||
_: UsageFilter,
|
||||
_: Granularity,
|
||||
_include_concurrency: bool,
|
||||
) -> AdminStoreResult<gateway_admin::model::observability::UsageInsightsObservation> {
|
||||
Err(unavailable("usage insights"))
|
||||
}
|
||||
|
||||
async fn usage_metric_observation(
|
||||
&self,
|
||||
range: gateway_admin::model::observability::TimeRange,
|
||||
filter: gateway_admin::model::observability::UsageFilter,
|
||||
granularity: gateway_admin::model::observability::Granularity,
|
||||
metric: gateway_admin::model::observability::UsageInsightMetric,
|
||||
) -> AdminStoreResult<gateway_admin::model::observability::UsageInsightsObservation> {
|
||||
self.usage_insights_observation(
|
||||
range,
|
||||
filter,
|
||||
granularity,
|
||||
metric == gateway_admin::model::observability::UsageInsightMetric::Load,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
async fn list_usage_records(&self, _: UsageQuery) -> AdminStoreResult<UsagePage> {
|
||||
Err(unavailable("usage records"))
|
||||
}
|
||||
|
||||
@@ -22,8 +22,9 @@ use gateway_admin::{
|
||||
DiagnosticDimension, DiagnosticObservation, DiagnosticsObservation, Granularity,
|
||||
HealthStatus, LatencyPercentiles, ObservabilityPageSize, OpsErrorPage, OpsErrorQuery,
|
||||
PercentileMilliseconds, RequestMetricPoint, RequestMetrics, TimeRange, TrendKind,
|
||||
UsageBilling, UsageDetail, UsageFilter, UsageInsightsObservation, UsageListRecord,
|
||||
UsageOverview, UsagePage, UsageProjectedCostPoint, UsageQuery, UsageSummaryObservation,
|
||||
UsageBilling, UsageConcurrencyPoint, UsageDetail, UsageFilter,
|
||||
UsageInsightsObservation, UsageListRecord, UsageOverview, UsagePage,
|
||||
UsageProjectedCostPoint, UsageQuery, UsageSummaryObservation,
|
||||
},
|
||||
settings::{
|
||||
AdminApiKey, AdminApiKeyMutation, ReplaceRuntimeSettings, RotationStrategy,
|
||||
@@ -678,10 +679,26 @@ async fn observability_services_should_calculate_usage_insights_and_diagnostic_s
|
||||
assert_eq!(insights.health.success_rate, 0.75);
|
||||
assert_eq!(insights.health.completion_rate, 1.0);
|
||||
assert_eq!(insights.granularity, Granularity::FifteenMinutes);
|
||||
assert_eq!(insights.performance.latency_coverage, 0.5);
|
||||
assert_eq!(insights.performance.admission_decision_coverage, 1.0);
|
||||
assert_eq!(insights.performance.account_selection_wait_coverage, 0.8);
|
||||
assert_eq!(insights.performance.capacity_coverage, 0.8);
|
||||
assert_eq!(
|
||||
insights
|
||||
.performance
|
||||
.load
|
||||
.admission_decision_p95_ms
|
||||
.map(|value| value.as_f64()),
|
||||
Some(5.0)
|
||||
);
|
||||
assert_eq!(
|
||||
insights
|
||||
.performance
|
||||
.load
|
||||
.account_selection_wait_p95_ms
|
||||
.map(|value| value.as_f64()),
|
||||
Some(40.0)
|
||||
);
|
||||
assert_eq!(
|
||||
insights.performance.points[0].load,
|
||||
insights.performance.load
|
||||
);
|
||||
assert_eq!(insights.performance.output_throughput_p50, Some(50));
|
||||
assert_eq!(insights.cost.tokens_per_request, 100.0);
|
||||
assert_eq!(
|
||||
@@ -742,6 +759,52 @@ async fn observability_services_should_calculate_usage_insights_and_diagnostic_s
|
||||
assert_eq!(diagnostics.items[1].request_share, 0.25);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn usage_insights_should_weight_concurrency_by_effective_bucket_duration() {
|
||||
let bucket_start = DateTime::parse_from_rfc3339("2026-09-01T10:00:00Z")
|
||||
.unwrap()
|
||||
.with_timezone(&Utc);
|
||||
let range = TimeRange::new(
|
||||
bucket_start + Duration::minutes(5),
|
||||
bucket_start + Duration::minutes(20),
|
||||
)
|
||||
.unwrap();
|
||||
let store = Arc::new(FixtureObservabilityStore::new(range));
|
||||
store.replace_trend(vec![
|
||||
health_metric_point(bucket_start, RequestMetrics::default()),
|
||||
health_metric_point(
|
||||
bucket_start + Duration::minutes(15),
|
||||
RequestMetrics::default(),
|
||||
),
|
||||
]);
|
||||
*store.concurrency.lock().unwrap() = vec![
|
||||
UsageConcurrencyPoint {
|
||||
bucket_start,
|
||||
average_concurrency: 1.0,
|
||||
peak_concurrency: 1,
|
||||
},
|
||||
UsageConcurrencyPoint {
|
||||
bucket_start: bucket_start + Duration::minutes(15),
|
||||
average_concurrency: 3.0,
|
||||
peak_concurrency: 4,
|
||||
},
|
||||
];
|
||||
|
||||
let insights = observability_services(store)
|
||||
.await
|
||||
.observability()
|
||||
.usage_insights(range, UsageFilter::default())
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!((insights.performance.load.average_concurrency - 5.0 / 3.0).abs() < 1e-10);
|
||||
assert_eq!(insights.performance.load.peak_concurrency, 4);
|
||||
assert_eq!(insights.performance.points[0].load.average_concurrency, 1.0);
|
||||
assert_eq!(insights.performance.points[0].load.peak_concurrency, 1);
|
||||
assert_eq!(insights.performance.points[1].load.average_concurrency, 3.0);
|
||||
assert_eq!(insights.performance.points[1].load.peak_concurrency, 4);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn diagnostics_should_preserve_request_order_when_a_small_sample_fails() {
|
||||
let now = Utc::now();
|
||||
@@ -937,6 +1000,7 @@ async fn dashboard_should_preserve_provider_plan_names() {
|
||||
authentication_kind: "oauth".to_owned(),
|
||||
name: (*plan).to_owned(),
|
||||
email: None,
|
||||
notes: None,
|
||||
plan_type: Some((*plan).to_owned()),
|
||||
plan_type_display: None,
|
||||
request_count: 0,
|
||||
@@ -982,12 +1046,59 @@ async fn dashboard_should_preserve_provider_plan_names() {
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn dashboard_and_usage_insights_keep_concurrency_cache_entries_separate() {
|
||||
for dashboard_first in [true, false] {
|
||||
let now = Utc::now();
|
||||
let range = observation_range(now);
|
||||
let store = Arc::new(FixtureObservabilityStore::new(range));
|
||||
store
|
||||
.concurrency
|
||||
.lock()
|
||||
.unwrap()
|
||||
.push(UsageConcurrencyPoint {
|
||||
bucket_start: range.start,
|
||||
average_concurrency: 2.0,
|
||||
peak_concurrency: 3,
|
||||
});
|
||||
let services = observability_services(store.clone()).await;
|
||||
for dashboard in [
|
||||
dashboard_first,
|
||||
!dashboard_first,
|
||||
dashboard_first,
|
||||
!dashboard_first,
|
||||
] {
|
||||
if dashboard {
|
||||
services
|
||||
.observability()
|
||||
.dashboard_trend(range, TrendKind::Usage)
|
||||
.await
|
||||
.unwrap();
|
||||
} else {
|
||||
let result = services
|
||||
.observability()
|
||||
.usage_insights(range, UsageFilter::default())
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(result.performance.load.peak_concurrency, 3);
|
||||
}
|
||||
}
|
||||
assert_eq!(
|
||||
*store.concurrency_selections.lock().unwrap(),
|
||||
vec![!dashboard_first, dashboard_first]
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
struct FixtureObservabilityStore {
|
||||
yesterday: Mutex<RequestMetrics>,
|
||||
trend: Mutex<Vec<RequestMetricPoint>>,
|
||||
overview: Mutex<UsageOverview>,
|
||||
projected_costs: Mutex<Vec<UsageProjectedCostPoint>>,
|
||||
concurrency: Mutex<Vec<UsageConcurrencyPoint>>,
|
||||
insights_fail: AtomicBool,
|
||||
concurrency_selections: Mutex<Vec<bool>>,
|
||||
metric_selections: Mutex<Vec<gateway_admin::model::observability::UsageInsightMetric>>,
|
||||
summary_fail: AtomicBool,
|
||||
diagnostics: Mutex<DiagnosticsObservation>,
|
||||
runtime_slots: Mutex<Option<DashboardRuntimeSlots>>,
|
||||
@@ -1014,7 +1125,10 @@ impl FixtureObservabilityStore {
|
||||
providers: Vec::new(),
|
||||
}),
|
||||
projected_costs: Mutex::new(Vec::new()),
|
||||
concurrency: Mutex::new(Vec::new()),
|
||||
insights_fail: AtomicBool::new(false),
|
||||
concurrency_selections: Mutex::new(Vec::new()),
|
||||
metric_selections: Mutex::new(Vec::new()),
|
||||
summary_fail: AtomicBool::new(false),
|
||||
diagnostics: Mutex::new(DiagnosticsObservation::default()),
|
||||
runtime_slots: Mutex::new(None),
|
||||
@@ -1131,8 +1245,13 @@ impl ObservabilityStore for FixtureObservabilityStore {
|
||||
_: TimeRange,
|
||||
_: UsageFilter,
|
||||
granularity: Granularity,
|
||||
include_concurrency: bool,
|
||||
) -> AdminStoreResult<UsageInsightsObservation> {
|
||||
self.usage_granularities.lock().unwrap().push(granularity);
|
||||
self.concurrency_selections
|
||||
.lock()
|
||||
.unwrap()
|
||||
.push(include_concurrency);
|
||||
if self.insights_fail.load(Ordering::SeqCst) {
|
||||
return Err(super::unavailable("projection read failed"));
|
||||
}
|
||||
@@ -1144,9 +1263,31 @@ impl ObservabilityStore for FixtureObservabilityStore {
|
||||
.lock()
|
||||
.expect("projected costs")
|
||||
.clone(),
|
||||
concurrency: if include_concurrency {
|
||||
self.concurrency.lock().expect("concurrency").clone()
|
||||
} else {
|
||||
Vec::new()
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
async fn usage_metric_observation(
|
||||
&self,
|
||||
range: gateway_admin::model::observability::TimeRange,
|
||||
filter: gateway_admin::model::observability::UsageFilter,
|
||||
granularity: gateway_admin::model::observability::Granularity,
|
||||
metric: gateway_admin::model::observability::UsageInsightMetric,
|
||||
) -> AdminStoreResult<gateway_admin::model::observability::UsageInsightsObservation> {
|
||||
self.metric_selections.lock().unwrap().push(metric);
|
||||
self.usage_insights_observation(
|
||||
range,
|
||||
filter,
|
||||
granularity,
|
||||
metric == gateway_admin::model::observability::UsageInsightMetric::Load,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
async fn list_usage_records(&self, query: UsageQuery) -> AdminStoreResult<UsagePage> {
|
||||
let items = self.usage_records.lock().expect("usage records").clone();
|
||||
let total = u64::try_from(items.len()).unwrap_or(u64::MAX);
|
||||
@@ -1253,13 +1394,7 @@ impl SettingsStore for FixtureSettingsStore {
|
||||
usage_retention_days: 31,
|
||||
ops_event_retention_days: 30,
|
||||
audit_retention_days: 30,
|
||||
account_auto_freeze_enabled: true,
|
||||
account_auto_freeze_threshold: 12,
|
||||
account_auto_freeze_window_seconds: 600,
|
||||
account_auto_freeze_duration_seconds: 7_200,
|
||||
account_auto_freeze_probe_enabled: true,
|
||||
account_auto_freeze_probe_model: None,
|
||||
account_auto_freeze_adaptive_concurrency: true,
|
||||
account_error_protection: gateway_core::account::AccountErrorProtection::default(),
|
||||
account_warmup_enabled: false,
|
||||
account_warmup_schedule_time: "08:00".to_owned(),
|
||||
account_warmup_model: None,
|
||||
@@ -1680,3 +1815,48 @@ async fn insights_cache_should_reject_deleted_building_and_old_generation_times(
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn usage_metric_cache_isolates_selections_and_filters() {
|
||||
use gateway_admin::model::observability::UsageInsightMetric as Metric;
|
||||
let range = observation_range(Utc::now());
|
||||
let store = Arc::new(FixtureObservabilityStore::new(range));
|
||||
let services = observability_services(store.clone()).await;
|
||||
let metrics = [
|
||||
Metric::Health,
|
||||
Metric::Latency,
|
||||
Metric::FirstToken,
|
||||
Metric::Throughput,
|
||||
Metric::Load,
|
||||
Metric::Cost,
|
||||
Metric::Savings,
|
||||
Metric::Cache,
|
||||
];
|
||||
for _ in 0..2 {
|
||||
for metric in metrics {
|
||||
services
|
||||
.observability()
|
||||
.usage_metric(range, UsageFilter::default(), metric)
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
assert_eq!(*store.metric_selections.lock().unwrap(), metrics);
|
||||
assert_eq!(
|
||||
*store.concurrency_selections.lock().unwrap(),
|
||||
vec![false, false, false, false, true, false, false, false]
|
||||
);
|
||||
services
|
||||
.observability()
|
||||
.usage_metric(
|
||||
range,
|
||||
UsageFilter {
|
||||
provider_kind: Some("openai".into()),
|
||||
..Default::default()
|
||||
},
|
||||
Metric::Latency,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(store.metric_selections.lock().unwrap().len(), 9);
|
||||
}
|
||||
|
||||
@@ -213,6 +213,16 @@ fn sdk_settings_contract_matches_host_declarations() {
|
||||
let mut types = Vec::new();
|
||||
let mut replacement_fields = Vec::new();
|
||||
for (path, names) in [
|
||||
(
|
||||
"../gateway-core/src/account/error_protection.rs",
|
||||
&[
|
||||
"AccountErrorProtection",
|
||||
"AccountErrorRule",
|
||||
"AccountErrorCondition",
|
||||
"AccountErrorField",
|
||||
"AccountErrorOperator",
|
||||
][..],
|
||||
),
|
||||
(
|
||||
"../gateway-core/src/settings/privacy.rs",
|
||||
&[
|
||||
|
||||
@@ -345,13 +345,8 @@ async fn settings_should_reject_zero_refresh_margin_before_store_call() {
|
||||
usage_retention_days: 31,
|
||||
ops_event_retention_days: 30,
|
||||
audit_retention_days: 30,
|
||||
account_auto_freeze_enabled: true,
|
||||
account_auto_freeze_threshold: 12,
|
||||
account_auto_freeze_window_seconds: 600,
|
||||
account_auto_freeze_duration_seconds: 7_200,
|
||||
account_auto_freeze_probe_enabled: true,
|
||||
account_auto_freeze_probe_model: None,
|
||||
account_auto_freeze_adaptive_concurrency: true,
|
||||
account_error_protection:
|
||||
gateway_core::account::AccountErrorProtection::default(),
|
||||
account_warmup_enabled: false,
|
||||
account_warmup_schedule_time: "08:00".to_owned(),
|
||||
account_warmup_model: None,
|
||||
|
||||
@@ -16,11 +16,15 @@ use gateway_admin::{
|
||||
struct RecordingSystemOperations {
|
||||
target: Mutex<Option<Option<String>>>,
|
||||
channel: Mutex<Option<gateway_admin::model::system::SystemUpdateChannel>>,
|
||||
failure: Option<SystemOperationError>,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl SystemOperations for RecordingSystemOperations {
|
||||
async fn version(&self) -> Result<SystemVersion, SystemOperationError> {
|
||||
if let Some(error) = &self.failure {
|
||||
return Err(error.clone());
|
||||
}
|
||||
Ok(SystemVersion {
|
||||
version: "1.0.0".to_owned(),
|
||||
git_sha: "unknown".to_owned(),
|
||||
@@ -119,6 +123,33 @@ impl SystemOperations for RecordingSystemOperations {
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn system_failure_should_preserve_safe_message_source_and_redaction_marker() {
|
||||
let operations = std::sync::Arc::new(RecordingSystemOperations {
|
||||
failure: Some(
|
||||
SystemOperationError::new(
|
||||
gateway_admin::ports::system::SystemOperationErrorKind::Upstream,
|
||||
"更新包下载失败",
|
||||
)
|
||||
.with_source(std::io::Error::other("native transport diagnostic"))
|
||||
.redact_sensitive_context(),
|
||||
),
|
||||
..RecordingSystemOperations::default()
|
||||
});
|
||||
let services = super::AdminHarness::new().system(operations).build().await;
|
||||
let error = services.system().version().await.expect_err("host failure");
|
||||
assert_eq!(
|
||||
error.kind(),
|
||||
gateway_admin::model::AdminErrorKind::BadGateway
|
||||
);
|
||||
assert_eq!(error.message(), "更新包下载失败");
|
||||
assert!(!format!("{error} {error:?}").contains("native transport diagnostic"));
|
||||
let details = error.error_details().expect("host error details");
|
||||
assert!(details.as_str().contains("native transport diagnostic"));
|
||||
let details: serde_json::Value = serde_json::from_str(details.as_str()).expect("details JSON");
|
||||
assert_eq!(details["redacted"], true);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn system_update_should_normalize_blank_target_to_latest() {
|
||||
let operations = std::sync::Arc::new(RecordingSystemOperations::default());
|
||||
|
||||
@@ -25,13 +25,16 @@ gateway-admin = { path = "../gateway-admin" }
|
||||
gateway-core = { path = "../gateway-core" }
|
||||
gateway-protocol = { path = "../gateway-protocol" }
|
||||
headers = "0.4.1"
|
||||
hyper = { version = "=1.11.1", default-features = false }
|
||||
hyper-util = { version = "=0.1.21", default-features = false, features = ["tokio"] }
|
||||
regex = "1.13.1"
|
||||
secrecy = "0.10"
|
||||
serde = { version = "1.0.228", features = ["derive"] }
|
||||
serde_json = { version = "1.0.150", features = ["arbitrary_precision", "preserve_order"] }
|
||||
sha2 = "0.11.0"
|
||||
thiserror = "2.0.18"
|
||||
tokio = { version = "1.52.3", default-features = false, features = ["macros", "rt", "sync", "time"] }
|
||||
tokio = { version = "1.52.3", default-features = false, features = ["io-util", "macros", "rt", "sync", "time"] }
|
||||
tokio-tungstenite = { version = "0.29.0", default-features = false, features = ["handshake"] }
|
||||
tower = { version = "0.5.3", features = ["util"] }
|
||||
tower-http = { version = "0.7.0", features = ["compression-gzip", "cors", "fs", "request-id", "timeout", "trace"] }
|
||||
tracing = "0.1.44"
|
||||
|
||||
@@ -569,7 +569,8 @@ where
|
||||
.models(&account_id, false)
|
||||
.await
|
||||
.map_err(map_service_error)?;
|
||||
let data = account_models_data(result);
|
||||
let time = crate::time::TimePresenter::new(state.admin_services().timezone());
|
||||
let data = account_models_data(result, time);
|
||||
Ok(AdminResponse::new(StatusCode::OK, AdminEnvelope::ok(data)))
|
||||
}
|
||||
|
||||
@@ -607,7 +608,8 @@ where
|
||||
.models(&account_id, true)
|
||||
.await
|
||||
.map_err(map_service_error)?;
|
||||
let data = account_models_data(result);
|
||||
let time = crate::time::TimePresenter::new(state.admin_services().timezone());
|
||||
let data = account_models_data(result, time);
|
||||
Ok(AdminResponse::new(StatusCode::OK, AdminEnvelope::ok(data)))
|
||||
}
|
||||
|
||||
|
||||
@@ -43,7 +43,11 @@ pub(super) fn account_refresh_data(
|
||||
}
|
||||
}
|
||||
|
||||
pub(super) fn account_models_data(result: ProviderModels) -> AccountModelsData {
|
||||
pub(super) fn account_models_data(
|
||||
result: ProviderModels,
|
||||
time: crate::time::TimePresenter,
|
||||
) -> AccountModelsData {
|
||||
let observed_at = result.observed_at;
|
||||
AccountModelsData {
|
||||
models: result
|
||||
.models
|
||||
@@ -56,6 +60,9 @@ pub(super) fn account_models_data(result: ProviderModels) -> AccountModelsData {
|
||||
}
|
||||
})
|
||||
.collect(),
|
||||
observed_at: observed_at.map(|value| time.rfc3339(&value)),
|
||||
observed_at_display: observed_at
|
||||
.map_or_else(|| "—".to_owned(), |value| time.datetime(&value)),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -72,6 +79,7 @@ pub(super) fn account_view(
|
||||
projection,
|
||||
usage,
|
||||
quota,
|
||||
protection_rules,
|
||||
} = item;
|
||||
let status = projection.status.as_str().to_owned();
|
||||
let cooldown = projection.cooldown;
|
||||
@@ -86,7 +94,8 @@ pub(super) fn account_view(
|
||||
if account.authentication_kind == "api_key" {
|
||||
usage.window_kind = AccountUsageWindowKind::General;
|
||||
}
|
||||
let (quota, refresh_token_expires_at) = account_quota_view(quota, cooldown, now, time);
|
||||
let (mut quota, refresh_token_expires_at) = account_quota_view(quota, cooldown, now, time);
|
||||
quota.protection_rules = protection_rules;
|
||||
AccountView {
|
||||
capabilities: capabilities.into(),
|
||||
id: account.id.clone(),
|
||||
@@ -232,8 +241,8 @@ pub(super) fn account_quota_view(
|
||||
}
|
||||
});
|
||||
let rate_limit_reason = cooldown.map(|value| {
|
||||
if value.kind.is_capacity_freeze() {
|
||||
"capacity_freeze".to_owned()
|
||||
if value.kind.is_error_protection() {
|
||||
"error_protection".to_owned()
|
||||
} else {
|
||||
"upstream_rate_limit".to_owned()
|
||||
}
|
||||
@@ -251,6 +260,7 @@ pub(super) fn account_quota_view(
|
||||
rate_limit_recovery_display,
|
||||
rate_limit_reason,
|
||||
recovery_probe_required,
|
||||
protection_rules: Vec::new(),
|
||||
windows,
|
||||
credits: quota.credits.map(|credits| AccountQuotaCreditsView {
|
||||
has_credits: credits.has_credits,
|
||||
|
||||
@@ -394,6 +394,7 @@ pub struct AccountQuotaView {
|
||||
pub rate_limit_recovery_display: Option<String>,
|
||||
pub rate_limit_reason: Option<String>,
|
||||
pub recovery_probe_required: bool,
|
||||
pub protection_rules: Vec<gateway_core::account::AccountProtectionMatch>,
|
||||
pub windows: Vec<AccountQuotaWindowView>,
|
||||
pub credits: Option<AccountQuotaCreditsView>,
|
||||
}
|
||||
@@ -721,8 +722,11 @@ pub struct AccountModelView {
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct AccountModelsData {
|
||||
pub models: Vec<AccountModelView>,
|
||||
pub observed_at: Option<String>,
|
||||
pub observed_at_display: String,
|
||||
}
|
||||
|
||||
/// 账号模型目录文件;`catalog` 直接作为 Codex `model_catalog_json` 的文件正文落盘
|
||||
|
||||
@@ -1066,13 +1066,28 @@ fn mutation_response(
|
||||
}
|
||||
|
||||
fn map_wire_error(error: WireValidationError) -> AdminError {
|
||||
match error.field() {
|
||||
"cursor" => AdminError::bad_request("Client API Key 游标不合法"),
|
||||
let message = match error.field() {
|
||||
"cursor" => "API Key 分页游标不合法",
|
||||
"clientKeyRevealNotFound" | "clientKeyMutationNotFound" => {
|
||||
AdminError::not_found("Client API Key 不存在")
|
||||
return AdminError::not_found("API Key 不存在");
|
||||
}
|
||||
_ => AdminError::bad_request("Client API Key 请求不合法"),
|
||||
}
|
||||
"id" => "API Key ID 不能为空",
|
||||
"name" => "API Key 名称不能为空",
|
||||
"label" => "API Key 标签不能为空",
|
||||
"customKey" => "自定义 API Key 仅支持英文字母、数字和英文符号",
|
||||
"groupIds" => "账号分组无效、重复或数量超限",
|
||||
"maxConcurrency" => "并发请求上限超出支持范围",
|
||||
"requestsPerMinute" => "每分钟请求上限超出支持范围",
|
||||
"dailyLimitUsd" => "每日美元限额须非负,整数和小数各最多 10 位",
|
||||
"weeklyLimitUsd" => "每周美元限额须非负,整数和小数各最多 10 位",
|
||||
"providerRequestProfileOverrides" => "API Key 客户端身份配置无效或字段冲突",
|
||||
"limit" => "每页数量须为 1 至 65535",
|
||||
"search" => "搜索内容过长或包含控制字符",
|
||||
"sortBy" => "排序字段不受支持",
|
||||
"sortDirection" => "排序方向必须为 asc(升序)或 desc(降序)",
|
||||
_ => "API Key 请求参数不合法",
|
||||
};
|
||||
AdminError::bad_request(message)
|
||||
}
|
||||
|
||||
fn map_service_error(error: gateway_admin::model::AdminError) -> AdminError {
|
||||
|
||||
@@ -41,10 +41,6 @@ pub(crate) fn display_rate(value: f64) -> String {
|
||||
}
|
||||
}
|
||||
|
||||
fn basis_points_rate(value: u64) -> f64 {
|
||||
value as f64 / 10_000.0
|
||||
}
|
||||
|
||||
pub(crate) fn outcome_name(outcome: &domain::RequestOutcome) -> &str {
|
||||
outcome.as_str()
|
||||
}
|
||||
@@ -930,6 +926,7 @@ pub(crate) fn dashboard_view(
|
||||
.email
|
||||
.clone()
|
||||
.unwrap_or_else(|| credential.name.clone()),
|
||||
notes: credential.notes,
|
||||
plan_type: credential.plan_type.clone(),
|
||||
plan_type_display: credential
|
||||
.plan_type_display
|
||||
@@ -1050,26 +1047,21 @@ pub(crate) fn usage_summary_view(
|
||||
}
|
||||
}
|
||||
|
||||
fn overview_load_view(load: &domain::UsageInsightsLoad) -> OverviewLoadView {
|
||||
OverviewLoadView {
|
||||
average_concurrency: load.average_concurrency,
|
||||
peak_concurrency: load.peak_concurrency,
|
||||
admission_decision_p95_ms: load.admission_decision_p95_ms.map(|value| value.as_f64()),
|
||||
account_selection_wait_p95_ms: load
|
||||
.account_selection_wait_p95_ms
|
||||
.map(|value| value.as_f64()),
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn usage_insights_view(
|
||||
insights: domain::UsageInsights,
|
||||
time: crate::time::TimePresenter,
|
||||
) -> UsageInsightsOverviewView {
|
||||
let health_points = insights
|
||||
.health
|
||||
.points
|
||||
.iter()
|
||||
.map(|point| OverviewHealthPointView {
|
||||
bucket: point.bucket_start,
|
||||
label: time.label(point.bucket_start, "%m-%d %H:%M"),
|
||||
total_requests: point.total_requests,
|
||||
success_requests: point.success_requests,
|
||||
failed_requests: point.failed_requests,
|
||||
cancelled_requests: point.cancelled_requests,
|
||||
incomplete_requests: point.incomplete_requests,
|
||||
caller_error_requests: point.caller_error_requests,
|
||||
error_rate: point.error_rate,
|
||||
})
|
||||
.collect();
|
||||
let performance_points = insights
|
||||
.performance
|
||||
.points
|
||||
@@ -1092,31 +1084,10 @@ pub(crate) fn usage_insights_view(
|
||||
.first_token_latency_percentiles
|
||||
.p99_ms
|
||||
.map(|value| value.as_f64()),
|
||||
admission_decision_p50_ms: point
|
||||
.admission_decision_percentiles
|
||||
.p50_ms
|
||||
.map(|value| value.as_f64()),
|
||||
admission_decision_p95_ms: point
|
||||
.admission_decision_percentiles
|
||||
.p95_ms
|
||||
.map(|value| value.as_f64()),
|
||||
account_selection_wait_p50_ms: point
|
||||
.account_selection_wait_percentiles
|
||||
.p50_ms
|
||||
.map(|value| value.as_f64()),
|
||||
account_selection_wait_p95_ms: point
|
||||
.account_selection_wait_percentiles
|
||||
.p95_ms
|
||||
.map(|value| value.as_f64()),
|
||||
output_throughput_p10: point.output_throughput_p10,
|
||||
output_throughput_p50: point.output_throughput_p50,
|
||||
output_throughput_p90: point.output_throughput_p90,
|
||||
capacity_utilization: point
|
||||
.capacity_utilization_avg_basis_points
|
||||
.map(basis_points_rate),
|
||||
capacity_utilization_p95: point
|
||||
.capacity_utilization_p95_basis_points
|
||||
.map(basis_points_rate),
|
||||
load: overview_load_view(&point.load),
|
||||
})
|
||||
.collect();
|
||||
let cost_points = insights
|
||||
@@ -1141,19 +1112,7 @@ pub(crate) fn usage_insights_view(
|
||||
UsageInsightsOverviewView {
|
||||
analytics: analytics_view(insights.analytics, time),
|
||||
granularity: granularity_name(insights.granularity).to_owned(),
|
||||
health: OverviewHealthView {
|
||||
total_requests: insights.health.total_requests,
|
||||
success_requests: insights.health.success_requests,
|
||||
failed_requests: insights.health.failed_requests,
|
||||
cancelled_requests: insights.health.cancelled_requests,
|
||||
incomplete_requests: insights.health.incomplete_requests,
|
||||
caller_error_requests: insights.health.caller_error_requests,
|
||||
success_rate: insights.health.success_rate,
|
||||
completion_rate: insights.health.completion_rate,
|
||||
request_change_rate: None,
|
||||
success_rate_change: None,
|
||||
points: health_points,
|
||||
},
|
||||
health: overview_health_view(&insights.health, time),
|
||||
performance: OverviewPerformanceView {
|
||||
latency_p50_ms: insights
|
||||
.performance
|
||||
@@ -1185,42 +1144,10 @@ pub(crate) fn usage_insights_view(
|
||||
.first_token_latency_percentiles
|
||||
.p99_ms
|
||||
.map(|value| value.as_f64()),
|
||||
admission_decision_p50_ms: insights
|
||||
.performance
|
||||
.admission_decision_percentiles
|
||||
.p50_ms
|
||||
.map(|value| value.as_f64()),
|
||||
admission_decision_p95_ms: insights
|
||||
.performance
|
||||
.admission_decision_percentiles
|
||||
.p95_ms
|
||||
.map(|value| value.as_f64()),
|
||||
account_selection_wait_p50_ms: insights
|
||||
.performance
|
||||
.account_selection_wait_percentiles
|
||||
.p50_ms
|
||||
.map(|value| value.as_f64()),
|
||||
account_selection_wait_p95_ms: insights
|
||||
.performance
|
||||
.account_selection_wait_percentiles
|
||||
.p95_ms
|
||||
.map(|value| value.as_f64()),
|
||||
output_throughput_p10: insights.performance.output_throughput_p10,
|
||||
output_throughput_p50: insights.performance.output_throughput_p50,
|
||||
output_throughput_p90: insights.performance.output_throughput_p90,
|
||||
capacity_utilization: insights
|
||||
.performance
|
||||
.capacity_utilization_avg_basis_points
|
||||
.map(basis_points_rate),
|
||||
capacity_utilization_p95: insights
|
||||
.performance
|
||||
.capacity_utilization_p95_basis_points
|
||||
.map(basis_points_rate),
|
||||
latency_coverage: insights.performance.latency_coverage,
|
||||
first_token_coverage: insights.performance.first_token_coverage,
|
||||
admission_decision_coverage: insights.performance.admission_decision_coverage,
|
||||
account_selection_wait_coverage: insights.performance.account_selection_wait_coverage,
|
||||
capacity_coverage: insights.performance.capacity_coverage,
|
||||
load: overview_load_view(&insights.performance.load),
|
||||
points: performance_points,
|
||||
},
|
||||
cost: OverviewCostView {
|
||||
@@ -1281,6 +1208,116 @@ pub(crate) fn usage_insights_view(
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn usage_metric_view(
|
||||
insight: domain::UsageInsight,
|
||||
time: crate::time::TimePresenter,
|
||||
) -> UsageMetricView {
|
||||
use domain::UsageInsightData;
|
||||
let data = match insight.data {
|
||||
UsageInsightData::Health(health) => {
|
||||
UsageMetricDataView::Health(overview_health_view(&health, time))
|
||||
}
|
||||
UsageInsightData::Latency(series) => UsageMetricDataView::Latency(metric_series_view(
|
||||
series,
|
||||
time,
|
||||
metric_latency_view,
|
||||
metric_latency_view,
|
||||
)),
|
||||
UsageInsightData::FirstToken(series) => UsageMetricDataView::FirstToken(
|
||||
metric_series_view(series, time, metric_latency_view, metric_latency_view),
|
||||
),
|
||||
UsageInsightData::Throughput(series) => UsageMetricDataView::Throughput(
|
||||
metric_series_view(series, time, metric_throughput_view, metric_throughput_view),
|
||||
),
|
||||
UsageInsightData::Load(series) => UsageMetricDataView::Load(metric_series_view(
|
||||
series,
|
||||
time,
|
||||
|value| overview_load_view(&value),
|
||||
|value| overview_load_view(&value),
|
||||
)),
|
||||
UsageInsightData::Cost(series) => UsageMetricDataView::Cost(metric_series_view(
|
||||
series,
|
||||
time,
|
||||
metric_cost_summary_view,
|
||||
|value| UsageCostComparisonView {
|
||||
estimated_cost: value.estimated_cost.as_ref().map(ToString::to_string),
|
||||
no_cache_cost: value.no_cache_cost.as_ref().map(ToString::to_string),
|
||||
},
|
||||
)),
|
||||
UsageInsightData::Savings(series) => UsageMetricDataView::Savings(metric_series_view(
|
||||
series,
|
||||
time,
|
||||
metric_cost_summary_view,
|
||||
|value| value.as_ref().map(ToString::to_string),
|
||||
)),
|
||||
UsageInsightData::Cache(series) => UsageMetricDataView::Cache(metric_series_view(
|
||||
series,
|
||||
time,
|
||||
metric_cost_summary_view,
|
||||
|value| UsageCacheView {
|
||||
input_tokens: value.input_tokens,
|
||||
cached_token_rate: value.cached_token_rate,
|
||||
cache_hit_request_rate: value.cache_hit_request_rate,
|
||||
},
|
||||
)),
|
||||
};
|
||||
UsageMetricView {
|
||||
analytics: analytics_view(insight.analytics, time),
|
||||
granularity: granularity_name(insight.granularity).to_owned(),
|
||||
data,
|
||||
}
|
||||
}
|
||||
|
||||
fn metric_series_view<S, P, SV, PV>(
|
||||
series: domain::UsageMetricSeries<S, P>,
|
||||
time: crate::time::TimePresenter,
|
||||
summary: impl FnOnce(S) -> SV,
|
||||
point: impl Fn(P) -> PV,
|
||||
) -> UsageMetricSeriesView<SV, PV> {
|
||||
UsageMetricSeriesView {
|
||||
summary: summary(series.summary),
|
||||
points: series
|
||||
.points
|
||||
.into_iter()
|
||||
.map(|value| UsageMetricPointView {
|
||||
bucket: value.bucket_start,
|
||||
label: time.label(value.bucket_start, "%m-%d %H:%M"),
|
||||
request_count: value.request_count,
|
||||
value: point(value.value),
|
||||
})
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
|
||||
fn metric_latency_view(value: domain::LatencyPercentiles) -> UsageLatencyView {
|
||||
UsageLatencyView {
|
||||
p50_ms: value.p50_ms.map(|value| value.as_f64()),
|
||||
p95_ms: value.p95_ms.map(|value| value.as_f64()),
|
||||
p99_ms: value.p99_ms.map(|value| value.as_f64()),
|
||||
}
|
||||
}
|
||||
|
||||
fn metric_throughput_view(value: domain::UsageThroughputPercentiles) -> UsageThroughputView {
|
||||
UsageThroughputView {
|
||||
p10: value.p10,
|
||||
p50: value.p50,
|
||||
p90: value.p90,
|
||||
}
|
||||
}
|
||||
|
||||
fn metric_cost_summary_view(value: domain::UsageCostSummary) -> UsageCostSummaryView {
|
||||
UsageCostSummaryView {
|
||||
estimated_cost: value.estimated_cost.as_ref().map(ToString::to_string),
|
||||
cache_savings: value.cache_savings.as_ref().map(ToString::to_string),
|
||||
tier_premium: value.tier_premium.as_ref().map(ToString::to_string),
|
||||
cost_per_successful_request: value
|
||||
.cost_per_successful_request
|
||||
.as_ref()
|
||||
.map(ToString::to_string),
|
||||
coverage: cost_coverage_view(&value.coverage),
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn granularity_name(granularity: domain::Granularity) -> &'static str {
|
||||
match granularity {
|
||||
domain::Granularity::FifteenMinutes => "15m",
|
||||
@@ -1367,3 +1404,37 @@ pub(crate) fn analytics_view(
|
||||
histogram_version: metadata.histogram_version,
|
||||
}
|
||||
}
|
||||
|
||||
fn overview_health_view(
|
||||
health: &domain::UsageInsightsHealth,
|
||||
time: crate::time::TimePresenter,
|
||||
) -> OverviewHealthView {
|
||||
let health_points = health
|
||||
.points
|
||||
.iter()
|
||||
.map(|point| OverviewHealthPointView {
|
||||
bucket: point.bucket_start,
|
||||
label: time.label(point.bucket_start, "%m-%d %H:%M"),
|
||||
total_requests: point.total_requests,
|
||||
success_requests: point.success_requests,
|
||||
failed_requests: point.failed_requests,
|
||||
cancelled_requests: point.cancelled_requests,
|
||||
incomplete_requests: point.incomplete_requests,
|
||||
caller_error_requests: point.caller_error_requests,
|
||||
error_rate: point.error_rate,
|
||||
})
|
||||
.collect();
|
||||
OverviewHealthView {
|
||||
total_requests: health.total_requests,
|
||||
success_requests: health.success_requests,
|
||||
failed_requests: health.failed_requests,
|
||||
cancelled_requests: health.cancelled_requests,
|
||||
incomplete_requests: health.incomplete_requests,
|
||||
caller_error_requests: health.caller_error_requests,
|
||||
success_rate: health.success_rate,
|
||||
completion_rate: health.completion_rate,
|
||||
request_change_rate: None,
|
||||
success_rate_change: None,
|
||||
points: health_points,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -53,6 +53,71 @@ pub struct UsageAnalyticsQuery {
|
||||
pub end_time: Option<String>,
|
||||
}
|
||||
|
||||
/// 卡片内的指标选择与时间、Provider 范围
|
||||
#[derive(Clone, Deserialize)]
|
||||
#[serde(rename_all = "camelCase", deny_unknown_fields)]
|
||||
pub struct UsageMetricQuery<T> {
|
||||
pub metric: T,
|
||||
pub provider: Option<String>,
|
||||
pub period: Option<String>,
|
||||
pub as_of: Option<i64>,
|
||||
pub start_time: Option<String>,
|
||||
pub end_time: Option<String>,
|
||||
}
|
||||
|
||||
impl<T> UsageMetricQuery<T> {
|
||||
pub fn into_parts(self) -> (UsageAnalyticsQuery, T) {
|
||||
(
|
||||
UsageAnalyticsQuery {
|
||||
provider: self.provider,
|
||||
period: self.period,
|
||||
as_of: self.as_of,
|
||||
start_time: self.start_time,
|
||||
end_time: self.end_time,
|
||||
},
|
||||
self.metric,
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub enum UsagePerformanceMetricKind {
|
||||
Latency,
|
||||
FirstToken,
|
||||
Throughput,
|
||||
Load,
|
||||
}
|
||||
|
||||
impl From<UsagePerformanceMetricKind> for domain::UsageInsightMetric {
|
||||
fn from(value: UsagePerformanceMetricKind) -> Self {
|
||||
match value {
|
||||
UsagePerformanceMetricKind::Latency => Self::Latency,
|
||||
UsagePerformanceMetricKind::FirstToken => Self::FirstToken,
|
||||
UsagePerformanceMetricKind::Throughput => Self::Throughput,
|
||||
UsagePerformanceMetricKind::Load => Self::Load,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub enum UsageCostMetricKind {
|
||||
Cost,
|
||||
Savings,
|
||||
Cache,
|
||||
}
|
||||
|
||||
impl From<UsageCostMetricKind> for domain::UsageInsightMetric {
|
||||
fn from(value: UsageCostMetricKind) -> Self {
|
||||
match value {
|
||||
UsageCostMetricKind::Cost => Self::Cost,
|
||||
UsageCostMetricKind::Savings => Self::Savings,
|
||||
UsageCostMetricKind::Cache => Self::Cache,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl UsageQuery {
|
||||
/// 校验 Element Plus 风格的页码分页字段
|
||||
pub fn validate_pagination(&self) -> Result<(u32, u16), WireValidationError> {
|
||||
|
||||
@@ -24,6 +24,12 @@ where
|
||||
"/api/admin/usage/insights/overview",
|
||||
get(usage_insights_overview::<S>),
|
||||
)
|
||||
.route("/api/admin/usage/insights/health", get(usage_health::<S>))
|
||||
.route(
|
||||
"/api/admin/usage/insights/performance",
|
||||
get(usage_performance::<S>),
|
||||
)
|
||||
.route("/api/admin/usage/insights/cost", get(usage_cost::<S>))
|
||||
.route(
|
||||
"/api/admin/usage/insights/diagnostics",
|
||||
get(usage_insights_diagnostics::<S>),
|
||||
@@ -207,6 +213,74 @@ where
|
||||
))
|
||||
}
|
||||
|
||||
pub(crate) async fn usage_health<S>(
|
||||
_auth: AdminAuth,
|
||||
State(state): State<S>,
|
||||
AdminQuery(query): AdminQuery<UsageAnalyticsQuery>,
|
||||
) -> Result<impl IntoResponse, AdminError>
|
||||
where
|
||||
S: SessionState + Send + Sync,
|
||||
{
|
||||
usage_metric_response(state, query, domain::UsageInsightMetric::Health).await
|
||||
}
|
||||
|
||||
pub(crate) async fn usage_performance<S>(
|
||||
_auth: AdminAuth,
|
||||
State(state): State<S>,
|
||||
AdminQuery(query): AdminQuery<UsageMetricQuery<UsagePerformanceMetricKind>>,
|
||||
) -> Result<impl IntoResponse, AdminError>
|
||||
where
|
||||
S: SessionState + Send + Sync,
|
||||
{
|
||||
let (range, metric) = query.into_parts();
|
||||
usage_metric_response(state, range, metric.into()).await
|
||||
}
|
||||
|
||||
pub(crate) async fn usage_cost<S>(
|
||||
_auth: AdminAuth,
|
||||
State(state): State<S>,
|
||||
AdminQuery(query): AdminQuery<UsageMetricQuery<UsageCostMetricKind>>,
|
||||
) -> Result<impl IntoResponse, AdminError>
|
||||
where
|
||||
S: SessionState + Send + Sync,
|
||||
{
|
||||
let (range, metric) = query.into_parts();
|
||||
usage_metric_response(state, range, metric.into()).await
|
||||
}
|
||||
|
||||
async fn usage_metric_response<S>(
|
||||
state: S,
|
||||
query: UsageAnalyticsQuery,
|
||||
metric: domain::UsageInsightMetric,
|
||||
) -> Result<impl IntoResponse, AdminError>
|
||||
where
|
||||
S: SessionState + Send + Sync,
|
||||
{
|
||||
let time = crate::time::TimePresenter::new(state.admin_services().timezone());
|
||||
let range = usage_range(
|
||||
query.start_time.as_deref(),
|
||||
query.end_time.as_deref(),
|
||||
query.period.as_deref(),
|
||||
query.as_of,
|
||||
state.admin_services().timezone(),
|
||||
)
|
||||
.map_err(map_wire_error)?;
|
||||
let filter = domain::UsageFilter {
|
||||
provider_kind: non_empty(query.provider),
|
||||
..Default::default()
|
||||
};
|
||||
let result = state
|
||||
.admin_services()
|
||||
.observability()
|
||||
.usage_metric(range, filter, metric)
|
||||
.await
|
||||
.map_err(map_service_error)?;
|
||||
Ok(AdminResponse::new(
|
||||
StatusCode::OK,
|
||||
AdminEnvelope::ok(usage_metric_view(result, time)),
|
||||
))
|
||||
}
|
||||
|
||||
pub(crate) async fn usage_insights_diagnostics<S>(
|
||||
_auth: AdminAuth,
|
||||
State(state): State<S>,
|
||||
|
||||
@@ -406,6 +406,7 @@ pub struct DashboardAccountUsageView {
|
||||
pub provider: String,
|
||||
pub authentication_kind: String,
|
||||
pub email: String,
|
||||
pub notes: Option<String>,
|
||||
pub plan_type: Option<String>,
|
||||
pub plan_type_display: String,
|
||||
pub tokens: String,
|
||||
@@ -648,6 +649,16 @@ pub struct OverviewHealthView {
|
||||
pub points: Vec<OverviewHealthPointView>,
|
||||
}
|
||||
|
||||
/// 并发与调度等待的负载指标
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct OverviewLoadView {
|
||||
pub average_concurrency: f64,
|
||||
pub peak_concurrency: u64,
|
||||
pub admission_decision_p95_ms: Option<f64>,
|
||||
pub account_selection_wait_p95_ms: Option<f64>,
|
||||
}
|
||||
|
||||
/// 洞察性能趋势点
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
@@ -660,15 +671,10 @@ pub struct OverviewPerformancePointView {
|
||||
pub first_token_p50_ms: Option<f64>,
|
||||
pub first_token_p95_ms: Option<f64>,
|
||||
pub first_token_p99_ms: Option<f64>,
|
||||
pub admission_decision_p50_ms: Option<f64>,
|
||||
pub admission_decision_p95_ms: Option<f64>,
|
||||
pub account_selection_wait_p50_ms: Option<f64>,
|
||||
pub account_selection_wait_p95_ms: Option<f64>,
|
||||
pub output_throughput_p10: Option<u64>,
|
||||
pub output_throughput_p50: Option<u64>,
|
||||
pub output_throughput_p90: Option<u64>,
|
||||
pub capacity_utilization: Option<f64>,
|
||||
pub capacity_utilization_p95: Option<f64>,
|
||||
pub load: OverviewLoadView,
|
||||
}
|
||||
|
||||
/// 洞察性能摘要
|
||||
@@ -681,20 +687,10 @@ pub struct OverviewPerformanceView {
|
||||
pub first_token_p50_ms: Option<f64>,
|
||||
pub first_token_p95_ms: Option<f64>,
|
||||
pub first_token_p99_ms: Option<f64>,
|
||||
pub admission_decision_p50_ms: Option<f64>,
|
||||
pub admission_decision_p95_ms: Option<f64>,
|
||||
pub account_selection_wait_p50_ms: Option<f64>,
|
||||
pub account_selection_wait_p95_ms: Option<f64>,
|
||||
pub output_throughput_p10: Option<u64>,
|
||||
pub output_throughput_p50: Option<u64>,
|
||||
pub output_throughput_p90: Option<u64>,
|
||||
pub capacity_utilization: Option<f64>,
|
||||
pub capacity_utilization_p95: Option<f64>,
|
||||
pub latency_coverage: f64,
|
||||
pub first_token_coverage: f64,
|
||||
pub admission_decision_coverage: f64,
|
||||
pub account_selection_wait_coverage: f64,
|
||||
pub capacity_coverage: f64,
|
||||
pub load: OverviewLoadView,
|
||||
pub points: Vec<OverviewPerformancePointView>,
|
||||
}
|
||||
|
||||
@@ -763,6 +759,86 @@ pub struct UsageInsightsOverviewView {
|
||||
pub providers: Vec<ProviderOverviewView>,
|
||||
}
|
||||
|
||||
/// 单项洞察响应
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct UsageMetricView {
|
||||
pub analytics: AnalyticsView,
|
||||
pub granularity: String,
|
||||
#[serde(flatten)]
|
||||
pub data: UsageMetricDataView,
|
||||
}
|
||||
|
||||
/// 指标标签与响应字段一一对应,未查询的数据不会出现在响应中
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(tag = "metric", content = "data", rename_all = "camelCase")]
|
||||
pub enum UsageMetricDataView {
|
||||
Health(OverviewHealthView),
|
||||
Latency(UsageMetricSeriesView<UsageLatencyView>),
|
||||
FirstToken(UsageMetricSeriesView<UsageLatencyView>),
|
||||
Throughput(UsageMetricSeriesView<UsageThroughputView>),
|
||||
Load(UsageMetricSeriesView<OverviewLoadView>),
|
||||
Cost(UsageMetricSeriesView<UsageCostSummaryView, UsageCostComparisonView>),
|
||||
Savings(UsageMetricSeriesView<UsageCostSummaryView, Option<String>>),
|
||||
Cache(UsageMetricSeriesView<UsageCostSummaryView, UsageCacheView>),
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct UsageMetricSeriesView<S, P = S> {
|
||||
pub summary: S,
|
||||
pub points: Vec<UsageMetricPointView<P>>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct UsageMetricPointView<P> {
|
||||
pub bucket: DateTime<Utc>,
|
||||
pub label: String,
|
||||
pub request_count: u64,
|
||||
pub value: P,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct UsageLatencyView {
|
||||
pub p50_ms: Option<f64>,
|
||||
pub p95_ms: Option<f64>,
|
||||
pub p99_ms: Option<f64>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
pub struct UsageThroughputView {
|
||||
pub p10: Option<u64>,
|
||||
pub p50: Option<u64>,
|
||||
pub p90: Option<u64>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct UsageCostSummaryView {
|
||||
pub estimated_cost: Option<String>,
|
||||
pub cache_savings: Option<String>,
|
||||
pub tier_premium: Option<String>,
|
||||
pub cost_per_successful_request: Option<String>,
|
||||
pub coverage: CostCoverageView,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct UsageCostComparisonView {
|
||||
pub estimated_cost: Option<String>,
|
||||
pub no_cache_cost: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct UsageCacheView {
|
||||
pub input_tokens: u64,
|
||||
pub cached_token_rate: f64,
|
||||
pub cache_hit_request_rate: Option<f64>,
|
||||
}
|
||||
|
||||
/// 诊断聚合项目
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
|
||||
@@ -56,13 +56,7 @@ pub struct RuntimeSettingsFields {
|
||||
pub usage_retention_days: u64,
|
||||
pub ops_event_retention_days: u64,
|
||||
pub audit_retention_days: u64,
|
||||
pub account_auto_freeze_enabled: bool,
|
||||
pub account_auto_freeze_threshold: u64,
|
||||
pub account_auto_freeze_window_seconds: u64,
|
||||
pub account_auto_freeze_duration_seconds: u64,
|
||||
pub account_auto_freeze_probe_enabled: bool,
|
||||
pub account_auto_freeze_probe_model: Option<String>,
|
||||
pub account_auto_freeze_adaptive_concurrency: bool,
|
||||
pub account_error_protection: gateway_core::account::AccountErrorProtection,
|
||||
pub account_warmup_enabled: bool,
|
||||
pub account_warmup_schedule_time: String,
|
||||
pub account_warmup_model: Option<String>,
|
||||
@@ -587,18 +581,13 @@ fn wire_model_mappings(mappings: DomainModelMappings) -> ModelMappings {
|
||||
|
||||
fn map_wire_error(error: WireValidationError) -> AdminError {
|
||||
let message = match error.field() {
|
||||
"settingsRefreshConcurrencyOverflow" => "refreshConcurrency 不合法".to_owned(),
|
||||
"settingsMaxConcurrencyOverflow" => "maxConcurrentPerAccount 不合法".to_owned(),
|
||||
"settingsUsageRetentionOverflow" => "usageRetentionDays 不合法".to_owned(),
|
||||
"settingsOpsRetentionOverflow" => "opsEventRetentionDays 不合法".to_owned(),
|
||||
"settingsAuditRetentionOverflow" => "auditRetentionDays 不合法".to_owned(),
|
||||
"requestLocation" => "请求位置不合法,请检查国家代码、地区和城市".to_owned(),
|
||||
"settingsFreezeThresholdOverflow" => "accountAutoFreezeThreshold 不合法".to_owned(),
|
||||
"accountAutoFreezeThreshold" => "账号自动冻结阈值应为 2~1000 的整数".to_owned(),
|
||||
"accountAutoFreezeWindowSeconds" => "账号自动冻结统计窗口应为 60~3600 秒".to_owned(),
|
||||
"accountAutoFreezeDurationSeconds" => "账号自动冻结时长应为 300~604800 秒".to_owned(),
|
||||
"accountAutoFreezeProbeModel" => "探测模型格式不合法".to_owned(),
|
||||
"providerRequestProfiles" => "Provider 请求画像格式不合法或字段冲突".to_owned(),
|
||||
"settingsRefreshConcurrencyOverflow" => "刷新并发数不合法".to_owned(),
|
||||
"settingsMaxConcurrencyOverflow" => "单账号并发上限不合法".to_owned(),
|
||||
"settingsUsageRetentionOverflow" => "用量记录保留天数不合法".to_owned(),
|
||||
"settingsOpsRetentionOverflow" => "运维事件保留天数不合法".to_owned(),
|
||||
"settingsAuditRetentionOverflow" => "审计记录保留天数不合法".to_owned(),
|
||||
"requestLocation" => "请求位置的国家代码、地区或城市不合法".to_owned(),
|
||||
"providerRequestProfiles" => "服务商客户端身份配置无效或字段冲突".to_owned(),
|
||||
"minCodexDesktopVersion" => "Codex Desktop 最低版本格式不合法".to_owned(),
|
||||
"minCodexCliVersion" => "Codex CLI 最低版本格式不合法".to_owned(),
|
||||
field => format!("{field} 字段不合法"),
|
||||
@@ -776,14 +765,7 @@ impl RuntimeSettingsFields {
|
||||
.map_err(|_| WireValidationError::new("settingsOpsRetentionOverflow"))?,
|
||||
audit_retention_days: u32::try_from(self.audit_retention_days)
|
||||
.map_err(|_| WireValidationError::new("settingsAuditRetentionOverflow"))?,
|
||||
account_auto_freeze_enabled: self.account_auto_freeze_enabled,
|
||||
account_auto_freeze_threshold: u32::try_from(self.account_auto_freeze_threshold)
|
||||
.map_err(|_| WireValidationError::new("settingsFreezeThresholdOverflow"))?,
|
||||
account_auto_freeze_window_seconds: self.account_auto_freeze_window_seconds,
|
||||
account_auto_freeze_duration_seconds: self.account_auto_freeze_duration_seconds,
|
||||
account_auto_freeze_probe_enabled: self.account_auto_freeze_probe_enabled,
|
||||
account_auto_freeze_probe_model: self.account_auto_freeze_probe_model,
|
||||
account_auto_freeze_adaptive_concurrency: self.account_auto_freeze_adaptive_concurrency,
|
||||
account_error_protection: self.account_error_protection,
|
||||
account_warmup_enabled: self.account_warmup_enabled,
|
||||
account_warmup_schedule_time: self.account_warmup_schedule_time,
|
||||
account_warmup_model: self.account_warmup_model,
|
||||
@@ -819,14 +801,7 @@ impl From<gateway_admin::model::settings::RuntimeSettingsValues> for RuntimeSett
|
||||
usage_retention_days: u64::from(settings.usage_retention_days),
|
||||
ops_event_retention_days: u64::from(settings.ops_event_retention_days),
|
||||
audit_retention_days: u64::from(settings.audit_retention_days),
|
||||
account_auto_freeze_enabled: settings.account_auto_freeze_enabled,
|
||||
account_auto_freeze_threshold: u64::from(settings.account_auto_freeze_threshold),
|
||||
account_auto_freeze_window_seconds: settings.account_auto_freeze_window_seconds,
|
||||
account_auto_freeze_duration_seconds: settings.account_auto_freeze_duration_seconds,
|
||||
account_auto_freeze_probe_enabled: settings.account_auto_freeze_probe_enabled,
|
||||
account_auto_freeze_probe_model: settings.account_auto_freeze_probe_model,
|
||||
account_auto_freeze_adaptive_concurrency: settings
|
||||
.account_auto_freeze_adaptive_concurrency,
|
||||
account_error_protection: settings.account_error_protection,
|
||||
account_warmup_enabled: settings.account_warmup_enabled,
|
||||
account_warmup_schedule_time: settings.account_warmup_schedule_time,
|
||||
account_warmup_model: settings.account_warmup_model,
|
||||
@@ -839,13 +814,7 @@ impl From<gateway_admin::model::settings::RuntimeSettingsValues> for RuntimeSett
|
||||
|
||||
fn settings_validation_error(field: &'static str) -> WireValidationError {
|
||||
WireValidationError::new(match field {
|
||||
"account_auto_freeze_adaptive_concurrency" => "accountAutoFreezeAdaptiveConcurrency",
|
||||
"account_auto_freeze_duration_seconds" => "accountAutoFreezeDurationSeconds",
|
||||
"account_auto_freeze_enabled" => "accountAutoFreezeEnabled",
|
||||
"account_auto_freeze_probe_enabled" => "accountAutoFreezeProbeEnabled",
|
||||
"account_auto_freeze_probe_model" => "accountAutoFreezeProbeModel",
|
||||
"account_auto_freeze_threshold" => "accountAutoFreezeThreshold",
|
||||
"account_auto_freeze_window_seconds" => "accountAutoFreezeWindowSeconds",
|
||||
"account_error_protection" => "accountErrorProtection",
|
||||
"account_warmup_enabled" => "accountWarmupEnabled",
|
||||
"account_warmup_model" => "accountWarmupModel",
|
||||
"reset_credit_lead_minutes" => "resetCreditLeadMinutes",
|
||||
|
||||
@@ -153,6 +153,19 @@ pub(crate) fn client_access_error_response(error: ClientAccessError) -> Response
|
||||
#[must_use]
|
||||
pub fn identify_codex_client(headers: &HeaderMap) -> Option<IdentifiedCodexClient> {
|
||||
const MAXIMUM_HEADER_LENGTH: usize = 4096;
|
||||
// 官方 CLI、IDE、SDK 与 App Server 的 UA 产品版本均来自所运行的 Codex Core
|
||||
const CORE_CLIENT_PRODUCTS: &[&str] = &[
|
||||
"codex_cli_rs",
|
||||
"codex-cli",
|
||||
"codex-tui",
|
||||
"codex_exec",
|
||||
"codex_vscode",
|
||||
"codex_sdk_ts",
|
||||
"codex_python_sdk",
|
||||
"codex-app-server",
|
||||
"codex-app-server-sdk",
|
||||
"codex_mcp_server",
|
||||
];
|
||||
|
||||
let originator = bounded_ascii_header(headers, "originator", MAXIMUM_HEADER_LENGTH);
|
||||
let user_agent = bounded_ascii_header(headers, "user-agent", MAXIMUM_HEADER_LENGTH);
|
||||
@@ -186,22 +199,17 @@ pub fn identify_codex_client(headers: &HeaderMap) -> Option<IdentifiedCodexClien
|
||||
for product in user_agent.split(|character: char| {
|
||||
character.is_ascii_whitespace() || matches!(character, '(' | ')' | ';' | ',' | '[' | ']')
|
||||
}) {
|
||||
for prefix in ["codex_cli_rs", "codex-cli"] {
|
||||
if product.eq_ignore_ascii_case(prefix) {
|
||||
return Some(IdentifiedCodexClient {
|
||||
kind: CodexClientKind::Cli,
|
||||
version: None,
|
||||
});
|
||||
}
|
||||
let Some((name, version)) = product.split_once('/') else {
|
||||
continue;
|
||||
};
|
||||
if name.eq_ignore_ascii_case(prefix) {
|
||||
return Some(IdentifiedCodexClient {
|
||||
kind: CodexClientKind::Cli,
|
||||
version: CodexClientVersion::parse(version).ok(),
|
||||
});
|
||||
}
|
||||
let (name, version) = product
|
||||
.split_once('/')
|
||||
.map_or((product, None), |(name, version)| (name, Some(version)));
|
||||
if CORE_CLIENT_PRODUCTS
|
||||
.iter()
|
||||
.any(|known| name.eq_ignore_ascii_case(known))
|
||||
{
|
||||
return Some(IdentifiedCodexClient {
|
||||
kind: CodexClientKind::Cli,
|
||||
version: version.and_then(|value| CodexClientVersion::parse(value).ok()),
|
||||
});
|
||||
}
|
||||
}
|
||||
None
|
||||
|
||||
@@ -0,0 +1,319 @@
|
||||
//! Chat 出口持有协议投影缓冲 提交与资源回收始终委托原受管正文
|
||||
|
||||
use std::collections::VecDeque;
|
||||
|
||||
use axum::body::to_bytes;
|
||||
use bytes::Bytes;
|
||||
use futures::future::BoxFuture;
|
||||
use gateway_core::{
|
||||
engine::middleware::{
|
||||
MiddlewareBody, MiddlewareError, MiddlewareFrame, MiddlewareFraming, MiddlewareHeader,
|
||||
MiddlewareResponse,
|
||||
},
|
||||
error::{GatewayError, GatewayErrorKind},
|
||||
};
|
||||
use gateway_protocol::openai::sse::{DONE_SSE_FRAME, SseEvent, SseEventDecoder, sse_frame_is_done};
|
||||
use serde_json::Value;
|
||||
|
||||
use crate::openai::{middleware::error_response, responses::ProtocolErrorBody};
|
||||
|
||||
use super::response::{ChatIdentity, ChatStreamEncoder, project_response};
|
||||
|
||||
pub(super) fn project_middleware_response(
|
||||
response: MiddlewareResponse,
|
||||
identity: ChatIdentity,
|
||||
streaming: bool,
|
||||
include_usage: bool,
|
||||
) -> MiddlewareResponse {
|
||||
if !(200..300).contains(&response.status_code()) {
|
||||
return response;
|
||||
}
|
||||
let (protocol, status, mut headers, body, envelope) = response.into_parts();
|
||||
// 投影改变正文长度和表示形式 不继承 Responses 的实体长度或编码
|
||||
headers.retain(|header| {
|
||||
!["content-length", "content-type", "content-encoding"]
|
||||
.iter()
|
||||
.any(|name| header.name().eq_ignore_ascii_case(name))
|
||||
});
|
||||
headers.push(MiddlewareHeader::new(
|
||||
"content-type",
|
||||
Bytes::from_static(if streaming {
|
||||
b"text/event-stream"
|
||||
} else {
|
||||
b"application/json"
|
||||
}),
|
||||
));
|
||||
let projection = if streaming {
|
||||
Projection::Stream(Box::new(StreamProjection {
|
||||
encoder: ChatStreamEncoder::new(identity, include_usage),
|
||||
decoder: SseEventDecoder::default(),
|
||||
done_received: false,
|
||||
}))
|
||||
} else {
|
||||
Projection::Json(identity)
|
||||
};
|
||||
let mut response = MiddlewareResponse::new(
|
||||
protocol,
|
||||
status,
|
||||
headers,
|
||||
Box::new(ChatBody {
|
||||
inner: Some(body),
|
||||
projection,
|
||||
pending: VecDeque::new(),
|
||||
committed: false,
|
||||
finished: false,
|
||||
}),
|
||||
);
|
||||
if let Some(envelope) = envelope {
|
||||
response = response.with_envelope(envelope);
|
||||
}
|
||||
response
|
||||
}
|
||||
|
||||
pub(super) fn chat_error_frame(data: &Value) -> Bytes {
|
||||
let error = data
|
||||
.pointer("/response/error")
|
||||
.or_else(|| data.get("error"));
|
||||
Bytes::from(format!("data: {}\n\n", serde_json::json!({"error": error})))
|
||||
}
|
||||
|
||||
enum Projection {
|
||||
Json(ChatIdentity),
|
||||
Stream(Box<StreamProjection>),
|
||||
}
|
||||
|
||||
impl Projection {
|
||||
fn terminal_seen(&self) -> bool {
|
||||
matches!(self, Self::Stream(stream) if stream.terminal_seen())
|
||||
}
|
||||
}
|
||||
|
||||
struct StreamProjection {
|
||||
encoder: ChatStreamEncoder,
|
||||
decoder: SseEventDecoder,
|
||||
done_received: bool,
|
||||
}
|
||||
|
||||
impl StreamProjection {
|
||||
fn terminal_seen(&self) -> bool {
|
||||
self.encoder.is_completed() || self.encoder.has_wire_failure()
|
||||
}
|
||||
|
||||
fn project(&mut self, events: Vec<SseEvent>) -> Result<Vec<Bytes>, MiddlewareError> {
|
||||
let mut output = Vec::new();
|
||||
for event in events {
|
||||
if self.done_received {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
let data = serde_json::from_str(&event.data).map_err(|_| invalid_response())?;
|
||||
// codec 区分辅助事件与终态后的业务输出 交付层不再维护一份事件分类
|
||||
output.extend(
|
||||
self.encoder
|
||||
.push_event(event.event.as_deref(), &data)
|
||||
.map_err(projection_error)?,
|
||||
);
|
||||
}
|
||||
Ok(output)
|
||||
}
|
||||
}
|
||||
|
||||
struct ChatBody {
|
||||
inner: Option<Box<dyn MiddlewareBody>>,
|
||||
projection: Projection,
|
||||
pending: VecDeque<MiddlewareFrame>,
|
||||
committed: bool,
|
||||
finished: bool,
|
||||
}
|
||||
|
||||
impl ChatBody {
|
||||
async fn read_one(&mut self) -> Result<(), MiddlewareError> {
|
||||
let inner = self.inner.as_mut().ok_or(MiddlewareError::InvalidState)?;
|
||||
let Some(frame) = inner.next_frame().await? else {
|
||||
if let Projection::Stream(stream) = &mut self.projection {
|
||||
let events = stream.decoder.finish().map_err(|_| invalid_response())?;
|
||||
let frames = stream.project(events)?;
|
||||
if !frames.is_empty() || !stream.done_received || !inner.is_finalized() {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
self.pending.push_back(done_frame());
|
||||
}
|
||||
self.finished = true;
|
||||
return Ok(());
|
||||
};
|
||||
let transformed = frame.transformed();
|
||||
let (bytes, framing, terminal, envelope) = frame.into_parts();
|
||||
let output = match &mut self.projection {
|
||||
Projection::Json(identity) => {
|
||||
if framing != MiddlewareFraming::JsonDocument || !terminal {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
let value = serde_json::from_slice(&bytes).map_err(|_| invalid_response())?;
|
||||
let value = project_response(&value, identity).map_err(projection_error)?;
|
||||
vec![MiddlewareFrame::new(
|
||||
Bytes::from(serde_json::to_vec(&value).map_err(|_| invalid_response())?),
|
||||
MiddlewareFraming::JsonDocument,
|
||||
true,
|
||||
)]
|
||||
}
|
||||
Projection::Stream(stream) => {
|
||||
if framing != MiddlewareFraming::SseEvent {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
let events = stream
|
||||
.decoder
|
||||
.push(&bytes)
|
||||
.map_err(|_| invalid_response())?;
|
||||
let output = stream.project(events)?;
|
||||
// 共用 SSE 解码器只返回业务事件 DONE 由受管正文的控制帧单独确认
|
||||
if std::str::from_utf8(&bytes).is_ok_and(sse_frame_is_done) {
|
||||
if stream.done_received || !stream.terminal_seen() {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
stream.done_received = true;
|
||||
}
|
||||
if terminal && !stream.done_received {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
output
|
||||
.into_iter()
|
||||
.map(|bytes| MiddlewareFrame::new(bytes, MiddlewareFraming::SseEvent, false))
|
||||
.collect()
|
||||
}
|
||||
};
|
||||
let mut output: VecDeque<_> = output
|
||||
.into_iter()
|
||||
.map(|frame| frame.with_transformed(transformed))
|
||||
.collect();
|
||||
if let Some(envelope) = envelope
|
||||
&& let Some(last) = output.pop_back()
|
||||
{
|
||||
output.push_back(last.with_envelope(envelope));
|
||||
}
|
||||
self.pending.extend(output);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn verify_terminal(&mut self) -> Result<(), MiddlewareError> {
|
||||
while !self.finished {
|
||||
self.read_one().await?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn fail_stream(&mut self, error: MiddlewareError) {
|
||||
self.pending.clear();
|
||||
if let Some(inner) = self.inner.take() {
|
||||
inner.close().await;
|
||||
}
|
||||
let response = error_response(error);
|
||||
let bytes = to_bytes(response.into_body(), usize::MAX)
|
||||
.await
|
||||
.unwrap_or_default();
|
||||
let value = serde_json::from_slice(&bytes).unwrap_or_else(|_| {
|
||||
serde_json::json!({"error": {
|
||||
"type": "server_error", "code": "invalid_upstream_response",
|
||||
"message": "The gateway could not deliver the Chat response."
|
||||
}})
|
||||
});
|
||||
self.pending.push_back(MiddlewareFrame::new(
|
||||
chat_error_frame(&value),
|
||||
MiddlewareFraming::SseEvent,
|
||||
false,
|
||||
));
|
||||
self.pending.push_back(done_frame());
|
||||
self.finished = true;
|
||||
}
|
||||
}
|
||||
|
||||
impl MiddlewareBody for ChatBody {
|
||||
fn next_frame(&mut self) -> BoxFuture<'_, Result<Option<MiddlewareFrame>, MiddlewareError>> {
|
||||
Box::pin(async move {
|
||||
while self.pending.is_empty() && !self.finished {
|
||||
let result = match self.read_one().await {
|
||||
Ok(()) if self.committed && self.projection.terminal_seen() => {
|
||||
self.verify_terminal().await
|
||||
}
|
||||
result => result,
|
||||
};
|
||||
if let Err(error) = result {
|
||||
if !self.committed {
|
||||
return Err(error);
|
||||
}
|
||||
self.fail_stream(error).await;
|
||||
}
|
||||
}
|
||||
Ok(self.pending.pop_front())
|
||||
})
|
||||
}
|
||||
|
||||
fn commit_downstream(
|
||||
&mut self,
|
||||
client_status_code: Option<u16>,
|
||||
) -> BoxFuture<'_, Result<(), MiddlewareError>> {
|
||||
Box::pin(async move {
|
||||
self.inner
|
||||
.as_mut()
|
||||
.ok_or(MiddlewareError::InvalidState)?
|
||||
.commit_downstream(client_status_code)
|
||||
.await?;
|
||||
self.committed = true;
|
||||
// HTTP 尚未发送首帧 即使首批已经包含 finish 或 usage 也先确认原会话完成结算
|
||||
if self.projection.terminal_seen()
|
||||
&& let Err(error) = self.verify_terminal().await
|
||||
{
|
||||
// 原会话已提交 不能把已记录的 200 改成另一 HTTP 状态
|
||||
// 首个 Chat 帧为 role 或 error 成功终态仍在 pending 内可安全撤回
|
||||
self.fail_stream(error).await;
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
fn record_client_status(&mut self, status: u16) -> BoxFuture<'_, Result<(), MiddlewareError>> {
|
||||
match self.inner.as_mut() {
|
||||
Some(inner) => inner.record_client_status(status),
|
||||
None => Box::pin(async { Ok(()) }),
|
||||
}
|
||||
}
|
||||
|
||||
fn is_finalized(&self) -> bool {
|
||||
self.inner.as_ref().is_none_or(|inner| inner.is_finalized())
|
||||
}
|
||||
|
||||
fn close(mut self: Box<Self>) -> BoxFuture<'static, ()> {
|
||||
let inner = self.inner.take();
|
||||
Box::pin(async move {
|
||||
if let Some(inner) = inner {
|
||||
inner.close().await;
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
fn projection_error(error: ProtocolErrorBody) -> MiddlewareError {
|
||||
MiddlewareError::Gateway(
|
||||
GatewayError::new(
|
||||
GatewayErrorKind::UpstreamUnavailable,
|
||||
"The gateway could not project the upstream response to Chat Completions.",
|
||||
)
|
||||
.with_client_code(error.error.code),
|
||||
)
|
||||
}
|
||||
|
||||
fn invalid_response() -> MiddlewareError {
|
||||
MiddlewareError::Gateway(
|
||||
GatewayError::new(
|
||||
GatewayErrorKind::UpstreamUnavailable,
|
||||
"The gateway could not deliver the Chat response.",
|
||||
)
|
||||
.with_client_code("invalid_upstream_response"),
|
||||
)
|
||||
}
|
||||
|
||||
fn done_frame() -> MiddlewareFrame {
|
||||
MiddlewareFrame::new(
|
||||
Bytes::from_static(DONE_SSE_FRAME.as_bytes()),
|
||||
MiddlewareFraming::SseEvent,
|
||||
true,
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
//! Chat HTTP 入口只规范化协议 共用 Responses 的执行与交付链路
|
||||
|
||||
use std::{net::SocketAddr, time::SystemTime};
|
||||
|
||||
use axum::{
|
||||
body::Bytes,
|
||||
extract::{Extension, State, connect_info::ConnectInfo},
|
||||
http::{HeaderMap, StatusCode, header::CONTENT_ENCODING},
|
||||
response::Response,
|
||||
};
|
||||
|
||||
use crate::ApiState;
|
||||
use crate::openai::{
|
||||
auth::{authenticate_client, client_access_error_response},
|
||||
error::{gateway_error_response, protocol_error_response},
|
||||
responses::{
|
||||
DecodedResponsesRequest, HttpResponseAdapter, RequestDecodeError, ResponsesHttpRequest,
|
||||
decode_request_with_headers, decompress_request_body, execute_prepared_response,
|
||||
},
|
||||
router::CHAT_COMPLETIONS_PATH,
|
||||
};
|
||||
|
||||
use super::{
|
||||
delivery::{chat_error_frame, project_middleware_response},
|
||||
request::convert_request,
|
||||
response::ChatIdentity,
|
||||
};
|
||||
|
||||
pub(in crate::openai) async fn chat_completions(
|
||||
State(state): State<ApiState>,
|
||||
connect_info: Option<Extension<ConnectInfo<SocketAddr>>>,
|
||||
ingress_id: Option<Extension<tower_http::request_id::RequestId>>,
|
||||
headers: HeaderMap,
|
||||
body: Bytes,
|
||||
) -> Response {
|
||||
let service = state.openai().clone();
|
||||
let client = match authenticate_client(&service, &headers).await {
|
||||
Ok(client) => client,
|
||||
Err(error) => return client_access_error_response(error),
|
||||
};
|
||||
let (decoded, middleware_body, include_usage) = match decode_request(
|
||||
&body,
|
||||
&headers,
|
||||
client.snapshot().responses_max_decompressed_body_bytes(),
|
||||
) {
|
||||
Ok(request) => request,
|
||||
Err(error) => {
|
||||
return protocol_error_response(StatusCode::BAD_REQUEST, error.protocol_body());
|
||||
}
|
||||
};
|
||||
let prepared = match service.execution().prepare_execution(client).await {
|
||||
Ok(prepared) => prepared,
|
||||
Err(error) => return gateway_error_response(&error),
|
||||
};
|
||||
let identity = ChatIdentity {
|
||||
id: format!("chatcmpl-{}", prepared.request_id().as_str()),
|
||||
created: SystemTime::now()
|
||||
.duration_since(SystemTime::UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_secs(),
|
||||
model: decoded.metadata().requested_model().to_owned(),
|
||||
};
|
||||
let streaming = decoded.metadata().stream();
|
||||
let adapter = HttpResponseAdapter {
|
||||
endpoint: CHAT_COMPLETIONS_PATH,
|
||||
wrap: Box::new(move |response| {
|
||||
project_middleware_response(response, identity, streaming, include_usage)
|
||||
}),
|
||||
stream_error: chat_error_frame,
|
||||
};
|
||||
execute_prepared_response(
|
||||
service,
|
||||
prepared,
|
||||
ResponsesHttpRequest {
|
||||
peer_address: connect_info.map(|Extension(ConnectInfo(address))| address),
|
||||
ingress_id,
|
||||
headers,
|
||||
decoded,
|
||||
middleware_body,
|
||||
},
|
||||
None,
|
||||
Some(adapter),
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
fn decode_request(
|
||||
body: &[u8],
|
||||
headers: &HeaderMap,
|
||||
maximum_body_bytes: usize,
|
||||
) -> Result<(DecodedResponsesRequest, Bytes, bool), RequestDecodeError> {
|
||||
let body = decompress_request_body(body, headers, maximum_body_bytes)?;
|
||||
let converted = convert_request(&body)?;
|
||||
let body =
|
||||
serde_json::to_vec(&converted.body).map_err(|_| RequestDecodeError::MalformedJson)?;
|
||||
let mut decoded_headers = headers.clone();
|
||||
decoded_headers.remove(CONTENT_ENCODING);
|
||||
let decoded = decode_request_with_headers(&body, &decoded_headers, maximum_body_bytes)?;
|
||||
Ok((decoded, Bytes::from(body), converted.include_usage))
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
//! Chat Completions 与 Responses 之间的 API 协议适配
|
||||
|
||||
mod delivery;
|
||||
mod http;
|
||||
mod request;
|
||||
mod response;
|
||||
|
||||
pub(super) use http::chat_completions;
|
||||
@@ -0,0 +1,550 @@
|
||||
//! 将 Chat 请求转换为 Responses 正文,协议选项留在 API,业务 JSON 与上游兼容规则保持原有归属
|
||||
|
||||
use serde_json::{Map, Value, json};
|
||||
|
||||
use crate::openai::responses::RequestDecodeError;
|
||||
|
||||
pub(in crate::openai) struct ConvertedChatRequest {
|
||||
pub(in crate::openai) body: Map<String, Value>,
|
||||
pub(in crate::openai) include_usage: bool,
|
||||
}
|
||||
|
||||
pub(in crate::openai) fn convert_request(
|
||||
body: &[u8],
|
||||
) -> Result<ConvertedChatRequest, RequestDecodeError> {
|
||||
let value = serde_json::from_slice(body).map_err(|_| RequestDecodeError::MalformedJson)?;
|
||||
let Value::Object(mut chat) = value else {
|
||||
return Err(RequestDecodeError::ExpectedObject);
|
||||
};
|
||||
validate_options(&chat)?;
|
||||
let model = required_string(&mut chat, "model", "model", false)?;
|
||||
let stream = optional_bool(&mut chat, "stream", "stream")?.unwrap_or(false);
|
||||
let include_usage = stream_options(optional(&mut chat, "stream_options"), stream)?;
|
||||
let input = messages(required(&mut chat, "messages", "messages")?)?;
|
||||
let mut body = Map::from_iter([
|
||||
("model".into(), model.into()),
|
||||
("input".into(), input.into()),
|
||||
("stream".into(), stream.into()),
|
||||
("store".into(), false.into()),
|
||||
]);
|
||||
if let Some(value) = optional(&mut chat, "tools") {
|
||||
body.insert("tools".into(), function_tools(value)?.into());
|
||||
}
|
||||
if let Some(value) = optional(&mut chat, "tool_choice") {
|
||||
let choice = tool_choice(value, body.get("tools"))?;
|
||||
body.insert("tool_choice".into(), choice);
|
||||
}
|
||||
if let Some(value) = optional_bool(&mut chat, "parallel_tool_calls", "parallel_tool_calls")? {
|
||||
body.insert("parallel_tool_calls".into(), value.into());
|
||||
}
|
||||
|
||||
let mut text = Map::new();
|
||||
if let Some(value) = optional(&mut chat, "response_format") {
|
||||
text.insert("format".into(), response_format(value)?);
|
||||
}
|
||||
if let Some(value) = optional(&mut chat, "verbosity") {
|
||||
string(&value, "verbosity", false)?;
|
||||
text.insert("verbosity".into(), value);
|
||||
}
|
||||
if !text.is_empty() {
|
||||
body.insert("text".into(), text.into());
|
||||
}
|
||||
if let Some(value) = optional(&mut chat, "reasoning_effort") {
|
||||
string(&value, "reasoning_effort", false)?;
|
||||
body.insert("reasoning".into(), json!({"effort": value}));
|
||||
}
|
||||
for field in ["max_tokens", "max_completion_tokens"] {
|
||||
if let Some(value) = optional(&mut chat, field) {
|
||||
if !value.as_u64().is_some_and(|tokens| tokens > 0) {
|
||||
return Err(invalid(field));
|
||||
}
|
||||
// 新字段优先,具体上游是否接受长度控制仍由现有 Provider 决定
|
||||
body.insert("max_output_tokens".into(), value);
|
||||
}
|
||||
}
|
||||
for (field, maximum) in [("temperature", 2.0), ("top_p", 1.0)] {
|
||||
if let Some(value) = optional(&mut chat, field) {
|
||||
if !value
|
||||
.as_f64()
|
||||
.is_some_and(|number| (0.0..=maximum).contains(&number))
|
||||
{
|
||||
return Err(invalid(field));
|
||||
}
|
||||
body.insert(field.into(), value);
|
||||
}
|
||||
}
|
||||
for field in [
|
||||
"user",
|
||||
"safety_identifier",
|
||||
"service_tier",
|
||||
"prompt_cache_key",
|
||||
"prompt_cache_retention",
|
||||
] {
|
||||
if let Some(value) = optional(&mut chat, field) {
|
||||
string(&value, field, true)?;
|
||||
body.insert(field.into(), value);
|
||||
}
|
||||
}
|
||||
for field in ["metadata", "prompt_cache_options", "moderation"] {
|
||||
if let Some(value) = optional(&mut chat, field) {
|
||||
let object = object(value, field)?;
|
||||
body.insert(field.into(), object.into());
|
||||
}
|
||||
}
|
||||
// 未识别的客户端顶层扩展不等于 Responses 参数,不进入上游正文
|
||||
Ok(ConvertedChatRequest {
|
||||
body,
|
||||
include_usage,
|
||||
})
|
||||
}
|
||||
|
||||
fn validate_options(chat: &Map<String, Value>) -> Result<(), RequestDecodeError> {
|
||||
for (field, value) in chat.iter().filter(|(_, value)| !value.is_null()) {
|
||||
let supported = match field.as_str() {
|
||||
"n" => value.as_u64() == Some(1),
|
||||
"store" | "logprobs" => value == false,
|
||||
"frequency_penalty" | "presence_penalty" => value.as_f64() == Some(0.0),
|
||||
"top_logprobs" => value.as_u64() == Some(0),
|
||||
"logit_bias" => value.as_object().is_some_and(Map::is_empty),
|
||||
"functions" | "stop" => value.as_array().is_some_and(Vec::is_empty),
|
||||
"modalities" => value == &json!(["text"]),
|
||||
"audio"
|
||||
| "prediction"
|
||||
| "seed"
|
||||
| "function_call"
|
||||
| "web_search_options"
|
||||
| "reasoning_content"
|
||||
| "input"
|
||||
| "previous_response_id"
|
||||
| "conversation"
|
||||
| "instructions"
|
||||
| "reasoning"
|
||||
| "text"
|
||||
| "include"
|
||||
| "background"
|
||||
| "max_output_tokens" => false,
|
||||
_ => true,
|
||||
};
|
||||
if !supported {
|
||||
return Err(unsupported(field));
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn stream_options(value: Option<Value>, stream: bool) -> Result<bool, RequestDecodeError> {
|
||||
let Some(value) = value else {
|
||||
return Ok(false);
|
||||
};
|
||||
if !stream {
|
||||
return Err(invalid("stream_options"));
|
||||
}
|
||||
let mut options = object(value, "stream_options")?;
|
||||
let include_usage = optional_bool(
|
||||
&mut options,
|
||||
"include_usage",
|
||||
"stream_options.include_usage",
|
||||
)?
|
||||
.unwrap_or(false);
|
||||
if optional_bool(
|
||||
&mut options,
|
||||
"include_obfuscation",
|
||||
"stream_options.include_obfuscation",
|
||||
)? == Some(true)
|
||||
{
|
||||
// Chat 重编码会改变帧长度,不能声称继承上游流的长度混淆保证
|
||||
return Err(unsupported("stream_options.include_obfuscation"));
|
||||
}
|
||||
Ok(include_usage)
|
||||
}
|
||||
|
||||
fn messages(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
|
||||
let messages = array(value, "messages")?;
|
||||
if messages.is_empty() {
|
||||
return Err(RequestDecodeError::EmptyField {
|
||||
field: "messages".into(),
|
||||
});
|
||||
}
|
||||
let mut input = Vec::with_capacity(messages.len());
|
||||
for (index, value) in messages.into_iter().enumerate() {
|
||||
let path = format!("messages[{index}]");
|
||||
let mut message = object(value, &path)?;
|
||||
let role = required_string(&mut message, "role", &format!("{path}.role"), false)?;
|
||||
if !matches!(
|
||||
role.as_str(),
|
||||
"system" | "developer" | "user" | "assistant" | "tool"
|
||||
) {
|
||||
return Err(unsupported(&format!("{path}.role")));
|
||||
}
|
||||
if let Some(name) = optional(&mut message, "name")
|
||||
&& !string(&name, &format!("{path}.name"), true)?.is_empty()
|
||||
{
|
||||
return Err(unsupported(&format!("{path}.name")));
|
||||
}
|
||||
for field in ["function_call", "reasoning_content", "reasoning", "audio"] {
|
||||
if optional(&mut message, field).is_some() {
|
||||
return Err(unsupported(&format!("{path}.{field}")));
|
||||
}
|
||||
}
|
||||
for (field, allowed_role) in [
|
||||
("tool_calls", "assistant"),
|
||||
("refusal", "assistant"),
|
||||
("tool_call_id", "tool"),
|
||||
] {
|
||||
if role != allowed_role && message.get(field).is_some_and(|value| !value.is_null()) {
|
||||
return Err(invalid(&format!("{path}.{field}")));
|
||||
}
|
||||
}
|
||||
if role == "tool" {
|
||||
let call_id = required_string(
|
||||
&mut message,
|
||||
"tool_call_id",
|
||||
&format!("{path}.tool_call_id"),
|
||||
false,
|
||||
)?;
|
||||
let content_path = format!("{path}.content");
|
||||
let content = required(&mut message, "content", &content_path)?;
|
||||
let output = if content.is_string() {
|
||||
content
|
||||
} else {
|
||||
content_parts(content, &role, &content_path, false)?.into()
|
||||
};
|
||||
input.push(
|
||||
json!({"type": "function_call_output", "call_id": call_id, "output": output}),
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
let calls = optional(&mut message, "tool_calls")
|
||||
.map(|value| tool_calls(value, &format!("{path}.tool_calls")))
|
||||
.transpose()?
|
||||
.unwrap_or_default();
|
||||
let refusal = optional(&mut message, "refusal");
|
||||
if let Some(value) = &refusal {
|
||||
string(value, &format!("{path}.refusal"), true)?;
|
||||
}
|
||||
let content = optional(&mut message, "content");
|
||||
if content.is_none() && refusal.is_none() && calls.is_empty() {
|
||||
return Err(invalid(&format!("{path}.content")));
|
||||
}
|
||||
let is_assistant = role == "assistant";
|
||||
let mut parts = content
|
||||
.map(|value| content_parts(value, &role, &format!("{path}.content"), is_assistant))
|
||||
.transpose()?;
|
||||
if let Some(refusal) = refusal {
|
||||
parts
|
||||
.get_or_insert_with(Vec::new)
|
||||
.push(json!({"type": "refusal", "refusal": refusal}));
|
||||
}
|
||||
if let Some(parts) = parts {
|
||||
let mut item = json!({"type": "message", "role": role, "content": parts});
|
||||
if is_assistant {
|
||||
// assistant 历史采用输出消息合同,普通文本和 refusal 都不能编码成 input_text
|
||||
item["id"] = format!("msg_chat_{index}").into();
|
||||
item["status"] = "completed".into();
|
||||
}
|
||||
input.push(item);
|
||||
}
|
||||
input.extend(calls);
|
||||
}
|
||||
Ok(input)
|
||||
}
|
||||
|
||||
fn content_parts(
|
||||
value: Value,
|
||||
role: &str,
|
||||
path: &str,
|
||||
is_assistant: bool,
|
||||
) -> Result<Vec<Value>, RequestDecodeError> {
|
||||
let parts = match value {
|
||||
Value::String(text) => return Ok(vec![text_part(text, is_assistant)]),
|
||||
Value::Array(parts) => parts,
|
||||
_ => return Err(invalid_type(path, "a string or an array")),
|
||||
};
|
||||
let mut result = Vec::with_capacity(parts.len());
|
||||
for (index, value) in parts.into_iter().enumerate() {
|
||||
let path = format!("{path}[{index}]");
|
||||
let mut part = object(value, &path)?;
|
||||
let kind = required_string(&mut part, "type", &format!("{path}.type"), false)?;
|
||||
let mut converted = match kind.as_str() {
|
||||
"text" => text_part(
|
||||
required_string(&mut part, "text", &format!("{path}.text"), true)?,
|
||||
is_assistant,
|
||||
),
|
||||
"image_url" if role == "user" => image_part(
|
||||
required(&mut part, "image_url", &format!("{path}.image_url"))?,
|
||||
&format!("{path}.image_url"),
|
||||
)?,
|
||||
"refusal" if role == "assistant" => json!({
|
||||
"type": "refusal",
|
||||
"refusal": required_string(&mut part, "refusal", &format!("{path}.refusal"), true)?,
|
||||
}),
|
||||
_ => return Err(unsupported(&format!("{path}.type"))),
|
||||
};
|
||||
if let Some(value) = optional(&mut part, "prompt_cache_breakpoint") {
|
||||
if is_assistant {
|
||||
return Err(unsupported(&format!("{path}.prompt_cache_breakpoint")));
|
||||
}
|
||||
converted["prompt_cache_breakpoint"] =
|
||||
object(value, &format!("{path}.prompt_cache_breakpoint"))?.into();
|
||||
}
|
||||
result.push(converted);
|
||||
}
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
fn text_part(text: String, is_assistant: bool) -> Value {
|
||||
if is_assistant {
|
||||
json!({"type": "output_text", "text": text, "annotations": []})
|
||||
} else {
|
||||
json!({"type": "input_text", "text": text})
|
||||
}
|
||||
}
|
||||
|
||||
fn image_part(value: Value, path: &str) -> Result<Value, RequestDecodeError> {
|
||||
let mut image = object(value, path)?;
|
||||
let url = required_string(&mut image, "url", &format!("{path}.url"), false)?;
|
||||
let valid = url::Url::parse(&url).is_ok_and(|parsed| {
|
||||
matches!(parsed.scheme(), "http" | "https") && parsed.host_str().is_some()
|
||||
|| parsed.scheme() == "data"
|
||||
&& url.starts_with("data:image/")
|
||||
&& url
|
||||
.split_once(";base64,")
|
||||
.is_some_and(|(_, data)| !data.trim().is_empty())
|
||||
});
|
||||
if !valid {
|
||||
return Err(invalid(&format!("{path}.url")));
|
||||
}
|
||||
let mut converted = json!({"type": "input_image", "image_url": url});
|
||||
if let Some(detail) = optional(&mut image, "detail") {
|
||||
string(&detail, &format!("{path}.detail"), false)?;
|
||||
converted["detail"] = detail;
|
||||
}
|
||||
Ok(converted)
|
||||
}
|
||||
|
||||
fn tool_calls(value: Value, path: &str) -> Result<Vec<Value>, RequestDecodeError> {
|
||||
let calls = array(value, path)?;
|
||||
let mut result = Vec::with_capacity(calls.len());
|
||||
for (index, value) in calls.into_iter().enumerate() {
|
||||
let path = format!("{path}[{index}]");
|
||||
let mut call = object(value, &path)?;
|
||||
let id = required_string(&mut call, "id", &format!("{path}.id"), false)?;
|
||||
if required_string(&mut call, "type", &format!("{path}.type"), false)? != "function" {
|
||||
return Err(unsupported(&format!("{path}.type")));
|
||||
}
|
||||
let function_path = format!("{path}.function");
|
||||
let mut function = object(
|
||||
required(&mut call, "function", &function_path)?,
|
||||
&function_path,
|
||||
)?;
|
||||
let name = required_string(
|
||||
&mut function,
|
||||
"name",
|
||||
&format!("{function_path}.name"),
|
||||
false,
|
||||
)?;
|
||||
let arguments = required_string(
|
||||
&mut function,
|
||||
"arguments",
|
||||
&format!("{function_path}.arguments"),
|
||||
true,
|
||||
)?;
|
||||
result.push(
|
||||
json!({"type": "function_call", "call_id": id, "name": name, "arguments": arguments}),
|
||||
);
|
||||
}
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
fn function_tools(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
|
||||
array(value, "tools")?
|
||||
.into_iter()
|
||||
.enumerate()
|
||||
.map(|(index, value)| {
|
||||
let path = format!("tools[{index}]");
|
||||
let mut tool = object(value, &path)?;
|
||||
if required_string(&mut tool, "type", &format!("{path}.type"), false)? != "function" {
|
||||
return Err(unsupported(&format!("{path}.type")));
|
||||
}
|
||||
let function_path = format!("{path}.function");
|
||||
let mut function = object(
|
||||
required(&mut tool, "function", &function_path)?,
|
||||
&function_path,
|
||||
)?;
|
||||
let name = required_string(
|
||||
&mut function,
|
||||
"name",
|
||||
&format!("{function_path}.name"),
|
||||
false,
|
||||
)?;
|
||||
// Chat 的缺省是宽松函数 schema,不能让 Responses 的缺省收紧原始参数合同
|
||||
let strict =
|
||||
optional_bool(&mut function, "strict", &format!("{function_path}.strict"))?
|
||||
.unwrap_or(false);
|
||||
let mut converted = json!({"type": "function", "name": name, "strict": strict});
|
||||
if let Some(description) = optional(&mut function, "description") {
|
||||
string(&description, &format!("{function_path}.description"), true)?;
|
||||
converted["description"] = description;
|
||||
}
|
||||
if let Some(parameters) = optional(&mut function, "parameters") {
|
||||
converted["parameters"] = parameters;
|
||||
}
|
||||
Ok(converted)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn tool_choice(value: Value, tools: Option<&Value>) -> Result<Value, RequestDecodeError> {
|
||||
let tools = tools.and_then(Value::as_array);
|
||||
if let Value::String(choice) = &value {
|
||||
return match choice.as_str() {
|
||||
"auto" | "none" => Ok(value),
|
||||
"required" if tools.is_some_and(|tools| !tools.is_empty()) => Ok(value),
|
||||
"required" => Err(invalid("tool_choice")),
|
||||
_ => Err(unsupported("tool_choice")),
|
||||
};
|
||||
}
|
||||
let mut choice = object(value, "tool_choice")?;
|
||||
if required_string(&mut choice, "type", "tool_choice.type", false)? != "function" {
|
||||
return Err(unsupported("tool_choice.type"));
|
||||
}
|
||||
let mut function = object(
|
||||
required(&mut choice, "function", "tool_choice.function")?,
|
||||
"tool_choice.function",
|
||||
)?;
|
||||
let name = required_string(&mut function, "name", "tool_choice.function.name", false)?;
|
||||
if !tools.is_some_and(|tools| {
|
||||
tools
|
||||
.iter()
|
||||
.any(|tool| tool.get("name").and_then(Value::as_str) == Some(name.as_str()))
|
||||
}) {
|
||||
return Err(invalid("tool_choice.function.name"));
|
||||
}
|
||||
Ok(json!({"type": "function", "name": name}))
|
||||
}
|
||||
|
||||
fn response_format(value: Value) -> Result<Value, RequestDecodeError> {
|
||||
let mut format = object(value, "response_format")?;
|
||||
let kind = required_string(&mut format, "type", "response_format.type", false)?;
|
||||
match kind.as_str() {
|
||||
"text" | "json_object" => Ok(json!({"type": kind})),
|
||||
"json_schema" => {
|
||||
let mut schema = object(
|
||||
required(&mut format, "json_schema", "response_format.json_schema")?,
|
||||
"response_format.json_schema",
|
||||
)?;
|
||||
let name = required_string(
|
||||
&mut schema,
|
||||
"name",
|
||||
"response_format.json_schema.name",
|
||||
false,
|
||||
)?;
|
||||
let definition = required(&mut schema, "schema", "response_format.json_schema.schema")?;
|
||||
if definition.is_null() {
|
||||
return Err(invalid("response_format.json_schema.schema"));
|
||||
}
|
||||
let mut converted = json!({"type": "json_schema", "name": name, "schema": definition});
|
||||
if let Some(strict) =
|
||||
optional_bool(&mut schema, "strict", "response_format.json_schema.strict")?
|
||||
{
|
||||
converted["strict"] = strict.into();
|
||||
}
|
||||
if let Some(description) = optional(&mut schema, "description") {
|
||||
string(
|
||||
&description,
|
||||
"response_format.json_schema.description",
|
||||
true,
|
||||
)?;
|
||||
converted["description"] = description;
|
||||
}
|
||||
Ok(converted)
|
||||
}
|
||||
_ => Err(unsupported("response_format.type")),
|
||||
}
|
||||
}
|
||||
|
||||
fn optional(object: &mut Map<String, Value>, key: &str) -> Option<Value> {
|
||||
object.remove(key).filter(|value| !value.is_null())
|
||||
}
|
||||
|
||||
fn required(
|
||||
object: &mut Map<String, Value>,
|
||||
key: &str,
|
||||
path: &str,
|
||||
) -> Result<Value, RequestDecodeError> {
|
||||
object
|
||||
.remove(key)
|
||||
.ok_or_else(|| RequestDecodeError::MissingField { field: path.into() })
|
||||
}
|
||||
|
||||
fn object(value: Value, path: &str) -> Result<Map<String, Value>, RequestDecodeError> {
|
||||
match value {
|
||||
Value::Object(object) => Ok(object),
|
||||
_ => Err(invalid_type(path, "an object")),
|
||||
}
|
||||
}
|
||||
|
||||
fn array(value: Value, path: &str) -> Result<Vec<Value>, RequestDecodeError> {
|
||||
match value {
|
||||
Value::Array(array) => Ok(array),
|
||||
_ => Err(invalid_type(path, "an array")),
|
||||
}
|
||||
}
|
||||
|
||||
fn string<'a>(
|
||||
value: &'a Value,
|
||||
path: &str,
|
||||
allow_empty: bool,
|
||||
) -> Result<&'a str, RequestDecodeError> {
|
||||
let value = value
|
||||
.as_str()
|
||||
.ok_or_else(|| invalid_type(path, "a string"))?;
|
||||
if !allow_empty && value.trim().is_empty() {
|
||||
return Err(RequestDecodeError::EmptyField { field: path.into() });
|
||||
}
|
||||
Ok(value)
|
||||
}
|
||||
|
||||
fn required_string(
|
||||
object: &mut Map<String, Value>,
|
||||
key: &str,
|
||||
path: &str,
|
||||
allow_empty: bool,
|
||||
) -> Result<String, RequestDecodeError> {
|
||||
let value = required(object, key, path)?;
|
||||
string(&value, path, allow_empty)?;
|
||||
match value {
|
||||
Value::String(value) => Ok(value),
|
||||
_ => Err(invalid_type(path, "a string")),
|
||||
}
|
||||
}
|
||||
|
||||
fn optional_bool(
|
||||
object: &mut Map<String, Value>,
|
||||
key: &str,
|
||||
path: &str,
|
||||
) -> Result<Option<bool>, RequestDecodeError> {
|
||||
optional(object, key)
|
||||
.map(|value| {
|
||||
value
|
||||
.as_bool()
|
||||
.ok_or_else(|| invalid_type(path, "a boolean"))
|
||||
})
|
||||
.transpose()
|
||||
}
|
||||
|
||||
fn invalid_type(path: &str, expected: &'static str) -> RequestDecodeError {
|
||||
RequestDecodeError::InvalidType {
|
||||
field: path.into(),
|
||||
expected,
|
||||
}
|
||||
}
|
||||
|
||||
fn invalid(path: &str) -> RequestDecodeError {
|
||||
RequestDecodeError::InvalidValue { field: path.into() }
|
||||
}
|
||||
|
||||
fn unsupported(path: &str) -> RequestDecodeError {
|
||||
RequestDecodeError::UnsupportedField { field: path.into() }
|
||||
}
|
||||
@@ -0,0 +1,664 @@
|
||||
//! 将实际 Responses wire 投影为 Chat JSON 和 SSE,不拥有执行、计费或终结生命周期
|
||||
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
|
||||
use bytes::Bytes;
|
||||
use gateway_protocol::openai::events::{billable_usage_is_complete, extract_usage};
|
||||
use gateway_protocol::openai::sse::encode_sse_event;
|
||||
use serde::Serialize;
|
||||
use serde_json::{Value, json};
|
||||
|
||||
use crate::openai::responses::{ProtocolError, ProtocolErrorBody};
|
||||
|
||||
#[derive(Debug, Clone, Serialize)]
|
||||
pub(in crate::openai) struct ChatIdentity {
|
||||
pub id: String,
|
||||
pub created: u64,
|
||||
pub model: String,
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
struct Envelope<'a> {
|
||||
#[serde(flatten)]
|
||||
identity: &'a ChatIdentity,
|
||||
object: &'static str,
|
||||
choices: Value,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
usage: Option<Value>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
metadata: Option<&'a Value>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
service_tier: Option<&'a Value>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
moderation: Option<Value>,
|
||||
}
|
||||
|
||||
/// 输入已由现有 Responses collector 聚合,投影不再恢复或重建缺失的 output
|
||||
pub(in crate::openai) fn project_response(
|
||||
response: &Value,
|
||||
identity: &ChatIdentity,
|
||||
) -> Result<Value, ProtocolErrorBody> {
|
||||
let output = response
|
||||
.get("output")
|
||||
.and_then(Value::as_array)
|
||||
.ok_or_else(invalid_response)?;
|
||||
validate_output_identities(output)?;
|
||||
let mut content = None;
|
||||
let mut refusal = None;
|
||||
let mut tools = Vec::new();
|
||||
for item in output {
|
||||
match required_str(item, "type")? {
|
||||
"message" => {
|
||||
for part in message_parts(item)? {
|
||||
let (kind, text) = part_text(part)?;
|
||||
let target = match kind {
|
||||
PartKind::Text => &mut content,
|
||||
PartKind::Refusal => &mut refusal,
|
||||
};
|
||||
target.get_or_insert_with(String::new).push_str(text);
|
||||
}
|
||||
}
|
||||
"function_call" => {
|
||||
let (id, name, arguments) = function_fields(item)?;
|
||||
tools.push(json!({
|
||||
"id": id, "type": "function",
|
||||
"function": {"name": name, "arguments": arguments}
|
||||
}));
|
||||
}
|
||||
"reasoning" => {}
|
||||
_ => return Err(invalid_response()),
|
||||
}
|
||||
}
|
||||
let finish_reason = finish_reason(response, !tools.is_empty())?;
|
||||
let mut message = json!({"role": "assistant", "content": content, "refusal": refusal});
|
||||
if !tools.is_empty() {
|
||||
message["tool_calls"] = Value::Array(tools);
|
||||
}
|
||||
serde_json::to_value(Envelope {
|
||||
identity,
|
||||
object: "chat.completion",
|
||||
choices: json!([{"index": 0, "message": message, "finish_reason": finish_reason, "logprobs": null}]),
|
||||
usage: chat_usage(response),
|
||||
metadata: response.get("metadata"),
|
||||
service_tier: response.get("service_tier"),
|
||||
moderation: project_moderation(response)?,
|
||||
})
|
||||
.map_err(|_| invalid_response())
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, PartialEq, Eq)]
|
||||
enum PartKind {
|
||||
Text,
|
||||
Refusal,
|
||||
}
|
||||
|
||||
impl PartKind {
|
||||
const fn field(self) -> &'static str {
|
||||
match self {
|
||||
Self::Text => "content",
|
||||
Self::Refusal => "refusal",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct Part {
|
||||
kind: PartKind,
|
||||
text: String,
|
||||
}
|
||||
|
||||
struct Tool {
|
||||
index: usize,
|
||||
call_id: String,
|
||||
name: String,
|
||||
arguments: String,
|
||||
}
|
||||
|
||||
enum Output {
|
||||
Message(BTreeMap<u64, Part>),
|
||||
Function(Tool),
|
||||
Reasoning,
|
||||
}
|
||||
|
||||
struct OutputItem {
|
||||
id: Option<String>,
|
||||
output: Output,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, PartialEq, Eq)]
|
||||
enum Terminal {
|
||||
Completed,
|
||||
Failed,
|
||||
}
|
||||
|
||||
/// 只保存身份关联与快照去重所需内容;终态帧由外层受管 body 决定何时交付
|
||||
pub(in crate::openai) struct ChatStreamEncoder {
|
||||
identity: ChatIdentity,
|
||||
include_usage: bool,
|
||||
role_sent: bool,
|
||||
output: BTreeMap<u64, OutputItem>,
|
||||
terminal: Option<Terminal>,
|
||||
}
|
||||
|
||||
impl ChatStreamEncoder {
|
||||
pub(in crate::openai) fn new(identity: ChatIdentity, include_usage: bool) -> Self {
|
||||
Self {
|
||||
identity,
|
||||
include_usage,
|
||||
role_sent: false,
|
||||
output: BTreeMap::new(),
|
||||
terminal: None,
|
||||
}
|
||||
}
|
||||
|
||||
pub(in crate::openai) fn is_completed(&self) -> bool {
|
||||
self.terminal == Some(Terminal::Completed)
|
||||
}
|
||||
|
||||
pub(in crate::openai) fn has_wire_failure(&self) -> bool {
|
||||
self.terminal == Some(Terminal::Failed)
|
||||
}
|
||||
|
||||
pub(in crate::openai) fn push_event(
|
||||
&mut self,
|
||||
event_type: Option<&str>,
|
||||
data: &Value,
|
||||
) -> Result<Vec<Bytes>, ProtocolErrorBody> {
|
||||
let event_type = event_type.or_else(|| data.get("type").and_then(Value::as_str));
|
||||
if self.has_wire_failure() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
let mut frames = Vec::new();
|
||||
match event_type {
|
||||
Some("response.created") => {
|
||||
self.require_open()?;
|
||||
if !self.role_sent {
|
||||
frames.push(self.delta(json!({"role": "assistant"}))?);
|
||||
self.role_sent = true;
|
||||
}
|
||||
}
|
||||
Some("response.output_text.delta" | "response.refusal.delta") => {
|
||||
self.require_open()?;
|
||||
let kind = if event_type == Some("response.refusal.delta") {
|
||||
PartKind::Refusal
|
||||
} else {
|
||||
PartKind::Text
|
||||
};
|
||||
let delta = required_str(data, "delta")?;
|
||||
let part = self.event_part(data, kind)?;
|
||||
part.text.push_str(delta);
|
||||
if !delta.is_empty() {
|
||||
frames.push(self.delta(json!({kind.field(): delta}))?);
|
||||
}
|
||||
}
|
||||
Some("response.output_text.done" | "response.refusal.done") => {
|
||||
self.require_open()?;
|
||||
let (kind, field) = if event_type == Some("response.refusal.done") {
|
||||
(PartKind::Refusal, "refusal")
|
||||
} else {
|
||||
(PartKind::Text, "text")
|
||||
};
|
||||
let text = required_str(data, field)?;
|
||||
let part = self.event_part(data, kind)?;
|
||||
let suffix = append_snapshot(&mut part.text, text)?;
|
||||
if !suffix.is_empty() {
|
||||
frames.push(self.delta(json!({kind.field(): suffix}))?);
|
||||
}
|
||||
}
|
||||
Some("response.content_part.added" | "response.content_part.done") => {
|
||||
self.require_open()?;
|
||||
let part = data.get("part").ok_or_else(invalid_response)?;
|
||||
let (kind, text) = part_text(part)?;
|
||||
let part = self.event_part(data, kind)?;
|
||||
let suffix = append_snapshot(&mut part.text, text)?;
|
||||
if !suffix.is_empty() {
|
||||
frames.push(self.delta(json!({kind.field(): suffix}))?);
|
||||
}
|
||||
}
|
||||
Some("response.output_item.added" | "response.output_item.done") => {
|
||||
self.require_open()?;
|
||||
self.item_snapshot(
|
||||
required_index(data, "output_index")?,
|
||||
data.get("item").ok_or_else(invalid_response)?,
|
||||
false,
|
||||
&mut frames,
|
||||
)?;
|
||||
}
|
||||
Some(
|
||||
"response.function_call_arguments.delta" | "response.function_call_arguments.done",
|
||||
) => {
|
||||
self.require_open()?;
|
||||
let index = required_index(data, "output_index")?;
|
||||
let item = self.output.get_mut(&index).ok_or_else(invalid_response)?;
|
||||
bind_id(item, optional_id(data, "item_id")?)?;
|
||||
let Output::Function(tool) = &mut item.output else {
|
||||
return Err(invalid_response());
|
||||
};
|
||||
let delta = if event_type == Some("response.function_call_arguments.delta") {
|
||||
let delta = required_str(data, "delta")?;
|
||||
tool.arguments.push_str(delta);
|
||||
delta
|
||||
} else {
|
||||
append_snapshot(&mut tool.arguments, required_str(data, "arguments")?)?
|
||||
};
|
||||
let tool_index = tool.index;
|
||||
if !delta.is_empty() {
|
||||
frames.push(self.delta(json!({"tool_calls": [{"index": tool_index, "function": {"arguments": delta}}]}))?);
|
||||
}
|
||||
}
|
||||
Some("response.completed" | "response.incomplete") => {
|
||||
self.require_open()?;
|
||||
let response = data.get("response").ok_or_else(invalid_response)?;
|
||||
let status = required_str(response, "status")?;
|
||||
if (event_type == Some("response.completed") && status != "completed")
|
||||
|| (event_type == Some("response.incomplete") && status != "incomplete")
|
||||
{
|
||||
return Err(invalid_response());
|
||||
}
|
||||
if let Some(output) = response.get("output") {
|
||||
let output = output.as_array().ok_or_else(invalid_response)?;
|
||||
validate_output_identities(output)?;
|
||||
for (index, item) in output.iter().enumerate() {
|
||||
self.item_snapshot(index as u64, item, true, &mut frames)?;
|
||||
}
|
||||
} else if self.output.is_empty() {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
let finish_reason = finish_reason(response, self.tool_count() > 0)?;
|
||||
frames.push(self.chunk(
|
||||
json!([{"index": 0, "delta": {}, "finish_reason": finish_reason, "logprobs": null}]),
|
||||
self.include_usage.then_some(Value::Null),
|
||||
Some(response),
|
||||
)?);
|
||||
if self.include_usage
|
||||
&& let Some(usage) = chat_usage(response)
|
||||
{
|
||||
frames.push(self.chunk(json!([]), Some(usage), None)?);
|
||||
}
|
||||
self.terminal = Some(Terminal::Completed);
|
||||
}
|
||||
Some("response.failed" | "error") => {
|
||||
self.require_open()?;
|
||||
let error = data
|
||||
.pointer("/response/error")
|
||||
.or_else(|| data.get("error"))
|
||||
.unwrap_or(data);
|
||||
required_str(error, "message")?;
|
||||
// 错误恢复信号和扩展由既有出口拥有,Chat 仅转换外层包装
|
||||
let mut error = error.clone();
|
||||
if error
|
||||
.get("type")
|
||||
.and_then(Value::as_str)
|
||||
.is_none_or(|kind| kind == "error")
|
||||
{
|
||||
error["type"] = json!("server_error");
|
||||
}
|
||||
frames.push(sse_frame(&json!({"error": error})));
|
||||
self.terminal = Some(Terminal::Failed);
|
||||
}
|
||||
// 辅助事件可随上游演进;实际 output item 和 content part 仍逐项检查可表达性
|
||||
_ => {}
|
||||
}
|
||||
// done-only 流也要提供完整 Chat 消息身份,辅助事件或私有推理不触发角色帧
|
||||
if !self.role_sent && !frames.is_empty() && !self.has_wire_failure() {
|
||||
frames.insert(0, self.delta(json!({"role": "assistant"}))?);
|
||||
self.role_sent = true;
|
||||
}
|
||||
Ok(frames)
|
||||
}
|
||||
|
||||
fn require_open(&self) -> Result<(), ProtocolErrorBody> {
|
||||
if self.terminal.is_some() {
|
||||
Err(invalid_response())
|
||||
} else {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
fn tool_count(&self) -> usize {
|
||||
self.output
|
||||
.values()
|
||||
.filter(|item| matches!(item.output, Output::Function(_)))
|
||||
.count()
|
||||
}
|
||||
|
||||
fn event_part(&mut self, data: &Value, kind: PartKind) -> Result<&mut Part, ProtocolErrorBody> {
|
||||
let index = required_index(data, "output_index")?;
|
||||
let content_index = required_index(data, "content_index")?;
|
||||
let id = optional_id(data, "item_id")?;
|
||||
let item = self.output.entry(index).or_insert_with(|| OutputItem {
|
||||
id: None,
|
||||
output: Output::Message(BTreeMap::new()),
|
||||
});
|
||||
bind_id(item, id)?;
|
||||
message_part(item, content_index, kind)
|
||||
}
|
||||
|
||||
fn item_snapshot(
|
||||
&mut self,
|
||||
index: u64,
|
||||
snapshot: &Value,
|
||||
terminal: bool,
|
||||
frames: &mut Vec<Bytes>,
|
||||
) -> Result<(), ProtocolErrorBody> {
|
||||
let id = optional_id(snapshot, "id")?;
|
||||
let kind = required_str(snapshot, "type")?;
|
||||
let call_id = if kind == "function_call" {
|
||||
Some(required_nonempty_str(snapshot, "call_id")?)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
let known_index = self.find_item(id, call_id)?;
|
||||
let index = match known_index {
|
||||
Some(known) if terminal || known == index => known,
|
||||
Some(_) => return Err(invalid_response()),
|
||||
None => index,
|
||||
};
|
||||
if !self.output.contains_key(&index) {
|
||||
let output = match kind {
|
||||
"message" => Output::Message(BTreeMap::new()),
|
||||
"reasoning" => Output::Reasoning,
|
||||
"function_call" => {
|
||||
let (call_id, name, _) = function_fields(snapshot)?;
|
||||
let tool_index = self.tool_count();
|
||||
frames.push(self.delta(json!({"tool_calls": [{"index": tool_index, "id": call_id, "type": "function", "function": {"name": name, "arguments": ""}}]}))?);
|
||||
Output::Function(Tool {
|
||||
index: tool_index,
|
||||
call_id: call_id.to_owned(),
|
||||
name: name.to_owned(),
|
||||
arguments: String::new(),
|
||||
})
|
||||
}
|
||||
_ => return Err(invalid_response()),
|
||||
};
|
||||
self.output.insert(index, OutputItem { id: None, output });
|
||||
}
|
||||
let item = self.output.get_mut(&index).ok_or_else(invalid_response)?;
|
||||
bind_id(item, id)?;
|
||||
match (&mut item.output, kind) {
|
||||
(Output::Message(parts), "message") => {
|
||||
for (index, part) in message_parts(snapshot)?.iter().enumerate() {
|
||||
let (kind, text) = part_text(part)?;
|
||||
let part = part_entry(parts, index as u64, kind)?;
|
||||
let suffix = append_snapshot(&mut part.text, text)?;
|
||||
if !suffix.is_empty() {
|
||||
frames.push(delta_frame(
|
||||
&self.identity,
|
||||
self.include_usage,
|
||||
json!({kind.field(): suffix}),
|
||||
)?);
|
||||
}
|
||||
}
|
||||
}
|
||||
(Output::Function(tool), "function_call") => {
|
||||
let (id, name, arguments) = function_fields(snapshot)?;
|
||||
if tool.call_id != id || tool.name != name {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
let suffix = append_snapshot(&mut tool.arguments, arguments)?;
|
||||
if !suffix.is_empty() {
|
||||
frames.push(delta_frame(&self.identity, self.include_usage, json!({"tool_calls": [{"index": tool.index, "function": {"arguments": suffix}}]}))?);
|
||||
}
|
||||
}
|
||||
(Output::Reasoning, "reasoning") => {}
|
||||
_ => return Err(invalid_response()),
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn find_item(
|
||||
&self,
|
||||
id: Option<&str>,
|
||||
call_id: Option<&str>,
|
||||
) -> Result<Option<u64>, ProtocolErrorBody> {
|
||||
let mut found = None;
|
||||
for (&index, item) in &self.output {
|
||||
let same_id = id.is_some() && item.id.as_deref() == id;
|
||||
let same_call = matches!(&item.output, Output::Function(tool) if call_id == Some(tool.call_id.as_str()));
|
||||
if same_id || same_call {
|
||||
if found.is_some_and(|previous| previous != index) {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
found = Some(index);
|
||||
}
|
||||
}
|
||||
Ok(found)
|
||||
}
|
||||
|
||||
fn delta(&self, delta: Value) -> Result<Bytes, ProtocolErrorBody> {
|
||||
delta_frame(&self.identity, self.include_usage, delta)
|
||||
}
|
||||
|
||||
fn chunk(
|
||||
&self,
|
||||
choices: Value,
|
||||
usage: Option<Value>,
|
||||
response: Option<&Value>,
|
||||
) -> Result<Bytes, ProtocolErrorBody> {
|
||||
chunk_frame(&self.identity, choices, usage, response)
|
||||
}
|
||||
}
|
||||
|
||||
fn message_part(
|
||||
item: &mut OutputItem,
|
||||
index: u64,
|
||||
kind: PartKind,
|
||||
) -> Result<&mut Part, ProtocolErrorBody> {
|
||||
let Output::Message(parts) = &mut item.output else {
|
||||
return Err(invalid_response());
|
||||
};
|
||||
part_entry(parts, index, kind)
|
||||
}
|
||||
|
||||
fn part_entry(
|
||||
parts: &mut BTreeMap<u64, Part>,
|
||||
index: u64,
|
||||
kind: PartKind,
|
||||
) -> Result<&mut Part, ProtocolErrorBody> {
|
||||
let part = parts.entry(index).or_insert_with(|| Part {
|
||||
kind,
|
||||
text: String::new(),
|
||||
});
|
||||
if part.kind != kind {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
Ok(part)
|
||||
}
|
||||
|
||||
fn bind_id(item: &mut OutputItem, id: Option<&str>) -> Result<(), ProtocolErrorBody> {
|
||||
if let Some(id) = id {
|
||||
if item.id.as_deref().is_some_and(|previous| previous != id) {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
if item.id.is_none() {
|
||||
item.id = Some(id.to_owned());
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn append_snapshot<'a>(text: &mut String, snapshot: &'a str) -> Result<&'a str, ProtocolErrorBody> {
|
||||
let suffix = snapshot
|
||||
.strip_prefix(text.as_str())
|
||||
.ok_or_else(invalid_response)?;
|
||||
text.push_str(suffix);
|
||||
Ok(suffix)
|
||||
}
|
||||
|
||||
fn message_parts(item: &Value) -> Result<&[Value], ProtocolErrorBody> {
|
||||
if required_str(item, "role")? != "assistant" {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
item.get("content")
|
||||
.and_then(Value::as_array)
|
||||
.map(Vec::as_slice)
|
||||
.ok_or_else(invalid_response)
|
||||
}
|
||||
|
||||
fn part_text(part: &Value) -> Result<(PartKind, &str), ProtocolErrorBody> {
|
||||
match required_str(part, "type")? {
|
||||
"output_text" => Ok((PartKind::Text, required_str(part, "text")?)),
|
||||
"refusal" => Ok((PartKind::Refusal, required_str(part, "refusal")?)),
|
||||
_ => Err(invalid_response()),
|
||||
}
|
||||
}
|
||||
|
||||
fn function_fields(item: &Value) -> Result<(&str, &str, &str), ProtocolErrorBody> {
|
||||
Ok((
|
||||
required_nonempty_str(item, "call_id")?,
|
||||
required_nonempty_str(item, "name")?,
|
||||
required_str(item, "arguments")?,
|
||||
))
|
||||
}
|
||||
|
||||
fn validate_output_identities(output: &[Value]) -> Result<(), ProtocolErrorBody> {
|
||||
let mut item_ids = BTreeSet::new();
|
||||
let mut call_ids = BTreeSet::new();
|
||||
for item in output {
|
||||
if let Some(id) = optional_id(item, "id")?
|
||||
&& !item_ids.insert(id)
|
||||
{
|
||||
return Err(invalid_response());
|
||||
}
|
||||
if item.get("type").and_then(Value::as_str) == Some("function_call")
|
||||
&& !call_ids.insert(required_nonempty_str(item, "call_id")?)
|
||||
{
|
||||
return Err(invalid_response());
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn finish_reason(response: &Value, has_tools: bool) -> Result<&'static str, ProtocolErrorBody> {
|
||||
if response.get("error").is_some_and(|value| !value.is_null()) {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
match required_str(response, "status")? {
|
||||
"completed" => Ok(if has_tools { "tool_calls" } else { "stop" }),
|
||||
"incomplete" => match response
|
||||
.pointer("/incomplete_details/reason")
|
||||
.and_then(Value::as_str)
|
||||
{
|
||||
Some("max_output_tokens") => Ok("length"),
|
||||
Some("content_filter") => Ok("content_filter"),
|
||||
_ => Err(invalid_response()),
|
||||
},
|
||||
_ => Err(invalid_response()),
|
||||
}
|
||||
}
|
||||
|
||||
fn chat_usage(response: &Value) -> Option<Value> {
|
||||
let usage = extract_usage(response)?;
|
||||
if !billable_usage_is_complete(response, usage) {
|
||||
return None;
|
||||
}
|
||||
let mut result = json!({"prompt_tokens": usage.input_tokens, "completion_tokens": usage.output_tokens, "total_tokens": usage.total_tokens});
|
||||
if response
|
||||
.pointer("/usage/input_tokens_details/cached_tokens")
|
||||
.is_some()
|
||||
{
|
||||
result["prompt_tokens_details"] = json!({"cached_tokens": usage.cached_input_tokens});
|
||||
}
|
||||
if response
|
||||
.pointer("/usage/output_tokens_details/reasoning_tokens")
|
||||
.and_then(Value::as_u64)
|
||||
.is_some_and(|tokens| tokens <= usage.output_tokens)
|
||||
{
|
||||
result["completion_tokens_details"] =
|
||||
json!({"reasoning_tokens": usage.reasoning_output_tokens});
|
||||
}
|
||||
Some(result)
|
||||
}
|
||||
|
||||
fn project_moderation(response: &Value) -> Result<Option<Value>, ProtocolErrorBody> {
|
||||
let Some(moderation) = response.get("moderation").filter(|value| !value.is_null()) else {
|
||||
return Ok(None);
|
||||
};
|
||||
let mut result = json!({});
|
||||
for side in ["input", "output"] {
|
||||
let value = moderation.get(side).ok_or_else(invalid_response)?;
|
||||
result[side] = match required_str(value, "type")? {
|
||||
"moderation_result" => {
|
||||
json!({"type": "moderation_results", "model": required_str(value, "model")?, "results": [value]})
|
||||
}
|
||||
"error" => value.clone(),
|
||||
_ => return Err(invalid_response()),
|
||||
};
|
||||
}
|
||||
Ok(Some(result))
|
||||
}
|
||||
|
||||
fn delta_frame(
|
||||
identity: &ChatIdentity,
|
||||
include_usage: bool,
|
||||
delta: Value,
|
||||
) -> Result<Bytes, ProtocolErrorBody> {
|
||||
chunk_frame(
|
||||
identity,
|
||||
json!([{"index": 0, "delta": delta, "finish_reason": null, "logprobs": null}]),
|
||||
include_usage.then_some(Value::Null),
|
||||
None,
|
||||
)
|
||||
}
|
||||
|
||||
fn chunk_frame(
|
||||
identity: &ChatIdentity,
|
||||
choices: Value,
|
||||
usage: Option<Value>,
|
||||
response: Option<&Value>,
|
||||
) -> Result<Bytes, ProtocolErrorBody> {
|
||||
let value = serde_json::to_value(Envelope {
|
||||
identity,
|
||||
object: "chat.completion.chunk",
|
||||
choices,
|
||||
usage,
|
||||
metadata: None,
|
||||
service_tier: response.and_then(|value| value.get("service_tier")),
|
||||
moderation: response.map(project_moderation).transpose()?.flatten(),
|
||||
})
|
||||
.map_err(|_| invalid_response())?;
|
||||
Ok(sse_frame(&value))
|
||||
}
|
||||
|
||||
fn sse_frame(value: &Value) -> Bytes {
|
||||
Bytes::from(encode_sse_event("", &value.to_string()))
|
||||
}
|
||||
|
||||
fn required_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, ProtocolErrorBody> {
|
||||
value
|
||||
.get(field)
|
||||
.and_then(Value::as_str)
|
||||
.ok_or_else(invalid_response)
|
||||
}
|
||||
|
||||
fn required_nonempty_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, ProtocolErrorBody> {
|
||||
let value = required_str(value, field)?;
|
||||
if value.is_empty() {
|
||||
return Err(invalid_response());
|
||||
}
|
||||
Ok(value)
|
||||
}
|
||||
|
||||
fn optional_id<'a>(value: &'a Value, field: &str) -> Result<Option<&'a str>, ProtocolErrorBody> {
|
||||
value
|
||||
.get(field)
|
||||
.map(|_| required_nonempty_str(value, field))
|
||||
.transpose()
|
||||
}
|
||||
|
||||
fn required_index(value: &Value, field: &str) -> Result<u64, ProtocolErrorBody> {
|
||||
value
|
||||
.get(field)
|
||||
.and_then(Value::as_u64)
|
||||
.ok_or_else(invalid_response)
|
||||
}
|
||||
|
||||
fn invalid_response() -> ProtocolErrorBody {
|
||||
ProtocolErrorBody {
|
||||
error: ProtocolError {
|
||||
kind: "server_error",
|
||||
code: "invalid_upstream_response",
|
||||
message: "The upstream response cannot be represented as a Chat completion.".to_owned(),
|
||||
param: None,
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -33,7 +33,8 @@ pub fn openai_error_response(
|
||||
}
|
||||
}
|
||||
|
||||
/// Codex 将这两个容量码视为不可重试;仅在交付边界转换,内部仍保留真实上游事实
|
||||
/// 无重试提示的容量拒绝及旧版 Codex 限流错误需要可重试投影
|
||||
/// 仅在交付边界转换,内部仍保留真实上游事实
|
||||
pub(super) fn client_error_code(code: &str) -> &str {
|
||||
match code {
|
||||
"server_is_overloaded" | "slow_down" => "server_error",
|
||||
@@ -83,7 +84,7 @@ pub fn protocol_error_response(status: StatusCode, body: ProtocolErrorBody) -> R
|
||||
pub fn missing_client_api_key_response() -> (StatusCode, Json<Value>) {
|
||||
openai_error_response(
|
||||
StatusCode::UNAUTHORIZED,
|
||||
"Missing or invalid API key",
|
||||
"Missing or invalid API key.",
|
||||
"invalid_request_error",
|
||||
"invalid_api_key",
|
||||
)
|
||||
@@ -93,7 +94,7 @@ pub fn missing_client_api_key_response() -> (StatusCode, Json<Value>) {
|
||||
pub fn runtime_unavailable_response() -> (StatusCode, Json<Value>) {
|
||||
openai_error_response(
|
||||
StatusCode::SERVICE_UNAVAILABLE,
|
||||
"Gateway runtime configuration is temporarily unavailable",
|
||||
"Gateway configuration is temporarily unavailable.",
|
||||
"server_error",
|
||||
"runtime_configuration_unavailable",
|
||||
)
|
||||
@@ -110,7 +111,7 @@ pub fn client_version_rejection_response(
|
||||
let (message, code, current_version) = match rejection.current() {
|
||||
Some(current) => (
|
||||
format!(
|
||||
"{client_label} {current} is below the minimum required version {}. Upgrade {client_label} and retry.",
|
||||
"{client_label} {current} is outdated. Upgrade to {} or later.",
|
||||
rejection.min()
|
||||
),
|
||||
"client_version_too_old",
|
||||
@@ -118,15 +119,15 @@ pub fn client_version_rejection_response(
|
||||
),
|
||||
None => (
|
||||
format!(
|
||||
"A valid {client_label} version is required. Upgrade to version {} or newer and retry.",
|
||||
rejection.min()
|
||||
"Cannot read the {client_label} version. Check the User-Agent and version headers."
|
||||
),
|
||||
"client_version_unavailable",
|
||||
None,
|
||||
),
|
||||
};
|
||||
// Codex 将 400 视为不可重试的请求错误,426 则会触发 WS 回退和 HTTP 重试
|
||||
(
|
||||
StatusCode::UPGRADE_REQUIRED,
|
||||
StatusCode::BAD_REQUEST,
|
||||
Json(json!({
|
||||
"error": {
|
||||
"message": message,
|
||||
@@ -144,7 +145,7 @@ pub fn client_version_rejection_response(
|
||||
pub fn model_not_found_response() -> (StatusCode, Json<Value>) {
|
||||
openai_error_response(
|
||||
StatusCode::NOT_FOUND,
|
||||
"The requested model does not exist or is not available",
|
||||
"The requested model does not exist or is not available.",
|
||||
"invalid_request_error",
|
||||
"model_not_found",
|
||||
)
|
||||
|
||||
@@ -124,7 +124,7 @@ async fn image_operation(
|
||||
.map_err(|_| {
|
||||
GatewayError::new(
|
||||
GatewayErrorKind::Internal,
|
||||
"OpenAI protocol identifier is invalid",
|
||||
"OpenAI protocol identifier is invalid.",
|
||||
)
|
||||
})?
|
||||
.with_context(context);
|
||||
|
||||
@@ -26,7 +26,7 @@ pub(super) async fn normalize_edit_body(
|
||||
return Ok(body);
|
||||
}
|
||||
if headers.get_all(CONTENT_TYPE).iter().count() != 1 {
|
||||
return Err(invalid("Image edit requires a single Content-Type header"));
|
||||
return Err(invalid("Image edit requires a single Content-Type header."));
|
||||
}
|
||||
|
||||
let mut request = Request::new(Body::from(body));
|
||||
@@ -35,7 +35,7 @@ pub(super) async fn normalize_edit_body(
|
||||
DefaultBodyLimit::disable().apply(&mut request);
|
||||
let mut multipart = Multipart::from_request(request, &())
|
||||
.await
|
||||
.map_err(|_| invalid("Image edit multipart boundary is invalid"))?;
|
||||
.map_err(|_| invalid("Image edit multipart boundary is invalid."))?;
|
||||
let mut body = Vec::new();
|
||||
let mut serializer = serde_json::Serializer::new(&mut body);
|
||||
// 直接序列化字段,避免经过 Map 后覆盖同名参数
|
||||
@@ -44,17 +44,17 @@ pub(super) async fn normalize_edit_body(
|
||||
while let Some(field) = multipart
|
||||
.next_field()
|
||||
.await
|
||||
.map_err(|_| invalid("Image edit multipart body is invalid"))?
|
||||
.map_err(|_| invalid("Image edit multipart body is invalid."))?
|
||||
{
|
||||
let name = field
|
||||
.name()
|
||||
.ok_or_else(|| invalid("Image edit multipart fields must have names"))?
|
||||
.ok_or_else(|| invalid("Image edit multipart fields must have names."))?
|
||||
.to_owned();
|
||||
let is_file = field.file_name().is_some();
|
||||
let bytes = field
|
||||
.bytes()
|
||||
.await
|
||||
.map_err(|_| invalid("Image edit multipart field is invalid"))?;
|
||||
.map_err(|_| invalid("Image edit multipart field is invalid."))?;
|
||||
if matches!(name.as_str(), "image" | "image[]") {
|
||||
// 按表单中的出现顺序追加,多图顺序参与编辑语义
|
||||
images.push(image_reference(&bytes)?);
|
||||
@@ -65,7 +65,7 @@ pub(super) async fn normalize_edit_body(
|
||||
} else {
|
||||
if is_file {
|
||||
return Err(invalid(
|
||||
"Only image, image[] and mask may contain image files",
|
||||
"Only image, image[] and mask may contain image files.",
|
||||
));
|
||||
}
|
||||
scalar_field(&name, &bytes)?
|
||||
@@ -76,7 +76,7 @@ pub(super) async fn normalize_edit_body(
|
||||
}
|
||||
if images.is_empty() {
|
||||
return Err(invalid(
|
||||
"Image edit multipart requires an image or image[] upload",
|
||||
"Image edit multipart requires an image or image[] upload.",
|
||||
));
|
||||
}
|
||||
fields
|
||||
@@ -92,14 +92,14 @@ fn scalar_field(name: &str, bytes: &[u8]) -> Result<Value, GatewayError> {
|
||||
match name {
|
||||
"n" | "output_compression" | "partial_images" => serde_json::from_slice::<Number>(bytes)
|
||||
.map(Value::Number)
|
||||
.map_err(|_| invalid("Image edit numeric fields must contain JSON numbers")),
|
||||
.map_err(|_| invalid("Image edit numeric fields must contain JSON numbers.")),
|
||||
"stream" => serde_json::from_slice::<bool>(bytes)
|
||||
.map(Value::Bool)
|
||||
.map_err(|_| invalid("Image edit stream field must contain a JSON boolean")),
|
||||
.map_err(|_| invalid("Image edit stream field must contain a JSON boolean.")),
|
||||
// 其余表单字段保留名称与文本值,不因字段尚未识别而丢弃或拒绝
|
||||
_ => {
|
||||
let text = std::str::from_utf8(bytes)
|
||||
.map_err(|_| invalid("Image edit text fields must be valid UTF-8"))?;
|
||||
.map_err(|_| invalid("Image edit text fields must be valid UTF-8."))?;
|
||||
Ok(Value::String(text.to_owned()))
|
||||
}
|
||||
}
|
||||
@@ -114,7 +114,7 @@ fn image_reference(bytes: &[u8]) -> Result<Value, GatewayError> {
|
||||
} else if bytes.starts_with(b"RIFF") && bytes.get(8..12) == Some(b"WEBP") {
|
||||
"image/webp"
|
||||
} else {
|
||||
return Err(invalid("Image edit uploads must be PNG, JPEG or WebP"));
|
||||
return Err(invalid("Image edit uploads must be PNG, JPEG or WebP."));
|
||||
};
|
||||
Ok(json!({ "image_url": format!("data:{mime};base64,{}", STANDARD.encode(bytes)) }))
|
||||
}
|
||||
@@ -126,6 +126,6 @@ fn invalid(message: &'static str) -> GatewayError {
|
||||
fn encode_error(_: serde_json::Error) -> GatewayError {
|
||||
GatewayError::new(
|
||||
GatewayErrorKind::Internal,
|
||||
"Could not encode image edit JSON",
|
||||
"Could not encode image edit JSON.",
|
||||
)
|
||||
}
|
||||
|
||||
@@ -70,7 +70,7 @@ pub(crate) async fn live_call(
|
||||
return live_error_response(
|
||||
LiveErrorShape::Live,
|
||||
StatusCode::BAD_REQUEST,
|
||||
"Codex live call request has an invalid model",
|
||||
"Codex live call request has an invalid model.",
|
||||
"invalid_request",
|
||||
);
|
||||
}
|
||||
@@ -184,7 +184,7 @@ async fn read_call_body(
|
||||
Box::new(live_error_response(
|
||||
LiveErrorShape::Live,
|
||||
StatusCode::PAYLOAD_TOO_LARGE,
|
||||
"Codex live request body too large",
|
||||
"Codex live request body too large.",
|
||||
"invalid_request",
|
||||
))
|
||||
})?;
|
||||
@@ -194,7 +194,7 @@ async fn read_call_body(
|
||||
Box::new(live_error_response(
|
||||
LiveErrorShape::Live,
|
||||
StatusCode::BAD_REQUEST,
|
||||
"Codex live multipart body is invalid",
|
||||
"Codex live multipart body is invalid.",
|
||||
"invalid_request",
|
||||
))
|
||||
})?;
|
||||
@@ -205,7 +205,7 @@ async fn read_call_body(
|
||||
Box::new(live_error_response(
|
||||
LiveErrorShape::Live,
|
||||
StatusCode::BAD_REQUEST,
|
||||
"Codex live multipart body is invalid",
|
||||
"Codex live multipart body is invalid.",
|
||||
"invalid_request",
|
||||
))
|
||||
})?
|
||||
@@ -218,7 +218,7 @@ async fn read_call_body(
|
||||
Box::new(live_error_response(
|
||||
LiveErrorShape::Live,
|
||||
StatusCode::BAD_REQUEST,
|
||||
"Codex live multipart body is invalid",
|
||||
"Codex live multipart body is invalid.",
|
||||
"invalid_request",
|
||||
))
|
||||
})?;
|
||||
@@ -229,7 +229,7 @@ async fn read_call_body(
|
||||
Box::new(live_error_response(
|
||||
LiveErrorShape::Live,
|
||||
StatusCode::BAD_REQUEST,
|
||||
"Codex live multipart body is invalid",
|
||||
"Codex live multipart body is invalid.",
|
||||
"invalid_request",
|
||||
))
|
||||
})?;
|
||||
@@ -237,7 +237,7 @@ async fn read_call_body(
|
||||
Box::new(live_error_response(
|
||||
LiveErrorShape::Live,
|
||||
StatusCode::BAD_REQUEST,
|
||||
"Codex live session field must contain valid JSON",
|
||||
"Codex live session field must contain valid JSON.",
|
||||
"invalid_request",
|
||||
))
|
||||
})?;
|
||||
@@ -251,7 +251,7 @@ async fn read_call_body(
|
||||
None => Err(Box::new(live_error_response(
|
||||
LiveErrorShape::Live,
|
||||
StatusCode::BAD_REQUEST,
|
||||
"Codex live multipart body requires an sdp field",
|
||||
"Codex live multipart body requires an sdp field.",
|
||||
"invalid_request",
|
||||
))),
|
||||
}
|
||||
|
||||
@@ -121,7 +121,7 @@ async fn sip_unsupported() -> Response {
|
||||
pub(crate) fn realtime_unsupported_response(capability: &str) -> Response {
|
||||
openai_error_response(
|
||||
StatusCode::NOT_IMPLEMENTED,
|
||||
&format!("{capability} are not supported by the Codex OAuth upstream"),
|
||||
&format!("Unsupported by Codex OAuth: {capability}."),
|
||||
"not_supported_error",
|
||||
"realtime_capability_not_supported",
|
||||
)
|
||||
@@ -168,7 +168,7 @@ pub(crate) fn live_call_operation(
|
||||
Box::new(live_error_response(
|
||||
LiveErrorShape::Live,
|
||||
StatusCode::BAD_REQUEST,
|
||||
"Codex live call request is invalid",
|
||||
"Codex live call request is invalid.",
|
||||
"invalid_request",
|
||||
))
|
||||
};
|
||||
@@ -251,7 +251,7 @@ pub(crate) fn encode_call_request(
|
||||
))
|
||||
};
|
||||
if sdp.trim().is_empty() {
|
||||
return Err(invalid("Codex live call request requires an SDP offer"));
|
||||
return Err(invalid("Codex live call request requires an SDP offer."));
|
||||
}
|
||||
let mut payload = serde_json::Map::new();
|
||||
payload.insert("sdp".to_owned(), Value::String(sdp.to_owned()));
|
||||
@@ -260,7 +260,7 @@ pub(crate) fn encode_call_request(
|
||||
}
|
||||
serde_json::to_vec(&payload)
|
||||
.map(Bytes::from)
|
||||
.map_err(|_| invalid("Codex live call request is invalid"))
|
||||
.map_err(|_| invalid("Codex live call request is invalid."))
|
||||
}
|
||||
|
||||
/// 共用响应身份与传输边界;Live 原始正文另保留上游 Content-Type
|
||||
@@ -310,7 +310,7 @@ pub(crate) fn normalize_call_request(
|
||||
Box::new(live_error_response(
|
||||
LiveErrorShape::Live,
|
||||
StatusCode::BAD_REQUEST,
|
||||
"Codex live call request requires a UTF-8 SDP offer",
|
||||
"Codex live call request requires a UTF-8 SDP offer.",
|
||||
"invalid_request",
|
||||
))
|
||||
})?;
|
||||
|
||||
@@ -149,7 +149,7 @@ fn upgrade_required_response() -> Response {
|
||||
let mut response = live_error_response(
|
||||
LiveErrorShape::Realtime,
|
||||
StatusCode::UPGRADE_REQUIRED,
|
||||
"WebSocket upgrade required",
|
||||
"WebSocket upgrade required.",
|
||||
"websocket_upgrade_required",
|
||||
);
|
||||
response.headers_mut().insert(
|
||||
|
||||
@@ -246,7 +246,7 @@ pub(crate) fn error_response(error: MiddlewareError) -> Response {
|
||||
MiddlewareError::Rejected | MiddlewareError::Remote { rejected: true, .. } => {
|
||||
openai_error_response(
|
||||
StatusCode::FORBIDDEN,
|
||||
"Request rejected by middleware",
|
||||
"Request rejected by middleware.",
|
||||
"invalid_request_error",
|
||||
"middleware_rejected",
|
||||
)
|
||||
@@ -255,7 +255,7 @@ pub(crate) fn error_response(error: MiddlewareError) -> Response {
|
||||
MiddlewareError::Fault | MiddlewareError::Remote { .. } | MiddlewareError::InvalidState => {
|
||||
openai_error_response(
|
||||
StatusCode::BAD_GATEWAY,
|
||||
"Request middleware failed",
|
||||
"Request middleware failed.",
|
||||
"server_error",
|
||||
"middleware_failed",
|
||||
)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
//! OpenAI 客户端协议 adapter
|
||||
|
||||
pub mod auth;
|
||||
mod chat_completions;
|
||||
mod endpoint;
|
||||
pub mod error;
|
||||
pub mod images;
|
||||
|
||||
@@ -72,7 +72,7 @@ async fn models_response(
|
||||
{
|
||||
return openai_error_response(
|
||||
StatusCode::BAD_REQUEST,
|
||||
"Invalid client_version",
|
||||
"Invalid client_version.",
|
||||
"invalid_request_error",
|
||||
"invalid_client_version",
|
||||
)
|
||||
@@ -134,7 +134,7 @@ async fn models_response(
|
||||
fn catalog_unavailable_response() -> Response {
|
||||
openai_error_response(
|
||||
StatusCode::SERVICE_UNAVAILABLE,
|
||||
"Model catalog is temporarily unavailable",
|
||||
"Model catalog is temporarily unavailable.",
|
||||
"server_error",
|
||||
"model_catalog_unavailable",
|
||||
)
|
||||
|
||||
@@ -31,6 +31,7 @@ use gateway_core::event::{ProviderEvent, ProviderResponseHeader};
|
||||
use gateway_core::lifecycle::{CancellationToken, ConnectionGuard};
|
||||
use gateway_core::operation::OperationKind;
|
||||
use gateway_protocol::openai::sse::{DONE_SSE_FRAME, response_failed_sse_event_with_id};
|
||||
use serde_json::Value;
|
||||
use tokio::time::Instant;
|
||||
|
||||
use crate::ApiState;
|
||||
@@ -120,6 +121,23 @@ pub(crate) async fn execute_prepared_responses(
|
||||
prepared: PreparedRootExecution,
|
||||
request: ResponsesHttpRequest,
|
||||
authorization: Option<Arc<dyn ResponseAuthorization>>,
|
||||
) -> Response {
|
||||
execute_prepared_response(service, prepared, request, authorization, None).await
|
||||
}
|
||||
|
||||
/// 下游协议只投影出口字节 执行会话和计量仍归原 owner
|
||||
pub(in crate::openai) struct HttpResponseAdapter {
|
||||
pub endpoint: &'static str,
|
||||
pub wrap: Box<dyn FnOnce(MiddlewareResponse) -> MiddlewareResponse + Send>,
|
||||
pub stream_error: fn(&Value) -> Bytes,
|
||||
}
|
||||
|
||||
pub(in crate::openai) async fn execute_prepared_response(
|
||||
service: OpenAiService,
|
||||
prepared: PreparedRootExecution,
|
||||
request: ResponsesHttpRequest,
|
||||
authorization: Option<Arc<dyn ResponseAuthorization>>,
|
||||
adapter: Option<HttpResponseAdapter>,
|
||||
) -> Response {
|
||||
let ResponsesHttpRequest {
|
||||
peer_address,
|
||||
@@ -148,8 +166,14 @@ pub(crate) async fn execute_prepared_responses(
|
||||
// 中间件正文已经由入口按上限解压;传输编码和旧长度不能与改写后的正文混用
|
||||
middleware_headers.remove(CONTENT_ENCODING);
|
||||
middleware_headers.remove(CONTENT_LENGTH);
|
||||
let endpoint = adapter
|
||||
.as_ref()
|
||||
.map_or(crate::openai::router::RESPONSES_PATH, |adapter| {
|
||||
adapter.endpoint
|
||||
});
|
||||
let stream_error = adapter.as_ref().map(|adapter| adapter.stream_error);
|
||||
let input = RequestInput {
|
||||
endpoint: crate::openai::router::RESPONSES_PATH.to_owned(),
|
||||
endpoint: endpoint.to_owned(),
|
||||
protocol: OPENAI_PROTOCOL.to_owned(),
|
||||
operation: Some(OperationKind::Generate),
|
||||
transport,
|
||||
@@ -194,12 +218,7 @@ pub(crate) async fn execute_prepared_responses(
|
||||
None
|
||||
};
|
||||
let started = terminal_service
|
||||
.start_prepared_response(
|
||||
prepared,
|
||||
decoded,
|
||||
transport,
|
||||
crate::openai::router::RESPONSES_PATH,
|
||||
)
|
||||
.start_prepared_response(prepared, decoded, transport, endpoint)
|
||||
.await
|
||||
.map_err(MiddlewareError::Gateway)?;
|
||||
let _ = terminal_request_id.set(started.request_id.clone());
|
||||
@@ -242,17 +261,18 @@ pub(crate) async fn execute_prepared_responses(
|
||||
} else {
|
||||
response
|
||||
};
|
||||
into_http_response(
|
||||
validated_middleware_response(response, validation, stream),
|
||||
expected,
|
||||
)
|
||||
.await
|
||||
let response = validated_middleware_response(response, validation, stream);
|
||||
let response = match adapter {
|
||||
Some(adapter) => (adapter.wrap)(response),
|
||||
None => response,
|
||||
};
|
||||
into_http_response(response, expected).await
|
||||
}
|
||||
Err(error) => error_response(error),
|
||||
}
|
||||
};
|
||||
let response = if stream {
|
||||
super::keepalive::response(response, keepalive).await
|
||||
super::keepalive::response_with_encoder(response, keepalive, stream_error).await
|
||||
} else {
|
||||
response.await
|
||||
};
|
||||
@@ -964,7 +984,7 @@ impl MiddlewareBody for StreamingExecutionBody {
|
||||
MiddlewareError::Engine(error) => gateway_error_from_engine(&error),
|
||||
_ => GatewayError::new(
|
||||
GatewayErrorKind::Internal,
|
||||
"gateway response stream could not be delivered",
|
||||
"Gateway response stream could not be delivered.",
|
||||
),
|
||||
};
|
||||
self.push_stream_error(error);
|
||||
@@ -1048,6 +1068,6 @@ fn done_frame() -> MiddlewareFrame {
|
||||
fn invalid_execution_stream() -> MiddlewareError {
|
||||
MiddlewareError::Gateway(GatewayError::new(
|
||||
GatewayErrorKind::Internal,
|
||||
"gateway response stream violated its execution lifecycle",
|
||||
"Gateway response stream violated its execution lifecycle.",
|
||||
))
|
||||
}
|
||||
|
||||
@@ -21,6 +21,14 @@ use crate::openai::middleware::{SSE_KEEPALIVE_FRAME, SSE_KEEPALIVE_INTERVAL};
|
||||
pub(super) async fn response(
|
||||
response: impl Future<Output = Response> + Send + 'static,
|
||||
admitted: oneshot::Receiver<ResponseCommit>,
|
||||
) -> Response {
|
||||
response_with_encoder(response, admitted, None).await
|
||||
}
|
||||
|
||||
pub(super) async fn response_with_encoder(
|
||||
response: impl Future<Output = Response> + Send + 'static,
|
||||
admitted: oneshot::Receiver<ResponseCommit>,
|
||||
error_encoder: Option<fn(&Value) -> Bytes>,
|
||||
) -> Response {
|
||||
let mut response = Box::pin(response);
|
||||
// 鉴权、模型门禁和准入失败仍保留原 HTTP 状态,不提前建立 SSE
|
||||
@@ -40,7 +48,7 @@ pub(super) async fn response(
|
||||
if !commit.record(Some(StatusCode::OK.as_u16())) {
|
||||
return response.await;
|
||||
}
|
||||
let pending = PendingResponse::Waiting(response);
|
||||
let pending = PendingResponse::Waiting(response, error_encoder);
|
||||
let stream = stream::once(async { Ok::<_, axum::Error>(SSE_KEEPALIVE_FRAME) })
|
||||
.chain(stream::try_unfold(pending, PendingResponse::next));
|
||||
let mut response = Response::new(Body::from_stream(stream));
|
||||
@@ -58,7 +66,7 @@ pub(super) async fn response(
|
||||
}
|
||||
|
||||
enum PendingResponse {
|
||||
Waiting(BoxFuture<'static, Response>),
|
||||
Waiting(BoxFuture<'static, Response>, Option<fn(&Value) -> Bytes>),
|
||||
Streaming(BodyDataStream),
|
||||
}
|
||||
|
||||
@@ -66,7 +74,7 @@ impl PendingResponse {
|
||||
async fn next(mut self) -> Result<Option<(Bytes, Self)>, axum::Error> {
|
||||
loop {
|
||||
match &mut self {
|
||||
Self::Waiting(response) => {
|
||||
Self::Waiting(response, error_encoder) => {
|
||||
// 心跳只轮询同一个 future,不取消重建首帧等待或插件调用
|
||||
let response = tokio::select! {
|
||||
biased;
|
||||
@@ -85,7 +93,7 @@ impl PendingResponse {
|
||||
let body = if sse {
|
||||
response.into_body()
|
||||
} else {
|
||||
stream_failure(response).await
|
||||
stream_failure(response, *error_encoder).await
|
||||
};
|
||||
// 正式响应的正文已经带有交付期心跳,后续只转交,不叠加定时器
|
||||
self = Self::Streaming(body.into_data_stream());
|
||||
@@ -102,7 +110,7 @@ impl PendingResponse {
|
||||
}
|
||||
}
|
||||
|
||||
async fn stream_failure(response: Response) -> Body {
|
||||
async fn stream_failure(response: Response, error_encoder: Option<fn(&Value) -> Bytes>) -> Body {
|
||||
let status = response.status();
|
||||
let retry_after = response
|
||||
.headers()
|
||||
@@ -129,7 +137,7 @@ async fn stream_failure(response: Response) -> Body {
|
||||
raw_message.unwrap_or_else(|| {
|
||||
status
|
||||
.canonical_reason()
|
||||
.unwrap_or("Response stream failed")
|
||||
.unwrap_or("Response stream failed.")
|
||||
}),
|
||||
)
|
||||
});
|
||||
@@ -145,7 +153,10 @@ async fn stream_failure(response: Response) -> Body {
|
||||
.entry("retry-after")
|
||||
.or_insert(Value::String(retry_after));
|
||||
}
|
||||
let mut event = encode_sse_event("response.failed", &data.to_string());
|
||||
event.push_str(DONE_SSE_FRAME);
|
||||
let mut event = match error_encoder {
|
||||
Some(encode) => encode(&data).to_vec(),
|
||||
None => encode_sse_event("response.failed", &data.to_string()).into_bytes(),
|
||||
};
|
||||
event.extend_from_slice(DONE_SSE_FRAME.as_bytes());
|
||||
Body::from(event)
|
||||
}
|
||||
|
||||
@@ -10,10 +10,12 @@ pub mod websocket;
|
||||
|
||||
pub(crate) use crate::request_context::request_client_context;
|
||||
pub use error::{ProtocolError, ProtocolErrorBody, RequestDecodeError, ResponseEncodeError};
|
||||
pub(in crate::openai) use http::{HttpResponseAdapter, execute_prepared_response};
|
||||
pub(crate) use http::{
|
||||
ResponseAuthorization, ResponsesHttpRequest, execute_prepared_responses, responses,
|
||||
};
|
||||
pub use http::{collect_execution_response, stream_execution_response};
|
||||
pub(in crate::openai) use request::decompress_request_body;
|
||||
pub use request::{
|
||||
ContinuationIntent, DecodedResponsesRequest, OpenAiRequestHeaders, ResponsesRequestMetadata,
|
||||
decode_request_with_headers,
|
||||
|
||||
@@ -371,7 +371,7 @@ const MAX_ZSTD_WINDOW_LOG: u32 = 26;
|
||||
/// 未压缩与 `identity` 借用原始正文;压缩正文在读取过程中限制展开大小,
|
||||
/// 避免先完整分配再检查
|
||||
/// 只接受单一编码,重复头和叠加编码不能只解释第一项
|
||||
fn decompress_request_body<'a>(
|
||||
pub(in crate::openai) fn decompress_request_body<'a>(
|
||||
body: &'a [u8],
|
||||
headers: &HeaderMap,
|
||||
max_decompressed_bytes: usize,
|
||||
|
||||
@@ -10,6 +10,8 @@ pub(super) const DOWNSTREAM_WRITE_TIMEOUT: Duration = Duration::from_secs(5 * 60
|
||||
pub(super) const DOWNSTREAM_CLOSE_TIMEOUT: Duration = Duration::from_secs(1);
|
||||
pub(super) const CONNECTION_MAX_AGE: Duration = Duration::from_secs(60 * 60);
|
||||
pub(super) const DOWNSTREAM_PING_INTERVAL: Duration = Duration::from_secs(25);
|
||||
pub(super) const DOWNSTREAM_PONG_TIMEOUT: Duration = Duration::from_secs(30);
|
||||
pub(super) const DOWNSTREAM_PING_WRITE_TIMEOUT: Duration = Duration::from_secs(5);
|
||||
|
||||
/// WebSocket pump 的写入与生命周期预算
|
||||
#[derive(Clone, Copy)]
|
||||
@@ -124,6 +126,7 @@ pub enum PumpExitReason {
|
||||
ReadError,
|
||||
WriteError,
|
||||
WriteTimeout,
|
||||
PongTimeout,
|
||||
LifecycleShutdown,
|
||||
ConnectionMaxAge,
|
||||
CoordinatorDropped,
|
||||
@@ -141,6 +144,7 @@ impl PumpExitReason {
|
||||
Self::ReadError => "read_error",
|
||||
Self::WriteError => "write_error",
|
||||
Self::WriteTimeout => "write_timeout",
|
||||
Self::PongTimeout => "pong_timeout",
|
||||
Self::LifecycleShutdown => "lifecycle_shutdown",
|
||||
Self::ConnectionMaxAge => "connection_max_age",
|
||||
Self::CoordinatorDropped => "coordinator_dropped",
|
||||
|
||||
@@ -4,8 +4,9 @@ use super::{
|
||||
frame::*,
|
||||
pump::run_pump,
|
||||
state::{ConnectionCommand, ConnectionStats},
|
||||
transport::Socket,
|
||||
};
|
||||
use axum::extract::ws::{CloseFrame, Message, Utf8Bytes, WebSocket, close_code};
|
||||
use axum::extract::ws::{CloseFrame, Message, Utf8Bytes, close_code};
|
||||
use futures::{Sink, Stream};
|
||||
use gateway_core::{
|
||||
engine::middleware::{FrozenMiddlewarePlan, MiddlewareHeader},
|
||||
@@ -21,7 +22,7 @@ use std::{
|
||||
time::Duration,
|
||||
};
|
||||
use tokio::{
|
||||
sync::{mpsc, oneshot},
|
||||
sync::{mpsc, oneshot, watch},
|
||||
task::JoinHandle,
|
||||
time::{Instant, timeout},
|
||||
};
|
||||
@@ -30,6 +31,7 @@ use tokio::{
|
||||
pub struct ResponsesWebSocketConnection {
|
||||
connection_id: Arc<str>,
|
||||
opened_at: Instant,
|
||||
cancellation: CancellationToken,
|
||||
expired: Arc<AtomicBool>,
|
||||
commands: Option<mpsc::Sender<ConnectionCommand>>,
|
||||
incoming: mpsc::Receiver<PendingConnectionEvent>,
|
||||
@@ -37,19 +39,21 @@ pub struct ResponsesWebSocketConnection {
|
||||
exited: oneshot::Receiver<PumpExitReason>,
|
||||
pump_task: Option<JoinHandle<()>>,
|
||||
stats: Arc<ConnectionStats>,
|
||||
activity: watch::Sender<Instant>,
|
||||
config: ConnectionConfig,
|
||||
exit_reason: Option<PumpExitReason>,
|
||||
}
|
||||
|
||||
impl ResponsesWebSocketConnection {
|
||||
pub(in super::super) fn new(
|
||||
socket: WebSocket,
|
||||
socket: Socket,
|
||||
connection_id: String,
|
||||
cancellation: CancellationToken,
|
||||
middleware: Option<FrozenMiddlewarePlan>,
|
||||
headers: Arc<[MiddlewareHeader]>,
|
||||
) -> Self {
|
||||
spawn_connection(
|
||||
socket.activity.clone(),
|
||||
socket,
|
||||
Arc::<str>::from(connection_id),
|
||||
cancellation,
|
||||
@@ -266,7 +270,9 @@ impl ResponsesWebSocketConnection {
|
||||
}
|
||||
|
||||
fn terminate(&mut self, reason: PumpExitReason) {
|
||||
let reason = self.exited.try_recv().unwrap_or(reason);
|
||||
self.exit_reason.get_or_insert(reason);
|
||||
self.cancellation.cancel();
|
||||
self.commands.take();
|
||||
if let Some(task) = self.pump_task.take() {
|
||||
task.abort();
|
||||
@@ -275,8 +281,7 @@ impl ResponsesWebSocketConnection {
|
||||
|
||||
pub(in super::super) fn log_summary(&self, request_count: u64) {
|
||||
let connection_age_ms = self.age().as_millis();
|
||||
let read_idle_ms = connection_age_ms
|
||||
.saturating_sub(u128::from(self.stats.last_read_ms.load(Ordering::Relaxed)));
|
||||
let read_idle_ms = self.activity.borrow().elapsed().as_millis();
|
||||
let write_idle_ms = connection_age_ms
|
||||
.saturating_sub(u128::from(self.stats.last_write_ms.load(Ordering::Relaxed)));
|
||||
tracing::info!(
|
||||
@@ -306,6 +311,7 @@ impl ResponsesWebSocketConnection {
|
||||
|
||||
impl Drop for ResponsesWebSocketConnection {
|
||||
fn drop(&mut self) {
|
||||
self.cancellation.cancel();
|
||||
self.commands.take();
|
||||
if let Some(task) = self.pump_task.take() {
|
||||
task.abort();
|
||||
@@ -314,7 +320,10 @@ impl Drop for ResponsesWebSocketConnection {
|
||||
}
|
||||
|
||||
/// 为 WebSocket transport 启动包含冻结中间件计划的单 owner pump
|
||||
/// cancellation 必须属于本连接,连接退出或句柄释放时会取消它
|
||||
/// activity 仅由 transport 的实际入站字节或 pump 的完整消息更新,不能由写入刷新
|
||||
pub fn spawn_connection<S, E>(
|
||||
activity: watch::Sender<Instant>,
|
||||
socket: S,
|
||||
connection_id: Arc<str>,
|
||||
cancellation: CancellationToken,
|
||||
@@ -337,11 +346,12 @@ where
|
||||
.unwrap_or_default();
|
||||
let pump_task = tokio::spawn(run_pump(
|
||||
socket,
|
||||
activity.clone(),
|
||||
command_rx,
|
||||
incoming_tx,
|
||||
exit_tx,
|
||||
Arc::clone(&connection_id),
|
||||
cancellation,
|
||||
cancellation.clone(),
|
||||
opened_at,
|
||||
Arc::clone(&expired),
|
||||
Arc::clone(&stats),
|
||||
@@ -353,6 +363,7 @@ where
|
||||
ResponsesWebSocketConnection {
|
||||
connection_id,
|
||||
opened_at,
|
||||
cancellation,
|
||||
expired,
|
||||
commands: Some(command_tx),
|
||||
incoming: incoming_rx,
|
||||
@@ -360,6 +371,7 @@ where
|
||||
exited: exit_rx,
|
||||
pump_task: Some(pump_task),
|
||||
stats,
|
||||
activity,
|
||||
config,
|
||||
exit_reason: None,
|
||||
}
|
||||
|
||||
@@ -5,9 +5,12 @@ mod handle;
|
||||
mod middleware;
|
||||
mod pump;
|
||||
mod state;
|
||||
mod transport;
|
||||
|
||||
pub use frame::{
|
||||
ConnectionConfig, ConnectionEvent, ConnectionWriteError, FramePhase, PumpExitReason,
|
||||
WriteContext, WriteOutcome,
|
||||
};
|
||||
pub use handle::{ResponsesWebSocketConnection, spawn_connection};
|
||||
pub(super) use transport::Socket;
|
||||
pub(crate) use transport::Upgrade;
|
||||
|
||||
@@ -19,13 +19,14 @@ use std::{
|
||||
},
|
||||
};
|
||||
use tokio::{
|
||||
sync::{Semaphore, mpsc, oneshot},
|
||||
sync::{Semaphore, mpsc, oneshot, watch},
|
||||
time::Instant,
|
||||
};
|
||||
|
||||
#[expect(clippy::too_many_arguments)]
|
||||
pub(super) async fn run_pump<S, E>(
|
||||
socket: S,
|
||||
activity: watch::Sender<Instant>,
|
||||
commands: mpsc::Receiver<ConnectionCommand>,
|
||||
incoming: mpsc::Sender<PendingConnectionEvent>,
|
||||
exited: oneshot::Sender<PumpExitReason>,
|
||||
@@ -42,6 +43,7 @@ pub(super) async fn run_pump<S, E>(
|
||||
S: Stream<Item = Result<Message, E>> + Sink<Message, Error = E> + Unpin + Send + 'static,
|
||||
E: fmt::Display + Send + 'static,
|
||||
{
|
||||
let _connection_lifetime = middleware::CancelOnDrop(Some(cancellation.clone()));
|
||||
let context = Arc::new(PumpContext {
|
||||
extensions,
|
||||
connection_id,
|
||||
@@ -57,11 +59,13 @@ pub(super) async fn run_pump<S, E>(
|
||||
let (writer, mut reader) = socket.split();
|
||||
let writer = Arc::new(middleware::Writer::new(writer, context.clone()));
|
||||
let sender: Arc<dyn contract::Sender> = writer.clone();
|
||||
// 同一个受管任务同时驱动两侧;等待写入不能阻塞入站控制、关闭或取消
|
||||
let received_activity = activity.subscribe();
|
||||
// 心跳独立于业务写入等待,但共享唯一 writer,避免长写入或排队阻止失联检测
|
||||
let reason = tokio::select! {
|
||||
biased;
|
||||
reason = read_connection(&mut reader, &incoming, sender.clone(), &context) => reason,
|
||||
reason = read_connection(&mut reader, &incoming, sender.clone(), &activity, &context) => reason,
|
||||
reason = write_connection(writer.clone(), commands, &context) => reason,
|
||||
reason = heartbeat_connection(writer.clone(), received_activity, &context) => reason,
|
||||
};
|
||||
if matches!(reason, PumpExitReason::ClientClose) {
|
||||
if let Err(error) = writer.finish_peer_close(&cancellation).await {
|
||||
@@ -78,6 +82,7 @@ async fn read_connection<S, E>(
|
||||
socket: &mut S,
|
||||
incoming: &mpsc::Sender<PendingConnectionEvent>,
|
||||
sender: Arc<dyn contract::Sender>,
|
||||
activity: &watch::Sender<Instant>,
|
||||
context: &PumpContext,
|
||||
) -> PumpExitReason
|
||||
where
|
||||
@@ -102,7 +107,10 @@ where
|
||||
}
|
||||
}
|
||||
message = socket.next() => {
|
||||
if matches!(&message, Some(Ok(_))) { context.stats.record_read(context.opened_at); }
|
||||
if matches!(&message, Some(Ok(_))) {
|
||||
// 链路活动以真实入站为准,插件丢弃消息不应抹掉对端已经响应的事实
|
||||
activity.send_replace(Instant::now());
|
||||
}
|
||||
let message = match message {
|
||||
Some(Ok(message)) => match middleware::transform(message, contract::Direction::Incoming, sender.clone(), context).await {
|
||||
Ok(Some(message)) => Some(Ok(message)),
|
||||
@@ -145,40 +153,22 @@ where
|
||||
S: Sink<Message, Error = E> + Unpin + Send + 'static,
|
||||
E: fmt::Display + Send + 'static,
|
||||
{
|
||||
let mut heartbeat = tokio::time::interval_at(
|
||||
context.opened_at + DOWNSTREAM_PING_INTERVAL,
|
||||
DOWNSTREAM_PING_INTERVAL,
|
||||
);
|
||||
heartbeat.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
|
||||
let mut ping_sequence = 0_u64;
|
||||
loop {
|
||||
let message = tokio::select! {
|
||||
let command = tokio::select! {
|
||||
biased;
|
||||
() = context.cancellation.cancelled() => return PumpExitReason::LifecycleShutdown,
|
||||
command = commands.recv() => {
|
||||
let Some(command) = command else { return PumpExitReason::CoordinatorDropped; };
|
||||
let ConnectionCommand { message, context: write_context, acknowledged } = command;
|
||||
let closing = matches!(message, Message::Close(_));
|
||||
let result = match middleware::transform(message, contract::Direction::Outgoing, writer.clone(), context).await {
|
||||
Ok(Some(message)) => writer.write(message).await.map(|()| WriteOutcome::Written),
|
||||
Ok(None) => Ok(WriteOutcome::Suppressed),
|
||||
Err(error) => Err(ConnectionWriteError::Transport { message: error.to_string() }),
|
||||
};
|
||||
let reason = result.as_ref().err().map(|error| write_exit_reason(error, context));
|
||||
if let Err(error) = &result {
|
||||
tracing::debug!(websocket_connection_id = %context.connection_id, request_id = write_context.request_id.as_deref().unwrap_or(""), frame_phase = write_context.phase.as_str(), error = %error, "Responses WebSocket pump transport write failed");
|
||||
}
|
||||
let _ = acknowledged.send(result);
|
||||
if let Some(reason) = reason { return reason; }
|
||||
if closing { return PumpExitReason::ServerClose; }
|
||||
continue;
|
||||
}
|
||||
_ = heartbeat.tick() => {
|
||||
ping_sequence = ping_sequence.wrapping_add(1);
|
||||
Message::Ping(ping_sequence.to_be_bytes().to_vec().into())
|
||||
}
|
||||
command = commands.recv() => command,
|
||||
};
|
||||
let message = match middleware::transform(
|
||||
let Some(command) = command else {
|
||||
return PumpExitReason::CoordinatorDropped;
|
||||
};
|
||||
let ConnectionCommand {
|
||||
message,
|
||||
context: write_context,
|
||||
acknowledged,
|
||||
} = command;
|
||||
let closing = matches!(message, Message::Close(_));
|
||||
let result = match middleware::transform(
|
||||
message,
|
||||
contract::Direction::Outgoing,
|
||||
writer.clone(),
|
||||
@@ -186,21 +176,125 @@ where
|
||||
)
|
||||
.await
|
||||
{
|
||||
Ok(Some(message)) => message,
|
||||
Ok(None) => continue,
|
||||
Err(_) => return PumpExitReason::MiddlewareFailed,
|
||||
Ok(Some(message)) => writer.write(message).await.map(|()| WriteOutcome::Written),
|
||||
Ok(None) => Ok(WriteOutcome::Suppressed),
|
||||
Err(error) => Err(ConnectionWriteError::Transport {
|
||||
message: error.to_string(),
|
||||
}),
|
||||
};
|
||||
if let Err(error) = writer.write(message).await {
|
||||
tracing::info!(websocket_connection_id = %context.connection_id, error = %error, "Responses WebSocket control write failed");
|
||||
return write_exit_reason(&error, context);
|
||||
let reason = result
|
||||
.as_ref()
|
||||
.err()
|
||||
.map(|error| write_exit_reason(error, context));
|
||||
if let Err(error) = &result {
|
||||
tracing::debug!(websocket_connection_id = %context.connection_id, request_id = write_context.request_id.as_deref().unwrap_or(""), frame_phase = write_context.phase.as_str(), error = %error, "Responses WebSocket pump transport write failed");
|
||||
}
|
||||
let _ = acknowledged.send(result);
|
||||
if let Some(reason) = reason {
|
||||
return reason;
|
||||
}
|
||||
if closing {
|
||||
return PumpExitReason::ServerClose;
|
||||
}
|
||||
context
|
||||
.stats
|
||||
.ping_written_count
|
||||
.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
|
||||
async fn heartbeat_connection<S, E>(
|
||||
writer: Arc<middleware::Writer<S>>,
|
||||
mut activity: watch::Receiver<Instant>,
|
||||
context: &PumpContext,
|
||||
) -> PumpExitReason
|
||||
where
|
||||
S: Sink<Message, Error = E> + Unpin + Send + 'static,
|
||||
E: fmt::Display + Send + 'static,
|
||||
{
|
||||
let mut heartbeat = tokio::time::interval_at(
|
||||
context.opened_at + DOWNSTREAM_PING_INTERVAL,
|
||||
DOWNSTREAM_PING_INTERVAL,
|
||||
);
|
||||
heartbeat.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
|
||||
let mut ping_sequence = 0_u64;
|
||||
let pong_deadline = tokio::time::sleep(DOWNSTREAM_PONG_TIMEOUT);
|
||||
tokio::pin!(pong_deadline);
|
||||
let mut pending_ping = false;
|
||||
loop {
|
||||
tokio::select! {
|
||||
biased;
|
||||
() = context.cancellation.cancelled() => return PumpExitReason::LifecycleShutdown,
|
||||
changed = activity.changed() => {
|
||||
if changed.is_err() { return PumpExitReason::PumpStopped; }
|
||||
pending_ping = false;
|
||||
continue;
|
||||
}
|
||||
() = &mut pong_deadline, if pending_ping => return PumpExitReason::PongTimeout,
|
||||
_ = heartbeat.tick() => {}
|
||||
}
|
||||
ping_sequence = ping_sequence.wrapping_add(1);
|
||||
let write = tokio::time::timeout(
|
||||
DOWNSTREAM_PING_WRITE_TIMEOUT,
|
||||
write_heartbeat(&writer, ping_sequence, context),
|
||||
);
|
||||
tokio::pin!(write);
|
||||
let mut received_during_write = false;
|
||||
loop {
|
||||
tokio::select! {
|
||||
biased;
|
||||
() = context.cancellation.cancelled() => return PumpExitReason::LifecycleShutdown,
|
||||
changed = activity.changed() => {
|
||||
if changed.is_err() { return PumpExitReason::PumpStopped; }
|
||||
pending_ping = false;
|
||||
received_during_write = true;
|
||||
}
|
||||
() = &mut pong_deadline, if pending_ping => return PumpExitReason::PongTimeout,
|
||||
result = &mut write => {
|
||||
match result {
|
||||
Ok(Ok(true)) => {
|
||||
context.stats.ping_written_count.fetch_add(1, Ordering::Relaxed);
|
||||
// 重复 Ping 不顺延截止,写入完成前已经收到的响应也不能丢失
|
||||
if !pending_ping && !received_during_write {
|
||||
pong_deadline.as_mut().reset(Instant::now() + DOWNSTREAM_PONG_TIMEOUT);
|
||||
pending_ping = true;
|
||||
}
|
||||
}
|
||||
Ok(Ok(false)) => {}
|
||||
Ok(Err(reason)) => return reason,
|
||||
Err(_) => return PumpExitReason::WriteTimeout,
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async fn write_heartbeat<S, E>(
|
||||
writer: &Arc<middleware::Writer<S>>,
|
||||
sequence: u64,
|
||||
context: &PumpContext,
|
||||
) -> Result<bool, PumpExitReason>
|
||||
where
|
||||
S: Sink<Message, Error = E> + Unpin + Send + 'static,
|
||||
E: fmt::Display + Send + 'static,
|
||||
{
|
||||
let message = middleware::transform(
|
||||
Message::Ping(sequence.to_be_bytes().to_vec().into()),
|
||||
contract::Direction::Outgoing,
|
||||
writer.clone(),
|
||||
context,
|
||||
)
|
||||
.await
|
||||
.map_err(|_| PumpExitReason::MiddlewareFailed)?;
|
||||
let Some(message) = message else {
|
||||
return Ok(false);
|
||||
};
|
||||
let is_ping = matches!(message, Message::Ping(_));
|
||||
writer.write(message).await.map_err(|error| {
|
||||
tracing::info!(websocket_connection_id = %context.connection_id, error = %error, "Responses WebSocket control write failed");
|
||||
write_exit_reason(&error, context)
|
||||
})?;
|
||||
Ok(is_ping)
|
||||
}
|
||||
|
||||
fn write_exit_reason(error: &ConnectionWriteError, context: &PumpContext) -> PumpExitReason {
|
||||
match error {
|
||||
ConnectionWriteError::Timeout { .. } => PumpExitReason::WriteTimeout,
|
||||
|
||||
@@ -30,18 +30,10 @@ pub(super) struct ConnectionStats {
|
||||
pub(super) ping_received_count: AtomicU64,
|
||||
pub(super) ping_written_count: AtomicU64,
|
||||
pub(super) pong_received_count: AtomicU64,
|
||||
pub(super) last_read_ms: AtomicU64,
|
||||
pub(super) last_write_ms: AtomicU64,
|
||||
}
|
||||
|
||||
impl ConnectionStats {
|
||||
pub(super) fn record_read(&self, opened_at: Instant) {
|
||||
self.last_read_ms.store(
|
||||
u64::try_from(opened_at.elapsed().as_millis()).unwrap_or(u64::MAX),
|
||||
Ordering::Relaxed,
|
||||
);
|
||||
}
|
||||
|
||||
pub(super) fn record_write(&self, opened_at: Instant) {
|
||||
self.last_write_ms.store(
|
||||
u64::try_from(opened_at.elapsed().as_millis()).unwrap_or(u64::MAX),
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
//! 保留 Axum 握手校验,在 WS 消息解码前记录下游真实接收进度
|
||||
|
||||
use std::{
|
||||
io,
|
||||
pin::Pin,
|
||||
task::{Context, Poll},
|
||||
};
|
||||
|
||||
use axum::{
|
||||
extract::{
|
||||
FromRequestParts,
|
||||
ws::{self, Message},
|
||||
},
|
||||
http::{HeaderValue, StatusCode, Version, header, request::Parts},
|
||||
response::{IntoResponse, Response},
|
||||
};
|
||||
use futures::{Sink, Stream, ready};
|
||||
use hyper::upgrade::{OnUpgrade, Upgraded};
|
||||
use hyper_util::rt::TokioIo;
|
||||
use tokio::{
|
||||
io::{AsyncRead, AsyncWrite, ReadBuf},
|
||||
sync::watch,
|
||||
time::Instant,
|
||||
};
|
||||
use tokio_tungstenite::{
|
||||
WebSocketStream,
|
||||
tungstenite::{
|
||||
self,
|
||||
protocol::{Role, WebSocketConfig},
|
||||
},
|
||||
};
|
||||
|
||||
pub(crate) struct Upgrade {
|
||||
pending: OnUpgrade,
|
||||
key: Option<HeaderValue>,
|
||||
}
|
||||
|
||||
impl<S: Send + Sync> FromRequestParts<S> for Upgrade {
|
||||
type Rejection = ws::rejection::WebSocketUpgradeRejection;
|
||||
|
||||
async fn from_request_parts(parts: &mut Parts, state: &S) -> Result<Self, Self::Rejection> {
|
||||
let pending = parts.extensions.get::<OnUpgrade>().cloned();
|
||||
// Axum 的 WebSocket 不暴露底层 I/O;只复用其握手校验,由本层接管唯一升级流
|
||||
let _validated = ws::WebSocketUpgrade::from_request_parts(parts, state).await?;
|
||||
let pending = pending.expect("Axum validated the presence of OnUpgrade");
|
||||
let key = (parts.version <= Version::HTTP_11)
|
||||
.then(|| parts.headers.get(header::SEC_WEBSOCKET_KEY).cloned())
|
||||
.flatten();
|
||||
Ok(Self { pending, key })
|
||||
}
|
||||
}
|
||||
|
||||
impl Upgrade {
|
||||
pub(in super::super) fn on_upgrade<F, Fut>(self, callback: F) -> Response
|
||||
where
|
||||
F: FnOnce(Socket) -> Fut + Send + 'static,
|
||||
Fut: Future<Output = ()> + Send + 'static,
|
||||
{
|
||||
tokio::spawn(async move {
|
||||
let Ok(upgraded) = self.pending.await else {
|
||||
return;
|
||||
};
|
||||
let (activity, _) = watch::channel(Instant::now());
|
||||
let io = Receiving {
|
||||
inner: TokioIo::new(upgraded),
|
||||
activity: activity.clone(),
|
||||
};
|
||||
// 与原生 Responses 升级保持一致,不引入消息或帧长度上限
|
||||
let config = WebSocketConfig::default()
|
||||
.max_message_size(None)
|
||||
.max_frame_size(None);
|
||||
let inner = WebSocketStream::from_raw_socket(io, Role::Server, Some(config)).await;
|
||||
callback(Socket { inner, activity }).await;
|
||||
});
|
||||
if let Some(key) = self.key {
|
||||
(
|
||||
StatusCode::SWITCHING_PROTOCOLS,
|
||||
[
|
||||
(header::CONNECTION, "upgrade".to_owned()),
|
||||
(header::UPGRADE, "websocket".to_owned()),
|
||||
(
|
||||
header::SEC_WEBSOCKET_ACCEPT,
|
||||
tungstenite::handshake::derive_accept_key(key.as_bytes()),
|
||||
),
|
||||
],
|
||||
)
|
||||
.into_response()
|
||||
} else {
|
||||
StatusCode::OK.into_response()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct Receiving {
|
||||
inner: TokioIo<Upgraded>,
|
||||
activity: watch::Sender<Instant>,
|
||||
}
|
||||
|
||||
impl AsyncRead for Receiving {
|
||||
fn poll_read(
|
||||
mut self: Pin<&mut Self>,
|
||||
cx: &mut Context<'_>,
|
||||
buffer: &mut ReadBuf<'_>,
|
||||
) -> Poll<io::Result<()>> {
|
||||
let before = buffer.filled().len();
|
||||
let result = Pin::new(&mut self.inner).poll_read(cx, buffer);
|
||||
if buffer.filled().len() > before {
|
||||
// 完整消息尚未收齐时,客户端可能仍在发送、暂时无法读取 Ping
|
||||
self.activity.send_replace(Instant::now());
|
||||
}
|
||||
result
|
||||
}
|
||||
}
|
||||
|
||||
impl AsyncWrite for Receiving {
|
||||
fn poll_write(
|
||||
mut self: Pin<&mut Self>,
|
||||
cx: &mut Context<'_>,
|
||||
buffer: &[u8],
|
||||
) -> Poll<io::Result<usize>> {
|
||||
Pin::new(&mut self.inner).poll_write(cx, buffer)
|
||||
}
|
||||
fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
|
||||
Pin::new(&mut self.inner).poll_flush(cx)
|
||||
}
|
||||
fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
|
||||
Pin::new(&mut self.inner).poll_shutdown(cx)
|
||||
}
|
||||
}
|
||||
|
||||
pub(in super::super) struct Socket {
|
||||
inner: WebSocketStream<Receiving>,
|
||||
pub(super) activity: watch::Sender<Instant>,
|
||||
}
|
||||
|
||||
impl Stream for Socket {
|
||||
type Item = Result<Message, axum::Error>;
|
||||
|
||||
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
|
||||
loop {
|
||||
let message = match ready!(Pin::new(&mut self.inner).poll_next(cx)) {
|
||||
Some(Ok(message)) => message,
|
||||
Some(Err(error)) => return Poll::Ready(Some(Err(axum::Error::new(error)))),
|
||||
None => return Poll::Ready(None),
|
||||
};
|
||||
let message = match message {
|
||||
tungstenite::Message::Text(text) => Message::Text(
|
||||
ws::Utf8Bytes::try_from(bytes::Bytes::from(text)).map_err(axum::Error::new)?,
|
||||
),
|
||||
tungstenite::Message::Binary(data) => Message::Binary(data),
|
||||
tungstenite::Message::Ping(data) => Message::Ping(data),
|
||||
tungstenite::Message::Pong(data) => Message::Pong(data),
|
||||
tungstenite::Message::Close(frame) => {
|
||||
Message::Close(frame.map(|frame| ws::CloseFrame {
|
||||
code: frame.code.into(),
|
||||
reason: frame.reason.to_string().into(),
|
||||
}))
|
||||
}
|
||||
tungstenite::Message::Frame(_) => continue,
|
||||
};
|
||||
return Poll::Ready(Some(Ok(message)));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Sink<Message> for Socket {
|
||||
type Error = axum::Error;
|
||||
|
||||
fn poll_ready(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
|
||||
Pin::new(&mut self.inner)
|
||||
.poll_ready(cx)
|
||||
.map_err(axum::Error::new)
|
||||
}
|
||||
fn start_send(mut self: Pin<&mut Self>, message: Message) -> Result<(), Self::Error> {
|
||||
let message = match message {
|
||||
Message::Text(text) => tungstenite::Message::Text(
|
||||
tungstenite::Utf8Bytes::try_from(bytes::Bytes::from(text))
|
||||
.map_err(axum::Error::new)?,
|
||||
),
|
||||
Message::Binary(data) => tungstenite::Message::Binary(data),
|
||||
Message::Ping(data) => tungstenite::Message::Ping(data),
|
||||
Message::Pong(data) => tungstenite::Message::Pong(data),
|
||||
Message::Close(frame) => {
|
||||
tungstenite::Message::Close(frame.map(|frame| tungstenite::protocol::CloseFrame {
|
||||
code: frame.code.into(),
|
||||
reason: frame.reason.to_string().into(),
|
||||
}))
|
||||
}
|
||||
};
|
||||
Pin::new(&mut self.inner)
|
||||
.start_send(message)
|
||||
.map_err(axum::Error::new)
|
||||
}
|
||||
fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
|
||||
Pin::new(&mut self.inner)
|
||||
.poll_flush(cx)
|
||||
.map_err(axum::Error::new)
|
||||
}
|
||||
fn poll_close(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
|
||||
Pin::new(&mut self.inner)
|
||||
.poll_close(cx)
|
||||
.map_err(axum::Error::new)
|
||||
}
|
||||
}
|
||||
@@ -400,13 +400,13 @@ fn middleware_gateway_error(error: MiddlewareError) -> GatewayError {
|
||||
MiddlewareError::Rejected | MiddlewareError::Remote { rejected: true, .. } => {
|
||||
GatewayError::new(
|
||||
GatewayErrorKind::PolicyDenied,
|
||||
"request middleware rejected the request",
|
||||
"Request middleware rejected the request.",
|
||||
)
|
||||
}
|
||||
MiddlewareError::Fault | MiddlewareError::Remote { .. } | MiddlewareError::InvalidState => {
|
||||
GatewayError::new(
|
||||
GatewayErrorKind::Internal,
|
||||
"request middleware returned an invalid response",
|
||||
"Request middleware returned an invalid response.",
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -770,6 +770,21 @@ pub(super) async fn send_control(
|
||||
payload: &str,
|
||||
request_id: &Arc<str>,
|
||||
) -> ForwardOutcome {
|
||||
// HTTP 桥接没有原连接可中断;普通 error 会使 Codex 丢弃仍可正常完成的响应流
|
||||
// 仅在发送前确认无绑定时拒绝中断,不把发送结果不明的失败伪装成明确拒绝
|
||||
if control.binding().is_none()
|
||||
&& let Some(message) = super::interrupt::ResponseInterrupt::unavailable_event(payload)
|
||||
{
|
||||
return if connection
|
||||
.send_text(message, WriteContext::request(request_id, FramePhase::Data))
|
||||
.await
|
||||
.is_ok()
|
||||
{
|
||||
ForwardOutcome::Continue
|
||||
} else {
|
||||
ForwardOutcome::Disconnect
|
||||
};
|
||||
}
|
||||
let result = tokio::select! {
|
||||
_ = connection.wait_for_exit() => return ForwardOutcome::Disconnect,
|
||||
result = control.send(payload) => result,
|
||||
@@ -779,7 +794,7 @@ pub(super) async fn send_control(
|
||||
connection,
|
||||
&GatewayError::new(
|
||||
GatewayErrorKind::InvalidRequest,
|
||||
"No upstream WebSocket connection is available for control messages",
|
||||
"No upstream WebSocket connection is available for control messages.",
|
||||
),
|
||||
request_id,
|
||||
)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
//! 当前逻辑响应的中断意图,跨重试连接保留并随新响应 ID 转发
|
||||
//! 当前逻辑响应的中断意图、无控制通道时的拒绝与跨重试转发
|
||||
|
||||
use std::{borrow::Cow, collections::BTreeSet};
|
||||
|
||||
@@ -53,14 +53,12 @@ impl ResponseInterrupt {
|
||||
|
||||
/// 未知控制消息、模式和其他逻辑响应的 ID 仍走原始透明通道
|
||||
pub(super) fn retain(&mut self, payload: &str) -> bool {
|
||||
let Ok(event) = serde_json::from_str::<Value>(payload) else {
|
||||
let Some(event) = interrupt_request(payload) else {
|
||||
return false;
|
||||
};
|
||||
if event["type"] != "response.interrupt"
|
||||
|| event["mode"] != "discard_partial_items"
|
||||
|| !event["response_id"]
|
||||
.as_str()
|
||||
.is_some_and(|id| self.delivered_ids.contains(id))
|
||||
if !event["response_id"]
|
||||
.as_str()
|
||||
.is_some_and(|id| self.delivered_ids.contains(id))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
@@ -69,6 +67,18 @@ impl ResponseInterrupt {
|
||||
true
|
||||
}
|
||||
|
||||
/// 中断失败不代表模型响应失败,客户端仍可收完真实终态后续接
|
||||
pub(super) fn unavailable_event(payload: &str) -> Option<String> {
|
||||
let event = interrupt_request(payload)?;
|
||||
Some(
|
||||
serde_json::json!({
|
||||
"type": "response.interrupt.failed",
|
||||
"response_id": event["response_id"],
|
||||
})
|
||||
.to_string(),
|
||||
)
|
||||
}
|
||||
|
||||
pub(super) async fn forward_pending(&mut self, control: &ResponseControl) {
|
||||
if self.sent {
|
||||
return;
|
||||
@@ -89,3 +99,13 @@ impl ResponseInterrupt {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn interrupt_request(payload: &str) -> Option<Value> {
|
||||
let event: Value = serde_json::from_str(payload).ok()?;
|
||||
(event["type"] == "response.interrupt"
|
||||
&& event["mode"] == "discard_partial_items"
|
||||
&& event["response_id"]
|
||||
.as_str()
|
||||
.is_some_and(|id| !id.is_empty()))
|
||||
.then_some(event)
|
||||
}
|
||||
|
||||
@@ -11,11 +11,7 @@ use std::{
|
||||
};
|
||||
|
||||
use axum::{
|
||||
extract::{
|
||||
Extension, State,
|
||||
connect_info::ConnectInfo,
|
||||
ws::{WebSocket, WebSocketUpgrade},
|
||||
},
|
||||
extract::{Extension, State, connect_info::ConnectInfo},
|
||||
http::{HeaderMap, StatusCode},
|
||||
response::{IntoResponse, Response},
|
||||
};
|
||||
@@ -45,7 +41,9 @@ use crate::{
|
||||
use super::{
|
||||
request::OpenAiRequestHeaders, request_client_context, validation::ResponseValidationFacts,
|
||||
};
|
||||
use connection::{ConnectionEvent, FramePhase, ResponsesWebSocketConnection, WriteContext};
|
||||
use connection::{
|
||||
ConnectionEvent, FramePhase, ResponsesWebSocketConnection, Socket, Upgrade, WriteContext,
|
||||
};
|
||||
use forward::{
|
||||
ConnectionReplaySnapshot, ForwardOutcome, execution_response, forward_control_event,
|
||||
forward_response, new_replay_capture, send_control, send_gateway_error, send_middleware_error,
|
||||
@@ -54,7 +52,7 @@ use forward::{
|
||||
pub use protocol::{ResponseCreateFrameError, decode_response_create_with_context};
|
||||
use protocol::{connection_limit_event, is_response_create};
|
||||
|
||||
const TEXT_FRAMES_ONLY: &str = "Responses WebSocket accepts text frames only";
|
||||
const TEXT_FRAMES_ONLY: &str = "Responses WebSocket accepts text frames only.";
|
||||
const CONNECTION_LIMIT_CLOSE_REASON: &str = "Responses websocket connection limit reached";
|
||||
|
||||
/// 将已认证的 `GET /v1/responses` 升级为 Responses WebSocket
|
||||
@@ -64,7 +62,7 @@ pub(crate) async fn responses_websocket(
|
||||
middleware: Option<Extension<FrozenMiddlewarePlan>>,
|
||||
lifetime: Option<Extension<Arc<crate::middleware::CallLifetime>>>,
|
||||
headers: HeaderMap,
|
||||
websocket: WebSocketUpgrade,
|
||||
websocket: Upgrade,
|
||||
) -> Response {
|
||||
let service = state.openai().clone();
|
||||
let client = match authenticate_client(&service, &headers).await {
|
||||
@@ -104,7 +102,7 @@ impl ResponsesWebSocketAdapter {
|
||||
#[expect(clippy::too_many_arguments)]
|
||||
fn upgrade_with_client_context(
|
||||
&self,
|
||||
websocket: WebSocketUpgrade,
|
||||
websocket: Upgrade,
|
||||
client: AuthenticatedClient,
|
||||
client_ip: Option<IpAddr>,
|
||||
user_agent: Option<String>,
|
||||
@@ -138,15 +136,10 @@ impl ResponsesWebSocketAdapter {
|
||||
middleware,
|
||||
};
|
||||
let origin = crate::middleware::current();
|
||||
websocket
|
||||
// 覆盖 axum/tungstenite 的私有 64 MiB message 与 16 MiB frame 默认值
|
||||
// Responses JSON 的协议可接受性由上游决定,代理不另设 wire 长度上限
|
||||
.max_message_size(usize::MAX)
|
||||
.max_frame_size(usize::MAX)
|
||||
.on_upgrade(move |socket| async move {
|
||||
let _lifetime = lifetime;
|
||||
crate::middleware::scope(origin, serve_responses_websocket(socket, session)).await;
|
||||
})
|
||||
websocket.on_upgrade(move |socket| async move {
|
||||
let _lifetime = lifetime;
|
||||
crate::middleware::scope(origin, serve_responses_websocket(socket, session)).await;
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -163,7 +156,7 @@ struct ResponsesWebSocketSession {
|
||||
middleware: Option<FrozenMiddlewarePlan>,
|
||||
}
|
||||
|
||||
async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocketSession) {
|
||||
async fn serve_responses_websocket(socket: Socket, session: ResponsesWebSocketSession) {
|
||||
let ResponsesWebSocketSession {
|
||||
service,
|
||||
client,
|
||||
@@ -181,13 +174,15 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
|
||||
"Responses WebSocket connected"
|
||||
);
|
||||
let _connection_guard = connection_guard;
|
||||
let cancellation = lifecycle.cancellation();
|
||||
let cancellation = lifecycle.cancellation().child_token();
|
||||
let _connection_cancellation =
|
||||
crate::middleware::websocket::CancelOnDrop(Some(cancellation.clone()));
|
||||
let request_headers =
|
||||
request_headers.with_downstream_websocket_connection_id(connection_id.clone());
|
||||
let mut connection = ResponsesWebSocketConnection::new(
|
||||
socket,
|
||||
connection_id,
|
||||
cancellation,
|
||||
cancellation.clone(),
|
||||
middleware,
|
||||
encode_headers(&raw_headers).into(),
|
||||
);
|
||||
@@ -286,20 +281,24 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
|
||||
let current = execution.request_settings().ok_or_else(|| {
|
||||
gateway_core::error::GatewayError::new(
|
||||
gateway_core::error::GatewayErrorKind::Internal,
|
||||
"runtime snapshot is unavailable",
|
||||
"Gateway configuration is unavailable.",
|
||||
)
|
||||
})?;
|
||||
let rebased = settings.rebase(current.snapshot()).map_err(|_| {
|
||||
gateway_core::error::GatewayError::new(
|
||||
gateway_core::error::GatewayErrorKind::InvalidRequest,
|
||||
"request settings are invalid",
|
||||
"Request settings are invalid.",
|
||||
)
|
||||
})?;
|
||||
client = client.with_request_settings(rebased);
|
||||
}
|
||||
execution.prepare_execution(client).await
|
||||
};
|
||||
let prepared = match preparation.await {
|
||||
let prepared = match tokio::select! {
|
||||
biased;
|
||||
_ = connection.wait_for_exit() => break,
|
||||
prepared = preparation => prepared,
|
||||
} {
|
||||
Ok(prepared) => prepared,
|
||||
Err(error) => {
|
||||
trace_rejected_request(
|
||||
@@ -317,6 +316,7 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
|
||||
continue;
|
||||
}
|
||||
};
|
||||
let prepared = prepared.with_connection_cancellation(cancellation.clone());
|
||||
let request_id = Arc::<str>::from(prepared.request_id().to_string());
|
||||
response_control = prepared.response_control();
|
||||
let capture = new_replay_capture();
|
||||
|
||||
@@ -7,6 +7,7 @@ use axum::{
|
||||
};
|
||||
|
||||
use super::{
|
||||
chat_completions::chat_completions,
|
||||
images::{image_edits, image_generations},
|
||||
live,
|
||||
models::{model_detail, models},
|
||||
@@ -19,6 +20,7 @@ use crate::ApiState;
|
||||
|
||||
// 公开协议路径由 API 层统一定义;Core 使用操作类型,不根据这些路径分派业务
|
||||
pub(super) const RESPONSES_PATH: &str = "/v1/responses";
|
||||
pub(super) const CHAT_COMPLETIONS_PATH: &str = "/v1/chat/completions";
|
||||
pub(super) const IMAGE_GENERATIONS_PATH: &str = "/v1/images/generations";
|
||||
pub(super) const IMAGE_EDITS_PATH: &str = "/v1/images/edits";
|
||||
pub(super) const SEARCH_PATH: &str = "/v1/alpha/search";
|
||||
@@ -33,6 +35,7 @@ pub(crate) fn router() -> Router<ApiState> {
|
||||
.route(IMAGE_EDITS_PATH, post(image_edits))
|
||||
.route(SEARCH_PATH, post(standalone_search))
|
||||
.route(RESPONSES_PATH, get(responses_websocket).post(responses))
|
||||
.route(CHAT_COMPLETIONS_PATH, post(chat_completions))
|
||||
.route(MODELS_PATH, get(models))
|
||||
// 官方 OpenAI 模型详情合同使用 path ID;它不属于 Admin API 约束
|
||||
.route(MODEL_DETAIL_PATH, get(model_detail))
|
||||
|
||||
@@ -65,7 +65,7 @@ fn search_operation(body: Bytes, headers: &HeaderMap) -> Result<Operation, Gatew
|
||||
.map_err(|_| {
|
||||
GatewayError::new(
|
||||
GatewayErrorKind::Internal,
|
||||
"OpenAI protocol identifier is invalid",
|
||||
"OpenAI protocol identifier is invalid.",
|
||||
)
|
||||
})?
|
||||
.with_context(context);
|
||||
|
||||
@@ -101,7 +101,7 @@ impl OpenAiService {
|
||||
.map_err(|_| {
|
||||
GatewayError::new(
|
||||
GatewayErrorKind::ModelNotFound,
|
||||
"requested model was not found",
|
||||
"Requested model was not found.",
|
||||
)
|
||||
})?;
|
||||
let previous_response_id = match metadata.continuation() {
|
||||
@@ -142,7 +142,7 @@ impl OpenAiService {
|
||||
let provider = ProviderKind::new("openai").map_err(|_| {
|
||||
GatewayError::new(
|
||||
GatewayErrorKind::Internal,
|
||||
"OpenAI provider identifier is invalid",
|
||||
"OpenAI provider identifier is invalid.",
|
||||
)
|
||||
})?;
|
||||
self.execution
|
||||
|
||||
@@ -63,7 +63,7 @@ async fn usage_response(
|
||||
if Query::<UsageQuery>::try_from_uri(&uri).is_err() {
|
||||
return openai_error_response(
|
||||
StatusCode::BAD_REQUEST,
|
||||
"Usage query does not accept query parameters",
|
||||
"Usage query does not accept query parameters.",
|
||||
"invalid_request_error",
|
||||
"invalid_usage_query",
|
||||
)
|
||||
@@ -80,7 +80,7 @@ async fn usage_response(
|
||||
Err(_) => {
|
||||
return openai_error_response(
|
||||
StatusCode::SERVICE_UNAVAILABLE,
|
||||
"Key usage is temporarily unavailable",
|
||||
"Key usage is temporarily unavailable.",
|
||||
"server_error",
|
||||
"usage_unavailable",
|
||||
)
|
||||
|
||||
@@ -73,18 +73,18 @@ async fn count_tokens(
|
||||
};
|
||||
let provider = match ProviderKind::new(provider) {
|
||||
Ok(provider) => provider,
|
||||
Err(_) => return invalid_request("provider identifier is invalid"),
|
||||
Err(_) => return invalid_request("Provider identifier is invalid."),
|
||||
};
|
||||
let model = match UpstreamModelId::new(model) {
|
||||
Ok(model) => model,
|
||||
Err(_) => return invalid_request("provider model identifier is invalid"),
|
||||
Err(_) => return invalid_request("Provider model identifier is invalid."),
|
||||
};
|
||||
if serde_json::from_slice::<serde::de::IgnoredAny>(&body).is_err() {
|
||||
return invalid_request("token count request must be valid JSON");
|
||||
return invalid_request("Token count request must be valid JSON.");
|
||||
}
|
||||
let payload = match RawJsonPayload::new(TOKEN_COUNT_PROTOCOL, body) {
|
||||
Ok(payload) => payload,
|
||||
Err(_) => return invalid_request("token count request is invalid"),
|
||||
Err(_) => return invalid_request("Token count request is invalid."),
|
||||
};
|
||||
let (client_ip, user_agent) = request_client_context(
|
||||
&headers,
|
||||
@@ -124,12 +124,12 @@ async fn provider_http(
|
||||
body: Bytes,
|
||||
) -> Response {
|
||||
if method == Method::GET && !body.is_empty() {
|
||||
return invalid_request("GET provider HTTP operations cannot carry a request body");
|
||||
return invalid_request("GET provider HTTP operations cannot carry a request body.");
|
||||
}
|
||||
let method = match method {
|
||||
Method::GET => ProviderHttpMethod::Get,
|
||||
Method::POST => ProviderHttpMethod::Post,
|
||||
_ => return invalid_request("provider HTTP method is unsupported"),
|
||||
_ => return invalid_request("Provider HTTP method is unsupported."),
|
||||
};
|
||||
let service = state.openai();
|
||||
let client = match authenticate_client(service, &headers).await {
|
||||
@@ -138,7 +138,7 @@ async fn provider_http(
|
||||
};
|
||||
let provider = match ProviderKind::new(provider) {
|
||||
Ok(provider) => provider,
|
||||
Err(_) => return invalid_request("provider identifier is invalid"),
|
||||
Err(_) => return invalid_request("Provider identifier is invalid."),
|
||||
};
|
||||
let connection_options = headers
|
||||
.get_all(header::CONNECTION)
|
||||
@@ -158,7 +158,7 @@ async fn provider_http(
|
||||
.collect();
|
||||
let payload = match RawHttpPayload::new(PROVIDER_HTTP_PROTOCOL, body) {
|
||||
Ok(payload) => payload,
|
||||
Err(_) => return invalid_request("provider HTTP request is invalid"),
|
||||
Err(_) => return invalid_request("Provider HTTP request is invalid."),
|
||||
};
|
||||
let operation = match ProviderHttpRequest::new(
|
||||
endpoint,
|
||||
@@ -168,7 +168,7 @@ async fn provider_http(
|
||||
payload,
|
||||
) {
|
||||
Ok(request) => Operation::ProviderHttp(request),
|
||||
Err(_) => return invalid_request("provider HTTP request is invalid"),
|
||||
Err(_) => return invalid_request("Provider HTTP request is invalid."),
|
||||
};
|
||||
let (client_ip, user_agent) = request_client_context(
|
||||
&headers,
|
||||
|
||||
@@ -9,6 +9,86 @@ use tower::ServiceExt as _;
|
||||
|
||||
use super::super::{AdminTestFixture, AdminTestState};
|
||||
|
||||
#[tokio::test]
|
||||
async fn account_model_reads_and_refreshes_preserve_provider_observation_time() {
|
||||
use chrono::{DateTime, Utc};
|
||||
use gateway_admin::model::provider_credentials::{ProviderModel, ProviderModels};
|
||||
use gateway_core::{
|
||||
account::{AccountStatusFacts, CredentialState, QuotaState},
|
||||
routing::UpstreamModelId,
|
||||
};
|
||||
use serde_json::json;
|
||||
|
||||
let fixture = AdminTestFixture::new().await;
|
||||
fixture.auth.insert_session("valid-session");
|
||||
let observed_at: DateTime<Utc> = "2026-09-12T08:00:00Z".parse().unwrap();
|
||||
*fixture.account.lock().unwrap() = Some(account_page_item(
|
||||
AccountStatusFacts {
|
||||
enabled: true,
|
||||
credential_state: CredentialState::Ready,
|
||||
access_token_expires_at: None,
|
||||
quota: QuotaState::unknown(),
|
||||
cooldown: None,
|
||||
last_error_reason: None,
|
||||
last_error_message: None,
|
||||
},
|
||||
observed_at,
|
||||
"api_key",
|
||||
));
|
||||
let router = admin::accounts::router::<AdminTestState>().with_state(fixture.state());
|
||||
for observed_at in [Some(observed_at), None] {
|
||||
*fixture.provider_models.lock().unwrap() = Some(ProviderModels {
|
||||
models: vec![ProviderModel {
|
||||
id: UpstreamModelId::new("model-a").unwrap(),
|
||||
name: "Model A".to_owned(),
|
||||
}],
|
||||
observed_at,
|
||||
});
|
||||
for (method, uri, body) in [
|
||||
(
|
||||
"GET",
|
||||
"/api/admin/accounts/models?accountId=acct_cooldown",
|
||||
"",
|
||||
),
|
||||
(
|
||||
"POST",
|
||||
"/api/admin/accounts/models/refresh",
|
||||
r#"{"accountId":"acct_cooldown"}"#,
|
||||
),
|
||||
] {
|
||||
let response = router
|
||||
.clone()
|
||||
.oneshot(
|
||||
Request::builder()
|
||||
.method(method)
|
||||
.uri(uri)
|
||||
.header("x-request-id", "req_model_catalog")
|
||||
.header(header::COOKIE, "cpr_session=valid-session")
|
||||
.header(header::CONTENT_TYPE, "application/json")
|
||||
.body(Body::from(body))
|
||||
.unwrap(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(response.status(), StatusCode::OK, "{method} {uri}");
|
||||
let body = to_bytes(response.into_body(), 8192).await.unwrap();
|
||||
let value: serde_json::Value = serde_json::from_slice(&body).unwrap();
|
||||
assert_eq!(
|
||||
value["data"]["models"],
|
||||
json!([{ "id": "model-a", "label": "model-a" }])
|
||||
);
|
||||
assert_eq!(
|
||||
value["data"]["observedAt"],
|
||||
json!(observed_at.map(|_| "2026-09-12T16:00:00+08:00"))
|
||||
);
|
||||
assert_eq!(
|
||||
value["data"]["observedAtDisplay"],
|
||||
observed_at.map_or("—", |_| "2026-09-12 16:00:00")
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn account_proxy_binding_is_distinct_from_the_redacted_endpoint() {
|
||||
use chrono::Utc;
|
||||
@@ -91,17 +171,17 @@ async fn account_cooldown_returns_an_instant_and_server_formatted_recovery() {
|
||||
),
|
||||
(
|
||||
Some(7200),
|
||||
AccountCooldownKind::CapacityFreeze,
|
||||
AccountCooldownKind::ErrorProtection,
|
||||
Some("剩余 2 小时"),
|
||||
),
|
||||
(
|
||||
Some(7500),
|
||||
AccountCooldownKind::CapacityFreezeProbe,
|
||||
AccountCooldownKind::ErrorProtectionProbe,
|
||||
Some("剩余 2 小时 5 分"),
|
||||
),
|
||||
(
|
||||
Some(-60),
|
||||
AccountCooldownKind::CapacityFreezeProbe,
|
||||
AccountCooldownKind::ErrorProtectionProbe,
|
||||
Some("等待探测成功"),
|
||||
),
|
||||
(Some(20), AccountCooldownKind::RateLimit, None),
|
||||
@@ -121,7 +201,14 @@ async fn account_cooldown_returns_an_instant_and_server_formatted_recovery() {
|
||||
last_error_reason: None,
|
||||
last_error_message: None,
|
||||
};
|
||||
*fixture.account.lock().unwrap() = Some(account_page_item(facts, now, "oauth"));
|
||||
let mut item = account_page_item(facts, now, "oauth");
|
||||
if kind.is_error_protection() {
|
||||
item.protection_rules = vec![gateway_core::account::AccountProtectionMatch {
|
||||
id: "removed-rule".to_owned(),
|
||||
name: "触发时的规则名称".to_owned(),
|
||||
}];
|
||||
}
|
||||
*fixture.account.lock().unwrap() = Some(item);
|
||||
let response = admin::accounts::router::<AdminTestState>()
|
||||
.with_state(fixture.state())
|
||||
.oneshot(
|
||||
@@ -138,6 +225,17 @@ async fn account_cooldown_returns_an_instant_and_server_formatted_recovery() {
|
||||
let body = to_bytes(response.into_body(), 32768).await.unwrap();
|
||||
let value: serde_json::Value = serde_json::from_slice(&body).unwrap();
|
||||
let quota = &value["data"]["items"][0]["quota"];
|
||||
if kind.is_error_protection() {
|
||||
assert_eq!(quota["rateLimitReason"], "error_protection");
|
||||
assert_eq!(
|
||||
quota["protectionRules"],
|
||||
serde_json::json!([
|
||||
{"id":"removed-rule", "name":"触发时的规则名称"}
|
||||
])
|
||||
);
|
||||
} else {
|
||||
assert_eq!(quota["protectionRules"], serde_json::json!([]));
|
||||
}
|
||||
match until {
|
||||
Some(until) => {
|
||||
assert_eq!(
|
||||
@@ -256,6 +354,7 @@ fn account_page_item(
|
||||
use gateway_core::account::resolve_account_status;
|
||||
|
||||
AccountPageItem {
|
||||
protection_rules: Vec::new(),
|
||||
account: AccountRecord {
|
||||
outbound_proxy_id: None,
|
||||
outbound_proxy_enabled: true,
|
||||
|
||||
@@ -69,7 +69,7 @@ mod personal_info {
|
||||
fn personal_info_response_preserves_unknown_subscription_and_profile_error() {
|
||||
let response = AccountPersonalInfoData::from((
|
||||
AccountPersonalInfo {
|
||||
profile: Err(AdminError::unavailable("Provider 服务暂不可用")),
|
||||
profile: Err(AdminError::unavailable("服务商暂不可用")),
|
||||
subscription: None,
|
||||
},
|
||||
gateway_api::TimePresenter::new(Default::default()),
|
||||
@@ -78,7 +78,7 @@ mod personal_info {
|
||||
serde_json::to_value(response).unwrap(),
|
||||
json!({
|
||||
"profile": null,
|
||||
"profileError": "Provider 服务暂不可用",
|
||||
"profileError": "服务商暂不可用",
|
||||
"subscription": null,
|
||||
})
|
||||
);
|
||||
|
||||
@@ -171,6 +171,70 @@ async fn malformed_admin_query_should_use_a_safe_bad_request_envelope() {
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn client_key_validation_should_identify_the_invalid_field_without_echoing_values() {
|
||||
let fixture = AdminTestFixture::new().await;
|
||||
fixture.auth.insert_session("valid-session");
|
||||
let secret = "private-key-must-not-leak\n";
|
||||
for (field, value, message) in [
|
||||
("name", json!(" "), "API Key 名称不能为空"),
|
||||
(
|
||||
"customKey",
|
||||
json!(secret),
|
||||
"自定义 API Key 仅支持英文字母、数字和英文符号",
|
||||
),
|
||||
(
|
||||
"dailyLimitUsd",
|
||||
json!("-1"),
|
||||
"每日美元限额须非负,整数和小数各最多 10 位",
|
||||
),
|
||||
(
|
||||
"weeklyLimitUsd",
|
||||
json!("0.00000000001"),
|
||||
"每周美元限额须非负,整数和小数各最多 10 位",
|
||||
),
|
||||
(
|
||||
"maxConcurrency",
|
||||
json!(u64::MAX),
|
||||
"并发请求上限超出支持范围",
|
||||
),
|
||||
] {
|
||||
for path in [
|
||||
"/api/admin/client-keys/create",
|
||||
"/api/admin/client-keys/update",
|
||||
] {
|
||||
if path.ends_with("/update") && field == "customKey" {
|
||||
continue;
|
||||
}
|
||||
let mut payload = json!({
|
||||
"name": "key",
|
||||
"groupIds": [],
|
||||
"maxConcurrency": 0,
|
||||
"requestsPerMinute": 0,
|
||||
});
|
||||
if path.ends_with("/update") {
|
||||
payload["id"] = json!("key_existing");
|
||||
}
|
||||
payload[field] = value.clone();
|
||||
let mut request = request(Method::POST, path, Body::from(payload.to_string()));
|
||||
request
|
||||
.headers_mut()
|
||||
.insert(header::COOKIE, SESSION_COOKIE.parse().unwrap());
|
||||
request
|
||||
.headers_mut()
|
||||
.insert(header::CONTENT_TYPE, "application/json".parse().unwrap());
|
||||
let response = app(fixture.state()).oneshot(request).await.unwrap();
|
||||
let (status, _, body) = response_json(response).await;
|
||||
assert_eq!(status, StatusCode::BAD_REQUEST);
|
||||
assert_eq!(
|
||||
body,
|
||||
json!({"code": 40001, "message": message, "data": null})
|
||||
);
|
||||
assert!(!body.to_string().contains(secret.trim()));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn invalid_time_range_should_use_its_published_business_code() {
|
||||
let fixture = AdminTestFixture::new().await;
|
||||
@@ -445,8 +509,10 @@ mod provider {
|
||||
let fixture = AdminTestFixture::new().await;
|
||||
fixture.auth.insert_session("valid-session");
|
||||
for (kind, status, code, message) in [
|
||||
(Kind::Invalid, 400, 40001, "服务商请求参数不合法"),
|
||||
(Kind::Unsupported, 400, 40001, "该服务商不支持此操作"),
|
||||
(Kind::BadGateway, 502, 50201, "上游服务请求失败"),
|
||||
(Kind::Unavailable, 503, 50301, "Provider 服务暂不可用"),
|
||||
(Kind::Unavailable, 503, 50301, "服务商暂不可用"),
|
||||
(
|
||||
Kind::Ambiguous,
|
||||
502,
|
||||
@@ -506,6 +572,7 @@ mod provider {
|
||||
last_error_message: None,
|
||||
};
|
||||
AccountPageItem {
|
||||
protection_rules: Vec::new(),
|
||||
capacity: gateway_admin::model::accounts::AccountCapacity {
|
||||
used_slots: None,
|
||||
total_slots: None,
|
||||
|
||||
@@ -113,6 +113,7 @@ pub(super) struct AdminTestFixture {
|
||||
pub dashboard_summary_range: Arc<Mutex<Option<TimeRange>>>,
|
||||
pub provider_error: Arc<Mutex<Option<ProviderAdminError>>>,
|
||||
pub provider_quota: Arc<Mutex<Option<ProviderQuota>>>,
|
||||
pub provider_models: Arc<Mutex<Option<ProviderModels>>>,
|
||||
pub account: Arc<Mutex<Option<AccountPageItem>>>,
|
||||
pub account_usage: Arc<Mutex<Vec<AccountUsage>>>,
|
||||
}
|
||||
@@ -189,16 +190,19 @@ impl AdminTestFixture {
|
||||
plugin_ports.clone(),
|
||||
);
|
||||
let provider_quota = Arc::new(Mutex::new(None));
|
||||
let provider_models = Arc::new(Mutex::new(None));
|
||||
let providers: Vec<Arc<dyn ProviderAdmin>> = vec![
|
||||
Arc::new(UnusedProvider::new(
|
||||
"openai",
|
||||
Arc::clone(&provider_error),
|
||||
Arc::clone(&provider_quota),
|
||||
Arc::clone(&provider_models),
|
||||
)),
|
||||
Arc::new(UnusedProvider::new(
|
||||
"xai",
|
||||
Arc::clone(&provider_error),
|
||||
Arc::clone(&provider_quota),
|
||||
Arc::clone(&provider_models),
|
||||
)),
|
||||
];
|
||||
let bundle = gateway_admin::initialize(
|
||||
@@ -247,6 +251,7 @@ impl AdminTestFixture {
|
||||
dashboard_summary_range,
|
||||
provider_error,
|
||||
provider_quota,
|
||||
provider_models,
|
||||
account,
|
||||
account_usage,
|
||||
}
|
||||
@@ -647,13 +652,7 @@ impl SettingsStore for MemorySettingsStore {
|
||||
usage_retention_days: command.values.usage_retention_days,
|
||||
ops_event_retention_days: command.values.ops_event_retention_days,
|
||||
audit_retention_days: command.values.audit_retention_days,
|
||||
account_auto_freeze_enabled: true,
|
||||
account_auto_freeze_threshold: 12,
|
||||
account_auto_freeze_window_seconds: 600,
|
||||
account_auto_freeze_duration_seconds: 7_200,
|
||||
account_auto_freeze_probe_enabled: true,
|
||||
account_auto_freeze_probe_model: None,
|
||||
account_auto_freeze_adaptive_concurrency: true,
|
||||
account_error_protection: command.values.account_error_protection,
|
||||
account_warmup_enabled: false,
|
||||
account_warmup_schedule_time: "08:00".to_owned(),
|
||||
account_warmup_model: None,
|
||||
@@ -697,7 +696,9 @@ pub(super) struct MemoryClientKeyStore(Arc<Mutex<Option<ClientKeyRecord>>>);
|
||||
pub(super) struct MemoryObservations {
|
||||
pub summary: Option<UsageOverview>,
|
||||
pub trend: Option<Vec<RequestMetricPoint>>,
|
||||
pub concurrency: Vec<gateway_admin::model::observability::UsageConcurrencyPoint>,
|
||||
pub summaries: Vec<(TimeRange, UsageFilter)>,
|
||||
pub concurrency_selections: Vec<bool>,
|
||||
pub trends: Vec<(TimeRange, UsageFilter)>,
|
||||
pub records: Vec<UsageQuery>,
|
||||
pub errors: Vec<OpsErrorQuery>,
|
||||
@@ -1326,15 +1327,6 @@ impl AccountStore for UnusedStore {
|
||||
Err(unavailable("account enabled"))
|
||||
}
|
||||
|
||||
async fn lower_concurrency_limit(
|
||||
&self,
|
||||
_: &gateway_core::account::ProviderAccountId,
|
||||
_: gateway_core::account::AccountConcurrencyLimit,
|
||||
_: &MutationContext,
|
||||
) -> AdminStoreResult<Option<gateway_admin::model::accounts::AccountUpdateResult>> {
|
||||
Ok(None)
|
||||
}
|
||||
|
||||
async fn recover_account(
|
||||
&self,
|
||||
_: &ProviderAccountId,
|
||||
@@ -1399,6 +1391,7 @@ impl AccountRuntimeStore for UnusedStore {
|
||||
async fn active_rate_limits(&self) -> AdminStoreResult<AccountRuntimeSnapshot> {
|
||||
Ok(AccountRuntimeSnapshot {
|
||||
cooldown: BTreeMap::new(),
|
||||
protection_rules: BTreeMap::new(),
|
||||
in_flight: Some(BTreeMap::new()),
|
||||
})
|
||||
}
|
||||
@@ -1406,6 +1399,7 @@ impl AccountRuntimeStore for UnusedStore {
|
||||
async fn account_runtime(&self, _: &[String]) -> AdminStoreResult<AccountRuntimeSnapshot> {
|
||||
Ok(AccountRuntimeSnapshot {
|
||||
cooldown: BTreeMap::new(),
|
||||
protection_rules: BTreeMap::new(),
|
||||
in_flight: Some(BTreeMap::new()),
|
||||
})
|
||||
}
|
||||
@@ -1416,13 +1410,6 @@ impl AccountRuntimeStore for UnusedStore {
|
||||
Ok(BTreeMap::new())
|
||||
}
|
||||
|
||||
async fn capacity_peaks(
|
||||
&self,
|
||||
_account_ids: &[String],
|
||||
) -> AdminStoreResult<BTreeMap<String, u32>> {
|
||||
Ok(BTreeMap::new())
|
||||
}
|
||||
|
||||
async fn finish_freeze(
|
||||
&self,
|
||||
_account_id: &str,
|
||||
@@ -1510,8 +1497,10 @@ impl ObservabilityStore for UnusedStore {
|
||||
range: TimeRange,
|
||||
filter: UsageFilter,
|
||||
_: Granularity,
|
||||
include_concurrency: bool,
|
||||
) -> AdminStoreResult<gateway_admin::model::observability::UsageInsightsObservation> {
|
||||
let mut data = self.observations.lock().expect("observations");
|
||||
data.concurrency_selections.push(include_concurrency);
|
||||
data.summaries.push((range, filter.clone()));
|
||||
data.trends.push((range, filter));
|
||||
Ok(
|
||||
@@ -1526,10 +1515,31 @@ impl ObservabilityStore for UnusedStore {
|
||||
.clone()
|
||||
.ok_or_else(|| unavailable("usage trend"))?,
|
||||
costs: Vec::new(),
|
||||
concurrency: if include_concurrency {
|
||||
data.concurrency.clone()
|
||||
} else {
|
||||
Vec::new()
|
||||
},
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
async fn usage_metric_observation(
|
||||
&self,
|
||||
range: gateway_admin::model::observability::TimeRange,
|
||||
filter: gateway_admin::model::observability::UsageFilter,
|
||||
granularity: gateway_admin::model::observability::Granularity,
|
||||
metric: gateway_admin::model::observability::UsageInsightMetric,
|
||||
) -> AdminStoreResult<gateway_admin::model::observability::UsageInsightsObservation> {
|
||||
self.usage_insights_observation(
|
||||
range,
|
||||
filter,
|
||||
granularity,
|
||||
metric == gateway_admin::model::observability::UsageInsightMetric::Load,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
async fn list_usage_records(&self, query: UsageQuery) -> AdminStoreResult<UsagePage> {
|
||||
self.observations
|
||||
.lock()
|
||||
@@ -1602,6 +1612,7 @@ struct UnusedProvider {
|
||||
kind: ProviderKind,
|
||||
error: Arc<Mutex<Option<ProviderAdminError>>>,
|
||||
quota: Arc<Mutex<Option<ProviderQuota>>>,
|
||||
models: Arc<Mutex<Option<ProviderModels>>>,
|
||||
}
|
||||
|
||||
impl UnusedProvider {
|
||||
@@ -1609,11 +1620,13 @@ impl UnusedProvider {
|
||||
kind: &str,
|
||||
error: Arc<Mutex<Option<ProviderAdminError>>>,
|
||||
quota: Arc<Mutex<Option<ProviderQuota>>>,
|
||||
models: Arc<Mutex<Option<ProviderModels>>>,
|
||||
) -> Self {
|
||||
Self {
|
||||
kind: ProviderKind::new(kind).expect("provider kind"),
|
||||
error,
|
||||
quota,
|
||||
models,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1724,7 +1737,11 @@ impl ProviderAdmin for UnusedProvider {
|
||||
_: &ProviderAccountId,
|
||||
_: bool,
|
||||
) -> Result<ProviderModels, ProviderAdminError> {
|
||||
Err(unsupported_provider())
|
||||
self.models
|
||||
.lock()
|
||||
.expect("provider models")
|
||||
.clone()
|
||||
.ok_or_else(unsupported_provider)
|
||||
}
|
||||
|
||||
async fn export_credentials(
|
||||
@@ -1841,13 +1858,7 @@ fn test_runtime_settings() -> RuntimeSettings {
|
||||
usage_retention_days: 31,
|
||||
ops_event_retention_days: 30,
|
||||
audit_retention_days: 90,
|
||||
account_auto_freeze_enabled: true,
|
||||
account_auto_freeze_threshold: 12,
|
||||
account_auto_freeze_window_seconds: 600,
|
||||
account_auto_freeze_duration_seconds: 7_200,
|
||||
account_auto_freeze_probe_enabled: true,
|
||||
account_auto_freeze_probe_model: None,
|
||||
account_auto_freeze_adaptive_concurrency: true,
|
||||
account_error_protection: Default::default(),
|
||||
account_warmup_enabled: false,
|
||||
account_warmup_schedule_time: "08:00".to_owned(),
|
||||
account_warmup_model: None,
|
||||
|
||||
@@ -1286,6 +1286,253 @@ async fn calendar_query_rejects_an_anchor_outside_the_local_calendar() {
|
||||
assert!(fixture.dashboard_summary_range.lock().unwrap().is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn usage_insights_maps_load_summary_and_points_to_camel_case() {
|
||||
use crate::admin::{AdminTestFixture, AdminTestState};
|
||||
use axum::{
|
||||
body::{Body, to_bytes},
|
||||
http::{Request, StatusCode, header},
|
||||
};
|
||||
use chrono::Duration;
|
||||
use gateway_admin::model::observability::{
|
||||
Granularity, LatencyPercentiles, PercentileMilliseconds, RequestMetricPoint,
|
||||
RequestMetrics, TimeRange, UsageConcurrencyPoint, UsageOverview,
|
||||
};
|
||||
use gateway_api::admin::observability;
|
||||
use tower::ServiceExt as _;
|
||||
|
||||
let fixture = AdminTestFixture::new().await;
|
||||
fixture.auth.insert_session("valid-session");
|
||||
let now = Utc::now();
|
||||
let range = TimeRange::new(now - Duration::minutes(15), now).unwrap();
|
||||
let bucket = range.start;
|
||||
{
|
||||
let mut observations = fixture.observations.lock().unwrap();
|
||||
observations.summary = Some(UsageOverview {
|
||||
analytics: Default::default(),
|
||||
range,
|
||||
requests: RequestMetrics {
|
||||
admission_decision_percentiles: LatencyPercentiles {
|
||||
p95_ms: Some(PercentileMilliseconds::new(12.5).unwrap()),
|
||||
..Default::default()
|
||||
},
|
||||
account_selection_wait_percentiles: LatencyPercentiles {
|
||||
p95_ms: Some(PercentileMilliseconds::new(128.75).unwrap()),
|
||||
..Default::default()
|
||||
},
|
||||
..Default::default()
|
||||
},
|
||||
attempts: Default::default(),
|
||||
providers: Vec::new(),
|
||||
});
|
||||
observations.trend = Some(vec![RequestMetricPoint {
|
||||
bucket_start: bucket,
|
||||
granularity: Granularity::FifteenMinutes,
|
||||
metrics: RequestMetrics {
|
||||
admission_decision_percentiles: LatencyPercentiles {
|
||||
p95_ms: Some(PercentileMilliseconds::new(2.5).unwrap()),
|
||||
..Default::default()
|
||||
},
|
||||
..Default::default()
|
||||
},
|
||||
cost_coverage: Default::default(),
|
||||
costs: Vec::new(),
|
||||
}]);
|
||||
observations.concurrency = vec![UsageConcurrencyPoint {
|
||||
bucket_start: bucket,
|
||||
average_concurrency: 1.25,
|
||||
peak_concurrency: 3,
|
||||
}];
|
||||
}
|
||||
|
||||
let response = observability::router::<AdminTestState>()
|
||||
.with_state(fixture.state())
|
||||
.oneshot(
|
||||
Request::builder()
|
||||
.uri("/api/admin/usage/insights/overview")
|
||||
.header(header::COOKIE, "cpr_session=valid-session")
|
||||
.header("x-request-id", "req_usage_insights_load")
|
||||
.body(Body::empty())
|
||||
.unwrap(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let status = response.status();
|
||||
let body = to_bytes(response.into_body(), 1024 * 1024).await.unwrap();
|
||||
let value: serde_json::Value = serde_json::from_slice(&body).unwrap();
|
||||
assert_eq!(status, StatusCode::OK, "{value}");
|
||||
let performance = &value["data"]["performance"];
|
||||
assert_eq!(
|
||||
performance["load"],
|
||||
json!({
|
||||
"averageConcurrency": 1.25,
|
||||
"peakConcurrency": 3,
|
||||
"admissionDecisionP95Ms": 12.5,
|
||||
"accountSelectionWaitP95Ms": 128.75,
|
||||
})
|
||||
);
|
||||
assert_eq!(
|
||||
performance["points"][0]["load"],
|
||||
json!({
|
||||
"averageConcurrency": 1.25,
|
||||
"peakConcurrency": 3,
|
||||
"admissionDecisionP95Ms": 2.5,
|
||||
"accountSelectionWaitP95Ms": null,
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn usage_card_routes_return_only_the_selected_metric_and_validate_options() {
|
||||
use crate::admin::{AdminTestFixture, AdminTestState};
|
||||
use axum::{
|
||||
body::{Body, to_bytes},
|
||||
http::{Request, StatusCode, header},
|
||||
};
|
||||
use chrono::Duration;
|
||||
use gateway_admin::model::observability::{
|
||||
AttemptMetrics, CurrencyCost, Granularity, LatencyPercentiles, PercentileMilliseconds,
|
||||
RequestMetricPoint, RequestMetrics, TimeRange, UsageConcurrencyPoint, UsageOverview,
|
||||
};
|
||||
use gateway_api::admin::observability;
|
||||
use tower::ServiceExt as _;
|
||||
|
||||
let fixture = AdminTestFixture::new().await;
|
||||
fixture.auth.insert_session("valid-session");
|
||||
let now = Utc::now();
|
||||
let range = TimeRange::new(now - Duration::minutes(15), now).unwrap();
|
||||
let metrics = RequestMetrics {
|
||||
request_count: 10,
|
||||
success_count: 8,
|
||||
failure_count: 2,
|
||||
latency_percentiles: LatencyPercentiles {
|
||||
p95_ms: Some(PercentileMilliseconds::new(43.5).unwrap()),
|
||||
..Default::default()
|
||||
},
|
||||
first_token_latency_percentiles: LatencyPercentiles {
|
||||
p95_ms: Some(PercentileMilliseconds::new(17.25).unwrap()),
|
||||
..Default::default()
|
||||
},
|
||||
output_throughput_p50: Some(22),
|
||||
..Default::default()
|
||||
};
|
||||
{
|
||||
let mut observations = fixture.observations.lock().unwrap();
|
||||
observations.summary = Some(UsageOverview {
|
||||
analytics: Default::default(),
|
||||
range,
|
||||
requests: metrics.clone(),
|
||||
attempts: AttemptMetrics {
|
||||
costs: vec![CurrencyCost {
|
||||
currency: "USD".into(),
|
||||
amount: "12.5".parse().unwrap(),
|
||||
}],
|
||||
..Default::default()
|
||||
},
|
||||
providers: Vec::new(),
|
||||
});
|
||||
observations.trend = Some(vec![RequestMetricPoint {
|
||||
bucket_start: range.start,
|
||||
granularity: Granularity::FifteenMinutes,
|
||||
metrics,
|
||||
cost_coverage: Default::default(),
|
||||
costs: Vec::new(),
|
||||
}]);
|
||||
observations.concurrency = vec![UsageConcurrencyPoint {
|
||||
bucket_start: range.start,
|
||||
average_concurrency: 1.25,
|
||||
peak_concurrency: 3,
|
||||
}];
|
||||
}
|
||||
let router = observability::router::<AdminTestState>().with_state(fixture.state());
|
||||
for (path, metric, field, expected) in [
|
||||
("health", "health", "totalRequests", json!(10)),
|
||||
(
|
||||
"performance?metric=latency",
|
||||
"latency",
|
||||
"p95Ms",
|
||||
json!(43.5),
|
||||
),
|
||||
(
|
||||
"performance?metric=firstToken",
|
||||
"firstToken",
|
||||
"p95Ms",
|
||||
json!(17.25),
|
||||
),
|
||||
(
|
||||
"performance?metric=throughput",
|
||||
"throughput",
|
||||
"p50",
|
||||
json!(22),
|
||||
),
|
||||
(
|
||||
"performance?metric=load",
|
||||
"load",
|
||||
"peakConcurrency",
|
||||
json!(3),
|
||||
),
|
||||
("cost?metric=cost", "cost", "estimatedCost", json!("12.5")),
|
||||
(
|
||||
"cost?metric=savings",
|
||||
"savings",
|
||||
"estimatedCost",
|
||||
json!("12.5"),
|
||||
),
|
||||
("cost?metric=cache", "cache", "estimatedCost", json!("12.5")),
|
||||
] {
|
||||
let response = router
|
||||
.clone()
|
||||
.oneshot(
|
||||
Request::builder()
|
||||
.uri(format!("/api/admin/usage/insights/{path}"))
|
||||
.header(header::COOKIE, "cpr_session=valid-session")
|
||||
.header("x-request-id", "req_usage_card_metric")
|
||||
.body(Body::empty())
|
||||
.unwrap(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let status = response.status();
|
||||
let body = to_bytes(response.into_body(), 1024 * 1024).await.unwrap();
|
||||
let value: serde_json::Value = serde_json::from_slice(&body).unwrap();
|
||||
assert_eq!(status, StatusCode::OK, "{path}: {value}");
|
||||
let result = &value["data"];
|
||||
assert_eq!(result["metric"], metric);
|
||||
assert_eq!(result.as_object().unwrap().len(), 4);
|
||||
let data = &result["data"];
|
||||
if metric == "health" {
|
||||
assert_eq!(data[field], expected);
|
||||
} else {
|
||||
assert_eq!(data["summary"][field], expected, "{path}");
|
||||
assert_eq!(data["points"][0]["requestCount"], 10);
|
||||
assert_eq!(data.as_object().unwrap().len(), 2);
|
||||
}
|
||||
}
|
||||
for path in [
|
||||
"performance",
|
||||
"cost",
|
||||
"performance?metric=cost",
|
||||
"cost?metric=load",
|
||||
"health?metric=load",
|
||||
"performance?metric=unknown",
|
||||
"cost?metric=cache&status=error",
|
||||
] {
|
||||
let response = router
|
||||
.clone()
|
||||
.oneshot(
|
||||
Request::builder()
|
||||
.uri(format!("/api/admin/usage/insights/{path}"))
|
||||
.header(header::COOKIE, "cpr_session=valid-session")
|
||||
.header("x-request-id", "req_usage_card_metric")
|
||||
.body(Body::empty())
|
||||
.unwrap(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(response.status(), StatusCode::BAD_REQUEST, "{path}");
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn observability_queries_reject_group_filters_and_analytics_search() {
|
||||
use crate::admin::{AdminTestFixture, AdminTestState};
|
||||
@@ -1511,6 +1758,7 @@ async fn dashboard_summary_should_identify_metrics_without_localized_labels() {
|
||||
authentication_kind: "oauth".to_owned(),
|
||||
name: account_id.clone(),
|
||||
email: None,
|
||||
notes: None,
|
||||
plan_type: None,
|
||||
plan_type_display: None,
|
||||
request_count: 10,
|
||||
@@ -1709,6 +1957,7 @@ async fn dashboard_relative_time_preserves_server_anchor_boundaries_and_deployme
|
||||
authentication_kind: "oauth".to_owned(),
|
||||
name: "Relative time fixture".to_owned(),
|
||||
email: None,
|
||||
notes: None,
|
||||
plan_type: None,
|
||||
plan_type_display: None,
|
||||
request_count: 1,
|
||||
|
||||
@@ -65,13 +65,7 @@ fn update_body() -> Value {
|
||||
"usageRetentionDays": 31,
|
||||
"opsEventRetentionDays": 31,
|
||||
"auditRetentionDays": 91,
|
||||
"accountAutoFreezeEnabled": true,
|
||||
"accountAutoFreezeThreshold": 12,
|
||||
"accountAutoFreezeWindowSeconds": 600,
|
||||
"accountAutoFreezeDurationSeconds": 7200,
|
||||
"accountAutoFreezeProbeEnabled": true,
|
||||
"accountAutoFreezeProbeModel": null,
|
||||
"accountAutoFreezeAdaptiveConcurrency": true,
|
||||
"accountErrorProtection": gateway_core::account::AccountErrorProtection::default(),
|
||||
"accountWarmupEnabled": false,
|
||||
"accountWarmupScheduleTime": "08:00",
|
||||
"accountWarmupModel": null,
|
||||
@@ -185,6 +179,87 @@ fn privacy_policy_is_required_and_rejects_unknown_contract_fields() {
|
||||
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(unknown).is_err());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn error_protection_rules_round_trip_and_invalid_policy_does_not_replace_them() {
|
||||
let fixture = AdminTestFixture::new().await;
|
||||
fixture.auth.insert_session("valid-session");
|
||||
let mut body = update_body();
|
||||
let policy = json!({
|
||||
"enabled": true,
|
||||
"rules": [{
|
||||
"id": "custom-rejection", "name": "自定义上游拒绝",
|
||||
"conditions": [
|
||||
{"field": "provider", "operator": "equals", "values": ["openai", "xai"], "ignoreCase": false},
|
||||
{"field": "code", "operator": "equals", "values": ["custom_vendor_rejection", null], "ignoreCase": true},
|
||||
{"field": "message", "operator": "contains", "values": ["please pause"], "ignoreCase": false}
|
||||
]
|
||||
}],
|
||||
"threshold": 3, "windowSeconds": 120, "durationSeconds": 300,
|
||||
"probeEnabled": false, "probeModel": null
|
||||
});
|
||||
body["accountErrorProtection"] = policy.clone();
|
||||
let response = app(fixture.state())
|
||||
.oneshot(request(
|
||||
Method::POST,
|
||||
"/api/admin/settings/update",
|
||||
Some(body.clone()),
|
||||
))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
let saved = response_json(response).await;
|
||||
assert_eq!(saved["data"]["accountErrorProtection"], policy);
|
||||
body["configRevision"] = saved["data"]["configRevision"].clone();
|
||||
|
||||
let mut invalid_policies = Vec::new();
|
||||
let mut empty = policy.clone();
|
||||
empty["rules"] = json!([]);
|
||||
invalid_policies.push(empty);
|
||||
let mut provider_only = policy.clone();
|
||||
provider_only["rules"][0]["conditions"] = json!([
|
||||
{"field": "provider", "operator": "equals", "values": ["openai"], "ignoreCase": false}
|
||||
]);
|
||||
invalid_policies.push(provider_only);
|
||||
let mut empty_contains = policy.clone();
|
||||
empty_contains["rules"][0]["conditions"][2]["values"] = json!([" "]);
|
||||
invalid_policies.push(empty_contains);
|
||||
for invalid in invalid_policies {
|
||||
body["accountErrorProtection"] = invalid;
|
||||
let response = app(fixture.state())
|
||||
.oneshot(request(
|
||||
Method::POST,
|
||||
"/api/admin/settings/update",
|
||||
Some(body.clone()),
|
||||
))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
|
||||
}
|
||||
let response = app(fixture.state())
|
||||
.oneshot(request(Method::GET, "/api/admin/settings", None))
|
||||
.await
|
||||
.unwrap();
|
||||
let loaded = response_json(response).await;
|
||||
assert_eq!(loaded["data"]["accountErrorProtection"], policy);
|
||||
assert_eq!(
|
||||
loaded["data"]["configRevision"],
|
||||
saved["data"]["configRevision"]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn error_protection_requires_an_explicit_policy_in_settings_replacement() {
|
||||
let mut missing = update_body();
|
||||
missing
|
||||
.as_object_mut()
|
||||
.unwrap()
|
||||
.remove("accountErrorProtection");
|
||||
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(missing).is_err());
|
||||
let mut unknown = update_body();
|
||||
unknown["accountErrorProtection"]["unknownField"] = json!(true);
|
||||
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(unknown).is_err());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn smart_settings_round_trip_and_invalid_updates_leave_the_saved_value_intact() {
|
||||
let fixture = AdminTestFixture::new().await;
|
||||
@@ -400,13 +475,7 @@ fn settings_response_should_cover_the_full_runtime_settings_contract() {
|
||||
usage_retention_days: 31,
|
||||
ops_event_retention_days: 31,
|
||||
audit_retention_days: 91,
|
||||
account_auto_freeze_enabled: true,
|
||||
account_auto_freeze_threshold: 12,
|
||||
account_auto_freeze_window_seconds: 600,
|
||||
account_auto_freeze_duration_seconds: 7_200,
|
||||
account_auto_freeze_probe_enabled: true,
|
||||
account_auto_freeze_probe_model: None,
|
||||
account_auto_freeze_adaptive_concurrency: true,
|
||||
account_error_protection: Default::default(),
|
||||
account_warmup_enabled: false,
|
||||
account_warmup_schedule_time: "08:00".to_owned(),
|
||||
account_warmup_model: None,
|
||||
@@ -453,13 +522,7 @@ fn settings_response_should_cover_the_full_runtime_settings_contract() {
|
||||
"usageRetentionDays": 31,
|
||||
"opsEventRetentionDays": 31,
|
||||
"auditRetentionDays": 91,
|
||||
"accountAutoFreezeEnabled": true,
|
||||
"accountAutoFreezeThreshold": 12,
|
||||
"accountAutoFreezeWindowSeconds": 600,
|
||||
"accountAutoFreezeDurationSeconds": 7200,
|
||||
"accountAutoFreezeProbeEnabled": true,
|
||||
"accountAutoFreezeProbeModel": null,
|
||||
"accountAutoFreezeAdaptiveConcurrency": true,
|
||||
"accountErrorProtection": gateway_core::account::AccountErrorProtection::default(),
|
||||
"accountWarmupEnabled": false,
|
||||
"accountWarmupScheduleTime": "08:00",
|
||||
"accountWarmupModel": null,
|
||||
|
||||
@@ -218,6 +218,7 @@ async fn overview_scopes_every_query_and_projects_only_key_visible_fields() {
|
||||
assert!(!data.to_string().contains("private-sentinel"));
|
||||
let observations = fixture.observations.lock().unwrap();
|
||||
assert_eq!(observations.summaries.len(), 1);
|
||||
assert_eq!(observations.concurrency_selections, vec![false]);
|
||||
assert_eq!(observations.trends.len(), 2);
|
||||
for (_, filter) in observations
|
||||
.summaries
|
||||
|
||||
@@ -229,29 +229,67 @@ fn desktop_headers_should_take_precedence_over_embedded_cli_marker() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cli_user_agent_should_expose_semver_or_recognized_missing_version() {
|
||||
let mut headers = HeaderMap::new();
|
||||
headers.insert(
|
||||
"user-agent",
|
||||
HeaderValue::from_static("codex_cli_rs/0.39.0 (Linux; x86_64)"),
|
||||
);
|
||||
let identified = identify_codex_client(&headers).expect("recognized CLI");
|
||||
assert_eq!(identified.kind(), CodexClientKind::Cli);
|
||||
assert_eq!(
|
||||
identified.version().map(ToString::to_string).as_deref(),
|
||||
Some("0.39.0")
|
||||
);
|
||||
fn official_core_clients_should_use_the_core_version_instead_of_the_app_suffix() {
|
||||
for product in [
|
||||
"codex_cli_rs",
|
||||
"codex-cli",
|
||||
"codex-tui",
|
||||
"codex_exec",
|
||||
"codex_vscode",
|
||||
"codex_sdk_ts",
|
||||
"codex_python_sdk",
|
||||
"codex-app-server",
|
||||
"codex-app-server-sdk",
|
||||
"codex_mcp_server",
|
||||
"CoDeX-TuI",
|
||||
] {
|
||||
for (version, expected) in [
|
||||
(Some("0.158.0"), Some("0.158.0")),
|
||||
(
|
||||
Some("0.165.0-alpha.1+build.2"),
|
||||
Some("0.165.0-alpha.1+build.2"),
|
||||
),
|
||||
(Some("not-a-version"), None),
|
||||
(Some(""), None),
|
||||
(None, None),
|
||||
] {
|
||||
let prefix = version.map_or_else(|| product.to_owned(), |v| format!("{product}/{v}"));
|
||||
let mut headers = HeaderMap::new();
|
||||
headers.insert(
|
||||
"user-agent",
|
||||
HeaderValue::from_str(&format!(
|
||||
"{prefix} (Mac OS 26.6.2; arm64) Apple_Terminal ({product}; 0.165.0)"
|
||||
))
|
||||
.unwrap(),
|
||||
);
|
||||
let identified = identify_codex_client(&headers).expect("recognized official client");
|
||||
assert_eq!(identified.kind(), CodexClientKind::Cli, "{prefix}");
|
||||
assert_eq!(
|
||||
identified.version().map(ToString::to_string).as_deref(),
|
||||
expected,
|
||||
"{prefix}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
headers.insert(
|
||||
"user-agent",
|
||||
HeaderValue::from_static("codex-cli/not-a-version"),
|
||||
);
|
||||
assert!(
|
||||
identify_codex_client(&headers)
|
||||
.expect("recognized invalid CLI")
|
||||
.version()
|
||||
.is_none()
|
||||
);
|
||||
#[test]
|
||||
fn unknown_codex_like_products_should_remain_unrestricted() {
|
||||
for product in [
|
||||
"custom-codex-tui",
|
||||
"codex-tui-custom",
|
||||
"codex_cli_rs_custom",
|
||||
"codex_python_sdk_custom",
|
||||
"my_app",
|
||||
"deepseek-harness",
|
||||
] {
|
||||
let mut headers = HeaderMap::new();
|
||||
headers.insert(
|
||||
"user-agent",
|
||||
HeaderValue::from_str(&format!("{product}/0.158.0 (Linux; x86_64)")).unwrap(),
|
||||
);
|
||||
assert_eq!(identify_codex_client(&headers), None, "{product}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -524,9 +562,12 @@ async fn http_settings_freeze_before_plan_resolution_and_apply_before_admission(
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
response.status(),
|
||||
StatusCode::UPGRADE_REQUIRED,
|
||||
StatusCode::BAD_REQUEST,
|
||||
"request changes must not alter the published baseline"
|
||||
);
|
||||
let body: serde_json::Value =
|
||||
serde_json::from_slice(&response.into_body().collect().await.unwrap().to_bytes()).unwrap();
|
||||
assert_eq!(body["error"]["code"], "client_version_too_old");
|
||||
assert_eq!(
|
||||
snapshots
|
||||
.acquire()
|
||||
|
||||
@@ -0,0 +1,624 @@
|
||||
//! Chat HTTP 夹具和共享执行、鉴权、插件及终结边界
|
||||
|
||||
use std::{
|
||||
collections::VecDeque,
|
||||
io::Write as _,
|
||||
sync::{
|
||||
Arc, Mutex,
|
||||
atomic::{AtomicBool, AtomicUsize, Ordering},
|
||||
},
|
||||
time::Duration,
|
||||
};
|
||||
|
||||
use axum::{
|
||||
body::{Body, to_bytes},
|
||||
http::{Request, StatusCode},
|
||||
response::Response,
|
||||
};
|
||||
use bytes::Bytes;
|
||||
use futures::future::BoxFuture;
|
||||
use gateway_core::{
|
||||
engine::{
|
||||
CommitRequirement, CoordinatedEvent, EngineError, ModelRequestId,
|
||||
execution::{
|
||||
AuthenticatedClient, ClientAuthenticationError, ExecutionService, ExecutionSession,
|
||||
PreparedRootExecution, ResponseCommit, StartExecution, StartProviderExecution,
|
||||
StartedExecution,
|
||||
},
|
||||
middleware::{
|
||||
FrozenMiddlewarePlan, MiddlewareContext, MiddlewareError, MiddlewareNext,
|
||||
MiddlewarePlan, MiddlewareRequest, MiddlewareResponse,
|
||||
},
|
||||
},
|
||||
error::GatewayError,
|
||||
event::{ProviderEvent, ProviderResponseHeader},
|
||||
operation::Operation,
|
||||
routing::{
|
||||
PublicModelId,
|
||||
extensions::{ExtensionSetId, ExtensionSetLease, ExtensionSetReference},
|
||||
},
|
||||
};
|
||||
use serde_json::{Value, json};
|
||||
use tower::ServiceExt as _;
|
||||
|
||||
use super::{chat_request, completed, created, message};
|
||||
use crate::openai::{
|
||||
api_router, authenticated_client, authenticated_client_for_provider_with_limit,
|
||||
middleware::{RequestMiddleware, ResponseFrameAction},
|
||||
};
|
||||
|
||||
pub(super) const PATH: &str = "/v1/chat/completions";
|
||||
const KEY: &str = "sk_chat_fixture";
|
||||
|
||||
#[derive(Default)]
|
||||
pub(super) struct Trace {
|
||||
pub commits: AtomicUsize,
|
||||
pub discarded: AtomicUsize,
|
||||
pub cancelled: AtomicBool,
|
||||
pub finalized: AtomicBool,
|
||||
pub detached: AtomicUsize,
|
||||
response_commit: ResponseCommit,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
pub(super) enum Finish {
|
||||
#[default]
|
||||
Success,
|
||||
Error,
|
||||
Pending,
|
||||
}
|
||||
|
||||
pub(super) struct Session {
|
||||
events: VecDeque<Vec<ProviderEvent>>,
|
||||
pub trace: Arc<Trace>,
|
||||
pub finish: Finish,
|
||||
pub delay: Option<Duration>,
|
||||
pub fail_commit: bool,
|
||||
pub response_headers: Vec<ProviderResponseHeader>,
|
||||
committed: bool,
|
||||
buffered: bool,
|
||||
}
|
||||
|
||||
impl Session {
|
||||
pub fn new(events: Vec<ProviderEvent>) -> Self {
|
||||
Self {
|
||||
events: events.into_iter().map(|event| vec![event]).collect(),
|
||||
trace: Arc::default(),
|
||||
finish: Finish::Success,
|
||||
delay: None,
|
||||
fail_commit: false,
|
||||
response_headers: Vec::new(),
|
||||
committed: false,
|
||||
buffered: false,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn with_batch(mut self, events: Vec<ProviderEvent>) -> Self {
|
||||
self.events.push_back(events);
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
impl ExecutionSession for Session {
|
||||
fn response_commit(&self) -> ResponseCommit {
|
||||
self.trace.response_commit.clone()
|
||||
}
|
||||
|
||||
fn next_event(&mut self) -> BoxFuture<'_, Result<Option<CoordinatedEvent>, EngineError>> {
|
||||
Box::pin(async move {
|
||||
if let Some(delay) = self.delay.take() {
|
||||
tokio::time::sleep(delay).await;
|
||||
}
|
||||
if let Some(event) = self.events.pop_front() {
|
||||
let requirement = if self.committed {
|
||||
CommitRequirement::AlreadyCommitted
|
||||
} else {
|
||||
CommitRequirement::CommitBeforeDelivery
|
||||
};
|
||||
return CoordinatedEvent::try_batch(event, requirement).map(Some);
|
||||
}
|
||||
match self.finish {
|
||||
Finish::Success => {
|
||||
self.trace.finalized.store(true, Ordering::SeqCst);
|
||||
Ok(None)
|
||||
}
|
||||
Finish::Error => {
|
||||
self.trace.finalized.store(true, Ordering::SeqCst);
|
||||
Err(EngineError::ProviderMetadataMismatch)
|
||||
}
|
||||
Finish::Pending => futures::future::pending().await,
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
fn collect_uncommitted(&mut self) -> BoxFuture<'_, Result<Vec<ProviderEvent>, EngineError>> {
|
||||
self.buffered = true;
|
||||
Box::pin(async move { Ok(self.events.drain(..).flatten().collect()) })
|
||||
}
|
||||
|
||||
fn response_headers(&self) -> &[ProviderResponseHeader] {
|
||||
&self.response_headers
|
||||
}
|
||||
|
||||
fn discard_pending_delivery(&mut self) -> Result<(), EngineError> {
|
||||
self.trace.discarded.fetch_add(1, Ordering::SeqCst);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn commit_downstream(&mut self, status: Option<u16>) -> BoxFuture<'_, Result<(), EngineError>> {
|
||||
Box::pin(async move {
|
||||
self.trace.commits.fetch_add(1, Ordering::SeqCst);
|
||||
if self.fail_commit {
|
||||
return Err(EngineError::ProviderMetadataMismatch);
|
||||
}
|
||||
self.trace.response_commit.record(status);
|
||||
self.committed = true;
|
||||
if self.buffered {
|
||||
self.trace.finalized.store(true, Ordering::SeqCst);
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
fn record_client_status(&mut self, status: u16) -> BoxFuture<'_, Result<(), EngineError>> {
|
||||
Box::pin(async move {
|
||||
self.trace.response_commit.record(Some(status));
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
fn is_finalized(&self) -> bool {
|
||||
self.trace.finalized.load(Ordering::SeqCst)
|
||||
}
|
||||
fn cancel(&self) {
|
||||
self.trace.cancelled.store(true, Ordering::SeqCst);
|
||||
}
|
||||
fn detach_finalize(self: Box<Self>) -> BoxFuture<'static, ()> {
|
||||
Box::pin(async move {
|
||||
self.trace.detached.fetch_add(1, Ordering::SeqCst);
|
||||
self.trace.finalized.store(true, Ordering::SeqCst);
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub(super) struct Captured {
|
||||
pub body: Value,
|
||||
pub context: Value,
|
||||
pub endpoint: String,
|
||||
}
|
||||
|
||||
pub(super) struct Execution {
|
||||
pub captured: Mutex<Option<Captured>>,
|
||||
pub client: AuthenticatedClient,
|
||||
pub middleware: Option<FrozenMiddlewarePlan>,
|
||||
session: Mutex<Option<Session>>,
|
||||
}
|
||||
|
||||
impl Execution {
|
||||
pub fn new(session: Session) -> Self {
|
||||
Self {
|
||||
captured: Mutex::default(),
|
||||
client: authenticated_client(KEY),
|
||||
middleware: None,
|
||||
session: Mutex::new(Some(session)),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl ExecutionService for Execution {
|
||||
fn authenticate(&self, key: &str) -> Result<AuthenticatedClient, ClientAuthenticationError> {
|
||||
if key == KEY {
|
||||
Ok(self.client.clone())
|
||||
} else {
|
||||
Err(ClientAuthenticationError::InvalidKey)
|
||||
}
|
||||
}
|
||||
fn public_models(&self, _: &AuthenticatedClient) -> Vec<PublicModelId> {
|
||||
Vec::new()
|
||||
}
|
||||
fn contains_public_model(&self, _: &AuthenticatedClient, _: &PublicModelId) -> bool {
|
||||
true
|
||||
}
|
||||
fn middleware_plan(&self, _: &PreparedRootExecution) -> Option<FrozenMiddlewarePlan> {
|
||||
self.middleware.clone()
|
||||
}
|
||||
fn start(
|
||||
&self,
|
||||
request: StartExecution,
|
||||
) -> BoxFuture<'_, Result<StartedExecution, GatewayError>> {
|
||||
Box::pin(async move {
|
||||
let Operation::Generate(generation) = request.operation else {
|
||||
panic!("Chat must generate")
|
||||
};
|
||||
*self.captured.lock().unwrap() = Some(Captured {
|
||||
body: Value::Object(generation.protocol_payload().body().clone()),
|
||||
context: Value::Object(generation.protocol_payload().context().clone()),
|
||||
endpoint: request.metadata.endpoint,
|
||||
});
|
||||
Ok(StartedExecution {
|
||||
request_id: ModelRequestId::new("req_chat_fixture").unwrap(),
|
||||
created_at: std::time::SystemTime::now(),
|
||||
stream: request.metadata.stream,
|
||||
session: Box::new(self.session.lock().unwrap().take().expect("one request")),
|
||||
})
|
||||
})
|
||||
}
|
||||
fn start_provider_endpoint(
|
||||
&self,
|
||||
_: StartProviderExecution,
|
||||
) -> BoxFuture<'_, Result<StartedExecution, GatewayError>> {
|
||||
Box::pin(async { panic!("Chat must use model execution") })
|
||||
}
|
||||
}
|
||||
|
||||
pub(super) struct Fixture {
|
||||
pub router: axum::Router,
|
||||
pub execution: Arc<Execution>,
|
||||
pub trace: Arc<Trace>,
|
||||
}
|
||||
|
||||
impl Fixture {
|
||||
pub async fn new(session: Session) -> Self {
|
||||
Self::with_execution(Execution::new(session)).await
|
||||
}
|
||||
pub async fn with_execution(execution: Execution) -> Self {
|
||||
let trace = execution
|
||||
.session
|
||||
.lock()
|
||||
.unwrap()
|
||||
.as_ref()
|
||||
.unwrap()
|
||||
.trace
|
||||
.clone();
|
||||
let execution = Arc::new(execution);
|
||||
Self {
|
||||
router: api_router(execution.clone()).await,
|
||||
execution,
|
||||
trace,
|
||||
}
|
||||
}
|
||||
pub async fn post(&self, body: Value) -> Response {
|
||||
self.router.clone().oneshot(request(body)).await.unwrap()
|
||||
}
|
||||
pub fn captured(&self) -> Captured {
|
||||
self.execution
|
||||
.captured
|
||||
.lock()
|
||||
.unwrap()
|
||||
.clone()
|
||||
.expect("started request")
|
||||
}
|
||||
}
|
||||
|
||||
pub(super) fn request(body: Value) -> Request<Body> {
|
||||
Request::post(PATH)
|
||||
.header("authorization", format!("Bearer {KEY}"))
|
||||
.header("content-type", "application/json")
|
||||
.body(Body::from(body.to_string()))
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
pub(super) async fn json_body(response: Response) -> Value {
|
||||
serde_json::from_slice(&to_bytes(response.into_body(), usize::MAX).await.unwrap()).unwrap()
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
pub(super) struct InspectMiddleware {
|
||||
pub inner: RequestMiddleware,
|
||||
pub requests: Mutex<Vec<Value>>,
|
||||
pub update: Option<fn(&mut Value)>,
|
||||
}
|
||||
|
||||
impl InspectMiddleware {
|
||||
pub fn frozen(self: &Arc<Self>) -> FrozenMiddlewarePlan {
|
||||
FrozenMiddlewarePlan::new(
|
||||
self.clone(),
|
||||
ExtensionSetReference::new(
|
||||
ExtensionSetId::new("chat-inspector".to_owned()).unwrap(),
|
||||
Arc::new(ReadyLease),
|
||||
),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
struct ReadyLease;
|
||||
impl ExtensionSetLease for ReadyLease {
|
||||
fn is_ready(&self) -> bool {
|
||||
true
|
||||
}
|
||||
}
|
||||
|
||||
impl MiddlewarePlan for InspectMiddleware {
|
||||
fn handle(
|
||||
&self,
|
||||
context: MiddlewareContext,
|
||||
mut request: MiddlewareRequest,
|
||||
next: MiddlewareNext,
|
||||
) -> BoxFuture<'static, Result<MiddlewareResponse, MiddlewareError>> {
|
||||
let mut body: Value = serde_json::from_slice(request.body()).unwrap();
|
||||
self.requests.lock().unwrap().push(body.clone());
|
||||
if let Some(update) = self.update {
|
||||
update(&mut body);
|
||||
let headers = request.headers().to_vec();
|
||||
request = request
|
||||
.replace_parts(
|
||||
"openai".to_owned(),
|
||||
headers,
|
||||
Bytes::from(body.to_string()),
|
||||
None,
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
self.inner.handle(context, request, next)
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn rejects_missing_authentication_before_decoding_or_execution() {
|
||||
let fixture = Fixture::new(Session::new(Vec::new())).await;
|
||||
for authorization in [None, Some("Bearer sk_invalid")] {
|
||||
let mut request = Request::post(PATH);
|
||||
if let Some(authorization) = authorization {
|
||||
request = request.header("authorization", authorization);
|
||||
}
|
||||
let response = fixture
|
||||
.router
|
||||
.clone()
|
||||
.oneshot(request.body(Body::from("broken json")).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(response.status(), StatusCode::UNAUTHORIZED);
|
||||
}
|
||||
assert!(fixture.execution.captured.lock().unwrap().is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn request_middleware_sees_responses_and_preserves_the_chat_endpoint() {
|
||||
let middleware = Arc::new(InspectMiddleware {
|
||||
inner: RequestMiddleware {
|
||||
expected_operation: Some(gateway_core::operation::OperationKind::Generate),
|
||||
..RequestMiddleware::default()
|
||||
},
|
||||
update: Some(|body| {
|
||||
body["input"] = json!("rewritten by request middleware");
|
||||
}),
|
||||
..InspectMiddleware::default()
|
||||
});
|
||||
let mut execution = Execution::new(Session::new(vec![completed(json!([message("ok")]))]));
|
||||
execution.middleware = Some(middleware.frozen());
|
||||
let fixture = Fixture::with_execution(execution).await;
|
||||
let response = fixture.post(chat_request(false)).await;
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
let downstream = json_body(response).await;
|
||||
assert_eq!(downstream["object"], "chat.completion");
|
||||
let observed = middleware.requests.lock().unwrap();
|
||||
assert!(observed[0].get("messages").is_none());
|
||||
assert_eq!(observed[0]["input"][0]["role"], "user");
|
||||
assert_eq!(
|
||||
middleware.inner.endpoints.lock().unwrap().as_slice(),
|
||||
[PATH]
|
||||
);
|
||||
let captured = fixture.captured();
|
||||
assert_eq!(captured.endpoint, PATH);
|
||||
assert_eq!(captured.body["input"], "rewritten by request middleware");
|
||||
assert_eq!(fixture.trace.commits.load(Ordering::SeqCst), 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn dropping_chat_body_cancels_and_finalizes_the_original_execution() {
|
||||
let mut session = Session::new(vec![created()]);
|
||||
session.finish = Finish::Pending;
|
||||
let fixture = Fixture::new(session).await;
|
||||
let response = fixture.post(chat_request(true)).await;
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
drop(response);
|
||||
for _ in 0..8 {
|
||||
tokio::task::yield_now().await;
|
||||
}
|
||||
assert!(fixture.trace.cancelled.load(Ordering::SeqCst));
|
||||
assert_eq!(fixture.trace.detached.load(Ordering::SeqCst), 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn gzip_and_headers_reuse_the_responses_ingress_contract() {
|
||||
let mut session = Session::new(vec![completed(json!([message("ok")]))]);
|
||||
session.response_headers = vec![
|
||||
ProviderResponseHeader::new("x-future-multi", Bytes::from_static(b"first")),
|
||||
ProviderResponseHeader::new("x-future-multi", Bytes::from_static(b"second")),
|
||||
ProviderResponseHeader::new("connection", Bytes::from_static(b"x-hop-only")),
|
||||
ProviderResponseHeader::new("x-hop-only", Bytes::from_static(b"removed")),
|
||||
];
|
||||
let fixture = Fixture::new(session).await;
|
||||
let body = chat_request(false).to_string();
|
||||
let mut encoder = flate2::write::GzEncoder::new(Vec::new(), flate2::Compression::fast());
|
||||
encoder.write_all(body.as_bytes()).unwrap();
|
||||
let compressed = encoder.finish().unwrap();
|
||||
let request = Request::post(PATH)
|
||||
.header("authorization", format!("Bearer {KEY}"))
|
||||
.header("content-type", "application/json")
|
||||
.header("content-encoding", "gzip")
|
||||
.header("content-length", compressed.len())
|
||||
.header("session_id", "chat-session")
|
||||
.header("x-future-input", "first")
|
||||
.header("x-future-input", "second")
|
||||
.body(Body::from(compressed))
|
||||
.unwrap();
|
||||
let response = fixture.router.clone().oneshot(request).await.unwrap();
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
assert_eq!(
|
||||
response
|
||||
.headers()
|
||||
.get_all("x-future-multi")
|
||||
.iter()
|
||||
.map(|value| value.to_str().unwrap())
|
||||
.collect::<Vec<_>>(),
|
||||
["first", "second"]
|
||||
);
|
||||
assert!(!response.headers().contains_key("x-hop-only"));
|
||||
assert!(!response.headers().contains_key("content-encoding"));
|
||||
assert_eq!(
|
||||
json_body(response).await["choices"][0]["message"]["content"],
|
||||
"ok"
|
||||
);
|
||||
let captured = fixture.captured();
|
||||
assert_eq!(captured.context["session_id"], "chat-session");
|
||||
let headers = captured.context["opaque_request_headers"]
|
||||
.as_array()
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
headers
|
||||
.iter()
|
||||
.filter(|header| header[0] == "x-future-input")
|
||||
.count(),
|
||||
2
|
||||
);
|
||||
assert!(!headers.iter().any(|header| matches!(
|
||||
header[0].as_str(),
|
||||
Some("content-encoding" | "content-length" | "authorization")
|
||||
)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn corrupt_or_oversized_compressed_chat_never_starts_execution() {
|
||||
let mut encoder = flate2::write::GzEncoder::new(Vec::new(), flate2::Compression::fast());
|
||||
encoder
|
||||
.write_all(chat_request(false).to_string().as_bytes())
|
||||
.unwrap();
|
||||
let valid_compressed = encoder.finish().unwrap();
|
||||
for bytes in [b"broken gzip".to_vec(), valid_compressed] {
|
||||
let mut execution = Execution::new(Session::new(Vec::new()));
|
||||
execution.client = authenticated_client_for_provider_with_limit(KEY, "openai", 16);
|
||||
let fixture = Fixture::with_execution(execution).await;
|
||||
let request = Request::post(PATH)
|
||||
.header("authorization", format!("Bearer {KEY}"))
|
||||
.header("content-encoding", "gzip")
|
||||
.body(Body::from(bytes))
|
||||
.unwrap();
|
||||
let response = fixture.router.clone().oneshot(request).await.unwrap();
|
||||
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
|
||||
assert!(fixture.execution.captured.lock().unwrap().is_none());
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn response_middleware_is_validated_as_responses_before_chat_projection() {
|
||||
for matching_identity in [true, false] {
|
||||
let replacement = json!({"id":if matching_identity {"resp_chat_test"} else {"resp_other"},
|
||||
"status":"completed","output":[message("plugin text")]});
|
||||
let middleware = Arc::new(
|
||||
RequestMiddleware {
|
||||
expected_operation: Some(gateway_core::operation::OperationKind::Generate),
|
||||
..RequestMiddleware::default()
|
||||
}
|
||||
.with_response_actions(vec![ResponseFrameAction::Replace(Bytes::from(
|
||||
replacement.to_string(),
|
||||
))]),
|
||||
);
|
||||
let mut execution =
|
||||
Execution::new(Session::new(vec![completed(json!([message("original")]))]));
|
||||
execution.middleware = Some(middleware.frozen());
|
||||
let fixture = Fixture::with_execution(execution).await;
|
||||
let response = fixture.post(chat_request(false)).await;
|
||||
if matching_identity {
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
assert_eq!(
|
||||
json_body(response).await["choices"][0]["message"]["content"],
|
||||
"plugin text"
|
||||
);
|
||||
} else {
|
||||
assert_eq!(response.status(), StatusCode::BAD_GATEWAY);
|
||||
assert_eq!(
|
||||
json_body(response).await["error"]["code"],
|
||||
"middleware_failed"
|
||||
);
|
||||
assert_eq!(fixture.trace.commits.load(Ordering::SeqCst), 0);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn commit_failure_does_not_deliver_buffered_or_first_terminal_chat_success() {
|
||||
for streaming in [false, true] {
|
||||
let mut session = Session::new(vec![completed(json!([message("not delivered")]))]);
|
||||
session.fail_commit = true;
|
||||
let fixture = Fixture::new(session).await;
|
||||
let response = fixture.post(chat_request(streaming)).await;
|
||||
assert!(response.status().is_server_error());
|
||||
let body = json_body(response).await;
|
||||
assert!(body.get("error").is_some());
|
||||
assert!(body.get("choices").is_none());
|
||||
assert!(fixture.trace.cancelled.load(Ordering::SeqCst));
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn the_same_execution_fixture_finishes_native_responses_with_one_commit() {
|
||||
for already_started in [false, true] {
|
||||
let mut events = Vec::new();
|
||||
if already_started {
|
||||
events.push(created());
|
||||
}
|
||||
events.push(completed(json!([message("native baseline")])));
|
||||
let fixture = Fixture::new(Session::new(events)).await;
|
||||
let mut request = request(json!({"model":"model-a","input":"hello","stream":true}));
|
||||
*request.uri_mut() = "/v1/responses".parse().unwrap();
|
||||
let response = fixture.router.clone().oneshot(request).await.unwrap();
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
let bytes = to_bytes(response.into_body(), usize::MAX).await.unwrap();
|
||||
let text = String::from_utf8(bytes.to_vec()).unwrap();
|
||||
assert!(text.contains("event: response.completed"), "{text}");
|
||||
assert!(text.contains("native baseline"), "{text}");
|
||||
assert!(!text.contains("response.failed"), "{text}");
|
||||
assert_eq!(text.matches("data: [DONE]").count(), 1, "{text}");
|
||||
assert!(text.ends_with("data: [DONE]\n\n"), "{text}");
|
||||
assert!(fixture.trace.finalized.load(Ordering::SeqCst));
|
||||
assert_eq!(fixture.trace.commits.load(Ordering::SeqCst), 1);
|
||||
assert_eq!(fixture.trace.response_commit.status_code(), Some(200));
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn finalization_failure_never_leaks_success_finish_or_usage() {
|
||||
for already_started in [false, true] {
|
||||
let mut events = Vec::new();
|
||||
if already_started {
|
||||
events.push(created());
|
||||
}
|
||||
events.push(completed(json!([])));
|
||||
let mut session = Session::new(events);
|
||||
session.finish = Finish::Error;
|
||||
let fixture = Fixture::new(session).await;
|
||||
let mut body = chat_request(true);
|
||||
body["stream_options"] = json!({"include_usage":true});
|
||||
let response = fixture.post(body).await;
|
||||
// 原会话已经接受 200 提交,后续终结错误只能作为流内错误交付
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
let (_, chunks) = super::response::sse_body(response).await;
|
||||
assert!(chunks.iter().any(|chunk| chunk.get("error").is_some()));
|
||||
assert!(
|
||||
!chunks
|
||||
.iter()
|
||||
.any(|chunk| chunk["choices"][0]["finish_reason"].is_string())
|
||||
);
|
||||
assert!(!chunks.iter().any(|chunk| chunk["choices"] == json!([])));
|
||||
assert!(fixture.trace.finalized.load(Ordering::SeqCst));
|
||||
assert_eq!(fixture.trace.commits.load(Ordering::SeqCst), 1);
|
||||
assert_eq!(fixture.trace.response_commit.status_code(), Some(200));
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test(start_paused = true)]
|
||||
async fn error_after_admission_keepalive_uses_chat_error_encoding() {
|
||||
let mut session = Session::new(Vec::new());
|
||||
session.delay = Some(Duration::from_secs(31));
|
||||
session.finish = Finish::Error;
|
||||
let fixture = Fixture::new(session).await;
|
||||
let response = fixture.post(chat_request(true)).await;
|
||||
let (body, chunks) = super::response::sse_body(response).await;
|
||||
assert!(body.starts_with(": keep-alive\n\n"));
|
||||
assert!(chunks.iter().any(|chunk| chunk.get("error").is_some()));
|
||||
assert!(!body.contains("response.failed"));
|
||||
assert_eq!(fixture.trace.commits.load(Ordering::SeqCst), 0);
|
||||
assert_eq!(fixture.trace.response_commit.status_code(), Some(200));
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
//! Chat Completions 的请求转换、输出投影与 HTTP 生命周期测试
|
||||
|
||||
mod http;
|
||||
mod request;
|
||||
mod response;
|
||||
|
||||
use gateway_core::event::{ProtocolWireEvent, ProviderEvent};
|
||||
use serde_json::{Value, json};
|
||||
|
||||
fn chat_request(stream: bool) -> Value {
|
||||
json!({"model":"model-a","messages":[{"role":"user","content":"hello"}],"stream":stream})
|
||||
}
|
||||
|
||||
fn wire(kind: &str, mut value: Value) -> ProviderEvent {
|
||||
value["type"] = json!(kind);
|
||||
ProviderEvent::wire(ProtocolWireEvent::json("openai", Some(kind.to_owned()), value).unwrap())
|
||||
}
|
||||
|
||||
fn completed(output: Value) -> ProviderEvent {
|
||||
wire(
|
||||
"response.completed",
|
||||
json!({"response":{
|
||||
"id":"resp_chat_test","object":"response","model":"upstream-model",
|
||||
"status":"completed","output":output,
|
||||
"usage":{"input_tokens":11,"output_tokens":7,"total_tokens":18,
|
||||
"input_tokens_details":{"cached_tokens":3},
|
||||
"output_tokens_details":{"reasoning_tokens":2}}
|
||||
}}),
|
||||
)
|
||||
}
|
||||
|
||||
fn message(text: &str) -> Value {
|
||||
json!({"id":"msg_chat_test","type":"message","role":"assistant","status":"completed",
|
||||
"content":[{"type":"output_text","text":text,"annotations":[]}]})
|
||||
}
|
||||
|
||||
fn created() -> ProviderEvent {
|
||||
wire(
|
||||
"response.created",
|
||||
json!({"response":{
|
||||
"id":"resp_chat_test","object":"response","model":"upstream-model",
|
||||
"status":"in_progress","output":[]
|
||||
}}),
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
//! Chat 请求映射保留消息、工具和不透明业务数据的语义
|
||||
|
||||
use axum::http::StatusCode;
|
||||
use serde_json::{Value, json};
|
||||
|
||||
use super::{
|
||||
chat_request, completed,
|
||||
http::{Fixture, Session, json_body},
|
||||
message,
|
||||
};
|
||||
|
||||
async fn convert(body: Value) -> Value {
|
||||
let fixture = Fixture::new(Session::new(vec![completed(json!([message("ok")]))])).await;
|
||||
let response = fixture.post(body).await;
|
||||
let status = response.status();
|
||||
let result = json_body(response).await;
|
||||
assert_eq!(status, StatusCode::OK, "{result}");
|
||||
fixture.captured().body
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn preserves_ordered_roles_images_and_modern_tool_round_trip() {
|
||||
let arguments = "{ \"city\" : \"北京\", \"store\":true }";
|
||||
let converted = convert(json!({"model":"model-a","messages":[
|
||||
{"role":"system","content":"system text"},
|
||||
{"role":"developer","content":"developer text"},
|
||||
{"role":"user","content":[{"type":"text","text":"describe"},
|
||||
{"type":"image_url","image_url":{"url":"data:image/png;base64,aGVsbG8=","detail":"high"}}]},
|
||||
{"role":"assistant","content":"checking","tool_calls":[
|
||||
{"id":"call-weather","type":"function","function":{"name":"weather","arguments":arguments}}]},
|
||||
{"role":"tool","tool_call_id":"call-weather","content":"{\"temperature\":25}"},
|
||||
{"role":"user","content":"continue"}
|
||||
]})).await;
|
||||
let input = converted["input"].as_array().unwrap();
|
||||
assert_eq!(
|
||||
input
|
||||
.iter()
|
||||
.filter_map(|item| item["role"].as_str())
|
||||
.collect::<Vec<_>>(),
|
||||
["system", "developer", "user", "assistant", "user"]
|
||||
);
|
||||
assert_eq!(
|
||||
input[2]["content"][1],
|
||||
json!({"type":"input_image","image_url":"data:image/png;base64,aGVsbG8=","detail":"high"})
|
||||
);
|
||||
assert_eq!(
|
||||
input[3]["content"],
|
||||
json!([{"type":"output_text","text":"checking","annotations":[]}])
|
||||
);
|
||||
assert_eq!(
|
||||
input[4],
|
||||
json!({"type":"function_call","call_id":"call-weather","name":"weather","arguments":arguments})
|
||||
);
|
||||
assert_eq!(
|
||||
input[5],
|
||||
json!({"type":"function_call_output","call_id":"call-weather","output":"{\"temperature\":25}"})
|
||||
);
|
||||
assert!(converted.get("previous_response_id").is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn assistant_history_uses_output_content_for_text_and_refusals() {
|
||||
for content in [
|
||||
json!("previous answer"),
|
||||
json!([{"type":"text","text":"previous answer"}]),
|
||||
json!([{"type":"text","text":"previous answer"},{"type":"refusal","refusal":"declined"}]),
|
||||
] {
|
||||
let converted = convert(json!({"model":"model-a","messages":[
|
||||
{"role":"user","content":"first turn"},
|
||||
{"role":"assistant","content":content},
|
||||
{"role":"user","content":"continue from the previous answer"}
|
||||
]}))
|
||||
.await;
|
||||
let history = &converted["input"][1];
|
||||
assert_eq!(history["role"], "assistant");
|
||||
assert_eq!(history["status"], "completed");
|
||||
assert!(!history["id"].as_str().unwrap().is_empty());
|
||||
assert_eq!(
|
||||
history["content"][0],
|
||||
json!({"type":"output_text","text":"previous answer","annotations":[]})
|
||||
);
|
||||
if content.is_array() && content.as_array().unwrap().len() == 2 {
|
||||
assert_eq!(
|
||||
history["content"][1],
|
||||
json!({"type":"refusal","refusal":"declined"})
|
||||
);
|
||||
}
|
||||
assert_eq!(converted["input"][2]["content"][0]["type"], "input_text");
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn preserves_nested_schema_metadata_and_future_opaque_values() {
|
||||
let schema = json!({"type":"object","properties":{
|
||||
"messages":{"type":"string"},"store":{"type":"boolean"},
|
||||
"function_call":{"type":"string"},"future":{"x-extension":[1,2]}},
|
||||
"required":["messages"],"additionalProperties":true});
|
||||
let mut body = chat_request(false);
|
||||
body["model"] = json!("future-model-opaque:β");
|
||||
body["tools"] = json!([{"type":"function","function":{"name":"inspect","parameters":schema}}]);
|
||||
body["tool_choice"] = json!({"type":"function","function":{"name":"inspect"}});
|
||||
body["response_format"] = json!({"type":"json_schema","json_schema":{
|
||||
"name":"result","description":"preserve","strict":true,"schema":schema}});
|
||||
body["metadata"] = json!({"messages":"business data","future":"kept"});
|
||||
body["reasoning_effort"] = json!("future-effort");
|
||||
body["verbosity"] = json!("future-verbosity");
|
||||
body["service_tier"] = json!("future-tier");
|
||||
body["future_client_extension"] = json!({"opaque":true});
|
||||
let converted = convert(body).await;
|
||||
assert_eq!(converted["model"], "future-model-opaque:β");
|
||||
assert_eq!(converted["tools"][0]["parameters"], schema);
|
||||
assert_eq!(converted["tools"][0]["strict"], false);
|
||||
assert_eq!(
|
||||
converted["tool_choice"],
|
||||
json!({"type":"function","name":"inspect"})
|
||||
);
|
||||
assert_eq!(converted["text"]["format"]["schema"], schema);
|
||||
assert_eq!(converted["text"]["format"]["strict"], true);
|
||||
assert_eq!(
|
||||
converted["metadata"],
|
||||
json!({"messages":"business data","future":"kept"})
|
||||
);
|
||||
assert_eq!(converted["reasoning"]["effort"], "future-effort");
|
||||
assert_eq!(converted["text"]["verbosity"], "future-verbosity");
|
||||
assert_eq!(converted["service_tier"], "future-tier");
|
||||
assert!(converted.get("future_client_extension").is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn normalizes_neutral_defaults_without_dropping_shared_provider_parameters() {
|
||||
let mut body = chat_request(false);
|
||||
for name in [
|
||||
"stream",
|
||||
"store",
|
||||
"n",
|
||||
"functions",
|
||||
"function_call",
|
||||
"logprobs",
|
||||
"seed",
|
||||
] {
|
||||
body[name] = Value::Null;
|
||||
}
|
||||
body["frequency_penalty"] = json!(0);
|
||||
body["presence_penalty"] = json!(0);
|
||||
body["stop"] = json!([]);
|
||||
body["max_tokens"] = json!(12);
|
||||
body["max_completion_tokens"] = json!(24);
|
||||
body["temperature"] = json!(0.2);
|
||||
body["top_p"] = json!(0.9);
|
||||
body["prompt_cache_retention"] = json!("24h");
|
||||
let converted = convert(body).await;
|
||||
assert_eq!(converted["stream"], false);
|
||||
assert_eq!(converted["store"], false);
|
||||
assert_eq!(converted["max_output_tokens"], 24);
|
||||
assert_eq!(converted["temperature"], 0.2);
|
||||
assert_eq!(converted["top_p"], 0.9);
|
||||
assert_eq!(converted["prompt_cache_retention"], "24h");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn refuses_known_unrepresentable_features_before_starting_execution() {
|
||||
for (name, value) in [
|
||||
("n", json!(2)),
|
||||
("store", json!(true)),
|
||||
("stop", json!(["stop"])),
|
||||
("seed", json!(7)),
|
||||
("frequency_penalty", json!(0.3)),
|
||||
("logprobs", json!(true)),
|
||||
("functions", json!([{"name":"old"}])),
|
||||
("function_call", json!("auto")),
|
||||
("input", json!("not chat")),
|
||||
("previous_response_id", json!("resp_previous")),
|
||||
("audio", json!({"voice":"alloy","format":"wav"})),
|
||||
(
|
||||
"prediction",
|
||||
json!({"type":"content","content":"predicted"}),
|
||||
),
|
||||
] {
|
||||
let fixture = Fixture::new(Session::new(Vec::new())).await;
|
||||
let mut body = chat_request(false);
|
||||
body[name] = value;
|
||||
let response = fixture.post(body).await;
|
||||
assert_eq!(response.status(), StatusCode::BAD_REQUEST, "{name}");
|
||||
let body = json_body(response).await;
|
||||
assert_eq!(body["error"]["param"], name, "{body}");
|
||||
assert!(
|
||||
fixture.execution.captured.lock().unwrap().is_none(),
|
||||
"{name}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn rejects_unmapped_message_content_and_identity_fields() {
|
||||
for message in [
|
||||
json!({"role":"user","name":"speaker","content":"hello"}),
|
||||
json!({"role":"user","content":[{"type":"file","file":{"file_id":"file-example"}}]}),
|
||||
json!({"role":"assistant","content":"hello","reasoning_content":"private"}),
|
||||
json!({"role":"assistant","content":[{"type":"text","text":"history","prompt_cache_breakpoint":{"type":"default"}}]}),
|
||||
json!({"role":"tool","content":"unassociated output"}),
|
||||
] {
|
||||
let fixture = Fixture::new(Session::new(Vec::new())).await;
|
||||
let response = fixture
|
||||
.post(json!({"model":"model-a","messages":[message]}))
|
||||
.await;
|
||||
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
|
||||
let body = json_body(response).await;
|
||||
assert!(
|
||||
body["error"]["param"]
|
||||
.as_str()
|
||||
.unwrap()
|
||||
.starts_with("messages")
|
||||
);
|
||||
assert!(fixture.execution.captured.lock().unwrap().is_none());
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn assistant_tool_history_keeps_arguments_opaque_and_cache_annotations_structural() {
|
||||
let cache = json!({"type":"default","future_setting":{"keep":true}});
|
||||
let arguments = "not valid JSON {\n \"raw\" : 🦀";
|
||||
let converted = convert(json!({"model":"model-a","messages":[
|
||||
{"role":"user","content":[{"type":"text","text":"cached","prompt_cache_breakpoint":cache}]},
|
||||
{"role":"assistant","content":null,"tool_calls":[{"id":"call-raw","type":"function",
|
||||
"function":{"name":"raw","arguments":arguments}}]},
|
||||
{"role":"tool","tool_call_id":"call-raw","content":[{"type":"text","text":"result"}]}
|
||||
],"prompt_cache_options":{"future":{"keep":true}},"moderation":{"future":{"keep":true}}}))
|
||||
.await;
|
||||
assert_eq!(
|
||||
converted["input"][0]["content"][0]["prompt_cache_breakpoint"],
|
||||
cache
|
||||
);
|
||||
assert_eq!(converted["input"][1]["arguments"], arguments);
|
||||
assert_eq!(
|
||||
converted["input"][2]["output"],
|
||||
json!([{"type":"input_text","text":"result"}])
|
||||
);
|
||||
assert_eq!(
|
||||
converted["prompt_cache_options"],
|
||||
json!({"future":{"keep":true}})
|
||||
);
|
||||
assert_eq!(converted["moderation"], json!({"future":{"keep":true}}));
|
||||
}
|
||||
@@ -0,0 +1,596 @@
|
||||
//! Chat JSON 与 SSE 的真实 wire 投影、去重和协议扩展边界
|
||||
|
||||
use axum::{body::to_bytes, http::StatusCode, response::Response};
|
||||
use serde_json::{Value, json};
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use super::{
|
||||
chat_request, completed, created,
|
||||
http::{Fixture, Session, json_body},
|
||||
message, wire,
|
||||
};
|
||||
|
||||
pub(super) async fn sse_body(response: Response) -> (String, Vec<Value>) {
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
assert_eq!(response.headers()["content-type"], "text/event-stream");
|
||||
let bytes = to_bytes(response.into_body(), usize::MAX).await.unwrap();
|
||||
let text = String::from_utf8(bytes.to_vec()).unwrap();
|
||||
assert_eq!(text.matches("data: [DONE]").count(), 1, "{text}");
|
||||
assert!(text.ends_with("data: [DONE]\n\n"), "{text}");
|
||||
assert!(!text.contains("event: response."), "{text}");
|
||||
let chunks = text
|
||||
.lines()
|
||||
.filter_map(|line| line.strip_prefix("data: "))
|
||||
.filter(|data| *data != "[DONE]")
|
||||
.map(|data| serde_json::from_str(data).unwrap())
|
||||
.collect();
|
||||
(text, chunks)
|
||||
}
|
||||
|
||||
fn combined(chunks: &[Value], name: &str) -> String {
|
||||
chunks
|
||||
.iter()
|
||||
.filter_map(|value| value["choices"][0]["delta"][name].as_str())
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn tool(id: &str, call_id: &str, name: &str, arguments: &str) -> Value {
|
||||
json!({"type":"function_call","id":id,"call_id":call_id,"name":name,"arguments":arguments,"status":"completed"})
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn json_projects_public_identity_text_and_actual_usage() {
|
||||
let fixture = Fixture::new(Session::new(vec![completed(json!([message("你好 🌍")]))])).await;
|
||||
let response = fixture.post(chat_request(false)).await;
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
assert_eq!(
|
||||
response.headers()["x-gateway-request-id"],
|
||||
"req_chat_fixture"
|
||||
);
|
||||
let body = json_body(response).await;
|
||||
assert_eq!(body["object"], "chat.completion");
|
||||
assert!(body["id"].as_str().unwrap().starts_with("chatcmpl-"));
|
||||
assert!(body["created"].as_u64().is_some());
|
||||
assert_eq!(body["model"], "model-a");
|
||||
assert_eq!(body["choices"][0]["message"]["content"], "你好 🌍");
|
||||
assert_eq!(body["choices"][0]["finish_reason"], "stop");
|
||||
assert_eq!(
|
||||
body["usage"],
|
||||
json!({"prompt_tokens":11,"completion_tokens":7,"total_tokens":18,
|
||||
"prompt_tokens_details":{"cached_tokens":3},"completion_tokens_details":{"reasoning_tokens":2}})
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn json_tool_arguments_and_refusal_do_not_become_assistant_text() {
|
||||
for output in [
|
||||
json!([tool("fc1", "call1", "weather", "{ \"city\" : \"北京\" }")]),
|
||||
json!([{"type":"reasoning","id":"rs1","summary":[{"type":"summary_text","text":"private"}]},
|
||||
{"type":"message","id":"refusal1","role":"assistant","status":"completed",
|
||||
"content":[{"type":"refusal","refusal":"cannot help"}]}]),
|
||||
] {
|
||||
let fixture = Fixture::new(Session::new(vec![completed(output.clone())])).await;
|
||||
let response = fixture.post(chat_request(false)).await;
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
let body = json_body(response).await;
|
||||
let choice = &body["choices"][0];
|
||||
assert_eq!(choice["message"]["content"], Value::Null);
|
||||
assert!(!body.to_string().contains("private"));
|
||||
if output[0]["type"] == "function_call" {
|
||||
assert_eq!(choice["finish_reason"], "tool_calls");
|
||||
assert_eq!(
|
||||
choice["message"]["tool_calls"][0]["function"]["arguments"],
|
||||
"{ \"city\" : \"北京\" }"
|
||||
);
|
||||
assert_eq!(choice["message"]["tool_calls"][0]["id"], "call1");
|
||||
} else {
|
||||
assert_eq!(choice["message"]["refusal"], "cannot help");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn sse_deduplicates_text_snapshots_and_emits_usage_only_when_requested() {
|
||||
for include_usage in [false, true] {
|
||||
let fixture = Fixture::new(Session::new(vec![created(),
|
||||
wire("response.output_text.delta",json!({"item_id":"msg_chat_test","output_index":0,"content_index":0,"delta":"你好 "})),
|
||||
wire("response.output_text.done",json!({"item_id":"msg_chat_test","output_index":0,"content_index":0,"text":"你好 🌍"})),
|
||||
wire("response.content_part.done",json!({"item_id":"msg_chat_test","output_index":0,"content_index":0,"part":{"type":"output_text","text":"你好 🌍"}})),
|
||||
wire("response.output_item.done",json!({"output_index":0,"item":message("你好 🌍")})),
|
||||
completed(json!([message("你好 🌍")]))])).await;
|
||||
let mut body = chat_request(true);
|
||||
body["stream_options"] = json!({"include_usage":include_usage});
|
||||
let (_, chunks) = sse_body(fixture.post(body).await).await;
|
||||
assert_eq!(combined(&chunks, "content"), "你好 🌍");
|
||||
assert_eq!(
|
||||
chunks
|
||||
.iter()
|
||||
.filter(|chunk| chunk["choices"][0]["delta"]["role"] == "assistant")
|
||||
.count(),
|
||||
1
|
||||
);
|
||||
assert_eq!(
|
||||
chunks
|
||||
.iter()
|
||||
.filter(|chunk| chunk["choices"][0]["finish_reason"] == "stop")
|
||||
.count(),
|
||||
1
|
||||
);
|
||||
for chunk in &chunks {
|
||||
assert_eq!(chunk["object"], "chat.completion.chunk");
|
||||
assert_eq!(chunk["id"], chunks[0]["id"]);
|
||||
assert_eq!(chunk["created"], chunks[0]["created"]);
|
||||
assert_eq!(chunk["model"], "model-a");
|
||||
}
|
||||
let usage: Vec<_> = chunks
|
||||
.iter()
|
||||
.filter(|chunk| chunk["choices"] == json!([]))
|
||||
.collect();
|
||||
assert_eq!(usage.len(), usize::from(include_usage));
|
||||
if include_usage {
|
||||
assert_eq!(usage[0]["usage"]["total_tokens"], 18);
|
||||
}
|
||||
let captured = fixture.captured();
|
||||
assert!(captured.body.get("stream_options").is_none());
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn sse_preserves_interleaved_tools_and_reordered_terminal_snapshots() {
|
||||
let first = tool(
|
||||
"fc_first",
|
||||
"call_first",
|
||||
"weather",
|
||||
"{ \"city\": \"北京\" }",
|
||||
);
|
||||
let second = tool("fc_second", "call_second", "search", "{\"query\":\"Rust\"}");
|
||||
let fixture = Fixture::new(Session::new(vec![
|
||||
created(),
|
||||
wire(
|
||||
"response.output_item.added",
|
||||
json!({"output_index":2,"item":tool("fc_first","call_first","weather","")}),
|
||||
),
|
||||
wire(
|
||||
"response.output_item.added",
|
||||
json!({"output_index":5,"item":tool("fc_second","call_second","search","")}),
|
||||
),
|
||||
wire(
|
||||
"response.function_call_arguments.delta",
|
||||
json!({"output_index":2,"item_id":"fc_first","delta":"{ \"city\": "}),
|
||||
),
|
||||
wire(
|
||||
"response.function_call_arguments.delta",
|
||||
json!({"output_index":5,"item_id":"fc_second","delta":"{\"query\":"}),
|
||||
),
|
||||
wire(
|
||||
"response.function_call_arguments.delta",
|
||||
json!({"output_index":2,"item_id":"fc_first","delta":"\"北京\" }"}),
|
||||
),
|
||||
wire(
|
||||
"response.function_call_arguments.done",
|
||||
json!({"output_index":5,"item_id":"fc_second","arguments":second["arguments"]}),
|
||||
),
|
||||
wire(
|
||||
"response.output_item.done",
|
||||
json!({"output_index":2,"item":first}),
|
||||
),
|
||||
completed(json!([second, first])),
|
||||
]))
|
||||
.await;
|
||||
let (_, chunks) = sse_body(fixture.post(chat_request(true)).await).await;
|
||||
let mut tools = BTreeMap::<u64, (String, String, String)>::new();
|
||||
for chunk in &chunks {
|
||||
if let Some(calls) = chunk["choices"][0]["delta"]["tool_calls"].as_array() {
|
||||
for call in calls {
|
||||
let entry = tools.entry(call["index"].as_u64().unwrap()).or_default();
|
||||
if let Some(id) = call["id"].as_str() {
|
||||
entry.0.push_str(id);
|
||||
}
|
||||
if let Some(name) = call["function"]["name"].as_str() {
|
||||
entry.1.push_str(name);
|
||||
}
|
||||
if let Some(arguments) = call["function"]["arguments"].as_str() {
|
||||
entry.2.push_str(arguments);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
assert_eq!(
|
||||
tools,
|
||||
BTreeMap::from([
|
||||
(
|
||||
0,
|
||||
(
|
||||
"call_first".into(),
|
||||
"weather".into(),
|
||||
"{ \"city\": \"北京\" }".into()
|
||||
)
|
||||
),
|
||||
(
|
||||
1,
|
||||
(
|
||||
"call_second".into(),
|
||||
"search".into(),
|
||||
"{\"query\":\"Rust\"}".into()
|
||||
)
|
||||
),
|
||||
])
|
||||
);
|
||||
assert_eq!(
|
||||
chunks.last().unwrap()["choices"][0]["finish_reason"],
|
||||
"tool_calls"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn extra_attributes_and_auxiliary_events_preserve_existing_chat_results() {
|
||||
let mut text = message("preserved");
|
||||
text["future_optional"] = json!({"opaque":true});
|
||||
text["content"][0]["future_attribute"] = json!([1, 2, 3]);
|
||||
let fixture = Fixture::new(Session::new(vec![
|
||||
wire("response.future_auxiliary", json!({"future":true})),
|
||||
wire(
|
||||
"response.reasoning_summary_text.delta",
|
||||
json!({"output_index":0,"summary_index":0,"delta":"private"}),
|
||||
),
|
||||
created(),
|
||||
completed(json!([text])),
|
||||
]))
|
||||
.await;
|
||||
let (_, chunks) = sse_body(fixture.post(chat_request(true)).await).await;
|
||||
assert_eq!(combined(&chunks, "content"), "preserved");
|
||||
assert!(
|
||||
!chunks
|
||||
.iter()
|
||||
.any(|chunk| chunk.to_string().contains("private"))
|
||||
);
|
||||
assert_eq!(
|
||||
fixture
|
||||
.trace
|
||||
.discarded
|
||||
.load(std::sync::atomic::Ordering::SeqCst),
|
||||
2
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn incomplete_projects_only_supported_finish_reasons() {
|
||||
for (reason, expected) in [
|
||||
("max_output_tokens", "length"),
|
||||
("content_filter", "content_filter"),
|
||||
] {
|
||||
for streaming in [false, true] {
|
||||
let event = wire(
|
||||
"response.incomplete",
|
||||
json!({"response":{
|
||||
"id":"resp_chat_test","status":"incomplete","output":[message("partial")],
|
||||
"incomplete_details":{"reason":reason}}}),
|
||||
);
|
||||
let fixture = Fixture::new(Session::new(vec![created(), event])).await;
|
||||
let response = fixture.post(chat_request(streaming)).await;
|
||||
if streaming {
|
||||
let (_, chunks) = sse_body(response).await;
|
||||
assert_eq!(
|
||||
chunks.last().unwrap()["choices"][0]["finish_reason"],
|
||||
expected
|
||||
);
|
||||
} else {
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
assert_eq!(
|
||||
json_body(response).await["choices"][0]["finish_reason"],
|
||||
expected
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn encrypted_reasoning_stays_private_while_public_text_and_usage_are_projected() {
|
||||
let reasoning = json!({
|
||||
"id":"rs_private","type":"reasoning",
|
||||
"summary":[{"type":"summary_text","text":"PRIVATE_SUMMARY_MARKER"}],
|
||||
"encrypted_content":"OPAQUE_ENCRYPTED_REASONING_MARKER"
|
||||
});
|
||||
for streaming in [false, true] {
|
||||
let fixture = Fixture::new(Session::new(vec![
|
||||
created(),
|
||||
wire(
|
||||
"response.output_item.added",
|
||||
json!({"output_index":0,"item":{"id":"rs_private","type":"reasoning","summary":[]}}),
|
||||
),
|
||||
wire(
|
||||
"response.reasoning_summary_text.delta",
|
||||
json!({"item_id":"rs_private","output_index":0,"summary_index":0,"delta":"PRIVATE_SUMMARY_MARKER"}),
|
||||
),
|
||||
wire(
|
||||
"response.output_item.done",
|
||||
json!({"output_index":0,"item":reasoning}),
|
||||
),
|
||||
wire(
|
||||
"response.output_item.done",
|
||||
json!({"output_index":1,"item":message("public answer")}),
|
||||
),
|
||||
completed(json!([reasoning, message("public answer")])),
|
||||
]))
|
||||
.await;
|
||||
let mut request = chat_request(streaming);
|
||||
if streaming {
|
||||
request["stream_options"] = json!({"include_usage":true});
|
||||
}
|
||||
let response = fixture.post(request).await;
|
||||
let (body, content, usage) = if streaming {
|
||||
let (body, chunks) = sse_body(response).await;
|
||||
let usage = chunks
|
||||
.iter()
|
||||
.find(|chunk| chunk["choices"] == json!([]))
|
||||
.expect("terminal usage chunk")["usage"]
|
||||
.clone();
|
||||
(body, combined(&chunks, "content"), usage)
|
||||
} else {
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
let body = json_body(response).await;
|
||||
(
|
||||
body.to_string(),
|
||||
body["choices"][0]["message"]["content"]
|
||||
.as_str()
|
||||
.unwrap()
|
||||
.to_owned(),
|
||||
body["usage"].clone(),
|
||||
)
|
||||
};
|
||||
assert_eq!(content, "public answer");
|
||||
assert!(!body.contains("PRIVATE_SUMMARY_MARKER"), "{body}");
|
||||
assert!(
|
||||
!body.contains("OPAQUE_ENCRYPTED_REASONING_MARKER"),
|
||||
"{body}"
|
||||
);
|
||||
assert_eq!(usage["completion_tokens_details"]["reasoning_tokens"], 2);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn unrepresentable_business_output_fails_without_successful_chat_completion() {
|
||||
for output in [
|
||||
json!({"id":"future_output","type":"future_business_output","payload":"unrepresentable"}),
|
||||
json!({"id":"ig_native","type":"image_generation_call","status":"completed","result":"OPAQUE_IMAGE_MARKER"}),
|
||||
] {
|
||||
for streaming in [false, true] {
|
||||
let fixture =
|
||||
Fixture::new(Session::new(vec![created(), completed(json!([output]))])).await;
|
||||
let mut request = chat_request(streaming);
|
||||
if streaming {
|
||||
request["stream_options"] = json!({"include_usage":true});
|
||||
}
|
||||
let response = fixture.post(request).await;
|
||||
if streaming {
|
||||
let (body, chunks) = sse_body(response).await;
|
||||
assert!(chunks.iter().any(|chunk| chunk.get("error").is_some()));
|
||||
assert!(
|
||||
!chunks
|
||||
.iter()
|
||||
.any(|chunk| chunk["choices"][0]["finish_reason"].is_string())
|
||||
);
|
||||
assert!(!chunks.iter().any(|chunk| chunk["choices"] == json!([])));
|
||||
assert!(!body.contains("OPAQUE_IMAGE_MARKER"));
|
||||
} else {
|
||||
assert!(response.status().is_server_error());
|
||||
let body = json_body(response).await;
|
||||
assert!(body.get("error").is_some());
|
||||
assert!(body.get("choices").is_none());
|
||||
assert!(!body.to_string().contains("OPAQUE_IMAGE_MARKER"));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn stream_without_terminal_does_not_emit_a_successful_finish_or_usage() {
|
||||
let fixture = Fixture::new(Session::new(vec![
|
||||
created(),
|
||||
wire(
|
||||
"response.output_text.delta",
|
||||
json!({"item_id":"msg_chat_test","output_index":0,"content_index":0,"delta":"partial"}),
|
||||
),
|
||||
]))
|
||||
.await;
|
||||
let mut body = chat_request(true);
|
||||
body["stream_options"] = json!({"include_usage":true});
|
||||
let (_, chunks) = sse_body(fixture.post(body).await).await;
|
||||
assert!(chunks.iter().any(|chunk| chunk.get("error").is_some()));
|
||||
assert!(
|
||||
!chunks
|
||||
.iter()
|
||||
.any(|chunk| chunk["choices"][0]["finish_reason"].is_string())
|
||||
);
|
||||
assert!(!chunks.iter().any(|chunk| chunk["choices"] == json!([])));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn first_batch_terminal_can_finish_without_creating_usage_facts() {
|
||||
let fixture = Fixture::new(Session::new(vec![wire(
|
||||
"response.completed",
|
||||
json!({"response":{
|
||||
"id":"resp_chat_test","status":"completed","output":[message("done only")]
|
||||
}}),
|
||||
)]))
|
||||
.await;
|
||||
let mut body = chat_request(true);
|
||||
body["stream_options"] = json!({"include_usage":true});
|
||||
let (_, chunks) = sse_body(fixture.post(body).await).await;
|
||||
assert_eq!(combined(&chunks, "content"), "done only");
|
||||
assert_eq!(
|
||||
chunks
|
||||
.iter()
|
||||
.filter(|chunk| chunk["choices"][0]["finish_reason"] == "stop")
|
||||
.count(),
|
||||
1
|
||||
);
|
||||
assert!(!chunks.iter().any(|chunk| chunk["usage"].is_object()));
|
||||
assert!(
|
||||
fixture
|
||||
.trace
|
||||
.finalized
|
||||
.load(std::sync::atomic::Ordering::SeqCst)
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn native_sse_byte_fragments_and_multiple_events_preserve_unicode() {
|
||||
let first = gateway_protocol::openai::sse::encode_sse_event("response.output_text.delta",&json!({
|
||||
"type":"response.output_text.delta","item_id":"msg_chat_test","output_index":0,"content_index":0,"delta":"你"
|
||||
}).to_string());
|
||||
let second = gateway_protocol::openai::sse::encode_sse_event("response.output_text.delta",&json!({
|
||||
"type":"response.output_text.delta","item_id":"msg_chat_test","output_index":0,"content_index":0,"delta":"好"
|
||||
}).to_string());
|
||||
let split = first.find('你').unwrap() + 1;
|
||||
let mut remainder = first.as_bytes()[split..].to_vec();
|
||||
remainder.extend_from_slice(second.as_bytes());
|
||||
let raw = |bytes| {
|
||||
gateway_core::event::ProviderEvent::wire(
|
||||
gateway_core::event::ProtocolWireEvent::raw_sse("openai", bytes).unwrap(),
|
||||
)
|
||||
};
|
||||
let fixture = Fixture::new(Session::new(vec![
|
||||
created(),
|
||||
raw(bytes::Bytes::copy_from_slice(&first.as_bytes()[..split])),
|
||||
raw(bytes::Bytes::from(remainder)),
|
||||
completed(json!([message("你好")])),
|
||||
]))
|
||||
.await;
|
||||
let (_, chunks) = sse_body(fixture.post(chat_request(true)).await).await;
|
||||
assert_eq!(combined(&chunks, "content"), "你好");
|
||||
assert_eq!(
|
||||
chunks.last().unwrap()["choices"][0]["finish_reason"],
|
||||
"stop"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn refusal_deltas_and_done_snapshots_produce_only_standard_refusal() {
|
||||
let output = json!({"id":"msg_refusal","type":"message","role":"assistant","status":"completed",
|
||||
"content":[{"type":"refusal","refusal":"cannot help"}]});
|
||||
let fixture = Fixture::new(Session::new(vec![created(),
|
||||
wire("response.refusal.delta",json!({"item_id":"msg_refusal","output_index":0,"content_index":0,"delta":"cannot "})),
|
||||
wire("response.refusal.done",json!({"item_id":"msg_refusal","output_index":0,"content_index":0,"refusal":"cannot help"})),
|
||||
completed(json!([output]))])).await;
|
||||
let (_, chunks) = sse_body(fixture.post(chat_request(true)).await).await;
|
||||
assert_eq!(combined(&chunks, "refusal"), "cannot help");
|
||||
assert_eq!(combined(&chunks, "content"), "");
|
||||
assert_eq!(
|
||||
chunks.last().unwrap()["choices"][0]["finish_reason"],
|
||||
"stop"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn auxiliary_events_after_completed_preserve_exactly_one_finish_and_usage() {
|
||||
for with_created in [false, true] {
|
||||
// 终态与辅助事件同属一批 Core 交付 不构造已终结会话继续产出事件的另一种异常
|
||||
let session = Session::new(if with_created {
|
||||
vec![created()]
|
||||
} else {
|
||||
Vec::new()
|
||||
})
|
||||
.with_batch(vec![
|
||||
completed(json!([message("completed text")])),
|
||||
wire(
|
||||
"response.future_metadata",
|
||||
json!({"response_id":"resp_chat_test","future":{"kept":true}}),
|
||||
),
|
||||
]);
|
||||
let fixture = Fixture::new(session).await;
|
||||
let mut body = chat_request(true);
|
||||
body["stream_options"] = json!({"include_usage":true});
|
||||
let (_, chunks) = sse_body(fixture.post(body).await).await;
|
||||
assert_eq!(combined(&chunks, "content"), "completed text");
|
||||
assert_eq!(
|
||||
chunks
|
||||
.iter()
|
||||
.filter(|chunk| chunk["choices"][0]["finish_reason"] == "stop")
|
||||
.count(),
|
||||
1
|
||||
);
|
||||
let usage = chunks
|
||||
.iter()
|
||||
.filter(|chunk| chunk["choices"] == json!([]))
|
||||
.collect::<Vec<_>>();
|
||||
assert_eq!(usage.len(), 1);
|
||||
assert_eq!(usage[0]["usage"]["total_tokens"], 18);
|
||||
assert!(!chunks.iter().any(|chunk| chunk.get("error").is_some()));
|
||||
assert_eq!(
|
||||
fixture
|
||||
.trace
|
||||
.commits
|
||||
.load(std::sync::atomic::Ordering::SeqCst),
|
||||
1
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn wire_failures_preserve_existing_recovery_fields_without_success_chunks() {
|
||||
for shape in ["response.failed", "error", "bare_error"] {
|
||||
let mut expected = json!({
|
||||
"type":"rate_limit_error","code":"rate_limit_exceeded","param":null,
|
||||
"message":"retry later","headers":{"retry-after":"2"},
|
||||
"recovery":{"action":"retry","future":[1,2]},
|
||||
"future_error_property":{"enabled":true}
|
||||
});
|
||||
let failure = match shape {
|
||||
"response.failed" => wire(
|
||||
"response.failed",
|
||||
json!({"response":{
|
||||
"id":"resp_chat_test","object":"response","model":"upstream-model",
|
||||
"status":"failed","output":[],"error":expected.clone()
|
||||
}}),
|
||||
),
|
||||
"error" => wire("error", json!({"error":expected.clone()})),
|
||||
_ => {
|
||||
// 裸事件的 type 是 wire 名称,仅该包装身份归一化为 Chat 错误类型
|
||||
let failure = wire("error", expected.clone());
|
||||
expected["type"] = json!("server_error");
|
||||
failure
|
||||
}
|
||||
};
|
||||
let fixture = Fixture::new(Session::new(vec![created(), failure])).await;
|
||||
let mut body = chat_request(true);
|
||||
body["stream_options"] = json!({"include_usage":true});
|
||||
let (_, chunks) = sse_body(fixture.post(body).await).await;
|
||||
let errors = chunks
|
||||
.iter()
|
||||
.filter_map(|chunk| chunk.get("error"))
|
||||
.collect::<Vec<_>>();
|
||||
assert_eq!(errors, vec![&expected], "{shape}");
|
||||
assert!(
|
||||
!chunks
|
||||
.iter()
|
||||
.any(|chunk| chunk["choices"][0]["finish_reason"].is_string()),
|
||||
"{shape}"
|
||||
);
|
||||
assert!(!chunks.iter().any(|chunk| chunk["choices"] == json!([])));
|
||||
assert!(!chunks.iter().any(|chunk| chunk["usage"].is_object()));
|
||||
assert_eq!(
|
||||
fixture
|
||||
.trace
|
||||
.commits
|
||||
.load(std::sync::atomic::Ordering::SeqCst),
|
||||
1
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn conflicting_text_snapshot_fails_instead_of_appending_replacement_content() {
|
||||
let fixture = Fixture::new(Session::new(vec![created(),
|
||||
wire("response.output_text.delta",json!({"item_id":"msg_chat_test","output_index":0,"content_index":0,"delta":"already sent"})),
|
||||
completed(json!([message("different result")]))])).await;
|
||||
let (_, chunks) = sse_body(fixture.post(chat_request(true)).await).await;
|
||||
assert_eq!(combined(&chunks, "content"), "already sent");
|
||||
assert!(chunks.iter().any(|chunk| chunk.get("error").is_some()));
|
||||
assert!(
|
||||
!chunks
|
||||
.iter()
|
||||
.any(|chunk| chunk["choices"][0]["finish_reason"].is_string())
|
||||
);
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user