Compare commits

...
12 Commits
Author SHA1 Message Date
ZyyandCodex 9039cef3d8 chore(release): v3.41.0
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 12:18:54 +08:00
a9a77f5f9d fix(accounts): calculate model success rates from request outcomes (#611)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 12:14:27 +08:00
ZyyandCodex 6f7d92594e fix(frontend): defer incomplete configuration errors until submit
Avoid validating model access and plugin forms when switching modes.
Keep error protection drafts quiet until input or submission, and retain
condition identity when editing or removing entries.

Co-authored-by: Codex <noreply@openai.com>
2026-11-05 12:11:52 +08:00
6859520969 fix(api): broaden Chat compatibility and capture ingress failures (#610)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 11:53:21 +08:00
ZyyandCodex 21d84af2d8 fix: simplify user-facing messages and error feedback
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 00:48:19 +08:00
ZyyandCodex 311e4d228f docs(contributing): prioritize concise text in development and feedback
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 00:47:45 +08:00
ZyyandCodex d604adfbeb fix(login): center the sign-in form on mobile
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 00:47:45 +08:00
ZyyandCodex 46b4665c5b feat(openai): allow API keys to override the installation ID
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 00:31:37 +08:00
ZyyandCodex 1daa0623a0 fix(openai): preserve client turn state without local ownership records
Preserve client-supplied turn state on full requests with unknown ownership
and remove the process-local proof cache. Keep confirmed account-switch
cleanup and existing continuation authorization boundaries.

Cover HTTP and WebSocket retries, provider recreation, expired affinity,
and conflicting account ownership; update the API and architecture docs.

Co-authored-by: Codex <noreply@openai.com>
2026-11-05 00:14:53 +08:00
ZyyandCodex 860fb39622 fix(frontend): satisfy account formatting rules
Co-authored-by: Codex <noreply@openai.com>
2026-11-04 22:49:25 +08:00
f5c5a6e5b7 feat(proxies): test account egress and assign accounts in bulk (#607)
* feat(proxies): test account egress and assign accounts in bulk

Co-authored-by: Codex <noreply@openai.com>

* fix(accounts): refine proxy controls and quota display

Keep egress results with their proxy options and refresh stale proxy revisions before retrying. Compact account assignment and settings controls, and hide missing quota refresh timestamps.

Co-authored-by: Codex <noreply@openai.com>

* fix(proxies): reset account selection without closing dialog

Co-authored-by: Codex <noreply@openai.com>

* fix(proxies): clarify assignment reset and unavailable accounts

Use a red reset button and disabled checkboxes for accounts already using the target proxy.

Co-authored-by: Codex <noreply@openai.com>

* feat(proxies): show account details in assignment dialog

Add platform, subscription, and group columns, remove inline provider icons, and use the existing subtle danger style for reset.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-11-04 22:42:16 +08:00
ZyyandCodex bdfc201795 fix(peer-link): update account identity on definitive sharing results
Clear the peer badge when the upstream explicitly disables sharing or does not support CPR Link. Preserve identity on transient failures and allow manual rediscovery from the quota panel.

Co-authored-by: Codex <noreply@openai.com>
2026-11-04 19:58:55 +08:00
164 changed files with 3890 additions and 1268 deletions
+8 -1
View File
@@ -5,6 +5,13 @@ description: Codex Proxy RS 开发、修复、排障、代码审查和文档修
# 开发指南
## 首要写作要求:言简意赅
所有面向人的文字都按此要求编写和审查,包括界面、通知、后端错误、任务状态、命令行、日志、注释、文档和交付说明
- 先写读者需要知道的事实或动作。删掉一句不影响理解、判断或操作,就删;标题、正文和通知不重复同一信息
- 保留会改变决定的条件、权限、后果和恢复步骤。错误通知只说已知问题,有明确办法才补一句;后端返回的提示在后端改,前端不重复解释。上游原文、用户内容和诊断事实保持原样
## 先确定领域与必读规范
按用户意图、受影响行为与对象初判,定位后按实际文件复核,不要求用户说出“前端”“后端”或技能名。**所有命中行共同构成本次必读规范,先读完再确定方案、修改或给出审查结论**;跨领域叠加,不能只选最像当前步骤的一行
@@ -39,7 +46,7 @@ description: Codex Proxy RS 开发、修复、排障、代码审查和文档修
2. 以最终差异重新匹配上表,补齐新增领域的规范并检查实际实现,再处理本次引入的职责越界、重复规则、冗余状态与无依据分支;删除或替换功能时一并清理失效的配置、类型、调用、测试与文档,已部署数据和公开合同仍遵守所属迁移及兼容规则;仅审查时报告,不自行修复
3. 按最终行为执行[文档检查](references/documentation.md),核对所属正文及相关概述、示例和引用,在本任务内完成必要同步;不以尚无文档差异跳过检查。内部重构等未改变文档所述事实时无需改文档;常规文档只写当前状态,普通开发不自动修改 `release/notes.md`
4. 按变更范围执行[验证](../../../CONTRIBUTING.md#验证)及其停止条件,后端命令见[验证命令](../../../docs/architecture.md#验证命令);沿用本轮有效证据与用户已确定的验收分工,区分通过、失败、跳过与未执行;提交时补读[提交约定](../../../CONTRIBUTING.md#项目约定)
5. 交付实际结果与验证缺口;前端按[前端流程](references/frontend.md)核对最终差异和运行证据,后端按上表检查生命周期,不把构建通过当作行为验收
5. 交付前按[首要写作要求](#首要写作要求言简意赅)通读本次新增或修改的文字。交代实际结果与验证缺口;前端按[前端流程](references/frontend.md)核对运行证据,后端按上表检查生命周期,不把构建通过当作行为验收
## 测试维护
@@ -139,7 +139,6 @@ body:
id: expected-behavior
attributes:
label: 预期行为
description: 说明原本期望得到的结果。
validations:
required: true
@@ -108,7 +108,6 @@ body:
id: expected-behavior
attributes:
label: 预期行为
description: 说明原本期望得到的结果。
validations:
required: true
+1 -1
View File
@@ -91,7 +91,7 @@ API Key 持有者可在登录页切换身份,查看自己的用量与额度,
1. 在「账号」中添加账号,完成授权或导入
2. 按需建立账号分组,再创建客户端密钥并选择可用分组,**不选分组表示可使用全部账号**
3. 打开密钥的「使用密钥」,复制客户端配置
3. [连接客户端](#客户端接入)
## 客户端接入
+1 -1
View File
@@ -91,7 +91,7 @@ fn runtime() -> Result<tokio::runtime::Runtime, io::Error> {
fn reject_extra_arguments(mut arguments: impl Iterator) -> Result<(), io::Error> {
if arguments.next().is_some() {
return Err(invalid_cli("unexpected extra command arguments"));
return Err(invalid_cli("too many arguments"));
}
Ok(())
}
+1 -1
View File
@@ -15,7 +15,7 @@ cargo +1.99.1 install --locked --path backend/apps/plugin-cli --root /path/to/lo
## 打包
先构建目标平台的插件二进制;插件提供管理页面时,还需构建页面静态资源。打包命令传入作者清单、二进制和所需的资源目录映射,通过 `cpr-plugin package --help` 查看完整参数:
先构建目标平台的二进制;有管理页面时,也构建静态资源。全部参数见 `cpr-plugin package --help`
```bash
cpr-plugin package \
@@ -472,6 +472,80 @@ impl CostCoverage {
}
}
/// 请求健康计数与服务成功率,调用方错误和取消不计入有效样本
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct RequestHealth {
pub success_requests: u64,
pub failed_requests: u64,
pub cancelled_requests: u64,
pub incomplete_requests: u64,
pub caller_error_requests: u64,
}
impl RequestHealth {
#[must_use]
pub fn from_counts(
success: u64,
failed: u64,
cancelled: u64,
incomplete: u64,
caller_error: u64,
) -> Self {
Self {
success_requests: success,
failed_requests: failed.saturating_sub(caller_error),
cancelled_requests: cancelled,
incomplete_requests: incomplete,
caller_error_requests: caller_error,
}
}
pub(crate) fn from_metrics(metrics: &RequestMetrics) -> Self {
Self::from_counts(
metrics.success_count,
metrics.failure_count,
metrics.cancelled_count,
metrics.incomplete_count,
metrics.caller_error_count,
)
}
#[must_use]
pub fn success_rate(self) -> Option<f64> {
let total = self.eligible_requests();
(total > 0).then(|| self.success_requests as f64 / total as f64 * 100.0)
}
// 未完成保留交付终态,但上游失败、超时和进程中断都不能视为服务成功
pub(crate) fn non_success_requests(self) -> u64 {
self.failed_requests
.saturating_add(self.incomplete_requests)
}
pub(crate) fn eligible_requests(self) -> u64 {
self.success_requests
.saturating_add(self.non_success_requests())
}
pub(crate) fn add_metrics(&mut self, metrics: &RequestMetrics) {
self.add_window(Self::from_metrics(metrics));
}
pub(crate) fn add_window(&mut self, other: Self) {
self.success_requests = self.success_requests.saturating_add(other.success_requests);
self.failed_requests = self.failed_requests.saturating_add(other.failed_requests);
self.cancelled_requests = self
.cancelled_requests
.saturating_add(other.cancelled_requests);
self.incomplete_requests = self
.incomplete_requests
.saturating_add(other.incomplete_requests);
self.caller_error_requests = self
.caller_error_requests
.saturating_add(other.caller_error_requests);
}
}
/// 请求级聚合指标
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct RequestMetrics {
@@ -622,7 +696,7 @@ pub struct AccountPoolMetrics {
pub struct AccountModelUsage {
pub model: String,
pub request_count: u64,
pub success_count: u64,
pub health: RequestHealth,
pub input_tokens: Option<u64>,
pub output_tokens: Option<u64>,
pub cached_input_tokens: Option<u64>,
@@ -635,7 +709,7 @@ pub struct AccountModelUsage {
pub total_tokens: Option<u64>,
pub cost_coverage: CostCoverage,
pub costs: Vec<CurrencyCost>,
pub last_used_at: DateTime<Utc>,
pub last_used_at: Option<DateTime<Utc>>,
}
/// 账号在单小时窗口内的请求数
@@ -4,7 +4,7 @@ use chrono::{DateTime, Utc};
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ProviderUpstreamInfo {
/// 当前连接曾成功识别互联,失权时保留身份但清除观测内容
/// 当前连接的互联识别结果,明确关闭共享或不支持协议时清除
pub discovered: bool,
pub observed_at: Option<DateTime<Utc>>,
pub stale: bool,
@@ -222,6 +222,15 @@ pub trait ProviderAdmin: Send + Sync {
Err(ProviderAdminError::new(ProviderAdminErrorKind::Unsupported))
}
/// Key 可以继承当前全局画像;额外字段的合法性由对应 Provider 判定
fn preview_key_client_profile(
&self,
key: &gateway_core::account::OpaqueProviderData,
_global: Option<&gateway_core::account::OpaqueProviderData>,
) -> Result<gateway_core::account::OpaqueProviderData, ProviderAdminError> {
self.preview_client_profile(key)
}
/// 提供最终套餐展示名称;未覆盖时采用通用的大驼峰格式
fn plan_type_display(&self, plan_type: &str) -> String {
default_plan_type_display(plan_type)
@@ -162,7 +162,7 @@ impl BackupService for DefaultBackupService {
ok: false,
stage: "putObject",
code: Some(code::INVALID_CONFIG),
message: "备份存储配置不完整,无法执行连接测试".to_owned(),
message: "备份存储配置不完整".to_owned(),
});
};
let result = self
@@ -198,14 +198,10 @@ impl BackupService for DefaultBackupService {
.await
.map_err(|error| map_store_error(error, "backup settings"))?;
if !settings.storage_configured() {
return Err(AdminError::bad_request(
"备份存储配置不完整,请先保存存储配置",
));
return Err(AdminError::bad_request("请先补全并保存备份存储配置"));
}
if !settings.storage_verified() {
return Err(AdminError::bad_request(
"备份存储尚未通过连接测试,请先测试存储配置",
));
return Err(AdminError::bad_request("请先通过备份存储连接测试"));
}
let prefix = settings.prefix.as_deref().unwrap_or_default();
let seed = build_backup_seed(
@@ -241,9 +237,7 @@ impl BackupService for DefaultBackupService {
) -> Result<DownloadUrlResult, AdminError> {
let record = self.require_record(backup_id).await?;
if record.status != BackupStatus::Completed {
return Err(AdminError::conflict(
"只有 completed 的备份才能创建下载地址",
));
return Err(AdminError::conflict("只能下载已完成的备份"));
}
let settings = self
.repository
@@ -280,9 +274,7 @@ impl BackupService for DefaultBackupService {
) -> Result<BackupRecord, AdminError> {
let record = self.require_record(backup_id).await?;
if !record.status.can_be_deleted() {
return Err(AdminError::conflict(
"只有 completed 或 failed 的备份才能删除",
));
return Err(AdminError::conflict("只能删除已完成或失败的备份"));
}
let now = Utc::now();
let deleting = self
@@ -290,7 +282,7 @@ impl BackupService for DefaultBackupService {
.transition_to_deleting(backup_id, now, Some(context))
.await
.map_err(|error| map_store_error(error, "backup record"))?
.ok_or_else(|| AdminError::conflict("备份记录已进入删除流程"))?;
.ok_or_else(|| AdminError::conflict("备份状态已变更,请刷新列表"))?;
Ok(deleting)
}
}
@@ -162,7 +162,7 @@ impl ClientKeyService for DefaultClientKeyService {
for (provider, profile) in &command.request_profile_overrides {
self.providers
.require(provider)
.and_then(|provider| provider.preview_client_profile(profile))
.and_then(|provider| provider.preview_key_client_profile(profile, None))
.map_err(|error| super::map_provider_error(error, "client profile"))?;
}
validate_peer_sharing(command.peer_sharing, &command.group_ids)?;
@@ -209,7 +209,7 @@ impl ClientKeyService for DefaultClientKeyService {
if let Some(profile) = profile {
self.providers
.require(provider)
.and_then(|provider| provider.preview_client_profile(profile))
.and_then(|provider| provider.preview_key_client_profile(profile, None))
.map_err(|error| super::map_provider_error(error, "client profile"))?;
}
}
@@ -207,7 +207,7 @@ impl DefaultImportTasksService {
}
Err(_) => {
item.status = ImportItemStatus::Unknown;
item.message = Some("执行中断,请先核对账号列表再决定是否重新导入".to_owned());
item.message = Some("导入结果未知,重新导入前请核对账号列表".to_owned());
}
}
entry.finish_if_done();
@@ -19,8 +19,8 @@ use crate::{
DashboardPeriodMetrics, DashboardQuery, DashboardResult, DecimalAmount,
DiagnosticDimension, DiagnosticsItem, DiagnosticsResult, Granularity, HealthStatus,
HealthTimeline, HealthTimelinePoint, OpsErrorPage, OpsErrorQuery, ProviderBillingInput,
RequestMetricPoint, RequestMetrics, TimeRange, Trend, TrendKind, TrendPoint,
TrendSummary, UsageBilling, UsageBillingProjection, UsageCacheEfficiency,
RequestHealth, RequestMetricPoint, RequestMetrics, TimeRange, Trend, TrendKind,
TrendPoint, TrendSummary, UsageBilling, UsageBillingProjection, UsageCacheEfficiency,
UsageCalculatedBillingFact, UsageConcurrencyPoint, UsageCostComparison,
UsageCostSummary, UsageDetail, UsageFilter, UsageInsight, UsageInsightData,
UsageInsightMetric, UsageInsights, UsageInsightsCost, UsageInsightsCostPoint,
@@ -574,7 +574,7 @@ fn build_usage_insights(
}
let requests = &overview.requests;
let health_window = HealthWindow::from_metrics(requests);
let health_window = RequestHealth::from_metrics(requests);
let failed_requests = health_window.failed_requests;
let health_requests = health_window.eligible_requests();
let health = UsageInsightsHealth {
@@ -594,7 +594,7 @@ fn build_usage_insights(
points: trend
.iter()
.map(|point| {
let health_window = HealthWindow::from_metrics(&point.metrics);
let health_window = RequestHealth::from_metrics(&point.metrics);
UsageInsightsHealthPoint {
total_requests: point.metrics.request_count,
bucket_start: point.bucket_start,
@@ -1125,7 +1125,7 @@ pub(super) fn health_timeline_at(
.map(|index| {
(
start + Duration::minutes(HEALTH_TIMELINE_SLOT_MINUTES * index),
HealthWindow::default(),
RequestHealth::default(),
)
})
.collect::<Vec<_>>();
@@ -1151,7 +1151,7 @@ pub(super) fn health_timeline_at(
let totals = buckets
.iter()
.filter(|(bucket_start, _)| *bucket_start <= current_slot)
.fold(HealthWindow::default(), |mut totals, (_, bucket)| {
.fold(RequestHealth::default(), |mut totals, (_, bucket)| {
totals.add_window(*bucket);
totals
});
@@ -1215,10 +1215,10 @@ fn trend(
}
}
let average_latency_ms = average(metrics.latency_sum_ms, metrics.latency_count);
let service_failure_count = service_failure_count(&metrics);
let service_failure_count = RequestHealth::from_metrics(&metrics).failed_requests;
let success_rate = ratio(
metrics.success_count,
HealthWindow::from_metrics(&metrics).eligible_requests(),
RequestHealth::from_metrics(&metrics).eligible_requests(),
);
let cache_hit_request_rate = ratio(
metrics.cache_hit_request_count,
@@ -1267,7 +1267,7 @@ fn trend(
}
fn trend_point(point: RequestMetricPoint) -> TrendPoint {
let service_failure_count = service_failure_count(&point.metrics);
let service_failure_count = RequestHealth::from_metrics(&point.metrics).failed_requests;
TrendPoint {
bucket_start: point.bucket_start,
granularity: point.granularity,
@@ -1287,7 +1287,7 @@ fn trend_point(point: RequestMetricPoint) -> TrendPoint {
),
success_rate: ratio(
point.metrics.success_count,
HealthWindow::from_metrics(&point.metrics).eligible_requests(),
RequestHealth::from_metrics(&point.metrics).eligible_requests(),
),
metrics: point.metrics,
cost_coverage: point.cost_coverage,
@@ -1372,57 +1372,7 @@ fn rate_or_zero(numerator: u64, denominator: u64) -> f64 {
ratio(numerator, denominator).unwrap_or(0.0)
}
#[derive(Debug, Clone, Copy, Default)]
struct HealthWindow {
success_requests: u64,
failed_requests: u64,
cancelled_requests: u64,
incomplete_requests: u64,
caller_error_requests: u64,
}
impl HealthWindow {
fn from_metrics(metrics: &RequestMetrics) -> Self {
Self {
success_requests: metrics.success_count,
failed_requests: service_failure_count(metrics),
cancelled_requests: metrics.cancelled_count,
incomplete_requests: metrics.incomplete_count,
caller_error_requests: metrics.caller_error_count,
}
}
// 未完成保留交付终态,但上游失败、超时和进程中断都不能视为服务成功
fn non_success_requests(self) -> u64 {
self.failed_requests
.saturating_add(self.incomplete_requests)
}
fn eligible_requests(self) -> u64 {
self.success_requests
.saturating_add(self.non_success_requests())
}
fn add_metrics(&mut self, metrics: &RequestMetrics) {
self.add_window(Self::from_metrics(metrics));
}
fn add_window(&mut self, other: Self) {
self.success_requests = self.success_requests.saturating_add(other.success_requests);
self.failed_requests = self.failed_requests.saturating_add(other.failed_requests);
self.cancelled_requests = self
.cancelled_requests
.saturating_add(other.cancelled_requests);
self.incomplete_requests = self
.incomplete_requests
.saturating_add(other.incomplete_requests);
self.caller_error_requests = self
.caller_error_requests
.saturating_add(other.caller_error_requests);
}
}
fn health_status(bucket: HealthWindow, is_future: bool) -> HealthStatus {
fn health_status(bucket: RequestHealth, is_future: bool) -> HealthStatus {
let eligible_requests = bucket.eligible_requests();
if is_future {
HealthStatus::Future
@@ -1443,14 +1393,6 @@ fn health_status(bucket: HealthWindow, is_future: bool) -> HealthStatus {
}
}
fn health_reliability(bucket: HealthWindow) -> Option<f64> {
let eligible_requests = bucket.eligible_requests();
(eligible_requests > 0)
.then(|| bucket.success_requests as f64 / eligible_requests as f64 * 100.0)
}
fn service_failure_count(metrics: &RequestMetrics) -> u64 {
metrics
.failure_count
.saturating_sub(metrics.caller_error_count)
fn health_reliability(bucket: RequestHealth) -> Option<f64> {
bucket.success_rate()
}
@@ -56,8 +56,7 @@ impl PeerInfoService {
checked_at: now,
scheduling: None,
summary: None,
unsupported: false,
unavailable: None,
stored_failure: None,
},
));
}
@@ -93,21 +92,19 @@ impl PeerInfoService {
(
connection.clone(),
cached.and_then(|cached| cached.summary.clone()),
cached.and_then(|cached| cached.unavailable),
cached.is_some_and(|cached| cached.unsupported),
cached.and_then(|cached| cached.stored_failure),
)
})
.collect::<Vec<_>>()
};
Ok(summaries
.into_iter()
.map(|(connection, peer, unavailable, unsupported)| {
.map(|(connection, peer, stored_failure)| {
let quota = Self::project_peer(
&connection,
peer.as_deref(),
unavailable,
self.last_error(&connection)
.or(unsupported.then_some(UpstreamInfoFailure::Unsupported)),
stored_failure,
self.last_error(&connection),
now,
);
(connection, quota)
@@ -138,10 +135,8 @@ impl PeerInfoService {
checked_at: now,
scheduling,
summary,
unsupported: observation
.is_some_and(|value| snapshot::is_unsupported(&value.quota, connection)),
unavailable: observation
.and_then(|value| snapshot::unavailable_reason(&value.quota, connection)),
stored_failure: observation
.and_then(|value| snapshot::stored_failure(&value.quota, connection)),
};
let previous = self
.observations
@@ -156,6 +151,9 @@ impl PeerInfoService {
.read()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.get(&connection.account_id)
.is_some_and(|cached| cached.connection == *connection && cached.unsupported)
.is_some_and(|cached| {
cached.connection == *connection
&& cached.stored_failure == Some(UpstreamInfoFailure::Unsupported)
})
}
}
@@ -40,7 +40,7 @@ impl PeerInfoService {
pub(super) fn project_peer(
connection: &UpstreamConnection,
peer: Option<&PeerLink>,
unavailable: Option<UpstreamInfoFailure>,
stored_failure: Option<UpstreamInfoFailure>,
last_error: Option<UpstreamInfoFailure>,
now: SystemTime,
) -> ProviderQuota {
@@ -52,12 +52,12 @@ impl PeerInfoService {
});
let limits = peer.and_then(|peer| peer.key_limits);
let info = ProviderUpstreamInfo {
discovered: peer.is_some() || unavailable.is_some(),
discovered: peer.is_some() || stored_failure == Some(UpstreamInfoFailure::Unauthorized),
observed_at,
stale,
connection_revision: connection.revision.clone(),
last_error: last_error
.or(unavailable)
.or(stored_failure)
.map(|error| error.as_str().to_owned()),
max_concurrency: limits.and_then(|limits| limits.max_concurrency),
requests_per_minute: limits.and_then(|limits| limits.requests_per_minute),
@@ -105,8 +105,8 @@ impl PeerInfoService {
{
let error = state.last_error;
drop(state);
if let Some(error) = error.filter(|error| *error != UpstreamInfoFailure::Unsupported) {
return Err(refresh_error(error));
if let Some(error) = error.and_then(refresh_error) {
return Err(error);
}
return self
.read_quota(id)
@@ -130,7 +130,9 @@ impl PeerInfoService {
let failure = result.as_ref().err().copied();
let delay = match failure {
None => Duration::from_secs(240 + stable_jitter(id.as_str())),
Some(UpstreamInfoFailure::Unauthorized) => Duration::from_secs(1800),
Some(UpstreamInfoFailure::Unauthorized | UpstreamInfoFailure::SharingDisabled) => {
Duration::from_secs(1800)
}
Some(_) => Duration::from_secs(30),
}
.max(retry_after.unwrap_or_default());
@@ -141,7 +143,11 @@ impl PeerInfoService {
state.last_error = failure;
let document = match result {
Ok(document) => Some(document),
Err(error @ (UpstreamInfoFailure::Unauthorized | UpstreamInfoFailure::Unsupported)) => {
Err(UpstreamInfoFailure::Unsupported) => Some(snapshot::unsupported(&connection)),
Err(error @ UpstreamInfoFailure::SharingDisabled) => {
Some(snapshot::unavailable(&connection, error))
}
Err(error @ UpstreamInfoFailure::Unauthorized) => {
let discovered = self
.read_summary(id)
.await
@@ -153,8 +159,6 @@ impl PeerInfoService {
// 首次探测失败不能证明对方是互联节点;已确认的连接只留下无共享内容的识别记录
Some(if discovered {
snapshot::unavailable(&connection, error)
} else if error == UpstreamInfoFailure::Unsupported {
snapshot::unsupported(&connection)
} else {
OpaqueProviderData::new(Default::default())
})
@@ -199,8 +203,8 @@ impl PeerInfoService {
}
drop(state);
// 不支持协议是正常发现结果,管理端保留分类,后台不重复报警
if let Some(error) = failure.filter(|error| *error != UpstreamInfoFailure::Unsupported) {
return Err(refresh_error(error));
if let Some(error) = failure.and_then(refresh_error) {
return Err(error);
}
self.read_quota(id)
.await
@@ -215,12 +219,13 @@ fn stable_jitter(id: &str) -> u64 {
}) % 60
}
fn refresh_error(error: UpstreamInfoFailure) -> ProviderAdminError {
ProviderAdminError::new(ProviderAdminErrorKind::BadGateway).with_public_message(match error {
fn refresh_error(error: UpstreamInfoFailure) -> Option<ProviderAdminError> {
let message = match error {
UpstreamInfoFailure::Unauthorized => "上游未授权读取互联信息",
UpstreamInfoFailure::Unsupported => "上游不支持互联信息查询",
UpstreamInfoFailure::Unsupported | UpstreamInfoFailure::SharingDisabled => return None,
UpstreamInfoFailure::RateLimited => "上游互联查询已限流",
UpstreamInfoFailure::InvalidResponse => "上游互联响应无效",
UpstreamInfoFailure::Unavailable => "上游互联服务暂不可用",
})
};
Some(ProviderAdminError::new(ProviderAdminErrorKind::BadGateway).with_public_message(message))
}
@@ -34,8 +34,7 @@ pub(super) struct CachedObservation {
pub checked_at: SystemTime,
pub scheduling: Option<Arc<SchedulingObservation>>,
pub summary: Option<Arc<PeerLink>>,
pub unsupported: bool,
pub unavailable: Option<UpstreamInfoFailure>,
pub stored_failure: Option<UpstreamInfoFailure>,
}
pub(crate) struct PeerInfoService {
@@ -173,21 +172,17 @@ impl PeerInfoService {
let peer = observation
.as_ref()
.and_then(|value| snapshot::decode(&value.quota));
let unsupported = observation
let stored_failure = observation
.as_ref()
.is_some_and(|value| snapshot::is_unsupported(&value.quota, &connection));
let unavailable = observation
.as_ref()
.and_then(|value| snapshot::unavailable_reason(&value.quota, &connection));
.and_then(|value| snapshot::stored_failure(&value.quota, &connection));
if self.current_connection(id).await?.as_ref() != Some(&connection) {
return Ok(None);
}
Ok(Some(Self::project_peer(
&connection,
peer.as_ref(),
unavailable,
self.last_error(&connection)
.or(unsupported.then_some(UpstreamInfoFailure::Unsupported)),
stored_failure,
self.last_error(&connection),
SystemTime::now(),
)))
}
@@ -28,21 +28,7 @@ pub(super) fn unsupported(connection: &UpstreamConnection) -> OpaqueProviderData
]))
}
pub(super) fn is_unsupported(data: &OpaqueProviderData, connection: &UpstreamConnection) -> bool {
let fields = data.expose_to_provider();
(fields.get("object").and_then(serde_json::Value::as_str) == Some("cpr.peer.unsupported")
&& fields
.get("schema_version")
.and_then(serde_json::Value::as_u64)
== Some(1)
&& fields
.get("connection_revision")
.and_then(serde_json::Value::as_str)
== Some(connection.revision.as_str()))
|| unavailable_reason(data, connection) == Some(UpstreamInfoFailure::Unsupported)
}
// 失权后只保留同一连接已确认互联的事实,不保留已撤销的池、预算与模型数据
// 授权撤销或共享关闭后只保存连接与失败原因,丢弃全部共享数据
pub(super) fn unavailable(
connection: &UpstreamConnection,
reason: UpstreamInfoFailure,
@@ -58,20 +44,24 @@ pub(super) fn unavailable(
]))
}
pub(super) fn unavailable_reason(
pub(super) fn stored_failure(
data: &OpaqueProviderData,
connection: &UpstreamConnection,
) -> Option<UpstreamInfoFailure> {
let fields = data.expose_to_provider();
if fields.get("object")?.as_str()? != "cpr.peer.unavailable"
|| fields.get("schema_version")?.as_u64()? != 1
if fields.get("schema_version")?.as_u64()? != 1
|| fields.get("connection_revision")?.as_str()? != connection.revision
{
return None;
}
match fields.get("reason")?.as_str()? {
"unauthorized" => Some(UpstreamInfoFailure::Unauthorized),
"unsupported" => Some(UpstreamInfoFailure::Unsupported),
match fields.get("object")?.as_str()? {
"cpr.peer.unsupported" => Some(UpstreamInfoFailure::Unsupported),
"cpr.peer.unavailable" => match fields.get("reason")?.as_str()? {
"unauthorized" => Some(UpstreamInfoFailure::Unauthorized),
"sharing_disabled" => Some(UpstreamInfoFailure::SharingDisabled),
"unsupported" => Some(UpstreamInfoFailure::Unsupported),
_ => None,
},
_ => None,
}
}
@@ -361,9 +361,10 @@ impl ProxiesService for DefaultProxiesService {
proxy: Option<&OutboundProxy>,
detect_location: bool,
) -> Result<ProxyTestResult, AdminError> {
let _permit = self.test_slots.try_acquire().map_err(|_| {
AdminError::new(AdminErrorKind::RateLimited, "代理测试繁忙,请稍后重试")
})?;
let _permit = self
.test_slots
.try_acquire()
.map_err(|_| AdminError::new(AdminErrorKind::RateLimited, "代理测试已达并发上限"))?;
Ok(self.probe.test(proxy, detect_location).await)
}
@@ -374,9 +375,10 @@ impl ProxiesService for DefaultProxiesService {
detect_location: bool,
context: &MutationContext,
) -> Result<ProxyRecord, AdminError> {
let _permit = self.test_slots.try_acquire().map_err(|_| {
AdminError::new(AdminErrorKind::RateLimited, "代理测试繁忙,请稍后重试")
})?;
let _permit = self
.test_slots
.try_acquire()
.map_err(|_| AdminError::new(AdminErrorKind::RateLimited, "代理测试已达并发上限"))?;
let record = self
.store
.get(id)
@@ -252,7 +252,7 @@ impl SettingsService for DefaultSettingsService {
.is_some_and(|models| models.contains_key(model))
})
{
return Err(AdminError::invalid("内置价目不能删除,仅支持人工覆盖"));
return Err(AdminError::invalid("内置价目只能覆盖,不能删除"));
}
let defaults = gateway_core::metering::merge_pricing(catalog.defaults, &catalog.synced);
let defaults = defaults.get(&command.provider);
@@ -332,24 +332,35 @@ impl SettingsService for DefaultSettingsService {
configuration: Option<&gateway_core::account::OpaqueProviderData>,
) -> Result<gateway_core::account::OpaqueProviderData, AdminError> {
let profile_provider = self.profile_provider(provider)?;
let global;
let (configuration, source) = if let Some(configuration) = configuration {
(configuration, "override")
let independent = configuration.filter(|key| {
gateway_core::settings::request_profiles::inherited_fields(key).is_none()
});
let preview = if let Some(key) = independent {
profile_provider.preview_key_client_profile(key, None)
} else {
global = self
let global = self
.load()
.await?
.client_profile(provider)
.cloned()
.or_else(|| profile_provider.default_client_profile())
.ok_or_else(|| AdminError::internal("通用客户端身份尚未初始化"))?;
(&global, "global")
};
let mut preview = profile_provider
.preview_client_profile(configuration)
.map_err(|error| super::map_provider_error(error, "client profile"))?
.into_inner();
preview.insert("source".to_owned(), serde_json::Value::from(source));
if let Some(key) = configuration {
profile_provider.preview_key_client_profile(key, Some(&global))
} else {
profile_provider.preview_client_profile(&global)
}
}
.map_err(|error| super::map_provider_error(error, "client profile"))?;
let mut preview = preview.into_inner();
preview.insert(
"source".to_owned(),
serde_json::Value::from(if independent.is_some() {
"override"
} else {
"global"
}),
);
Ok(gateway_core::account::OpaqueProviderData::new(preview))
}
@@ -419,7 +430,7 @@ impl SettingsService for DefaultSettingsService {
crate::ports::store::AdminStoreErrorKind::Conflict
| crate::ports::store::AdminStoreErrorKind::StaleRevision
) {
AdminError::conflict("运行设置已被其他调用修改,请重新读取并确认后提交")
AdminError::conflict("运行设置已变更,请刷新核对后提交")
} else {
map_store_error(error, "runtime settings")
}
@@ -176,10 +176,11 @@ async fn uncertain_discovery_failures_retry_without_becoming_unsupported() {
}
#[tokio::test(start_paused = true)]
async fn known_peer_failures_keep_identity_and_only_revocation_discards_shared_data() {
async fn known_peer_identity_changes_only_on_definitive_detection() {
for failure in [
UpstreamInfoFailure::Unauthorized,
UpstreamInfoFailure::Unsupported,
UpstreamInfoFailure::SharingDisabled,
UpstreamInfoFailure::Unavailable,
UpstreamInfoFailure::RateLimited,
UpstreamInfoFailure::InvalidResponse,
@@ -191,14 +192,15 @@ async fn known_peer_failures_keep_identity_and_only_revocation_discards_shared_d
tokio::time::advance(Duration::from_secs(15)).await;
*transport.failure.lock().unwrap() = Some(failure.into());
let result = bundle.service().refresh_quota(&id).await;
assert_eq!(result.is_ok(), failure == UpstreamInfoFailure::Unsupported);
let clears_data = matches!(
let clears_identity = matches!(
failure,
UpstreamInfoFailure::Unauthorized | UpstreamInfoFailure::Unsupported
UpstreamInfoFailure::Unsupported | UpstreamInfoFailure::SharingDisabled
);
assert_eq!(result.is_ok(), clears_identity);
let clears_data = clears_identity || failure == UpstreamInfoFailure::Unauthorized;
let quota = bundle.service().read_quota(&id).await.unwrap().unwrap();
let info = quota.upstream.as_ref().unwrap();
assert!(info.discovered);
assert_eq!(info.discovered, !clears_identity);
assert_eq!(info.last_error.as_deref(), Some(failure.as_str()));
assert_eq!(quota.windows.is_empty(), clears_data);
assert_eq!(info.observed_at.is_none(), clears_data);
@@ -221,10 +223,24 @@ async fn known_peer_failures_keep_identity_and_only_revocation_discards_shared_d
clears_data
);
if failure == UpstreamInfoFailure::Unsupported {
// 已运行 v3.40.0 的节点可能持有这种不支持记录,读取时应直接显示为普通 API Key
accounts.records.lock().unwrap().get_mut(&id).unwrap().quota =
serde_json::from_value(serde_json::json!({
"object": "cpr.peer.unavailable",
"schema_version": 1,
"connection_revision": info.connection_revision,
"reason": "unsupported",
}))
.unwrap();
}
drop(bundle);
let restarted = initialize(accounts, transport);
let restored = restarted.service().read_quota(&id).await.unwrap().unwrap();
assert!(restored.upstream.as_ref().unwrap().discovered);
assert_eq!(
restored.upstream.as_ref().unwrap().discovered,
!clears_identity
);
assert_eq!(restored.windows.is_empty(), clears_data);
if clears_data {
assert_eq!(
@@ -246,7 +262,7 @@ async fn known_peer_failures_keep_identity_and_only_revocation_discards_shared_d
}
#[tokio::test(start_paused = true)]
async fn known_peer_identity_is_scoped_to_the_connection_after_revocation() {
async fn peer_detection_after_revocation_is_scoped_to_the_connection() {
for failure in [
UpstreamInfoFailure::Unauthorized,
UpstreamInfoFailure::Unsupported,
@@ -261,7 +277,7 @@ async fn known_peer_identity_is_scoped_to_the_connection_after_revocation() {
let edited = initial.clone().with_notes(Some("保留连接身份".into()));
*accounts.accounts.lock().unwrap() = vec![edited.clone()];
bundle.service().account_changed(&id).await.unwrap();
assert!(
assert_eq!(
bundle
.service()
.read_summary(&id)
@@ -270,7 +286,8 @@ async fn known_peer_identity_is_scoped_to_the_connection_after_revocation() {
.unwrap()
.upstream
.unwrap()
.discovered
.discovered,
failure == UpstreamInfoFailure::Unauthorized
);
// 地址和密钥修改推进凭据版本,出口变化则改变同一版本的连接身份
+1 -1
View File
@@ -12,7 +12,7 @@ name = "main"
path = "tests/main.rs"
[dependencies]
axum = { version = "0.8.9", default-features = false, features = ["json", "multipart", "query", "ws"] }
axum = { version = "0.8.9", default-features = false, features = ["json", "matched-path", "multipart", "query", "ws"] }
base64 = "0.23.0"
bytes = "1.11.1"
chrono = { version = "0.4.45", features = ["serde"] }
@@ -455,20 +455,14 @@ pub(super) fn account_model_usage_view(
.costs
.iter()
.find(|cost| cost.currency.eq_ignore_ascii_case("USD"));
let success_rate = usage.health.success_rate();
ModelUsageView {
model: usage.model,
request_count: usage.request_count,
request_count_display: format_number(usage.request_count),
success_rate: (usage.request_count > 0)
.then(|| usage.success_count as f64 * 100.0 / usage.request_count as f64),
success_rate_display: if usage.request_count == 0 {
"—".to_owned()
} else {
format!(
"{:.1}%",
usage.success_count as f64 * 100.0 / usage.request_count as f64
)
},
success_rate,
success_rate_display: success_rate
.map_or_else(|| "—".to_owned(), |rate| format!("{rate:.1}%")),
input_tokens: usage.input_tokens,
input_tokens_display: display_optional_tokens(usage.input_tokens),
output_tokens: usage.output_tokens,
@@ -495,9 +489,11 @@ pub(super) fn account_model_usage_view(
partial_cost_count: u64::from(cost_estimate_status == "partial"),
unknown_cost_count: usage.cost_coverage.unavailable_count,
costs: usage.costs.iter().map(account_currency_cost_view).collect(),
last_used_at: time.rfc3339(&usage.last_used_at),
last_used_at_full_display: Some(time.datetime(&usage.last_used_at)),
last_used_at_display: time.relative(usage.last_used_at, now),
last_used_at: usage.last_used_at.map(|value| time.rfc3339(&value)),
last_used_at_full_display: usage.last_used_at.map(|value| time.datetime(&value)),
last_used_at_display: usage
.last_used_at
.map_or_else(|| "—".to_owned(), |value| time.relative(value, now)),
}
}
@@ -515,7 +515,7 @@ pub struct ModelUsageView {
pub partial_cost_count: u64,
pub unknown_cost_count: u64,
pub costs: Vec<CurrencyCostView>,
pub last_used_at: String,
pub last_used_at: Option<String>,
pub last_used_at_display: String,
pub last_used_at_full_display: Option<String>,
}
@@ -537,14 +537,10 @@ async fn probe<S>(
where
S: SessionState + Send + Sync,
{
let proxy = request
.proxy_url
.0
.ok_or_else(|| AdminError::bad_request("代理 URL 不能为空"))?;
let result = state
.admin_services()
.proxies()
.probe(Some(&proxy), request.detect_location)
.probe(request.proxy_url.0.as_ref(), request.detect_location)
.await
.map_err(map_error)?;
Ok(AdminResponse::new(
@@ -9,6 +9,14 @@ use gateway_core::error::{GatewayError, GatewayErrorKind};
use serde::Serialize;
use serde_json::{Value, json};
/// 认证边界提供的稳定拒绝原因,不携带认证输入
#[derive(Clone)]
pub(crate) struct AccessRejection(pub &'static str);
/// 执行已交给 Core 观测,HTTP 入口只补充交付边界日志
#[derive(Clone)]
pub(crate) struct ExecutionObserved;
/// OpenAI 风格的协议错误 JSON
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
pub struct ProtocolErrorBody {
@@ -107,6 +115,7 @@ pub fn gateway_error_response(error: &GatewayError) -> Response {
error.client_error_code().unwrap_or(default_code),
)
.into_response();
response.extensions_mut().insert(error.clone());
if let Some(delay) = error.retry_after()
&& let Ok(value) = HeaderValue::from_str(
&delay
@@ -0,0 +1,218 @@
//! 数据面 HTTP 入口诊断;执行前失败进入运维事件,不伪造模型请求或上游状态
use std::{
pin::Pin,
sync::Arc,
task::{Context, Poll},
};
use axum::{
body::{Body, Bytes},
extract::{MatchedPath, Request, State},
http::StatusCode,
middleware::Next,
response::Response,
};
use gateway_core::diagnostics::{OperationalDiagnostics, OperationalFailure, TraceContext};
use http_body::{Body as _, Frame, SizeHint};
use serde_json::json;
use tower_http::request_id::RequestId;
use crate::client_error::{AccessRejection, ProtocolError, ProtocolErrorBody};
pub(crate) async fn record_ingress(
State(diagnostics): State<Arc<dyn OperationalDiagnostics>>,
mut request: Request,
next: Next,
) -> Response {
if request.uri().path() != "/v1" && !request.uri().path().starts_with("/v1/") {
return next.run(request).await;
}
let correlation_id = request
.extensions()
.get::<RequestId>()
.and_then(|id| id.header_value().to_str().ok())
.map(str::to_owned);
let endpoint = request
.extensions()
.get::<MatchedPath>()
.map(|path| path.as_str().to_owned());
let method = request.method().clone();
let dump = if tracing::enabled!(target: "request_dump", tracing::Level::INFO) {
let trace = TraceContext::new(correlation_id.as_deref().unwrap_or("ingress"));
trace.dump(
"ingress.request",
&serde_json::to_vec(&json!({
"method":method.as_str(), "uri":request.uri().to_string(),
}))
.unwrap_or_default(),
);
trace.headers(
"ingress.request.headers",
json!({}),
request
.headers()
.iter()
.map(|(name, value)| (name.as_str(), value.as_bytes())),
);
let body = std::mem::replace(request.body_mut(), Body::empty());
*request.body_mut() = Body::new(DumpBody::new(body, trace.clone(), "ingress.request.body"));
Some(trace)
} else {
None
};
let mut response = if endpoint.is_none() {
// 数据面未知路径不能落入 SPA fallback,否则会把协议错误伪装成 HTML 成功
crate::openai::error::protocol_error_response(
StatusCode::NOT_FOUND,
ProtocolErrorBody {
error: ProtocolError {
kind: "invalid_request_error",
code: "unknown_endpoint",
message: "The requested API endpoint does not exist.".into(),
param: None,
},
},
)
} else {
next.run(request).await
};
let protocol = response.extensions_mut().remove::<ProtocolErrorBody>();
let gateway = response
.extensions_mut()
.remove::<gateway_core::error::GatewayError>();
let access = response.extensions_mut().remove::<AccessRejection>();
let execution_observed = response
.extensions_mut()
.remove::<crate::client_error::ExecutionObserved>()
.is_some();
let status = response.status();
if status.is_client_error() || status.is_server_error() {
let code = access
.map(|error| error.0)
.or_else(|| protocol.as_ref().map(|body| body.error.code))
.or_else(|| gateway.as_ref().map(|error| error.kind().as_str()))
.unwrap_or(match status {
StatusCode::UNAUTHORIZED => "invalid_api_key",
StatusCode::METHOD_NOT_ALLOWED => "method_not_allowed",
StatusCode::NOT_FOUND => "not_found",
_ => "http_rejection",
});
// 未知参数名可由客户端任意构造,只保留 decoder 生成的已知参数路径
let parameter = protocol
.as_ref()
.filter(|body| body.error.code != "unknown_parameter")
.and_then(|body| body.error.param.as_deref())
.filter(|path| {
path.len() <= 256
&& path.bytes().all(|byte| {
byte.is_ascii_alphanumeric() || matches!(byte, b'_' | b'.' | b'[' | b']')
})
});
let endpoint = endpoint.as_deref().unwrap_or("/v1/<unmatched>");
let model_request_id = execution_observed
.then(|| {
response
.headers()
.get("x-gateway-request-id")
.and_then(|id| id.to_str().ok())
})
.flatten();
tracing::info!(target: "request_trace", stage="ingress.rejected", status=status.as_u16(),
code, parameter, endpoint, model_request_id, method=method.as_str(), "Data-plane HTTP request rejected");
if !execution_observed {
let mut failure = OperationalFailure::new(
"api",
"http_ingress",
code,
json!({"method":method.as_str(), "endpoint":endpoint, "status":status.as_u16(),
"code":code, "param":parameter})
.to_string(),
);
failure.correlation_id = correlation_id;
failure.details = gateway.as_ref().and_then(|error| error.error_details());
if diagnostics.record_failure(failure).await.is_err() {
tracing::warn!("HTTP ingress diagnostic could not be recorded");
}
}
}
if let Some(trace) = dump {
trace.dump(
"ingress.response",
&serde_json::to_vec(&json!({"status":status.as_u16()})).unwrap_or_default(),
);
trace.headers(
"ingress.response.headers",
json!({"status":status.as_u16()}),
response
.headers()
.iter()
.map(|(name, value)| (name.as_str(), value.as_bytes())),
);
let body = std::mem::replace(response.body_mut(), Body::empty());
*response.body_mut() = Body::new(DumpBody::new(body, trace, "ingress.response.body"));
}
response
}
struct DumpBody {
inner: Body,
trace: TraceContext,
stage: &'static str,
complete: bool,
}
impl DumpBody {
fn new(inner: Body, trace: TraceContext, stage: &'static str) -> Self {
let complete = inner.is_end_stream();
Self {
inner,
trace,
stage,
complete,
}
}
}
impl http_body::Body for DumpBody {
type Data = Bytes;
type Error = axum::Error;
fn poll_frame(
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
) -> Poll<Option<Result<Frame<Bytes>, axum::Error>>> {
let result = Pin::new(&mut self.inner).poll_frame(cx);
if let Poll::Ready(Some(Ok(frame))) = &result
&& let Some(bytes) = frame.data_ref()
{
self.trace.dump(self.stage, bytes);
}
if matches!(result, Poll::Ready(Some(Err(_)))) {
self.complete = false;
} else if matches!(result, Poll::Ready(None)) || self.inner.is_end_stream() {
self.complete = true;
}
result
}
fn is_end_stream(&self) -> bool {
self.inner.is_end_stream()
}
fn size_hint(&self) -> SizeHint {
self.inner.size_hint()
}
}
impl Drop for DumpBody {
fn drop(&mut self) {
// 不为诊断主动读取剩余正文;取消或未消费的请求必须明确标出转储不完整
self.trace.dump(
"ingress.body.finished",
&serde_json::to_vec(&json!({
"stage":self.stage, "complete":self.complete,
}))
.unwrap_or_default(),
);
}
}
+5
View File
@@ -30,6 +30,7 @@ use crate::openai::service::OpenAiService;
pub mod admin;
pub mod auth;
mod client_error;
mod diagnostics;
mod health;
mod key_usage;
mod middleware;
@@ -280,6 +281,10 @@ pub fn initialize(
headers_ms = latency.as_millis(), "HTTP response headers ready");
});
let router = router
.layer(axum::middleware::from_fn_with_state(
diagnostics.clone(),
diagnostics::record_ingress,
))
.layer(axum::middleware::from_fn_with_state(
diagnostics,
admin::diagnostics::record_failure,
+11 -2
View File
@@ -128,7 +128,12 @@ fn client_authentication_request(
}
pub(crate) fn client_access_error_response(error: ClientAccessError) -> Response {
match error {
let code = match &error {
ClientAccessError::Authentication(error) => error.reason(),
ClientAccessError::Version(error) if error.current().is_some() => "client_version_too_old",
ClientAccessError::Version(_) => "client_version_unavailable",
};
let mut response = match error {
ClientAccessError::Authentication(error) => {
log_client_api_key_auth_failure(error);
if error == ClientApiKeyAuthError::RuntimeUnavailable {
@@ -146,7 +151,11 @@ pub(crate) fn client_access_error_response(error: ClientAccessError) -> Response
);
client_version_rejection_response(&error).into_response()
}
}
};
response
.extensions_mut()
.insert(crate::client_error::AccessRejection(code));
response
}
/// Desktop 优先于内嵌 CLI/Core;未知客户端及未提供应用版本的手机远程客户端返回 `None`
@@ -1,5 +1,7 @@
//! 将 Chat 请求转换为 Responses 正文,协议选项留在 API,业务 JSON 与上游兼容规则保持原有归属
use std::collections::HashSet;
use serde_json::{Map, Value, json};
use crate::openai::responses::RequestDecodeError;
@@ -50,10 +52,10 @@ pub(in crate::openai) fn convert_request(
body.insert("tool_choice".into(), json!({"type": "web_search"}));
} else {
if let Some(value) = optional(&mut chat, "tools") {
body.insert("tools".into(), function_tools(value)?.into());
body.insert("tools".into(), tools(value)?.into());
}
if let Some(value) = optional(&mut chat, "tool_choice") {
let choice = tool_choice(value, body.get("tools"))?;
let choice = tool_choice(value, "tool_choice")?;
body.insert("tool_choice".into(), choice);
}
}
@@ -61,7 +63,10 @@ pub(in crate::openai) fn convert_request(
body.insert("parallel_tool_calls".into(), value.into());
}
let mut text = Map::new();
let mut text = optional(&mut chat, "text")
.map(|value| object(value, "text"))
.transpose()?
.unwrap_or_default();
if let Some(value) = optional(&mut chat, "response_format") {
text.insert("format".into(), response_format(value)?);
}
@@ -72,47 +77,81 @@ pub(in crate::openai) fn convert_request(
if !text.is_empty() {
body.insert("text".into(), text.into());
}
let mut reasoning = optional(&mut chat, "reasoning")
.map(|value| object(value, "reasoning"))
.transpose()?
.unwrap_or_default();
if let Some(value) = optional(&mut chat, "reasoning_effort") {
string(&value, "reasoning_effort", false)?;
body.insert("reasoning".into(), json!({"effort": value}));
reasoning.insert("effort".into(), value);
}
for field in ["max_tokens", "max_completion_tokens"] {
if !reasoning.is_empty() {
body.insert("reasoning".into(), reasoning.into());
}
for field in ["max_output_tokens", "max_tokens", "max_completion_tokens"] {
if let Some(value) = optional(&mut chat, field) {
if !value.as_u64().is_some_and(|tokens| tokens > 0) {
return Err(invalid(field));
}
// 新字段优先,具体上游是否接受长度控制仍由现有 Provider 决定
body.insert("max_output_tokens".into(), value);
}
}
for (field, maximum) in [("temperature", 2.0), ("top_p", 1.0)] {
if let Some(value) = optional(&mut chat, field) {
if !value
.as_f64()
.is_some_and(|number| (0.0..=maximum).contains(&number))
{
return Err(invalid(field));
}
body.insert(field.into(), value);
}
}
// 同语义字段不在入口预判模型支持范围,交由既有 Provider 与上游校验
for field in [
"temperature",
"top_p",
"user",
"safety_identifier",
"service_tier",
"prompt_cache_key",
"prompt_cache_retention",
"metadata",
"prompt_cache_options",
"moderation",
"instructions",
"include",
"max_tool_calls",
"truncation",
"context_management",
] {
if let Some(value) = optional(&mut chat, field) {
string(&value, field, true)?;
body.insert(field.into(), value);
}
}
for field in ["metadata", "prompt_cache_options", "moderation"] {
if let Some(value) = optional(&mut chat, field) {
let object = object(value, field)?;
body.insert(field.into(), object.into());
if optional_bool(&mut chat, "logprobs", "logprobs")? == Some(true) {
let mut include = body
.remove("include")
.map(|value| array(value, "include"))
.transpose()?
.unwrap_or_default();
let logprobs = json!("message.output_text.logprobs");
if !include.contains(&logprobs) {
include.push(logprobs);
}
body.insert("include".into(), include.into());
if let Some(value) = optional(&mut chat, "top_logprobs") {
body.insert("top_logprobs".into(), value);
}
}
let ignored = [
"stop",
"seed",
"frequency_penalty",
"presence_penalty",
"logit_bias",
"prediction",
]
.into_iter()
.filter(|field| {
chat.get(*field).is_some_and(|value| {
!value.is_null()
&& value.as_f64() != Some(0.0)
&& !value.as_array().is_some_and(Vec::is_empty)
&& !value.as_object().is_some_and(Map::is_empty)
})
})
.collect::<Vec<_>>();
if !ignored.is_empty() {
tracing::info!(target: "request_trace", stage="chat.compatibility", ignored_options=?ignored,
"Chat-only optional controls omitted from Responses request");
}
// 未识别的客户端顶层扩展不等于 Responses 参数,不进入上游正文
Ok(ConvertedChatRequest {
@@ -125,26 +164,10 @@ fn validate_options(chat: &Map<String, Value>) -> Result<(), RequestDecodeError>
for (field, value) in chat.iter().filter(|(_, value)| !value.is_null()) {
let supported = match field.as_str() {
"n" => value.as_u64() == Some(1),
"store" | "logprobs" => value == false,
"frequency_penalty" | "presence_penalty" => value.as_f64() == Some(0.0),
"top_logprobs" => value.as_u64() == Some(0),
"logit_bias" => value.as_object().is_some_and(Map::is_empty),
"functions" | "stop" => value.as_array().is_some_and(Vec::is_empty),
"store" | "background" => value == false,
"functions" => value.as_array().is_some_and(Vec::is_empty),
"modalities" => value == &json!(["text"]),
"audio"
| "prediction"
| "seed"
| "function_call"
| "reasoning_content"
| "input"
| "previous_response_id"
| "conversation"
| "instructions"
| "reasoning"
| "text"
| "include"
| "background"
| "max_output_tokens" => false,
"audio" | "function_call" | "input" | "previous_response_id" | "conversation" => false,
_ => true,
};
if !supported {
@@ -215,6 +238,7 @@ fn messages(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
});
}
let mut input = Vec::with_capacity(messages.len());
let mut custom_calls = HashSet::new();
for (index, value) in messages.into_iter().enumerate() {
let path = format!("messages[{index}]");
let mut message = object(value, &path)?;
@@ -229,11 +253,9 @@ fn messages(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
if let Some(name) = &name {
let name_path = format!("{path}.name");
string(name, &name_path, true)?;
if role != "tool" {
return Err(unsupported(&name_path));
}
}
for field in ["function_call", "reasoning_content", "reasoning", "audio"] {
// Chat 的显示名称和明文推理历史没有对应的 Responses 输入语义
for field in ["function_call", "audio"] {
if optional(&mut message, field).is_some() {
return Err(unsupported(&format!("{path}.{field}")));
}
@@ -261,8 +283,12 @@ fn messages(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
} else {
content_parts(content, &role, &content_path, false)?.into()
};
let mut item =
json!({"type": "function_call_output", "call_id": call_id, "output": output});
let kind = if custom_calls.contains(&call_id) {
"custom_tool_call_output"
} else {
"function_call_output"
};
let mut item = json!({"type": kind, "call_id": call_id, "output": output});
if let Some(name) = name {
item["name"] = name;
}
@@ -300,7 +326,15 @@ fn messages(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
}
input.push(item);
}
input.extend(calls);
for call in calls {
let id = string(&call["call_id"], &format!("{path}.tool_calls.id"), false)?.to_owned();
if call["type"] == "custom_tool_call" {
custom_calls.insert(id);
} else {
custom_calls.remove(&id);
}
input.push(call);
}
}
Ok(input)
}
@@ -322,17 +356,22 @@ fn content_parts(
let mut part = object(value, &path)?;
let kind = required_string(&mut part, "type", &format!("{path}.type"), false)?;
let mut converted = match kind.as_str() {
"text" => text_part(
"text" | "input_text" | "output_text" => text_part(
required_string(&mut part, "text", &format!("{path}.text"), true)?,
is_assistant,
),
"image_url" if role == "user" => image_part(
"image_url" if matches!(role, "user" | "tool") => image_part(
required(&mut part, "image_url", &format!("{path}.image_url"))?,
&format!("{path}.image_url"),
)?,
"file" if role == "user" => {
"file" if matches!(role, "user" | "tool") => {
let file_path = format!("{path}.file");
inline_pdf_part(required(&mut part, "file", &file_path)?, &file_path)?
file_part(required(&mut part, "file", &file_path)?, &file_path)?
}
"input_audio" if matches!(role, "user" | "tool") => audio_part(&mut part, &path)?,
"input_image" | "input_file" if matches!(role, "user" | "tool") => {
part.insert("type".into(), kind.into());
Value::Object(std::mem::take(&mut part))
}
"refusal" if role == "assistant" => json!({
"type": "refusal",
@@ -360,23 +399,32 @@ fn text_part(text: String, is_assistant: bool) -> Value {
}
}
fn inline_pdf_part(value: Value, path: &str) -> Result<Value, RequestDecodeError> {
fn file_part(value: Value, path: &str) -> Result<Value, RequestDecodeError> {
let mut file = object(value, path)?;
if optional(&mut file, "file_id").is_some() {
return Err(unsupported(&format!("{path}.file_id")));
}
let data_path = format!("{path}.file_data");
let data = required_string(&mut file, "file_data", &data_path, false)?;
if !data
.strip_prefix("data:application/pdf;base64,")
.is_some_and(|payload| !payload.trim().is_empty())
if !["file_id", "file_url", "file_data"]
.iter()
.any(|field| file.get(*field).is_some_and(|value| !value.is_null()))
{
return Err(invalid(&data_path));
return Err(invalid(path));
}
// 文件可用性、格式和组合由接收文件的上游判断,网关不下载或重编码
file.insert("type".into(), "input_file".into());
Ok(file.into())
}
fn audio_part(part: &mut Map<String, Value>, path: &str) -> Result<Value, RequestDecodeError> {
if let Some(value) = optional(part, "input_audio") {
let path = format!("{path}.input_audio");
let mut audio = object(value, &path)?;
let data = required_string(&mut audio, "data", &format!("{path}.data"), false)?;
let format = required_string(&mut audio, "format", &format!("{path}.format"), false)?;
let mime = if format == "mp3" { "mpeg" } else { &format };
Ok(json!({"type":"input_audio", "audio_url":format!("data:audio/{mime};base64,{data}")}))
} else {
// Codex 使用 audio_url/file_id 引用,原生扩展保持其结构
part.insert("type".into(), "input_audio".into());
Ok(Value::Object(std::mem::take(part)))
}
let name = required_string(&mut file, "filename", &format!("{path}.filename"), false)?;
let mut converted = json!({"type": "input_file", "filename": name});
converted["file_data"] = data.into();
Ok(converted)
}
fn image_part(value: Value, path: &str) -> Result<Value, RequestDecodeError> {
@@ -402,104 +450,82 @@ fn image_part(value: Value, path: &str) -> Result<Value, RequestDecodeError> {
}
fn tool_calls(value: Value, path: &str) -> Result<Vec<Value>, RequestDecodeError> {
let calls = array(value, path)?;
let mut result = Vec::with_capacity(calls.len());
for (index, value) in calls.into_iter().enumerate() {
let path = format!("{path}[{index}]");
let mut call = object(value, &path)?;
let id = required_string(&mut call, "id", &format!("{path}.id"), false)?;
if required_string(&mut call, "type", &format!("{path}.type"), false)? != "function" {
return Err(unsupported(&format!("{path}.type")));
}
let function_path = format!("{path}.function");
let mut function = object(
required(&mut call, "function", &function_path)?,
&function_path,
)?;
let name = required_string(
&mut function,
"name",
&format!("{function_path}.name"),
false,
)?;
let arguments = required_string(
&mut function,
"arguments",
&format!("{function_path}.arguments"),
true,
)?;
result.push(
json!({"type": "function_call", "call_id": id, "name": name, "arguments": arguments}),
);
}
Ok(result)
array(value, path)?
.into_iter()
.enumerate()
.map(|(index, value)| {
let path = format!("{path}[{index}]");
let mut call = object(value, &path)?;
let id = required_string(&mut call, "id", &format!("{path}.id"), false)?;
let kind = required_string(&mut call, "type", &format!("{path}.type"), false)?;
let (output_type, data_field) = match kind.as_str() {
"function" => ("function_call", "arguments"),
"custom" => ("custom_tool_call", "input"),
_ => return Err(unsupported(&format!("{path}.type"))),
};
let nested_path = format!("{path}.{kind}");
let mut nested = object(required(&mut call, &kind, &nested_path)?, &nested_path)?;
let name = required_string(&mut nested, "name", &format!("{nested_path}.name"), false)?;
let data = required_string(
&mut nested,
data_field,
&format!("{nested_path}.{data_field}"),
true,
)?;
Ok(json!({"type":output_type, "call_id":id, "name":name, data_field:data}))
})
.collect()
}
fn function_tools(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
fn tools(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
array(value, "tools")?
.into_iter()
.enumerate()
.map(|(index, value)| {
let path = format!("tools[{index}]");
let mut tool = object(value, &path)?;
if required_string(&mut tool, "type", &format!("{path}.type"), false)? != "function" {
return Err(unsupported(&format!("{path}.type")));
let kind = required_string(&mut tool, "type", &format!("{path}.type"), false)?;
if matches!(kind.as_str(), "function" | "custom") {
if let Some(nested) = optional(&mut tool, &kind) {
tool = object(nested, &format!("{path}.{kind}"))?;
}
if kind == "function" && tool.get("strict").is_none_or(Value::is_null) {
// Chat 的缺省是宽松 schema,不能让 Responses 收紧参数合同
tool.insert("strict".into(), false.into());
}
}
let function_path = format!("{path}.function");
let mut function = object(
required(&mut tool, "function", &function_path)?,
&function_path,
)?;
let name = required_string(
&mut function,
"name",
&format!("{function_path}.name"),
false,
)?;
// Chat 的缺省是宽松函数 schema,不能让 Responses 的缺省收紧原始参数合同
let strict =
optional_bool(&mut function, "strict", &format!("{function_path}.strict"))?
.unwrap_or(false);
let mut converted = json!({"type": "function", "name": name, "strict": strict});
if let Some(description) = optional(&mut function, "description") {
string(&description, &format!("{function_path}.description"), true)?;
converted["description"] = description;
}
if let Some(parameters) = optional(&mut function, "parameters") {
converted["parameters"] = parameters;
}
Ok(converted)
// hosted 工具采用原生 Responses 结构,支持范围由选中的 Provider 决定
tool.insert("type".into(), kind.into());
Ok(tool.into())
})
.collect()
}
fn tool_choice(value: Value, tools: Option<&Value>) -> Result<Value, RequestDecodeError> {
let tools = tools.and_then(Value::as_array);
if let Value::String(choice) = &value {
return match choice.as_str() {
"auto" | "none" => Ok(value),
"required" if tools.is_some_and(|tools| !tools.is_empty()) => Ok(value),
"required" => Err(invalid("tool_choice")),
_ => Err(unsupported("tool_choice")),
};
fn tool_choice(value: Value, path: &str) -> Result<Value, RequestDecodeError> {
if value.is_string() {
return Ok(value);
}
let mut choice = object(value, "tool_choice")?;
if required_string(&mut choice, "type", "tool_choice.type", false)? != "function" {
return Err(unsupported("tool_choice.type"));
let mut choice = object(value, path)?;
let kind = required_string(&mut choice, "type", &format!("{path}.type"), false)?;
if matches!(kind.as_str(), "function" | "custom") {
if let Some(nested) = optional(&mut choice, &kind) {
choice = object(nested, &format!("{path}.{kind}"))?;
}
} else if kind == "allowed_tools" {
if let Some(nested) = optional(&mut choice, "allowed_tools") {
choice = object(nested, &format!("{path}.allowed_tools"))?;
}
if let Some(value) = optional(&mut choice, "tools") {
let mapped = array(value, &format!("{path}.tools"))?
.into_iter()
.enumerate()
.map(|(index, value)| tool_choice(value, &format!("{path}.tools[{index}]")))
.collect::<Result<Vec<_>, _>>()?;
choice.insert("tools".into(), mapped.into());
}
}
let mut function = object(
required(&mut choice, "function", "tool_choice.function")?,
"tool_choice.function",
)?;
let name = required_string(&mut function, "name", "tool_choice.function.name", false)?;
if !tools.is_some_and(|tools| {
tools
.iter()
.any(|tool| tool.get("name").and_then(Value::as_str) == Some(name.as_str()))
}) {
return Err(invalid("tool_choice.function.name"));
}
Ok(json!({"type": "function", "name": name}))
choice.insert("type".into(), kind.into());
Ok(choice.into())
}
fn response_format(value: Value) -> Result<Value, RequestDecodeError> {
@@ -538,7 +564,10 @@ fn response_format(value: Value) -> Result<Value, RequestDecodeError> {
}
Ok(converted)
}
_ => Err(unsupported("response_format.type")),
_ => {
format.insert("type".into(), kind.into());
Ok(format.into())
}
}
}
@@ -7,6 +7,7 @@ use gateway_protocol::openai::events::{billable_usage_is_complete, extract_usage
use gateway_protocol::openai::sse::encode_sse_event;
use serde::Serialize;
use serde_json::{Value, json};
use sha2::{Digest as _, Sha256};
use crate::client_error::{ProtocolError, ProtocolErrorBody};
@@ -48,16 +49,23 @@ pub(in crate::openai) fn project_response(
let mut tools = Vec::new();
let mut annotations = Vec::new();
let mut content_chars = 0;
let mut reasoning = Vec::new();
let mut images = Vec::new();
let mut logprobs = None;
for item in output {
match required_str(item, "type")? {
"message" => {
// 两侧索引都按消息中的字符计数,合并多条 assistant 消息时平移引用
let message_offset = content_chars;
for part in message_parts(item)? {
let (kind, text) = part_text(part)?;
let target = match kind {
PartKind::Text => {
append_url_citations(part, message_offset, &mut annotations)?;
append_url_citations(part, content_chars, &mut annotations)?;
if let Some(values) = part.get("logprobs").and_then(Value::as_array) {
logprobs
.get_or_insert_with(Vec::new)
.extend(values.iter().cloned());
}
content_chars += text.chars().count() as u64;
&mut content
}
@@ -66,14 +74,25 @@ pub(in crate::openai) fn project_response(
target.get_or_insert_with(String::new).push_str(text);
}
}
"function_call" => {
let (id, name, arguments) = function_fields(item)?;
tools.push(json!({
"id": id, "type": "function",
"function": {"name": name, "arguments": arguments}
}));
"function_call" | "custom_tool_call" => {
let kind = ToolKind::from_item(item)?;
let (id, name, data) = tool_fields(item, kind)?;
tools.push(json!({"id":id, "type":kind.chat_type(),
kind.chat_type():{"name":name, kind.data_field():data}}));
}
"reasoning" | "web_search_call" => {}
"reasoning" => {
for (_, _, text) in reasoning_parts(item)? {
if !text.is_empty() {
reasoning.push(text);
}
}
}
"image_generation_call" => {
if let Some(image) = image_payload(item)? {
images.push(image);
}
}
kind if is_auxiliary_output(kind) => {}
_ => return Err(invalid_response()),
}
}
@@ -85,10 +104,17 @@ pub(in crate::openai) fn project_response(
if !annotations.is_empty() {
message["annotations"] = Value::Array(annotations);
}
if !reasoning.is_empty() {
message["reasoning_content"] = reasoning.join("\n\n").into();
}
if !images.is_empty() {
message["images"] = images.into();
}
let logprobs = logprobs.map(|content| json!({"content":content, "refusal":null}));
serde_json::to_value(Envelope {
identity,
object: "chat.completion",
choices: json!([{"index": 0, "message": message, "finish_reason": finish_reason, "logprobs": null}]),
choices: json!([{"index": 0, "message": message, "finish_reason": finish_reason, "logprobs": logprobs}]),
usage: chat_usage(response),
metadata: response.get("metadata"),
service_tier: response.get("service_tier"),
@@ -115,9 +141,47 @@ impl PartKind {
struct Part {
kind: PartKind,
text: String,
logprobs: Vec<Value>,
annotations: BTreeMap<u64, Value>,
spans: Vec<TextSpan>,
}
struct TextSpan {
start: u64,
end: u64,
delivered_start: u64,
}
#[derive(Clone, Copy, PartialEq, Eq)]
enum ToolKind {
Function,
Custom,
}
impl ToolKind {
fn from_item(item: &Value) -> Result<Self, ProtocolErrorBody> {
match required_str(item, "type")? {
"function_call" => Ok(Self::Function),
"custom_tool_call" => Ok(Self::Custom),
_ => Err(invalid_response()),
}
}
const fn chat_type(self) -> &'static str {
match self {
Self::Function => "function",
Self::Custom => "custom",
}
}
const fn data_field(self) -> &'static str {
match self {
Self::Function => "arguments",
Self::Custom => "input",
}
}
}
struct Tool {
kind: ToolKind,
index: usize,
call_id: String,
name: String,
@@ -126,9 +190,13 @@ struct Tool {
enum Output {
Message(BTreeMap<u64, Part>),
Function(Tool),
Reasoning,
WebSearch,
Tool(Tool),
Reasoning(BTreeMap<(bool, u64), String>),
Image {
index: usize,
fingerprint: Option<[u8; 32]>,
},
Auxiliary(String),
}
struct OutputItem {
@@ -149,6 +217,7 @@ pub(in crate::openai) struct ChatStreamEncoder {
role_sent: bool,
output: BTreeMap<u64, OutputItem>,
terminal: Option<Terminal>,
content_chars: u64,
}
impl ChatStreamEncoder {
@@ -159,6 +228,7 @@ impl ChatStreamEncoder {
role_sent: false,
output: BTreeMap::new(),
terminal: None,
content_chars: 0,
}
}
@@ -196,11 +266,7 @@ impl ChatStreamEncoder {
PartKind::Text
};
let delta = required_str(data, "delta")?;
let part = self.event_part(data, kind)?;
part.text.push_str(delta);
if !delta.is_empty() {
frames.push(self.delta(json!({kind.field(): delta}))?);
}
self.text_event(data, kind, delta, false, data, &mut frames)?;
}
Some("response.output_text.done" | "response.refusal.done") => {
self.require_open()?;
@@ -209,22 +275,29 @@ impl ChatStreamEncoder {
} else {
(PartKind::Text, "text")
};
let text = required_str(data, field)?;
let part = self.event_part(data, kind)?;
let suffix = append_snapshot(&mut part.text, text)?;
if !suffix.is_empty() {
frames.push(self.delta(json!({kind.field(): suffix}))?);
}
self.text_event(
data,
kind,
required_str(data, field)?,
true,
data,
&mut frames,
)?;
}
Some("response.content_part.added" | "response.content_part.done") => {
self.require_open()?;
let part = data.get("part").ok_or_else(invalid_response)?;
let (kind, text) = part_text(part)?;
let part = self.event_part(data, kind)?;
let suffix = append_snapshot(&mut part.text, text)?;
if !suffix.is_empty() {
frames.push(self.delta(json!({kind.field(): suffix}))?);
}
self.text_event(data, kind, text, true, part, &mut frames)?;
}
Some("response.output_text.annotation.added") => {
self.require_open()?;
let part = self.event_part(data, PartKind::Text)?;
insert_annotation(
part,
required_index(data, "annotation_index")?,
data.get("annotation").ok_or_else(invalid_response)?,
)?;
}
Some("response.output_item.added" | "response.output_item.done") => {
self.require_open()?;
@@ -236,27 +309,73 @@ impl ChatStreamEncoder {
)?;
}
Some(
"response.function_call_arguments.delta" | "response.function_call_arguments.done",
"response.function_call_arguments.delta"
| "response.function_call_arguments.done"
| "response.custom_tool_call_input.delta"
| "response.custom_tool_call_input.done",
) => {
self.require_open()?;
let index = required_index(data, "output_index")?;
let item = self.output.get_mut(&index).ok_or_else(invalid_response)?;
bind_id(item, optional_id(data, "item_id")?)?;
let Output::Function(tool) = &mut item.output else {
let Output::Tool(tool) = &mut item.output else {
return Err(invalid_response());
};
let delta = if event_type == Some("response.function_call_arguments.delta") {
let kind = tool.kind;
let custom_event = event_type
.is_some_and(|event| event.starts_with("response.custom_tool_call_input."));
if custom_event != (kind == ToolKind::Custom) {
return Err(invalid_response());
}
let delta = if event_type.is_some_and(|event| event.ends_with(".delta")) {
let delta = required_str(data, "delta")?;
tool.arguments.push_str(delta);
delta
} else {
append_snapshot(&mut tool.arguments, required_str(data, "arguments")?)?
append_snapshot(&mut tool.arguments, required_str(data, kind.data_field())?)?
};
let tool_index = tool.index;
if !delta.is_empty() {
frames.push(self.delta(json!({"tool_calls": [{"index": tool_index, "function": {"arguments": delta}}]}))?);
frames.push(self.delta(json!({"tool_calls": [{"index": tool_index, kind.chat_type(): {kind.data_field(): delta}}]}))?);
}
}
Some(
"response.reasoning_summary_text.delta"
| "response.reasoning_summary_text.done"
| "response.reasoning_text.delta"
| "response.reasoning_text.done",
) => {
self.require_open()?;
let summary =
event_type.is_some_and(|event| event.starts_with("response.reasoning_summary"));
let snapshot = event_type.is_some_and(|event| event.ends_with(".done"));
self.reasoning_text(
required_index(data, "output_index")?,
optional_id(data, "item_id")?,
(
summary,
required_index(
data,
if summary {
"summary_index"
} else {
"content_index"
},
)?,
),
required_str(data, if snapshot { "text" } else { "delta" })?,
snapshot,
&mut frames,
)?;
}
Some("response.image_generation_call.partial_image") => {
self.require_open()?;
self.item_snapshot(required_index(data, "output_index")?, &json!({
"id": required_nonempty_str(data, "item_id")?, "type":"image_generation_call",
"result": required_str(data, "partial_image_b64")?,
"output_format": data.get("output_format").unwrap_or(&json!("png")),
}), false, &mut frames)?;
}
Some("response.completed" | "response.incomplete") => {
self.require_open()?;
let response = data.get("response").ok_or_else(invalid_response)?;
@@ -275,6 +394,10 @@ impl ChatStreamEncoder {
} else if self.output.is_empty() {
return Err(invalid_response());
}
let annotations = self.stream_annotations()?;
if !annotations.is_empty() {
frames.push(self.delta(json!({"annotations":annotations}))?);
}
let finish_reason = finish_reason(response, self.tool_count() > 0)?;
frames.push(self.chunk(
json!([{"index": 0, "delta": {}, "finish_reason": finish_reason, "logprobs": null}]),
@@ -310,7 +433,7 @@ impl ChatStreamEncoder {
// 辅助事件可随上游演进;实际 output item 和 content part 仍逐项检查可表达性
_ => {}
}
// done-only 流也要提供完整 Chat 消息身份,辅助事件或私有推理不触发角色帧
// done-only 流也要提供完整 Chat 消息身份,不含可交付内容的辅助事件不触发角色帧
if !self.role_sent && !frames.is_empty() && !self.has_wire_failure() {
frames.insert(0, self.delta(json!({"role": "assistant"}))?);
self.role_sent = true;
@@ -318,6 +441,106 @@ impl ChatStreamEncoder {
Ok(frames)
}
fn text_event(
&mut self,
data: &Value,
kind: PartKind,
text: &str,
snapshot: bool,
metadata: &Value,
frames: &mut Vec<Bytes>,
) -> Result<(), ProtocolErrorBody> {
let mut content_chars = self.content_chars;
let part = self.event_part(data, kind)?;
let delta = if snapshot {
append_snapshot(&mut part.text, text)?
} else {
part.text.push_str(text);
text
};
record_text_span(part, &mut content_chars, delta);
merge_annotations(part, metadata)?;
let logprobs = append_logprobs(part, metadata, snapshot)?;
self.content_chars = content_chars;
if !delta.is_empty() {
frames.push(self.delta(json!({kind.field():delta}))?);
}
if !logprobs.is_empty() {
frames.push(logprobs_frame(
&self.identity,
self.include_usage,
kind,
logprobs,
)?);
}
Ok(())
}
fn stream_annotations(&self) -> Result<Vec<Value>, ProtocolErrorBody> {
let mut result = Vec::new();
for item in self.output.values() {
if let Output::Message(parts) = &item.output {
for part in parts.values() {
let mut citations = Vec::new();
append_url_citations(
&json!({"annotations":part.annotations.values().collect::<Vec<_>>()}),
0,
&mut citations,
)?;
for mut citation in citations {
for field in ["start_index", "end_index"] {
let position = required_index(&citation["url_citation"], field)?;
citation["url_citation"][field] =
delivered_position(part, position, field == "end_index")?.into();
}
result.push(citation);
}
}
}
}
Ok(result)
}
fn reasoning_text(
&mut self,
index: u64,
id: Option<&str>,
part: (bool, u64),
text: &str,
snapshot: bool,
frames: &mut Vec<Bytes>,
) -> Result<(), ProtocolErrorBody> {
let has_text = self.output.values().any(|item| {
matches!(&item.output,
Output::Reasoning(parts) if parts.values().any(|text| !text.is_empty()))
});
let item = self.output.entry(index).or_insert_with(|| OutputItem {
id: None,
output: Output::Reasoning(BTreeMap::new()),
});
bind_id(item, id)?;
let Output::Reasoning(parts) = &mut item.output else {
return Err(invalid_response());
};
let previous = parts.entry(part).or_default();
let separator = previous.is_empty() && has_text;
let delta = if snapshot {
append_snapshot(previous, text)?
} else {
previous.push_str(text);
text
};
if !delta.is_empty() {
let delta = if separator {
format!("\n\n{delta}")
} else {
delta.to_owned()
};
frames.push(self.delta(json!({"reasoning_content":delta}))?);
}
Ok(())
}
fn require_open(&self) -> Result<(), ProtocolErrorBody> {
if self.terminal.is_some() {
Err(invalid_response())
@@ -329,7 +552,7 @@ impl ChatStreamEncoder {
fn tool_count(&self) -> usize {
self.output
.values()
.filter(|item| matches!(item.output, Output::Function(_)))
.filter(|item| matches!(item.output, Output::Tool(_)))
.count()
}
@@ -354,7 +577,7 @@ impl ChatStreamEncoder {
) -> Result<(), ProtocolErrorBody> {
let id = optional_id(snapshot, "id")?;
let kind = required_str(snapshot, "type")?;
let call_id = if kind == "function_call" {
let call_id = if matches!(kind, "function_call" | "custom_tool_call") {
Some(required_nonempty_str(snapshot, "call_id")?)
} else {
None
@@ -368,13 +591,23 @@ impl ChatStreamEncoder {
if !self.output.contains_key(&index) {
let output = match kind {
"message" => Output::Message(BTreeMap::new()),
"reasoning" => Output::Reasoning,
"web_search_call" => Output::WebSearch,
"function_call" => {
let (call_id, name, _) = function_fields(snapshot)?;
"reasoning" => Output::Reasoning(BTreeMap::new()),
"image_generation_call" => Output::Image {
index: self
.output
.values()
.filter(|item| matches!(item.output, Output::Image { .. }))
.count(),
fingerprint: None,
},
kind if is_auxiliary_output(kind) => Output::Auxiliary(kind.to_owned()),
"function_call" | "custom_tool_call" => {
let kind = ToolKind::from_item(snapshot)?;
let (call_id, name, _) = tool_fields(snapshot, kind)?;
let tool_index = self.tool_count();
frames.push(self.delta(json!({"tool_calls": [{"index": tool_index, "id": call_id, "type": "function", "function": {"name": name, "arguments": ""}}]}))?);
Output::Function(Tool {
frames.push(self.delta(json!({"tool_calls": [{"index": tool_index, "id": call_id, "type": kind.chat_type(), kind.chat_type(): {"name": name, kind.data_field(): ""}}]}))?);
Output::Tool(Tool {
kind,
index: tool_index,
call_id: call_id.to_owned(),
name: name.to_owned(),
@@ -391,8 +624,20 @@ impl ChatStreamEncoder {
(Output::Message(parts), "message") => {
for (index, part) in message_parts(snapshot)?.iter().enumerate() {
let (kind, text) = part_text(part)?;
let snapshot = part;
let part = part_entry(parts, index as u64, kind)?;
let suffix = append_snapshot(&mut part.text, text)?;
record_text_span(part, &mut self.content_chars, suffix);
merge_annotations(part, snapshot)?;
let logprobs = append_logprobs(part, snapshot, true)?;
if !logprobs.is_empty() {
frames.push(logprobs_frame(
&self.identity,
self.include_usage,
kind,
logprobs,
)?);
}
if !suffix.is_empty() {
frames.push(delta_frame(
&self.identity,
@@ -402,17 +647,41 @@ impl ChatStreamEncoder {
}
}
}
(Output::Function(tool), "function_call") => {
let (id, name, arguments) = function_fields(snapshot)?;
(Output::Tool(tool), "function_call" | "custom_tool_call") => {
let kind = ToolKind::from_item(snapshot)?;
if kind != tool.kind {
return Err(invalid_response());
}
let (id, name, arguments) = tool_fields(snapshot, kind)?;
if tool.call_id != id || tool.name != name {
return Err(invalid_response());
}
let suffix = append_snapshot(&mut tool.arguments, arguments)?;
if !suffix.is_empty() {
frames.push(delta_frame(&self.identity, self.include_usage, json!({"tool_calls": [{"index": tool.index, "function": {"arguments": suffix}}]}))?);
frames.push(delta_frame(&self.identity, self.include_usage, json!({"tool_calls": [{"index": tool.index, kind.chat_type(): {kind.data_field(): suffix}}]}))?);
}
}
(Output::Reasoning, "reasoning") | (Output::WebSearch, "web_search_call") => {}
(Output::Reasoning(_), "reasoning") => {
for (summary, part_index, text) in reasoning_parts(snapshot)? {
self.reasoning_text(index, id, (summary, part_index), text, true, frames)?;
}
}
(Output::Image { index, fingerprint }, "image_generation_call") => {
if let Some(mut image) = image_payload(snapshot)? {
let hash: [u8; 32] =
Sha256::digest(required_str(&image["image_url"], "url")?.as_bytes()).into();
if *fingerprint != Some(hash) {
*fingerprint = Some(hash);
image["index"] = (*index).into();
frames.push(delta_frame(
&self.identity,
self.include_usage,
json!({"images":[image]}),
)?);
}
}
}
(Output::Auxiliary(previous), kind) if previous == kind => {}
_ => return Err(invalid_response()),
}
Ok(())
@@ -426,7 +695,7 @@ impl ChatStreamEncoder {
let mut found = None;
for (&index, item) in &self.output {
let same_id = id.is_some() && item.id.as_deref() == id;
let same_call = matches!(&item.output, Output::Function(tool) if call_id == Some(tool.call_id.as_str()));
let same_call = matches!(&item.output, Output::Tool(tool) if call_id == Some(tool.call_id.as_str()));
if same_id || same_call {
if found.is_some_and(|previous| previous != index) {
return Err(invalid_response());
@@ -470,6 +739,9 @@ fn part_entry(
let part = parts.entry(index).or_insert_with(|| Part {
kind,
text: String::new(),
logprobs: Vec::new(),
annotations: BTreeMap::new(),
spans: Vec::new(),
});
if part.kind != kind {
return Err(invalid_response());
@@ -477,6 +749,68 @@ fn part_entry(
Ok(part)
}
fn record_text_span(part: &mut Part, delivered: &mut u64, delta: &str) {
if part.kind != PartKind::Text || delta.is_empty() {
return;
}
let length = delta.chars().count() as u64;
let start = part.spans.last().map_or(0, |span| span.end);
if let Some(last) = part.spans.last_mut()
&& last.delivered_start + last.end - last.start == *delivered
{
last.end += length;
} else {
part.spans.push(TextSpan {
start,
end: start + length,
delivered_start: *delivered,
});
}
*delivered += length;
}
fn delivered_position(part: &Part, position: u64, end: bool) -> Result<u64, ProtocolErrorBody> {
let matches = |span: &&TextSpan| {
if end && position > 0 {
span.start < position && position <= span.end
} else {
span.start <= position && position < span.end
}
};
let span = part
.spans
.iter()
.find(matches)
.or_else(|| part.spans.last().filter(|span| position == span.end))
.ok_or_else(invalid_response)?;
Ok(span.delivered_start + position - span.start)
}
fn merge_annotations(part: &mut Part, value: &Value) -> Result<(), ProtocolErrorBody> {
if let Some(values) = value.get("annotations").filter(|value| !value.is_null()) {
for (index, annotation) in values
.as_array()
.ok_or_else(invalid_response)?
.iter()
.enumerate()
{
insert_annotation(part, index as u64, annotation)?;
}
}
Ok(())
}
fn insert_annotation(part: &mut Part, index: u64, value: &Value) -> Result<(), ProtocolErrorBody> {
if let Some(previous) = part.annotations.get(&index) {
if previous != value {
return Err(invalid_response());
}
} else {
part.annotations.insert(index, value.clone());
}
Ok(())
}
fn bind_id(item: &mut OutputItem, id: Option<&str>) -> Result<(), ProtocolErrorBody> {
if let Some(id) = id {
if item.id.as_deref().is_some_and(|previous| previous != id) {
@@ -542,11 +876,109 @@ fn append_url_citations(
Ok(())
}
fn function_fields(item: &Value) -> Result<(&str, &str, &str), ProtocolErrorBody> {
fn is_auxiliary_output(kind: &str) -> bool {
// 服务端工具记录和压缩密文不要求 Chat 客户端执行动作,也不投影为可读内容
matches!(
kind,
"web_search_call"
| "file_search_call"
| "code_interpreter_call"
| "mcp_call"
| "mcp_list_tools"
| "compaction"
| "compaction_summary"
| "context_compaction"
)
}
fn reasoning_parts(item: &Value) -> Result<Vec<(bool, u64, &str)>, ProtocolErrorBody> {
let mut result = Vec::new();
// 只交付上游明确公开的推理文本,不解读或暴露 encrypted_content
for (field, kind) in [("summary", "summary_text"), ("content", "reasoning_text")] {
if let Some(parts) = item.get(field).filter(|value| !value.is_null()) {
for (index, part) in parts
.as_array()
.ok_or_else(invalid_response)?
.iter()
.enumerate()
{
if part["type"] == kind {
result.push((
field == "summary",
index as u64,
required_str(part, "text")?,
));
}
}
}
}
Ok(result)
}
fn image_payload(item: &Value) -> Result<Option<Value>, ProtocolErrorBody> {
let Some(data) = item.get("result").filter(|value| !value.is_null()) else {
return Ok(None);
};
let data = data.as_str().ok_or_else(invalid_response)?;
if data.is_empty() {
return Ok(None);
}
let format = item
.get("output_format")
.and_then(Value::as_str)
.unwrap_or("png");
let mime = match format {
"png" => "image/png",
"jpeg" | "jpg" => "image/jpeg",
"webp" => "image/webp",
_ => return Err(invalid_response()),
};
Ok(Some(
json!({"type":"image_url", "image_url":{"url":format!("data:{mime};base64,{data}")}}),
))
}
fn append_logprobs(
part: &mut Part,
value: &Value,
snapshot: bool,
) -> Result<Vec<Value>, ProtocolErrorBody> {
let Some(values) = value.get("logprobs").filter(|value| !value.is_null()) else {
return Ok(Vec::new());
};
let values = values.as_array().ok_or_else(invalid_response)?;
let delta = if snapshot {
values
.strip_prefix(part.logprobs.as_slice())
.ok_or_else(invalid_response)?
} else {
values
};
let delta = delta.to_vec();
part.logprobs.extend(delta.iter().cloned());
Ok(delta)
}
fn logprobs_frame(
identity: &ChatIdentity,
include_usage: bool,
kind: PartKind,
values: Vec<Value>,
) -> Result<Bytes, ProtocolErrorBody> {
chunk_frame(
identity,
json!([{"index":0, "delta":{}, "finish_reason":null,
"logprobs":{kind.field():values}}]),
include_usage.then_some(Value::Null),
None,
)
}
fn tool_fields(item: &Value, kind: ToolKind) -> Result<(&str, &str, &str), ProtocolErrorBody> {
Ok((
required_nonempty_str(item, "call_id")?,
required_nonempty_str(item, "name")?,
required_str(item, "arguments")?,
required_str(item, kind.data_field())?,
))
}
@@ -559,8 +991,10 @@ fn validate_output_identities(output: &[Value]) -> Result<(), ProtocolErrorBody>
{
return Err(invalid_response());
}
if item.get("type").and_then(Value::as_str) == Some("function_call")
&& !call_ids.insert(required_nonempty_str(item, "call_id")?)
if matches!(
item.get("type").and_then(Value::as_str),
Some("function_call" | "custom_tool_call")
) && !call_ids.insert(required_nonempty_str(item, "call_id")?)
{
return Err(invalid_response());
}
@@ -592,19 +1026,41 @@ fn chat_usage(response: &Value) -> Option<Value> {
return None;
}
let mut result = json!({"prompt_tokens": usage.input_tokens, "completion_tokens": usage.output_tokens, "total_tokens": usage.total_tokens});
if response
.pointer("/usage/input_tokens_details/cached_tokens")
.is_some()
{
result["prompt_tokens_details"] = json!({"cached_tokens": usage.cached_input_tokens});
for (source, target) in [
("input_tokens_details", "prompt_tokens_details"),
("output_tokens_details", "completion_tokens_details"),
] {
if let Some(details) = response
.get("usage")
.and_then(|usage| usage.get(source))
.filter(|value| value.is_object())
{
result[target] = details.clone();
}
}
if response
.pointer("/usage/output_tokens_details/reasoning_tokens")
.and_then(Value::as_u64)
.is_some_and(|tokens| tokens <= usage.output_tokens)
if let Some(details) = result
.get_mut("prompt_tokens_details")
.and_then(Value::as_object_mut)
&& details.contains_key("cached_tokens")
{
result["completion_tokens_details"] =
json!({"reasoning_tokens": usage.reasoning_output_tokens});
details.insert("cached_tokens".into(), usage.cached_input_tokens.into());
}
if let Some(details) = result
.get_mut("completion_tokens_details")
.and_then(Value::as_object_mut)
{
if details
.get("reasoning_tokens")
.and_then(Value::as_u64)
.is_some_and(|tokens| tokens <= usage.output_tokens)
{
details.insert(
"reasoning_tokens".into(),
usage.reasoning_output_tokens.into(),
);
} else {
details.remove("reasoning_tokens");
}
}
Some(result)
}
@@ -1,6 +1,13 @@
//! OpenAI Provider 原生非流式 JSON 端点的公共交付边界
use std::{future::Future, net::IpAddr};
use std::{
future::Future,
net::IpAddr,
sync::{
Arc,
atomic::{AtomicBool, Ordering},
},
};
use axum::{
body::{Body, Bytes},
@@ -49,6 +56,8 @@ where
};
let request_id = prepared.request_id().clone();
let endpoint = input.endpoint.clone();
let execution_entered = Arc::new(AtomicBool::new(false));
let terminal_entered = Arc::clone(&execution_entered);
let result = invoke_request(
execution,
prepared,
@@ -60,6 +69,7 @@ where
return Err(MiddlewareError::Rejected);
}
let operation = request.apply_capabilities(decode(body, headers).await?)?;
terminal_entered.store(true, Ordering::Relaxed);
let started = match service
.start_prepared_provider_endpoint(
prepared, operation,
@@ -82,11 +92,18 @@ where
Ok(response) => into_http_response(response, ExpectedBody::SingleJson).await,
Err(error) => error_response(error),
};
if response.headers().contains_key("x-gateway-request-id") {
if !execution_entered.load(Ordering::Relaxed) {
return response;
}
let mut response = if response.headers().contains_key("x-gateway-request-id") {
response
} else {
super::with_model_request_id(response, &request_id)
}
};
response
.extensions_mut()
.insert(crate::client_error::ExecutionObserved);
response
}
/// 只收集正文,不提前 commit;外层完成响应变换与校验后才提交 Core
@@ -23,7 +23,9 @@ pub use crate::client_error::{
/// 严格协议 decoder/encoder 返回的安全错误 body
pub fn protocol_error_response(status: StatusCode, body: ProtocolErrorBody) -> Response {
(status, Json(body.into_value())).into_response()
let mut response = (status, Json(body.clone().into_value())).into_response();
response.extensions_mut().insert(body);
response
}
/// 下游 Client API Key 无效
@@ -91,6 +91,7 @@ pub(crate) async fn live_call(
Ok(prepared) => prepared,
Err(error) => return gateway_error_response(&error),
};
let request_id = prepared.request_id().clone();
let started = match service
.start_prepared_provider_endpoint(
prepared,
@@ -103,7 +104,7 @@ pub(crate) async fn live_call(
.await
{
Ok(started) => started,
Err(error) => return gateway_error_response(&error),
Err(error) => return with_model_request_id(gateway_error_response(&error), &request_id),
};
collect_live_call_response(started, uri.path().to_owned()).await
}
@@ -25,6 +25,9 @@ pub(crate) fn with_model_request_id(
let Ok(gateway_id) = HeaderValue::from_str(request_id.as_str()) else {
return response;
};
response
.extensions_mut()
.insert(crate::client_error::ExecutionObserved);
let headers = response.headers_mut();
let usable = |value: &HeaderValue| value.to_str().is_ok_and(|id| !id.trim().is_empty());
if !headers.get("x-request-id").is_some_and(usable) {
@@ -85,7 +85,7 @@ async fn peer_response(state: ApiState, client: AuthenticatedClient, uri: Uri) -
StatusCode::FORBIDDEN,
"CPR Link sharing is not enabled for this API key.",
"permission_error",
"peer_sharing_disabled",
gateway_protocol::peer::SHARING_DISABLED_CODE,
)
.into_response(),
Err(PeerLinkError::RateLimited { retry_after }) => {
@@ -5,7 +5,10 @@ use crate::request_context::request_client_context;
use std::collections::VecDeque;
use std::future::Future;
use std::net::SocketAddr;
use std::sync::{Arc, OnceLock};
use std::sync::{
Arc, OnceLock,
atomic::{AtomicBool, Ordering},
};
use std::time::Duration;
use axum::{
@@ -159,6 +162,8 @@ pub(in crate::openai) async fn execute_prepared_response(
let execution = service.execution();
let request_id = prepared.request_id().clone();
let cancellation = prepared.cancellation();
let execution_entered = Arc::new(AtomicBool::new(false));
let terminal_entered = Arc::clone(&execution_entered);
let started_request_id = Arc::new(OnceLock::new());
let terminal_request_id = Arc::clone(&started_request_id);
let (admitted, keepalive) = tokio::sync::oneshot::channel();
@@ -217,6 +222,7 @@ pub(in crate::openai) async fn execute_prepared_response(
} else {
None
};
terminal_entered.store(true, Ordering::Relaxed);
let started = terminal_service
.start_prepared_response(prepared, decoded, transport, endpoint)
.await
@@ -276,7 +282,15 @@ pub(in crate::openai) async fn execute_prepared_response(
} else {
response.await
};
super::super::with_model_request_id(response, started_request_id.get().unwrap_or(&request_id))
// 预留 ID 没有对应执行记录,中间件提前返回时保留 HTTP 入口关联 ID
if execution_entered.load(Ordering::Relaxed) {
super::super::with_model_request_id(
response,
started_request_id.get().unwrap_or(&request_id),
)
} else {
response
}
}
async fn await_with_authorization<T>(
@@ -446,7 +446,8 @@ fn account_page_item(
#[tokio::test]
async fn account_usage_preserves_periods_cache_counts_and_missing_values() {
use chrono::Utc;
use gateway_admin::model::accounts::AccountUsage;
use gateway_admin::model::accounts::{AccountModelUsage, AccountUsage};
use gateway_admin::model::observability::RequestHealth;
use gateway_admin::model::provider_credentials::{
ProviderQuota, ProviderQuotaWindow, QuotaLocalUsageAttribution,
};
@@ -497,7 +498,35 @@ async fn account_usage_preserves_periods_cache_counts_and_missing_values() {
bucket_start: now,
request_count: 1,
}],
models: Vec::new(),
models: [
(
"mixed",
RequestHealth::from_counts(1, 2, 3, 1, 1),
Some(now),
),
("failed", RequestHealth::from_counts(0, 1, 0, 0, 0), None),
("no-sample", RequestHealth::from_counts(0, 1, 1, 0, 1), None),
]
.into_iter()
.map(|(model, health, last_used_at)| AccountModelUsage {
model: model.to_owned(),
request_count: u64::from(last_used_at.is_some()),
health,
input_tokens: None,
output_tokens: None,
cached_input_tokens: None,
cache_write_input_tokens: None,
reasoning_output_tokens: None,
image_input_tokens: None,
image_output_tokens: None,
image_request_count: 0,
image_request_failed_count: 0,
total_tokens: None,
cost_coverage: Default::default(),
costs: Vec::new(),
last_used_at,
})
.collect(),
};
*fixture.account_usage.lock().unwrap() = vec![local_usage.clone()];
*fixture.provider_quota.lock().unwrap() =
@@ -545,6 +574,13 @@ async fn account_usage_preserves_periods_cache_counts_and_missing_values() {
"general"
}
);
assert_eq!(usage["models"][0]["successRateDisplay"], "33.3%");
assert!((usage["models"][0]["successRate"].as_f64().unwrap() - 100.0 / 3.0).abs() < 0.001);
assert_eq!(usage["models"][1]["successRate"], 0.0);
assert_eq!(usage["models"][1]["successRateDisplay"], "0.0%");
assert!(usage["models"][1]["lastUsedAt"].is_null());
assert!(usage["models"][2]["successRate"].is_null());
assert_eq!(usage["models"][2]["successRateDisplay"], "—");
assert_eq!(usage["requestCount"], 1);
assert_eq!(usage["totalTokens"], 120);
if authentication_kind == "oauth" {
@@ -354,9 +354,12 @@ impl ProxyProbe for SuccessfulProbe {
proxy.expose_url(),
"http://test-user:private-password@proxy.example:8080/"
);
} else {
assert!(detect_location);
}
let exit_ip = if proxy.is_some() {
"203.0.113.2"
} else {
"203.0.113.3"
};
ProxyTestResult {
location: if detect_location {
ProxyLocationDetection::Detected { location: serde_json::from_value(json!({"country":"JP","region":"Tokyo","city":"Tokyo","timezone":"Asia/Tokyo"})).unwrap() }
@@ -365,8 +368,8 @@ impl ProxyProbe for SuccessfulProbe {
},
success: true,
latency_ms: 15,
exit_ip: Some("203.0.113.2".parse().unwrap()),
exit_ipv4: Some("203.0.113.2".parse().unwrap()),
exit_ip: Some(exit_ip.parse().unwrap()),
exit_ipv4: Some(exit_ip.parse().unwrap()),
exit_ipv6: None,
message: "Connected".to_owned(),
}
@@ -567,11 +570,50 @@ async fn proxy_probe_checks_unsaved_address_without_creating_or_changing_records
}
#[tokio::test]
async fn proxy_probe_rejects_empty_or_invalid_addresses() {
async fn proxy_probe_checks_direct_egress_without_changing_saved_proxies() {
let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session");
let (_, created) = request(
&fixture,
"/api/admin/proxies/create",
Some(json!([{"name": "Saved", "proxyUrl": "http://saved.example:8080"}])),
true,
)
.await;
for detect_location in [false, true] {
let (status, probed) = request(
&fixture,
"/api/admin/proxies/probe",
Some(json!({"proxyUrl": "", "detectLocation": detect_location})),
true,
)
.await;
assert_eq!(status, StatusCode::OK);
assert_eq!(probed["data"]["success"], true);
assert_eq!(probed["data"]["exitIpv4"], "203.0.113.3");
assert_eq!(
probed["data"]["location"]["status"],
if detect_location {
"detected"
} else {
"notRequested"
}
);
}
let (_, listed) = request(&fixture, "/api/admin/proxies", None, true).await;
assert_eq!(listed["data"]["page"]["total"], 1);
assert_eq!(
listed["data"]["items"][0],
created["data"][0]["data"]["record"]
);
}
#[tokio::test]
async fn proxy_probe_rejects_missing_or_invalid_addresses() {
let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session");
for (body, expected) in [
(json!({"proxyUrl": ""}), StatusCode::BAD_REQUEST),
(json!({"proxyUrl": " "}), StatusCode::UNPROCESSABLE_ENTITY),
(json!({"proxyUrl": null}), StatusCode::UNPROCESSABLE_ENTITY),
(json!({}), StatusCode::UNPROCESSABLE_ENTITY),
(
@@ -0,0 +1,257 @@
//! HTTP 入口拒绝可检索,诊断摘要不保存凭据和请求内容
use std::sync::Arc;
use axum::{
body::{Body, to_bytes},
http::{Request, StatusCode},
};
use serde_json::{Value, json};
use tower::ServiceExt as _;
use crate::support::RecordingDiagnostics;
async fn fixture() -> (axum::Router, Arc<RecordingDiagnostics>) {
let admin = crate::admin::AdminTestFixture::new().await;
let diagnostics = Arc::new(RecordingDiagnostics::default());
let router = crate::openai::api_bundle_with_diagnostics(
admin.services,
gateway_api::ApiConfig {
asset_directory: env!("CARGO_MANIFEST_DIR").into(),
cors_allowed_origins: Vec::new(),
request_timeout_seconds: None,
request_id_header: "x-correlation-id".into(),
},
diagnostics.clone(),
)
.router();
(router, diagnostics)
}
#[tokio::test]
async fn decode_rejections_record_the_ingress_id_and_safe_parameter_without_model_facts() {
for (path, body, field) in [
(
"/v1/chat/completions",
json!({"model":"model-a", "messages":[{"role":"user", "content":[{"type":"SENSITIVE_CONTENT"}]}]}),
"messages[0].content[0].type",
),
(
"/v1/responses",
json!({"model":42, "input":"SENSITIVE_CONTENT"}),
"model",
),
] {
let (router, diagnostics) = fixture().await;
let response = router
.oneshot(
Request::post(path)
.header("authorization", "Bearer unused-client-route-key")
.header("x-correlation-id", "ingress-test-608")
.body(Body::from(body.to_string()))
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
assert_eq!(response.headers()["x-correlation-id"], "ingress-test-608");
let body: Value =
serde_json::from_slice(&to_bytes(response.into_body(), usize::MAX).await.unwrap())
.unwrap();
assert_eq!(body["error"]["param"], field);
let failures = diagnostics.0.lock().unwrap();
assert_eq!(failures.len(), 1);
let failure = &failures[0];
assert_eq!(failure.component, "api");
assert_eq!(failure.operation, "http_ingress");
assert_eq!(failure.correlation_id.as_deref(), Some("ingress-test-608"));
let summary: Value = serde_json::from_str(&failure.message).unwrap();
assert_eq!(summary["param"], field);
assert_eq!(summary["endpoint"], path);
assert!(failure.upstream_status.is_none());
assert!(failure.account_id.is_none());
assert!(!failure.message.contains("SENSITIVE_CONTENT"));
assert!(!failure.message.contains("unused-client-route-key"));
}
}
#[tokio::test]
async fn authentication_unknown_routes_and_method_errors_are_observable() {
for (method, path, status, code) in [
(
"POST",
"/v1/chat/completions",
StatusCode::UNAUTHORIZED,
"missing_authorization",
),
(
"POST",
"/v1/messages?secret=SENSITIVE_QUERY",
StatusCode::NOT_FOUND,
"unknown_endpoint",
),
(
"DELETE",
"/v1/chat/completions",
StatusCode::METHOD_NOT_ALLOWED,
"method_not_allowed",
),
] {
let (router, diagnostics) = fixture().await;
let response = router
.oneshot(
Request::builder()
.method(method)
.uri(path)
.body(Body::from("SENSITIVE_CONTENT"))
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), status);
let failures = diagnostics.0.lock().unwrap();
assert_eq!(failures.len(), 1);
assert_eq!(failures[0].kind, code);
assert!(!failures[0].message.contains("SENSITIVE"));
assert!(failures[0].correlation_id.is_some());
}
}
#[derive(Clone)]
struct Recorder {
dump_enabled: bool,
events: Arc<std::sync::Mutex<Vec<std::collections::BTreeMap<String, String>>>>,
}
impl tracing::Subscriber for Recorder {
fn enabled(&self, metadata: &tracing::Metadata<'_>) -> bool {
metadata.target() != "request_dump" || self.dump_enabled
}
fn register_callsite(
&self,
_: &'static tracing::Metadata<'static>,
) -> tracing::subscriber::Interest {
tracing::subscriber::Interest::sometimes()
}
fn new_span(&self, _: &tracing::span::Attributes<'_>) -> tracing::span::Id {
tracing::span::Id::from_u64(1)
}
fn record(&self, _: &tracing::span::Id, _: &tracing::span::Record<'_>) {}
fn record_follows_from(&self, _: &tracing::span::Id, _: &tracing::span::Id) {}
fn enter(&self, _: &tracing::span::Id) {}
fn exit(&self, _: &tracing::span::Id) {}
fn event(&self, event: &tracing::Event<'_>) {
struct Fields(std::collections::BTreeMap<String, String>);
impl tracing::field::Visit for Fields {
fn record_str(&mut self, field: &tracing::field::Field, value: &str) {
self.0.insert(field.name().to_owned(), value.to_owned());
}
fn record_debug(&mut self, field: &tracing::field::Field, value: &dyn std::fmt::Debug) {
self.0.insert(field.name().to_owned(), format!("{value:?}"));
}
}
let mut fields = Fields(std::collections::BTreeMap::new());
event.record(&mut fields);
fields
.0
.insert("target".into(), event.metadata().target().into());
self.events.lock().unwrap().push(fields.0);
}
}
#[tokio::test]
async fn original_rejected_request_is_dumped_only_when_explicitly_enabled() {
use base64::{Engine as _, engine::general_purpose::STANDARD};
use tracing::instrument::WithSubscriber as _;
for enabled in [false, true] {
let recorder = Recorder {
dump_enabled: enabled,
events: Arc::default(),
};
let events = recorder.events.clone();
async {
let (router, _) = fixture().await;
let response = router
.clone()
.oneshot(
Request::post("/v1/chat/completions")
.header("authorization", "Bearer unused-client-route-key")
.body(Body::from("{\"secret\":\"SENSITIVE_CONTENT\","))
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
to_bytes(response.into_body(), usize::MAX).await.unwrap();
let response = router
.oneshot(
Request::post("/v1/unmatched")
.body(Body::from("UNCONSUMED_CONTENT"))
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::NOT_FOUND);
to_bytes(response.into_body(), usize::MAX).await.unwrap();
}
.with_subscriber(recorder)
.await;
let events = events.lock().unwrap();
let dumps = events
.iter()
.filter(|event| {
event
.get("target")
.is_some_and(|target| target == "request_dump")
})
.collect::<Vec<_>>();
assert_eq!(!dumps.is_empty(), enabled);
if enabled {
let original = dumps
.iter()
.filter(|event| {
event
.get("stage")
.is_some_and(|stage| stage == "ingress.request.body")
})
.flat_map(|event| STANDARD.decode(event.get("body_base64").unwrap()).unwrap())
.collect::<Vec<_>>();
assert_eq!(original, b"{\"secret\":\"SENSITIVE_CONTENT\",");
let completion = dumps
.iter()
.filter(|event| {
event
.get("stage")
.is_some_and(|stage| stage == "ingress.body.finished")
})
.map(|event| {
serde_json::from_slice::<Value>(
&STANDARD.decode(event.get("body_base64").unwrap()).unwrap(),
)
.unwrap()
})
.collect::<Vec<_>>();
assert!(completion.iter().any(
|event| event["stage"] == "ingress.request.body" && event["complete"] == false
));
assert!(
completion
.iter()
.any(|event| event["stage"] == "ingress.request.body"
&& event["complete"] == true)
);
}
for event in events.iter().filter(|event| {
event
.get("target")
.is_none_or(|target| target != "request_dump")
}) {
assert!(
!event
.values()
.any(|value| value.contains("SENSITIVE_CONTENT")
|| value.contains("unused-client-route-key"))
);
}
}
}
+1
View File
@@ -3,6 +3,7 @@
mod admin;
mod architecture;
mod auth;
mod diagnostics;
mod health;
mod key_usage;
mod middleware;
@@ -622,3 +622,50 @@ async fn error_after_admission_keepalive_uses_chat_error_encoding() {
assert_eq!(fixture.trace.commits.load(Ordering::SeqCst), 0);
assert_eq!(fixture.trace.response_commit.status_code(), Some(200));
}
#[tokio::test]
async fn early_middleware_failure_is_recorded_at_ingress_without_a_reserved_model_id() {
let middleware = Arc::new(
RequestMiddleware {
expected_operation: Some(gateway_core::operation::OperationKind::Generate),
..Default::default()
}
.with_json_short_circuit(Bytes::from_static(b"{}")),
);
let mut execution = Execution::new(Session::new(vec![completed(json!([message("OK")]))]));
execution.middleware = Some(middleware.frozen());
let execution = Arc::new(execution);
let diagnostics = Arc::new(crate::support::RecordingDiagnostics::default());
let admin = crate::admin::AdminTestFixture::new().await;
let router = gateway_api::initialize(
gateway_api::ApiConfig {
asset_directory: env!("CARGO_MANIFEST_DIR").into(),
cors_allowed_origins: Vec::new(),
request_timeout_seconds: None,
request_id_header: "x-correlation-id".into(),
},
execution.clone(),
admin.services,
Vec::new(),
Arc::new(crate::openai::EmptyWorkerHealth),
Arc::new(crate::openai::TestLifecycle::default()),
diagnostics.clone(),
)
.unwrap()
.router();
let mut request = request(chat_request(false));
request
.headers_mut()
.insert("x-correlation-id", "ingress-short-circuit".parse().unwrap());
let response = router.oneshot(request).await.unwrap();
assert_eq!(response.status(), StatusCode::BAD_GATEWAY);
assert!(!response.headers().contains_key("x-gateway-request-id"));
assert!(execution.captured.lock().unwrap().is_none());
let failures = diagnostics.0.lock().unwrap();
assert_eq!(failures.len(), 1);
assert_eq!(failures[0].operation, "http_ingress");
assert_eq!(
failures[0].correlation_id.as_deref(),
Some("ingress-short-circuit")
);
}
@@ -229,19 +229,11 @@ async fn refuses_known_unrepresentable_features_before_starting_execution() {
for (name, value) in [
("n", json!(2)),
("store", json!(true)),
("stop", json!(["stop"])),
("seed", json!(7)),
("frequency_penalty", json!(0.3)),
("logprobs", json!(true)),
("functions", json!([{"name":"old"}])),
("function_call", json!("auto")),
("input", json!("not chat")),
("previous_response_id", json!("resp_previous")),
("audio", json!({"voice":"alloy","format":"wav"})),
(
"prediction",
json!({"type":"content","content":"predicted"}),
),
] {
let fixture = Fixture::new(Session::new(Vec::new())).await;
let mut body = chat_request(false);
@@ -260,14 +252,8 @@ async fn refuses_known_unrepresentable_features_before_starting_execution() {
#[tokio::test]
async fn rejects_unmapped_message_content_and_identity_fields() {
for message in [
json!({"role":"user","name":"speaker","content":"hello"}),
json!({"role":"tool","tool_call_id":"call-lookup","name":42,"content":"result"}),
json!({"role":"user","content":[{"type":"file","file":{"file_id":"file-example"}}]}),
json!({"role":"user","content":[{"type":"file","file":{"file_data":"data:application/pdf;base64,JVBERg=="}}]}),
json!({"role":"user","content":[{"type":"file","file":{"filename":"a.pdf","file_data":"data:application/pdf;base64,"}}]}),
json!({"role":"user","content":[{"type":"file","file":{"filename":"a.txt","file_data":"data:text/plain;base64,SGk="}}]}),
json!({"role":"assistant","content":[{"type":"file","file":{"filename":"a.pdf","file_data":"data:application/pdf;base64,JVBERg=="}}]}),
json!({"role":"assistant","content":"hello","reasoning_content":"private"}),
json!({"role":"assistant","content":[{"type":"text","text":"history","prompt_cache_breakpoint":{"type":"default"}}]}),
json!({"role":"tool","content":"unassociated output"}),
] {
@@ -313,3 +299,121 @@ async fn assistant_tool_history_keeps_arguments_opaque_and_cache_annotations_str
);
assert_eq!(converted["moderation"], json!({"future":{"keep":true}}));
}
#[tokio::test]
async fn optional_chat_controls_degrade_without_blocking_upstream_execution() {
for stop in [json!("end"), json!(["end", "stop"]), json!([])] {
let converted = convert(json!({"model":"model-a", "messages":[
{"role":"user", "name":"speaker", "content":"hello"},
{"role":"assistant", "reasoning_content":"display-only", "content":"previous"}
], "stop":stop, "seed":7, "frequency_penalty":0.3, "presence_penalty":0.5,
"logit_bias":{"10":-10}, "prediction":{"type":"content", "content":"hint"}}))
.await;
assert_eq!(converted["input"][0]["content"][0]["text"], "hello");
assert_eq!(converted["input"][1]["content"][0]["text"], "previous");
for field in [
"stop",
"seed",
"frequency_penalty",
"presence_penalty",
"logit_bias",
"prediction",
] {
assert!(converted.get(field).is_none(), "{field}");
}
}
}
#[tokio::test]
async fn shared_options_and_native_extensions_reach_upstream_without_model_prejudgment() {
let mut request = chat_request(false);
request.as_object_mut().unwrap().extend(json!({
"max_output_tokens":10, "temperature":3.0, "top_p":1.5,
"reasoning":{"effort":"low", "summary":"auto", "future":true}, "reasoning_effort":"high",
"text":{"verbosity":"low", "future":true}, "verbosity":"high",
"response_format":{"type":"future_format", "future":true},
"tools":[{"type":"file_search", "vector_store_ids":["vs_test"]}],
"tool_choice":"future_choice", "logprobs":true, "top_logprobs":3,
"include":["reasoning.encrypted_content"], "truncation":"auto", "max_tool_calls":5
}).as_object().unwrap().clone());
let converted = convert(request).await;
assert_eq!(converted["max_output_tokens"], 10);
assert_eq!(converted["temperature"], 3.0);
assert_eq!(converted["top_p"], 1.5);
assert_eq!(
converted["reasoning"],
json!({"effort":"high", "summary":"auto", "future":true})
);
assert_eq!(
converted["text"],
json!({"verbosity":"high", "future":true, "format":{"type":"future_format", "future":true}})
);
assert_eq!(converted["tool_choice"], "future_choice");
assert_eq!(
converted["include"],
json!([
"reasoning.encrypted_content",
"message.output_text.logprobs"
])
);
assert_eq!(
converted["tools"][0]["vector_store_ids"],
json!(["vs_test"])
);
assert_eq!(converted["top_logprobs"], 3);
}
#[tokio::test]
async fn custom_tools_keep_native_input_and_mixed_call_families_through_history() {
let schema = json!({"type":"grammar", "grammar":{"syntax":"lark", "definition":"start: /.+/"}});
let converted = convert(json!({"model":"model-a", "tools":[
{"type":"custom", "custom":{"name":"patch", "format":schema}},
{"type":"function", "function":{"name":"lookup"}}
], "tool_choice":{"type":"allowed_tools", "allowed_tools":{"mode":"auto", "tools":[
{"type":"custom", "custom":{"name":"patch"}}, {"type":"function", "function":{"name":"lookup"}}
]}}, "messages":[
{"role":"assistant", "tool_calls":[
{"id":"call_patch", "type":"custom", "custom":{"name":"patch", "input":"raw\ninput"}},
{"id":"call_lookup", "type":"function", "function":{"name":"lookup", "arguments":"{}"}}
]},
{"role":"tool", "tool_call_id":"call_patch", "content":"done"},
{"role":"tool", "tool_call_id":"call_lookup", "content":[{"type":"image_url", "image_url":{"url":"https://example.com/result.png"}}]},
{"role":"assistant", "tool_calls":[{"id":"call_patch", "type":"function", "function":{"name":"lookup", "arguments":"{}"}}]},
{"role":"tool", "tool_call_id":"call_patch", "content":"reused id"}
]})).await;
assert_eq!(
converted["tools"][0],
json!({"type":"custom", "name":"patch", "format":schema})
);
assert_eq!(
converted["tool_choice"],
json!({"type":"allowed_tools", "mode":"auto", "tools":[{"type":"custom","name":"patch"},{"type":"function","name":"lookup"}]})
);
assert_eq!(
converted["input"][0],
json!({"type":"custom_tool_call","call_id":"call_patch","name":"patch","input":"raw\ninput"})
);
assert_eq!(converted["input"][2]["type"], "custom_tool_call_output");
assert_eq!(converted["input"][3]["output"][0]["type"], "input_image");
assert_eq!(converted["input"][5]["type"], "function_call_output");
}
#[tokio::test]
async fn files_and_audio_are_preserved_as_inputs_instead_of_losing_the_user_turn() {
for file in [
json!({"file_id":"file_test"}),
json!({"file_url":"https://example.com/report.txt"}),
json!({"file_data":"data:text/plain;base64,aGk=", "filename":"report.txt", "detail":"high"}),
] {
let converted = convert(json!({"model":"model-a", "messages":[{"role":"user", "content":[
{"type":"file", "file":file}, {"type":"input_audio", "input_audio":{"data":"aGk=", "format":"mp3"}}
]}]})).await;
let mut expected = file;
expected["type"] = json!("input_file");
assert_eq!(converted["input"][0]["content"][0], expected);
assert_eq!(
converted["input"][0]["content"][1],
json!({"type":"input_audio","audio_url":"data:audio/mpeg;base64,aGk="})
);
}
}
@@ -131,7 +131,7 @@ async fn web_search_projects_citations_in_json_and_streams_the_original_links()
async fn json_tool_arguments_and_refusal_do_not_become_assistant_text() {
for output in [
json!([tool("fc1", "call1", "weather", "{ \"city\" : \"北京\" }")]),
json!([{"type":"reasoning","id":"rs1","summary":[{"type":"summary_text","text":"private"}]},
json!([{"type":"reasoning","id":"rs1","summary":[{"type":"summary_text","text":"public summary"}]},
{"type":"message","id":"refusal1","role":"assistant","status":"completed",
"content":[{"type":"refusal","refusal":"cannot help"}]}]),
] {
@@ -141,7 +141,9 @@ async fn json_tool_arguments_and_refusal_do_not_become_assistant_text() {
let body = json_body(response).await;
let choice = &body["choices"][0];
assert_eq!(choice["message"]["content"], Value::Null);
assert!(!body.to_string().contains("private"));
if output[0]["type"] == "reasoning" {
assert_eq!(choice["message"]["reasoning_content"], "public summary");
}
if output[0]["type"] == "function_call" {
assert_eq!(choice["finish_reason"], "tool_calls");
assert_eq!(
@@ -296,7 +298,7 @@ async fn extra_attributes_and_auxiliary_events_preserve_existing_chat_results()
let fixture = Fixture::new(Session::new(vec![
wire("response.future_auxiliary", json!({"future":true})),
wire(
"response.reasoning_summary_text.delta",
"response.future_reasoning.delta",
json!({"output_index":0,"summary_index":0,"delta":"private"}),
),
created(),
@@ -319,6 +321,45 @@ async fn extra_attributes_and_auxiliary_events_preserve_existing_chat_results()
);
}
#[tokio::test]
async fn compaction_state_preserves_chat_answer_without_exposing_ciphertext() {
for kind in ["compaction", "compaction_summary", "context_compaction"] {
for streaming in [false, true] {
let state =
json!({"type":kind,"id":"ctx_1","encrypted_content":"opaque-private-state"});
let fixture = Fixture::new(Session::new(vec![
created(),
wire(
"response.output_item.added",
json!({"output_index":0,"item":state}),
),
wire(
"response.output_item.done",
json!({"output_index":0,"item":state}),
),
completed(json!([state, message("preserved")])),
]))
.await;
let response = fixture.post(chat_request(streaming)).await;
if streaming {
let (text, chunks) = sse_body(response).await;
assert_eq!(combined(&chunks, "content"), "preserved");
assert!(!text.contains("opaque-private-state"));
assert_eq!(
chunks.last().unwrap()["choices"][0]["finish_reason"],
"stop"
);
} else {
assert_eq!(response.status(), StatusCode::OK);
let body = json_body(response).await;
assert_eq!(body["choices"][0]["message"]["content"], "preserved");
assert_eq!(body["choices"][0]["finish_reason"], "stop");
assert!(!body.to_string().contains("opaque-private-state"));
}
}
}
}
#[tokio::test]
async fn incomplete_projects_only_supported_finish_reasons() {
for (reason, expected) in [
@@ -355,7 +396,7 @@ async fn incomplete_projects_only_supported_finish_reasons() {
async fn encrypted_reasoning_stays_private_while_public_text_and_usage_are_projected() {
let reasoning = json!({
"id":"rs_private","type":"reasoning",
"summary":[{"type":"summary_text","text":"PRIVATE_SUMMARY_MARKER"}],
"summary":[{"type":"summary_text","text":"PUBLIC_SUMMARY_MARKER"}],
"encrypted_content":"OPAQUE_ENCRYPTED_REASONING_MARKER"
});
for streaming in [false, true] {
@@ -367,7 +408,7 @@ async fn encrypted_reasoning_stays_private_while_public_text_and_usage_are_proje
),
wire(
"response.reasoning_summary_text.delta",
json!({"item_id":"rs_private","output_index":0,"summary_index":0,"delta":"PRIVATE_SUMMARY_MARKER"}),
json!({"item_id":"rs_private","output_index":0,"summary_index":0,"delta":"PUBLIC_SUMMARY_MARKER"}),
),
wire(
"response.output_item.done",
@@ -377,7 +418,13 @@ async fn encrypted_reasoning_stays_private_while_public_text_and_usage_are_proje
"response.output_item.done",
json!({"output_index":1,"item":message("public answer")}),
),
completed(json!([reasoning, message("public answer")])),
wire("response.completed", json!({"response":{
"id":"resp_chat_test","object":"response","model":"upstream-model","status":"completed",
"output":[reasoning,message("public answer")],
"usage":{"input_tokens":11,"output_tokens":7,"total_tokens":18,
"input_tokens_details":{"cached_tokens":3,"cache_write_tokens":4},
"output_tokens_details":{"reasoning_tokens":2,"accepted_prediction_tokens":1}}
}})),
]))
.await;
let mut request = chat_request(streaming);
@@ -406,12 +453,17 @@ async fn encrypted_reasoning_stays_private_while_public_text_and_usage_are_proje
)
};
assert_eq!(content, "public answer");
assert!(!body.contains("PRIVATE_SUMMARY_MARKER"), "{body}");
assert!(body.contains("PUBLIC_SUMMARY_MARKER"), "{body}");
assert!(
!body.contains("OPAQUE_ENCRYPTED_REASONING_MARKER"),
"{body}"
);
assert_eq!(usage["completion_tokens_details"]["reasoning_tokens"], 2);
assert_eq!(
usage["completion_tokens_details"]["accepted_prediction_tokens"],
1
);
assert_eq!(usage["prompt_tokens_details"]["cache_write_tokens"], 4);
}
}
@@ -419,7 +471,7 @@ async fn encrypted_reasoning_stays_private_while_public_text_and_usage_are_proje
async fn unrepresentable_business_output_fails_without_successful_chat_completion() {
for output in [
json!({"id":"future_output","type":"future_business_output","payload":"unrepresentable"}),
json!({"id":"ig_native","type":"image_generation_call","status":"completed","result":"OPAQUE_IMAGE_MARKER"}),
json!({"id":"action_native","type":"computer_call","status":"completed","call_id":"call_action","action":{"type":"click","x":1,"y":2}}),
] {
for streaming in [false, true] {
let fixture =
@@ -660,3 +712,183 @@ async fn conflicting_text_snapshot_fails_instead_of_appending_replacement_conten
.any(|chunk| chunk["choices"][0]["finish_reason"].is_string())
);
}
#[tokio::test]
async fn custom_and_function_calls_keep_distinct_shapes_in_json_and_interleaved_streams() {
let custom = json!({"id":"ct_1","type":"custom_tool_call","call_id":"call_custom","name":"patch","input":"first\nsecond"});
let function = json!({"id":"fc_1","type":"function_call","call_id":"call_function","name":"lookup","arguments":"{}"});
for streaming in [false, true] {
let mut initial_custom = custom.clone();
initial_custom["input"] = json!("");
let mut initial_function = function.clone();
initial_function["arguments"] = json!("");
let fixture = Fixture::new(Session::new(vec![
created(),
wire(
"response.output_item.added",
json!({"output_index":0,"item":initial_custom}),
),
wire(
"response.output_item.added",
json!({"output_index":1,"item":initial_function}),
),
wire(
"response.custom_tool_call_input.delta",
json!({"output_index":0,"item_id":"ct_1","delta":"first\n"}),
),
wire(
"response.function_call_arguments.delta",
json!({"output_index":1,"item_id":"fc_1","delta":"{}"}),
),
wire(
"response.custom_tool_call_input.done",
json!({"output_index":0,"item_id":"ct_1","input":"first\nsecond"}),
),
completed(json!([custom, function])),
]))
.await;
let response = fixture.post(chat_request(streaming)).await;
assert_eq!(response.status(), StatusCode::OK);
if streaming {
let (_, chunks) = sse_body(response).await;
let calls = chunks
.iter()
.filter_map(|chunk| chunk["choices"][0]["delta"]["tool_calls"].as_array())
.flatten()
.collect::<Vec<_>>();
assert_eq!(
calls.iter().filter(|call| call["type"] == "custom").count(),
1
);
assert_eq!(
calls
.iter()
.filter(|call| call["type"] == "function")
.count(),
1
);
assert_eq!(
calls
.iter()
.filter_map(|call| call["custom"]["input"].as_str())
.collect::<String>(),
"first\nsecond"
);
assert_eq!(
calls
.iter()
.filter_map(|call| call["function"]["arguments"].as_str())
.collect::<String>(),
"{}"
);
assert_eq!(
chunks.last().unwrap()["choices"][0]["finish_reason"],
"tool_calls"
);
} else {
let body = json_body(response).await;
assert_eq!(
body["choices"][0]["message"]["tool_calls"],
json!([
{"id":"call_custom","type":"custom","custom":{"name":"patch","input":"first\nsecond"}},
{"id":"call_function","type":"function","function":{"name":"lookup","arguments":"{}"}}
])
);
}
}
}
#[tokio::test]
async fn image_previews_and_final_images_are_projected_once_per_distinct_snapshot() {
let image = json!({"id":"ig_one","type":"image_generation_call","status":"completed","result":"ZmluYWw=","output_format":"webp"});
for streaming in [false, true] {
let fixture = Fixture::new(Session::new(vec![
created(),
wire("response.image_generation_call.partial_image", json!({"output_index":0,"item_id":"ig_one","partial_image_b64":"cHJldmlldw==","output_format":"webp"})),
wire("response.output_item.done", json!({"output_index":0,"item":image})),
completed(json!([image, message("created")])),
])).await;
let response = fixture.post(chat_request(streaming)).await;
assert_eq!(response.status(), StatusCode::OK);
if streaming {
let (_, chunks) = sse_body(response).await;
let images = chunks
.iter()
.filter_map(|chunk| chunk["choices"][0]["delta"]["images"].as_array())
.flatten()
.collect::<Vec<_>>();
assert_eq!(images.len(), 2);
assert_eq!(images[1]["index"], 0);
assert_eq!(
images[1]["image_url"]["url"],
"data:image/webp;base64,ZmluYWw="
);
} else {
let body = json_body(response).await;
assert_eq!(
body["choices"][0]["message"]["images"][0]["image_url"]["url"],
"data:image/webp;base64,ZmluYWw="
);
}
}
}
#[tokio::test]
async fn output_logprobs_survive_json_projection_and_stream_snapshot_deduplication() {
let probabilities =
json!([{"token":"hello","logprob":-0.1,"bytes":[104,101,108,108,111],"top_logprobs":[]}]);
let mut output = message("hello");
output["content"][0]["logprobs"] = probabilities.clone();
for streaming in [false, true] {
let fixture = Fixture::new(Session::new(vec![
created(),
wire("response.output_text.delta", json!({"output_index":0,"content_index":0,"item_id":"msg_chat_test","delta":"hello","logprobs":probabilities})),
wire("response.output_text.done", json!({"output_index":0,"content_index":0,"item_id":"msg_chat_test","text":"hello","logprobs":probabilities})),
completed(json!([output])),
])).await;
let response = fixture.post(chat_request(streaming)).await;
assert_eq!(response.status(), StatusCode::OK);
if streaming {
let (_, chunks) = sse_body(response).await;
let values = chunks
.iter()
.filter_map(|chunk| chunk["choices"][0]["logprobs"]["content"].as_array())
.flatten()
.cloned()
.collect::<Vec<_>>();
assert_eq!(json!(values), probabilities);
} else {
assert_eq!(
json_body(response).await["choices"][0]["logprobs"]["content"],
probabilities
);
}
}
}
#[tokio::test]
async fn streaming_citation_offsets_follow_delivered_text_when_messages_interleave() {
let citation = json!({"type":"url_citation","start_index":2,"end_index":4,"url":"https://example.com","title":"source"});
let mut first = message("abcd");
first["content"][0]["annotations"] = json!([citation]);
let mut second = message("xy");
second["id"] = json!("msg_second");
let fixture = Fixture::new(Session::new(vec![
created(),
wire("response.output_text.delta", json!({"output_index":0,"content_index":0,"item_id":"msg_chat_test","delta":"ab"})),
wire("response.output_text.delta", json!({"output_index":1,"content_index":0,"item_id":"msg_second","delta":"xy"})),
wire("response.output_text.delta", json!({"output_index":0,"content_index":0,"item_id":"msg_chat_test","delta":"cd"})),
wire("response.output_text.annotation.added", json!({"output_index":0,"content_index":0,"item_id":"msg_chat_test","annotation_index":0,"annotation":citation})),
completed(json!([first, second])),
])).await;
let (_, chunks) = sse_body(fixture.post(chat_request(true)).await).await;
assert_eq!(combined(&chunks, "content"), "abxycd");
let annotations = chunks
.iter()
.filter_map(|chunk| chunk["choices"][0]["delta"]["annotations"].as_array())
.flatten()
.collect::<Vec<_>>();
assert_eq!(annotations.len(), 1);
assert_eq!(annotations[0]["url_citation"]["start_index"], 4);
assert_eq!(annotations[0]["url_citation"]["end_index"], 6);
}
@@ -390,7 +390,7 @@ fn no_eligible_provider_account_should_preserve_safe_provider_detail() {
);
assert_eq!(
gateway.safe_message(),
"No upstream provider is currently available for this request."
"No upstream provider is available for this request."
);
}
@@ -423,7 +423,7 @@ fn no_eligible_provider_account_should_map_to_service_unavailable() {
assert_eq!(projected.kind(), GatewayErrorKind::NoAvailableProvider);
assert_eq!(
projected.safe_message(),
"No upstream provider is currently available for this request."
"No upstream provider is available for this request."
);
}
@@ -54,6 +54,7 @@ impl UpstreamConnection {
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum UpstreamInfoFailure {
Unauthorized,
SharingDisabled,
Unsupported,
RateLimited,
InvalidResponse,
@@ -65,6 +66,7 @@ impl UpstreamInfoFailure {
pub const fn as_str(self) -> &'static str {
match self {
Self::Unauthorized => "unauthorized",
Self::SharingDisabled => "sharing_disabled",
Self::Unsupported => "unsupported",
Self::RateLimited => "rate_limited",
Self::InvalidResponse => "invalid_response",
@@ -411,23 +411,7 @@ impl DefaultExecutionService {
started_at: SystemTime,
request_id: Option<ModelRequestId>,
) -> Result<StartedExecution, GatewayError> {
crate::engine::image_generation::check_image_generation(
authorization.account_scope.allows_image_generation(),
&request.operation,
)?;
let request_id = request_id.map_or_else(new_request_id, Ok)?;
if authorization.cancellation.is_cancelled() {
return Err(GatewayError::new(
GatewayErrorKind::Cancelled,
"Parent request was cancelled.",
));
}
if authorization.deadline_at.is_elapsed() {
return Err(GatewayError::new(
GatewayErrorKind::Timeout,
"Request deadline elapsed.",
));
}
let account_group_ids = authorization
.account_scope
.routing_snapshot()
@@ -494,6 +478,22 @@ impl DefaultExecutionService {
let budget_key_id = request.client.policy.key_id().clone();
let mut entered_execution = false;
let result = async {
crate::engine::image_generation::check_image_generation(
authorization.account_scope.allows_image_generation(),
&request.operation,
)?;
if authorization.cancellation.is_cancelled() {
return Err(GatewayError::new(
GatewayErrorKind::Cancelled,
"Parent request was cancelled.",
));
}
if authorization.deadline_at.is_elapsed() {
return Err(GatewayError::new(
GatewayErrorKind::Timeout,
"Request deadline elapsed.",
));
}
if request_policy.is_some() {
start_guard = Some(self.prepare_execution_start(&request, &request_id, &mut authorization).await?);
}
@@ -680,7 +680,7 @@ impl DefaultExecutionService {
request_id = request_id.as_str(),
key_id = budget_key_id.as_str(),
failure_kind = error.kind().as_str(),
"请求在路由或准入阶段被拒绝"
"请求在执行前被拒绝"
);
let rejection = crate::engine::EntryRejection {
request_id: request_id.clone(),
+4 -5
View File
@@ -1226,10 +1226,9 @@ impl GatewayError {
GatewayErrorKind::UpstreamUnavailable,
"The upstream account cannot access the service.",
),
ProviderErrorKind::RequestPolicyDenied => Self::new(
GatewayErrorKind::PolicyDenied,
"Request policy rejected the request.",
),
ProviderErrorKind::RequestPolicyDenied => {
Self::new(GatewayErrorKind::PolicyDenied, "Request blocked by policy.")
}
ProviderErrorKind::RateLimited | ProviderErrorKind::QuotaExhausted => Self::new(
GatewayErrorKind::RateLimited,
"Upstream capacity is temporarily unavailable.",
@@ -1248,7 +1247,7 @@ impl GatewayError {
),
ProviderErrorKind::NoEligibleAccount => Self::new(
GatewayErrorKind::NoAvailableProvider,
"No upstream provider is currently available for this request.",
"No upstream provider is available for this request.",
),
ProviderErrorKind::ProviderInfrastructureUnavailable => Self::new(
GatewayErrorKind::ProviderInfrastructureUnavailable,
@@ -236,7 +236,13 @@ impl RequestSettings {
if !self.overrides.contains_key("request_profiles") && !defaults.request_profiles.is_empty()
{
let mut profiles = runtime.request_profiles().clone();
profiles.extend(defaults.request_profiles.clone());
for (provider, key_profile) in &defaults.request_profiles {
let profile = crate::settings::request_profiles::resolve_key_profile(
key_profile,
profiles.get(provider),
);
profiles.insert(provider.clone(), profile.into_owned());
}
runtime = runtime.with_request_profiles(profiles);
}
ExecutionSettings {
@@ -2,6 +2,7 @@
pub(crate) mod compiled;
pub mod privacy;
pub mod request_profiles;
mod values;
pub use values::{
SettingsValues, client_min_versions, response_body_limit, validate_request_limits,
@@ -0,0 +1,31 @@
//! Key 使用完整画像,或显式继承全局画像并附加 Provider 所有的字段
use std::borrow::Cow;
use crate::account::OpaqueProviderData;
use serde_json::{Map, Value};
/// `inherit` 是 Key 配置的外层协议;内部字段名和值只由对应 Provider 校验
#[must_use]
pub fn inherited_fields(key: &OpaqueProviderData) -> Option<&Map<String, Value>> {
let document = key.expose_to_provider();
(document.len() == 1)
.then(|| document.get("inherit")?.as_object())
.flatten()
}
/// 在请求快照上应用继承,不把全局字段复制回 Key;没有宿主配置时由 Provider 解析默认值
#[must_use]
pub fn resolve_key_profile<'a>(
key: &'a OpaqueProviderData,
global: Option<&OpaqueProviderData>,
) -> Cow<'a, OpaqueProviderData> {
if let Some(fields) = inherited_fields(key)
&& let Some(global) = global
{
let mut resolved = global.clone().into_inner();
resolved.extend(fields.clone());
return Cow::Owned(OpaqueProviderData::new(resolved));
}
Cow::Borrowed(key)
}
@@ -5316,3 +5316,39 @@ fn image_policy_stays_frozen_for_prepared_requests_and_refreshes_for_the_next_tu
assert_eq!(admissions.limits.lock().unwrap().len(), 1);
});
}
#[test]
fn cancelled_prepared_execution_records_rejection_without_admission_or_model_request() {
block_on(async {
let store = Arc::new(TrackingExecutionStore::default());
let service = DefaultExecutionService::new(
RuntimeSnapshotHandle::new(start_snapshot()),
store.clone(),
ProviderRegistry::default(),
Arc::new(UnusedAdmissions),
Arc::new(UnusedContinuation),
Arc::new(RecordingClientApiKeyUsage::default()),
Arc::new(crate::RecordingDiagnostics::default()),
);
let request = request(&service, ClientTransport::HttpJson);
let prepared = service.prepare_execution(request.client).await.unwrap();
let id = prepared.request_id().clone();
prepared.cancellation().cancel();
let result = service
.start_prepared(
prepared,
gateway_core::engine::execution::PreparedExecutionRequest {
public_model: request.public_model,
operation: request.operation,
metadata: request.metadata,
},
)
.await;
assert!(matches!(result, Err(error) if error.kind() == GatewayErrorKind::Cancelled));
let rejected = store.entry_rejections.lock().unwrap();
assert_eq!(rejected.len(), 1);
assert_eq!(rejected[0].request_id, id);
assert_eq!(rejected[0].error.kind(), GatewayErrorKind::Cancelled);
assert!(store.requests.lock().unwrap().is_empty());
});
}
@@ -40,12 +40,19 @@ fn snapshot(revision: u64, name: &str) -> Arc<RuntimeSnapshot> {
}
fn policy(id: &str) -> ClientPolicy {
policy_with_profiles(id, profiles(&[("openai", id)]))
}
fn policy_with_profiles(
id: &str,
overrides: BTreeMap<ProviderKind, OpaqueProviderData>,
) -> ClientPolicy {
ClientPolicy::new(
ClientApiKeyId::new(id).unwrap(),
PlaintextClientApiKey::new(format!("fixture-{id}")).unwrap(),
Arc::new(
FrozenAccountScope::new(Arc::default(), ClientRoutingScope::all_accounts())
.with_request_profiles(profiles(&[("openai", id)]))
.with_request_profiles(overrides)
.with_fast_mode(FastMode::Disabled),
),
true,
@@ -86,6 +93,34 @@ fn settings_facts_are_shared_and_invalid_compilation_does_not_change_them() {
assert!(std::ptr::eq(original.settings(), frozen.settings()));
}
#[test]
fn inherited_key_fields_follow_the_current_global_profile_without_changing_key_defaults() {
let provider = ProviderKind::new("openai").unwrap();
let key_profile = OpaqueProviderData::new(
json!({"inherit":{"device":"test-device"}})
.as_object()
.unwrap()
.clone(),
);
let mut key = policy_with_profiles(
"device-key",
BTreeMap::from([(provider.clone(), key_profile.clone())]),
);
for (revision, name) in [(1, "first"), (2, "second")] {
key = RequestSettings::new(snapshot(revision, name)).apply_policy(key);
assert_eq!(
key.account_scope()
.request_profile(&provider)
.unwrap()
.expose_to_provider(),
json!({"identity":name,"device":"test-device"})
.as_object()
.unwrap()
);
assert_eq!(key.defaults().request_profiles[&provider], key_profile);
}
}
#[test]
fn resolving_an_existing_policy_uses_key_defaults_and_the_current_host() {
let key = policy("key");
+1 -1
View File
@@ -53,7 +53,7 @@ pub fn load_config<T: LoadableConfig>() -> Result<T, ConfigError> {
// 此时日志尚未初始化;只报告字段路径,不回显可能包含凭据的配置值
for field in unused {
if !T::EXTERNAL_SECTIONS.contains(&field.as_str()) {
eprintln!("[警告] 配置字段 {field:?} 未使用,已忽略");
eprintln!("[警告] 已忽略未知配置字段 {field:?}");
}
}
value.resolve_and_validate(source_dir)?;
+41 -22
View File
@@ -61,12 +61,6 @@ impl PeerTransport for HttpPeerTransport {
.await
.map_err(|_| UpstreamInfoFailure::Unavailable)?;
if !response.status().is_success() {
let kind = match response.status().as_u16() {
401 | 403 => UpstreamInfoFailure::Unauthorized,
404 | 405 | 501 => UpstreamInfoFailure::Unsupported,
429 => UpstreamInfoFailure::RateLimited,
_ => UpstreamInfoFailure::Unavailable,
};
let retry_after = response
.headers()
.get(reqwest::header::RETRY_AFTER)
@@ -87,23 +81,48 @@ impl PeerTransport for HttpPeerTransport {
})
})
.map(|value| value.min(Duration::from_secs(24 * 60 * 60)));
let kind = match response.status().as_u16() {
401 => UpstreamInfoFailure::Unauthorized,
403 => forbidden_reason(response).await,
404 | 405 | 501 => UpstreamInfoFailure::Unsupported,
429 => UpstreamInfoFailure::RateLimited,
_ => UpstreamInfoFailure::Unavailable,
};
return Err(PeerFetchFailure { kind, retry_after });
}
if response
.content_length()
.is_some_and(|length| length > BODY_LIMIT as u64)
{
return Err(UpstreamInfoFailure::InvalidResponse.into());
}
let mut bytes = Vec::new();
let mut stream = response.bytes_stream();
while let Some(chunk) = stream.next().await {
let chunk = chunk.map_err(|_| UpstreamInfoFailure::Unavailable)?;
if bytes.len().saturating_add(chunk.len()) > BODY_LIMIT {
return Err(UpstreamInfoFailure::InvalidResponse.into());
}
bytes.extend_from_slice(&chunk);
}
Ok(bytes)
read_body(response).await.map_err(Into::into)
}
}
// 普通 403 只说明当前请求失权,上游明确关闭共享才清除互联身份
async fn forbidden_reason(response: reqwest::Response) -> UpstreamInfoFailure {
if let Ok(bytes) = read_body(response).await
&& let Ok(body) = serde_json::from_slice::<serde_json::Value>(&bytes)
&& body
.pointer("/error/code")
.and_then(serde_json::Value::as_str)
== Some(gateway_protocol::peer::SHARING_DISABLED_CODE)
{
return UpstreamInfoFailure::SharingDisabled;
}
UpstreamInfoFailure::Unauthorized
}
async fn read_body(response: reqwest::Response) -> Result<Vec<u8>, UpstreamInfoFailure> {
if response
.content_length()
.is_some_and(|length| length > BODY_LIMIT as u64)
{
return Err(UpstreamInfoFailure::InvalidResponse);
}
let mut bytes = Vec::new();
let mut stream = response.bytes_stream();
while let Some(chunk) = stream.next().await {
let chunk = chunk.map_err(|_| UpstreamInfoFailure::Unavailable)?;
if bytes.len().saturating_add(chunk.len()) > BODY_LIMIT {
return Err(UpstreamInfoFailure::InvalidResponse);
}
bytes.extend_from_slice(&chunk);
}
Ok(bytes)
}
@@ -173,7 +173,7 @@ impl PluginDistribution for HttpPluginDistribution {
self.download_inner(location, credentials, egress.as_ref()),
)
.await
.map_err(|_| AdminError::bad_gateway("插件下载超时,请重试"))?
.map_err(|_| AdminError::bad_gateway("插件下载超时"))?
}
}
@@ -185,7 +185,7 @@ fn credential_ids(credentials: &[SourceCredential]) -> Vec<String> {
}
fn source_error() -> AdminError {
AdminError::bad_gateway("插件来源请求失败,请检查来源与下载授权后重试")
AdminError::bad_gateway("插件来源请求失败")
}
fn validate_digest(value: &str) -> Result<(), AdminError> {
+1 -1
View File
@@ -43,7 +43,7 @@ impl PricingSource for ModelsDevPricing {
}
fn source_error() -> AdminError {
AdminError::bad_gateway("无法读取 models.dev 价目,请稍后重试;现有价格未变更")
AdminError::bad_gateway("无法读取 models.dev 价目,现有价格不变")
}
/// 只接受当前网关 Provider 的文本 Token 价格;不猜测缺失字段或模态换算
@@ -69,9 +69,7 @@ impl HttpProxyProbe {
mut self,
build: impl Fn(reqwest::ClientBuilder) -> Result<reqwest::Client, E> + Send + Sync + 'static,
) -> Self {
self.build_client = Arc::new(move |builder| {
build(builder).map_err(|_| "无法创建代理连接,请检查证书信任配置")
});
self.build_client = Arc::new(move |builder| build(builder).map_err(|_| "无法创建代理连接"));
self
}
@@ -94,7 +92,7 @@ impl HttpProxyProbe {
if error.is_timeout() {
"出口连接超时"
} else {
"出口连接失败,请检查网络或代理配置"
"出口连接失败"
}
})?;
if !response.status().is_success() {
@@ -319,7 +317,7 @@ impl HttpProxyProbe {
.query(&[("fields", "ip,success,country_code,region,city,timezone.id")])
.send()
.await
.map_err(|_| "位置查询失败,请手动重试")?;
.map_err(|_| "位置查询失败")?;
if response.status() == reqwest::StatusCode::TOO_MANY_REQUESTS {
return Err("位置查询服务限流,请稍后手动重试");
}
@@ -102,10 +102,7 @@ async fn source_failures_log_status_without_credentials_url_or_response_body() {
.err()
.unwrap();
assert_eq!(error.kind(), AdminErrorKind::BadGateway);
assert_eq!(
error.to_string(),
"插件来源请求失败,请检查来源与下载授权后重试"
);
assert_eq!(error.to_string(), "插件来源请求失败");
output.seek(SeekFrom::Start(0)).unwrap();
let mut log = String::new();
output.read_to_string(&mut log).unwrap();
@@ -2,6 +2,7 @@
use serde::{Deserialize, Serialize};
pub const SHARING_DISABLED_CODE: &str = "peer_sharing_disabled";
pub const OBJECT: &str = "cpr.peer";
pub const SCHEMA_VERSION: u32 = 3;
pub const MAX_ACCOUNTS: usize = 3_000;
@@ -498,10 +498,10 @@ fn s3_error_message<E: ProvideErrorMetadata>(error: &SdkError<E, HttpResponse>)
// 不拼入原始 message,避免泄露请求细节
let detail = error.code().unwrap_or("unknown");
match status_of(error) {
Some(401) => format!("当前凭据认证失败({detail})"),
Some(403) => format!("当前凭据没有目标前缀的读写权限({detail})"),
Some(401) => format!("对象存储认证失败({detail})"),
Some(403) => format!("对象存储拒绝访问({detail})"),
Some(404) => format!("目标存储或对象不存在({detail})"),
Some(500..=599) => format!("上游对象存储服务返回失败响应({detail})"),
Some(500..=599) => format!("对象存储服务异常({detail})"),
_ => format!("对象存储请求失败({detail})"),
}
}
@@ -88,6 +88,11 @@ impl AnalyticsSnapshot<'_> {
.account_fact_states("model_requests", range, account_ids, false, by_model)
.await;
}
let retained_counter = if by_model {
"request_count"
} else {
"completed_request_count"
};
let mut states = BTreeMap::new();
// 排序与分组费用只需要账号总量,不额外展开模型维度
let (collection, model) = if by_model {
@@ -117,7 +122,7 @@ impl AnalyticsSnapshot<'_> {
.push_bind(account_ids)
.push("::text[])");
push_summary_buckets(&mut query, &buckets, seconds);
let counters = usage_counter_names();
let counters = usage_counter_names(by_model);
// 每行一次展开指标,避免对可能位于 TOAST 的同一 JSON 反复解码
query.push(format!(
"), totals as (select account_id,model,jsonb_build_object({}) as counts,max(last_used_at) as last_used_at from selected cross join lateral jsonb_to_record(counts) as metric({}) group by account_id,model),
@@ -149,7 +154,7 @@ impl AnalyticsSnapshot<'_> {
merge_account_states(&mut states, boundary)?;
}
if !self.has_pending_requests().await? {
states.retain(|_, state| state.count("completed_request_count") > 0);
states.retain(|_, state| state.count(retained_counter) > 0);
return Ok(states);
}
let old = self
@@ -192,7 +197,10 @@ impl AnalyticsSnapshot<'_> {
.map_err(|_| invalid("invalid account cost difference"))?;
state.costs.insert(currency.clone(), value.canonical());
}
repair.insert(key);
// 失败终态没有用量极值,更新它们只需调整计数
if previous.last_used_at().is_some() {
repair.insert(key);
}
}
merge_account_states(&mut states, current)?;
// 撤销最大值不能用减法;只对发生源版本替换的账号模型做索引定位
@@ -201,12 +209,12 @@ impl AnalyticsSnapshot<'_> {
let state = states
.get_mut(&(account.clone(), model.clone()))
.ok_or_else(|| invalid("pending account state disappeared"))?;
if state.count("completed_request_count") == 0 {
if state.count(retained_counter) == 0 {
*state = MetricState::default();
continue;
}
let statement = format!(
"select started_at from model_requests mr where provider_account_ref=$1 and ($2::text is null or coalesce(upstream_model_id,requested_model_id,'')=$2) and started_at >= $3 and started_at < $4 and {completed} order by started_at desc,id desc limit 1"
"select started_at from model_requests mr where provider_account_ref=$1 and ($2::text is null or coalesce(upstream_model_id,requested_model_id,'')=$2) and started_at >= $3 and started_at < $4 and {completed} and mr.recovered_at is null order by started_at desc,id desc limit 1"
);
state.last_used_at = sqlx::query_scalar(sqlx::AssertSqlSafe(statement))
.bind(&account)
@@ -224,7 +232,7 @@ impl AnalyticsSnapshot<'_> {
.collect::<Vec<_>>();
for currency in zero_currencies {
let statement = format!(
"select exists(select 1 from model_requests mr where provider_account_ref=$1 and ($2::text is null or coalesce(upstream_model_id,requested_model_id,'')=$2) and started_at >= $3 and started_at < $4 and {completed} and cost_currency=$5 and cost_amount is not null)"
"select exists(select 1 from model_requests mr where provider_account_ref=$1 and ($2::text is null or coalesce(upstream_model_id,requested_model_id,'')=$2) and started_at >= $3 and started_at < $4 and {completed} and mr.recovered_at is null and cost_currency=$5 and cost_amount is not null)"
);
let exists: bool = sqlx::query_scalar(sqlx::AssertSqlSafe(statement))
.bind(&account)
@@ -240,7 +248,7 @@ impl AnalyticsSnapshot<'_> {
}
}
}
states.retain(|_, state| state.count("completed_request_count") > 0);
states.retain(|_, state| state.count(retained_counter) > 0);
Ok(states)
}
@@ -409,7 +417,10 @@ impl AnalyticsSnapshot<'_> {
.push_bind(range.start)
.push(" and mr.started_at < ")
.push_bind(range.end);
push_completed_usage_fact_filter(&mut query, "mr");
if !by_model {
push_completed_usage_fact_filter(&mut query, "mr");
}
push_unrecovered_request_filter(&mut query, "mr");
query.push(" group by mr.provider_account_ref,mr.cost_currency");
if by_model {
query.push(",coalesce(mr.upstream_model_id,mr.requested_model_id,'')");
@@ -465,7 +476,7 @@ fn merge_account_states(states: &mut AccountStates, incoming: AccountStates) ->
Ok(())
}
fn usage_counter_names() -> Vec<String> {
fn usage_counter_names(by_model: bool) -> Vec<String> {
let mut names = [
"completed_request_count",
"image_request_count",
@@ -477,6 +488,19 @@ fn usage_counter_names() -> Vec<String> {
]
.map(str::to_owned)
.to_vec();
if by_model {
names.extend(
[
"request_count",
"success_count",
"failure_count",
"cancelled_count",
"incomplete_count",
"caller_error_count",
]
.map(str::to_owned),
);
}
for name in super::metrics::TOKEN_METRICS {
names.push(name.to_owned());
names.push(format!("{name}_known_count"));
@@ -491,25 +515,49 @@ fn usage_fact_aggregate_sql(table: &str, by_model: bool) -> String {
"''::text"
};
let mut counters = vec![
"'completed_request_count',count(*)".to_owned(),
"'image_request_count',count(*) filter(where image_generation_succeeded is true)".to_owned(),
"'image_request_failed_count',count(*) filter(where image_generation_succeeded is false)".to_owned(),
"'window_total_tokens',coalesce(sum(coalesce(total_tokens,coalesce(input_tokens,0)+coalesce(output_tokens,0))),0)".to_owned(),
"'completed_request_count',count(*) filter(where is_completed_usage)".to_owned(),
"'image_request_count',count(*) filter(where is_completed_usage and image_generation_succeeded is true)".to_owned(),
"'image_request_failed_count',count(*) filter(where is_completed_usage and image_generation_succeeded is false)".to_owned(),
"'window_total_tokens',coalesce(sum(coalesce(total_tokens,coalesce(input_tokens,0)+coalesce(output_tokens,0))) filter(where is_completed_usage),0)".to_owned(),
];
if by_model {
counters.push("'request_count',count(*)".to_owned());
counters.push(
"'caller_error_count',count(*) filter(where client_status_code between 400 and 499)"
.to_owned(),
);
for (outcome, name) in [
("succeeded", "success"),
("failed", "failure"),
("cancelled", "cancelled"),
("incomplete", "incomplete"),
] {
counters.push(format!(
"'{name}_count',count(*) filter(where outcome='{outcome}')"
));
}
}
for name in super::metrics::TOKEN_METRICS {
counters.push(format!(
"'{name}',coalesce(sum({name}),0),'{name}_known_count',count({name})"
"'{name}',coalesce(sum({name}) filter(where is_completed_usage),0),'{name}_known_count',count({name}) filter(where is_completed_usage)"
));
}
for source in ["provider_reported", "calculated", "unavailable"] {
counters.push(format!(
"'{source}_count',count(*) filter(where cost_source='{source}')"
"'{source}_count',count(*) filter(where is_completed_usage and cost_source='{source}')"
));
}
// 排序与分组费用已在 WHERE 限定完整交付,只在模型健康查询中按指标分别筛选
let completed = if by_model {
completed_usage_fact_predicate("mr")
} else {
"true".to_owned()
};
format!("select mr.provider_account_ref as account_id,{model} as model,
jsonb_build_object({}) as counts,case when mr.cost_currency is not null and sum(mr.cost_amount) is not null
then jsonb_build_object(mr.cost_currency,sum(mr.cost_amount)::text) else '{{}}'::jsonb end as costs,
'{{}}'::jsonb as distributions,max(mr.started_at) as last_used_at from {table} mr",counters.join(","))
jsonb_build_object({}) as counts,case when mr.cost_currency is not null and sum(mr.cost_amount) filter(where is_completed_usage) is not null
then jsonb_build_object(mr.cost_currency,(sum(mr.cost_amount) filter(where is_completed_usage))::text) else '{{}}'::jsonb end as costs,
'{{}}'::jsonb as distributions,max(mr.started_at) filter(where is_completed_usage) as last_used_at
from (select mr.*,coalesce(({completed}),false) as is_completed_usage from {table} mr) mr",counters.join(","))
}
pub(crate) async fn account_usage_windows(
@@ -542,13 +590,11 @@ pub(crate) async fn account_usage_windows(
continue;
}
total.merge(state)?;
if !model.is_empty()
&& let Some(last_used_at) = state.last_used_at()
{
if !model.is_empty() {
models.push(AccountModelUsage {
model: model.clone(),
request_count: state.count("completed_request_count"),
success_count: state.count("completed_request_count"),
health: state.request_health(),
input_tokens: state.optional_count("input_tokens"),
output_tokens: state.optional_count("output_tokens"),
cached_input_tokens: state.optional_count("cached_input_tokens"),
@@ -561,7 +607,7 @@ pub(crate) async fn account_usage_windows(
image_request_failed_count: state.count("image_request_failed_count"),
cost_coverage: admin_cost_coverage(state.coverage()),
costs: account_costs(state)?,
last_used_at,
last_used_at: state.last_used_at(),
});
}
}
@@ -320,6 +320,16 @@ impl MetricState {
p99_ms: percentile(0.99)?,
})
}
pub fn request_health(&self) -> admin_observability::RequestHealth {
admin_observability::RequestHealth::from_counts(
self.count("success_count"),
self.count("failure_count"),
self.count("cancelled_count"),
self.count("incomplete_count"),
self.count("caller_error_count"),
)
}
pub fn request_summary_metrics(&self) -> admin_observability::RequestSummaryMetrics {
admin_observability::RequestSummaryMetrics {
request_count: self.count("request_count"),
@@ -174,7 +174,7 @@ pub(crate) fn admin_account_model_usage(
admin_observability::AccountModelUsage {
model: usage.model,
request_count: usage.request_count,
success_count: usage.success_count,
health: usage.health,
input_tokens: usage.input_tokens,
output_tokens: usage.output_tokens,
cached_input_tokens: usage.cached_input_tokens,
@@ -252,7 +252,7 @@ pub struct ProviderAccountUsageObservation {
pub struct ProviderAccountModelUsageObservation {
pub model: String,
pub request_count: u64,
pub success_count: u64,
pub health: admin_observability::RequestHealth,
pub input_tokens: Option<u64>,
pub output_tokens: Option<u64>,
pub cached_input_tokens: Option<u64>,
@@ -265,7 +265,7 @@ pub struct ProviderAccountModelUsageObservation {
pub total_tokens: Option<u64>,
pub cost_coverage: CostCoverage,
pub costs: Vec<CurrencyCostTotal>,
pub last_used_at: DateTime<Utc>,
pub last_used_at: Option<DateTime<Utc>>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -42,14 +42,12 @@ pub(crate) async fn provider_account_usage(
.await?
{
totals.entry(account.clone()).or_default().merge(&state)?;
if !model.is_empty()
&& let Some(last_used_at) = state.last_used_at()
{
if !model.is_empty() {
models_by_account.entry(account).or_default().push(
ProviderAccountModelUsageObservation {
model,
request_count: state.count("completed_request_count"),
success_count: state.count("completed_request_count"),
health: state.request_health(),
input_tokens: state.optional_count("input_tokens"),
output_tokens: state.optional_count("output_tokens"),
cached_input_tokens: state.optional_count("cached_input_tokens"),
@@ -62,7 +60,7 @@ pub(crate) async fn provider_account_usage(
image_request_failed_count: state.count("image_request_failed_count"),
cost_coverage: state.coverage(),
costs: state.costs()?,
last_used_at,
last_used_at: state.last_used_at(),
},
);
}
@@ -148,3 +148,162 @@ async fn quota_windows_reconcile_moved_requests_and_pending_deletions_without_wa
assert!(old.usage.costs.is_empty());
database.close().await;
}
#[tokio::test]
async fn model_success_rates_preserve_usage_across_source_summary_and_pending_reads() {
let Some(database) = TestDatabase::create("account_model_health").await else {
return;
};
PgProviderAccountRepository::new(database.pool.clone())
.insert_provider_account(account("acct_health", "user-health"))
.await
.unwrap();
let start = chrono::DateTime::from_timestamp(Utc::now().timestamp().div_euclid(3600) * 3600, 0)
.unwrap()
- TimeDelta::hours(3);
for (index, (id, model, outcome, status)) in [
("health_success", "mixed", "succeeded", 200),
("health_retry", "mixed", "succeeded", 200),
("health_failed", "mixed", "failed", 502),
("health_incomplete", "mixed", "incomplete", 200),
("health_cancelled", "mixed", "cancelled", 200),
("health_caller", "mixed", "failed", 400),
("health_recovered", "mixed", "failed", 502),
("health_only_failed", "failed-only", "failed", 502),
("health_only_cancelled", "no-sample", "cancelled", 200),
("health_only_caller", "no-sample", "failed", 400),
]
.into_iter()
.enumerate()
{
seed_model_request(
&database.pool,
ModelRequestSeed {
request_id: id,
account_id: "acct_health",
provider_kind: "openai",
model,
total_tokens: 11,
cost_amount: "2.5",
started_at: start + TimeDelta::minutes(5 + index as i64 * 5),
},
)
.await
.unwrap();
sqlx::query("update model_requests set outcome=$2,client_status_code=$3,attempt_count=case when id='health_retry' then 3 else 1 end where id=$1")
.bind(id).bind(outcome).bind(status).execute(&database.pool).await.unwrap();
}
sqlx::query("update model_requests set recovery_request_id='health_retry',recovered_at=completed_at,recovery_attempt_count=1,request_observation_json=jsonb_set(request_observation_json,'{recovery}',jsonb_build_object('retryDelayMs',0,'totalLatencyMs',1)) where id='health_recovered'")
.execute(&database.pool).await.unwrap();
let windows = [
AccountUsageWindowQuery {
account_id: "acct_health".to_owned(),
key: "whole".to_owned(),
range: TimeRange {
start,
end: start + TimeDelta::hours(2),
},
},
AccountUsageWindowQuery {
account_id: "acct_health".to_owned(),
key: "boundary".to_owned(),
range: TimeRange {
start: start + TimeDelta::minutes(3),
end: start + TimeDelta::minutes(117),
},
},
];
let store = admin_account_store(&database.pool);
let source = store.load_account_usage_by_windows(&windows).await.unwrap();
for result in &source {
let usage = &result.usage;
assert_eq!((usage.request_count, usage.total_tokens), (2, Some(22)));
assert_eq!(usage.costs[0].amount.as_str(), "5");
assert_eq!(usage.cost_coverage.provider_reported_count, 2);
let mixed = usage
.models
.iter()
.find(|model| model.model == "mixed")
.unwrap();
assert_eq!(mixed.health.success_rate(), Some(50.0));
assert_eq!(mixed.health.success_requests, 2);
assert_eq!(mixed.health.failed_requests, 1);
assert_eq!(mixed.health.incomplete_requests, 1);
let failed = usage
.models
.iter()
.find(|model| model.model == "failed-only")
.unwrap();
assert_eq!(failed.health.success_rate(), Some(0.0));
assert_eq!(failed.last_used_at, None);
assert!(failed.costs.is_empty());
let excluded = usage
.models
.iter()
.find(|model| model.model == "no-sample")
.unwrap();
assert_eq!(excluded.health.success_rate(), None);
}
drain_analytics(&database.pool).await;
assert_eq!(
store.load_account_usage_by_windows(&windows).await.unwrap(),
source
);
let rolling = store
.load_account_usage(windows[0].range, &["acct_health".to_owned()])
.await
.unwrap();
for model in &rolling[0].models {
assert_eq!(
model.health,
source[0]
.usage
.models
.iter()
.find(|expected| expected.model == model.model)
.unwrap()
.health
);
}
// 待投影的成功变失败必须撤销费用与最近使用时间,失败变成功则补入当前源版本
sqlx::query("update model_requests set outcome='failed',client_status_code=502 where id in ('health_success','health_retry')")
.execute(&database.pool).await.unwrap();
sqlx::query("update model_requests set outcome='succeeded',client_status_code=200 where id='health_only_failed'")
.execute(&database.pool).await.unwrap();
sqlx::query("delete from model_requests where id='health_failed'")
.execute(&database.pool)
.await
.unwrap();
let pending = store.load_account_usage_by_windows(&windows).await.unwrap();
for result in &pending {
assert_eq!(
(result.usage.request_count, result.usage.total_tokens),
(1, Some(11))
);
assert_eq!(result.usage.costs[0].amount.as_str(), "2.5");
let mixed = result
.usage
.models
.iter()
.find(|model| model.model == "mixed")
.unwrap();
assert_eq!(mixed.health.success_rate(), Some(0.0));
assert_eq!(mixed.health.failed_requests, 2);
assert_eq!(mixed.last_used_at, None);
assert!(mixed.costs.is_empty());
let recovered = result
.usage
.models
.iter()
.find(|model| model.model == "failed-only")
.unwrap();
assert_eq!(recovered.health.success_rate(), Some(100.0));
}
drain_analytics(&database.pool).await;
assert_eq!(
store.load_account_usage_by_windows(&windows).await.unwrap(),
pending
);
database.close().await;
}
@@ -231,11 +231,32 @@ impl ProviderAdmin for OpenAiAdminProvider {
&self,
configuration: &gateway_core::account::OpaqueProviderData,
) -> Result<gateway_core::account::OpaqueProviderData, ProviderAdminError> {
if configuration
.expose_to_provider()
.contains_key("installationId")
|| gateway_core::settings::request_profiles::inherited_fields(configuration).is_some()
{
return Err(provider_admin_error(ProviderAdminErrorKind::Invalid)
.with_public_message("安装 ID 只能在 API 密钥中设置"));
}
self.profile
.preview_selection(configuration)
.map_err(map_client_profile_error)
}
fn preview_key_client_profile(
&self,
key: &gateway_core::account::OpaqueProviderData,
global: Option<&gateway_core::account::OpaqueProviderData>,
) -> Result<gateway_core::account::OpaqueProviderData, ProviderAdminError> {
crate::transport::profile::identity::inherited_identity_fields(key)
.map_err(map_client_profile_error)?;
let profile = gateway_core::settings::request_profiles::resolve_key_profile(key, global);
self.profile
.preview_selection(&profile)
.map_err(map_client_profile_error)
}
fn provider_kind(&self) -> &ProviderKind {
&self.provider_kind
}
@@ -2077,6 +2098,7 @@ fn map_client_profile_error(
use crate::transport::profile::selection::ClientProfileError;
let message = match error {
ClientProfileError::Invalid => "客户端身份字段或版本组合不合法",
ClientProfileError::InvalidInstallationId => "安装 ID 必须是带连字符的 UUID",
ClientProfileError::InvalidUserAgent => {
"User-Agent 必须是 1 至 4096 字节的单行 ASCII 文本,且首尾不能含空白"
}
@@ -829,7 +829,7 @@ impl CodexCredentialCatalogService {
authorization.expose_secret(),
account.upstream_account_id(),
&request_id,
None,
client_version.map(|_| credential.installation_id.as_str()),
),
client_version,
)
@@ -330,6 +330,10 @@ impl CodexProvider {
));
// Standalone Provider 端点没有可证明的账号 owner;Images/Search metadata 必须按
// 跨账号输入收敛到当前 lease,不能沿用下游声明的账号或 installation identity
let client = self.client_for_request(&context)?;
let installation_id = client
.profile_state()
.effective_installation_id(lease.installation_id());
let turn_metadata = payload
.context()
.get("turn_metadata")
@@ -337,13 +341,13 @@ impl CodexProvider {
.and_then(|metadata| {
crate::transport::request::scope_turn_metadata(
metadata,
lease.installation_id(),
true,
&installation_id,
RequestAccountScope::Different,
)
});
let events = cold_json_response_stream(ColdJsonResponse {
client: self
.client_for_request(&context)?
installation_id,
client: client
.for_account(lease.account())
.map_err(|error| map_client_error(error, UpstreamSendState::NotSent, false).error)?
.with_responses_api_base_url(lease.authentication().responses_api_base_url())
@@ -384,6 +388,7 @@ struct RawJsonEndpointRequest {
pub(super) struct ColdResponse {
pub(super) client: CodexBackendClient,
pub(super) installation_id: String,
pub(super) response_origin: Url,
pub(super) request: CodexResponsesRequest,
pub(super) upstream_model: UpstreamModelId,
@@ -397,7 +402,6 @@ pub(super) struct ColdResponse {
pub(super) session_transport_key_hash: Option<String>,
pub(super) session_transport_recovery: CodexSessionTransportRecovery,
pub(super) guardian_routing: guardian::GuardianRouting,
pub(super) turn_state_scope: Option<turn_state::TurnStateScope>,
pub(super) websocket_retry_count: u32,
pub(super) stream_max_retries: u32,
pub(super) session_capture: Option<OpenAiSessionCapture>,
@@ -405,6 +409,7 @@ pub(super) struct ColdResponse {
pub(super) struct ColdJsonResponse {
pub(super) client: CodexBackendClient,
pub(super) installation_id: String,
pub(super) response_origin: Url,
pub(super) endpoint_path: &'static str,
pub(super) body: Bytes,
@@ -594,7 +599,6 @@ pub(super) fn map_handshake_attempt_error(
pub(super) async fn create_json_attempt(
request: &ColdJsonResponse,
account: &ProviderAccount,
installation_id: &str,
authorization: &SecretString,
cookie_header: Option<&SecretString>,
account_selection: CodexAccountSelectionTelemetry<'_>,
@@ -608,7 +612,7 @@ pub(super) async fn create_json_attempt(
authorization.expose_secret(),
account.upstream_account_id(),
request_id,
Some(installation_id),
Some(&request.installation_id),
);
request_context.trace = Some(&trace);
request_context.cookie_header = cookie_header.map(ExposeSecret::expose_secret);
@@ -660,7 +664,6 @@ pub(super) fn cold_json_response_stream(request: ColdJsonResponse) -> EventStrea
let response = create_json_attempt(
&request,
&active_account,
request.lease.installation_id(),
&authorization,
cookie_header.as_ref(),
account_selection,
@@ -780,6 +783,7 @@ fn image_response_metering(
pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
let ColdResponse {
client,
installation_id,
response_origin,
request,
upstream_model,
@@ -793,7 +797,6 @@ pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
session_transport_key_hash,
session_transport_recovery,
guardian_routing,
turn_state_scope,
websocket_retry_count,
stream_max_retries,
mut session_capture,
@@ -837,7 +840,7 @@ pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
&request,
&request_id,
&active_account,
lease.installation_id(),
&installation_id,
&authorization,
cookie_header.as_ref(),
account_selection,
@@ -883,17 +886,6 @@ pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
},
);
}
if let Some(scope) = &turn_state_scope
&& let Some(response) = failure.error.client_visible_upstream_response()
{
for header in response.headers() {
if header.name().eq_ignore_ascii_case("x-codex-turn-state")
&& let Ok(token) = std::str::from_utf8(header.value().as_ref())
{
scope.remember(token);
}
}
}
if let Some(observation) = failure.observation.take() {
yield ProviderEvent::observation(observation);
}
@@ -918,11 +910,6 @@ pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
Err(failure.error)?;
return;
}
if let Some(scope) = &turn_state_scope
&& let Some(token) = response.turn_state.as_deref()
{
scope.remember(token);
}
if let Some(capture) = session_capture.as_mut() {
capture.continuation_scope = Some(if capture.response_store {
OpenAiContinuationScope::Persisted
@@ -1066,7 +1053,6 @@ pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
response_metadata_updates.as_ref(),
&mut session_capture,
&mut observation_state,
turn_state_scope.as_ref(),
)
.await;
let observation_event = if rate_limits_changed || metadata_merge.is_some()
@@ -1144,7 +1130,6 @@ pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
response_metadata_updates.as_ref(),
&mut session_capture,
&mut observation_state,
turn_state_scope.as_ref(),
)
.await;
let metadata_changed = metadata_merge.unwrap_or(false);
@@ -1345,7 +1330,6 @@ pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
response_metadata_updates.as_ref(),
&mut session_capture,
&mut observation_state,
turn_state_scope.as_ref(),
)
.await
.unwrap_or(false);
@@ -1413,12 +1397,8 @@ async fn merge_response_metadata_updates(
updates: Option<&CodexResponseMetadataUpdates>,
session_capture: &mut Option<OpenAiSessionCapture>,
observation_state: &mut OpenAiResponseObservationState,
turn_state_scope: Option<&turn_state::TurnStateScope>,
) -> Option<bool> {
let turn_state = updates?.lock().await.turn_state.take()?;
if let Some(scope) = turn_state_scope {
scope.remember(&turn_state);
}
if let Some(capture) = session_capture.as_mut() {
capture.turn_state = Some(turn_state.clone());
}
@@ -1098,6 +1098,7 @@ pub(super) fn profile_diagnostic(
use crate::transport::profile::selection::ClientProfileError;
let code = match error {
ClientProfileError::Invalid => "request_profile_invalid",
ClientProfileError::InvalidInstallationId => "request_profile_installation_id_invalid",
ClientProfileError::InvalidUserAgent => "request_profile_user_agent_invalid",
ClientProfileError::CompanionHeadersRequired => "request_profile_headers_required",
ClientProfileError::CompanionHeadersConflict => "request_profile_headers_conflict",
@@ -107,7 +107,6 @@ mod failure;
mod guardian;
mod live;
mod observation;
mod turn_state;
mod upstream_adapter;
mod workers;
pub(crate) use live::{CodexLiveGateway, CodexLiveRegistry};
@@ -155,7 +154,6 @@ pub struct CodexProvider {
session_identity: Option<CodexSessionIdentity>,
session_transport_recovery: CodexSessionTransportRecovery,
guardian_routing: guardian::GuardianRouting,
turn_state_ownership: turn_state::TurnStateOwnership,
stream_max_retries: u32,
live_registry: Arc<CodexLiveRegistry>,
live_gateway: Option<Arc<CodexLiveGateway>>,
@@ -361,7 +359,6 @@ impl CodexProvider {
session_identity: None,
session_transport_recovery: CodexSessionTransportRecovery::default(),
guardian_routing: guardian::GuardianRouting::default(),
turn_state_ownership: turn_state::TurnStateOwnership::default(),
stream_max_retries,
live_registry: Arc::new(CodexLiveRegistry::default()),
live_gateway: None,
@@ -964,26 +961,11 @@ impl CodexProvider {
)
.map(|affinity| (response_id.to_owned(), affinity))
});
let turn_state_scope = self
.turn_state_ownership
.scope(&context, &lease, &upstream_request);
// 全量请求的 token 证明不能用于放行 response ID 或其他账号状态
let verified_turn_state = upstream_request
.turn_state
.as_ref()
.filter(|token| {
matches!(account_scope, RequestAccountScope::Unknown)
&& turn_state_scope
.as_ref()
.is_some_and(|scope| scope.owns(token))
})
.cloned();
scope_request_to_account(
&mut upstream_request,
lease.installation_id(),
account_scope,
verified_turn_state.as_deref(),
);
let client = self.client_for_request(&context)?;
let installation_id = client
.profile_state()
.effective_installation_id(lease.installation_id());
scope_request_to_account(&mut upstream_request, &installation_id, account_scope);
// 父响应的可信关联只恢复此字段,不把首次 Guardian 请求的其他账号状态视为可信
if let Some((response_id, _)) = &guardian_parent
&& let Some(Value::Object(metadata)) =
@@ -1005,7 +987,6 @@ impl CodexProvider {
"ownerCrossAccount": state_owner_cross_account,
"accountSwitch": lease.account_switch(),
"accountStateReusable": account_scope.can_reuse_account_state(),
"verifiedTurnStateEcho": verified_turn_state.is_some(),
"before": before,
"after": crate::transport::diagnostics::evidence::request_state(&upstream_request),
}));
@@ -1162,8 +1143,8 @@ impl CodexProvider {
AttemptTransport::Default | AttemptTransport::Fallback => 0,
};
let events = cold_response_stream(ColdResponse {
client: self
.client_for_request(&context)?
installation_id,
client: client
.for_account(lease.account())
.map_err(|error| map_client_error(error, UpstreamSendState::NotSent, false).error)?
.with_responses_api_base_url(lease.authentication().responses_api_base_url())
@@ -1185,7 +1166,6 @@ impl CodexProvider {
session_transport_key_hash,
session_transport_recovery: self.session_transport_recovery.clone(),
guardian_routing: self.guardian_routing.clone(),
turn_state_scope,
websocket_retry_count,
stream_max_retries: self.stream_max_retries,
session_capture,
@@ -1,124 +0,0 @@
//! 只为上游已返回的 turn state 保存归属证明,不恢复正文或参与账号选择
use std::{
collections::HashMap,
sync::{Arc, Mutex},
time::{Duration, Instant},
};
use gateway_core::engine::AttemptContext;
use sha2::{Digest, Sha256};
use crate::{
credential::{CodexCredentialLease, CodexRuntimeAuthentication},
transport::protocol::responses::CodexResponsesRequest,
};
// 仅保存 32 字节摘要与过期时间,4096 项的条目数据约 192 KiB,不随 token 长度增长
const MAX_PROOFS: usize = 4096;
#[derive(Clone, Default)]
pub(super) struct TurnStateOwnership {
proofs: Arc<Mutex<HashMap<[u8; 32], Instant>>>,
}
pub(super) struct TurnStateScope {
ownership: TurnStateOwnership,
identity: Sha256,
retention: Duration,
}
impl TurnStateOwnership {
pub(super) fn scope(
&self,
context: &AttemptContext,
lease: &CodexCredentialLease,
request: &CodexResponsesRequest,
) -> Option<TurnStateScope> {
let turn_id = request
.client_turn_id
.as_deref()
.filter(|id| !id.is_empty())?;
let mut identity = Sha256::new();
for part in [
context.client_api_key_ref().as_str(),
lease.account_id().as_str(),
request.client_session_id.as_deref().unwrap_or_default(),
request.client_thread_id.as_deref().unwrap_or_default(),
turn_id,
] {
identity.update(part.len().to_le_bytes());
identity.update(part.as_bytes());
}
// OAuth 刷新不改变账号归属;API Key 替换则不能继承原认证的路由状态
if matches!(
lease.authentication(),
CodexRuntimeAuthentication::ApiKey(_)
) {
identity.update([1]);
identity.update(lease.account().revision().get().to_le_bytes());
} else {
identity.update([0]);
}
Some(TurnStateScope {
ownership: self.clone(),
identity,
retention: context
.account_selection_policy()
.openai_session_affinity_ttl(),
})
}
}
impl TurnStateScope {
pub(super) fn owns(&self, token: &str) -> bool {
let key = self.key(token);
let mut proofs = self
.ownership
.proofs
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let Some(expires_at) = proofs.get_mut(&key) else {
return false;
};
let now = Instant::now();
if *expires_at <= now {
proofs.remove(&key);
return false;
}
*expires_at = now + self.retention;
true
}
pub(super) fn remember(&self, token: &str) {
if token.is_empty() {
return;
}
let key = self.key(token);
let now = Instant::now();
let mut proofs = self
.ownership
.proofs
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if proofs.len() >= MAX_PROOFS && !proofs.contains_key(&key) {
proofs.retain(|_, expires_at| *expires_at > now);
if proofs.len() >= MAX_PROOFS
&& let Some(oldest) = proofs
.iter()
.min_by_key(|(_, expires_at)| **expires_at)
.map(|(key, _)| *key)
{
proofs.remove(&oldest);
}
}
proofs.insert(key, now + self.retention);
}
fn key(&self, token: &str) -> [u8; 32] {
let mut hash = self.identity.clone();
hash.update(token.len().to_le_bytes());
hash.update(token.as_bytes());
hash.finalize().into()
}
}
@@ -490,7 +490,7 @@ pub struct CodexRequestContext<'a> {
pub parent_thread_id: Option<&'a str>,
/// cookie 头
pub cookie_header: Option<&'a str>,
/// 当前账号稳定派生的 installation ID
/// 本次请求生效的安装 ID
pub installation_id: Option<&'a str>,
/// 客户端 session ID
pub session_id: Option<&'a str>,
@@ -25,7 +25,16 @@ impl CodexBackendClient {
OpenAiUpstreamProtocol::ResponsesApi => "models",
};
let profile = self.profile.snapshot();
let headers = self.model_request_headers(&profile, context)?;
let headers = self.model_request_headers(
&profile,
CodexRequestContext {
installation_id: profile
.installation_id
.as_deref()
.or(context.installation_id),
..context
},
)?;
let mut request = self
.client
.get(endpoint_url(&self.base_url, path))
@@ -31,7 +31,15 @@ impl CodexBackendClient {
extra_protocol_headers: &[ProviderHttpHeader],
content_type: Option<&str>,
) -> CodexClientResult<HeaderMap> {
let mut headers = self.model_request_headers(&self.profile.snapshot(), context)?;
let profile = self.profile.snapshot();
let context = CodexRequestContext {
installation_id: profile
.installation_id
.as_deref()
.or(context.installation_id),
..context
};
let mut headers = self.model_request_headers(&profile, context)?;
for header in extra_protocol_headers {
let name = HeaderName::from_bytes(header.name().as_bytes())?;
if is_managed_identity_header(name.as_str())
@@ -48,7 +56,11 @@ impl CodexBackendClient {
.installation_id
.zip(std::str::from_utf8(header.value()).ok())
.and_then(|(installation_id, raw)| {
super::request::scope_turn_metadata(raw, installation_id, true)
super::request::scope_turn_metadata(
raw,
installation_id,
super::request::RequestAccountScope::Different,
)
})
else {
continue;
@@ -591,7 +591,8 @@ impl CodexBackendClient {
let mut key = CodexWebSocketPoolKey::new(&self.base_url, account_id, conversation_id)
.with_egress_key(&self.egress_key)
.with_outbound_proxy(self.outbound_proxy.as_ref())
.with_connection_profile(connection_profile);
.with_connection_profile(connection_profile)
.with_installation_id(context.installation_id);
key = key.with_routing_affinity(self.routing_affinity.as_ref().map(|pin| pin.id()));
if let Some(connection_id) = request.downstream_websocket_connection_id.as_deref() {
key = key.with_downstream_connection_id(connection_id);
@@ -28,7 +28,7 @@ pub(super) fn is_managed_identity_header(name: &str) -> bool {
| "chatgpt-project-id"
| "openai-organization"
| "openai-project"
// 安装头与 client_metadata 共用当前账号的安装身份
// 安装头与 client_metadata 共用本次请求的安装身份
| "x-codex-installation-id"
// 画像和设备证明须属于选定账号,不能由下游覆盖
| "originator"
@@ -12,7 +12,10 @@ use super::{CodexWireProfile, CodexWireProfileState};
/// 已解析的配置;没有 mode 的预设配置复用官方版本与平台合同
#[derive(Debug, Clone)]
pub struct RequestProfileSelection(ParsedSelection);
pub struct RequestProfileSelection {
selection: ParsedSelection,
installation_id: Option<String>,
}
#[derive(Debug, Clone)]
enum ParsedSelection {
@@ -32,62 +35,100 @@ enum SelectionDocument {
},
}
pub(crate) fn inherited_identity_fields(
document: &OpaqueProviderData,
) -> Result<Option<&serde_json::Map<String, Value>>, ClientProfileError> {
let inherited = gateway_core::settings::request_profiles::inherited_fields(document);
if inherited.is_some_and(|fields| fields.len() != 1 || !fields.contains_key("installationId")) {
return Err(ClientProfileError::Invalid);
}
Ok(inherited)
}
impl RequestProfileSelection {
pub fn preset(&self) -> Option<&ClientProfileSelection> {
match &self.0 {
match &self.selection {
ParsedSelection::Preset(selection) => Some(selection),
_ => None,
}
}
pub fn version_mode(&self) -> VersionMode {
match &self.0 {
match &self.selection {
ParsedSelection::Preset(selection) => selection.version_mode,
ParsedSelection::Custom(_) => VersionMode::Fixed,
}
}
pub fn parse(document: &OpaqueProviderData) -> Result<Self, ClientProfileError> {
let fields = document.expose_to_provider();
if !fields.contains_key("mode") {
return ClientProfileSelection::parse(document)
.map(ParsedSelection::Preset)
.map(Self);
}
let document = serde_json::from_value(Value::Object(fields.clone()))
.map_err(|_| ClientProfileError::Invalid)?;
let selection = match document {
SelectionDocument::Custom {
user_agent,
originator,
codex_version,
} => ParsedSelection::Custom(ExactIdentity::parse(
&user_agent,
originator.as_deref(),
codex_version.as_deref(),
)?),
let inherited = inherited_identity_fields(document)?;
let mut fields = inherited
.unwrap_or_else(|| document.expose_to_provider())
.clone();
let installation_id = match fields.remove("installationId") {
None | Some(Value::Null) => None,
Some(Value::String(value)) if value.is_empty() => None,
Some(Value::String(value)) => {
let parsed = uuid::Uuid::try_parse(&value)
.map_err(|_| ClientProfileError::InvalidInstallationId)?;
if value.len() != uuid::fmt::Hyphenated::LENGTH {
return Err(ClientProfileError::InvalidInstallationId);
}
Some(parsed.to_string())
}
Some(_) => return Err(ClientProfileError::InvalidInstallationId),
};
Ok(Self(selection))
let selection = if inherited.is_some() {
ParsedSelection::Preset(ClientProfileSelection::default())
} else if fields.contains_key("mode") {
match serde_json::from_value(Value::Object(fields))
.map_err(|_| ClientProfileError::Invalid)?
{
SelectionDocument::Custom {
user_agent,
originator,
codex_version,
} => ParsedSelection::Custom(ExactIdentity::parse(
&user_agent,
originator.as_deref(),
codex_version.as_deref(),
)?),
}
} else {
ParsedSelection::Preset(ClientProfileSelection::from_fields(fields)?)
};
Ok(Self {
selection,
installation_id,
})
}
pub fn resolve(
&self,
state: &CodexWireProfileState,
) -> Result<CodexWireProfile, ClientProfileError> {
match &self.0 {
ParsedSelection::Preset(selection) => selection.resolve(state),
ParsedSelection::Custom(identity) => Ok(identity.profile(state)),
}
let mut profile = match &self.selection {
ParsedSelection::Preset(selection) => selection.resolve(state)?,
ParsedSelection::Custom(identity) => identity.profile(state),
};
profile.installation_id.clone_from(&self.installation_id);
Ok(profile)
}
pub(super) fn preview(
&self,
state: &CodexWireProfileState,
) -> Result<OpaqueProviderData, ClientProfileError> {
match &self.0 {
ParsedSelection::Preset(selection) => state.preview_preset_selection(selection),
ParsedSelection::Custom(identity) => identity.preview(state),
}
let preview = match &self.selection {
ParsedSelection::Preset(selection) => state.preview_preset_selection(selection)?,
ParsedSelection::Custom(identity) => identity.preview(state)?,
};
let Some(id) = &self.installation_id else {
return Ok(preview);
};
let mut fields = preview.into_inner();
fields["configuration"]["installationId"] = Value::String(id.clone());
Ok(OpaqueProviderData::new(fields))
}
}
@@ -181,6 +222,7 @@ impl ExactIdentity {
.map(|value| value.terminal.clone())
.unwrap_or_default(),
exact_user_agent: Some(self.user_agent.clone()),
installation_id: None,
residency: deployment.residency,
account_routing_override: deployment.account_routing_override,
verified_at: DateTime::UNIX_EPOCH,
@@ -69,6 +69,9 @@ pub struct CodexWireProfile {
/// 入口专属或自定义的完整值;存在时不得用默认预设重新拼接
#[serde(default, skip_serializing_if = "Option::is_none")]
pub exact_user_agent: Option<String>,
/// Key 独立配置的安装身份;省略时保留当前调用的账号基线
#[serde(default, skip_serializing_if = "Option::is_none")]
pub installation_id: Option<String>,
/// 未配置时不发送 residency 头;不随制品版本更新而改变
pub residency: Option<String>,
/// 部署指定的账号路由头;未配置时不发送,不随客户端身份选择改变
@@ -90,6 +93,7 @@ impl Default for CodexWireProfile {
arch: "arm64".to_owned(),
terminal: "unknown".to_owned(),
exact_user_agent: None,
installation_id: None,
residency: None,
account_routing_override: None,
// 制品核验于 2026-09-06T03:26:12.084Z;进程启动不构成重新核验
@@ -205,6 +209,16 @@ impl CodexWireProfileState {
state
}
pub(crate) fn effective_installation_id(&self, account_installation_id: &str) -> String {
self.profile
.read()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.installation_id
.as_deref()
.unwrap_or(account_installation_id)
.to_owned()
}
/// 返回当前画像的独立快照,避免持锁执行网络请求
pub fn snapshot(&self) -> CodexWireProfile {
self.profile
@@ -116,9 +116,14 @@ impl Default for ClientProfileSelection {
impl ClientProfileSelection {
pub fn parse(document: &OpaqueProviderData) -> Result<Self, ClientProfileError> {
let selection: Self =
serde_json::from_value(Value::Object(document.expose_to_provider().clone()))
.map_err(|_| ClientProfileError::Invalid)?;
Self::from_fields(document.expose_to_provider().clone())
}
pub(super) fn from_fields(
fields: serde_json::Map<String, Value>,
) -> Result<Self, ClientProfileError> {
let selection: Self = serde_json::from_value(Value::Object(fields))
.map_err(|_| ClientProfileError::Invalid)?;
selection.validate()?;
Ok(selection)
}
@@ -272,6 +277,7 @@ impl ClientProfileSelection {
.clone()
.unwrap_or_else(|| "unknown".to_owned()),
exact_user_agent: None,
installation_id: None,
residency: deployment.residency,
account_routing_override: deployment.account_routing_override,
verified_at: release.verified_at.unwrap_or(DateTime::UNIX_EPOCH),
@@ -383,6 +389,8 @@ pub(crate) fn object(value: &impl Serialize) -> Result<OpaqueProviderData, Clien
pub enum ClientProfileError {
#[error("客户端身份字段或版本组合不合法")]
Invalid,
#[error("安装 ID 必须是带连字符的 UUID")]
InvalidInstallationId,
#[error("User-Agent 必须是 1 至 4096 字节的单行 ASCII 文本,且首尾不能含空白")]
InvalidUserAgent,
#[error("无法识别 User-Agent,请补充 originator 和有效的 Core version")]
@@ -452,7 +452,7 @@ fn normalize_conversation_anchor_text(text: &str) -> String {
rest.to_owned()
}
/// 新 turn 不得沿用上一轮的路由 token,各 wire 入口须同步清理
/// 已确认换轮或换号时,各 wire 入口同步清理原有路由 token
pub(crate) fn clear_request_turn_state(request: &mut CodexResponsesRequest) {
request.turn_state = None;
request.passthrough_headers.remove("x-codex-turn-state");
@@ -470,16 +470,17 @@ pub(crate) fn clear_request_turn_state(request: &mut CodexResponsesRequest) {
///
/// 真实 account ID 由随后构造的 `CodexRequestContext` 注入请求头;installation ID
/// 统一写入 Core 的 `client_metadata["x-codex-installation-id"]`,并替换原有兼容字段
/// 绝不接受客户端提供的 token、cookie 或账号身份
/// 绝不接受客户端提供的认证 token、cookie 或账号身份
/// `input` 是 Responses 的可回放会话正文,
/// item ID、encrypted content 和 compaction 都必须原样保留
pub(crate) fn scope_request_to_account(
request: &mut CodexResponsesRequest,
installation_id: &str,
account_scope: RequestAccountScope,
verified_turn_state: Option<&str>,
) {
// 全量请求缺少续接归属并不表示换号,客户端回传的轮次 token 不依赖本地缓存
if account_scope == RequestAccountScope::Different {
clear_request_turn_state(request);
// 明确换号时同步清理各 wire 副本,避免把原账号的追踪与合规上下文带给新账号
for name in CROSS_ACCOUNT_CLIENT_HEADERS {
request.passthrough_headers.remove(*name);
@@ -490,27 +491,17 @@ pub(crate) fn scope_request_to_account(
}
}
let reset_account_state = !account_scope.can_reuse_account_state();
let client_metadata_turn_state = metadata_string(request, "x-codex-turn-state");
let turn_state = if reset_account_state {
verified_turn_state.map(ToOwned::to_owned)
} else {
request.turn_state.clone()
};
let client_metadata_turn_state = client_metadata_turn_state
.filter(|token| !reset_account_state || Some(token.as_str()) == verified_turn_state);
let turn_metadata = request
.turn_metadata
.as_deref()
.and_then(|metadata| scope_turn_metadata(metadata, installation_id, reset_account_state));
.and_then(|metadata| scope_turn_metadata(metadata, installation_id, account_scope));
let client_metadata_turn_metadata = metadata_string(request, "x-codex-turn-metadata")
.and_then(|metadata| scope_turn_metadata(&metadata, installation_id, reset_account_state));
.and_then(|metadata| scope_turn_metadata(&metadata, installation_id, account_scope));
if reset_account_state {
request.passthrough_headers.remove("x-codex-turn-state");
request.passthrough_headers.remove("x-codex-turn-metadata");
for key in CROSS_ACCOUNT_IDENTITY_KEYS
.iter()
.chain(TURN_STATE_KEYS)
.chain(ACCOUNT_BOUND_STATE_KEYS)
{
request.body_mut().remove(*key);
@@ -520,7 +511,7 @@ pub(crate) fn scope_request_to_account(
for (name, value) in &mut request.passthrough_headers {
if name == "x-codex-turn-metadata"
&& let Ok(raw) = value.to_str()
&& let Some(scoped) = scope_turn_metadata(raw, installation_id, false)
&& let Some(scoped) = scope_turn_metadata(raw, installation_id, account_scope)
&& let Ok(scoped) = HeaderValue::from_str(&scoped)
{
*value = scoped;
@@ -536,7 +527,7 @@ pub(crate) fn scope_request_to_account(
.body()
.get(*key)
.and_then(Value::as_str)
.and_then(|value| scope_turn_metadata(value, installation_id, reset_account_state));
.and_then(|value| scope_turn_metadata(value, installation_id, account_scope));
replace_existing_body_string(request, key, scoped.as_deref());
}
@@ -551,14 +542,13 @@ pub(crate) fn scope_request_to_account(
(
key,
metadata.get(key).and_then(Value::as_str).and_then(|value| {
scope_turn_metadata(value, installation_id, reset_account_state)
scope_turn_metadata(value, installation_id, account_scope)
}),
)
});
if reset_account_state {
for key in CROSS_ACCOUNT_IDENTITY_KEYS
.iter()
.chain(TURN_STATE_KEYS)
.chain(ACCOUNT_BOUND_STATE_KEYS)
{
metadata.remove(*key);
@@ -568,7 +558,7 @@ pub(crate) fn scope_request_to_account(
metadata.remove("parent_response_id");
}
// 官方 Core 在 client_metadata 使用带 x-codex 前缀的键,
// turn metadata 内仍使用 installation_id;两处均取当前账号的安装身份
// turn metadata 内仍使用 installation_id;两处均取本次请求生效的安装身份
metadata.insert(
"x-codex-installation-id".to_owned(),
Value::String(installation_id.to_owned()),
@@ -579,11 +569,6 @@ pub(crate) fn scope_request_to_account(
Some(installation_id),
);
replace_existing_metadata_field(&mut metadata, "installationId", Some(installation_id));
replace_metadata_field(
&mut metadata,
"x-codex-turn-state",
client_metadata_turn_state.as_deref(),
);
replace_metadata_field(
&mut metadata,
"x-codex-turn-metadata",
@@ -598,7 +583,6 @@ pub(crate) fn scope_request_to_account(
};
request.set_client_metadata(scoped);
request.turn_state = turn_state;
request.turn_metadata = turn_metadata;
}
@@ -658,16 +642,21 @@ pub(crate) fn serialize_ascii_turn_metadata(value: &Value) -> Option<String> {
pub(crate) fn scope_turn_metadata(
raw: &str,
installation_id: &str,
cross_account: bool,
account_scope: RequestAccountScope,
) -> Option<String> {
let reset_account_state = !account_scope.can_reuse_account_state();
let Ok(Value::Object(mut metadata)) = serde_json::from_str::<Value>(raw) else {
return (!cross_account).then(|| raw.to_owned());
return (!reset_account_state).then(|| raw.to_owned());
};
let mut changed = false;
if cross_account {
if account_scope == RequestAccountScope::Different {
for key in TURN_STATE_KEYS {
changed |= metadata.remove(*key).is_some();
}
}
if reset_account_state {
for key in CROSS_ACCOUNT_IDENTITY_KEYS
.iter()
.chain(TURN_STATE_KEYS)
.chain(ACCOUNT_BOUND_STATE_KEYS)
.chain(TURN_METADATA_KEYS)
{
@@ -675,7 +664,7 @@ pub(crate) fn scope_turn_metadata(
}
}
if raw.is_ascii()
&& !cross_account
&& !reset_account_state
&& !changed
&& !INSTALLATION_ID_KEYS
.iter()
@@ -24,6 +24,7 @@ pub struct CodexWebSocketPoolKey {
account_id: String,
conversation_id: String,
connection_profile: String,
installation_id: Option<String>,
routing_affinity_id: Option<Uuid>,
pub(super) downstream_connection_id: String,
egress_key: String,
@@ -42,6 +43,7 @@ impl CodexWebSocketPoolKey {
account_id: account_id.into(),
conversation_id: conversation_id.into(),
connection_profile: String::new(),
installation_id: None,
routing_affinity_id: None,
downstream_connection_id: String::new(),
egress_key: String::new(),
@@ -49,6 +51,12 @@ impl CodexWebSocketPoolKey {
}
}
/// 安装身份参与逻辑连接归属,精确续写也不能跨安装身份复用旧握手
pub(crate) fn with_installation_id(mut self, id: Option<&str>) -> Self {
self.installation_id = id.map(str::to_owned);
self
}
/// 区分实际 WebSocket opening 画像,防止复用旧 UA 或不同握手语义的连接
pub(crate) fn with_connection_profile(mut self, connection_profile: impl Into<String>) -> Self {
self.connection_profile = connection_profile.into();
@@ -100,6 +108,7 @@ impl CodexWebSocketPoolKey {
self.account_id.as_str(),
self.conversation_id.as_str(),
self.connection_profile.as_str(),
self.installation_id.as_deref().unwrap_or_default(),
self.downstream_connection_id.as_str(),
self.egress_key.as_str(),
routing_affinity.as_str(),
@@ -109,6 +118,7 @@ impl CodexWebSocketPoolKey {
pub(super) fn same_logical_connection(&self, other: &Self) -> bool {
self.base_url == other.base_url
&& self.account_id == other.account_id
&& self.installation_id == other.installation_id
&& self.conversation_id == other.conversation_id
&& self.egress_key == other.egress_key
&& self.routing_affinity_id == other.routing_affinity_id
@@ -168,6 +168,67 @@ async fn custom_identity_preview_and_dashboard_share_the_exact_request_profile()
);
}
#[tokio::test]
async fn installation_id_is_optional_and_validated_only_for_key_profiles() {
let bundle = provider_openai::initialize(
valid_config().config,
provider_ports(),
Arc::new(gateway_core::account::UpstreamInfoReader::default()),
)
.await
.unwrap();
let provider = bundle.admin_provider();
let id = "3B11BC82-C666-417D-B9FF-2A34FD040385";
for (value, valid) in [
(json!(id), true),
(json!(null), true),
(json!(""), true),
(json!(id.replace('-', "")), false),
(json!("bad"), false),
(json!(42), false),
] {
let configuration = OpaqueProviderData::new(
json!({
"mode": "custom", "userAgent": "codex-tui/1.0.0", "installationId": value
})
.as_object()
.unwrap()
.clone(),
);
let preview = provider.preview_key_client_profile(&configuration, None);
assert_eq!(preview.is_ok(), valid);
if value == id {
assert_eq!(
preview.unwrap().expose_to_provider()["configuration"]["installationId"],
id.to_lowercase()
);
assert!(provider.preview_client_profile(&configuration).is_err());
}
}
let doc =
|value: serde_json::Value| OpaqueProviderData::new(value.as_object().unwrap().clone());
let global = doc(json!({"mode": "custom", "userAgent": "codex-tui/2.0.0"}));
let key = doc(json!({"inherit":{"installationId":id}}));
let preview = provider
.preview_key_client_profile(&key, Some(&global))
.unwrap()
.into_inner();
assert_eq!(preview["userAgent"], "codex-tui/2.0.0");
assert_eq!(
preview["configuration"]["installationId"],
id.to_lowercase()
);
assert!(provider.preview_client_profile(&key).is_err());
assert!(
provider
.preview_key_client_profile(
&doc(json!({"inherit":{"userAgent":"other"}})),
Some(&global)
)
.is_err()
);
}
#[tokio::test]
async fn openai_bundle_exposes_one_core_provider_and_drains_worker_contributions_once() {
let config = valid_config();
@@ -663,7 +663,35 @@ async fn transient_failure_preserves_observation_and_revocation_erases_it() {
cycle(&worker).await;
let revoked = quota(&bundle, &id).await;
assert!(revoked.windows.is_empty());
assert!(revoked.upstream.unwrap().observed_at.is_none());
let info = revoked.upstream.unwrap();
assert!(info.observed_at.is_none());
assert!(info.discovered);
// 同为 403,仅共享关闭的协议码能改变识别结果
server.reset().await;
Mock::given(method("GET"))
.respond_with(ResponseTemplate::new(403).set_body_json(serde_json::json!({
"error": { "code": "peer_sharing_disabled" }
})))
.mount(&server)
.await;
bundle.admin_provider().forget(&id);
cycle(&worker).await;
let disabled = quota(&bundle, &id).await;
let info = disabled.upstream.unwrap();
assert!(!info.discovered);
assert_eq!(info.last_error.as_deref(), Some("sharing_disabled"));
assert!(disabled.windows.is_empty());
server.reset().await;
Mock::given(method("GET"))
.respond_with(ResponseTemplate::new(200).set_body_json(peer_document()))
.mount(&server)
.await;
bundle.admin_provider().forget(&id);
let recovered = bundle.admin_provider().refresh_quota(&id).await.unwrap();
assert!(recovered.upstream.unwrap().discovered);
assert!(!recovered.windows.is_empty());
}
async fn wait_for_request(server: &MockServer) {
@@ -68,6 +68,7 @@ fn wire_profile() -> CodexWireProfileState {
arch: "x86_64".to_owned(),
terminal: "catalog-contract".to_owned(),
exact_user_agent: None,
installation_id: None,
residency: None,
account_routing_override: None,
verified_at: Utc
@@ -225,6 +225,7 @@ fn selector_with_runtime(
arch: "x86_64".to_owned(),
terminal: "selector-contract".to_owned(),
exact_user_agent: None,
installation_id: None,
residency: None,
account_routing_override: None,
verified_at: chrono::Utc::now(),
@@ -25,6 +25,7 @@ fn wire_profile() -> CodexWireProfileState {
arch: "x86_64".to_owned(),
terminal: "profile-statistics-contract".to_owned(),
exact_user_agent: None,
installation_id: None,
residency: None,
account_routing_override: None,
verified_at: Utc
@@ -44,6 +44,7 @@ fn wire_profile() -> CodexWireProfileState {
arch: "x86_64".to_owned(),
terminal: "quota-contract".to_owned(),
exact_user_agent: None,
installation_id: None,
residency: None,
account_routing_override: None,
verified_at: Utc
@@ -1113,6 +1113,7 @@ fn wire_profile() -> CodexWireProfileState {
arch: "x86_64".to_owned(),
terminal: "provider-contract".to_owned(),
exact_user_agent: None,
installation_id: None,
residency: None,
account_routing_override: None,
verified_at: Utc::now(),
@@ -1724,6 +1725,15 @@ fn context_with_state_owner_and_location(
request_id: &str,
owner_account_id: &str,
location: Option<gateway_core::account::RequestLocation>,
) -> AttemptContext {
context_with_state_owner_and_profile(request_id, owner_account_id, location, None)
}
fn context_with_state_owner_and_profile(
request_id: &str,
owner_account_id: &str,
location: Option<gateway_core::account::RequestLocation>,
profile: Option<OpaqueProviderData>,
) -> AttemptContext {
let owner = ProviderAccountStateOwner::new(
ProviderKind::new("openai").expect("provider"),
@@ -1734,7 +1744,8 @@ fn context_with_state_owner_and_location(
ModelRequestId::new(request_id).expect("request id"),
ClientApiKeyId::new("key_openai_contract").expect("client key id"),
)
.with_request_location(location),
.with_request_location(location)
.with_request_profile(profile),
NonZeroU32::new(1).expect("attempt"),
SystemTime::now() + Duration::from_secs(30),
account_policy(),
@@ -6022,6 +6033,82 @@ async fn opaque_provider_options_do_not_change_openai_account_selection() {
assert_eq!(error.send_state(), UpstreamSendState::NotSent);
}
const KEY_INSTALLATION_ID: &str = "3b11bc82-c666-417d-b9ff-2a34fd040385";
#[tokio::test]
async fn key_installation_override_scopes_http_headers_metadata_and_cache_without_changing_account()
{
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_scope_same").await;
let server = MockServer::start().await;
Mock::given(method("POST"))
.and(path("/codex/responses"))
.respond_with(
ResponseTemplate::new(200)
.insert_header("content-type", "text/event-stream")
.set_body_string(CAPTURE_COMPLETED_SSE),
)
.expect(3)
.mount(&server)
.await;
let provider = provider_with_base_url(&store, server.uri());
for installation_id in [None, Some(KEY_INSTALLATION_ID), None] {
let configuration = OpaqueProviderData::new(json!({
"client": "cli", "platform": "linux", "versionMode": "fixed", "codexVersion":"1.0.0",
"installationId": installation_id
}).as_object().unwrap().clone());
let profile = provider.resolve_request_profile(&configuration).unwrap();
let payload = ProtocolPayload::json_object(
"openai",
json!({
"model":"gpt-5.4","input":"hello","prompt_cache_key":"same-cache-key"
})
.as_object()
.unwrap()
.clone(),
)
.unwrap()
.with_context(Map::from_iter([("use_websocket".to_owned(), json!(false))]));
let mut stream = Arc::clone(&provider)
.execute(
planned_request(
"openai",
Operation::Generate(GenerateRequest::from_protocol_payload(payload)),
),
context_with_state_owner_and_profile(
"req_device",
"acct_scope_same",
Some(global_request_location()),
Some(profile),
),
)
.await
.unwrap();
while let Some(event) = stream.next().await {
event.unwrap();
}
}
let requests = server.received_requests().await.unwrap();
let bodies = requests
.iter()
.map(captured_request_body)
.collect::<Vec<_>>();
for (request, body) in requests.iter().zip(&bodies) {
assert_eq!(
request.headers["x-codex-installation-id"].to_str().unwrap(),
body["client_metadata"]["x-codex-installation-id"]
.as_str()
.unwrap()
);
}
let id = |i: usize| &bodies[i]["client_metadata"]["x-codex-installation-id"];
assert_ne!(id(0), KEY_INSTALLATION_ID);
assert_eq!(id(1), KEY_INSTALLATION_ID);
assert_eq!(id(0), id(2));
assert_ne!(bodies[0]["prompt_cache_key"], bodies[1]["prompt_cache_key"]);
assert_eq!(bodies[0]["prompt_cache_key"], bodies[2]["prompt_cache_key"]);
}
#[tokio::test]
async fn http_account_scoping_replaces_downstream_installation_header() {
for owner in ["acct_scope_same", "acct_scope_old"] {
@@ -6144,13 +6231,23 @@ async fn websocket_account_scoping_replaces_downstream_installation_header() {
]),
),
]));
let mut stream = provider_with_base_url(&store, base_url)
let provider = provider_with_base_url(&store, base_url);
let configuration = OpaqueProviderData::new(json!({
"mode": "custom", "userAgent": "codex-tui/1.0.0", "installationId": KEY_INSTALLATION_ID
}).as_object().unwrap().clone());
let profile = provider.resolve_request_profile(&configuration).unwrap();
let mut stream = provider
.execute(
planned_request(
"openai",
Operation::Generate(GenerateRequest::from_protocol_payload(payload)),
),
context_with_state_owner("req_installation_ws", owner),
context_with_state_owner_and_profile(
"req_installation_ws",
owner,
Some(global_request_location()),
Some(profile),
),
)
.await
.expect("provider stream");
@@ -6161,8 +6258,7 @@ async fn websocket_account_scoping_replaces_downstream_installation_header() {
let installation_id = body["client_metadata"]["x-codex-installation-id"]
.as_str()
.expect("account installation ID");
assert_ne!(installation_id, "client-installation");
assert!(uuid::Uuid::parse_str(installation_id).is_ok());
assert_eq!(installation_id, KEY_INSTALLATION_ID);
assert_eq!(body["installation_id"], installation_id);
assert_eq!(header["x-codex-installation-id"], installation_id);
for encoded in [
@@ -6632,7 +6728,7 @@ async fn missing_affinity_continuation_should_require_client_replay_without_an_u
}
#[tokio::test]
async fn missing_affinity_full_request_should_clear_unowned_turn_state() {
async fn missing_affinity_full_request_should_preserve_client_turn_state() {
let trace = gateway_core::diagnostics::TraceContext::new("req_unknown_turn_state");
let input = json!((0..520).map(|index| json!({"role":"user", "content":format!("{index}:{}", "private-transcript".repeat(60))})).collect::<Vec<_>>());
let store = Arc::new(MemoryAccountStore::default());
@@ -6695,7 +6791,10 @@ async fn missing_affinity_full_request_should_clear_unowned_turn_state() {
.await
.expect("captured full replay request");
assert!(captured_header_values(&requests[0], "x-codex-turn-state").is_empty());
assert_eq!(
captured_header_values(&requests[0], "x-codex-turn-state"),
vec![b"unowned-turn-state".to_vec()]
);
assert_eq!(captured_request_body(&requests[0])["input"], input);
let snapshot = trace.snapshot().unwrap();
let events = snapshot["events"].as_array().unwrap();
@@ -6711,7 +6810,7 @@ async fn missing_affinity_full_request_should_clear_unowned_turn_state() {
let scope = evidence("provider.request.account_scope");
assert_eq!(scope["scope"], "unknown");
assert!(scope["before"]["turnState"].is_object());
assert!(scope["after"]["turnState"].is_null());
assert_eq!(scope["before"]["turnState"], scope["after"]["turnState"]);
for private in [
"private-transcript",
"private-instructions",
@@ -7398,7 +7497,7 @@ async fn queued_children_follow_root_migration_without_interrupting_the_running_
),
("session_id".to_owned(), json!("parent-session")),
("thread_id".to_owned(), json!(thread_id)),
("turnState".to_owned(), json!("old-account-turn-state")),
("turnState".to_owned(), json!("client-turn-state")),
]),
)
.expect("payload")
@@ -7502,10 +7601,16 @@ async fn queued_children_follow_root_migration_without_interrupting_the_running_
);
drop(resumed_child);
for thread in ["child-one", "cold-child"] {
// 已知原账号的子请求清理旧 token;冷启动子请求不能仅凭根绑定历史推断归属
let request_context = if thread == "child-one" {
context_with_state_owner("req_child_busy", "acct_subagent_a")
} else {
context("req_child_busy", CancellationToken::new())
};
let mut child = Arc::clone(&provider)
.execute(
planned_request("openai", operation(thread)),
context("req_child_busy", CancellationToken::new()),
request_context,
)
.await
.expect("child busy fallback");
@@ -7560,10 +7665,16 @@ async fn queued_children_follow_root_migration_without_interrupting_the_running_
assert_eq!(affinity.binding_count(), 1);
let requests = server.received_requests().await.expect("child requests");
assert_eq!(requests.len(), 2);
for request in requests {
assert!(captured_header_values(&request, "x-codex-turn-state").is_empty());
for (request, expected_turn_state) in requests
.iter()
.zip([Vec::<Vec<u8>>::new(), vec![b"client-turn-state".to_vec()]])
{
assert_eq!(
captured_request_body(&request).get("input"),
captured_header_values(request, "x-codex-turn-state"),
expected_turn_state
);
assert_eq!(
captured_request_body(request).get("input"),
Some(&json!([{"role":"user", "content":"keep the complete transcript"}]))
);
assert_eq!(
@@ -1,7 +1,6 @@
//! 验证同轮全量请求回传的 token 归属、失败路径与账号状态隔离
//! 验证客户端轮次 token 的全量回传、失败重试与账号状态隔离
use super::*;
use gateway_core::engine::probe::AccountProbeTarget;
const TOKEN: &str = "upstream-turn-state";
@@ -126,15 +125,8 @@ async fn same_turn_full_http_request_preserves_observed_token_without_restoring_
}
#[tokio::test]
async fn verified_header_echo_does_not_authorize_a_different_metadata_token() {
async fn full_http_request_preserves_client_header_and_metadata_tokens() {
let (_store, server, provider) = http_fixture().await;
collect_response(
&provider,
full_request(full_body(), None, false),
context("req_turn_first", CancellationToken::new()),
)
.await
.unwrap();
let mut body = full_body();
body["client_metadata"]["x-codex-turn-state"] = json!("unobserved-token");
collect_response(
@@ -145,16 +137,15 @@ async fn verified_header_echo_does_not_authorize_a_different_metadata_token() {
.await
.unwrap();
let requests = server.received_requests().await.unwrap();
assert_eq!(requests[1].headers["x-codex-turn-state"], TOKEN);
assert!(
captured_request_body(&requests[1])["client_metadata"]
.get("x-codex-turn-state")
.is_none()
assert_eq!(requests[0].headers["x-codex-turn-state"], TOKEN);
assert_eq!(
captured_request_body(&requests[0])["client_metadata"]["x-codex-turn-state"],
"unobserved-token"
);
}
#[tokio::test]
async fn turn_state_from_failed_http_response_is_owned_before_a_terminal_success() {
async fn turn_state_from_failed_http_response_survives_client_retry() {
let (_store, server, provider) = http_fixture().await;
Mock::given(method("POST"))
.and(path("/codex/responses"))
@@ -186,7 +177,7 @@ async fn turn_state_from_failed_http_response_is_owned_before_a_terminal_success
}
#[tokio::test]
async fn turn_state_proof_does_not_authorize_an_unowned_previous_response_id() {
async fn client_turn_state_does_not_authorize_an_unowned_previous_response_id() {
let (_store, server, provider) = http_fixture().await;
collect_response(
&provider,
@@ -214,21 +205,10 @@ async fn turn_state_proof_does_not_authorize_an_unowned_previous_response_id() {
}
#[tokio::test]
async fn full_request_turn_state_requires_matching_client_account_thread_turn_and_token() {
for case in [
"new_turn",
"missing_turn",
"new_thread",
"new_session",
"new_key",
"new_account",
"unknown_token",
"conflicting_owner",
"expired",
"new_provider",
] {
async fn full_http_replay_preserves_turn_state_without_local_proof() {
for case in ["new_provider", "expired_affinity", "missing_turn"] {
let (store, server, provider) = http_fixture().await;
let first_context = if case == "expired" {
let first_context = if case == "expired_affinity" {
AttemptContext::new(
RequestAttemptContext::new(
ModelRequestId::new("req_turn_first").unwrap(),
@@ -242,8 +222,6 @@ async fn full_request_turn_state_requires_matching_client_account_thread_turn_an
None,
CancellationToken::new(),
)
} else if case == "new_account" {
diagnostic_context("req_turn_first", "acct_session_affinity")
} else {
context("req_turn_first", CancellationToken::new())
};
@@ -254,83 +232,101 @@ async fn full_request_turn_state_requires_matching_client_account_thread_turn_an
)
.await
.unwrap();
let mut body = full_body();
let mut token = TOKEN;
let mut echo_context = context("req_turn_echo", CancellationToken::new());
let provider = if case == "new_provider" {
provider_with_base_url(&store, server.uri())
} else {
provider
};
match case {
"new_turn" => body["client_metadata"]["turn_id"] = json!("another-turn"),
"missing_turn" => {
body["client_metadata"]
.as_object_mut()
.unwrap()
.remove("turn_id");
}
"new_thread" => body["client_metadata"]["thread_id"] = json!("another-thread"),
"new_session" => {
body["client_metadata"]["session_id"] = json!("another-session");
echo_context = diagnostic_context("req_turn_echo", "acct_session_affinity");
}
"unknown_token" => token = "unobserved-token",
"new_account" => {
create_account(&store, "acct_scope_new").await;
echo_context = diagnostic_context("req_turn_echo", "acct_scope_new");
}
"conflicting_owner" => {
echo_context = context_with_state_owner("req_turn_echo", "acct_scope_old")
}
"new_key" => {
echo_context = AttemptContext::new(
RequestAttemptContext::new(
ModelRequestId::new("req_turn_echo").unwrap(),
ClientApiKeyId::new("another-key").unwrap(),
),
NonZeroU32::new(1).unwrap(),
SystemTime::now() + Duration::from_secs(30),
account_policy(),
AccountAttemptContext::diagnostic(
BTreeSet::new(),
AccountProbeTarget::ConnectionTest(
ProviderAccountId::new("acct_session_affinity").unwrap(),
),
None,
),
None,
CancellationToken::new(),
);
}
_ => {}
let mut body = full_body();
if case == "missing_turn" {
body["client_metadata"]
.as_object_mut()
.unwrap()
.remove("turn_id");
}
for key in ["turnState", "turn_state", "x-codex-turn-state"] {
body[key] = json!(TOKEN);
body["client_metadata"][key] = json!(TOKEN);
}
body["client_metadata"]["x-codex-turn-state"] = json!(token);
collect_response(
&provider,
full_request(body, Some(token), false),
echo_context,
full_request(body, Some(TOKEN), false),
context("req_turn_echo", CancellationToken::new()),
)
.await
.unwrap_or_else(|error| panic!("{case}: {error:?}"));
let requests = server.received_requests().await.unwrap();
assert_eq!(requests.len(), 2, "{case}");
assert!(
!requests[1].headers.contains_key("x-codex-turn-state"),
"{case}"
);
assert!(
captured_request_body(&requests[1])["client_metadata"]
.get("x-codex-turn-state")
.is_none(),
"{case}"
);
assert_eq!(requests[1].headers["x-codex-turn-state"], TOKEN, "{case}");
let body = captured_request_body(&requests[1]);
let metadata: Value = serde_json::from_str(
requests[1].headers["x-codex-turn-metadata"]
.to_str()
.unwrap(),
)
.unwrap();
for key in ["turnState", "turn_state", "x-codex-turn-state"] {
assert_eq!(body[key], TOKEN, "{case}: {key}");
assert_eq!(body["client_metadata"][key], TOKEN, "{case}: {key}");
assert_eq!(metadata[key], TOKEN, "{case}: {key}");
}
}
}
#[tokio::test]
async fn conflicting_account_owner_clears_client_turn_state() {
let (_store, server, provider) = http_fixture().await;
let mut body = full_body();
for key in ["turnState", "turn_state", "x-codex-turn-state"] {
body[key] = json!(TOKEN);
body["client_metadata"][key] = json!(TOKEN);
}
collect_response(
&provider,
full_request(body, Some(TOKEN), false),
context_with_state_owner("req_turn_switch", "acct_scope_old"),
)
.await
.unwrap();
let requests = server.received_requests().await.unwrap();
assert!(!requests[0].headers.contains_key("x-codex-turn-state"));
let body = captured_request_body(&requests[0]);
let metadata: Value = serde_json::from_str(
requests[0].headers["x-codex-turn-metadata"]
.to_str()
.unwrap(),
)
.unwrap();
for key in ["turnState", "turn_state", "x-codex-turn-state"] {
assert!(body.get(key).is_none(), "{key}");
assert!(body["client_metadata"].get(key).is_none(), "{key}");
assert!(metadata.get(key).is_none(), "{key}");
}
}
#[tokio::test]
async fn full_http_request_does_not_restore_a_token_omitted_by_the_client() {
let (_store, server, provider) = http_fixture().await;
for request_id in ["req_turn_first", "req_turn_omitted"] {
collect_response(
&provider,
full_request(full_body(), None, false),
context(request_id, CancellationToken::new()),
)
.await
.unwrap();
}
let requests = server.received_requests().await.unwrap();
assert!(!requests[1].headers.contains_key("x-codex-turn-state"));
assert!(
captured_request_body(&requests[1])["client_metadata"]
.get("x-codex-turn-state")
.is_none()
);
}
#[tokio::test]
async fn websocket_metadata_turn_state_survives_full_replay_after_completion_or_failure() {
for fail_first in [false, true] {
for (fail_first, new_provider) in [(false, false), (true, false), (false, true), (true, true)] {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_session_affinity").await;
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
@@ -349,11 +345,7 @@ async fn websocket_metadata_turn_state_survives_full_replay_after_completion_or_
))
.await
.unwrap();
if fail_first {
websocket.close(None).await.unwrap();
let (stream, _) = listener.accept().await.unwrap();
websocket = accept_codex_test_websocket(stream).await;
} else {
if !fail_first {
websocket
.send(Message::Text(
CAPTURE_COMPLETED_SSE
@@ -366,6 +358,9 @@ async fn websocket_metadata_turn_state_survives_full_replay_after_completion_or_
.await
.unwrap();
}
websocket.close(None).await.unwrap();
let (stream, _) = listener.accept().await.unwrap();
websocket = accept_codex_test_websocket(stream).await;
let echo: Value = loop {
match websocket.next().await.unwrap().unwrap() {
Message::Text(text) => break serde_json::from_str(&text).unwrap(),
@@ -390,7 +385,7 @@ async fn websocket_metadata_turn_state_survives_full_replay_after_completion_or_
.unwrap();
echo
});
let provider = provider_with_base_url(&store, base_url);
let provider = provider_with_base_url(&store, base_url.clone());
let first = collect_response(
&provider,
full_request(full_body(), None, true),
@@ -398,9 +393,16 @@ async fn websocket_metadata_turn_state_survives_full_replay_after_completion_or_
)
.await;
assert_eq!(first.is_err(), fail_first);
let provider = if new_provider {
provider_with_base_url(&store, base_url)
} else {
provider
};
let mut body = full_body();
body["client_metadata"]["x-codex-turn-state"] = json!(TOKEN);
collect_response(
&provider,
full_request(full_body(), Some(TOKEN), true),
full_request(body, None, true),
context("req_turn_echo", CancellationToken::new()),
)
.await
@@ -411,7 +413,7 @@ async fn websocket_metadata_turn_state_survives_full_replay_after_completion_or_
.unwrap();
assert_eq!(
echo["client_metadata"]["x-codex-turn-state"], TOKEN,
"failed first response: {fail_first}"
"failed first response: {fail_first}, new provider: {new_provider}"
);
assert!(echo.get("previous_response_id").is_none());
}
@@ -412,6 +412,7 @@ fn profile() -> CodexWireProfileState {
arch: "x86_64".to_owned(),
terminal: "xterm".to_owned(),
exact_user_agent: None,
installation_id: None,
residency: None,
account_routing_override: None,
verified_at: Utc
@@ -138,6 +138,7 @@ pub(crate) fn test_wire_profile() -> CodexWireProfileState {
arch: "x86_64".to_owned(),
terminal: "transport-test".to_owned(),
exact_user_agent: None,
installation_id: None,
residency: None,
account_routing_override: None,
verified_at: Utc::now(),
@@ -152,6 +152,7 @@ fn wire_profile_should_generate_bundled_core_app_server_user_agent() {
arch: "arm64".to_owned(),
terminal: "unknown".to_owned(),
exact_user_agent: None,
installation_id: None,
residency: None,
account_routing_override: None,
verified_at: Utc
@@ -501,6 +502,7 @@ fn wire_profile() -> CodexWireProfile {
arch: "arm64".to_owned(),
terminal: "unknown".to_owned(),
exact_user_agent: None,
installation_id: None,
residency: None,
account_routing_override: None,
verified_at: Utc
@@ -31,6 +31,7 @@ fn client(base_url: &str) -> CodexBackendClient {
arch: "arm64".to_owned(),
terminal: "xterm-256color".to_owned(),
exact_user_agent: None,
installation_id: None,
residency: None,
account_routing_override: None,
verified_at: Utc
@@ -898,6 +898,7 @@ fn profile() -> CodexWireProfileState {
arch: "x86_64".to_owned(),
terminal: "xterm".to_owned(),
exact_user_agent: None,
installation_id: None,
residency: None,
account_routing_override: None,
verified_at: Utc
@@ -309,6 +309,16 @@ async fn exact_continuation_reuses_owning_socket_after_connection_profile_update
let mut continuation = first_request;
continuation.set_previous_response_id(Some("resp_connection_local".to_owned()));
continuation.previous_response_scope = Some(PreviousResponseScope::ConnectionLocal);
let changed_installation = backend
.create_response(
&continuation,
CodexRequestContext {
installation_id: Some("3b11bc82-c666-417d-b9ff-2a34fd040385"),
..request_context("req_other_installation", Some("chatgpt-account"))
},
)
.await;
assert!(changed_installation.is_err(), "不同安装身份不能续用原握手");
let second = backend
.create_response(
&continuation,
+16 -5
View File
@@ -407,7 +407,7 @@ PostgreSQL/Redis 启动密码。日常校验使用 `config --quiet`
在下游导入或编辑 OpenAI API Key 账号,填写上游 API 地址和该 Key,后台自动发现 CPR 互联能力。
地址包含自定义前缀时保留完整前缀,后台在该地址下请求 `cpr/link`。
列表与详情显示已保存的观测。展开账号可查看上游密钥限制,普通账号并发留空时跟随上游有效的正数上限。手动刷新等待上游查询完成,旧观测和查询失败分别提示
列表与详情显示已保存的观测。展开账号可查看上游密钥限制,普通账号并发留空时跟随上游有效的正数上限。启用的 OpenAI API Key 账号可点击「刷新额度」重新识别。上游明确关闭共享或不支持互联时,账号显示为 API,重新识别成功后显示 CPR
第三方网关通过插件实现相同协议也可被识别,不按软件名称判定兼容。
上游明确返回 Key 预算耗尽时,下游暂停该账号;后续同步确认允许使用后自动恢复,普通限流冷却独立保留。
@@ -430,6 +430,13 @@ PostgreSQL/Redis 启动密码。日常校验使用 `config --quiet`
“修改未添加项”只带回明确未添加的条目;连接测试失败的代理已经保存,可在列表中重新测试。
请求中断或结果无法确认时暂停后续提交,添加结果待确认的条目可能已保存,应先刷新列表核对,不能直接重复添加
创建或编辑账号时,在“出站隧道”选择直连或已保存的代理,点击右侧的“测试出口”图标。
耗时显示在对应选项右侧,悬浮可查看 IPv4 / IPv6,测试失败通过通知提示。
禁用代理的账号使用服务器直连出口。出口测试不保存账号设置,也不验证账号权限或额度;账号可用性使用账号连接测试
在代理列表点击关联账号数量,再点击“分配账号”,搜索并跨页勾选账号后点击右上角的“分配”,一次最多 1000 个。
已使用此代理的账号不可重复选择;所选账号原有代理会被替换,凭据、分组与调度设置保留。禁用代理须先启用
### 插件命令行
使用与网关相同的工作目录和 `deploy/config.yaml`,读取制品已接受且当前已启用的插件实例:
@@ -752,7 +759,7 @@ OAuth 恢复开关为 `host.logging.oauth_recovery`,默认关闭,与普通
2. 收集响应中的 `x-gateway-request-id`、`x-request-id` / `x-oai-request-id`;配置了
`api.request_id_header` 时也记录该入口头。WebSocket 合成错误的关联头位于本条错误的 `headers`。
在管理端错误列表按 ID 和时间搜索;检查平台条件并主动刷新,翻页不会推进查询时间。
只有入口 ID 时,改用入口日志和时间定位
只有入口 ID 时,在运维错误中按该 ID 搜索,并结合入口日志和时间定位
3. 打开错误详情,核对上游/客户端状态、发送状态、attempt、失败阶段及后续恢复关联。
下载诊断包作为反馈材料,先检查 `availability.gaps`、`exportTruncated` 以及尝试和时间线的完整性。
该包包含经服务端脱敏的已存上游错误与时间线,分享前仍应检查业务内容及内部信息
@@ -763,11 +770,15 @@ OAuth 恢复开关为 `host.logging.oauth_recovery`,默认关闭,与普通
时,普通日志才按级别输出到 stdout;专用 dump/OAuth 恢复通道不会混入
5. 查不到请求记录时,先用模型执行 ID 在“全部平台”下搜索:已进入执行会话的建流前失败也会记录,
但其 attempt 为零,尚未确认的 Provider/账号/传输为空,不会命中具体平台或账号筛选。
再检查观测队列丢弃/写入失败告警及 `file_logging` 健康状态。鉴权、解析、路由和准入等入口拒绝
仍不保证进入错误列表;应结合入口状态与日志,不能据此认定请求未发生。异步投影有延迟,
鉴权、解析、未知路由、方法错误和执行前中间件失败记录为 `api/http_ingress` 运维事件,
路由、权限、取消和准入拒绝由执行层记录;它们没有模型请求或用量,不应按 Provider/账号筛选。
再检查观测队列丢弃/写入失败告警及 `file_logging` 健康状态。异步投影有延迟,
刷新后仍需核对缺口,而不是反复重放可能已发送的请求
6. 仅在上述信息不足且能够控制访问范围时临时开启 `host.logging.request_dump` 复现一次。
它会记录完整凭据与用户正文,应限定访问、摘取最小片段并人工脱敏;复现后关闭开关,
`ingress.request`、`ingress.request.headers` 与 `ingress.request.body` 按入口 ID 关联原始请求,
`ingress.response` 与对应 headers/body 记录响应;`ingress.body.finished` 的 `complete: false`
表示正文未读完或读取失败,不能当作完整转储。网关不会为日志读取尚未消费的正文。
转储含凭据与用户正文,应限定访问、摘取最小片段并人工脱敏;复现后关闭开关,
按配置的保留窗口及组织的数据处理要求管理已生成文件。不要为普通请求排查开启 OAuth 恢复记录,
也不要直接上传整个日志目录或完整转储
+3 -2
View File
@@ -79,9 +79,10 @@ api:
request_id_header: 'x-request-id'
openai:
# 数据驻留参考值:us;填写值原样发送,省略或 null 时删除对应请求头
# 以下两项原样发送配置值;省略或设为 null 时删除对应请求头
# 数据驻留参考值:us
residency: us
# 账号路由参考值:us、us_cr;填写值原样发送,省略或 null 时删除对应请求头
# 账号路由参考值:us、us_cr
account_routing_override: null
# 可选:完整的 Responses 上游 WS 池配置;省略整段使用以下默认值。
# ws_pool:
+2 -2
View File
@@ -286,7 +286,7 @@ main() {
ADMIN_PASSWORD="$(openssl rand -hex 16)"
fi
validate_admin_password "$ADMIN_PASSWORD"
log "生成 PostgreSQL、Redis 和管理员密码"
log "写入密码配置"
patch_config "$postgres_password" "$redis_password" "$ADMIN_PASSWORD"
fi
@@ -344,7 +344,7 @@ main() {
if [[ "$existing_config" -eq 0 ]]; then
printf '管理员密码: %s\n' "$ADMIN_PASSWORD"
else
printf '管理员密码: 保留现有配置(未修改)\n'
printf '管理员密码: 未更改\n'
fi
printf '安装目录: %s\n' "$INSTALL_DIR"
printf '配置文件: %s\n' "$CONFIG_FILE"

Some files were not shown because too many files have changed in this diff Show More