Compare commits

...
34 Commits
Author SHA1 Message Date
ZyyandCodex 9039cef3d8 chore(release): v3.41.0
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 12:18:54 +08:00
a9a77f5f9d fix(accounts): calculate model success rates from request outcomes (#611)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 12:14:27 +08:00
ZyyandCodex 6f7d92594e fix(frontend): defer incomplete configuration errors until submit
Avoid validating model access and plugin forms when switching modes.
Keep error protection drafts quiet until input or submission, and retain
condition identity when editing or removing entries.

Co-authored-by: Codex <noreply@openai.com>
2026-11-05 12:11:52 +08:00
6859520969 fix(api): broaden Chat compatibility and capture ingress failures (#610)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 11:53:21 +08:00
ZyyandCodex 21d84af2d8 fix: simplify user-facing messages and error feedback
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 00:48:19 +08:00
ZyyandCodex 311e4d228f docs(contributing): prioritize concise text in development and feedback
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 00:47:45 +08:00
ZyyandCodex d604adfbeb fix(login): center the sign-in form on mobile
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 00:47:45 +08:00
ZyyandCodex 46b4665c5b feat(openai): allow API keys to override the installation ID
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 00:31:37 +08:00
ZyyandCodex 1daa0623a0 fix(openai): preserve client turn state without local ownership records
Preserve client-supplied turn state on full requests with unknown ownership
and remove the process-local proof cache. Keep confirmed account-switch
cleanup and existing continuation authorization boundaries.

Cover HTTP and WebSocket retries, provider recreation, expired affinity,
and conflicting account ownership; update the API and architecture docs.

Co-authored-by: Codex <noreply@openai.com>
2026-11-05 00:14:53 +08:00
ZyyandCodex 860fb39622 fix(frontend): satisfy account formatting rules
Co-authored-by: Codex <noreply@openai.com>
2026-11-04 22:49:25 +08:00
f5c5a6e5b7 feat(proxies): test account egress and assign accounts in bulk (#607)
* feat(proxies): test account egress and assign accounts in bulk

Co-authored-by: Codex <noreply@openai.com>

* fix(accounts): refine proxy controls and quota display

Keep egress results with their proxy options and refresh stale proxy revisions before retrying. Compact account assignment and settings controls, and hide missing quota refresh timestamps.

Co-authored-by: Codex <noreply@openai.com>

* fix(proxies): reset account selection without closing dialog

Co-authored-by: Codex <noreply@openai.com>

* fix(proxies): clarify assignment reset and unavailable accounts

Use a red reset button and disabled checkboxes for accounts already using the target proxy.

Co-authored-by: Codex <noreply@openai.com>

* feat(proxies): show account details in assignment dialog

Add platform, subscription, and group columns, remove inline provider icons, and use the existing subtle danger style for reset.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-11-04 22:42:16 +08:00
ZyyandCodex bdfc201795 fix(peer-link): update account identity on definitive sharing results
Clear the peer badge when the upstream explicitly disables sharing or does not support CPR Link. Preserve identity on transient failures and allow manual rediscovery from the quota panel.

Co-authored-by: Codex <noreply@openai.com>
2026-11-04 19:58:55 +08:00
ZyyandCodex 789609dd53 chore(release): v3.40.0
Co-authored-by: Codex <noreply@openai.com>
2026-11-04 19:36:41 +08:00
10e25ce25f feat(api): extend Chat Completions compatibility with Responses (#606)
Map tool-result names, inline PDF inputs, and hosted web search to Responses. Project URL citations into Chat messages and preserve the requested output token limit in the OpenAI encoder.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-04 19:32:42 +08:00
6b362a5029 fix(accounts): keep failed recovery probes from thawing accounts (#605)
Carry the freeze recovery target into provider account selection so probe responses cannot update credentials or clear the active protection. Let the recovery worker postpone or release the same freeze after observing the upstream result.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-04 19:30:22 +08:00
13374e2a70 fix(api): separate traffic readiness from worker health (#604)
Expose /readyz for the runtime snapshot, PostgreSQL, and Redis. Keep worker and logging diagnostics on /healthz so a maintenance failure can alert without removing a serving instance from service discovery.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-04 19:29:50 +08:00
ZyyandCodex 19ef484969 feat(peer-link)!: share upstream key concurrency and RPM limits
Use a fresh upstream key concurrency limit as the account default for scheduling, leases, and capacity reporting. Show disclosed limits alongside the upstream pool summary.

BREAKING CHANGE: Both peers must support CPR Link schema v3.

Refs: #587

Co-authored-by: Codex <noreply@openai.com>
2026-11-04 18:32:42 +08:00
242ffb2324 feat(accounts): add a separate model allowlist cleanup button (#600)
* feat(accounts): add manual allowlist cleanup

Co-authored-by: Codex <noreply@openai.com>

* fix(accounts): refine model allowlist cleanup interactions

Co-authored-by: Codex <noreply@openai.com>

* chore(docs): remove PR screenshot assets

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-11-04 17:15:04 +08:00
ZyyandCodex 53a09bbfe9 docs(skills): require English pull request titles
Co-authored-by: Codex <noreply@openai.com>
2026-11-04 17:03:08 +08:00
239def9a5b fix(store): preserve client key ledgers after deletion (#602)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-04 17:00:44 +08:00
ZyyandCodex 5eaca14a64 fix(openai): clear client headers when switching accounts
Co-authored-by: Codex <noreply@openai.com>
2026-11-04 15:28:48 +08:00
78e943f406 feat(usage): add snapshot scans and complete grouped statistics (#599)
Expose bounded usage readers through Admin, Store, Runtime, and SDK. Preserve snapshot consistency, coverage metadata, cancellation, and parent-call cleanup.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-04 15:22:44 +08:00
ZyyandCodex 0663df14df fix(responses): reconnect idle WebSockets for replay regardless of client identity
Refs #596

Co-authored-by: Codex <noreply@openai.com>
2026-11-04 14:14:29 +08:00
39aa20a13a fix(plugins): prefer account-scoped profiles for adapted models (#588)
* fix(plugins): prefer account-scoped model profiles in catalogs

Co-authored-by: Codex <noreply@openai.com>

* fix(models): honor scoped plugin profiles in fallback catalogs

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-11-04 13:20:08 +08:00
Ke WangandCodex 621b1faceb fix(store): keep usage queries compatible with PostgreSQL 16 (#597)
Co-authored-by: Codex <noreply@openai.com>
2026-11-04 13:17:09 +08:00
28eb4708b5 fix(update): keep read-only official plugin files during extract (#595)
* fix(update): keep read-only official plugin files during extract

Docker Desktop returns EPERM when chmod runs again on an official plugin file that tar already unpacked as mode 0444.

Fixes #594

* test(update): cover writable official plugin archives

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: le-czs <243511553+le-czs@users.noreply.github.com>
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-04 13:09:57 +08:00
Ke Wang aa003f7d90 fix(build): keep frozen migrations in LF (#589) 2026-11-04 13:02:04 +08:00
ZyyandCodex 4c57a0405d fix(xai): preserve quota snapshots and restore daily usage
Preserve account profiles and quota observations during OAuth refresh and retry deferral. Project rolling daily usage independently of upstream snapshots and expose its period through the existing account usage API.

Keep quota capacity forecasts scoped to weekly and monthly windows. Cover refresh persistence, daily statistics, API projection, and PostgreSQL CAS behavior.

Co-authored-by: Codex <noreply@openai.com>
2026-11-04 12:39:10 +08:00
ZyyandCodex 49cee2397f fix(openai): avoid repeated replay errors after large SSE requests
Keep oversized Codex threads on SSE until a full request falls below 13.5 MiB. After delivering a response that requires full replay, close an idle Codex socket so the next step can send its history directly.

Co-authored-by: Codex <noreply@openai.com>
2026-11-04 12:17:26 +08:00
ZyyandCodex 1915806bd1 perf(theme): load preview content after the canvas
Co-authored-by: Codex <noreply@openai.com>
2026-11-03 21:18:02 +08:00
ZyyandCodex 5100f6333e fix(usage): improve error diagnostics and request inspection
Co-authored-by: Codex <noreply@openai.com>
2026-11-03 21:18:02 +08:00
ZyyandCodex 187af36306 fix(accounts): preserve known peer identity after sync failures
Keep undiscovered upstreams displayed as ordinary API keys and retain known peer errors without revoked shared data. Reuse quota refresh for retries and reset recognition when the connection changes.

Co-authored-by: Codex <noreply@openai.com>
2026-11-03 19:33:24 +08:00
ZyyandCodex be35f8a6ee fix(accounts): refine upstream quota display and layout
Show quota availability consistently for unlimited and stale observations, keep pool summaries at the bottom, and compact plan counts with an inline overflow popover.

Co-authored-by: Codex <noreply@openai.com>
2026-11-03 17:48:47 +08:00
ZyyandCodex 317c05c74a fix(accounts): show OAuth results in notifications
Co-authored-by: Codex <noreply@openai.com>
2026-11-03 15:55:56 +08:00
335 changed files with 10020 additions and 2211 deletions
+8 -1
View File
@@ -5,6 +5,13 @@ description: Codex Proxy RS 开发、修复、排障、代码审查和文档修
# 开发指南
## 首要写作要求:言简意赅
所有面向人的文字都按此要求编写和审查,包括界面、通知、后端错误、任务状态、命令行、日志、注释、文档和交付说明
- 先写读者需要知道的事实或动作。删掉一句不影响理解、判断或操作,就删;标题、正文和通知不重复同一信息
- 保留会改变决定的条件、权限、后果和恢复步骤。错误通知只说已知问题,有明确办法才补一句;后端返回的提示在后端改,前端不重复解释。上游原文、用户内容和诊断事实保持原样
## 先确定领域与必读规范
按用户意图、受影响行为与对象初判,定位后按实际文件复核,不要求用户说出“前端”“后端”或技能名。**所有命中行共同构成本次必读规范,先读完再确定方案、修改或给出审查结论**;跨领域叠加,不能只选最像当前步骤的一行
@@ -39,7 +46,7 @@ description: Codex Proxy RS 开发、修复、排障、代码审查和文档修
2. 以最终差异重新匹配上表,补齐新增领域的规范并检查实际实现,再处理本次引入的职责越界、重复规则、冗余状态与无依据分支;删除或替换功能时一并清理失效的配置、类型、调用、测试与文档,已部署数据和公开合同仍遵守所属迁移及兼容规则;仅审查时报告,不自行修复
3. 按最终行为执行[文档检查](references/documentation.md),核对所属正文及相关概述、示例和引用,在本任务内完成必要同步;不以尚无文档差异跳过检查。内部重构等未改变文档所述事实时无需改文档;常规文档只写当前状态,普通开发不自动修改 `release/notes.md`
4. 按变更范围执行[验证](../../../CONTRIBUTING.md#验证)及其停止条件,后端命令见[验证命令](../../../docs/architecture.md#验证命令);沿用本轮有效证据与用户已确定的验收分工,区分通过、失败、跳过与未执行;提交时补读[提交约定](../../../CONTRIBUTING.md#项目约定)
5. 交付实际结果与验证缺口;前端按[前端流程](references/frontend.md)核对最终差异和运行证据,后端按上表检查生命周期,不把构建通过当作行为验收
5. 交付前按[首要写作要求](#首要写作要求言简意赅)通读本次新增或修改的文字。交代实际结果与验证缺口;前端按[前端流程](references/frontend.md)核对运行证据,后端按上表检查生命周期,不把构建通过当作行为验收
## 测试维护
+2
View File
@@ -15,6 +15,8 @@ description: 准备、创建、更新或审查 GitHub PR,含回复讨论与处
本仓库目标分支为 `main`,其他仓库使用其可信基线规则与模板,PR 内修改的规则不能降低审查要求
为本仓库拟写或更新 PR 标题时,完整标题使用英文,沿用提交信息的 Conventional Commits 格式,例如 `fix(store): preserve client key ledgers after deletion`
涉及上游行为的判断按[问题与方案依据](../../../CONTRIBUTING.md#问题与方案依据)执行
## 按模式读取
+3
View File
@@ -0,0 +1,3 @@
# SQLx migration checksums are byte-sensitive; keep migration inputs portable across hosts.
backend/migrations/*.sql text eol=lf
backend/migrations/.frozen-sha256 text eol=lf
@@ -139,7 +139,6 @@ body:
id: expected-behavior
attributes:
label: 预期行为
description: 说明原本期望得到的结果。
validations:
required: true
@@ -108,7 +108,6 @@ body:
id: expected-behavior
attributes:
label: 预期行为
description: 说明原本期望得到的结果。
validations:
required: true
+1 -1
View File
@@ -91,7 +91,7 @@ API Key 持有者可在登录页切换身份,查看自己的用量与额度,
1. 在「账号」中添加账号,完成授权或导入
2. 按需建立账号分组,再创建客户端密钥并选择可用分组,**不选分组表示可使用全部账号**
3. 打开密钥的「使用密钥」,复制客户端配置
3. [连接客户端](#客户端接入)
## 客户端接入
@@ -146,6 +146,10 @@ async fn execute(
providers.admin,
);
or_shutdown!(plugin_runtime, services.register_usage_data(&usage_data));
or_shutdown!(
plugin_runtime,
plugin_runtime.bind_usage_queries(&usage_data)
);
let services = std::sync::Arc::new(services);
or_shutdown!(plugin_runtime, plugin_runtime.bind_services(&services));
let commands = or_shutdown!(plugin_runtime, plugin_runtime.prepare_command_line().await);
@@ -160,6 +160,10 @@ pub async fn run() -> Result<(), BootstrapError> {
plugin_runtime,
plugin_runtime.bind_services(&admin.services().public_services())
);
or_shutdown!(
plugin_runtime,
plugin_runtime.bind_usage_queries(&admin.services().usage_data())
);
let mut plan = store.take_worker_contributions();
plan.push(analytics_worker);
+1 -1
View File
@@ -91,7 +91,7 @@ fn runtime() -> Result<tokio::runtime::Runtime, io::Error> {
fn reject_extra_arguments(mut arguments: impl Iterator) -> Result<(), io::Error> {
if arguments.next().is_some() {
return Err(invalid_cli("unexpected extra command arguments"));
return Err(invalid_cli("too many arguments"));
}
Ok(())
}
+1 -1
View File
@@ -15,7 +15,7 @@ cargo +1.99.1 install --locked --path backend/apps/plugin-cli --root /path/to/lo
## 打包
先构建目标平台的插件二进制;插件提供管理页面时,还需构建页面静态资源。打包命令传入作者清单、二进制和所需的资源目录映射,通过 `cpr-plugin package --help` 查看完整参数:
先构建目标平台的二进制;有管理页面时,也构建静态资源。全部参数见 `cpr-plugin package --help`
```bash
cpr-plugin package \
@@ -197,6 +197,7 @@ async fn initialize_inner(
auth,
accounts: accounts.clone(),
account_groups: Arc::new(DefaultAccountGroupService::new(
upstream_info.clone(),
store.account_groups(),
store.account_runtime(),
snapshot.clone(),
@@ -7,6 +7,7 @@ use std::time::{Duration, SystemTime};
use chrono::{DateTime, Utc};
use futures::StreamExt as _;
use gateway_core::account::{AccountErrorProtection, ProviderAccountId};
use gateway_core::engine::probe::AccountProbeTarget;
use gateway_core::routing::UpstreamModelId;
use gateway_core::task::{
ScheduledTask, WorkerContribution, WorkerCycleContext, WorkerId, WorkerKind,
@@ -161,7 +162,7 @@ impl FreezeRecoveryTask {
let probe_succeeded = match self
.deps
.accounts
.test_connection(account.clone(), model)
.test_connection(AccountProbeTarget::FreezeRecovery(account.clone()), model)
.await
{
Ok(events) => drain_probe(events).await,
+2 -2
View File
@@ -288,8 +288,8 @@ impl AdminServices {
}
#[must_use]
pub fn usage_data(&self) -> &dyn UsageDataService {
self.usage_data.as_ref()
pub fn usage_data(&self) -> Arc<dyn UsageDataService> {
self.usage_data.clone()
}
#[must_use]
@@ -73,6 +73,7 @@ pub struct AccountGroupUsage {
/// 当前页账号组 membership 对应的持久账号事实;运行态在 Admin query service 合并
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct AccountGroupMemberFact {
pub inherited_version: Option<gateway_core::account::QuotaObservationVersion>,
pub group_id: AccountGroupId,
pub account_id: String,
pub status: AccountStatusFacts,
@@ -219,7 +219,7 @@ pub struct AccountPageItem {
pub struct AccountCapacity {
/// 实时租约读取失败时为 `None`,不能当作空闲
pub used_slots: Option<u64>,
/// 应用账号覆盖或全局默认值后的上限;`None` 表示不限
/// 应用账号覆盖或继承默认值后的普通并发上限;`None` 表示不限
pub total_slots: Option<u64>,
}
@@ -472,6 +472,80 @@ impl CostCoverage {
}
}
/// 请求健康计数与服务成功率,调用方错误和取消不计入有效样本
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct RequestHealth {
pub success_requests: u64,
pub failed_requests: u64,
pub cancelled_requests: u64,
pub incomplete_requests: u64,
pub caller_error_requests: u64,
}
impl RequestHealth {
#[must_use]
pub fn from_counts(
success: u64,
failed: u64,
cancelled: u64,
incomplete: u64,
caller_error: u64,
) -> Self {
Self {
success_requests: success,
failed_requests: failed.saturating_sub(caller_error),
cancelled_requests: cancelled,
incomplete_requests: incomplete,
caller_error_requests: caller_error,
}
}
pub(crate) fn from_metrics(metrics: &RequestMetrics) -> Self {
Self::from_counts(
metrics.success_count,
metrics.failure_count,
metrics.cancelled_count,
metrics.incomplete_count,
metrics.caller_error_count,
)
}
#[must_use]
pub fn success_rate(self) -> Option<f64> {
let total = self.eligible_requests();
(total > 0).then(|| self.success_requests as f64 / total as f64 * 100.0)
}
// 未完成保留交付终态,但上游失败、超时和进程中断都不能视为服务成功
pub(crate) fn non_success_requests(self) -> u64 {
self.failed_requests
.saturating_add(self.incomplete_requests)
}
pub(crate) fn eligible_requests(self) -> u64 {
self.success_requests
.saturating_add(self.non_success_requests())
}
pub(crate) fn add_metrics(&mut self, metrics: &RequestMetrics) {
self.add_window(Self::from_metrics(metrics));
}
pub(crate) fn add_window(&mut self, other: Self) {
self.success_requests = self.success_requests.saturating_add(other.success_requests);
self.failed_requests = self.failed_requests.saturating_add(other.failed_requests);
self.cancelled_requests = self
.cancelled_requests
.saturating_add(other.cancelled_requests);
self.incomplete_requests = self
.incomplete_requests
.saturating_add(other.incomplete_requests);
self.caller_error_requests = self
.caller_error_requests
.saturating_add(other.caller_error_requests);
}
}
/// 请求级聚合指标
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct RequestMetrics {
@@ -622,7 +696,7 @@ pub struct AccountPoolMetrics {
pub struct AccountModelUsage {
pub model: String,
pub request_count: u64,
pub success_count: u64,
pub health: RequestHealth,
pub input_tokens: Option<u64>,
pub output_tokens: Option<u64>,
pub cached_input_tokens: Option<u64>,
@@ -635,7 +709,7 @@ pub struct AccountModelUsage {
pub total_tokens: Option<u64>,
pub cost_coverage: CostCoverage,
pub costs: Vec<CurrencyCost>,
pub last_used_at: DateTime<Utc>,
pub last_used_at: Option<DateTime<Utc>>,
}
/// 账号在单小时窗口内的请求数
@@ -684,8 +758,10 @@ pub struct DashboardAccountUsage {
}
/// Dashboard 当前账号池的可重建运行时槽位事实
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DashboardRuntimeSlots {
/// 仅未设置本地覆盖的账号可能继承有效互联观测
pub inherited_versions: Vec<gateway_core::account::QuotaObservationVersion>,
/// 继承运行参数默认并发上限的可调度账号数
pub inherited_accounts: u64,
/// 已配置账号级并发上限的可调度账号槽位总数
@@ -724,6 +800,11 @@ pub struct UsageListRecord {
pub outcome: RequestOutcome,
pub error_code: Option<String>,
pub error_message: Option<String>,
pub error_kind: Option<String>,
pub websocket_close_code: Option<u16>,
pub continuation_unavailable_reason: Option<String>,
pub upstream_connection_exit_reason: Option<String>,
pub recovered: bool,
pub client_status_code: Option<u16>,
pub client_api_key_name: Option<String>,
pub id: String,
@@ -782,6 +863,23 @@ pub struct UsageListRecord {
pub started_at: DateTime<Utc>,
}
/// 受控请求详情中的上游错误字段,不参与错误分类或普通日志输出
#[derive(Clone, PartialEq)]
pub struct UsageUpstreamError {
pub code: Option<String>,
pub error_type: Option<String>,
pub message: Option<String>,
pub param: Option<String>,
pub close_reason: Option<String>,
pub last_event_type: Option<String>,
}
impl std::fmt::Debug for UsageUpstreamError {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str("UsageUpstreamError(<restricted>)")
}
}
/// 一次完整模型请求的公共观测记录
#[derive(Debug, Clone, PartialEq)]
pub struct UsageRecord {
@@ -820,6 +918,11 @@ pub struct UsageRecord {
pub error_kind: Option<String>,
pub provider_error_code: Option<String>,
pub error_message: Option<String>,
pub upstream_error: Option<UsageUpstreamError>,
pub websocket_close_code: Option<u16>,
pub continuation_unavailable_reason: Option<String>,
pub upstream_connection_exit_reason: Option<String>,
pub recovered: bool,
pub retry_after_ms: Option<u64>,
pub input_tokens: Option<u64>,
pub output_tokens: Option<u64>,
@@ -17,6 +17,7 @@ pub struct PeerLinkLimits {
pub struct PeerLinkExport {
pub observed_at: DateTime<Utc>,
pub expires_at: DateTime<Utc>,
pub key_limits: gateway_core::policy::RateLimits,
pub budget: ClientBudgetStatus,
pub budget_access: ClientBudgetAccess,
pub budget_observed_at: DateTime<Utc>,
@@ -806,11 +806,12 @@ pub struct ProviderQuotaWindow {
pub provider_data: Option<ProviderDocument>,
}
/// 账号用量统计周期,按周优先、月次之选择;短期限流窗口不参与统计面板
/// 账号用量统计周期,完整周/月额度优先,随后使用 Provider 提供的本地滚动日统计
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub enum AccountUsagePeriod {
Weekly,
Monthly,
Daily,
}
/// Provider 解释 quota 所需的公共请求事实
@@ -998,16 +999,27 @@ impl ProviderQuota {
}
}
/// 选择账号用量统计的周/月窗口,同时返回展示所需的周期事实
/// 选择账号用量统计窗口,同时返回展示所需的周期事实
///
/// 只使用边界完整、可归属到整个账号的窗口;周优先于月,同周期保持
/// Provider 投影顺序
/// 没有符合条件的窗口时不回退到短期或历史累计
/// 周/月额度使用边界完整、可归属到整个账号的窗口
/// 周优先于月,同周期保持 Provider 投影顺序
/// Provider 提供的无固定重置点的滚动日统计可独立展示
#[must_use]
pub fn usage_window(&self) -> Option<(&ProviderQuotaWindow, AccountUsagePeriod)> {
self.usage_windows()
.filter(|(window, _)| window.local_usage.is_some())
.min_by_key(|(_, period)| *period)
.or_else(|| {
self.windows
.iter()
.find(|window| {
window.local_usage_attribution == QuotaLocalUsageAttribution::AccountWide
&& window.window_seconds == Some(86_400)
&& window.reset_at.is_none()
&& window.local_usage.is_some()
})
.map(|window| (window, AccountUsagePeriod::Daily))
})
}
/// 互联账号按最长的 Key 消费周期统计本地用量,池内账号窗口不代表此 Key 的周期
@@ -1025,7 +1037,7 @@ impl ProviderQuota {
.max_by_key(|window| window.window_seconds)
}
/// 统计与预测共用窗口归属规则,避免把模型专属桶或短期限流当作账号容量
/// 周/月额度的归属规则,供额度周期统计与容量预测共用
pub(crate) fn usage_windows(
&self,
) -> impl Iterator<Item = (&ProviderQuotaWindow, AccountUsagePeriod)> {
@@ -62,14 +62,15 @@ pub fn account_quota_forecasts(
now: DateTime<Utc>,
samples: &[QuotaForecastSample],
) -> [AccountQuotaForecast; 2] {
[AccountUsagePeriod::Weekly, AccountUsagePeriod::Monthly].map(|period| {
[
(AccountUsagePeriod::Weekly, 7 * DAY_SECONDS),
(AccountUsagePeriod::Monthly, 30 * DAY_SECONDS),
]
.map(|(period, target_seconds)| {
let selected = quota_forecast_source_window(quota, period);
let mut forecast = AccountQuotaForecast {
period,
target_seconds: match period {
AccountUsagePeriod::Weekly => 7 * DAY_SECONDS,
AccountUsagePeriod::Monthly => 30 * DAY_SECONDS,
},
target_seconds,
extrapolated: false,
source: None,
unavailable_reason: Some("缺少周/月额度窗口,请刷新账号额度"),
@@ -4,11 +4,14 @@ use chrono::{DateTime, Utc};
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ProviderUpstreamInfo {
/// 当前连接的互联识别结果,明确关闭共享或不支持协议时清除
pub discovered: bool,
pub observed_at: Option<DateTime<Utc>>,
pub stale: bool,
pub connection_revision: String,
pub last_error: Option<String>,
pub max_concurrency: Option<u64>,
pub requests_per_minute: Option<u64>,
pub pool: Option<UpstreamPool>,
}
@@ -10,6 +10,8 @@ pub enum UsageErrorKind {
Conflict,
CapacityExceeded,
Unavailable,
Cancelled,
Timeout,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
@@ -20,3 +20,6 @@ pub use actor::*;
mod aggregate;
pub use aggregate::*;
mod read;
pub use read::*;
@@ -0,0 +1,162 @@
//! 历史用量的窄事实、分组统计与调用内分批读取合同
use super::{UsageCount, UsageMetric, UsageRecordData, UsageRecordFilter};
use serde::{Deserialize, Serialize};
use std::collections::BTreeMap;
pub const USAGE_READ_MAX_BATCH_ROWS: u16 = 1024;
pub const USAGE_READ_MAX_BATCH_BYTES: usize = 2 * 1024 * 1024;
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageReadFilter {
pub records: UsageRecordFilter,
pub routing_group_ref: Option<String>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct ScanUsageRecords {
pub filter: UsageReadFilter,
pub batch_size: Option<u16>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum UsageGrouping {
Model,
Key,
KeyModel,
RoutingGroup,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct QueryUsageStatistics {
pub filter: UsageReadFilter,
pub group_by: UsageGrouping,
pub batch_size: Option<u16>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "kind", rename_all = "snake_case")]
pub enum UsageReadQuery {
Records(ScanUsageRecords),
Statistics(QueryUsageStatistics),
}
impl UsageReadQuery {
pub fn filter(&self) -> &UsageReadFilter {
match self {
Self::Records(query) => &query.filter,
Self::Statistics(query) => &query.filter,
}
}
pub fn batch_size(&self) -> u16 {
match self {
Self::Records(query) => query.batch_size,
Self::Statistics(query) => query.batch_size,
}
.unwrap_or(USAGE_READ_MAX_BATCH_ROWS)
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageRecordFact {
pub id: String,
pub data: UsageRecordData,
pub user_agent: Option<String>,
pub reasoning_effort: Option<String>,
pub subagent_kind: Option<String>,
pub request_kind: Option<String>,
pub routing_scope: Option<String>,
pub routing_group_refs: Vec<String>,
pub recovered: bool,
/// 由宿主现行规则派生,原始审计中的恢复请求仍可读取
pub request_counted: bool,
/// 包含完整交付、预热排除及恢复关联规则
pub usage_counted: bool,
}
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageStatisticsGroup {
pub client_key_ref: Option<String>,
pub model: Option<String>,
pub routing_group_ref: Option<String>,
pub unassigned_contribution: bool,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageStatisticsCost {
pub currency: String,
pub amount: String,
pub native_amount: Option<String>,
/// 外部修正允许负值,原生费用来源覆盖单独返回
pub contribution_amount: Option<String>,
}
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageStatisticsCostCoverage {
pub provider_reported: UsageCount,
pub calculated: UsageCount,
pub unavailable: UsageCount,
}
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageStatisticsMetrics {
/// 缺失的 Token 指标表示未知,计数指标始终返回
pub values: BTreeMap<UsageMetric, UsageCount>,
pub native_request_count: UsageCount,
pub contribution_count: UsageCount,
pub native_token_samples: BTreeMap<UsageMetric, UsageCount>,
/// 数量表示提供该指标的贡献记录数,不能用作请求覆盖率的分母
pub contribution_token_samples: BTreeMap<UsageMetric, UsageCount>,
pub cost_coverage: UsageStatisticsCostCoverage,
pub costs: Vec<UsageStatisticsCost>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageStatisticsRow {
/// null 表示全局总计,分组重叠不会重复计入该行
pub group: Option<UsageStatisticsGroup>,
pub metrics: UsageStatisticsMetrics,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "kind", content = "value", rename_all = "snake_case")]
pub enum UsageReadItem {
Record(Box<UsageRecordFact>),
Statistics(Box<UsageStatisticsRow>),
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageReadAnalytics {
pub coverage_start_ms: Option<i64>,
pub coverage_end_ms: Option<i64>,
pub lag_seconds: u64,
pub state: String,
pub generation: u64,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageReadObservation {
pub generated_at_ms: i64,
pub analytics: Option<UsageReadAnalytics>,
pub total_groups: Option<UsageCount>,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageReadBatch {
pub observation: UsageReadObservation,
pub items: Vec<UsageReadItem>,
pub complete: bool,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageReadOpened {
pub reader: Option<String>,
pub batch: UsageReadBatch,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct UsageReadHandle {
pub reader: String,
}
@@ -38,6 +38,13 @@ pub trait PeerTransport: Send + Sync {
/// 账号用例读取公共互联观测,显式刷新等待额度查询完成
#[async_trait]
pub trait PeerUpstreamService: gateway_core::account::UpstreamInfoSource {
/// 仅在指定持久观测与当前调度缓存一致时估算容量
fn observed_concurrency_limit(
&self,
version: &gateway_core::account::QuotaObservationVersion,
now: std::time::SystemTime,
) -> Option<gateway_core::account::AccountConcurrencyLimit>;
async fn account_changed(
&self,
id: &gateway_core::account::ProviderAccountId,
@@ -222,6 +222,15 @@ pub trait ProviderAdmin: Send + Sync {
Err(ProviderAdminError::new(ProviderAdminErrorKind::Unsupported))
}
/// Key 可以继承当前全局画像;额外字段的合法性由对应 Provider 判定
fn preview_key_client_profile(
&self,
key: &gateway_core::account::OpaqueProviderData,
_global: Option<&gateway_core::account::OpaqueProviderData>,
) -> Result<gateway_core::account::OpaqueProviderData, ProviderAdminError> {
self.preview_client_profile(key)
}
/// 提供最终套餐展示名称;未覆盖时采用通用的大驼峰格式
fn plan_type_display(&self, plan_type: &str) -> String {
default_plan_type_display(plan_type)
@@ -2,6 +2,14 @@
use crate::model::{auth::AdminAuditEvent, usage_data::*};
use async_trait::async_trait;
use gateway_core::lifecycle::CancellationToken;
#[async_trait]
pub trait UsageReadSession: Send {
async fn next_batch(&mut self) -> UsageDataResult<UsageReadBatch>;
fn is_finished(&self) -> bool;
async fn close(&mut self);
}
pub struct StoredUsageWrite {
pub changes: Vec<(UsageRecordChange, AdminAuditEvent)>,
@@ -13,6 +21,11 @@ pub struct StoredAggregateWrite {
#[async_trait]
pub trait UsageDataStore: Send + Sync {
async fn open_read(
&self,
query: UsageReadQuery,
cancellation: CancellationToken,
) -> UsageDataResult<Box<dyn UsageReadSession>>;
async fn list_aggregates(
&self,
request: ListUsageAggregates,
@@ -60,6 +60,7 @@ pub trait AccountGroupService: Send + Sync {
}
pub(crate) struct DefaultAccountGroupService {
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
store: Arc<dyn AccountGroupStore>,
runtime: Arc<dyn AccountRuntimeStore>,
snapshot: Arc<dyn SnapshotControl>,
@@ -68,11 +69,13 @@ pub(crate) struct DefaultAccountGroupService {
impl DefaultAccountGroupService {
#[must_use]
pub(crate) fn new(
upstream_info: Option<Arc<dyn crate::ports::peer::PeerUpstreamService>>,
store: Arc<dyn AccountGroupStore>,
runtime: Arc<dyn AccountRuntimeStore>,
snapshot: Arc<dyn SnapshotControl>,
) -> Self {
Self {
upstream_info,
store,
runtime,
snapshot,
@@ -105,7 +108,7 @@ impl DefaultAccountGroupService {
Default::default()
}
};
project_group_runtime(records, &members, &runtime);
project_group_runtime(records, &members, &runtime, self.upstream_info.as_deref());
Ok(())
}
@@ -215,6 +218,7 @@ fn project_group_runtime(
records: &mut [AccountGroupRecord],
members: &[AccountGroupMemberFact],
runtime: &crate::model::accounts::AccountRuntimeSnapshot,
upstream: Option<&dyn crate::ports::peer::PeerUpstreamService>,
) {
let now = SystemTime::now();
let mut status_by_account = BTreeMap::new();
@@ -228,7 +232,14 @@ fn project_group_runtime(
.or_insert_with(|| resolve_account_status(&status, now).status);
slots_by_account
.entry(member.account_id.as_str())
.or_insert(member.total_slots);
.or_insert_with(|| {
member
.inherited_version
.as_ref()
.and_then(|version| upstream?.observed_concurrency_limit(version, now))
.map(|limit| u64::from(limit.get()))
.or(member.total_slots)
});
accounts_by_group
.entry(member.group_id.as_str())
.or_default()
@@ -12,7 +12,7 @@ use chrono::{Duration, Utc};
use futures::StreamExt as _;
use gateway_core::{
account::ProviderAccountId,
engine::probe::{AccountProbe, AccountProbeRequest},
engine::probe::{AccountProbe, AccountProbeRequest, AccountProbeTarget},
routing::{ProviderKind, UpstreamModelId},
runtime::SnapshotControl,
};
@@ -178,7 +178,7 @@ pub trait AccountsService: Send + Sync {
async fn test_connection(
&self,
account_id: ProviderAccountId,
target: AccountProbeTarget,
upstream_model: UpstreamModelId,
) -> Result<AccountConnectionTestEventStream, AdminError>;
}
@@ -504,7 +504,11 @@ impl DefaultAccountsService {
});
Ok(AccountDirectoryItem {
protection_rules: stored.protection_rules,
capacity: stored.capacity,
capacity: super::peer_link::upstream::account_capacity(
&stored.account,
stored.capacity,
&quota,
),
capabilities: provider
.account_capabilities(account_id, &stored.account.authentication_kind),
plan_type_display: self.providers.resolve_account_plan(
@@ -618,7 +622,11 @@ impl AccountsService for DefaultAccountsService {
used_slots: in_flight
.as_ref()
.map(|counts| counts.get(&item.account.id).copied().unwrap_or(0)),
..item.capacity
..super::peer_link::upstream::account_capacity(
&item.account,
item.capacity,
&quota,
)
},
capabilities,
plan_type_display: providers.resolve_account_plan(
@@ -1123,10 +1131,10 @@ impl AccountsService for DefaultAccountsService {
async fn test_connection(
&self,
account_id: ProviderAccountId,
target: AccountProbeTarget,
upstream_model: UpstreamModelId,
) -> Result<AccountConnectionTestEventStream, AdminError> {
let (stored, provider) = self.provider_for_account(&account_id).await?;
let (stored, provider) = self.provider_for_account(target.account_id()).await?;
let account = stored.account;
let model = upstream_model.as_str().to_owned();
let operation = provider
@@ -1149,7 +1157,7 @@ impl AccountsService for DefaultAccountsService {
let result = probe
.probe(
AccountProbeRequest {
account_id,
target,
provider_kind: account.provider_kind,
upstream_model,
operation,
@@ -162,7 +162,7 @@ impl BackupService for DefaultBackupService {
ok: false,
stage: "putObject",
code: Some(code::INVALID_CONFIG),
message: "备份存储配置不完整,无法执行连接测试".to_owned(),
message: "备份存储配置不完整".to_owned(),
});
};
let result = self
@@ -198,14 +198,10 @@ impl BackupService for DefaultBackupService {
.await
.map_err(|error| map_store_error(error, "backup settings"))?;
if !settings.storage_configured() {
return Err(AdminError::bad_request(
"备份存储配置不完整,请先保存存储配置",
));
return Err(AdminError::bad_request("请先补全并保存备份存储配置"));
}
if !settings.storage_verified() {
return Err(AdminError::bad_request(
"备份存储尚未通过连接测试,请先测试存储配置",
));
return Err(AdminError::bad_request("请先通过备份存储连接测试"));
}
let prefix = settings.prefix.as_deref().unwrap_or_default();
let seed = build_backup_seed(
@@ -241,9 +237,7 @@ impl BackupService for DefaultBackupService {
) -> Result<DownloadUrlResult, AdminError> {
let record = self.require_record(backup_id).await?;
if record.status != BackupStatus::Completed {
return Err(AdminError::conflict(
"只有 completed 的备份才能创建下载地址",
));
return Err(AdminError::conflict("只能下载已完成的备份"));
}
let settings = self
.repository
@@ -280,9 +274,7 @@ impl BackupService for DefaultBackupService {
) -> Result<BackupRecord, AdminError> {
let record = self.require_record(backup_id).await?;
if !record.status.can_be_deleted() {
return Err(AdminError::conflict(
"只有 completed 或 failed 的备份才能删除",
));
return Err(AdminError::conflict("只能删除已完成或失败的备份"));
}
let now = Utc::now();
let deleting = self
@@ -290,7 +282,7 @@ impl BackupService for DefaultBackupService {
.transition_to_deleting(backup_id, now, Some(context))
.await
.map_err(|error| map_store_error(error, "backup record"))?
.ok_or_else(|| AdminError::conflict("备份记录已进入删除流程"))?;
.ok_or_else(|| AdminError::conflict("备份状态已变更,请刷新列表"))?;
Ok(deleting)
}
}
@@ -162,7 +162,7 @@ impl ClientKeyService for DefaultClientKeyService {
for (provider, profile) in &command.request_profile_overrides {
self.providers
.require(provider)
.and_then(|provider| provider.preview_client_profile(profile))
.and_then(|provider| provider.preview_key_client_profile(profile, None))
.map_err(|error| super::map_provider_error(error, "client profile"))?;
}
validate_peer_sharing(command.peer_sharing, &command.group_ids)?;
@@ -209,7 +209,7 @@ impl ClientKeyService for DefaultClientKeyService {
if let Some(profile) = profile {
self.providers
.require(provider)
.and_then(|provider| provider.preview_client_profile(profile))
.and_then(|provider| provider.preview_key_client_profile(profile, None))
.map_err(|error| super::map_provider_error(error, "client profile"))?;
}
}
@@ -207,7 +207,7 @@ impl DefaultImportTasksService {
}
Err(_) => {
item.status = ImportItemStatus::Unknown;
item.message = Some("执行中断,请先核对账号列表再决定是否重新导入".to_owned());
item.message = Some("导入结果未知,重新导入前请核对账号列表".to_owned());
}
}
entry.finish_if_done();
@@ -19,8 +19,8 @@ use crate::{
DashboardPeriodMetrics, DashboardQuery, DashboardResult, DecimalAmount,
DiagnosticDimension, DiagnosticsItem, DiagnosticsResult, Granularity, HealthStatus,
HealthTimeline, HealthTimelinePoint, OpsErrorPage, OpsErrorQuery, ProviderBillingInput,
RequestMetricPoint, RequestMetrics, TimeRange, Trend, TrendKind, TrendPoint,
TrendSummary, UsageBilling, UsageBillingProjection, UsageCacheEfficiency,
RequestHealth, RequestMetricPoint, RequestMetrics, TimeRange, Trend, TrendKind,
TrendPoint, TrendSummary, UsageBilling, UsageBillingProjection, UsageCacheEfficiency,
UsageCalculatedBillingFact, UsageConcurrencyPoint, UsageCostComparison,
UsageCostSummary, UsageDetail, UsageFilter, UsageInsight, UsageInsightData,
UsageInsightMetric, UsageInsights, UsageInsightsCost, UsageInsightsCostPoint,
@@ -133,7 +133,7 @@ impl DefaultObservabilityService {
self.settings.load_runtime_settings(),
)
.map_err(|error| map_store_error(error, "dashboard"))?;
let runtime_slots = observation.runtime_slots;
let runtime_slots = observation.runtime_slots.as_ref();
self.enrich_list_records(&mut observation.recent_requests)
.await;
self.enrich_dashboard_quotas(&mut observation.account_usage)
@@ -180,11 +180,19 @@ impl DefaultObservabilityService {
.providers
.dashboard_wire_profiles(&settings.request_profiles);
let max_concurrent_per_account = u64::from(settings.values.max_concurrent_per_account);
let (inherited_accounts, overridden_slots) = runtime_slots
.as_ref()
let (mut inherited_accounts, mut overridden_slots) = runtime_slots
.map_or((observation.provider_accounts.normal, 0), |slots| {
(slots.inherited_accounts, slots.overridden_slots)
});
if let (Some(peer), Some(slots)) = (&self.upstream_info, runtime_slots) {
let now = std::time::SystemTime::now();
for version in &slots.inherited_versions {
if let Some(limit) = peer.observed_concurrency_limit(version, now) {
inherited_accounts -= 1;
overridden_slots = overridden_slots.saturating_add(u64::from(limit.get()));
}
}
}
let total_slots = (max_concurrent_per_account > 0 || inherited_accounts == 0).then(|| {
inherited_accounts
.saturating_mul(max_concurrent_per_account)
@@ -566,7 +574,7 @@ fn build_usage_insights(
}
let requests = &overview.requests;
let health_window = HealthWindow::from_metrics(requests);
let health_window = RequestHealth::from_metrics(requests);
let failed_requests = health_window.failed_requests;
let health_requests = health_window.eligible_requests();
let health = UsageInsightsHealth {
@@ -586,7 +594,7 @@ fn build_usage_insights(
points: trend
.iter()
.map(|point| {
let health_window = HealthWindow::from_metrics(&point.metrics);
let health_window = RequestHealth::from_metrics(&point.metrics);
UsageInsightsHealthPoint {
total_requests: point.metrics.request_count,
bucket_start: point.bucket_start,
@@ -1117,7 +1125,7 @@ pub(super) fn health_timeline_at(
.map(|index| {
(
start + Duration::minutes(HEALTH_TIMELINE_SLOT_MINUTES * index),
HealthWindow::default(),
RequestHealth::default(),
)
})
.collect::<Vec<_>>();
@@ -1143,7 +1151,7 @@ pub(super) fn health_timeline_at(
let totals = buckets
.iter()
.filter(|(bucket_start, _)| *bucket_start <= current_slot)
.fold(HealthWindow::default(), |mut totals, (_, bucket)| {
.fold(RequestHealth::default(), |mut totals, (_, bucket)| {
totals.add_window(*bucket);
totals
});
@@ -1207,10 +1215,10 @@ fn trend(
}
}
let average_latency_ms = average(metrics.latency_sum_ms, metrics.latency_count);
let service_failure_count = service_failure_count(&metrics);
let service_failure_count = RequestHealth::from_metrics(&metrics).failed_requests;
let success_rate = ratio(
metrics.success_count,
HealthWindow::from_metrics(&metrics).eligible_requests(),
RequestHealth::from_metrics(&metrics).eligible_requests(),
);
let cache_hit_request_rate = ratio(
metrics.cache_hit_request_count,
@@ -1259,7 +1267,7 @@ fn trend(
}
fn trend_point(point: RequestMetricPoint) -> TrendPoint {
let service_failure_count = service_failure_count(&point.metrics);
let service_failure_count = RequestHealth::from_metrics(&point.metrics).failed_requests;
TrendPoint {
bucket_start: point.bucket_start,
granularity: point.granularity,
@@ -1279,7 +1287,7 @@ fn trend_point(point: RequestMetricPoint) -> TrendPoint {
),
success_rate: ratio(
point.metrics.success_count,
HealthWindow::from_metrics(&point.metrics).eligible_requests(),
RequestHealth::from_metrics(&point.metrics).eligible_requests(),
),
metrics: point.metrics,
cost_coverage: point.cost_coverage,
@@ -1364,57 +1372,7 @@ fn rate_or_zero(numerator: u64, denominator: u64) -> f64 {
ratio(numerator, denominator).unwrap_or(0.0)
}
#[derive(Debug, Clone, Copy, Default)]
struct HealthWindow {
success_requests: u64,
failed_requests: u64,
cancelled_requests: u64,
incomplete_requests: u64,
caller_error_requests: u64,
}
impl HealthWindow {
fn from_metrics(metrics: &RequestMetrics) -> Self {
Self {
success_requests: metrics.success_count,
failed_requests: service_failure_count(metrics),
cancelled_requests: metrics.cancelled_count,
incomplete_requests: metrics.incomplete_count,
caller_error_requests: metrics.caller_error_count,
}
}
// 未完成保留交付终态,但上游失败、超时和进程中断都不能视为服务成功
fn non_success_requests(self) -> u64 {
self.failed_requests
.saturating_add(self.incomplete_requests)
}
fn eligible_requests(self) -> u64 {
self.success_requests
.saturating_add(self.non_success_requests())
}
fn add_metrics(&mut self, metrics: &RequestMetrics) {
self.add_window(Self::from_metrics(metrics));
}
fn add_window(&mut self, other: Self) {
self.success_requests = self.success_requests.saturating_add(other.success_requests);
self.failed_requests = self.failed_requests.saturating_add(other.failed_requests);
self.cancelled_requests = self
.cancelled_requests
.saturating_add(other.cancelled_requests);
self.incomplete_requests = self
.incomplete_requests
.saturating_add(other.incomplete_requests);
self.caller_error_requests = self
.caller_error_requests
.saturating_add(other.caller_error_requests);
}
}
fn health_status(bucket: HealthWindow, is_future: bool) -> HealthStatus {
fn health_status(bucket: RequestHealth, is_future: bool) -> HealthStatus {
let eligible_requests = bucket.eligible_requests();
if is_future {
HealthStatus::Future
@@ -1435,14 +1393,6 @@ fn health_status(bucket: HealthWindow, is_future: bool) -> HealthStatus {
}
}
fn health_reliability(bucket: HealthWindow) -> Option<f64> {
let eligible_requests = bucket.eligible_requests();
(eligible_requests > 0)
.then(|| bucket.success_requests as f64 / eligible_requests as f64 * 100.0)
}
fn service_failure_count(metrics: &RequestMetrics) -> u64 {
metrics
.failure_count
.saturating_sub(metrics.caller_error_count)
fn health_reliability(bucket: RequestHealth) -> Option<f64> {
bucket.success_rate()
}
@@ -117,6 +117,7 @@ impl PeerLinkService for DefaultPeerLinkService {
.budget
.limits
.access(key.budget.daily_used_usd, key.budget.weekly_used_usd),
key_limits: key.limits,
budget: key.budget,
budget_observed_at,
pool: None,
@@ -56,7 +56,7 @@ impl PeerInfoService {
checked_at: now,
scheduling: None,
summary: None,
unsupported: false,
stored_failure: None,
},
));
}
@@ -92,19 +92,19 @@ impl PeerInfoService {
(
connection.clone(),
cached.and_then(|cached| cached.summary.clone()),
cached.is_some_and(|cached| cached.unsupported),
cached.and_then(|cached| cached.stored_failure),
)
})
.collect::<Vec<_>>()
};
Ok(summaries
.into_iter()
.map(|(connection, peer, unsupported)| {
.map(|(connection, peer, stored_failure)| {
let quota = Self::project_peer(
&connection,
peer.as_deref(),
self.last_error(&connection)
.or(unsupported.then_some(UpstreamInfoFailure::Unsupported)),
stored_failure,
self.last_error(&connection),
now,
);
(connection, quota)
@@ -135,8 +135,8 @@ impl PeerInfoService {
checked_at: now,
scheduling,
summary,
unsupported: observation
.is_some_and(|value| snapshot::is_unsupported(&value.quota, connection)),
stored_failure: observation
.and_then(|value| snapshot::stored_failure(&value.quota, connection)),
};
let previous = self
.observations
@@ -151,6 +151,9 @@ impl PeerInfoService {
.read()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.get(&connection.account_id)
.is_some_and(|cached| cached.connection == *connection && cached.unsupported)
.is_some_and(|cached| {
cached.connection == *connection
&& cached.stored_failure == Some(UpstreamInfoFailure::Unsupported)
})
}
}
@@ -25,6 +25,7 @@ use gateway_core::{
error::StoreError,
task::{WorkerContribution, WorkerDefinitionError},
};
pub(crate) use projection::account_capacity;
use service::PeerInfoService;
/// 组合根独立注册公共互联 Worker,不属于任何具体 Provider 的任务集合
@@ -62,6 +63,24 @@ pub fn initialize(
#[async_trait::async_trait]
impl PeerUpstreamService for PeerInfoService {
fn observed_concurrency_limit(
&self,
version: &gateway_core::account::QuotaObservationVersion,
now: std::time::SystemTime,
) -> Option<gateway_core::account::AccountConcurrencyLimit> {
let observations = self
.observations
.read()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let cached = observations.get(&version.account_id)?;
if cached.observed_at != Some(version.observed_at)
|| cached.connection.credential_revision != version.credential_revision
{
return None;
}
cached.scheduling.as_ref()?.concurrency_limit(now)
}
async fn account_changed(&self, id: &ProviderAccountId) -> Result<(), StoreError> {
PeerInfoService::account_changed(self, id).await
}
@@ -16,10 +16,31 @@ use gateway_protocol::peer::{PeerComponentStatus, PeerLink};
use super::{PeerInfoService, quota, snapshot};
pub(crate) fn account_capacity(
account: &crate::model::accounts::AccountRecord,
mut capacity: crate::model::accounts::AccountCapacity,
quota: &ProviderQuota,
) -> crate::model::accounts::AccountCapacity {
if account.concurrency_limit.is_none()
&& let Some(info) = &quota.upstream
&& !info.stale
&& info
.observed_at
.is_some_and(|at| snapshot::fresh_at(at.into(), SystemTime::now()))
&& let Some(limit) = info
.max_concurrency
.and_then(gateway_core::account::AccountConcurrencyLimit::from_key_limit)
{
capacity.total_slots = Some(u64::from(limit.get()));
}
capacity
}
impl PeerInfoService {
pub(super) fn project_peer(
connection: &UpstreamConnection,
peer: Option<&PeerLink>,
stored_failure: Option<UpstreamInfoFailure>,
last_error: Option<UpstreamInfoFailure>,
now: SystemTime,
) -> ProviderQuota {
@@ -29,12 +50,17 @@ impl PeerInfoService {
let stale = peer.as_ref().is_none_or(|peer| {
!snapshot::component_fresh(Some(&peer.observed_at), Some(&peer.expires_at), now)
});
let limits = peer.and_then(|peer| peer.key_limits);
let info = ProviderUpstreamInfo {
discovered: peer.is_some(),
discovered: peer.is_some() || stored_failure == Some(UpstreamInfoFailure::Unauthorized),
observed_at,
stale,
connection_revision: connection.revision.clone(),
last_error: last_error.map(|error| error.as_str().to_owned()),
last_error: last_error
.or(stored_failure)
.map(|error| error.as_str().to_owned()),
max_concurrency: limits.and_then(|limits| limits.max_concurrency),
requests_per_minute: limits.and_then(|limits| limits.requests_per_minute),
pool: peer
.as_ref()
.and_then(|peer| peer.pool.as_ref())
@@ -105,8 +105,8 @@ impl PeerInfoService {
{
let error = state.last_error;
drop(state);
if let Some(error) = error.filter(|error| *error != UpstreamInfoFailure::Unsupported) {
return Err(refresh_error(error));
if let Some(error) = error.and_then(refresh_error) {
return Err(error);
}
return self
.read_quota(id)
@@ -130,26 +130,39 @@ impl PeerInfoService {
let failure = result.as_ref().err().copied();
let delay = match failure {
None => Duration::from_secs(240 + stable_jitter(id.as_str())),
Some(UpstreamInfoFailure::Unauthorized) => Duration::from_secs(1800),
Some(UpstreamInfoFailure::Unauthorized | UpstreamInfoFailure::SharingDisabled) => {
Duration::from_secs(1800)
}
Some(_) => Duration::from_secs(30),
}
.max(retry_after.unwrap_or_default());
state.next_attempt = Instant::now() + delay;
state.manual_after = Instant::now()
+ (if failure.is_some_and(|error| error != UpstreamInfoFailure::Unsupported) {
delay
} else {
Duration::from_secs(15)
})
.max(retry_after.unwrap_or_default());
// 管理员修复授权后可主动复核,不继承自动重试的长退避,仍遵守上游限流
state.manual_after =
Instant::now() + Duration::from_secs(15).max(retry_after.unwrap_or_default());
state.last_error = failure;
let document = match result {
Ok(document) => Some(document),
// 空观测清除已撤销内容,并用共有观察时间阻止更早请求重新写回
Err(UpstreamInfoFailure::Unauthorized) => {
Some(OpaqueProviderData::new(Default::default()))
}
Err(UpstreamInfoFailure::Unsupported) => Some(snapshot::unsupported(&connection)),
Err(error @ UpstreamInfoFailure::SharingDisabled) => {
Some(snapshot::unavailable(&connection, error))
}
Err(error @ UpstreamInfoFailure::Unauthorized) => {
let discovered = self
.read_summary(id)
.await
.map_err(store_error)?
.and_then(|quota| quota.upstream)
.is_some_and(|info| {
info.connection_revision == connection.revision && info.discovered
});
// 首次探测失败不能证明对方是互联节点;已确认的连接只留下无共享内容的识别记录
Some(if discovered {
snapshot::unavailable(&connection, error)
} else {
OpaqueProviderData::new(Default::default())
})
}
Err(_) => None,
};
if let Some(document) = document {
@@ -190,8 +203,8 @@ impl PeerInfoService {
}
drop(state);
// 不支持协议是正常发现结果,管理端保留分类,后台不重复报警
if let Some(error) = failure.filter(|error| *error != UpstreamInfoFailure::Unsupported) {
return Err(refresh_error(error));
if let Some(error) = failure.and_then(refresh_error) {
return Err(error);
}
self.read_quota(id)
.await
@@ -206,12 +219,13 @@ fn stable_jitter(id: &str) -> u64 {
}) % 60
}
fn refresh_error(error: UpstreamInfoFailure) -> ProviderAdminError {
ProviderAdminError::new(ProviderAdminErrorKind::BadGateway).with_public_message(match error {
fn refresh_error(error: UpstreamInfoFailure) -> Option<ProviderAdminError> {
let message = match error {
UpstreamInfoFailure::Unauthorized => "上游未授权读取互联信息",
UpstreamInfoFailure::Unsupported => "上游不支持互联信息查询",
UpstreamInfoFailure::Unsupported | UpstreamInfoFailure::SharingDisabled => return None,
UpstreamInfoFailure::RateLimited => "上游互联查询已限流",
UpstreamInfoFailure::InvalidResponse => "上游互联响应无效",
UpstreamInfoFailure::Unavailable => "上游互联服务暂不可用",
})
};
Some(ProviderAdminError::new(ProviderAdminErrorKind::BadGateway).with_public_message(message))
}
@@ -63,6 +63,7 @@ struct ModelObservation {
pub(super) struct SchedulingObservation {
validity: Validity,
concurrency_limit: Option<gateway_core::account::AccountConcurrencyLimit>,
exhausted_budgets: Vec<Validity>,
models: HashMap<String, Option<ModelObservation>>,
}
@@ -166,43 +167,52 @@ impl SchedulingObservation {
}
Some(Self {
validity,
concurrency_limit: peer
.key_limits
.and_then(|limits| limits.max_concurrency)
.and_then(gateway_core::account::AccountConcurrencyLimit::from_key_limit),
exhausted_budgets,
models,
})
}
pub(super) fn fresh(&self, now: SystemTime) -> bool {
self.validity.fresh(now)
&& now
.duration_since(self.validity.observed_at)
.unwrap_or_default()
<= snapshot::FRESH_FOR
self.validity.fresh(now) && snapshot::fresh_at(self.validity.observed_at, now)
}
pub(super) fn signal(&self, model: &str, now: SystemTime) -> Option<UpstreamSchedulingSignal> {
pub(super) fn concurrency_limit(
&self,
now: SystemTime,
) -> Option<gateway_core::account::AccountConcurrencyLimit> {
self.concurrency_limit.filter(|_| self.fresh(now))
}
pub(super) fn signal(
&self,
model: Option<&str>,
now: SystemTime,
) -> Option<UpstreamSchedulingSignal> {
if !self.fresh(now) {
return None;
}
if self
let budget_exhausted = self
.exhausted_budgets
.iter()
.any(|validity| validity.fresh(now))
{
return Some(UpstreamSchedulingSignal {
exhausted: true,
remaining_percent: None,
});
}
let model = self.models.get(model)?.as_ref()?;
if !model.validity.fresh(now) {
.any(|validity| validity.fresh(now));
let model = model
.and_then(|id| self.models.get(id)?.as_ref())
.filter(|model| model.validity.fresh(now));
if !budget_exhausted && model.is_none() && self.concurrency_limit.is_none() {
return None;
}
Some(UpstreamSchedulingSignal {
exhausted: model.exhausted,
remaining_percent: model
.account
.as_ref()
.and_then(|account| account.remaining_percent(now)),
concurrency_limit: self.concurrency_limit,
exhausted: budget_exhausted || model.is_some_and(|model| model.exhausted),
remaining_percent: if budget_exhausted {
None
} else {
model.and_then(|model| model.account.as_ref()?.remaining_percent(now))
},
})
}
}
@@ -34,7 +34,7 @@ pub(super) struct CachedObservation {
pub checked_at: SystemTime,
pub scheduling: Option<Arc<SchedulingObservation>>,
pub summary: Option<Arc<PeerLink>>,
pub unsupported: bool,
pub stored_failure: Option<UpstreamInfoFailure>,
}
pub(crate) struct PeerInfoService {
@@ -172,17 +172,17 @@ impl PeerInfoService {
let peer = observation
.as_ref()
.and_then(|value| snapshot::decode(&value.quota));
let unsupported = observation
let stored_failure = observation
.as_ref()
.is_some_and(|value| snapshot::is_unsupported(&value.quota, &connection));
.and_then(|value| snapshot::stored_failure(&value.quota, &connection));
if self.current_connection(id).await?.as_ref() != Some(&connection) {
return Ok(None);
}
Ok(Some(Self::project_peer(
&connection,
peer.as_ref(),
self.last_error(&connection)
.or(unsupported.then_some(UpstreamInfoFailure::Unsupported)),
stored_failure,
self.last_error(&connection),
SystemTime::now(),
)))
}
@@ -285,7 +285,7 @@ impl UpstreamInfoSource for PeerInfoService {
fn signal(
&self,
account: &ProviderAccount,
model: &str,
model: Option<&str>,
now: SystemTime,
) -> Option<UpstreamSchedulingSignal> {
if !account.enabled() {
@@ -28,17 +28,42 @@ pub(super) fn unsupported(connection: &UpstreamConnection) -> OpaqueProviderData
]))
}
pub(super) fn is_unsupported(data: &OpaqueProviderData, connection: &UpstreamConnection) -> bool {
// 授权撤销或共享关闭后只保存连接与失败原因,丢弃全部共享数据
pub(super) fn unavailable(
connection: &UpstreamConnection,
reason: UpstreamInfoFailure,
) -> OpaqueProviderData {
OpaqueProviderData::new(serde_json::Map::from_iter([
("object".into(), "cpr.peer.unavailable".into()),
("schema_version".into(), 1.into()),
(
"connection_revision".into(),
connection.revision.clone().into(),
),
("reason".into(), reason.as_str().into()),
]))
}
pub(super) fn stored_failure(
data: &OpaqueProviderData,
connection: &UpstreamConnection,
) -> Option<UpstreamInfoFailure> {
let fields = data.expose_to_provider();
fields.get("object").and_then(serde_json::Value::as_str) == Some("cpr.peer.unsupported")
&& fields
.get("schema_version")
.and_then(serde_json::Value::as_u64)
== Some(1)
&& fields
.get("connection_revision")
.and_then(serde_json::Value::as_str)
== Some(connection.revision.as_str())
if fields.get("schema_version")?.as_u64()? != 1
|| fields.get("connection_revision")?.as_str()? != connection.revision
{
return None;
}
match fields.get("object")?.as_str()? {
"cpr.peer.unsupported" => Some(UpstreamInfoFailure::Unsupported),
"cpr.peer.unavailable" => match fields.get("reason")?.as_str()? {
"unauthorized" => Some(UpstreamInfoFailure::Unauthorized),
"sharing_disabled" => Some(UpstreamInfoFailure::SharingDisabled),
"unsupported" => Some(UpstreamInfoFailure::Unsupported),
_ => None,
},
_ => None,
}
}
pub(super) fn timestamp(value: &str) -> Option<SystemTime> {
@@ -54,10 +79,12 @@ pub(super) fn datetime(value: &str) -> Option<DateTime<Utc>> {
}
pub(super) fn fresh(observed: &str, now: SystemTime) -> bool {
timestamp(observed).is_some_and(|observed| {
observed <= now + Duration::from_secs(60)
&& now.duration_since(observed).unwrap_or_default() <= FRESH_FOR
})
timestamp(observed).is_some_and(|observed| fresh_at(observed, now))
}
pub(super) fn fresh_at(observed: SystemTime, now: SystemTime) -> bool {
observed <= now + Duration::from_secs(60)
&& now.duration_since(observed).unwrap_or_default() <= FRESH_FOR
}
pub(super) fn component_fresh(
@@ -127,6 +154,9 @@ pub(super) fn parse(
}
// 未授权能力不能借同名数据块越过逐层公开边界
let disclosed = |capability| peer.capabilities.iter().any(|value| value == capability);
if !disclosed("key_limits") {
peer.key_limits = None;
}
if !disclosed("key_usage") {
peer.usage = None;
}
@@ -361,9 +361,10 @@ impl ProxiesService for DefaultProxiesService {
proxy: Option<&OutboundProxy>,
detect_location: bool,
) -> Result<ProxyTestResult, AdminError> {
let _permit = self.test_slots.try_acquire().map_err(|_| {
AdminError::new(AdminErrorKind::RateLimited, "代理测试繁忙,请稍后重试")
})?;
let _permit = self
.test_slots
.try_acquire()
.map_err(|_| AdminError::new(AdminErrorKind::RateLimited, "代理测试已达并发上限"))?;
Ok(self.probe.test(proxy, detect_location).await)
}
@@ -374,9 +375,10 @@ impl ProxiesService for DefaultProxiesService {
detect_location: bool,
context: &MutationContext,
) -> Result<ProxyRecord, AdminError> {
let _permit = self.test_slots.try_acquire().map_err(|_| {
AdminError::new(AdminErrorKind::RateLimited, "代理测试繁忙,请稍后重试")
})?;
let _permit = self
.test_slots
.try_acquire()
.map_err(|_| AdminError::new(AdminErrorKind::RateLimited, "代理测试已达并发上限"))?;
let record = self
.store
.get(id)
@@ -252,7 +252,7 @@ impl SettingsService for DefaultSettingsService {
.is_some_and(|models| models.contains_key(model))
})
{
return Err(AdminError::invalid("内置价目不能删除,仅支持人工覆盖"));
return Err(AdminError::invalid("内置价目只能覆盖,不能删除"));
}
let defaults = gateway_core::metering::merge_pricing(catalog.defaults, &catalog.synced);
let defaults = defaults.get(&command.provider);
@@ -332,24 +332,35 @@ impl SettingsService for DefaultSettingsService {
configuration: Option<&gateway_core::account::OpaqueProviderData>,
) -> Result<gateway_core::account::OpaqueProviderData, AdminError> {
let profile_provider = self.profile_provider(provider)?;
let global;
let (configuration, source) = if let Some(configuration) = configuration {
(configuration, "override")
let independent = configuration.filter(|key| {
gateway_core::settings::request_profiles::inherited_fields(key).is_none()
});
let preview = if let Some(key) = independent {
profile_provider.preview_key_client_profile(key, None)
} else {
global = self
let global = self
.load()
.await?
.client_profile(provider)
.cloned()
.or_else(|| profile_provider.default_client_profile())
.ok_or_else(|| AdminError::internal("通用客户端身份尚未初始化"))?;
(&global, "global")
};
let mut preview = profile_provider
.preview_client_profile(configuration)
.map_err(|error| super::map_provider_error(error, "client profile"))?
.into_inner();
preview.insert("source".to_owned(), serde_json::Value::from(source));
if let Some(key) = configuration {
profile_provider.preview_key_client_profile(key, Some(&global))
} else {
profile_provider.preview_client_profile(&global)
}
}
.map_err(|error| super::map_provider_error(error, "client profile"))?;
let mut preview = preview.into_inner();
preview.insert(
"source".to_owned(),
serde_json::Value::from(if independent.is_some() {
"override"
} else {
"global"
}),
);
Ok(gateway_core::account::OpaqueProviderData::new(preview))
}
@@ -419,7 +430,7 @@ impl SettingsService for DefaultSettingsService {
crate::ports::store::AdminStoreErrorKind::Conflict
| crate::ports::store::AdminStoreErrorKind::StaleRevision
) {
AdminError::conflict("运行设置已被其他调用修改,请重新读取并确认后提交")
AdminError::conflict("运行设置已变更,请刷新核对后提交")
} else {
map_store_error(error, "runtime settings")
}
@@ -15,6 +15,11 @@ use std::sync::Arc;
#[async_trait]
pub trait UsageDataService: Send + Sync {
async fn open_read(
&self,
query: UsageReadQuery,
cancellation: gateway_core::lifecycle::CancellationToken,
) -> UsageDataResult<Box<dyn UsageReadSession>>;
async fn list_aggregates(
&self,
request: ListUsageAggregates,
@@ -58,6 +63,21 @@ impl DefaultUsageDataService {
#[async_trait]
impl UsageDataService for DefaultUsageDataService {
async fn open_read(
&self,
query: UsageReadQuery,
cancellation: gateway_core::lifecycle::CancellationToken,
) -> UsageDataResult<Box<dyn UsageReadSession>> {
let filter = query.filter();
validation::query(&filter.records, None, None)?;
if let Some(group) = &filter.routing_group_ref {
validation::text(group, 128)?;
}
if !(1..=USAGE_READ_MAX_BATCH_ROWS).contains(&query.batch_size()) {
return Err(UsageDataError::invalid("读取批次条数应为 1 到 1024"));
}
self.store.open_read(query, cancellation).await
}
async fn list_aggregates(
&self,
request: ListUsageAggregates,
@@ -7,7 +7,6 @@ use gateway_core::account::OpaqueProviderData;
use gateway_admin::model::{
PageSize, Revision,
accounts::AccountUsage,
auth::LoginCommand,
client_keys::ClientKeyPageSize,
observability::{DecimalAmount, RequestOutcome, TimeRange},
@@ -146,57 +145,6 @@ fn representative_quota_should_prefer_account_wide_window_over_model_specific_wi
assert_eq!(quota.representative_used_percent(), Some(5.0));
}
#[test]
fn usage_window_should_not_use_daily_rolling_usage_for_weekly_statistics() {
let usage = AccountUsage {
account_id: "acct_xai".to_owned(),
request_count: 5,
success_count: 5,
input_tokens: Some(100),
output_tokens: Some(20),
cached_input_tokens: Some(80),
cache_write_input_tokens: Some(0),
reasoning_output_tokens: Some(0),
image_input_tokens: Some(0),
image_output_tokens: Some(0),
image_request_count: 0,
image_request_failed_count: 0,
total_tokens: Some(120),
cost_coverage: Default::default(),
costs: Vec::new(),
last_used_at: Some(Utc::now()),
request_buckets: Vec::new(),
models: Vec::new(),
};
let quota = ProviderQuota {
upstream: None,
credits: None,
plan_type: None,
observed_at: None,
refresh_token_expires_at: None,
windows: vec![ProviderQuotaWindow {
upstream: None,
key: "free-rolling-24h".to_owned(),
group: "shortTerm".to_owned(),
label: "日限额".to_owned(),
limit_id: None,
limit_name: None,
role: None,
local_usage_attribution: QuotaLocalUsageAttribution::AccountWide,
window_seconds: Some(86_400),
used_percent: None,
reset_at: None,
limit_reached: false,
local_usage: Some(usage),
provider_data: None,
}],
limit_reached: false,
provider_data: None,
};
assert_eq!(quota.usage_window(), None);
}
#[test]
fn exhausted_quota_should_project_full_usage_to_only_the_representative_window() {
let mut quota = ProviderQuota {
@@ -65,6 +65,36 @@ fn usage_window_should_not_fallback_to_short_daily_or_incomplete_windows() {
assert_eq!(quota.usage_window(), None);
}
#[test]
fn usage_window_should_preserve_provider_rolling_daily_statistics() {
let mut rolling = usage_window("rolling-day", "shortTerm", 86_400);
rolling.reset_at = None;
rolling.used_percent = None;
let mut quota = usage_quota(vec![rolling.clone()]);
assert_eq!(
selected_usage_window(&quota),
Some(("rolling-day", AccountUsagePeriod::Daily))
);
quota.limit_reached = true;
quota.apply_limit_reached_display();
assert_eq!(
selected_usage_window(&quota),
Some(("rolling-day", AccountUsagePeriod::Daily))
);
quota
.windows
.push(usage_window("week", "shortTerm", 7 * 86_400));
assert_eq!(
selected_usage_window(&quota),
Some(("week", AccountUsagePeriod::Weekly))
);
rolling.local_usage_attribution = QuotaLocalUsageAttribution::Unavailable;
assert_eq!(usage_quota(vec![rolling]).usage_window(), None);
}
#[test]
fn usage_window_should_preserve_provider_order_for_equal_periods() {
let quota = usage_quota(vec![
@@ -162,7 +162,10 @@ fn source_selection_keeps_provider_order_with_the_maximum_window_count() {
fn short_or_model_specific_windows_are_not_account_capacity() {
let mut model = window("model", 7);
model.local_usage_attribution = QuotaLocalUsageAttribution::Unavailable;
for item in forecast(&quota(vec![window("day", 1), model])) {
let mut rolling = window("rolling-day", 1);
rolling.reset_at = None;
rolling.used_percent = None;
for item in forecast(&quota(vec![window("day", 1), rolling, model])) {
assert!(item.source.is_none());
assert!(item.unavailable_reason.is_some());
assert!(item.estimated_tokens.is_none());
@@ -373,6 +373,7 @@ fn group_record() -> AccountGroupRecord {
fn member(account_id: &str, total_slots: u64) -> AccountGroupMemberFact {
AccountGroupMemberFact {
inherited_version: None,
group_id: group_id(),
account_id: account_id.to_owned(),
status: AccountStatusFacts {
@@ -2,6 +2,7 @@
mod reset_credit_automation;
use gateway_core::engine::probe::AccountProbeTarget;
use std::{
collections::BTreeMap,
sync::{Arc, Mutex},
@@ -1335,7 +1336,9 @@ async fn connection_test_should_probe_unavailable_account() {
let events = services
.accounts()
.test_connection(
ProviderAccountId::new("acct_test").expect("account ID"),
AccountProbeTarget::ConnectionTest(
ProviderAccountId::new("acct_test").expect("account ID"),
),
gateway_core::routing::UpstreamModelId::new("grok-4.5").expect("model"),
)
.await
@@ -1372,7 +1375,9 @@ async fn connection_test_rate_limited_probe_returns_provider_failure() {
let events = services
.accounts()
.test_connection(
ProviderAccountId::new("acct_test").expect("account ID"),
AccountProbeTarget::ConnectionTest(
ProviderAccountId::new("acct_test").expect("account ID"),
),
gateway_core::routing::UpstreamModelId::new("grok-4.5").expect("model"),
)
.await
@@ -1405,7 +1410,9 @@ async fn connection_test_should_preserve_disabled_account_status() {
let events = services
.accounts()
.test_connection(
ProviderAccountId::new("acct_test").expect("account ID"),
AccountProbeTarget::ConnectionTest(
ProviderAccountId::new("acct_test").expect("account ID"),
),
gateway_core::routing::UpstreamModelId::new("grok-4.5").expect("model"),
)
.await
@@ -2686,12 +2693,15 @@ async fn peer_account_usage_should_follow_budget_windows_without_borrowing_pool_
for longest in [86_400, 604_800] {
let provider = FakeProviderAdmin::new("openai", events());
let mut account = account_record("openai");
account.concurrency_limit = None;
account.authentication_kind = "api_key".to_owned();
account.created_at = Utc::now() - TimeDelta::days(60);
let store = FakeAccountStore::with_account(account, events());
let reset_at = Utc::now() + TimeDelta::hours(1);
let mut quota = empty_quota();
quota.upstream = Some(ProviderUpstreamInfo {
max_concurrency: Some(7),
requests_per_minute: Some(60),
discovered: true,
observed_at: Some(Utc::now()),
stale: false,
@@ -2762,6 +2772,8 @@ async fn peer_account_usage_should_follow_budget_windows_without_borrowing_pool_
page.items[0].usage.as_ref().unwrap().total_tokens,
Some(1_234)
);
assert_eq!(page.items[0].capacity.total_slots, Some(7));
assert_eq!(page.items[0].account.concurrency_limit, None);
let queries = store.quota_window_queries();
assert_eq!(queries.len(), window_usage.len());
for (query, expected) in queries.iter().zip(&window_usage) {
@@ -143,7 +143,7 @@ impl UpstreamInfoSource for PendingPeer {
fn signal(
&self,
_: &ProviderAccount,
_: &str,
_: Option<&str>,
_: std::time::SystemTime,
) -> Option<UpstreamSchedulingSignal> {
None
@@ -152,6 +152,14 @@ impl UpstreamInfoSource for PendingPeer {
#[async_trait]
impl PeerUpstreamService for PendingPeer {
fn observed_concurrency_limit(
&self,
_: &gateway_core::account::QuotaObservationVersion,
_: std::time::SystemTime,
) -> Option<gateway_core::account::AccountConcurrencyLimit> {
None
}
async fn account_changed(&self, id: &ProviderAccountId) -> Result<(), StoreError> {
self.changed.lock().unwrap().push(id.as_str().into());
Ok(())
@@ -1,5 +1,6 @@
//! 账号保护恢复编排:到期探测、快照参数与失败顺延
use gateway_core::engine::probe::AccountProbeTarget;
use std::collections::BTreeMap;
use std::sync::{Arc, Mutex};
@@ -242,6 +243,10 @@ impl AccountProbe for FailingProbe {
request: AccountProbeRequest,
_: Option<Arc<gateway_core::routing::RuntimeSnapshot>>,
) -> futures::future::BoxFuture<'_, Result<AccountProbeResult, AccountProbeError>> {
assert!(matches!(
request.target,
AccountProbeTarget::FreezeRecovery(_)
));
assert_eq!(request.upstream_model.as_str(), "snapshot-model");
Box::pin(async {
Err(AccountProbeError::new(
@@ -448,6 +448,7 @@ async fn dashboard_summary_should_project_rebuildable_runtime_slots() {
let now = Utc::now();
let store = Arc::new(FixtureObservabilityStore::new(observation_range(now)));
store.replace_runtime_slots(Some(DashboardRuntimeSlots {
inherited_versions: Vec::new(),
inherited_accounts: 2,
overridden_slots: 5,
used_slots: Some(2),
@@ -476,6 +477,7 @@ async fn dashboard_capacity_distinguishes_unlimited_inheritance_finite_overrides
let now = Utc::now();
let store = Arc::new(FixtureObservabilityStore::new(observation_range(now)));
store.replace_runtime_slots(Some(DashboardRuntimeSlots {
inherited_versions: Vec::new(),
inherited_accounts,
overridden_slots,
used_slots: Some(2),
@@ -1212,7 +1214,7 @@ impl ObservabilityStore for FixtureObservabilityStore {
range: query.range,
totals: Default::default(),
provider_accounts: AccountPoolMetrics::default(),
runtime_slots: *self.runtime_slots.lock().expect("runtime slots"),
runtime_slots: self.runtime_slots.lock().expect("runtime slots").clone(),
trend: self.trend.lock().expect("trend").clone(),
account_usage: self.account_usage.lock().expect("account usage").clone(),
recent_requests: Vec::new(),
@@ -1482,6 +1484,11 @@ fn total_record(
provider_account_upstream_peer: None,
error_code: None,
error_message: None,
error_kind: None,
websocket_close_code: None,
continuation_unavailable_reason: None,
upstream_connection_exit_reason: None,
recovered: false,
client_status_code: None,
outcome: gateway_admin::model::observability::RequestOutcome::Succeeded,
client_api_key_name: Some("Production".to_owned()),
@@ -241,7 +241,7 @@ async fn discovery_uses_provider_capabilities_and_publishes_neutral_signals() {
let reader = UpstreamInfoReader::default();
assert!(
reader
.signal(&initial_accounts[0], "model", SystemTime::now())
.signal(&initial_accounts[0], Some("model"), SystemTime::now())
.is_none()
);
let source = bundle.scheduling_source();
@@ -273,7 +273,11 @@ async fn discovery_uses_provider_capabilities_and_publishes_neutral_signals() {
);
for account in &initial_accounts[..2] {
if Some(account.provider().as_str()) == failed_provider {
assert!(reader.signal(account, "model", SystemTime::now()).is_none());
assert!(
reader
.signal(account, Some("model"), SystemTime::now())
.is_none()
);
continue;
}
let observed = bundle
@@ -288,21 +292,21 @@ async fn discovery_uses_provider_capabilities_and_publishes_neutral_signals() {
);
assert!(
reader
.signal(account, "model", SystemTime::now())
.signal(account, Some("model"), SystemTime::now())
.unwrap()
.exhausted
);
}
assert!(
reader
.signal(&initial_accounts[2], "model", SystemTime::now())
.signal(&initial_accounts[2], Some("model"), SystemTime::now())
.is_none()
);
drop(worker);
drop(bundle);
assert!(
reader
.signal(&initial_accounts[0], "model", SystemTime::now())
.signal(&initial_accounts[0], Some("model"), SystemTime::now())
.is_none()
);
}
@@ -332,7 +336,7 @@ async fn account_edits_preserve_observation_and_refresh_budget_until_connection_
assert!(
bundle
.scheduling_source()
.signal(&edited, "model", SystemTime::now())
.signal(&edited, Some("model"), SystemTime::now())
.is_some()
);
let after = service.refresh_quota(edited.id()).await.unwrap();
@@ -347,7 +351,7 @@ async fn account_edits_preserve_observation_and_refresh_budget_until_connection_
assert!(
bundle
.scheduling_source()
.signal(&reconnected, "model", SystemTime::now())
.signal(&reconnected, Some("model"), SystemTime::now())
.is_none()
);
service.refresh_quota(reconnected.id()).await.unwrap();
@@ -392,7 +396,7 @@ impl PeerTransport for TestTransport {
.push(account.id().as_str().to_owned());
let now = Utc::now();
Ok(serde_json::to_vec(&serde_json::json!({
"object":"cpr.peer", "schema_version":2,
"object":"cpr.peer", "schema_version":3,
"observed_at":now.to_rfc3339(),
"expires_at":(now + chrono::Duration::minutes(5)).to_rfc3339(),
"capabilities":["key_usage"],
@@ -175,6 +175,191 @@ async fn uncertain_discovery_failures_retry_without_becoming_unsupported() {
}
}
#[tokio::test(start_paused = true)]
async fn known_peer_identity_changes_only_on_definitive_detection() {
for failure in [
UpstreamInfoFailure::Unauthorized,
UpstreamInfoFailure::Unsupported,
UpstreamInfoFailure::SharingDisabled,
UpstreamInfoFailure::Unavailable,
UpstreamInfoFailure::RateLimited,
UpstreamInfoFailure::InvalidResponse,
] {
let (bundle, accounts, transport, id) = fixture(failure);
*transport.failure.lock().unwrap() = None;
let before = bundle.service().refresh_quota(&id).await.unwrap();
assert!(!before.windows.is_empty());
tokio::time::advance(Duration::from_secs(15)).await;
*transport.failure.lock().unwrap() = Some(failure.into());
let result = bundle.service().refresh_quota(&id).await;
let clears_identity = matches!(
failure,
UpstreamInfoFailure::Unsupported | UpstreamInfoFailure::SharingDisabled
);
assert_eq!(result.is_ok(), clears_identity);
let clears_data = clears_identity || failure == UpstreamInfoFailure::Unauthorized;
let quota = bundle.service().read_quota(&id).await.unwrap().unwrap();
let info = quota.upstream.as_ref().unwrap();
assert_eq!(info.discovered, !clears_identity);
assert_eq!(info.last_error.as_deref(), Some(failure.as_str()));
assert_eq!(quota.windows.is_empty(), clears_data);
assert_eq!(info.observed_at.is_none(), clears_data);
assert_eq!(
bundle
.service()
.read_summary(&id)
.await
.unwrap()
.unwrap()
.upstream,
quota.upstream
);
let account = accounts.accounts.lock().unwrap()[0].clone();
assert_eq!(
bundle
.scheduling_source()
.signal(&account, Some("model"), SystemTime::now())
.is_none(),
clears_data
);
if failure == UpstreamInfoFailure::Unsupported {
// 已运行 v3.40.0 的节点可能持有这种不支持记录,读取时应直接显示为普通 API Key
accounts.records.lock().unwrap().get_mut(&id).unwrap().quota =
serde_json::from_value(serde_json::json!({
"object": "cpr.peer.unavailable",
"schema_version": 1,
"connection_revision": info.connection_revision,
"reason": "unsupported",
}))
.unwrap();
}
drop(bundle);
let restarted = initialize(accounts, transport);
let restored = restarted.service().read_quota(&id).await.unwrap().unwrap();
assert_eq!(
restored.upstream.as_ref().unwrap().discovered,
!clears_identity
);
assert_eq!(restored.windows.is_empty(), clears_data);
if clears_data {
assert_eq!(
restored.upstream.as_ref().unwrap().last_error.as_deref(),
Some(failure.as_str())
);
}
assert_eq!(
restarted
.service()
.read_summary(&id)
.await
.unwrap()
.unwrap()
.upstream,
restored.upstream
);
}
}
#[tokio::test(start_paused = true)]
async fn peer_detection_after_revocation_is_scoped_to_the_connection() {
for failure in [
UpstreamInfoFailure::Unauthorized,
UpstreamInfoFailure::Unsupported,
] {
let (bundle, accounts, transport, id) = fixture(failure);
*transport.failure.lock().unwrap() = None;
bundle.service().refresh_quota(&id).await.unwrap();
tokio::time::advance(Duration::from_secs(15)).await;
*transport.failure.lock().unwrap() = Some(failure.into());
let _result = bundle.service().refresh_quota(&id).await;
let initial = accounts.accounts.lock().unwrap()[0].clone();
let edited = initial.clone().with_notes(Some("保留连接身份".into()));
*accounts.accounts.lock().unwrap() = vec![edited.clone()];
bundle.service().account_changed(&id).await.unwrap();
assert_eq!(
bundle
.service()
.read_summary(&id)
.await
.unwrap()
.unwrap()
.upstream
.unwrap()
.discovered,
failure == UpstreamInfoFailure::Unauthorized
);
// 地址和密钥修改推进凭据版本,出口变化则改变同一版本的连接身份
for replacement in [
edited.with_outbound_proxy(Some(
OutboundProxy::parse("http://other.invalid:8080").unwrap(),
)),
ProviderAccount::new(
id.clone(),
initial.provider().clone(),
"替换连接".into(),
None,
"peer_test".into(),
CredentialRevision::new(2).unwrap(),
None,
)
.with_account_facts(
true,
CredentialState::Ready,
QuotaState::unknown(),
None,
None,
),
] {
*accounts.accounts.lock().unwrap() = vec![replacement];
bundle.service().account_changed(&id).await.unwrap();
let detail = bundle.service().read_quota(&id).await.unwrap().unwrap();
assert!(!detail.upstream.as_ref().unwrap().discovered);
assert!(detail.upstream.as_ref().unwrap().last_error.is_none());
assert_eq!(
bundle
.service()
.read_summary(&id)
.await
.unwrap()
.unwrap()
.upstream,
detail.upstream
);
}
}
}
#[tokio::test(start_paused = true)]
async fn manual_rediscovery_does_not_wait_for_authorization_backoff_but_respects_retry_after() {
for retry_after in [None, Some(Duration::from_secs(90))] {
let (mut bundle, _, transport, id) = fixture(UpstreamInfoFailure::Unauthorized);
*transport.failure.lock().unwrap() = None;
bundle.service().refresh_quota(&id).await.unwrap();
tokio::time::advance(Duration::from_secs(15)).await;
*transport.failure.lock().unwrap() = Some(PeerFetchFailure {
kind: UpstreamInfoFailure::Unauthorized,
retry_after,
});
assert!(bundle.service().refresh_quota(&id).await.is_err());
let worker = worker(&mut bundle);
*transport.failure.lock().unwrap() = None;
let delay = retry_after.unwrap_or(Duration::from_secs(15));
tokio::time::advance(delay - Duration::from_secs(1)).await;
assert!(bundle.service().refresh_quota(&id).await.is_err());
assert_eq!(transport.requests.load(Ordering::SeqCst), 2);
tokio::time::advance(Duration::from_secs(1)).await;
cycle(&worker).await;
assert_eq!(transport.requests.load(Ordering::SeqCst), 2);
let recovered = bundle.service().refresh_quota(&id).await.unwrap();
assert_eq!(transport.requests.load(Ordering::SeqCst), 3);
assert!(recovered.upstream.as_ref().unwrap().discovered);
assert!(recovered.upstream.as_ref().unwrap().last_error.is_none());
assert!(!recovered.windows.is_empty());
}
}
fn fixture(
failure: UpstreamInfoFailure,
) -> (
@@ -3,6 +3,14 @@ use super::TestPluginPorts;
use gateway_admin::{model::usage_data::*, ports::usage_data::*};
#[async_trait::async_trait]
impl UsageDataStore for TestPluginPorts {
async fn open_read(
&self,
_: UsageReadQuery,
_: gateway_core::lifecycle::CancellationToken,
) -> UsageDataResult<Box<dyn UsageReadSession>> {
Err(UsageDataError::unavailable("test usage reader unavailable"))
}
async fn get_record(&self, _: GetUsageRecord) -> UsageDataResult<UsageRecord> {
unreachable!("unexpected usage call")
}
@@ -64,6 +64,7 @@ fn sdk_wire_types_match_host_contracts() {
"value.rs",
"record.rs",
"query.rs",
"read.rs",
"aggregate.rs",
"pricing.rs",
"error.rs",
+1 -1
View File
@@ -12,7 +12,7 @@ name = "main"
path = "tests/main.rs"
[dependencies]
axum = { version = "0.8.9", default-features = false, features = ["json", "multipart", "query", "ws"] }
axum = { version = "0.8.9", default-features = false, features = ["json", "matched-path", "multipart", "query", "ws"] }
base64 = "0.23.0"
bytes = "1.11.1"
chrono = { version = "0.4.45", features = ["serde"] }
@@ -1,5 +1,7 @@
//! 账号管理路由与 HTTP handler 编排
use gateway_core::engine::probe::AccountProbeTarget;
use crate::auth::SessionState;
use super::*;
@@ -626,7 +628,10 @@ where
let stream = state
.admin_services()
.accounts()
.test_connection(account_id, upstream_model)
.test_connection(
AccountProbeTarget::ConnectionTest(account_id),
upstream_model,
)
.await
.map_err(map_service_error)?
.map(move |event| {
@@ -193,6 +193,7 @@ fn quota_forecast_view(
period: match forecast.period {
AccountUsagePeriod::Weekly => "weekly",
AccountUsagePeriod::Monthly => "monthly",
AccountUsagePeriod::Daily => "daily",
},
target_days: forecast.target_seconds as f64 / 86_400.0,
extrapolated: forecast.extrapolated,
@@ -389,6 +390,7 @@ pub(super) fn account_usage_view(
};
AccountUsageView {
window_kind: match period {
Some(AccountUsagePeriod::Daily) => AccountUsageWindowKind::Daily,
Some(AccountUsagePeriod::Weekly) => AccountUsageWindowKind::Weekly,
Some(AccountUsagePeriod::Monthly) => AccountUsageWindowKind::Monthly,
None => AccountUsageWindowKind::Quota,
@@ -453,20 +455,14 @@ pub(super) fn account_model_usage_view(
.costs
.iter()
.find(|cost| cost.currency.eq_ignore_ascii_case("USD"));
let success_rate = usage.health.success_rate();
ModelUsageView {
model: usage.model,
request_count: usage.request_count,
request_count_display: format_number(usage.request_count),
success_rate: (usage.request_count > 0)
.then(|| usage.success_count as f64 * 100.0 / usage.request_count as f64),
success_rate_display: if usage.request_count == 0 {
"—".to_owned()
} else {
format!(
"{:.1}%",
usage.success_count as f64 * 100.0 / usage.request_count as f64
)
},
success_rate,
success_rate_display: success_rate
.map_or_else(|| "—".to_owned(), |rate| format!("{rate:.1}%")),
input_tokens: usage.input_tokens,
input_tokens_display: display_optional_tokens(usage.input_tokens),
output_tokens: usage.output_tokens,
@@ -493,9 +489,11 @@ pub(super) fn account_model_usage_view(
partial_cost_count: u64::from(cost_estimate_status == "partial"),
unknown_cost_count: usage.cost_coverage.unavailable_count,
costs: usage.costs.iter().map(account_currency_cost_view).collect(),
last_used_at: time.rfc3339(&usage.last_used_at),
last_used_at_full_display: Some(time.datetime(&usage.last_used_at)),
last_used_at_display: time.relative(usage.last_used_at, now),
last_used_at: usage.last_used_at.map(|value| time.rfc3339(&value)),
last_used_at_full_display: usage.last_used_at.map(|value| time.datetime(&value)),
last_used_at_display: usage
.last_used_at
.map_or_else(|| "—".to_owned(), |value| time.relative(value, now)),
}
}
@@ -14,6 +14,8 @@ pub struct AccountUpstreamInfoView {
pub stale: bool,
pub connection_revision: String,
pub last_error: Option<String>,
pub max_concurrency: Option<u64>,
pub requests_per_minute: Option<u64>,
pub pool: Option<UpstreamPoolView>,
}
@@ -78,6 +80,8 @@ pub(crate) fn view(
stale: info.stale,
connection_revision: info.connection_revision,
last_error: info.last_error,
max_concurrency: info.max_concurrency,
requests_per_minute: info.requests_per_minute,
pool: info.pool.map(|pool| UpstreamPoolView {
account_count: pool.account_count,
states: UpstreamPoolStatesView {
@@ -515,7 +515,7 @@ pub struct ModelUsageView {
pub partial_cost_count: u64,
pub unknown_cost_count: u64,
pub costs: Vec<CurrencyCostView>,
pub last_used_at: String,
pub last_used_at: Option<String>,
pub last_used_at_display: String,
pub last_used_at_full_display: Option<String>,
}
@@ -260,6 +260,11 @@ pub(crate) fn usage_list_record_view(
UsageListRecordView {
error_code: record.error_code,
error_message: record.error_message,
error_kind: record.error_kind,
websocket_close_code: record.websocket_close_code,
continuation_unavailable_reason: record.continuation_unavailable_reason,
upstream_connection_exit_reason: record.upstream_connection_exit_reason,
recovered: record.recovered,
logical_outcome: record.outcome.as_str().to_owned(),
client_api_key_name: record.client_api_key_name,
id: record.id,
@@ -357,6 +362,23 @@ pub(crate) fn usage_record_view(
.or_else(|| record.requested_model_id.clone());
let metadata = provider_metadata_fields(record.provider_metadata_json.as_deref());
UsageRecordView {
upstream_error: record
.upstream_error
.map(|error| super::wire::UsageUpstreamErrorView {
code: error.code,
error_type: error.error_type,
message: error.message,
param: error.param,
close_reason: error.close_reason,
last_event_type: error.last_event_type,
}),
retry_after_ms: record.retry_after_ms,
error_code: record.provider_error_code,
error_kind: record.error_kind,
websocket_close_code: record.websocket_close_code,
continuation_unavailable_reason: record.continuation_unavailable_reason,
upstream_connection_exit_reason: record.upstream_connection_exit_reason,
recovered: record.recovered,
id: record.id.clone(),
request_id: record.id,
client_api_key_id: Some(record.client_api_key_ref),
@@ -88,6 +88,11 @@ pub struct ImageBillingView {
pub struct UsageListRecordView {
pub error_code: Option<String>,
pub error_message: Option<String>,
pub error_kind: Option<String>,
pub websocket_close_code: Option<u16>,
pub continuation_unavailable_reason: Option<String>,
pub upstream_connection_exit_reason: Option<String>,
pub recovered: bool,
pub logical_outcome: String,
pub client_api_key_name: Option<String>,
pub id: String,
@@ -122,10 +127,30 @@ pub struct UsageListRecordView {
pub user_agent: Option<String>,
}
/// 受控详情中的原始上游错误字段
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
pub struct UsageUpstreamErrorView {
pub code: Option<String>,
pub error_type: Option<String>,
pub message: Option<String>,
pub param: Option<String>,
pub close_reason: Option<String>,
pub last_event_type: Option<String>,
}
/// 单条逻辑请求详情展示
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
pub struct UsageRecordView {
pub upstream_error: Option<UsageUpstreamErrorView>,
pub retry_after_ms: Option<u64>,
pub error_code: Option<String>,
pub error_kind: Option<String>,
pub websocket_close_code: Option<u16>,
pub continuation_unavailable_reason: Option<String>,
pub upstream_connection_exit_reason: Option<String>,
pub recovered: bool,
pub id: String,
pub request_id: String,
pub client_api_key_id: Option<String>,
@@ -537,14 +537,10 @@ async fn probe<S>(
where
S: SessionState + Send + Sync,
{
let proxy = request
.proxy_url
.0
.ok_or_else(|| AdminError::bad_request("代理 URL 不能为空"))?;
let result = state
.admin_services()
.proxies()
.probe(Some(&proxy), request.detect_location)
.probe(request.proxy_url.0.as_ref(), request.detect_location)
.await
.map_err(map_error)?;
Ok(AdminResponse::new(
@@ -9,6 +9,14 @@ use gateway_core::error::{GatewayError, GatewayErrorKind};
use serde::Serialize;
use serde_json::{Value, json};
/// 认证边界提供的稳定拒绝原因,不携带认证输入
#[derive(Clone)]
pub(crate) struct AccessRejection(pub &'static str);
/// 执行已交给 Core 观测,HTTP 入口只补充交付边界日志
#[derive(Clone)]
pub(crate) struct ExecutionObserved;
/// OpenAI 风格的协议错误 JSON
#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
pub struct ProtocolErrorBody {
@@ -107,6 +115,7 @@ pub fn gateway_error_response(error: &GatewayError) -> Response {
error.client_error_code().unwrap_or(default_code),
)
.into_response();
response.extensions_mut().insert(error.clone());
if let Some(delay) = error.retry_after()
&& let Ok(value) = HeaderValue::from_str(
&delay
@@ -0,0 +1,218 @@
//! 数据面 HTTP 入口诊断;执行前失败进入运维事件,不伪造模型请求或上游状态
use std::{
pin::Pin,
sync::Arc,
task::{Context, Poll},
};
use axum::{
body::{Body, Bytes},
extract::{MatchedPath, Request, State},
http::StatusCode,
middleware::Next,
response::Response,
};
use gateway_core::diagnostics::{OperationalDiagnostics, OperationalFailure, TraceContext};
use http_body::{Body as _, Frame, SizeHint};
use serde_json::json;
use tower_http::request_id::RequestId;
use crate::client_error::{AccessRejection, ProtocolError, ProtocolErrorBody};
pub(crate) async fn record_ingress(
State(diagnostics): State<Arc<dyn OperationalDiagnostics>>,
mut request: Request,
next: Next,
) -> Response {
if request.uri().path() != "/v1" && !request.uri().path().starts_with("/v1/") {
return next.run(request).await;
}
let correlation_id = request
.extensions()
.get::<RequestId>()
.and_then(|id| id.header_value().to_str().ok())
.map(str::to_owned);
let endpoint = request
.extensions()
.get::<MatchedPath>()
.map(|path| path.as_str().to_owned());
let method = request.method().clone();
let dump = if tracing::enabled!(target: "request_dump", tracing::Level::INFO) {
let trace = TraceContext::new(correlation_id.as_deref().unwrap_or("ingress"));
trace.dump(
"ingress.request",
&serde_json::to_vec(&json!({
"method":method.as_str(), "uri":request.uri().to_string(),
}))
.unwrap_or_default(),
);
trace.headers(
"ingress.request.headers",
json!({}),
request
.headers()
.iter()
.map(|(name, value)| (name.as_str(), value.as_bytes())),
);
let body = std::mem::replace(request.body_mut(), Body::empty());
*request.body_mut() = Body::new(DumpBody::new(body, trace.clone(), "ingress.request.body"));
Some(trace)
} else {
None
};
let mut response = if endpoint.is_none() {
// 数据面未知路径不能落入 SPA fallback,否则会把协议错误伪装成 HTML 成功
crate::openai::error::protocol_error_response(
StatusCode::NOT_FOUND,
ProtocolErrorBody {
error: ProtocolError {
kind: "invalid_request_error",
code: "unknown_endpoint",
message: "The requested API endpoint does not exist.".into(),
param: None,
},
},
)
} else {
next.run(request).await
};
let protocol = response.extensions_mut().remove::<ProtocolErrorBody>();
let gateway = response
.extensions_mut()
.remove::<gateway_core::error::GatewayError>();
let access = response.extensions_mut().remove::<AccessRejection>();
let execution_observed = response
.extensions_mut()
.remove::<crate::client_error::ExecutionObserved>()
.is_some();
let status = response.status();
if status.is_client_error() || status.is_server_error() {
let code = access
.map(|error| error.0)
.or_else(|| protocol.as_ref().map(|body| body.error.code))
.or_else(|| gateway.as_ref().map(|error| error.kind().as_str()))
.unwrap_or(match status {
StatusCode::UNAUTHORIZED => "invalid_api_key",
StatusCode::METHOD_NOT_ALLOWED => "method_not_allowed",
StatusCode::NOT_FOUND => "not_found",
_ => "http_rejection",
});
// 未知参数名可由客户端任意构造,只保留 decoder 生成的已知参数路径
let parameter = protocol
.as_ref()
.filter(|body| body.error.code != "unknown_parameter")
.and_then(|body| body.error.param.as_deref())
.filter(|path| {
path.len() <= 256
&& path.bytes().all(|byte| {
byte.is_ascii_alphanumeric() || matches!(byte, b'_' | b'.' | b'[' | b']')
})
});
let endpoint = endpoint.as_deref().unwrap_or("/v1/<unmatched>");
let model_request_id = execution_observed
.then(|| {
response
.headers()
.get("x-gateway-request-id")
.and_then(|id| id.to_str().ok())
})
.flatten();
tracing::info!(target: "request_trace", stage="ingress.rejected", status=status.as_u16(),
code, parameter, endpoint, model_request_id, method=method.as_str(), "Data-plane HTTP request rejected");
if !execution_observed {
let mut failure = OperationalFailure::new(
"api",
"http_ingress",
code,
json!({"method":method.as_str(), "endpoint":endpoint, "status":status.as_u16(),
"code":code, "param":parameter})
.to_string(),
);
failure.correlation_id = correlation_id;
failure.details = gateway.as_ref().and_then(|error| error.error_details());
if diagnostics.record_failure(failure).await.is_err() {
tracing::warn!("HTTP ingress diagnostic could not be recorded");
}
}
}
if let Some(trace) = dump {
trace.dump(
"ingress.response",
&serde_json::to_vec(&json!({"status":status.as_u16()})).unwrap_or_default(),
);
trace.headers(
"ingress.response.headers",
json!({"status":status.as_u16()}),
response
.headers()
.iter()
.map(|(name, value)| (name.as_str(), value.as_bytes())),
);
let body = std::mem::replace(response.body_mut(), Body::empty());
*response.body_mut() = Body::new(DumpBody::new(body, trace, "ingress.response.body"));
}
response
}
struct DumpBody {
inner: Body,
trace: TraceContext,
stage: &'static str,
complete: bool,
}
impl DumpBody {
fn new(inner: Body, trace: TraceContext, stage: &'static str) -> Self {
let complete = inner.is_end_stream();
Self {
inner,
trace,
stage,
complete,
}
}
}
impl http_body::Body for DumpBody {
type Data = Bytes;
type Error = axum::Error;
fn poll_frame(
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
) -> Poll<Option<Result<Frame<Bytes>, axum::Error>>> {
let result = Pin::new(&mut self.inner).poll_frame(cx);
if let Poll::Ready(Some(Ok(frame))) = &result
&& let Some(bytes) = frame.data_ref()
{
self.trace.dump(self.stage, bytes);
}
if matches!(result, Poll::Ready(Some(Err(_)))) {
self.complete = false;
} else if matches!(result, Poll::Ready(None)) || self.inner.is_end_stream() {
self.complete = true;
}
result
}
fn is_end_stream(&self) -> bool {
self.inner.is_end_stream()
}
fn size_hint(&self) -> SizeHint {
self.inner.size_hint()
}
}
impl Drop for DumpBody {
fn drop(&mut self) {
// 不为诊断主动读取剩余正文;取消或未消费的请求必须明确标出转储不完整
self.trace.dump(
"ingress.body.finished",
&serde_json::to_vec(&json!({
"stage":self.stage, "complete":self.complete,
}))
.unwrap_or_default(),
);
}
}
+37 -18
View File
@@ -1,4 +1,4 @@
//! `/healthz` 对 Core、Store 与 Host 关键 worker 健康事实的聚合
//! 请求就绪检查与综合健康诊断
use std::sync::{Arc, Mutex};
use std::time::Duration;
@@ -67,22 +67,7 @@ impl HealthStatus {
),
})
.collect();
// 逐个探针共享截止时间,保留已完成结果并指出具体超时组件
let deadline = tokio::time::Instant::now() + HEALTH_CHECK_TIMEOUT;
let checks = self.probes.iter().map(|probe| async move {
let (state, reason) = match tokio::time::timeout_at(deadline, probe.check()).await {
Ok(HealthState::Healthy) => return None,
Ok(HealthState::Degraded(reason)) => ("degraded", reason),
Ok(HealthState::Unhealthy(reason)) => ("unhealthy", reason),
Err(_) => ("timed_out", "health probe exceeded 2s deadline".to_owned()),
};
Some(HealthFailure {
component: format!("probe:{}", probe.name()),
state,
reason,
})
});
failures.extend(join_all(checks).await.into_iter().flatten());
failures.extend(probe_failures(self.probes.iter()).await);
let healthy = failures.is_empty();
let changed = {
let mut previous = self
@@ -128,8 +113,42 @@ impl HealthStatus {
}
}
async fn probe_failures<'a>(
probes: impl Iterator<Item = &'a Arc<dyn HealthProbe>>,
) -> Vec<HealthFailure> {
// 探针并行共用截止时间,保留已完成结果并指出具体超时组件
let deadline = tokio::time::Instant::now() + HEALTH_CHECK_TIMEOUT;
let checks = probes.map(|probe| async move {
let (state, reason) = match tokio::time::timeout_at(deadline, probe.check()).await {
Ok(HealthState::Healthy) => return None,
Ok(HealthState::Degraded(reason)) => ("degraded", reason),
Ok(HealthState::Unhealthy(reason)) => ("unhealthy", reason),
Err(_) => ("timed_out", "health probe exceeded 2s deadline".to_owned()),
};
Some(HealthFailure {
component: format!("probe:{}", probe.name()),
state,
reason,
})
});
join_all(checks).await.into_iter().flatten().collect()
}
pub(crate) async fn readyz(State(state): State<ApiState>) -> StatusCode {
let required = state
.health()
.probes
.iter()
.filter(|probe| probe.required_for_readiness());
status_code(probe_failures(required).await.is_empty())
}
pub(crate) async fn healthz(State(state): State<ApiState>) -> StatusCode {
if state.health().healthy().await {
status_code(state.health().healthy().await)
}
fn status_code(healthy: bool) -> StatusCode {
if healthy {
StatusCode::NO_CONTENT
} else {
StatusCode::SERVICE_UNAVAILABLE
+6
View File
@@ -30,6 +30,7 @@ use crate::openai::service::OpenAiService;
pub mod admin;
pub mod auth;
mod client_error;
mod diagnostics;
mod health;
mod key_usage;
mod middleware;
@@ -216,6 +217,7 @@ pub fn initialize(
};
let index = config.asset_directory.join("index.html");
let mut router = Router::new()
.route("/readyz", get(health::readyz))
.route("/healthz", get(health::healthz))
.merge(openai::router::router())
.merge(provider::router())
@@ -279,6 +281,10 @@ pub fn initialize(
headers_ms = latency.as_millis(), "HTTP response headers ready");
});
let router = router
.layer(axum::middleware::from_fn_with_state(
diagnostics.clone(),
diagnostics::record_ingress,
))
.layer(axum::middleware::from_fn_with_state(
diagnostics,
admin::diagnostics::record_failure,
+11 -2
View File
@@ -128,7 +128,12 @@ fn client_authentication_request(
}
pub(crate) fn client_access_error_response(error: ClientAccessError) -> Response {
match error {
let code = match &error {
ClientAccessError::Authentication(error) => error.reason(),
ClientAccessError::Version(error) if error.current().is_some() => "client_version_too_old",
ClientAccessError::Version(_) => "client_version_unavailable",
};
let mut response = match error {
ClientAccessError::Authentication(error) => {
log_client_api_key_auth_failure(error);
if error == ClientApiKeyAuthError::RuntimeUnavailable {
@@ -146,7 +151,11 @@ pub(crate) fn client_access_error_response(error: ClientAccessError) -> Response
);
client_version_rejection_response(&error).into_response()
}
}
};
response
.extensions_mut()
.insert(crate::client_error::AccessRejection(code));
response
}
/// Desktop 优先于内嵌 CLI/Core;未知客户端及未提供应用版本的手机远程客户端返回 `None`
@@ -1,5 +1,7 @@
//! 将 Chat 请求转换为 Responses 正文,协议选项留在 API,业务 JSON 与上游兼容规则保持原有归属
use std::collections::HashSet;
use serde_json::{Map, Value, json};
use crate::openai::responses::RequestDecodeError;
@@ -27,18 +29,44 @@ pub(in crate::openai) fn convert_request(
("stream".into(), stream.into()),
("store".into(), false.into()),
]);
if let Some(value) = optional(&mut chat, "tools") {
body.insert("tools".into(), function_tools(value)?.into());
}
if let Some(value) = optional(&mut chat, "tool_choice") {
let choice = tool_choice(value, body.get("tools"))?;
body.insert("tool_choice".into(), choice);
if let Some(options) = optional(&mut chat, "web_search_options") {
let tool = web_search_tool(options)?;
// Chat 搜索要求先检索,不能用强制搜索覆盖调用方的函数工具或选择
if let Some(tools) = optional(&mut chat, "tools")
&& !array(tools, "tools")?.is_empty()
{
return Err(RequestDecodeError::UnsupportedCombination {
field: "tools",
other: "web_search_options",
});
}
if let Some(choice) = optional(&mut chat, "tool_choice")
&& choice != "auto"
{
return Err(RequestDecodeError::UnsupportedCombination {
field: "tool_choice",
other: "web_search_options",
});
}
body.insert("tools".into(), Value::Array(vec![tool]));
body.insert("tool_choice".into(), json!({"type": "web_search"}));
} else {
if let Some(value) = optional(&mut chat, "tools") {
body.insert("tools".into(), tools(value)?.into());
}
if let Some(value) = optional(&mut chat, "tool_choice") {
let choice = tool_choice(value, "tool_choice")?;
body.insert("tool_choice".into(), choice);
}
}
if let Some(value) = optional_bool(&mut chat, "parallel_tool_calls", "parallel_tool_calls")? {
body.insert("parallel_tool_calls".into(), value.into());
}
let mut text = Map::new();
let mut text = optional(&mut chat, "text")
.map(|value| object(value, "text"))
.transpose()?
.unwrap_or_default();
if let Some(value) = optional(&mut chat, "response_format") {
text.insert("format".into(), response_format(value)?);
}
@@ -49,47 +77,81 @@ pub(in crate::openai) fn convert_request(
if !text.is_empty() {
body.insert("text".into(), text.into());
}
let mut reasoning = optional(&mut chat, "reasoning")
.map(|value| object(value, "reasoning"))
.transpose()?
.unwrap_or_default();
if let Some(value) = optional(&mut chat, "reasoning_effort") {
string(&value, "reasoning_effort", false)?;
body.insert("reasoning".into(), json!({"effort": value}));
reasoning.insert("effort".into(), value);
}
for field in ["max_tokens", "max_completion_tokens"] {
if !reasoning.is_empty() {
body.insert("reasoning".into(), reasoning.into());
}
for field in ["max_output_tokens", "max_tokens", "max_completion_tokens"] {
if let Some(value) = optional(&mut chat, field) {
if !value.as_u64().is_some_and(|tokens| tokens > 0) {
return Err(invalid(field));
}
// 新字段优先,具体上游是否接受长度控制仍由现有 Provider 决定
body.insert("max_output_tokens".into(), value);
}
}
for (field, maximum) in [("temperature", 2.0), ("top_p", 1.0)] {
if let Some(value) = optional(&mut chat, field) {
if !value
.as_f64()
.is_some_and(|number| (0.0..=maximum).contains(&number))
{
return Err(invalid(field));
}
body.insert(field.into(), value);
}
}
// 同语义字段不在入口预判模型支持范围,交由既有 Provider 与上游校验
for field in [
"temperature",
"top_p",
"user",
"safety_identifier",
"service_tier",
"prompt_cache_key",
"prompt_cache_retention",
"metadata",
"prompt_cache_options",
"moderation",
"instructions",
"include",
"max_tool_calls",
"truncation",
"context_management",
] {
if let Some(value) = optional(&mut chat, field) {
string(&value, field, true)?;
body.insert(field.into(), value);
}
}
for field in ["metadata", "prompt_cache_options", "moderation"] {
if let Some(value) = optional(&mut chat, field) {
let object = object(value, field)?;
body.insert(field.into(), object.into());
if optional_bool(&mut chat, "logprobs", "logprobs")? == Some(true) {
let mut include = body
.remove("include")
.map(|value| array(value, "include"))
.transpose()?
.unwrap_or_default();
let logprobs = json!("message.output_text.logprobs");
if !include.contains(&logprobs) {
include.push(logprobs);
}
body.insert("include".into(), include.into());
if let Some(value) = optional(&mut chat, "top_logprobs") {
body.insert("top_logprobs".into(), value);
}
}
let ignored = [
"stop",
"seed",
"frequency_penalty",
"presence_penalty",
"logit_bias",
"prediction",
]
.into_iter()
.filter(|field| {
chat.get(*field).is_some_and(|value| {
!value.is_null()
&& value.as_f64() != Some(0.0)
&& !value.as_array().is_some_and(Vec::is_empty)
&& !value.as_object().is_some_and(Map::is_empty)
})
})
.collect::<Vec<_>>();
if !ignored.is_empty() {
tracing::info!(target: "request_trace", stage="chat.compatibility", ignored_options=?ignored,
"Chat-only optional controls omitted from Responses request");
}
// 未识别的客户端顶层扩展不等于 Responses 参数,不进入上游正文
Ok(ConvertedChatRequest {
@@ -102,27 +164,10 @@ fn validate_options(chat: &Map<String, Value>) -> Result<(), RequestDecodeError>
for (field, value) in chat.iter().filter(|(_, value)| !value.is_null()) {
let supported = match field.as_str() {
"n" => value.as_u64() == Some(1),
"store" | "logprobs" => value == false,
"frequency_penalty" | "presence_penalty" => value.as_f64() == Some(0.0),
"top_logprobs" => value.as_u64() == Some(0),
"logit_bias" => value.as_object().is_some_and(Map::is_empty),
"functions" | "stop" => value.as_array().is_some_and(Vec::is_empty),
"store" | "background" => value == false,
"functions" => value.as_array().is_some_and(Vec::is_empty),
"modalities" => value == &json!(["text"]),
"audio"
| "prediction"
| "seed"
| "function_call"
| "web_search_options"
| "reasoning_content"
| "input"
| "previous_response_id"
| "conversation"
| "instructions"
| "reasoning"
| "text"
| "include"
| "background"
| "max_output_tokens" => false,
"audio" | "function_call" | "input" | "previous_response_id" | "conversation" => false,
_ => true,
};
if !supported {
@@ -132,6 +177,33 @@ fn validate_options(chat: &Map<String, Value>) -> Result<(), RequestDecodeError>
Ok(())
}
fn web_search_tool(value: Value) -> Result<Value, RequestDecodeError> {
let mut options = object(value, "web_search_options")?;
let mut tool = json!({"type": "web_search"});
if let Some(size) = optional(&mut options, "search_context_size") {
string(&size, "web_search_options.search_context_size", false)?;
tool["search_context_size"] = size;
}
if let Some(location) = optional(&mut options, "user_location") {
let mut location = object(location, "web_search_options.user_location")?;
let path = "web_search_options.user_location.type";
if required_string(&mut location, "type", path, false)? != "approximate" {
return Err(unsupported(path));
}
let path = "web_search_options.user_location.approximate";
let mut approximate = object(required(&mut location, "approximate", path)?, path)?;
let mut flat = json!({"type": "approximate"});
for field in ["city", "country", "region", "timezone"] {
if let Some(value) = optional(&mut approximate, field) {
string(&value, &format!("{path}.{field}"), true)?;
flat[field] = value;
}
}
tool["user_location"] = flat;
}
Ok(tool)
}
fn stream_options(value: Option<Value>, stream: bool) -> Result<bool, RequestDecodeError> {
let Some(value) = value else {
return Ok(false);
@@ -166,6 +238,7 @@ fn messages(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
});
}
let mut input = Vec::with_capacity(messages.len());
let mut custom_calls = HashSet::new();
for (index, value) in messages.into_iter().enumerate() {
let path = format!("messages[{index}]");
let mut message = object(value, &path)?;
@@ -176,12 +249,13 @@ fn messages(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
) {
return Err(unsupported(&format!("{path}.role")));
}
if let Some(name) = optional(&mut message, "name")
&& !string(&name, &format!("{path}.name"), true)?.is_empty()
{
return Err(unsupported(&format!("{path}.name")));
let name = optional(&mut message, "name").filter(|name| name != "");
if let Some(name) = &name {
let name_path = format!("{path}.name");
string(name, &name_path, true)?;
}
for field in ["function_call", "reasoning_content", "reasoning", "audio"] {
// Chat 的显示名称和明文推理历史没有对应的 Responses 输入语义
for field in ["function_call", "audio"] {
if optional(&mut message, field).is_some() {
return Err(unsupported(&format!("{path}.{field}")));
}
@@ -209,9 +283,16 @@ fn messages(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
} else {
content_parts(content, &role, &content_path, false)?.into()
};
input.push(
json!({"type": "function_call_output", "call_id": call_id, "output": output}),
);
let kind = if custom_calls.contains(&call_id) {
"custom_tool_call_output"
} else {
"function_call_output"
};
let mut item = json!({"type": kind, "call_id": call_id, "output": output});
if let Some(name) = name {
item["name"] = name;
}
input.push(item);
continue;
}
@@ -245,7 +326,15 @@ fn messages(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
}
input.push(item);
}
input.extend(calls);
for call in calls {
let id = string(&call["call_id"], &format!("{path}.tool_calls.id"), false)?.to_owned();
if call["type"] == "custom_tool_call" {
custom_calls.insert(id);
} else {
custom_calls.remove(&id);
}
input.push(call);
}
}
Ok(input)
}
@@ -267,14 +356,23 @@ fn content_parts(
let mut part = object(value, &path)?;
let kind = required_string(&mut part, "type", &format!("{path}.type"), false)?;
let mut converted = match kind.as_str() {
"text" => text_part(
"text" | "input_text" | "output_text" => text_part(
required_string(&mut part, "text", &format!("{path}.text"), true)?,
is_assistant,
),
"image_url" if role == "user" => image_part(
"image_url" if matches!(role, "user" | "tool") => image_part(
required(&mut part, "image_url", &format!("{path}.image_url"))?,
&format!("{path}.image_url"),
)?,
"file" if matches!(role, "user" | "tool") => {
let file_path = format!("{path}.file");
file_part(required(&mut part, "file", &file_path)?, &file_path)?
}
"input_audio" if matches!(role, "user" | "tool") => audio_part(&mut part, &path)?,
"input_image" | "input_file" if matches!(role, "user" | "tool") => {
part.insert("type".into(), kind.into());
Value::Object(std::mem::take(&mut part))
}
"refusal" if role == "assistant" => json!({
"type": "refusal",
"refusal": required_string(&mut part, "refusal", &format!("{path}.refusal"), true)?,
@@ -301,6 +399,34 @@ fn text_part(text: String, is_assistant: bool) -> Value {
}
}
fn file_part(value: Value, path: &str) -> Result<Value, RequestDecodeError> {
let mut file = object(value, path)?;
if !["file_id", "file_url", "file_data"]
.iter()
.any(|field| file.get(*field).is_some_and(|value| !value.is_null()))
{
return Err(invalid(path));
}
// 文件可用性、格式和组合由接收文件的上游判断,网关不下载或重编码
file.insert("type".into(), "input_file".into());
Ok(file.into())
}
fn audio_part(part: &mut Map<String, Value>, path: &str) -> Result<Value, RequestDecodeError> {
if let Some(value) = optional(part, "input_audio") {
let path = format!("{path}.input_audio");
let mut audio = object(value, &path)?;
let data = required_string(&mut audio, "data", &format!("{path}.data"), false)?;
let format = required_string(&mut audio, "format", &format!("{path}.format"), false)?;
let mime = if format == "mp3" { "mpeg" } else { &format };
Ok(json!({"type":"input_audio", "audio_url":format!("data:audio/{mime};base64,{data}")}))
} else {
// Codex 使用 audio_url/file_id 引用,原生扩展保持其结构
part.insert("type".into(), "input_audio".into());
Ok(Value::Object(std::mem::take(part)))
}
}
fn image_part(value: Value, path: &str) -> Result<Value, RequestDecodeError> {
let mut image = object(value, path)?;
let url = required_string(&mut image, "url", &format!("{path}.url"), false)?;
@@ -324,104 +450,82 @@ fn image_part(value: Value, path: &str) -> Result<Value, RequestDecodeError> {
}
fn tool_calls(value: Value, path: &str) -> Result<Vec<Value>, RequestDecodeError> {
let calls = array(value, path)?;
let mut result = Vec::with_capacity(calls.len());
for (index, value) in calls.into_iter().enumerate() {
let path = format!("{path}[{index}]");
let mut call = object(value, &path)?;
let id = required_string(&mut call, "id", &format!("{path}.id"), false)?;
if required_string(&mut call, "type", &format!("{path}.type"), false)? != "function" {
return Err(unsupported(&format!("{path}.type")));
}
let function_path = format!("{path}.function");
let mut function = object(
required(&mut call, "function", &function_path)?,
&function_path,
)?;
let name = required_string(
&mut function,
"name",
&format!("{function_path}.name"),
false,
)?;
let arguments = required_string(
&mut function,
"arguments",
&format!("{function_path}.arguments"),
true,
)?;
result.push(
json!({"type": "function_call", "call_id": id, "name": name, "arguments": arguments}),
);
}
Ok(result)
array(value, path)?
.into_iter()
.enumerate()
.map(|(index, value)| {
let path = format!("{path}[{index}]");
let mut call = object(value, &path)?;
let id = required_string(&mut call, "id", &format!("{path}.id"), false)?;
let kind = required_string(&mut call, "type", &format!("{path}.type"), false)?;
let (output_type, data_field) = match kind.as_str() {
"function" => ("function_call", "arguments"),
"custom" => ("custom_tool_call", "input"),
_ => return Err(unsupported(&format!("{path}.type"))),
};
let nested_path = format!("{path}.{kind}");
let mut nested = object(required(&mut call, &kind, &nested_path)?, &nested_path)?;
let name = required_string(&mut nested, "name", &format!("{nested_path}.name"), false)?;
let data = required_string(
&mut nested,
data_field,
&format!("{nested_path}.{data_field}"),
true,
)?;
Ok(json!({"type":output_type, "call_id":id, "name":name, data_field:data}))
})
.collect()
}
fn function_tools(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
fn tools(value: Value) -> Result<Vec<Value>, RequestDecodeError> {
array(value, "tools")?
.into_iter()
.enumerate()
.map(|(index, value)| {
let path = format!("tools[{index}]");
let mut tool = object(value, &path)?;
if required_string(&mut tool, "type", &format!("{path}.type"), false)? != "function" {
return Err(unsupported(&format!("{path}.type")));
let kind = required_string(&mut tool, "type", &format!("{path}.type"), false)?;
if matches!(kind.as_str(), "function" | "custom") {
if let Some(nested) = optional(&mut tool, &kind) {
tool = object(nested, &format!("{path}.{kind}"))?;
}
if kind == "function" && tool.get("strict").is_none_or(Value::is_null) {
// Chat 的缺省是宽松 schema,不能让 Responses 收紧参数合同
tool.insert("strict".into(), false.into());
}
}
let function_path = format!("{path}.function");
let mut function = object(
required(&mut tool, "function", &function_path)?,
&function_path,
)?;
let name = required_string(
&mut function,
"name",
&format!("{function_path}.name"),
false,
)?;
// Chat 的缺省是宽松函数 schema,不能让 Responses 的缺省收紧原始参数合同
let strict =
optional_bool(&mut function, "strict", &format!("{function_path}.strict"))?
.unwrap_or(false);
let mut converted = json!({"type": "function", "name": name, "strict": strict});
if let Some(description) = optional(&mut function, "description") {
string(&description, &format!("{function_path}.description"), true)?;
converted["description"] = description;
}
if let Some(parameters) = optional(&mut function, "parameters") {
converted["parameters"] = parameters;
}
Ok(converted)
// hosted 工具采用原生 Responses 结构,支持范围由选中的 Provider 决定
tool.insert("type".into(), kind.into());
Ok(tool.into())
})
.collect()
}
fn tool_choice(value: Value, tools: Option<&Value>) -> Result<Value, RequestDecodeError> {
let tools = tools.and_then(Value::as_array);
if let Value::String(choice) = &value {
return match choice.as_str() {
"auto" | "none" => Ok(value),
"required" if tools.is_some_and(|tools| !tools.is_empty()) => Ok(value),
"required" => Err(invalid("tool_choice")),
_ => Err(unsupported("tool_choice")),
};
fn tool_choice(value: Value, path: &str) -> Result<Value, RequestDecodeError> {
if value.is_string() {
return Ok(value);
}
let mut choice = object(value, "tool_choice")?;
if required_string(&mut choice, "type", "tool_choice.type", false)? != "function" {
return Err(unsupported("tool_choice.type"));
let mut choice = object(value, path)?;
let kind = required_string(&mut choice, "type", &format!("{path}.type"), false)?;
if matches!(kind.as_str(), "function" | "custom") {
if let Some(nested) = optional(&mut choice, &kind) {
choice = object(nested, &format!("{path}.{kind}"))?;
}
} else if kind == "allowed_tools" {
if let Some(nested) = optional(&mut choice, "allowed_tools") {
choice = object(nested, &format!("{path}.allowed_tools"))?;
}
if let Some(value) = optional(&mut choice, "tools") {
let mapped = array(value, &format!("{path}.tools"))?
.into_iter()
.enumerate()
.map(|(index, value)| tool_choice(value, &format!("{path}.tools[{index}]")))
.collect::<Result<Vec<_>, _>>()?;
choice.insert("tools".into(), mapped.into());
}
}
let mut function = object(
required(&mut choice, "function", "tool_choice.function")?,
"tool_choice.function",
)?;
let name = required_string(&mut function, "name", "tool_choice.function.name", false)?;
if !tools.is_some_and(|tools| {
tools
.iter()
.any(|tool| tool.get("name").and_then(Value::as_str) == Some(name.as_str()))
}) {
return Err(invalid("tool_choice.function.name"));
}
Ok(json!({"type": "function", "name": name}))
choice.insert("type".into(), kind.into());
Ok(choice.into())
}
fn response_format(value: Value) -> Result<Value, RequestDecodeError> {
@@ -460,7 +564,10 @@ fn response_format(value: Value) -> Result<Value, RequestDecodeError> {
}
Ok(converted)
}
_ => Err(unsupported("response_format.type")),
_ => {
format.insert("type".into(), kind.into());
Ok(format.into())
}
}
}
@@ -7,6 +7,7 @@ use gateway_protocol::openai::events::{billable_usage_is_complete, extract_usage
use gateway_protocol::openai::sse::encode_sse_event;
use serde::Serialize;
use serde_json::{Value, json};
use sha2::{Digest as _, Sha256};
use crate::client_error::{ProtocolError, ProtocolErrorBody};
@@ -46,26 +47,52 @@ pub(in crate::openai) fn project_response(
let mut content = None;
let mut refusal = None;
let mut tools = Vec::new();
let mut annotations = Vec::new();
let mut content_chars = 0;
let mut reasoning = Vec::new();
let mut images = Vec::new();
let mut logprobs = None;
for item in output {
match required_str(item, "type")? {
"message" => {
// 两侧索引都按消息中的字符计数,合并多条 assistant 消息时平移引用
for part in message_parts(item)? {
let (kind, text) = part_text(part)?;
let target = match kind {
PartKind::Text => &mut content,
PartKind::Text => {
append_url_citations(part, content_chars, &mut annotations)?;
if let Some(values) = part.get("logprobs").and_then(Value::as_array) {
logprobs
.get_or_insert_with(Vec::new)
.extend(values.iter().cloned());
}
content_chars += text.chars().count() as u64;
&mut content
}
PartKind::Refusal => &mut refusal,
};
target.get_or_insert_with(String::new).push_str(text);
}
}
"function_call" => {
let (id, name, arguments) = function_fields(item)?;
tools.push(json!({
"id": id, "type": "function",
"function": {"name": name, "arguments": arguments}
}));
"function_call" | "custom_tool_call" => {
let kind = ToolKind::from_item(item)?;
let (id, name, data) = tool_fields(item, kind)?;
tools.push(json!({"id":id, "type":kind.chat_type(),
kind.chat_type():{"name":name, kind.data_field():data}}));
}
"reasoning" => {}
"reasoning" => {
for (_, _, text) in reasoning_parts(item)? {
if !text.is_empty() {
reasoning.push(text);
}
}
}
"image_generation_call" => {
if let Some(image) = image_payload(item)? {
images.push(image);
}
}
kind if is_auxiliary_output(kind) => {}
_ => return Err(invalid_response()),
}
}
@@ -74,10 +101,20 @@ pub(in crate::openai) fn project_response(
if !tools.is_empty() {
message["tool_calls"] = Value::Array(tools);
}
if !annotations.is_empty() {
message["annotations"] = Value::Array(annotations);
}
if !reasoning.is_empty() {
message["reasoning_content"] = reasoning.join("\n\n").into();
}
if !images.is_empty() {
message["images"] = images.into();
}
let logprobs = logprobs.map(|content| json!({"content":content, "refusal":null}));
serde_json::to_value(Envelope {
identity,
object: "chat.completion",
choices: json!([{"index": 0, "message": message, "finish_reason": finish_reason, "logprobs": null}]),
choices: json!([{"index": 0, "message": message, "finish_reason": finish_reason, "logprobs": logprobs}]),
usage: chat_usage(response),
metadata: response.get("metadata"),
service_tier: response.get("service_tier"),
@@ -104,9 +141,47 @@ impl PartKind {
struct Part {
kind: PartKind,
text: String,
logprobs: Vec<Value>,
annotations: BTreeMap<u64, Value>,
spans: Vec<TextSpan>,
}
struct TextSpan {
start: u64,
end: u64,
delivered_start: u64,
}
#[derive(Clone, Copy, PartialEq, Eq)]
enum ToolKind {
Function,
Custom,
}
impl ToolKind {
fn from_item(item: &Value) -> Result<Self, ProtocolErrorBody> {
match required_str(item, "type")? {
"function_call" => Ok(Self::Function),
"custom_tool_call" => Ok(Self::Custom),
_ => Err(invalid_response()),
}
}
const fn chat_type(self) -> &'static str {
match self {
Self::Function => "function",
Self::Custom => "custom",
}
}
const fn data_field(self) -> &'static str {
match self {
Self::Function => "arguments",
Self::Custom => "input",
}
}
}
struct Tool {
kind: ToolKind,
index: usize,
call_id: String,
name: String,
@@ -115,8 +190,13 @@ struct Tool {
enum Output {
Message(BTreeMap<u64, Part>),
Function(Tool),
Reasoning,
Tool(Tool),
Reasoning(BTreeMap<(bool, u64), String>),
Image {
index: usize,
fingerprint: Option<[u8; 32]>,
},
Auxiliary(String),
}
struct OutputItem {
@@ -137,6 +217,7 @@ pub(in crate::openai) struct ChatStreamEncoder {
role_sent: bool,
output: BTreeMap<u64, OutputItem>,
terminal: Option<Terminal>,
content_chars: u64,
}
impl ChatStreamEncoder {
@@ -147,6 +228,7 @@ impl ChatStreamEncoder {
role_sent: false,
output: BTreeMap::new(),
terminal: None,
content_chars: 0,
}
}
@@ -184,11 +266,7 @@ impl ChatStreamEncoder {
PartKind::Text
};
let delta = required_str(data, "delta")?;
let part = self.event_part(data, kind)?;
part.text.push_str(delta);
if !delta.is_empty() {
frames.push(self.delta(json!({kind.field(): delta}))?);
}
self.text_event(data, kind, delta, false, data, &mut frames)?;
}
Some("response.output_text.done" | "response.refusal.done") => {
self.require_open()?;
@@ -197,22 +275,29 @@ impl ChatStreamEncoder {
} else {
(PartKind::Text, "text")
};
let text = required_str(data, field)?;
let part = self.event_part(data, kind)?;
let suffix = append_snapshot(&mut part.text, text)?;
if !suffix.is_empty() {
frames.push(self.delta(json!({kind.field(): suffix}))?);
}
self.text_event(
data,
kind,
required_str(data, field)?,
true,
data,
&mut frames,
)?;
}
Some("response.content_part.added" | "response.content_part.done") => {
self.require_open()?;
let part = data.get("part").ok_or_else(invalid_response)?;
let (kind, text) = part_text(part)?;
let part = self.event_part(data, kind)?;
let suffix = append_snapshot(&mut part.text, text)?;
if !suffix.is_empty() {
frames.push(self.delta(json!({kind.field(): suffix}))?);
}
self.text_event(data, kind, text, true, part, &mut frames)?;
}
Some("response.output_text.annotation.added") => {
self.require_open()?;
let part = self.event_part(data, PartKind::Text)?;
insert_annotation(
part,
required_index(data, "annotation_index")?,
data.get("annotation").ok_or_else(invalid_response)?,
)?;
}
Some("response.output_item.added" | "response.output_item.done") => {
self.require_open()?;
@@ -224,27 +309,73 @@ impl ChatStreamEncoder {
)?;
}
Some(
"response.function_call_arguments.delta" | "response.function_call_arguments.done",
"response.function_call_arguments.delta"
| "response.function_call_arguments.done"
| "response.custom_tool_call_input.delta"
| "response.custom_tool_call_input.done",
) => {
self.require_open()?;
let index = required_index(data, "output_index")?;
let item = self.output.get_mut(&index).ok_or_else(invalid_response)?;
bind_id(item, optional_id(data, "item_id")?)?;
let Output::Function(tool) = &mut item.output else {
let Output::Tool(tool) = &mut item.output else {
return Err(invalid_response());
};
let delta = if event_type == Some("response.function_call_arguments.delta") {
let kind = tool.kind;
let custom_event = event_type
.is_some_and(|event| event.starts_with("response.custom_tool_call_input."));
if custom_event != (kind == ToolKind::Custom) {
return Err(invalid_response());
}
let delta = if event_type.is_some_and(|event| event.ends_with(".delta")) {
let delta = required_str(data, "delta")?;
tool.arguments.push_str(delta);
delta
} else {
append_snapshot(&mut tool.arguments, required_str(data, "arguments")?)?
append_snapshot(&mut tool.arguments, required_str(data, kind.data_field())?)?
};
let tool_index = tool.index;
if !delta.is_empty() {
frames.push(self.delta(json!({"tool_calls": [{"index": tool_index, "function": {"arguments": delta}}]}))?);
frames.push(self.delta(json!({"tool_calls": [{"index": tool_index, kind.chat_type(): {kind.data_field(): delta}}]}))?);
}
}
Some(
"response.reasoning_summary_text.delta"
| "response.reasoning_summary_text.done"
| "response.reasoning_text.delta"
| "response.reasoning_text.done",
) => {
self.require_open()?;
let summary =
event_type.is_some_and(|event| event.starts_with("response.reasoning_summary"));
let snapshot = event_type.is_some_and(|event| event.ends_with(".done"));
self.reasoning_text(
required_index(data, "output_index")?,
optional_id(data, "item_id")?,
(
summary,
required_index(
data,
if summary {
"summary_index"
} else {
"content_index"
},
)?,
),
required_str(data, if snapshot { "text" } else { "delta" })?,
snapshot,
&mut frames,
)?;
}
Some("response.image_generation_call.partial_image") => {
self.require_open()?;
self.item_snapshot(required_index(data, "output_index")?, &json!({
"id": required_nonempty_str(data, "item_id")?, "type":"image_generation_call",
"result": required_str(data, "partial_image_b64")?,
"output_format": data.get("output_format").unwrap_or(&json!("png")),
}), false, &mut frames)?;
}
Some("response.completed" | "response.incomplete") => {
self.require_open()?;
let response = data.get("response").ok_or_else(invalid_response)?;
@@ -263,6 +394,10 @@ impl ChatStreamEncoder {
} else if self.output.is_empty() {
return Err(invalid_response());
}
let annotations = self.stream_annotations()?;
if !annotations.is_empty() {
frames.push(self.delta(json!({"annotations":annotations}))?);
}
let finish_reason = finish_reason(response, self.tool_count() > 0)?;
frames.push(self.chunk(
json!([{"index": 0, "delta": {}, "finish_reason": finish_reason, "logprobs": null}]),
@@ -298,7 +433,7 @@ impl ChatStreamEncoder {
// 辅助事件可随上游演进;实际 output item 和 content part 仍逐项检查可表达性
_ => {}
}
// done-only 流也要提供完整 Chat 消息身份,辅助事件或私有推理不触发角色帧
// done-only 流也要提供完整 Chat 消息身份,不含可交付内容的辅助事件不触发角色帧
if !self.role_sent && !frames.is_empty() && !self.has_wire_failure() {
frames.insert(0, self.delta(json!({"role": "assistant"}))?);
self.role_sent = true;
@@ -306,6 +441,106 @@ impl ChatStreamEncoder {
Ok(frames)
}
fn text_event(
&mut self,
data: &Value,
kind: PartKind,
text: &str,
snapshot: bool,
metadata: &Value,
frames: &mut Vec<Bytes>,
) -> Result<(), ProtocolErrorBody> {
let mut content_chars = self.content_chars;
let part = self.event_part(data, kind)?;
let delta = if snapshot {
append_snapshot(&mut part.text, text)?
} else {
part.text.push_str(text);
text
};
record_text_span(part, &mut content_chars, delta);
merge_annotations(part, metadata)?;
let logprobs = append_logprobs(part, metadata, snapshot)?;
self.content_chars = content_chars;
if !delta.is_empty() {
frames.push(self.delta(json!({kind.field():delta}))?);
}
if !logprobs.is_empty() {
frames.push(logprobs_frame(
&self.identity,
self.include_usage,
kind,
logprobs,
)?);
}
Ok(())
}
fn stream_annotations(&self) -> Result<Vec<Value>, ProtocolErrorBody> {
let mut result = Vec::new();
for item in self.output.values() {
if let Output::Message(parts) = &item.output {
for part in parts.values() {
let mut citations = Vec::new();
append_url_citations(
&json!({"annotations":part.annotations.values().collect::<Vec<_>>()}),
0,
&mut citations,
)?;
for mut citation in citations {
for field in ["start_index", "end_index"] {
let position = required_index(&citation["url_citation"], field)?;
citation["url_citation"][field] =
delivered_position(part, position, field == "end_index")?.into();
}
result.push(citation);
}
}
}
}
Ok(result)
}
fn reasoning_text(
&mut self,
index: u64,
id: Option<&str>,
part: (bool, u64),
text: &str,
snapshot: bool,
frames: &mut Vec<Bytes>,
) -> Result<(), ProtocolErrorBody> {
let has_text = self.output.values().any(|item| {
matches!(&item.output,
Output::Reasoning(parts) if parts.values().any(|text| !text.is_empty()))
});
let item = self.output.entry(index).or_insert_with(|| OutputItem {
id: None,
output: Output::Reasoning(BTreeMap::new()),
});
bind_id(item, id)?;
let Output::Reasoning(parts) = &mut item.output else {
return Err(invalid_response());
};
let previous = parts.entry(part).or_default();
let separator = previous.is_empty() && has_text;
let delta = if snapshot {
append_snapshot(previous, text)?
} else {
previous.push_str(text);
text
};
if !delta.is_empty() {
let delta = if separator {
format!("\n\n{delta}")
} else {
delta.to_owned()
};
frames.push(self.delta(json!({"reasoning_content":delta}))?);
}
Ok(())
}
fn require_open(&self) -> Result<(), ProtocolErrorBody> {
if self.terminal.is_some() {
Err(invalid_response())
@@ -317,7 +552,7 @@ impl ChatStreamEncoder {
fn tool_count(&self) -> usize {
self.output
.values()
.filter(|item| matches!(item.output, Output::Function(_)))
.filter(|item| matches!(item.output, Output::Tool(_)))
.count()
}
@@ -342,7 +577,7 @@ impl ChatStreamEncoder {
) -> Result<(), ProtocolErrorBody> {
let id = optional_id(snapshot, "id")?;
let kind = required_str(snapshot, "type")?;
let call_id = if kind == "function_call" {
let call_id = if matches!(kind, "function_call" | "custom_tool_call") {
Some(required_nonempty_str(snapshot, "call_id")?)
} else {
None
@@ -356,12 +591,23 @@ impl ChatStreamEncoder {
if !self.output.contains_key(&index) {
let output = match kind {
"message" => Output::Message(BTreeMap::new()),
"reasoning" => Output::Reasoning,
"function_call" => {
let (call_id, name, _) = function_fields(snapshot)?;
"reasoning" => Output::Reasoning(BTreeMap::new()),
"image_generation_call" => Output::Image {
index: self
.output
.values()
.filter(|item| matches!(item.output, Output::Image { .. }))
.count(),
fingerprint: None,
},
kind if is_auxiliary_output(kind) => Output::Auxiliary(kind.to_owned()),
"function_call" | "custom_tool_call" => {
let kind = ToolKind::from_item(snapshot)?;
let (call_id, name, _) = tool_fields(snapshot, kind)?;
let tool_index = self.tool_count();
frames.push(self.delta(json!({"tool_calls": [{"index": tool_index, "id": call_id, "type": "function", "function": {"name": name, "arguments": ""}}]}))?);
Output::Function(Tool {
frames.push(self.delta(json!({"tool_calls": [{"index": tool_index, "id": call_id, "type": kind.chat_type(), kind.chat_type(): {"name": name, kind.data_field(): ""}}]}))?);
Output::Tool(Tool {
kind,
index: tool_index,
call_id: call_id.to_owned(),
name: name.to_owned(),
@@ -378,8 +624,20 @@ impl ChatStreamEncoder {
(Output::Message(parts), "message") => {
for (index, part) in message_parts(snapshot)?.iter().enumerate() {
let (kind, text) = part_text(part)?;
let snapshot = part;
let part = part_entry(parts, index as u64, kind)?;
let suffix = append_snapshot(&mut part.text, text)?;
record_text_span(part, &mut self.content_chars, suffix);
merge_annotations(part, snapshot)?;
let logprobs = append_logprobs(part, snapshot, true)?;
if !logprobs.is_empty() {
frames.push(logprobs_frame(
&self.identity,
self.include_usage,
kind,
logprobs,
)?);
}
if !suffix.is_empty() {
frames.push(delta_frame(
&self.identity,
@@ -389,17 +647,41 @@ impl ChatStreamEncoder {
}
}
}
(Output::Function(tool), "function_call") => {
let (id, name, arguments) = function_fields(snapshot)?;
(Output::Tool(tool), "function_call" | "custom_tool_call") => {
let kind = ToolKind::from_item(snapshot)?;
if kind != tool.kind {
return Err(invalid_response());
}
let (id, name, arguments) = tool_fields(snapshot, kind)?;
if tool.call_id != id || tool.name != name {
return Err(invalid_response());
}
let suffix = append_snapshot(&mut tool.arguments, arguments)?;
if !suffix.is_empty() {
frames.push(delta_frame(&self.identity, self.include_usage, json!({"tool_calls": [{"index": tool.index, "function": {"arguments": suffix}}]}))?);
frames.push(delta_frame(&self.identity, self.include_usage, json!({"tool_calls": [{"index": tool.index, kind.chat_type(): {kind.data_field(): suffix}}]}))?);
}
}
(Output::Reasoning, "reasoning") => {}
(Output::Reasoning(_), "reasoning") => {
for (summary, part_index, text) in reasoning_parts(snapshot)? {
self.reasoning_text(index, id, (summary, part_index), text, true, frames)?;
}
}
(Output::Image { index, fingerprint }, "image_generation_call") => {
if let Some(mut image) = image_payload(snapshot)? {
let hash: [u8; 32] =
Sha256::digest(required_str(&image["image_url"], "url")?.as_bytes()).into();
if *fingerprint != Some(hash) {
*fingerprint = Some(hash);
image["index"] = (*index).into();
frames.push(delta_frame(
&self.identity,
self.include_usage,
json!({"images":[image]}),
)?);
}
}
}
(Output::Auxiliary(previous), kind) if previous == kind => {}
_ => return Err(invalid_response()),
}
Ok(())
@@ -413,7 +695,7 @@ impl ChatStreamEncoder {
let mut found = None;
for (&index, item) in &self.output {
let same_id = id.is_some() && item.id.as_deref() == id;
let same_call = matches!(&item.output, Output::Function(tool) if call_id == Some(tool.call_id.as_str()));
let same_call = matches!(&item.output, Output::Tool(tool) if call_id == Some(tool.call_id.as_str()));
if same_id || same_call {
if found.is_some_and(|previous| previous != index) {
return Err(invalid_response());
@@ -457,6 +739,9 @@ fn part_entry(
let part = parts.entry(index).or_insert_with(|| Part {
kind,
text: String::new(),
logprobs: Vec::new(),
annotations: BTreeMap::new(),
spans: Vec::new(),
});
if part.kind != kind {
return Err(invalid_response());
@@ -464,6 +749,68 @@ fn part_entry(
Ok(part)
}
fn record_text_span(part: &mut Part, delivered: &mut u64, delta: &str) {
if part.kind != PartKind::Text || delta.is_empty() {
return;
}
let length = delta.chars().count() as u64;
let start = part.spans.last().map_or(0, |span| span.end);
if let Some(last) = part.spans.last_mut()
&& last.delivered_start + last.end - last.start == *delivered
{
last.end += length;
} else {
part.spans.push(TextSpan {
start,
end: start + length,
delivered_start: *delivered,
});
}
*delivered += length;
}
fn delivered_position(part: &Part, position: u64, end: bool) -> Result<u64, ProtocolErrorBody> {
let matches = |span: &&TextSpan| {
if end && position > 0 {
span.start < position && position <= span.end
} else {
span.start <= position && position < span.end
}
};
let span = part
.spans
.iter()
.find(matches)
.or_else(|| part.spans.last().filter(|span| position == span.end))
.ok_or_else(invalid_response)?;
Ok(span.delivered_start + position - span.start)
}
fn merge_annotations(part: &mut Part, value: &Value) -> Result<(), ProtocolErrorBody> {
if let Some(values) = value.get("annotations").filter(|value| !value.is_null()) {
for (index, annotation) in values
.as_array()
.ok_or_else(invalid_response)?
.iter()
.enumerate()
{
insert_annotation(part, index as u64, annotation)?;
}
}
Ok(())
}
fn insert_annotation(part: &mut Part, index: u64, value: &Value) -> Result<(), ProtocolErrorBody> {
if let Some(previous) = part.annotations.get(&index) {
if previous != value {
return Err(invalid_response());
}
} else {
part.annotations.insert(index, value.clone());
}
Ok(())
}
fn bind_id(item: &mut OutputItem, id: Option<&str>) -> Result<(), ProtocolErrorBody> {
if let Some(id) = id {
if item.id.as_deref().is_some_and(|previous| previous != id) {
@@ -502,11 +849,136 @@ fn part_text(part: &Value) -> Result<(PartKind, &str), ProtocolErrorBody> {
}
}
fn function_fields(item: &Value) -> Result<(&str, &str, &str), ProtocolErrorBody> {
fn append_url_citations(
part: &Value,
message_offset: u64,
annotations: &mut Vec<Value>,
) -> Result<(), ProtocolErrorBody> {
let Some(values) = part.get("annotations").filter(|value| !value.is_null()) else {
return Ok(());
};
for value in values.as_array().ok_or_else(invalid_response)? {
if value["type"] != "url_citation" {
continue;
}
let start = required_index(value, "start_index")?;
let end = required_index(value, "end_index")?;
if start > end {
return Err(invalid_response());
}
annotations.push(json!({"type": "url_citation", "url_citation": {
"url": required_str(value, "url")?,
"title": required_str(value, "title")?,
"start_index": message_offset.checked_add(start).ok_or_else(invalid_response)?,
"end_index": message_offset.checked_add(end).ok_or_else(invalid_response)?,
}}));
}
Ok(())
}
fn is_auxiliary_output(kind: &str) -> bool {
// 服务端工具记录和压缩密文不要求 Chat 客户端执行动作,也不投影为可读内容
matches!(
kind,
"web_search_call"
| "file_search_call"
| "code_interpreter_call"
| "mcp_call"
| "mcp_list_tools"
| "compaction"
| "compaction_summary"
| "context_compaction"
)
}
fn reasoning_parts(item: &Value) -> Result<Vec<(bool, u64, &str)>, ProtocolErrorBody> {
let mut result = Vec::new();
// 只交付上游明确公开的推理文本,不解读或暴露 encrypted_content
for (field, kind) in [("summary", "summary_text"), ("content", "reasoning_text")] {
if let Some(parts) = item.get(field).filter(|value| !value.is_null()) {
for (index, part) in parts
.as_array()
.ok_or_else(invalid_response)?
.iter()
.enumerate()
{
if part["type"] == kind {
result.push((
field == "summary",
index as u64,
required_str(part, "text")?,
));
}
}
}
}
Ok(result)
}
fn image_payload(item: &Value) -> Result<Option<Value>, ProtocolErrorBody> {
let Some(data) = item.get("result").filter(|value| !value.is_null()) else {
return Ok(None);
};
let data = data.as_str().ok_or_else(invalid_response)?;
if data.is_empty() {
return Ok(None);
}
let format = item
.get("output_format")
.and_then(Value::as_str)
.unwrap_or("png");
let mime = match format {
"png" => "image/png",
"jpeg" | "jpg" => "image/jpeg",
"webp" => "image/webp",
_ => return Err(invalid_response()),
};
Ok(Some(
json!({"type":"image_url", "image_url":{"url":format!("data:{mime};base64,{data}")}}),
))
}
fn append_logprobs(
part: &mut Part,
value: &Value,
snapshot: bool,
) -> Result<Vec<Value>, ProtocolErrorBody> {
let Some(values) = value.get("logprobs").filter(|value| !value.is_null()) else {
return Ok(Vec::new());
};
let values = values.as_array().ok_or_else(invalid_response)?;
let delta = if snapshot {
values
.strip_prefix(part.logprobs.as_slice())
.ok_or_else(invalid_response)?
} else {
values
};
let delta = delta.to_vec();
part.logprobs.extend(delta.iter().cloned());
Ok(delta)
}
fn logprobs_frame(
identity: &ChatIdentity,
include_usage: bool,
kind: PartKind,
values: Vec<Value>,
) -> Result<Bytes, ProtocolErrorBody> {
chunk_frame(
identity,
json!([{"index":0, "delta":{}, "finish_reason":null,
"logprobs":{kind.field():values}}]),
include_usage.then_some(Value::Null),
None,
)
}
fn tool_fields(item: &Value, kind: ToolKind) -> Result<(&str, &str, &str), ProtocolErrorBody> {
Ok((
required_nonempty_str(item, "call_id")?,
required_nonempty_str(item, "name")?,
required_str(item, "arguments")?,
required_str(item, kind.data_field())?,
))
}
@@ -519,8 +991,10 @@ fn validate_output_identities(output: &[Value]) -> Result<(), ProtocolErrorBody>
{
return Err(invalid_response());
}
if item.get("type").and_then(Value::as_str) == Some("function_call")
&& !call_ids.insert(required_nonempty_str(item, "call_id")?)
if matches!(
item.get("type").and_then(Value::as_str),
Some("function_call" | "custom_tool_call")
) && !call_ids.insert(required_nonempty_str(item, "call_id")?)
{
return Err(invalid_response());
}
@@ -552,19 +1026,41 @@ fn chat_usage(response: &Value) -> Option<Value> {
return None;
}
let mut result = json!({"prompt_tokens": usage.input_tokens, "completion_tokens": usage.output_tokens, "total_tokens": usage.total_tokens});
if response
.pointer("/usage/input_tokens_details/cached_tokens")
.is_some()
{
result["prompt_tokens_details"] = json!({"cached_tokens": usage.cached_input_tokens});
for (source, target) in [
("input_tokens_details", "prompt_tokens_details"),
("output_tokens_details", "completion_tokens_details"),
] {
if let Some(details) = response
.get("usage")
.and_then(|usage| usage.get(source))
.filter(|value| value.is_object())
{
result[target] = details.clone();
}
}
if response
.pointer("/usage/output_tokens_details/reasoning_tokens")
.and_then(Value::as_u64)
.is_some_and(|tokens| tokens <= usage.output_tokens)
if let Some(details) = result
.get_mut("prompt_tokens_details")
.and_then(Value::as_object_mut)
&& details.contains_key("cached_tokens")
{
result["completion_tokens_details"] =
json!({"reasoning_tokens": usage.reasoning_output_tokens});
details.insert("cached_tokens".into(), usage.cached_input_tokens.into());
}
if let Some(details) = result
.get_mut("completion_tokens_details")
.and_then(Value::as_object_mut)
{
if details
.get("reasoning_tokens")
.and_then(Value::as_u64)
.is_some_and(|tokens| tokens <= usage.output_tokens)
{
details.insert(
"reasoning_tokens".into(),
usage.reasoning_output_tokens.into(),
);
} else {
details.remove("reasoning_tokens");
}
}
Some(result)
}
@@ -1,6 +1,13 @@
//! OpenAI Provider 原生非流式 JSON 端点的公共交付边界
use std::{future::Future, net::IpAddr};
use std::{
future::Future,
net::IpAddr,
sync::{
Arc,
atomic::{AtomicBool, Ordering},
},
};
use axum::{
body::{Body, Bytes},
@@ -49,6 +56,8 @@ where
};
let request_id = prepared.request_id().clone();
let endpoint = input.endpoint.clone();
let execution_entered = Arc::new(AtomicBool::new(false));
let terminal_entered = Arc::clone(&execution_entered);
let result = invoke_request(
execution,
prepared,
@@ -60,6 +69,7 @@ where
return Err(MiddlewareError::Rejected);
}
let operation = request.apply_capabilities(decode(body, headers).await?)?;
terminal_entered.store(true, Ordering::Relaxed);
let started = match service
.start_prepared_provider_endpoint(
prepared, operation,
@@ -82,11 +92,18 @@ where
Ok(response) => into_http_response(response, ExpectedBody::SingleJson).await,
Err(error) => error_response(error),
};
if response.headers().contains_key("x-gateway-request-id") {
if !execution_entered.load(Ordering::Relaxed) {
return response;
}
let mut response = if response.headers().contains_key("x-gateway-request-id") {
response
} else {
super::with_model_request_id(response, &request_id)
}
};
response
.extensions_mut()
.insert(crate::client_error::ExecutionObserved);
response
}
/// 只收集正文,不提前 commit;外层完成响应变换与校验后才提交 Core
@@ -23,7 +23,9 @@ pub use crate::client_error::{
/// 严格协议 decoder/encoder 返回的安全错误 body
pub fn protocol_error_response(status: StatusCode, body: ProtocolErrorBody) -> Response {
(status, Json(body.into_value())).into_response()
let mut response = (status, Json(body.clone().into_value())).into_response();
response.extensions_mut().insert(body);
response
}
/// 下游 Client API Key 无效
@@ -91,6 +91,7 @@ pub(crate) async fn live_call(
Ok(prepared) => prepared,
Err(error) => return gateway_error_response(&error),
};
let request_id = prepared.request_id().clone();
let started = match service
.start_prepared_provider_endpoint(
prepared,
@@ -103,7 +104,7 @@ pub(crate) async fn live_call(
.await
{
Ok(started) => started,
Err(error) => return gateway_error_response(&error),
Err(error) => return with_model_request_id(gateway_error_response(&error), &request_id),
};
collect_live_call_response(started, uri.path().to_owned()).await
}
@@ -25,6 +25,9 @@ pub(crate) fn with_model_request_id(
let Ok(gateway_id) = HeaderValue::from_str(request_id.as_str()) else {
return response;
};
response
.extensions_mut()
.insert(crate::client_error::ExecutionObserved);
let headers = response.headers_mut();
let usable = |value: &HeaderValue| value.to_str().is_ok_and(|id| !id.trim().is_empty());
if !headers.get("x-request-id").is_some_and(usable) {
@@ -85,7 +85,7 @@ async fn peer_response(state: ApiState, client: AuthenticatedClient, uri: Uri) -
StatusCode::FORBIDDEN,
"CPR Link sharing is not enabled for this API key.",
"permission_error",
"peer_sharing_disabled",
gateway_protocol::peer::SHARING_DISABLED_CODE,
)
.into_response(),
Err(PeerLinkError::RateLimited { retry_after }) => {
@@ -4,14 +4,14 @@ use chrono::{DateTime, Utc};
use gateway_admin::model::peer_link::{PeerLinkExport, PeerModelState as ModelState};
use gateway_core::metering::Decimal;
use gateway_protocol::peer::{
OBJECT, PeerBudgetWindow, PeerComponentStatus, PeerKeyUsage, PeerLink, PeerModelAvailability,
PeerModelCoverage, PeerModelState, PeerPlanCount, PeerPool, PeerPoolStates, PeerQuota,
PeerQuotaAccount, PeerQuotaWindow, PeerUsage, SCHEMA_VERSION,
OBJECT, PeerBudgetWindow, PeerComponentStatus, PeerKeyLimits, PeerKeyUsage, PeerLink,
PeerModelAvailability, PeerModelCoverage, PeerModelState, PeerPlanCount, PeerPool,
PeerPoolStates, PeerQuota, PeerQuotaAccount, PeerQuotaWindow, PeerUsage, SCHEMA_VERSION,
};
use std::time::SystemTime;
pub(super) fn response(export: PeerLinkExport) -> PeerLink {
let mut capabilities = vec!["key_usage".to_owned()];
let mut capabilities = vec!["key_usage".to_owned(), "key_limits".to_owned()];
let pool = export.pool.map(|pool| {
capabilities.push("pool_summary".to_owned());
PeerPool {
@@ -74,6 +74,10 @@ pub(super) fn response(export: PeerLinkExport) -> PeerLink {
capabilities,
observed_at: export.observed_at.to_rfc3339(),
expires_at: export.expires_at.to_rfc3339(),
key_limits: Some(PeerKeyLimits {
max_concurrency: Some(export.key_limits.max_concurrency),
requests_per_minute: Some(export.key_limits.requests_per_minute),
}),
usage: Some(PeerUsage {
status: PeerComponentStatus::Available,
observed_at: Some(export.budget_observed_at.to_rfc3339()),
@@ -62,6 +62,14 @@ pub enum RequestDecodeError {
/// 字段路径
field: String,
},
/// 两个参数的要求无法同时满足
#[error("requested `{field}` is not supported with `{other}`")]
UnsupportedCombination {
/// 返回给客户端的参数路径
field: &'static str,
/// 与之冲突的参数路径
other: &'static str,
},
/// Core operation 拒绝规范化后的字段
#[error("field `{field}` violates the canonical operation contract")]
CanonicalContract {
@@ -127,6 +135,11 @@ impl RequestDecodeError {
format!("Parameter `{field}` is not supported by this gateway."),
Some(field.clone()),
),
Self::UnsupportedCombination { field, other } => (
"unsupported_parameter",
format!("The requested `{field}` is not supported with `{other}`."),
Some((*field).to_owned()),
),
};
ProtocolErrorBody {
error: ProtocolError {
@@ -5,7 +5,10 @@ use crate::request_context::request_client_context;
use std::collections::VecDeque;
use std::future::Future;
use std::net::SocketAddr;
use std::sync::{Arc, OnceLock};
use std::sync::{
Arc, OnceLock,
atomic::{AtomicBool, Ordering},
};
use std::time::Duration;
use axum::{
@@ -159,6 +162,8 @@ pub(in crate::openai) async fn execute_prepared_response(
let execution = service.execution();
let request_id = prepared.request_id().clone();
let cancellation = prepared.cancellation();
let execution_entered = Arc::new(AtomicBool::new(false));
let terminal_entered = Arc::clone(&execution_entered);
let started_request_id = Arc::new(OnceLock::new());
let terminal_request_id = Arc::clone(&started_request_id);
let (admitted, keepalive) = tokio::sync::oneshot::channel();
@@ -217,6 +222,7 @@ pub(in crate::openai) async fn execute_prepared_response(
} else {
None
};
terminal_entered.store(true, Ordering::Relaxed);
let started = terminal_service
.start_prepared_response(prepared, decoded, transport, endpoint)
.await
@@ -276,7 +282,15 @@ pub(in crate::openai) async fn execute_prepared_response(
} else {
response.await
};
super::super::with_model_request_id(response, started_request_id.get().unwrap_or(&request_id))
// 预留 ID 没有对应执行记录,中间件提前返回时保留 HTTP 入口关联 ID
if execution_entered.load(Ordering::Relaxed) {
super::super::with_model_request_id(
response,
started_request_id.get().unwrap_or(&request_id),
)
} else {
response
}
}
async fn await_with_authorization<T>(
@@ -275,6 +275,11 @@ impl ResponsesDeliveryValidator {
}
}
pub(super) fn completed_response_id(&self) -> Option<&str> {
self.response_id()
.filter(|_| self.state.terminal == Some(TerminalKind::Completed))
}
pub(super) fn finish_websocket_delivery(&self) -> Result<(), ResponseValidationError> {
if !self.active || self.state.terminal.is_some() {
Ok(())
@@ -36,6 +36,7 @@ pub struct ResponsesWebSocketConnection {
commands: Option<mpsc::Sender<ConnectionCommand>>,
incoming: mpsc::Receiver<PendingConnectionEvent>,
deferred: VecDeque<PendingConnectionEvent>,
input_gate: Arc<tokio::sync::Mutex<()>>,
exited: oneshot::Receiver<PumpExitReason>,
pump_task: Option<JoinHandle<()>>,
stats: Arc<ConnectionStats>,
@@ -172,6 +173,26 @@ impl ResponsesWebSocketConnection {
.await;
}
pub(in super::super) async fn close_for_replay_if_idle(&mut self) -> bool {
// 原子划分入站帧与正常关闭,正在处理或已排队的帧都先按原合同执行
let Ok(_guard) = self.input_gate.clone().try_lock_owned() else {
return false;
};
if !self.deferred.is_empty() || !self.incoming.is_empty() {
return false;
}
self.close(
CloseFrame {
code: close_code::NORMAL,
reason: "Reconnect with full input.".into(),
},
WriteContext::connection(FramePhase::Close),
PumpExitReason::ServerClose,
)
.await;
true
}
pub(in super::super) async fn close_for_connection_limit(&mut self, reason: &'static str) {
self.close(
CloseFrame {
@@ -344,11 +365,13 @@ where
let extensions = crate::middleware::current()
.map(|context| context.extensions)
.unwrap_or_default();
let input_gate = Arc::new(tokio::sync::Mutex::new(()));
let pump_task = tokio::spawn(run_pump(
socket,
activity.clone(),
command_rx,
incoming_tx,
Arc::clone(&input_gate),
exit_tx,
Arc::clone(&connection_id),
cancellation.clone(),
@@ -368,6 +391,7 @@ where
commands: Some(command_tx),
incoming: incoming_rx,
deferred: VecDeque::new(),
input_gate,
exited: exit_rx,
pump_task: Some(pump_task),
stats,
@@ -29,6 +29,7 @@ pub(super) async fn run_pump<S, E>(
activity: watch::Sender<Instant>,
commands: mpsc::Receiver<ConnectionCommand>,
incoming: mpsc::Sender<PendingConnectionEvent>,
input_gate: Arc<tokio::sync::Mutex<()>>,
exited: oneshot::Sender<PumpExitReason>,
connection_id: Arc<str>,
cancellation: CancellationToken,
@@ -63,7 +64,7 @@ pub(super) async fn run_pump<S, E>(
// 心跳独立于业务写入等待,但共享唯一 writer,避免长写入或排队阻止失联检测
let reason = tokio::select! {
biased;
reason = read_connection(&mut reader, &incoming, sender.clone(), &activity, &context) => reason,
reason = read_connection(&mut reader, &incoming, &input_gate, sender.clone(), &activity, &context) => reason,
reason = write_connection(writer.clone(), commands, &context) => reason,
reason = heartbeat_connection(writer.clone(), received_activity, &context) => reason,
};
@@ -81,6 +82,7 @@ pub(super) async fn run_pump<S, E>(
async fn read_connection<S, E>(
socket: &mut S,
incoming: &mpsc::Sender<PendingConnectionEvent>,
input_gate: &tokio::sync::Mutex<()>,
sender: Arc<dyn contract::Sender>,
activity: &watch::Sender<Instant>,
context: &PumpContext,
@@ -107,6 +109,7 @@ where
}
}
message = socket.next() => {
let _guard = input_gate.lock().await;
if matches!(&message, Some(Ok(_))) {
// 链路活动以真实入站为准,插件丢弃消息不应抹掉对端已经响应的事实
activity.send_replace(Instant::now());
@@ -23,7 +23,9 @@ use crate::openai::responses::validation::{ResponseValidationFacts, ResponsesDel
use super::{
super::{DecodedResponsesRequest, OpenAiResponsesEncoder, ProtocolErrorBody},
connection::{ConnectionEvent, FramePhase, ResponsesWebSocketConnection, WriteContext},
connection::{
ConnectionEvent, FramePhase, ResponsesWebSocketConnection, WriteContext, WriteOutcome,
},
protocol::{
error_event, initial_engine_error_event, is_response_create, response_metadata_event,
},
@@ -32,6 +34,7 @@ use super::{
#[derive(Clone, Copy, PartialEq, Eq)]
pub(super) enum ForwardOutcome {
Continue,
ReconnectIfIdle,
Disconnect,
}
@@ -55,9 +58,25 @@ impl ConnectionReplaySnapshot {
}
}
fn commit(&mut self, response_id: String, provider_state: Option<ProviderSessionState>) {
self.last_response_id = Some(response_id);
self.provider_state = provider_state;
fn commit(&mut self, capture: &ReplayCaptureHandle, delivered: Option<&str>) -> ForwardOutcome {
let mut capture = capture
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let Some(id) = capture.response_id.take() else {
return ForwardOutcome::Continue;
};
self.provider_state = capture.provider_state.take();
let reconnect = delivered == Some(&id)
&& self
.provider_state
.as_ref()
.is_some_and(ProviderSessionState::prefers_websocket_reconnect);
self.last_response_id = Some(id);
if reconnect {
ForwardOutcome::ReconnectIfIdle
} else {
ForwardOutcome::Continue
}
}
}
@@ -199,7 +218,7 @@ pub(super) async fn forward_response(
let mut first_frame_written = false;
let mut current = Some(first);
let mut terminal_seen = false;
let mut terminal_write = None;
let mut first_protocol_validated = true;
loop {
let frame = match current.take() {
@@ -211,12 +230,14 @@ pub(super) async fn forward_response(
response_control,
&mut interrupt,
&request_id,
terminal_seen,
terminal_write.is_some(),
)
.await
{
BodyInput::Frame(Ok(Some(frame))) => frame,
BodyInput::Frame(Ok(None)) if terminal_seen && body.is_finalized() => {
BodyInput::Frame(Ok(None))
if terminal_write.is_some() && body.is_finalized() =>
{
if delivery_validator.finish_websocket_delivery().is_err() {
return fail_body(
connection,
@@ -226,16 +247,12 @@ pub(super) async fn forward_response(
)
.await;
}
{
let captured = capture
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if let Some(response_id) = captured.response_id.clone() {
replay.commit(response_id, captured.provider_state.clone());
}
}
let delivered = delivery_validator
.completed_response_id()
.filter(|_| terminal_write == Some(WriteOutcome::Written));
let outcome = replay.commit(&capture, delivered);
body.close().await;
return ForwardOutcome::Continue;
return outcome;
}
BodyInput::Frame(Ok(None)) => {
return fail_body(
@@ -256,7 +273,7 @@ pub(super) async fn forward_response(
}
}
};
if terminal_seen || validate_frame(&frame).is_err() {
if terminal_write.is_some() || validate_frame(&frame).is_err() {
return fail_body(connection, body, MiddlewareError::InvalidState, &request_id).await;
}
if first_protocol_validated {
@@ -278,30 +295,27 @@ pub(super) async fn forward_response(
(true, false) => FramePhase::Data,
};
let created_id = super::interrupt::ResponseInterrupt::created_id(&message);
match connection
let Ok(write) = connection
.send_text(message, WriteContext::request(&request_id, phase))
.await
{
Ok(super::connection::WriteOutcome::Written) => {
first_frame_written = true;
if let Some(id) = created_id {
interrupt.observe_created(id, response_control);
tokio::select! {
_ = connection.wait_for_exit() => {
detach_body(body);
return ForwardOutcome::Disconnect;
}
() = interrupt.forward_pending(response_control) => {}
else {
detach_body(body);
return ForwardOutcome::Disconnect;
};
if write == WriteOutcome::Written {
first_frame_written = true;
if let Some(id) = created_id {
interrupt.observe_created(id, response_control);
tokio::select! {
_ = connection.wait_for_exit() => {
detach_body(body);
return ForwardOutcome::Disconnect;
}
() = interrupt.forward_pending(response_control) => {}
}
}
Ok(super::connection::WriteOutcome::Suppressed) => {}
Err(_) => {
detach_body(body);
return ForwardOutcome::Disconnect;
}
}
terminal_seen = terminal;
terminal_write = terminal.then_some(write);
}
}
@@ -179,6 +179,10 @@ async fn serve_responses_websocket(socket: Socket, session: ResponsesWebSocketSe
crate::middleware::websocket::CancelOnDrop(Some(cancellation.clone()));
let request_headers =
request_headers.with_downstream_websocket_connection_id(connection_id.clone());
// 帧中间件可能改写终态,此时不能据原始响应关闭下游连接
let reconnect_for_replay = !middleware
.as_ref()
.is_some_and(FrozenMiddlewarePlan::has_websocket);
let mut connection = ResponsesWebSocketConnection::new(
socket,
connection_id,
@@ -401,7 +405,7 @@ async fn serve_responses_websocket(socket: Socket, session: ResponsesWebSocketSe
continue;
}
};
if forward_response(
let outcome = forward_response(
&mut connection,
response,
request_id,
@@ -410,8 +414,12 @@ async fn serve_responses_websocket(socket: Socket, session: ResponsesWebSocketSe
validation,
&response_control,
)
.await
== ForwardOutcome::Disconnect
.await;
if outcome == ForwardOutcome::Disconnect
|| (outcome == ForwardOutcome::ReconnectIfIdle
&& reconnect_for_replay
&& !connection.is_expired()
&& connection.close_for_replay_if_idle().await)
{
break;
}
@@ -444,9 +444,13 @@ fn account_page_item(
}
#[tokio::test]
async fn account_usage_preserves_cache_read_and_write_counts_and_missing_values() {
async fn account_usage_preserves_periods_cache_counts_and_missing_values() {
use chrono::Utc;
use gateway_admin::model::accounts::AccountUsage;
use gateway_admin::model::accounts::{AccountModelUsage, AccountUsage};
use gateway_admin::model::observability::RequestHealth;
use gateway_admin::model::provider_credentials::{
ProviderQuota, ProviderQuotaWindow, QuotaLocalUsageAttribution,
};
use gateway_core::account::{AccountStatusFacts, CredentialState, QuotaState};
let fixture = AdminTestFixture::new().await;
@@ -461,9 +465,19 @@ async fn account_usage_preserves_cache_read_and_write_counts_and_missing_values(
last_error_reason: None,
last_error_message: None,
};
*fixture.account.lock().unwrap() = Some(account_page_item(facts, now, "api_key"));
for (read, write) in [(Some(13), Some(7)), (None, None)] {
*fixture.account_usage.lock().unwrap() = vec![AccountUsage {
for (authentication_kind, read, write) in [
("api_key", Some(13), Some(7)),
("api_key", None, None),
("oauth", Some(13), Some(7)),
("oauth", None, None),
] {
let mut account = account_page_item(facts.clone(), now, authentication_kind);
if authentication_kind == "oauth" {
account.account.provider_kind =
gateway_core::routing::ProviderKind::new("xai").unwrap();
}
*fixture.account.lock().unwrap() = Some(account);
let local_usage = AccountUsage {
account_id: "acct_cooldown".to_owned(),
request_count: 1,
success_count: 1,
@@ -480,9 +494,61 @@ async fn account_usage_preserves_cache_read_and_write_counts_and_missing_values(
cost_coverage: Default::default(),
costs: Vec::new(),
last_used_at: Some(now),
request_buckets: Vec::new(),
models: Vec::new(),
}];
request_buckets: vec![gateway_admin::model::accounts::AccountRequestBucket {
bucket_start: now,
request_count: 1,
}],
models: [
(
"mixed",
RequestHealth::from_counts(1, 2, 3, 1, 1),
Some(now),
),
("failed", RequestHealth::from_counts(0, 1, 0, 0, 0), None),
("no-sample", RequestHealth::from_counts(0, 1, 1, 0, 1), None),
]
.into_iter()
.map(|(model, health, last_used_at)| AccountModelUsage {
model: model.to_owned(),
request_count: u64::from(last_used_at.is_some()),
health,
input_tokens: None,
output_tokens: None,
cached_input_tokens: None,
cache_write_input_tokens: None,
reasoning_output_tokens: None,
image_input_tokens: None,
image_output_tokens: None,
image_request_count: 0,
image_request_failed_count: 0,
total_tokens: None,
cost_coverage: Default::default(),
costs: Vec::new(),
last_used_at,
})
.collect(),
};
*fixture.account_usage.lock().unwrap() = vec![local_usage.clone()];
*fixture.provider_quota.lock().unwrap() =
(authentication_kind == "oauth").then(|| ProviderQuota {
windows: vec![ProviderQuotaWindow {
upstream: None,
key: "rolling-day".to_owned(),
group: "shortTerm".to_owned(),
label: "日限额".to_owned(),
limit_id: None,
limit_name: None,
role: None,
local_usage_attribution: QuotaLocalUsageAttribution::AccountWide,
window_seconds: Some(86_400),
used_percent: None,
reset_at: None,
limit_reached: false,
local_usage: Some(local_usage),
provider_data: None,
}],
..Default::default()
});
let response = admin::accounts::router::<AdminTestState>()
.with_state(fixture.state())
.oneshot(
@@ -500,6 +566,34 @@ async fn account_usage_preserves_cache_read_and_write_counts_and_missing_values(
let value: serde_json::Value = serde_json::from_slice(&body).unwrap();
assert_eq!(status, StatusCode::OK, "{value}");
let usage = &value["data"]["items"][0]["usage"];
assert_eq!(
usage["windowKind"],
if authentication_kind == "oauth" {
"daily"
} else {
"general"
}
);
assert_eq!(usage["models"][0]["successRateDisplay"], "33.3%");
assert!((usage["models"][0]["successRate"].as_f64().unwrap() - 100.0 / 3.0).abs() < 0.001);
assert_eq!(usage["models"][1]["successRate"], 0.0);
assert_eq!(usage["models"][1]["successRateDisplay"], "0.0%");
assert!(usage["models"][1]["lastUsedAt"].is_null());
assert!(usage["models"][2]["successRate"].is_null());
assert_eq!(usage["models"][2]["successRateDisplay"], "—");
assert_eq!(usage["requestCount"], 1);
assert_eq!(usage["totalTokens"], 120);
if authentication_kind == "oauth" {
let window = &value["data"]["items"][0]["quota"]["windows"][0];
assert_eq!(window["localUsage"]["requestCount"], 1);
assert_eq!(
window["localUsage"]["requestBuckets"]
.as_array()
.unwrap()
.len(),
1
);
}
assert_eq!(usage["cachedInputTokens"], serde_json::json!(read));
assert_eq!(usage["cacheWriteInputTokens"], serde_json::json!(write));
assert_eq!(
@@ -884,6 +884,7 @@ impl AccountGroupStore for MemoryAccountGroupStore {
}
Ok(vec![
AccountGroupMemberFact {
inherited_version: None,
group_id: group_id(PRIMARY_GROUP_ID),
account_id: "acct_group_ready".to_owned(),
status: AccountStatusFacts {
@@ -898,6 +899,7 @@ impl AccountGroupStore for MemoryAccountGroupStore {
total_slots: Some(1),
},
AccountGroupMemberFact {
inherited_version: None,
group_id: group_id(PRIMARY_GROUP_ID),
account_id: "acct_group_disabled".to_owned(),
status: AccountStatusFacts {
@@ -387,6 +387,21 @@ async fn usage_detail_should_keep_attempt_snapshot_contract() {
record.upstream_response_ms = Some(1_000);
record.upstream_api_overhead_ms = Some(120.25);
record.upstream_engine_ms = Some(6400.0);
record.outcome = gateway_admin::model::observability::RequestOutcome::Failed;
record.error_kind = Some("continuation_recovery_required".to_owned());
record.provider_error_code = Some("previous_response_not_found".to_owned());
record.continuation_unavailable_reason = Some("reused_connection_lost".to_owned());
record.upstream_connection_exit_reason = Some("max_age_expired".to_owned());
record.recovered = true;
record.upstream_error = Some(gateway_admin::model::observability::UsageUpstreamError {
code: Some("previous_response_not_found".to_owned()),
error_type: Some("invalid_request_error".to_owned()),
message: Some("Previous response is unavailable.".to_owned()),
param: Some("previous_response_id".to_owned()),
close_reason: None,
last_event_type: None,
});
record.retry_after_ms = Some(206_000);
fixture
.usage_detail
.lock()
@@ -447,6 +462,31 @@ async fn usage_detail_should_keep_attempt_snapshot_contract() {
.await
.expect("usage detail body");
let value: serde_json::Value = serde_json::from_slice(&body).expect("usage detail JSON");
assert_eq!(value["data"]["errorKind"], "continuation_recovery_required");
assert_eq!(value["data"]["errorCode"], "previous_response_not_found");
assert_eq!(
value["data"]["upstreamError"]["message"],
"Previous response is unavailable."
);
assert_eq!(
value["data"]["upstreamError"]["errorType"],
"invalid_request_error"
);
assert_eq!(
value["data"]["upstreamError"]["param"],
"previous_response_id"
);
assert_eq!(value["data"]["retryAfterMs"], 206_000);
assert_eq!(
value["data"]["continuationUnavailableReason"],
"reused_connection_lost"
);
assert_eq!(
value["data"]["upstreamConnectionExitReason"],
"max_age_expired"
);
assert_eq!(value["data"]["websocketCloseCode"], serde_json::Value::Null);
assert_eq!(value["data"]["recovered"], true);
assert_eq!(value["data"]["firstTokenLatencyMs"], 20);
assert_eq!(value["data"]["latencyDetails"]["upstreamResponseMs"], 1_000);
assert_eq!(
@@ -662,6 +702,8 @@ fn peer_summary(
) -> gateway_admin::model::provider_credentials::ProviderQuota {
gateway_admin::model::provider_credentials::ProviderQuota {
upstream: Some(gateway_admin::model::upstream_info::ProviderUpstreamInfo {
max_concurrency: None,
requests_per_minute: None,
discovered,
observed_at: observed.then(chrono::Utc::now),
stale: true,
@@ -786,6 +828,7 @@ pub(super) fn usage_record_with_account(
use gateway_admin::model::observability::{RequestOutcome, UsageRecord};
UsageRecord {
upstream_error: None,
id: id.to_owned(),
client_api_key_ref: "key_detail".to_owned(),
config_revision: 1,
@@ -817,6 +860,10 @@ pub(super) fn usage_record_with_account(
client_response_id: None,
upstream_request_id: None,
upstream_response_id: None,
websocket_close_code: None,
continuation_unavailable_reason: None,
upstream_connection_exit_reason: None,
recovered: false,
error_kind: None,
provider_error_code: None,
error_message: None,
@@ -934,6 +981,11 @@ async fn usage_route_projects_request_facts_and_billing() {
provider_account_upstream_peer: None,
error_code: None,
error_message: None,
error_kind: None,
websocket_close_code: None,
continuation_unavailable_reason: None,
upstream_connection_exit_reason: None,
recovered: false,
client_status_code: None,
outcome: gateway_admin::model::observability::RequestOutcome::Succeeded,
client_api_key_name: Some("Production".to_owned()),
@@ -1027,6 +1079,14 @@ async fn usage_route_projects_request_facts_and_billing() {
failed.error_code = Some("server_error".to_owned());
failed.error_message = Some("The upstream service is temporarily unavailable.".to_owned());
records.push(failed);
let mut closed = records[0].clone();
closed.id = "request_ws_closed".to_owned();
closed.outcome = gateway_admin::model::observability::RequestOutcome::Incomplete;
closed.error_kind = Some("upstream_unavailable".to_owned());
closed.websocket_close_code = Some(1012);
closed.error_message =
Some("OpenAI WebSocket closed before a terminal response".to_owned());
records.push(closed);
}
let response = observability::router::<AdminTestState>()
.with_state(fixture.state())
@@ -1049,6 +1109,17 @@ async fn usage_route_projects_request_facts_and_billing() {
assert!(value["data"]["items"][0]["errorCode"].is_null());
assert!(value["data"]["items"][0]["errorMessage"].is_null());
assert_eq!(value["data"]["items"][3]["errorCode"], "server_error");
assert_eq!(value["data"]["items"][4]["logicalOutcome"], "incomplete");
assert_eq!(
value["data"]["items"][4]["errorKind"],
"upstream_unavailable"
);
assert_eq!(value["data"]["items"][4]["websocketCloseCode"], 1012);
assert_eq!(
value["data"]["items"][4]["errorCode"],
serde_json::Value::Null
);
assert_eq!(value["data"]["items"][4]["recovered"], false);
assert_eq!(
value["data"]["items"][3]["errorMessage"],
"The upstream service is temporarily unavailable."
@@ -3,6 +3,14 @@ use super::TestPluginPorts;
use gateway_admin::{model::usage_data::*, ports::usage_data::*};
#[async_trait::async_trait]
impl UsageDataStore for TestPluginPorts {
async fn open_read(
&self,
_: UsageReadQuery,
_: gateway_core::lifecycle::CancellationToken,
) -> UsageDataResult<Box<dyn UsageReadSession>> {
Err(UsageDataError::unavailable("test usage reader unavailable"))
}
async fn get_record(&self, _: GetUsageRecord) -> UsageDataResult<UsageRecord> {
unreachable!("unexpected usage call")
}
@@ -354,9 +354,12 @@ impl ProxyProbe for SuccessfulProbe {
proxy.expose_url(),
"http://test-user:private-password@proxy.example:8080/"
);
} else {
assert!(detect_location);
}
let exit_ip = if proxy.is_some() {
"203.0.113.2"
} else {
"203.0.113.3"
};
ProxyTestResult {
location: if detect_location {
ProxyLocationDetection::Detected { location: serde_json::from_value(json!({"country":"JP","region":"Tokyo","city":"Tokyo","timezone":"Asia/Tokyo"})).unwrap() }
@@ -365,8 +368,8 @@ impl ProxyProbe for SuccessfulProbe {
},
success: true,
latency_ms: 15,
exit_ip: Some("203.0.113.2".parse().unwrap()),
exit_ipv4: Some("203.0.113.2".parse().unwrap()),
exit_ip: Some(exit_ip.parse().unwrap()),
exit_ipv4: Some(exit_ip.parse().unwrap()),
exit_ipv6: None,
message: "Connected".to_owned(),
}
@@ -567,11 +570,50 @@ async fn proxy_probe_checks_unsaved_address_without_creating_or_changing_records
}
#[tokio::test]
async fn proxy_probe_rejects_empty_or_invalid_addresses() {
async fn proxy_probe_checks_direct_egress_without_changing_saved_proxies() {
let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session");
let (_, created) = request(
&fixture,
"/api/admin/proxies/create",
Some(json!([{"name": "Saved", "proxyUrl": "http://saved.example:8080"}])),
true,
)
.await;
for detect_location in [false, true] {
let (status, probed) = request(
&fixture,
"/api/admin/proxies/probe",
Some(json!({"proxyUrl": "", "detectLocation": detect_location})),
true,
)
.await;
assert_eq!(status, StatusCode::OK);
assert_eq!(probed["data"]["success"], true);
assert_eq!(probed["data"]["exitIpv4"], "203.0.113.3");
assert_eq!(
probed["data"]["location"]["status"],
if detect_location {
"detected"
} else {
"notRequested"
}
);
}
let (_, listed) = request(&fixture, "/api/admin/proxies", None, true).await;
assert_eq!(listed["data"]["page"]["total"], 1);
assert_eq!(
listed["data"]["items"][0],
created["data"][0]["data"]["record"]
);
}
#[tokio::test]
async fn proxy_probe_rejects_missing_or_invalid_addresses() {
let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session");
for (body, expected) in [
(json!({"proxyUrl": ""}), StatusCode::BAD_REQUEST),
(json!({"proxyUrl": " "}), StatusCode::UNPROCESSABLE_ENTITY),
(json!({"proxyUrl": null}), StatusCode::UNPROCESSABLE_ENTITY),
(json!({}), StatusCode::UNPROCESSABLE_ENTITY),
(
@@ -13,7 +13,7 @@ impl UpstreamInfoSource for TestPeerInfo {
fn signal(
&self,
_: &ProviderAccount,
_: &str,
_: Option<&str>,
_: SystemTime,
) -> Option<UpstreamSchedulingSignal> {
None
@@ -22,6 +22,14 @@ impl UpstreamInfoSource for TestPeerInfo {
#[async_trait]
impl PeerUpstreamService for TestPeerInfo {
fn observed_concurrency_limit(
&self,
_: &gateway_core::account::QuotaObservationVersion,
_: std::time::SystemTime,
) -> Option<gateway_core::account::AccountConcurrencyLimit> {
None
}
async fn account_changed(&self, _: &ProviderAccountId) -> Result<(), StoreError> {
Ok(())
}
@@ -0,0 +1,257 @@
//! HTTP 入口拒绝可检索,诊断摘要不保存凭据和请求内容
use std::sync::Arc;
use axum::{
body::{Body, to_bytes},
http::{Request, StatusCode},
};
use serde_json::{Value, json};
use tower::ServiceExt as _;
use crate::support::RecordingDiagnostics;
async fn fixture() -> (axum::Router, Arc<RecordingDiagnostics>) {
let admin = crate::admin::AdminTestFixture::new().await;
let diagnostics = Arc::new(RecordingDiagnostics::default());
let router = crate::openai::api_bundle_with_diagnostics(
admin.services,
gateway_api::ApiConfig {
asset_directory: env!("CARGO_MANIFEST_DIR").into(),
cors_allowed_origins: Vec::new(),
request_timeout_seconds: None,
request_id_header: "x-correlation-id".into(),
},
diagnostics.clone(),
)
.router();
(router, diagnostics)
}
#[tokio::test]
async fn decode_rejections_record_the_ingress_id_and_safe_parameter_without_model_facts() {
for (path, body, field) in [
(
"/v1/chat/completions",
json!({"model":"model-a", "messages":[{"role":"user", "content":[{"type":"SENSITIVE_CONTENT"}]}]}),
"messages[0].content[0].type",
),
(
"/v1/responses",
json!({"model":42, "input":"SENSITIVE_CONTENT"}),
"model",
),
] {
let (router, diagnostics) = fixture().await;
let response = router
.oneshot(
Request::post(path)
.header("authorization", "Bearer unused-client-route-key")
.header("x-correlation-id", "ingress-test-608")
.body(Body::from(body.to_string()))
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
assert_eq!(response.headers()["x-correlation-id"], "ingress-test-608");
let body: Value =
serde_json::from_slice(&to_bytes(response.into_body(), usize::MAX).await.unwrap())
.unwrap();
assert_eq!(body["error"]["param"], field);
let failures = diagnostics.0.lock().unwrap();
assert_eq!(failures.len(), 1);
let failure = &failures[0];
assert_eq!(failure.component, "api");
assert_eq!(failure.operation, "http_ingress");
assert_eq!(failure.correlation_id.as_deref(), Some("ingress-test-608"));
let summary: Value = serde_json::from_str(&failure.message).unwrap();
assert_eq!(summary["param"], field);
assert_eq!(summary["endpoint"], path);
assert!(failure.upstream_status.is_none());
assert!(failure.account_id.is_none());
assert!(!failure.message.contains("SENSITIVE_CONTENT"));
assert!(!failure.message.contains("unused-client-route-key"));
}
}
#[tokio::test]
async fn authentication_unknown_routes_and_method_errors_are_observable() {
for (method, path, status, code) in [
(
"POST",
"/v1/chat/completions",
StatusCode::UNAUTHORIZED,
"missing_authorization",
),
(
"POST",
"/v1/messages?secret=SENSITIVE_QUERY",
StatusCode::NOT_FOUND,
"unknown_endpoint",
),
(
"DELETE",
"/v1/chat/completions",
StatusCode::METHOD_NOT_ALLOWED,
"method_not_allowed",
),
] {
let (router, diagnostics) = fixture().await;
let response = router
.oneshot(
Request::builder()
.method(method)
.uri(path)
.body(Body::from("SENSITIVE_CONTENT"))
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), status);
let failures = diagnostics.0.lock().unwrap();
assert_eq!(failures.len(), 1);
assert_eq!(failures[0].kind, code);
assert!(!failures[0].message.contains("SENSITIVE"));
assert!(failures[0].correlation_id.is_some());
}
}
#[derive(Clone)]
struct Recorder {
dump_enabled: bool,
events: Arc<std::sync::Mutex<Vec<std::collections::BTreeMap<String, String>>>>,
}
impl tracing::Subscriber for Recorder {
fn enabled(&self, metadata: &tracing::Metadata<'_>) -> bool {
metadata.target() != "request_dump" || self.dump_enabled
}
fn register_callsite(
&self,
_: &'static tracing::Metadata<'static>,
) -> tracing::subscriber::Interest {
tracing::subscriber::Interest::sometimes()
}
fn new_span(&self, _: &tracing::span::Attributes<'_>) -> tracing::span::Id {
tracing::span::Id::from_u64(1)
}
fn record(&self, _: &tracing::span::Id, _: &tracing::span::Record<'_>) {}
fn record_follows_from(&self, _: &tracing::span::Id, _: &tracing::span::Id) {}
fn enter(&self, _: &tracing::span::Id) {}
fn exit(&self, _: &tracing::span::Id) {}
fn event(&self, event: &tracing::Event<'_>) {
struct Fields(std::collections::BTreeMap<String, String>);
impl tracing::field::Visit for Fields {
fn record_str(&mut self, field: &tracing::field::Field, value: &str) {
self.0.insert(field.name().to_owned(), value.to_owned());
}
fn record_debug(&mut self, field: &tracing::field::Field, value: &dyn std::fmt::Debug) {
self.0.insert(field.name().to_owned(), format!("{value:?}"));
}
}
let mut fields = Fields(std::collections::BTreeMap::new());
event.record(&mut fields);
fields
.0
.insert("target".into(), event.metadata().target().into());
self.events.lock().unwrap().push(fields.0);
}
}
#[tokio::test]
async fn original_rejected_request_is_dumped_only_when_explicitly_enabled() {
use base64::{Engine as _, engine::general_purpose::STANDARD};
use tracing::instrument::WithSubscriber as _;
for enabled in [false, true] {
let recorder = Recorder {
dump_enabled: enabled,
events: Arc::default(),
};
let events = recorder.events.clone();
async {
let (router, _) = fixture().await;
let response = router
.clone()
.oneshot(
Request::post("/v1/chat/completions")
.header("authorization", "Bearer unused-client-route-key")
.body(Body::from("{\"secret\":\"SENSITIVE_CONTENT\","))
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
to_bytes(response.into_body(), usize::MAX).await.unwrap();
let response = router
.oneshot(
Request::post("/v1/unmatched")
.body(Body::from("UNCONSUMED_CONTENT"))
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::NOT_FOUND);
to_bytes(response.into_body(), usize::MAX).await.unwrap();
}
.with_subscriber(recorder)
.await;
let events = events.lock().unwrap();
let dumps = events
.iter()
.filter(|event| {
event
.get("target")
.is_some_and(|target| target == "request_dump")
})
.collect::<Vec<_>>();
assert_eq!(!dumps.is_empty(), enabled);
if enabled {
let original = dumps
.iter()
.filter(|event| {
event
.get("stage")
.is_some_and(|stage| stage == "ingress.request.body")
})
.flat_map(|event| STANDARD.decode(event.get("body_base64").unwrap()).unwrap())
.collect::<Vec<_>>();
assert_eq!(original, b"{\"secret\":\"SENSITIVE_CONTENT\",");
let completion = dumps
.iter()
.filter(|event| {
event
.get("stage")
.is_some_and(|stage| stage == "ingress.body.finished")
})
.map(|event| {
serde_json::from_slice::<Value>(
&STANDARD.decode(event.get("body_base64").unwrap()).unwrap(),
)
.unwrap()
})
.collect::<Vec<_>>();
assert!(completion.iter().any(
|event| event["stage"] == "ingress.request.body" && event["complete"] == false
));
assert!(
completion
.iter()
.any(|event| event["stage"] == "ingress.request.body"
&& event["complete"] == true)
);
}
for event in events.iter().filter(|event| {
event
.get("target")
.is_none_or(|target| target != "request_dump")
}) {
assert!(
!event
.values()
.any(|value| value.contains("SENSITIVE_CONTENT")
|| value.contains("unused-client-route-key"))
);
}
}
}
+67 -15
View File
@@ -1,4 +1,4 @@
//! 验证健康接口按基础设施与关键 Worker 状态判定可用性
//! 验证请求就绪、综合健康状态与故障诊断
use std::sync::{Arc, Mutex};
@@ -86,6 +86,54 @@ async fn healthz_should_reject_critical_worker_failure() {
assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE);
}
#[tokio::test]
async fn readiness_ignores_freeze_recovery_and_logging_failures_without_clearing_alerts() {
let diagnostics = Arc::new(RecordingDiagnostics::default());
let router = health_router(
vec![Arc::new(MutableProbe(Mutex::new(HealthState::Degraded(
"file logging maintenance failed".to_owned(),
))))],
vec![worker_snapshot(
WorkerKind::AccountFreezeRecovery,
WorkerRuntimeState::BackingOff,
)],
diagnostics.clone(),
)
.await;
for (path, expected) in [
("/healthz", StatusCode::SERVICE_UNAVAILABLE),
("/readyz", StatusCode::NO_CONTENT),
("/healthz", StatusCode::SERVICE_UNAVAILABLE),
] {
assert_eq!(
health_response(router.clone(), path).await.status(),
expected
);
}
assert_eq!(diagnostics.0.lock().unwrap().len(), 2);
}
#[tokio::test]
async fn readiness_rejects_an_unavailable_runtime_snapshot() {
let router = health_router(
vec![Arc::new(
gateway_core::runtime::RuntimeSnapshotHandle::default(),
)],
Vec::new(),
Arc::new(RecordingDiagnostics::default()),
)
.await;
let response = health_response(router, "/readyz").await;
assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE);
assert!(
axum::body::to_bytes(response.into_body(), 1024)
.await
.unwrap()
.is_empty()
);
}
#[tokio::test]
async fn healthz_records_probe_changes_once_and_records_recurrence_after_recovery() {
let probe = Arc::new(MutableProbe(Mutex::new(HealthState::Unhealthy(
@@ -95,7 +143,8 @@ async fn healthz_records_probe_changes_once_and_records_recurrence_after_recover
let router = health_router(vec![probe.clone()], Vec::new(), diagnostics.clone()).await;
let responses =
futures::future::join_all((0..8).map(|_| health_response(router.clone()))).await;
futures::future::join_all((0..8).map(|_| health_response(router.clone(), "/healthz")))
.await;
for response in responses {
assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE);
assert!(
@@ -126,18 +175,18 @@ async fn healthz_records_probe_changes_once_and_records_recurrence_after_recover
*probe.0.lock().unwrap() = HealthState::Degraded("file logging maintenance failed".to_owned());
assert_eq!(
health_response(router.clone()).await.status(),
health_response(router.clone(), "/healthz").await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert_eq!(diagnostics.0.lock().unwrap().len(), 2);
*probe.0.lock().unwrap() = HealthState::Healthy;
assert_eq!(
health_response(router.clone()).await.status(),
health_response(router.clone(), "/healthz").await.status(),
StatusCode::NO_CONTENT
);
*probe.0.lock().unwrap() = HealthState::Degraded("file logging maintenance failed".to_owned());
assert_eq!(
health_response(router).await.status(),
health_response(router, "/healthz").await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert_eq!(diagnostics.0.lock().unwrap().len(), 3);
@@ -168,7 +217,11 @@ async fn healthz_retains_worker_and_completed_probe_failures_when_another_probe_
.await;
assert_eq!(
health_response(router).await.status(),
health_response(router.clone(), "/readyz").await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert_eq!(
health_response(router, "/healthz").await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
let events = diagnostics.0.lock().unwrap();
@@ -202,12 +255,12 @@ async fn healthz_retries_failed_diagnostic_admission_without_changing_health_res
.await;
assert_eq!(
health_response(router.clone()).await.status(),
health_response(router.clone(), "/healthz").await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert!(diagnostics.recorded.0.lock().unwrap().is_empty());
assert_eq!(
health_response(router).await.status(),
health_response(router, "/healthz").await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert_eq!(diagnostics.recorded.0.lock().unwrap().len(), 1);
@@ -258,14 +311,9 @@ async fn health_router(
.router()
}
async fn health_response(router: axum::Router) -> axum::response::Response {
async fn health_response(router: axum::Router, path: &str) -> axum::response::Response {
router
.oneshot(
Request::builder()
.uri("/healthz")
.body(Body::empty())
.unwrap(),
)
.oneshot(Request::builder().uri(path).body(Body::empty()).unwrap())
.await
.unwrap()
}
@@ -289,6 +337,10 @@ impl HealthProbe for PendingProbe {
"pending"
}
fn required_for_readiness(&self) -> bool {
true
}
fn check(&self) -> BoxFuture<'_, HealthState> {
Box::pin(std::future::pending())
}
@@ -95,6 +95,11 @@ fn usage_record() -> UsageListRecord {
provider_account_upstream_peer: None,
error_code: None,
error_message: None,
error_kind: None,
websocket_close_code: None,
continuation_unavailable_reason: None,
upstream_connection_exit_reason: None,
recovered: false,
client_status_code: None,
outcome: gateway_admin::model::observability::RequestOutcome::Succeeded,
client_api_key_name: Some("Production".to_owned()),
+1
View File
@@ -3,6 +3,7 @@
mod admin;
mod architecture;
mod auth;
mod diagnostics;
mod health;
mod key_usage;
mod middleware;
@@ -622,3 +622,50 @@ async fn error_after_admission_keepalive_uses_chat_error_encoding() {
assert_eq!(fixture.trace.commits.load(Ordering::SeqCst), 0);
assert_eq!(fixture.trace.response_commit.status_code(), Some(200));
}
#[tokio::test]
async fn early_middleware_failure_is_recorded_at_ingress_without_a_reserved_model_id() {
let middleware = Arc::new(
RequestMiddleware {
expected_operation: Some(gateway_core::operation::OperationKind::Generate),
..Default::default()
}
.with_json_short_circuit(Bytes::from_static(b"{}")),
);
let mut execution = Execution::new(Session::new(vec![completed(json!([message("OK")]))]));
execution.middleware = Some(middleware.frozen());
let execution = Arc::new(execution);
let diagnostics = Arc::new(crate::support::RecordingDiagnostics::default());
let admin = crate::admin::AdminTestFixture::new().await;
let router = gateway_api::initialize(
gateway_api::ApiConfig {
asset_directory: env!("CARGO_MANIFEST_DIR").into(),
cors_allowed_origins: Vec::new(),
request_timeout_seconds: None,
request_id_header: "x-correlation-id".into(),
},
execution.clone(),
admin.services,
Vec::new(),
Arc::new(crate::openai::EmptyWorkerHealth),
Arc::new(crate::openai::TestLifecycle::default()),
diagnostics.clone(),
)
.unwrap()
.router();
let mut request = request(chat_request(false));
request
.headers_mut()
.insert("x-correlation-id", "ingress-short-circuit".parse().unwrap());
let response = router.oneshot(request).await.unwrap();
assert_eq!(response.status(), StatusCode::BAD_GATEWAY);
assert!(!response.headers().contains_key("x-gateway-request-id"));
assert!(execution.captured.lock().unwrap().is_none());
let failures = diagnostics.0.lock().unwrap();
assert_eq!(failures.len(), 1);
assert_eq!(failures[0].operation, "http_ingress");
assert_eq!(
failures[0].correlation_id.as_deref(),
Some("ingress-short-circuit")
);
}
@@ -28,7 +28,7 @@ async fn preserves_ordered_roles_images_and_modern_tool_round_trip() {
{"type":"image_url","image_url":{"url":"data:image/png;base64,aGVsbG8=","detail":"high"}}]},
{"role":"assistant","content":"checking","tool_calls":[
{"id":"call-weather","type":"function","function":{"name":"weather","arguments":arguments}}]},
{"role":"tool","tool_call_id":"call-weather","content":"{\"temperature\":25}"},
{"role":"tool","tool_call_id":"call-weather","name":"weather","content":"{\"temperature\":25}"},
{"role":"user","content":"continue"}
]})).await;
let input = converted["input"].as_array().unwrap();
@@ -53,11 +53,78 @@ async fn preserves_ordered_roles_images_and_modern_tool_round_trip() {
);
assert_eq!(
input[5],
json!({"type":"function_call_output","call_id":"call-weather","output":"{\"temperature\":25}"})
json!({"type":"function_call_output","call_id":"call-weather","name":"weather","output":"{\"temperature\":25}"})
);
assert!(converted.get("previous_response_id").is_none());
}
#[tokio::test]
async fn web_search_options_require_a_hosted_search_and_flatten_location() {
for (options, expected) in [
(json!({}), json!({"type":"web_search"})),
(
json!({"search_context_size":"low","user_location":{"type":"approximate",
"approximate":{"country":"US","city":"Seattle","region":"WA","timezone":"America/Los_Angeles"}}}),
json!({"type":"web_search","search_context_size":"low","user_location":{"type":"approximate",
"country":"US","city":"Seattle","region":"WA","timezone":"America/Los_Angeles"}}),
),
] {
let mut request = chat_request(false);
request["web_search_options"] = options;
let body = convert(request).await;
assert_eq!(body["tools"], json!([expected]));
assert_eq!(body["tool_choice"], json!({"type":"web_search"}));
}
}
#[tokio::test]
async fn invalid_search_options_and_conflicting_tool_policies_report_the_field() {
for (fields, param) in [
(
json!({"web_search_options":{"search_context_size":7}}),
"web_search_options.search_context_size",
),
(
json!({"web_search_options":{"user_location":{"type":"approximate"}}}),
"web_search_options.user_location.approximate",
),
(
json!({"web_search_options":{}, "tool_choice":"none"}),
"tool_choice",
),
(
json!({"web_search_options":{},"tools":[{"type":"function","function":{"name":"lookup"}}]}),
"tools",
),
] {
let mut request = chat_request(false);
request
.as_object_mut()
.unwrap()
.extend(fields.as_object().unwrap().clone());
let fixture = Fixture::new(Session::new(Vec::new())).await;
let response = fixture.post(request).await;
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
assert_eq!(json_body(response).await["error"]["param"], param);
assert!(fixture.execution.captured.lock().unwrap().is_none());
}
}
#[tokio::test]
async fn maps_inline_user_pdf_without_reencoding_its_data() {
let data = "data:application/pdf;base64,JVBERi0xLjQ=";
let converted = convert(json!({"model":"model-a","messages":[
{"role":"user","content":[{"type":"text","text":"Read the attachment"},
{"type":"file","file":{"filename":"document.pdf","file_data":data}}]}
]}))
.await;
assert_eq!(
converted["input"][0]["content"],
json!([{"type":"input_text","text":"Read the attachment"},
{"type":"input_file","filename":"document.pdf","file_data":data}])
);
}
#[tokio::test]
async fn assistant_history_uses_output_content_for_text_and_refusals() {
for content in [
@@ -162,19 +229,11 @@ async fn refuses_known_unrepresentable_features_before_starting_execution() {
for (name, value) in [
("n", json!(2)),
("store", json!(true)),
("stop", json!(["stop"])),
("seed", json!(7)),
("frequency_penalty", json!(0.3)),
("logprobs", json!(true)),
("functions", json!([{"name":"old"}])),
("function_call", json!("auto")),
("input", json!("not chat")),
("previous_response_id", json!("resp_previous")),
("audio", json!({"voice":"alloy","format":"wav"})),
(
"prediction",
json!({"type":"content","content":"predicted"}),
),
] {
let fixture = Fixture::new(Session::new(Vec::new())).await;
let mut body = chat_request(false);
@@ -193,9 +252,8 @@ async fn refuses_known_unrepresentable_features_before_starting_execution() {
#[tokio::test]
async fn rejects_unmapped_message_content_and_identity_fields() {
for message in [
json!({"role":"user","name":"speaker","content":"hello"}),
json!({"role":"user","content":[{"type":"file","file":{"file_id":"file-example"}}]}),
json!({"role":"assistant","content":"hello","reasoning_content":"private"}),
json!({"role":"tool","tool_call_id":"call-lookup","name":42,"content":"result"}),
json!({"role":"assistant","content":[{"type":"file","file":{"filename":"a.pdf","file_data":"data:application/pdf;base64,JVBERg=="}}]}),
json!({"role":"assistant","content":[{"type":"text","text":"history","prompt_cache_breakpoint":{"type":"default"}}]}),
json!({"role":"tool","content":"unassociated output"}),
] {
@@ -232,8 +290,8 @@ async fn assistant_tool_history_keeps_arguments_opaque_and_cache_annotations_str
);
assert_eq!(converted["input"][1]["arguments"], arguments);
assert_eq!(
converted["input"][2]["output"],
json!([{"type":"input_text","text":"result"}])
converted["input"][2],
json!({"type":"function_call_output","call_id":"call-raw","output":[{"type":"input_text","text":"result"}]})
);
assert_eq!(
converted["prompt_cache_options"],
@@ -241,3 +299,121 @@ async fn assistant_tool_history_keeps_arguments_opaque_and_cache_annotations_str
);
assert_eq!(converted["moderation"], json!({"future":{"keep":true}}));
}
#[tokio::test]
async fn optional_chat_controls_degrade_without_blocking_upstream_execution() {
for stop in [json!("end"), json!(["end", "stop"]), json!([])] {
let converted = convert(json!({"model":"model-a", "messages":[
{"role":"user", "name":"speaker", "content":"hello"},
{"role":"assistant", "reasoning_content":"display-only", "content":"previous"}
], "stop":stop, "seed":7, "frequency_penalty":0.3, "presence_penalty":0.5,
"logit_bias":{"10":-10}, "prediction":{"type":"content", "content":"hint"}}))
.await;
assert_eq!(converted["input"][0]["content"][0]["text"], "hello");
assert_eq!(converted["input"][1]["content"][0]["text"], "previous");
for field in [
"stop",
"seed",
"frequency_penalty",
"presence_penalty",
"logit_bias",
"prediction",
] {
assert!(converted.get(field).is_none(), "{field}");
}
}
}
#[tokio::test]
async fn shared_options_and_native_extensions_reach_upstream_without_model_prejudgment() {
let mut request = chat_request(false);
request.as_object_mut().unwrap().extend(json!({
"max_output_tokens":10, "temperature":3.0, "top_p":1.5,
"reasoning":{"effort":"low", "summary":"auto", "future":true}, "reasoning_effort":"high",
"text":{"verbosity":"low", "future":true}, "verbosity":"high",
"response_format":{"type":"future_format", "future":true},
"tools":[{"type":"file_search", "vector_store_ids":["vs_test"]}],
"tool_choice":"future_choice", "logprobs":true, "top_logprobs":3,
"include":["reasoning.encrypted_content"], "truncation":"auto", "max_tool_calls":5
}).as_object().unwrap().clone());
let converted = convert(request).await;
assert_eq!(converted["max_output_tokens"], 10);
assert_eq!(converted["temperature"], 3.0);
assert_eq!(converted["top_p"], 1.5);
assert_eq!(
converted["reasoning"],
json!({"effort":"high", "summary":"auto", "future":true})
);
assert_eq!(
converted["text"],
json!({"verbosity":"high", "future":true, "format":{"type":"future_format", "future":true}})
);
assert_eq!(converted["tool_choice"], "future_choice");
assert_eq!(
converted["include"],
json!([
"reasoning.encrypted_content",
"message.output_text.logprobs"
])
);
assert_eq!(
converted["tools"][0]["vector_store_ids"],
json!(["vs_test"])
);
assert_eq!(converted["top_logprobs"], 3);
}
#[tokio::test]
async fn custom_tools_keep_native_input_and_mixed_call_families_through_history() {
let schema = json!({"type":"grammar", "grammar":{"syntax":"lark", "definition":"start: /.+/"}});
let converted = convert(json!({"model":"model-a", "tools":[
{"type":"custom", "custom":{"name":"patch", "format":schema}},
{"type":"function", "function":{"name":"lookup"}}
], "tool_choice":{"type":"allowed_tools", "allowed_tools":{"mode":"auto", "tools":[
{"type":"custom", "custom":{"name":"patch"}}, {"type":"function", "function":{"name":"lookup"}}
]}}, "messages":[
{"role":"assistant", "tool_calls":[
{"id":"call_patch", "type":"custom", "custom":{"name":"patch", "input":"raw\ninput"}},
{"id":"call_lookup", "type":"function", "function":{"name":"lookup", "arguments":"{}"}}
]},
{"role":"tool", "tool_call_id":"call_patch", "content":"done"},
{"role":"tool", "tool_call_id":"call_lookup", "content":[{"type":"image_url", "image_url":{"url":"https://example.com/result.png"}}]},
{"role":"assistant", "tool_calls":[{"id":"call_patch", "type":"function", "function":{"name":"lookup", "arguments":"{}"}}]},
{"role":"tool", "tool_call_id":"call_patch", "content":"reused id"}
]})).await;
assert_eq!(
converted["tools"][0],
json!({"type":"custom", "name":"patch", "format":schema})
);
assert_eq!(
converted["tool_choice"],
json!({"type":"allowed_tools", "mode":"auto", "tools":[{"type":"custom","name":"patch"},{"type":"function","name":"lookup"}]})
);
assert_eq!(
converted["input"][0],
json!({"type":"custom_tool_call","call_id":"call_patch","name":"patch","input":"raw\ninput"})
);
assert_eq!(converted["input"][2]["type"], "custom_tool_call_output");
assert_eq!(converted["input"][3]["output"][0]["type"], "input_image");
assert_eq!(converted["input"][5]["type"], "function_call_output");
}
#[tokio::test]
async fn files_and_audio_are_preserved_as_inputs_instead_of_losing_the_user_turn() {
for file in [
json!({"file_id":"file_test"}),
json!({"file_url":"https://example.com/report.txt"}),
json!({"file_data":"data:text/plain;base64,aGk=", "filename":"report.txt", "detail":"high"}),
] {
let converted = convert(json!({"model":"model-a", "messages":[{"role":"user", "content":[
{"type":"file", "file":file}, {"type":"input_audio", "input_audio":{"data":"aGk=", "format":"mp3"}}
]}]})).await;
let mut expected = file;
expected["type"] = json!("input_file");
assert_eq!(converted["input"][0]["content"][0], expected);
assert_eq!(
converted["input"][0]["content"][1],
json!({"type":"input_audio","audio_url":"data:audio/mpeg;base64,aGk="})
);
}
}
@@ -61,11 +61,77 @@ async fn json_projects_public_identity_text_and_actual_usage() {
);
}
#[tokio::test]
async fn web_search_projects_citations_in_json_and_streams_the_original_links() {
let linked = "[IANA](https://www.iana.org/)";
let text = format!("见 {linked}");
let citation = json!({"type":"url_citation","url":"https://www.iana.org/","title":"IANA",
"start_index":2,"end_index":text.chars().count()});
let mut prefix = message("🌍 ");
prefix["id"] = json!("msg_intro");
let search = json!({"type":"web_search_call","id":"ws_lookup","status":"completed",
"action":{"type":"search","query":"IANA"}});
let mut answer = message(&text);
answer["content"][0]["annotations"] = json!([citation]);
for stream in [false, true] {
let fixture = Fixture::new(Session::new(vec![
created(),
wire(
"response.output_item.done",
json!({"output_index":0,"item":prefix}),
),
wire(
"response.output_item.done",
json!({"output_index":1,"item":search}),
),
wire(
"response.output_text.delta",
json!({"output_index":2,"content_index":0,
"item_id":"msg_chat_test","delta":text}),
),
wire(
"response.output_text.annotation.added",
json!({"output_index":2,"content_index":0,
"item_id":"msg_chat_test","annotation_index":0,"annotation":citation}),
),
wire(
"response.output_item.done",
json!({"output_index":2,"item":answer}),
),
completed(json!([])),
]))
.await;
let mut request = chat_request(stream);
request["web_search_options"] = json!({});
let response = fixture.post(request).await;
assert_eq!(response.status(), StatusCode::OK);
let expected = format!("🌍 {text}");
if stream {
let (_, chunks) = sse_body(response).await;
assert_eq!(combined(&chunks, "content"), expected);
assert_eq!(
chunks.last().unwrap()["choices"][0]["finish_reason"],
"stop"
);
} else {
let body = json_body(response).await;
let choice = &body["choices"][0];
assert_eq!(choice["finish_reason"], "stop");
assert_eq!(choice["message"]["content"], expected);
assert_eq!(
choice["message"]["annotations"],
json!([{"type":"url_citation","url_citation":{
"url":"https://www.iana.org/","title":"IANA","start_index":4,"end_index":expected.chars().count()}}])
);
}
}
}
#[tokio::test]
async fn json_tool_arguments_and_refusal_do_not_become_assistant_text() {
for output in [
json!([tool("fc1", "call1", "weather", "{ \"city\" : \"北京\" }")]),
json!([{"type":"reasoning","id":"rs1","summary":[{"type":"summary_text","text":"private"}]},
json!([{"type":"reasoning","id":"rs1","summary":[{"type":"summary_text","text":"public summary"}]},
{"type":"message","id":"refusal1","role":"assistant","status":"completed",
"content":[{"type":"refusal","refusal":"cannot help"}]}]),
] {
@@ -75,7 +141,9 @@ async fn json_tool_arguments_and_refusal_do_not_become_assistant_text() {
let body = json_body(response).await;
let choice = &body["choices"][0];
assert_eq!(choice["message"]["content"], Value::Null);
assert!(!body.to_string().contains("private"));
if output[0]["type"] == "reasoning" {
assert_eq!(choice["message"]["reasoning_content"], "public summary");
}
if output[0]["type"] == "function_call" {
assert_eq!(choice["finish_reason"], "tool_calls");
assert_eq!(
@@ -230,7 +298,7 @@ async fn extra_attributes_and_auxiliary_events_preserve_existing_chat_results()
let fixture = Fixture::new(Session::new(vec![
wire("response.future_auxiliary", json!({"future":true})),
wire(
"response.reasoning_summary_text.delta",
"response.future_reasoning.delta",
json!({"output_index":0,"summary_index":0,"delta":"private"}),
),
created(),
@@ -253,6 +321,45 @@ async fn extra_attributes_and_auxiliary_events_preserve_existing_chat_results()
);
}
#[tokio::test]
async fn compaction_state_preserves_chat_answer_without_exposing_ciphertext() {
for kind in ["compaction", "compaction_summary", "context_compaction"] {
for streaming in [false, true] {
let state =
json!({"type":kind,"id":"ctx_1","encrypted_content":"opaque-private-state"});
let fixture = Fixture::new(Session::new(vec![
created(),
wire(
"response.output_item.added",
json!({"output_index":0,"item":state}),
),
wire(
"response.output_item.done",
json!({"output_index":0,"item":state}),
),
completed(json!([state, message("preserved")])),
]))
.await;
let response = fixture.post(chat_request(streaming)).await;
if streaming {
let (text, chunks) = sse_body(response).await;
assert_eq!(combined(&chunks, "content"), "preserved");
assert!(!text.contains("opaque-private-state"));
assert_eq!(
chunks.last().unwrap()["choices"][0]["finish_reason"],
"stop"
);
} else {
assert_eq!(response.status(), StatusCode::OK);
let body = json_body(response).await;
assert_eq!(body["choices"][0]["message"]["content"], "preserved");
assert_eq!(body["choices"][0]["finish_reason"], "stop");
assert!(!body.to_string().contains("opaque-private-state"));
}
}
}
}
#[tokio::test]
async fn incomplete_projects_only_supported_finish_reasons() {
for (reason, expected) in [
@@ -289,7 +396,7 @@ async fn incomplete_projects_only_supported_finish_reasons() {
async fn encrypted_reasoning_stays_private_while_public_text_and_usage_are_projected() {
let reasoning = json!({
"id":"rs_private","type":"reasoning",
"summary":[{"type":"summary_text","text":"PRIVATE_SUMMARY_MARKER"}],
"summary":[{"type":"summary_text","text":"PUBLIC_SUMMARY_MARKER"}],
"encrypted_content":"OPAQUE_ENCRYPTED_REASONING_MARKER"
});
for streaming in [false, true] {
@@ -301,7 +408,7 @@ async fn encrypted_reasoning_stays_private_while_public_text_and_usage_are_proje
),
wire(
"response.reasoning_summary_text.delta",
json!({"item_id":"rs_private","output_index":0,"summary_index":0,"delta":"PRIVATE_SUMMARY_MARKER"}),
json!({"item_id":"rs_private","output_index":0,"summary_index":0,"delta":"PUBLIC_SUMMARY_MARKER"}),
),
wire(
"response.output_item.done",
@@ -311,7 +418,13 @@ async fn encrypted_reasoning_stays_private_while_public_text_and_usage_are_proje
"response.output_item.done",
json!({"output_index":1,"item":message("public answer")}),
),
completed(json!([reasoning, message("public answer")])),
wire("response.completed", json!({"response":{
"id":"resp_chat_test","object":"response","model":"upstream-model","status":"completed",
"output":[reasoning,message("public answer")],
"usage":{"input_tokens":11,"output_tokens":7,"total_tokens":18,
"input_tokens_details":{"cached_tokens":3,"cache_write_tokens":4},
"output_tokens_details":{"reasoning_tokens":2,"accepted_prediction_tokens":1}}
}})),
]))
.await;
let mut request = chat_request(streaming);
@@ -340,12 +453,17 @@ async fn encrypted_reasoning_stays_private_while_public_text_and_usage_are_proje
)
};
assert_eq!(content, "public answer");
assert!(!body.contains("PRIVATE_SUMMARY_MARKER"), "{body}");
assert!(body.contains("PUBLIC_SUMMARY_MARKER"), "{body}");
assert!(
!body.contains("OPAQUE_ENCRYPTED_REASONING_MARKER"),
"{body}"
);
assert_eq!(usage["completion_tokens_details"]["reasoning_tokens"], 2);
assert_eq!(
usage["completion_tokens_details"]["accepted_prediction_tokens"],
1
);
assert_eq!(usage["prompt_tokens_details"]["cache_write_tokens"], 4);
}
}
@@ -353,7 +471,7 @@ async fn encrypted_reasoning_stays_private_while_public_text_and_usage_are_proje
async fn unrepresentable_business_output_fails_without_successful_chat_completion() {
for output in [
json!({"id":"future_output","type":"future_business_output","payload":"unrepresentable"}),
json!({"id":"ig_native","type":"image_generation_call","status":"completed","result":"OPAQUE_IMAGE_MARKER"}),
json!({"id":"action_native","type":"computer_call","status":"completed","call_id":"call_action","action":{"type":"click","x":1,"y":2}}),
] {
for streaming in [false, true] {
let fixture =
@@ -594,3 +712,183 @@ async fn conflicting_text_snapshot_fails_instead_of_appending_replacement_conten
.any(|chunk| chunk["choices"][0]["finish_reason"].is_string())
);
}
#[tokio::test]
async fn custom_and_function_calls_keep_distinct_shapes_in_json_and_interleaved_streams() {
let custom = json!({"id":"ct_1","type":"custom_tool_call","call_id":"call_custom","name":"patch","input":"first\nsecond"});
let function = json!({"id":"fc_1","type":"function_call","call_id":"call_function","name":"lookup","arguments":"{}"});
for streaming in [false, true] {
let mut initial_custom = custom.clone();
initial_custom["input"] = json!("");
let mut initial_function = function.clone();
initial_function["arguments"] = json!("");
let fixture = Fixture::new(Session::new(vec![
created(),
wire(
"response.output_item.added",
json!({"output_index":0,"item":initial_custom}),
),
wire(
"response.output_item.added",
json!({"output_index":1,"item":initial_function}),
),
wire(
"response.custom_tool_call_input.delta",
json!({"output_index":0,"item_id":"ct_1","delta":"first\n"}),
),
wire(
"response.function_call_arguments.delta",
json!({"output_index":1,"item_id":"fc_1","delta":"{}"}),
),
wire(
"response.custom_tool_call_input.done",
json!({"output_index":0,"item_id":"ct_1","input":"first\nsecond"}),
),
completed(json!([custom, function])),
]))
.await;
let response = fixture.post(chat_request(streaming)).await;
assert_eq!(response.status(), StatusCode::OK);
if streaming {
let (_, chunks) = sse_body(response).await;
let calls = chunks
.iter()
.filter_map(|chunk| chunk["choices"][0]["delta"]["tool_calls"].as_array())
.flatten()
.collect::<Vec<_>>();
assert_eq!(
calls.iter().filter(|call| call["type"] == "custom").count(),
1
);
assert_eq!(
calls
.iter()
.filter(|call| call["type"] == "function")
.count(),
1
);
assert_eq!(
calls
.iter()
.filter_map(|call| call["custom"]["input"].as_str())
.collect::<String>(),
"first\nsecond"
);
assert_eq!(
calls
.iter()
.filter_map(|call| call["function"]["arguments"].as_str())
.collect::<String>(),
"{}"
);
assert_eq!(
chunks.last().unwrap()["choices"][0]["finish_reason"],
"tool_calls"
);
} else {
let body = json_body(response).await;
assert_eq!(
body["choices"][0]["message"]["tool_calls"],
json!([
{"id":"call_custom","type":"custom","custom":{"name":"patch","input":"first\nsecond"}},
{"id":"call_function","type":"function","function":{"name":"lookup","arguments":"{}"}}
])
);
}
}
}
#[tokio::test]
async fn image_previews_and_final_images_are_projected_once_per_distinct_snapshot() {
let image = json!({"id":"ig_one","type":"image_generation_call","status":"completed","result":"ZmluYWw=","output_format":"webp"});
for streaming in [false, true] {
let fixture = Fixture::new(Session::new(vec![
created(),
wire("response.image_generation_call.partial_image", json!({"output_index":0,"item_id":"ig_one","partial_image_b64":"cHJldmlldw==","output_format":"webp"})),
wire("response.output_item.done", json!({"output_index":0,"item":image})),
completed(json!([image, message("created")])),
])).await;
let response = fixture.post(chat_request(streaming)).await;
assert_eq!(response.status(), StatusCode::OK);
if streaming {
let (_, chunks) = sse_body(response).await;
let images = chunks
.iter()
.filter_map(|chunk| chunk["choices"][0]["delta"]["images"].as_array())
.flatten()
.collect::<Vec<_>>();
assert_eq!(images.len(), 2);
assert_eq!(images[1]["index"], 0);
assert_eq!(
images[1]["image_url"]["url"],
"data:image/webp;base64,ZmluYWw="
);
} else {
let body = json_body(response).await;
assert_eq!(
body["choices"][0]["message"]["images"][0]["image_url"]["url"],
"data:image/webp;base64,ZmluYWw="
);
}
}
}
#[tokio::test]
async fn output_logprobs_survive_json_projection_and_stream_snapshot_deduplication() {
let probabilities =
json!([{"token":"hello","logprob":-0.1,"bytes":[104,101,108,108,111],"top_logprobs":[]}]);
let mut output = message("hello");
output["content"][0]["logprobs"] = probabilities.clone();
for streaming in [false, true] {
let fixture = Fixture::new(Session::new(vec![
created(),
wire("response.output_text.delta", json!({"output_index":0,"content_index":0,"item_id":"msg_chat_test","delta":"hello","logprobs":probabilities})),
wire("response.output_text.done", json!({"output_index":0,"content_index":0,"item_id":"msg_chat_test","text":"hello","logprobs":probabilities})),
completed(json!([output])),
])).await;
let response = fixture.post(chat_request(streaming)).await;
assert_eq!(response.status(), StatusCode::OK);
if streaming {
let (_, chunks) = sse_body(response).await;
let values = chunks
.iter()
.filter_map(|chunk| chunk["choices"][0]["logprobs"]["content"].as_array())
.flatten()
.cloned()
.collect::<Vec<_>>();
assert_eq!(json!(values), probabilities);
} else {
assert_eq!(
json_body(response).await["choices"][0]["logprobs"]["content"],
probabilities
);
}
}
}
#[tokio::test]
async fn streaming_citation_offsets_follow_delivered_text_when_messages_interleave() {
let citation = json!({"type":"url_citation","start_index":2,"end_index":4,"url":"https://example.com","title":"source"});
let mut first = message("abcd");
first["content"][0]["annotations"] = json!([citation]);
let mut second = message("xy");
second["id"] = json!("msg_second");
let fixture = Fixture::new(Session::new(vec![
created(),
wire("response.output_text.delta", json!({"output_index":0,"content_index":0,"item_id":"msg_chat_test","delta":"ab"})),
wire("response.output_text.delta", json!({"output_index":1,"content_index":0,"item_id":"msg_second","delta":"xy"})),
wire("response.output_text.delta", json!({"output_index":0,"content_index":0,"item_id":"msg_chat_test","delta":"cd"})),
wire("response.output_text.annotation.added", json!({"output_index":0,"content_index":0,"item_id":"msg_chat_test","annotation_index":0,"annotation":citation})),
completed(json!([first, second])),
])).await;
let (_, chunks) = sse_body(fixture.post(chat_request(true)).await).await;
assert_eq!(combined(&chunks, "content"), "abxycd");
let annotations = chunks
.iter()
.filter_map(|chunk| chunk["choices"][0]["delta"]["annotations"].as_array())
.flatten()
.collect::<Vec<_>>();
assert_eq!(annotations.len(), 1);
assert_eq!(annotations[0]["url_citation"]["start_index"], 4);
assert_eq!(annotations[0]["url_citation"]["end_index"], 6);
}
@@ -390,7 +390,7 @@ fn no_eligible_provider_account_should_preserve_safe_provider_detail() {
);
assert_eq!(
gateway.safe_message(),
"No upstream provider is currently available for this request."
"No upstream provider is available for this request."
);
}
@@ -423,7 +423,7 @@ fn no_eligible_provider_account_should_map_to_service_unavailable() {
assert_eq!(projected.kind(), GatewayErrorKind::NoAvailableProvider);
assert_eq!(
projected.safe_message(),
"No upstream provider is currently available for this request."
"No upstream provider is available for this request."
);
}
@@ -69,6 +69,10 @@ async fn fixture(sharing: ClientPeerSharing, plaintext: &str) -> (AdminTestFixtu
.await
.unwrap()
.record;
key.limits = RateLimits {
max_concurrency: 5,
requests_per_minute: 60,
};
key.peer_sharing = sharing;
key.groups = vec![group.clone()];
key.name = "private-key-sentinel".into();
@@ -243,7 +247,11 @@ async fn exports_scoped_observations_even_when_the_key_budget_is_exhausted_witho
assert_eq!(response.status(), StatusCode::OK);
let body = response_json(response).await;
assert_eq!(body["object"], "cpr.peer");
assert_eq!(body["schema_version"], 2);
assert_eq!(body["schema_version"], 3);
assert_eq!(
body["key_limits"],
json!({"max_concurrency":5,"requests_per_minute":60})
);
assert_eq!(body["usage"]["data"]["windows"][0]["remaining"], "0");
assert_eq!(body["usage"]["data"]["allowed"], false);
assert_eq!(body["usage"]["data"]["windows"][0]["limit_reached"], true);
@@ -358,7 +366,10 @@ async fn plan_only_summary_limits_exported_data_and_preserves_unknown_on_quota_r
body["pool"]["plans"],
json!([{ "name": "Free", "count": 1 }])
);
assert_eq!(body["capabilities"], json!(["key_usage", "pool_summary"]));
assert_eq!(
body["capabilities"],
json!(["key_usage", "key_limits", "pool_summary"])
);
assert!(body.get("quota").is_none());
*fixture.provider_quota.lock().unwrap() = None;
@@ -356,7 +356,7 @@ async fn rewrite_upgrade_and_control_active_response_keep_http_context_until_con
),
);
let (trace, mut socket, server) =
start_active_response_with_middleware(Some(frozen), "/custom/websocket").await;
start_active_response_with(Some(frozen), "/custom/websocket", "codex-tui/0.166.0").await;
// 升级后的默认会话仍持有入口上下文;101 正文结束不再提前取消它
assert!(
!plan
@@ -388,6 +388,7 @@ async fn rewrite_upgrade_and_control_active_response_keep_http_context_until_con
assert!(matches!(reply, ClientMessage::Binary(bytes) if bytes.as_ref() == [0,255,1]));
assert_eq!(trace.starts.load(Ordering::SeqCst), 1);
assert!(!trace.cancelled.load(Ordering::SeqCst));
trace.reconnect_for_replay.store(true, Ordering::Release);
trace.release_terminal.notify_one();
let reply = tokio::time::timeout(Duration::from_secs(2), socket.next())
.await
@@ -400,6 +401,16 @@ async fn rewrite_upgrade_and_control_active_response_keep_http_context_until_con
let event: Value = serde_json::from_str(&text).unwrap();
assert_eq!(event["type"], "response.completed");
assert_eq!(event["plugin"], true);
socket
.send(ClientMessage::Ping(vec![9].into()))
.await
.unwrap();
let reply = tokio::time::timeout(Duration::from_secs(2), socket.next())
.await
.expect("Pong after middleware rewrote the terminal")
.unwrap()
.unwrap();
assert!(matches!(reply, ClientMessage::Pong(_)));
assert!(plan.incoming.load(Ordering::SeqCst) >= 2);
assert!(plan.outgoing.load(Ordering::SeqCst) >= 3);
socket.close(None).await.unwrap();
@@ -441,7 +452,7 @@ async fn input_plugin_runs_while_a_real_socket_write_is_blocked() {
),
);
let (trace, mut socket, server) =
start_active_response_with_middleware(Some(frozen), "/custom/websocket").await;
start_active_response_with(Some(frozen), "/custom/websocket", "codex-tui/0.166.0").await;
trace.release_terminal.notify_one();
// 仅窥视帧头以确认写入开始,不消费 64 MiB 输出,保留真实 TCP 接收窗口背压
let MaybeTlsStream::Plain(transport) = socket.get_ref() else {
@@ -49,6 +49,7 @@ use crate::openai::{api_router, authenticated_client};
#[derive(Default)]
struct CrossingLimitTrace {
reconnect_for_replay: AtomicBool,
release_terminal: tokio::sync::Notify,
starts: AtomicUsize,
inputs: Mutex<Vec<Value>>,
@@ -74,7 +75,9 @@ impl ExecutionSession for CrossingLimitSession {
if !self.terminal_emitted {
self.trace.release_terminal.notified().await;
self.terminal_emitted = true;
return Ok(Some(crossing_limit_terminal_batch()));
return Ok(Some(crossing_limit_terminal_batch(
self.trace.reconnect_for_replay.load(Ordering::Acquire),
)));
}
self.finalized = true;
self.trace.finalized.store(true, Ordering::Release);
@@ -197,8 +200,8 @@ fn crossing_limit_started_batch() -> CoordinatedEvent {
.expect("started WebSocket batch")
}
fn crossing_limit_terminal_batch() -> CoordinatedEvent {
let completed = ProviderEvent::wire(
fn crossing_limit_terminal_batch(reconnect: bool) -> CoordinatedEvent {
let mut completed = ProviderEvent::wire(
ProtocolWireEvent::json(
"openai",
Some("response.completed".to_owned()),
@@ -207,12 +210,20 @@ fn crossing_limit_terminal_batch() -> CoordinatedEvent {
"response": {
"id": "resp_ws_crossing_limit",
"model": "model-a",
"output": [],
"status": "completed"
}
}),
)
.expect("completed wire"),
);
if reconnect {
completed.attach_session_update(
gateway_core::operation::ProviderSessionState::new("openai", serde_json::Map::new())
.unwrap()
.with_websocket_reconnect(),
);
}
CoordinatedEvent::try_batch(vec![completed], CommitRequirement::AlreadyCommitted)
.expect("terminal WebSocket batch")
}
@@ -816,12 +827,13 @@ async fn start_active_response() -> (
TestClientSocket,
tokio::task::JoinHandle<()>,
) {
start_active_response_with_middleware(None, "/v1/responses").await
start_active_response_with(None, "/v1/responses", "integration-test").await
}
async fn start_active_response_with_middleware(
async fn start_active_response_with(
plan: Option<gateway_core::engine::middleware::FrozenMiddlewarePlan>,
path: &str,
user_agent: &str,
) -> (
Arc<CrossingLimitTrace>,
TestClientSocket,
@@ -872,6 +884,9 @@ async fn start_active_response_with_middleware(
.parse()
.expect("authorization"),
);
request
.headers_mut()
.insert("user-agent", user_agent.parse().unwrap());
let (mut socket, _) = connect_async(request).await.expect("upgrade WebSocket");
let response_create = ClientMessage::Text(
json!({
@@ -905,8 +920,9 @@ async fn start_active_response_with_middleware(
}
#[tokio::test]
async fn requests_received_during_an_active_response_should_execute_in_order() {
async fn replay_reconnect_waits_for_queued_requests_and_writes_the_last_terminal() {
let (trace, mut socket, server) = start_active_response().await;
trace.reconnect_for_replay.store(true, Ordering::Release);
for input in ["second", "third"] {
socket
.send(ClientMessage::Text(
@@ -964,7 +980,13 @@ async fn requests_received_during_an_active_response_should_execute_in_order() {
vec![json!("hello"), json!("second"), json!("third")]
);
assert!(!trace.cancelled.load(Ordering::Acquire));
socket.close(None).await.expect("close WebSocket");
let next = tokio::time::timeout(Duration::from_secs(2), socket.next())
.await
.expect("replay close timeout")
.unwrap()
.unwrap();
assert!(matches!(next, ClientMessage::Close(Some(frame))
if frame.code == CloseCode::Normal && frame.reason == "Reconnect with full input."));
server.abort();
}

Some files were not shown because too many files have changed in this diff Show More