Compare commits

...
15 Commits
Author SHA1 Message Date
ZyyandCodex bad2a245ab chore(release): v3.29.4
Co-authored-by: Codex <noreply@openai.com>
2026-10-19 12:59:43 +08:00
ZyyandCodex b49b4528e7 fix(store): prevent analytics stalls during history cleanup
Batch empty-group deletion and extrema rebuilds, and reserve queue capacity
for live requests while historical contributions are removed.

Keep adaptive batch sizes across worker cycles so timed-out batches can
shrink and make progress without changing transaction or version ACK rules.

Add regression coverage for rollback, fair processing, and timeout recovery.
Validated with 452 Store tests, strict Clippy, and delayed PostgreSQL stress
with 30,000 historical requests plus 12,000 new requests at 200 per second.

Co-authored-by: Codex <noreply@openai.com>
2026-10-19 12:57:08 +08:00
dependabot[bot]andCodex d678f350c1 chore(deps): update frontend dependencies with compatible vue-tsc (#473)
Update nine frontend dependencies and pin vue-tsc to 3.3.11 to avoid the upstream imported-binding code generation regression.

Refs: https://github.com/vuejs/language-tools/issues/6237

Co-authored-by: Codex <noreply@openai.com>
2026-10-19 12:14:01 +08:00
dependabot[bot]andCodex 17b05d4671 chore(deps): update backend dependencies and sha2 build profile (#471)
Update serde_json, sha2, syn, tokio-util, and uuid. Keep the sha2 dev/test optimization profile aligned with version 0.11.1.

Co-authored-by: Codex <noreply@openai.com>
2026-10-19 12:13:39 +08:00
ZyyandCodex 7148303ad6 chore(deps): update plugins submodule to v0.5.0
Pin modules/plugins to the published v0.5.0 release at 7500732.

Co-authored-by: Codex <noreply@openai.com>
2026-10-19 11:36:13 +08:00
ZyyandCodex f6538d433d test: align backend coverage with current behavior
Consolidate overlapping scenarios and remove obsolete assertions and
migration-specific SQL tests. Keep persistence fixtures on the current schema
and document the test maintenance rules in the development skill.

Validation: 3,997 tests passed across 12 crates, plus rustfmt, strict Clippy,
and skill validation. All 33 migration SQL files passed temporary isolated
execution, and the opt-in xAI official network test passed.

Co-authored-by: Codex <noreply@openai.com>
2026-10-19 11:36:06 +08:00
ZyyandCodex 406256bcc0 fix(ci): check frontend dependencies in the correct directory
Co-authored-by: Codex <noreply@openai.com>
2026-10-19 10:43:30 +08:00
ZyyandCodex d912fa0c59 fix(ui): auto-hide key usage log scrollbars
Co-authored-by: Codex <noreply@openai.com>
2026-10-18 23:58:15 +08:00
ZyyandCodex a166580aed fix(ui): remove redundant failure messages and concurrency controls
Co-authored-by: Codex <noreply@openai.com>
2026-10-18 23:31:12 +08:00
ZyyandCodex d1aec7311c fix(proxies): refresh regions and hide stale detection results
Co-authored-by: Codex <noreply@openai.com>
2026-10-18 23:31:01 +08:00
ZyyandCodex 16d8a3f873 chore(release): v3.29.3
Co-authored-by: Codex <noreply@openai.com>
2026-10-18 23:14:48 +08:00
ZyyandCodex ac6033f6fa docs: refine GitHub reply wording
Co-authored-by: Codex <noreply@openai.com>
2026-10-18 23:10:58 +08:00
cef9f0b21e fix(openai): honor usable credits and recover quota after reset (#469)
Refs #465

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-18 22:45:15 +08:00
591f95cc18 fix(store): accept managed PostgreSQL and Redis passwords (#467)
* fix(store): accept managed PostgreSQL passwords

Co-authored-by: Codex <noreply@openai.com>

* fix(store): unify PostgreSQL and Redis password validation

Co-authored-by: Codex <noreply@openai.com>

* fix(store): preserve literal percent sequences in passwords

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-10-18 20:34:20 +08:00
ZyyandCodex 99ba5b1120 fix(openai): align request ordering and simplify timing metrics
Preserve response.create field order and align OAuth query ordering with
current official Codex sources.

Stop adding the timing-metrics request header by default and remove the
four unused engine TTFT and token interval fields throughout the backend,
plugin observation contracts, and frontend. Keep rate calculations unchanged.

Co-authored-by: Codex <noreply@openai.com>
2026-10-18 16:32:15 +08:00
128 changed files with 2609 additions and 2972 deletions
+11
View File
@@ -15,6 +15,7 @@ description: Codex Proxy RS 前后端开发、排障、代码审查和文档修
| 修改服务端接口、路由、调度、账号与用量规则、持久化、后台任务或协议行为;目标是 `backend/` 中的实现、配置、迁移,或 Rust 源码 | [Workspace 边界](../../../docs/architecture.md#3-workspace-边界)、所属模块、[后端自审](../../../docs/architecture.md#后端自审);涉及 Rust 时使用 `$rust-best-practices` 和下方后端注释要求,验证时读[命令](../../../docs/architecture.md#验证命令) |
| 修改页面接口调用,或开发、排查、审查面向管理端或模型客户端的 HTTP/WS API 合同 | [API](../../../docs/api.md)对应章节;优先复用已有接口,核对调用方 → 服务端解析 → 实际查询或执行,不能只凭前端类型认定参数生效;按实际改动叠加前后端规范 |
| 模块或调用链变更 | [Workspace 边界](../../../docs/architecture.md#3-workspace-边界)及所属模块章节,不通读整份架构 |
| 新增、调整、清理或审查测试 | 下方[测试维护](#测试维护),按当前行为、职责归属与已有覆盖判断保留、合并或删除 |
| 修改部署行为、`deploy/` 配置或数据库迁移 | 按范围读取[部署](../../../deploy/README.md)、[迁移](../../../backend/migrations/README.md)对应章节 |
| 开发、排查或审查宿主 SDK、Runtime、扩展接口或独立插件;目标在 `backend/crates/gateway-plugin/` | [插件职责边界](references/plugin-boundaries.md)及所用 [SDK](../../../backend/crates/gateway-plugin/sdk/README.md) 类型与能力合同;宿主守住自身合同、资源与稳定性,不猜测或补偿插件业务错误;仅独立插件创作、排查另用 [cpr-plugin-dev](../cpr-plugin-dev/SKILL.md) |
| 修改或审查文档 | [文档检查](references/documentation.md)及目标章节,不默认读取代码开发、页面验收或发布流程 |
@@ -35,6 +36,16 @@ description: Codex Proxy RS 前后端开发、排障、代码审查和文档修
4. 按变更范围执行[验证](../../../CONTRIBUTING.md#验证)及其停止条件,沿用本轮有效证据与用户已确定的验收分工,区分通过、失败、跳过与未执行;提交时补读[提交约定](../../../CONTRIBUTING.md#项目约定)
5. 交付实际结果与验证缺口;前端按[前端流程](references/frontend.md)核对最终差异和运行证据,后端按上表检查生命周期,不把构建通过当作行为验收
## 测试维护
- 测试随当前功能维护,名称、夹具和断言描述当前输入、行为与结果;功能变更时优先更新所属模块的现有用例,不为每次改动叠加一组记录变更经过的测试
- 清理只证明旧字段、旧路由或旧实现已经消失的用例与附带断言;未知字段校验使用合法请求基线加无业务含义的未知字段,避免因缺少必填项等其他错误而误通过
- 同一层对同一行为的重复覆盖合并为清晰的场景矩阵,复用相邻测试夹具;跨层测试确有不同职责或失败路径时保留,不为统一而扩大测试辅助抽象
- 断言可观察的行为与业务不变量,避免复述结构体赋值、罗列表列或字段清单、匹配 SQL 文本与源码布局;项目明确要求的架构边界检查继续维护
- 当前协议兼容、数据读取兼容、安全边界、并发和资源生命周期仍需覆盖,不按 `old`、`legacy` 等名称机械删除,也不能用改名代替无效用例清理
- 不在仓库中新增或保留迁移 SQL 专项测试,包括迁移文本、schema 演变与指定版本升级用例;需要验证迁移执行、数据保留或回滚时,在临时目录编写一次性验证并使用隔离数据库,测试通过即可,临时脚本不纳入维护测试集
- 持久化功能测试使用当前完整 schema,验证现行读写、事务与数据约束;插件运行时状态迁移等业务能力按当前功能测试,不归为迁移 SQL 专项测试
## 后端注释
- 每个 Rust 文件顶层使用简短的中文 `//!` 注释说明文件职责,覆盖生产源码、测试与构建脚本;测试文件说明测试范围或辅助用途,生成文件同步维护生成器
+1 -1
View File
@@ -52,7 +52,7 @@ updates:
- patch
- package-ecosystem: npm
directory: /
directory: /frontend
labels: []
schedule:
interval: weekly
+1 -1
View File
@@ -56,7 +56,7 @@ jobs:
const marker = '<!-- agents-md-guard -->';
const body = [
marker,
'佬友,我看到这个 PR 涉及受保护的协作规则文件,按仓库约定需要先关闭',
'我看到这个 PR 涉及受保护的协作规则文件,按仓库约定需要先关闭',
'',
'保护范围包括根目录或子目录的 `AGENTS.md`、根目录的 `CONTRIBUTING.md`,以及 `.agents/skills/` 和 `.github/` 中的所有文件,涵盖新增、修改、删除和重命名',
'',
+2 -1
View File
@@ -23,7 +23,8 @@
AI 草拟或发布 Issue、PR 讨论及审查意见的回复时:
- 用第一人称“我”,自然称呼对方为“佬友”,不逐句重复,也不用“我们”“本助手”代称自己
- 用第一人称“我”,不用“我们”“本助手”代称自己
- 普通回复直接回应问题,不加问候语或称呼;向对方提问时称呼“佬”,同一条回复不重复称呼
- 客气、委婉地回应具体问题,补充信息或提出不同意见时说明缘由,避免命令式、责备式表达
- 使用自然短句,简单回复用短段,并列信息较多时再分点;少用句号,避免客服套话、机械模板和空泛保证
- 只把实际完成的排查、复现、修改和验证写成“我已……”;未采纳的意见说明依据,未确认、待处理或未验证的内容如实交代,说清结论、限制和下一步
+21 -19
View File
@@ -198,7 +198,7 @@ checksum = "82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.6",
"syn 3.0.9",
]
[[package]]
@@ -856,7 +856,7 @@ dependencies = [
"heck",
"proc-macro2",
"quote",
"syn 3.0.6",
"syn 3.0.9",
]
[[package]]
@@ -910,7 +910,7 @@ dependencies = [
"provider-xai",
"serde",
"serde_json",
"syn 3.0.6",
"syn 3.0.9",
"tempfile",
"thiserror",
"tokio",
@@ -1657,7 +1657,7 @@ checksum = "9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.6",
"syn 3.0.9",
]
[[package]]
@@ -1720,7 +1720,7 @@ dependencies = [
"serde_json",
"sha2",
"subtle",
"syn 3.0.6",
"syn 3.0.9",
"tempfile",
"thiserror",
"tokio",
@@ -1915,6 +1915,7 @@ dependencies = [
"gateway-admin",
"gateway-core",
"hex",
"percent-encoding",
"redis",
"secrecy",
"serde",
@@ -3266,7 +3267,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2bfe0f4c752e450fc2faf62654f1c134747922825d5b04ca717b8874f41a40c0"
dependencies = [
"proc-macro2",
"syn 3.0.6",
"syn 3.0.9",
]
[[package]]
@@ -3626,7 +3627,7 @@ checksum = "92ecd8964f8453721699a1ed72037b0db49ce2f5a5138486ee89bed6f67cdf3a"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.6",
"syn 3.0.9",
]
[[package]]
@@ -3999,7 +4000,7 @@ checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.6",
"syn 3.0.9",
]
[[package]]
@@ -4014,10 +4015,11 @@ dependencies = [
[[package]]
name = "serde_json"
version = "1.0.151"
version = "1.0.154"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
checksum = "e7e9cc8b1b85264074fbcc02a88680c4096b1e47df8f739dceb03bf482f04bd6"
dependencies = [
"foldhash",
"indexmap",
"itoa",
"memchr",
@@ -4098,9 +4100,9 @@ checksum = "bbfa15b3dddfee50a0fff136974b3e1bde555604ba463834a7eb7deb6417705d"
[[package]]
name = "sha2"
version = "0.11.0"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4"
checksum = "47d7069beb7d6ac7b9acd1039986e73443f24234f41074da099d6f994ac9ad19"
dependencies = [
"cfg-if",
"cpufeatures 0.3.0",
@@ -4480,9 +4482,9 @@ dependencies = [
[[package]]
name = "syn"
version = "3.0.6"
version = "3.0.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8593e8e72159ed2257d083c7a454a85cbf854f37a0966d8d483aff8c8a3ebcee"
checksum = "d78c8dee4c7bf0e14673097256fed6142ce9d3b85a408189d07482442145823b"
dependencies = [
"proc-macro2",
"quote",
@@ -4580,7 +4582,7 @@ checksum = "fe5197923287db20a58125f0bc85c062f7f2c892de97b18c356f9efb14b28524"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.6",
"syn 3.0.9",
]
[[package]]
@@ -4734,9 +4736,9 @@ dependencies = [
[[package]]
name = "tokio-util"
version = "0.7.19"
version = "0.7.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "494815d09bf52b5548659851081238f0ca39ff638363907596da739561c62c52"
checksum = "e464cf451ba96ebfc6f9b6542f17ee8b8956e33f1e40d9690624e59d7a7f8a4b"
dependencies = [
"bytes",
"futures-core",
@@ -5051,9 +5053,9 @@ checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
[[package]]
name = "uuid"
version = "1.27.0"
version = "1.28.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "97277d36b9c3ace13e58fa6e753f8b0bbbf302a18dd193240d70f9e29681059a"
checksum = "7cc1186384beb7dd8eedea376413fd654937285ea6c9cfbb928dc3043ea4b606"
dependencies = [
"getrandom 0.4.3",
"js-sys",
+1 -1
View File
@@ -43,7 +43,7 @@ opt-level = 2
[profile.dev.package.blake2]
opt-level = 2
[profile.dev.package."sha2@0.11.0"]
[profile.dev.package."sha2@0.11.1"]
opt-level = 2
[profile.dev.package."miniz_oxide@0.9.1"]
+52 -29
View File
@@ -106,8 +106,11 @@ fn config_loader_should_apply_only_explicit_topology_overrides() {
"postgres://codex_proxy@127.0.0.1:5432/codex_proxy",
)
.env("CPR_REDIS_URL", "redis://127.0.0.1:6379/")
.env("CPR_DATABASE_PASSWORD", POSTGRES_PASSWORD)
.env("CPR_REDIS_PASSWORD", REDIS_PASSWORD)
.env(
"CPR_DATABASE_PASSWORD",
"RdsTest9!abcdefGhRdsTest9!abcdefGh",
)
.env("CPR_REDIS_PASSWORD", "ManagedRedis9!abcdefGh")
.status()
.expect("run isolated environment override test");
assert!(status.success());
@@ -152,14 +155,6 @@ fn config_loader_should_ignore_unknown_fields_in_configuration_sections() {
}
}
#[test]
fn config_loader_should_ignore_removed_tls_and_fingerprint_sections() {
let mut document = valid_config_document();
document["openai"]["tls"] = serde_json::json!({});
document["openai"]["fingerprint"] = serde_json::json!({"browser": "removed"});
parse_config(&document.to_string()).expect("removed provider settings are ignored");
}
#[test]
fn config_loader_should_reject_invalid_known_fields_alongside_unknown_fields() {
for port in [serde_json::json!(0), serde_json::json!("not-a-port")] {
@@ -189,7 +184,7 @@ fn config_loader_should_report_startup_configuration_diagnostics() {
] {
let mut document = valid_config_document();
if case == "unused" {
document["openai"]["wire_profile"]["location"] = serde_json::json!(UNUSED_SECRET);
document["openai"]["unused_setting"] = serde_json::json!(UNUSED_SECRET);
} else if case == "missing" {
document["host"]["listen"]
.as_object_mut()
@@ -229,7 +224,7 @@ fn config_loader_should_report_startup_configuration_diagnostics() {
);
match case {
"unused" => assert!(
stderr.contains("openai.wire_profile") && stderr.contains("已忽略"),
stderr.contains("openai.unused_setting") && stderr.contains("已忽略"),
"{stderr}"
),
"missing" => assert!(stderr.contains("host.listen.port"), "{stderr}"),
@@ -253,19 +248,7 @@ fn config_loader_should_allow_missing_provider_configuration() {
let mut document = valid_config_document();
document.as_object_mut().unwrap().remove("openai");
document.as_object_mut().unwrap().remove("xai");
parse_config(&document.to_string()).expect("provider defaults require no YAML identity");
}
#[test]
fn config_loader_should_ignore_removed_provider_identity_sections() {
let mut document = valid_config_document();
document["openai"]["wire_profile"] = serde_json::json!({"codex_version":"legacy-invalid"});
document["xai"] = serde_json::json!({"wire_profile": {"client_identifier": "legacy-client"}});
let (config, _directory) =
parse_config(&document.to_string()).expect("removed identities ignored");
let debug = format!("{config:?}");
assert!(!debug.contains("legacy-invalid"));
assert!(!debug.contains("legacy-client"));
parse_config(&document.to_string()).expect("providers use defaults when omitted");
}
#[test]
@@ -282,13 +265,53 @@ fn config_loader_should_reject_embedded_database_password() {
}
#[test]
fn config_loader_should_reject_non_hex_postgres_password() {
assert_rejected(valid_config().replace(POSTGRES_PASSWORD, &"g".repeat(48)));
fn config_loader_should_accept_managed_store_passwords() {
for password in [
"RdsTest9!abcdefGh".to_owned(),
"RdsTest9!abcdefGhRdsTest9!abcdefGh".to_owned(),
POSTGRES_PASSWORD.to_owned(),
"x".repeat(128),
"Managed@:/?#%&+Password9!".to_owned(),
"数据库密码".repeat(4),
] {
for field in ["database", "redis"] {
let mut document = valid_config_document();
document["store"][field]["password"] = serde_json::json!(password);
let (config, _directory) =
parse_config(&document.to_string()).expect("managed store password");
assert!(!format!("{config:?}").contains(&password));
}
}
}
#[test]
fn config_loader_should_reject_wrong_length_redis_password() {
assert_rejected(valid_config().replace(REDIS_PASSWORD, "1234"));
fn config_loader_should_reject_invalid_store_passwords() {
for password in [
String::new(),
"x".repeat(15),
"x".repeat(129),
" ".repeat(32),
"\u{3000}".repeat(6),
"数据库密码".repeat(9),
"RdsTest9!abcdefGh\u{0000}".to_owned(),
"RdsTest9!abcdefGh\n".to_owned(),
"RdsTest9!abcdefGh\t".to_owned(),
"RdsTest9!abcdefGh\u{007f}".to_owned(),
] {
for field in ["database", "redis"] {
let mut document = valid_config_document();
document["store"][field]["password"] = serde_json::json!(password);
assert_rejected(document.to_string());
}
}
}
#[test]
fn config_loader_should_reject_embedded_redis_password() {
assert_rejected(valid_config().replace(
"redis://127.0.0.1:6379/",
"redis://default:embedded@127.0.0.1:6379/",
));
}
#[test]
@@ -722,10 +722,6 @@ pub struct UsageListRecord {
pub upstream_response_ms: Option<u64>,
pub upstream_api_overhead_ms: Option<f64>,
pub upstream_engine_ms: Option<f64>,
pub upstream_engine_iapi_ttft_ms: Option<f64>,
pub upstream_engine_service_ttft_ms: Option<f64>,
pub upstream_engine_iapi_tbt_ms: Option<f64>,
pub upstream_engine_service_tbt_ms: Option<f64>,
pub latency_ms: Option<u64>,
pub admission_decision_ms: Option<u64>,
pub account_selection_wait_ms: Option<u64>,
@@ -803,10 +799,6 @@ pub struct UsageRecord {
pub upstream_response_ms: Option<u64>,
pub upstream_api_overhead_ms: Option<f64>,
pub upstream_engine_ms: Option<f64>,
pub upstream_engine_iapi_ttft_ms: Option<f64>,
pub upstream_engine_service_ttft_ms: Option<f64>,
pub upstream_engine_iapi_tbt_ms: Option<f64>,
pub upstream_engine_service_tbt_ms: Option<f64>,
pub latency_ms: Option<u64>,
pub admission_decision_ms: Option<u64>,
pub account_selection_wait_ms: Option<u64>,
+9 -27
View File
@@ -5,20 +5,16 @@ use std::str::FromStr as _;
use chrono::{TimeDelta, Utc};
use gateway_core::account::OpaqueProviderData;
use gateway_admin::{
model::{
PageSize, Revision,
accounts::AccountUsage,
auth::LoginCommand,
client_keys::ClientKeyPageSize,
observability::{DecimalAmount, RequestOutcome, TimeRange},
provider_credentials::{
CredentialCommitGuard, ProviderDocument, ProviderQuota, ProviderQuotaWindow,
QuotaLocalUsageAttribution,
},
settings::AdminApiKey,
use gateway_admin::model::{
PageSize, Revision,
accounts::AccountUsage,
auth::LoginCommand,
client_keys::ClientKeyPageSize,
observability::{DecimalAmount, RequestOutcome, TimeRange},
provider_credentials::{
ProviderDocument, ProviderQuota, ProviderQuotaWindow, QuotaLocalUsageAttribution,
},
ports::store::{AccountStore, AuthStore, ClientKeyStore, ObservabilityStore, SettingsStore},
settings::AdminApiKey,
};
mod backup;
@@ -99,20 +95,6 @@ fn login_command_debug_should_redact_password() {
assert!(!format!("{command:?}").contains("secret-password"));
}
#[test]
fn all_store_capability_traits_should_be_object_safe() {
fn assert_object_safe<T: ?Sized + Send + Sync>() {}
assert_object_safe::<dyn AccountStore>();
assert_object_safe::<dyn AuthStore>();
assert_object_safe::<dyn ClientKeyStore>();
assert_object_safe::<dyn ObservabilityStore>();
assert_object_safe::<dyn SettingsStore>();
fn assert_send_object_safe<T: ?Sized + Send>() {}
assert_send_object_safe::<dyn CredentialCommitGuard>();
}
#[test]
fn provider_document_debug_should_not_expose_opaque_material() {
let document = ProviderDocument::new(OpaqueProviderData::new(Default::default()));
@@ -1390,10 +1390,6 @@ fn total_record(
upstream_response_ms: None,
upstream_api_overhead_ms: None,
upstream_engine_ms: None,
upstream_engine_iapi_ttft_ms: None,
upstream_engine_service_ttft_ms: None,
upstream_engine_iapi_tbt_ms: None,
upstream_engine_service_tbt_ms: None,
latency_ms: Some(100),
admission_decision_ms: None,
account_selection_wait_ms: None,
@@ -21,7 +21,7 @@ fn counters_preserve_integer_precision_and_reject_json_numbers() {
}
}
#[tokio::test]
async fn usage_mutations_require_trusted_actor_and_old_workflow_operations_are_absent() {
async fn usage_mutations_require_a_trusted_actor() {
let admin = super::AdminHarness::new().build().await;
let registry = admin.public_services();
let denied = registry
@@ -38,21 +38,6 @@ async fn usage_mutations_require_trusted_actor_and_old_workflow_operations_are_a
.await
.is_err()
);
let ops = registry
.operations()
.filter(|op| op.starts_with("usage.") || *op == "pricing.evaluate")
.collect::<std::collections::BTreeSet<_>>();
assert_eq!(
ops,
std::collections::BTreeSet::from([
"usage.records.get",
"usage.records.list",
"usage.records.write",
"usage.aggregates.list",
"usage.aggregates.write",
"pricing.evaluate"
])
);
}
#[test]
fn sdk_wire_types_match_host_contracts() {
@@ -304,10 +304,6 @@ pub(crate) fn usage_list_record_view(
upstream_response_ms: record.upstream_response_ms,
upstream_api_overhead_ms: record.upstream_api_overhead_ms,
upstream_engine_ms: record.upstream_engine_ms,
upstream_engine_iapi_ttft_ms: record.upstream_engine_iapi_ttft_ms,
upstream_engine_service_ttft_ms: record.upstream_engine_service_ttft_ms,
upstream_engine_iapi_tbt_ms: record.upstream_engine_iapi_tbt_ms,
upstream_engine_service_tbt_ms: record.upstream_engine_service_tbt_ms,
},
first_token_latency_ms: record.first_token_ms,
latency_ms: record.latency_ms,
@@ -411,10 +407,6 @@ pub(crate) fn usage_record_view(
upstream_response_ms: record.upstream_response_ms,
upstream_api_overhead_ms: record.upstream_api_overhead_ms,
upstream_engine_ms: record.upstream_engine_ms,
upstream_engine_iapi_ttft_ms: record.upstream_engine_iapi_ttft_ms,
upstream_engine_service_ttft_ms: record.upstream_engine_service_ttft_ms,
upstream_engine_iapi_tbt_ms: record.upstream_engine_iapi_tbt_ms,
upstream_engine_service_tbt_ms: record.upstream_engine_service_tbt_ms,
},
attempt_index: None,
attempt_count: u64::from(record.attempt_count),
@@ -230,14 +230,6 @@ pub struct UsageLatencyDetailsView {
pub upstream_api_overhead_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub upstream_engine_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub upstream_engine_iapi_ttft_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub upstream_engine_service_ttft_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub upstream_engine_iapi_tbt_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub upstream_engine_service_tbt_ms: Option<f64>,
}
/// 单次上游尝试展示
@@ -213,14 +213,6 @@ struct OutputTimingView {
#[serde(skip_serializing_if = "Option::is_none")]
upstream_engine_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
upstream_engine_iapi_ttft_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
upstream_engine_service_ttft_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
upstream_engine_iapi_tbt_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
upstream_engine_service_tbt_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
first_event_ms: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
first_reasoning_ms: Option<u64>,
@@ -260,10 +252,6 @@ fn usage_record(value: UsageListRecord, time: crate::time::TimePresenter) -> Rec
upstream_response_ms: value.upstream_response_ms,
upstream_api_overhead_ms: value.upstream_api_overhead_ms,
upstream_engine_ms: value.upstream_engine_ms,
upstream_engine_iapi_ttft_ms: value.upstream_engine_iapi_ttft_ms,
upstream_engine_service_ttft_ms: value.upstream_engine_service_ttft_ms,
upstream_engine_iapi_tbt_ms: value.upstream_engine_iapi_tbt_ms,
upstream_engine_service_tbt_ms: value.upstream_engine_service_tbt_ms,
first_event_ms: value.first_event_ms,
first_reasoning_ms: value.first_reasoning_ms,
first_text_ms: value.first_text_ms,
@@ -58,7 +58,8 @@ fn decode_response_create_inner(
else {
return Err(ResponseCreateFrameError::ExpectedObject);
};
match body.remove("type") {
// 保持剩余字段顺序,避免末尾元数据交换到模型等路由字段之前
match body.shift_remove("type") {
Some(Value::String(message_type)) if message_type == "response.create" => {}
_ => return Err(ResponseCreateFrameError::UnsupportedType),
}
@@ -46,7 +46,6 @@ async fn list_route_should_keep_camel_case_group_and_page_wire() {
value["data"]["items"][0]["usage"]["retainedTotalUsd"],
"5.5"
);
assert!(value["data"]["items"][0].get("member_count").is_none());
}
#[tokio::test]
@@ -149,7 +149,7 @@ fn account_page_item(
}
#[tokio::test]
async fn account_usage_returns_distinct_cache_read_and_write_tokens_without_aliases() {
async fn account_usage_preserves_cache_read_and_write_counts_and_missing_values() {
use chrono::Utc;
use gateway_admin::model::accounts::AccountUsage;
use gateway_core::account::{AccountStatusFacts, CredentialState, QuotaState};
@@ -215,37 +215,9 @@ async fn account_usage_returns_distinct_cache_read_and_write_tokens_without_alia
usage["cacheWriteInputTokensDisplay"],
if write.is_some() { "7" } else { "—" }
);
for alias in [
"createdTokens",
"createdTokensDisplay",
"readTokens",
"readTokensDisplay",
] {
assert!(usage.get(alias).is_none(), "obsolete token alias: {alias}");
}
}
}
#[tokio::test]
async fn standalone_credential_rotation_route_is_not_exposed() {
let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session");
let response = admin::router::<AdminTestState>()
.with_state(fixture.state())
.oneshot(
Request::builder()
.method("POST")
.uri("/api/admin/accounts/rotate")
.header(header::COOKIE, "cpr_session=valid-session")
.header(header::CONTENT_TYPE, "application/json")
.body(Body::from("{}"))
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::NOT_FOUND);
}
#[tokio::test]
async fn connection_update_requires_admin_and_validates_before_calling_the_service() {
let fixture = AdminTestFixture::new().await;
@@ -248,9 +248,8 @@ mod profile_statistics {
value["activityInsights"]["invocations"][0]["usageCount"],
29
);
assert!(value.get("cycle").is_none());
assert!(value.get("models").is_none());
assert!(value.get("estimatedCost").is_none());
assert_eq!(value["activityInsights"]["fastModePercent"], 0.0);
assert!(value["activityInsights"]["invocations"][0]["skillId"].is_null());
}
#[test]
@@ -375,7 +374,7 @@ mod batch_update {
assert!(request.concurrency_limit.is_none());
assert!(
serde_json::from_value::<BatchUpdateAccountsRequest>(json!({
"accountIds": ["acct_test"], "enabled": true, "legacy": true
"accountIds": ["acct_test"], "enabled": true, "unknownField": true
}))
.is_err()
);
@@ -583,12 +582,12 @@ mod actions {
}
#[test]
fn account_actions_should_require_frozen_account_ids_and_reject_unknown_revision() {
fn account_actions_should_validate_account_ids_and_reject_unknown_fields() {
let id: AccountIdQuery =
serde_json::from_value(json!({ "accountId": "acct_1" })).expect("decode ID query");
assert!(id.validate().is_ok());
let action: AccountActionRequest =
serde_json::from_value(json!({ "accountId": "legacy-id" })).expect("decode action");
serde_json::from_value(json!({ "accountId": "invalid-id" })).expect("decode action");
assert_eq!(action.validate().unwrap_err().field(), "accountId");
let refresh: AccountRefreshRequest = serde_json::from_value(json!({
"accountId": "acct_1"
@@ -598,7 +597,7 @@ mod actions {
assert!(
serde_json::from_value::<AccountRefreshRequest>(json!({
"accountId": "acct_1",
"expectedConfigRevision": 0
"unknownField": true
}))
.is_err()
);
@@ -772,15 +771,14 @@ mod actions {
assert_eq!(invalid.validate().unwrap_err().field(), "data");
assert!(
serde_json::from_value::<AccountImportRequest>(json!({
"provider": "openai",
"document": {}
"provider": "openai"
}))
.is_err()
);
assert!(
serde_json::from_value::<AccountImportRequest>(json!({
"provider": "openai",
"expectedConfigRevision": 7,
"unknownField": true,
"data": {}
}))
.is_err()
@@ -11,7 +11,7 @@ use gateway_api::admin::accounts::{
use gateway_core::operation::RawJsonPayload;
#[test]
fn quota_forecast_projection_only_exposes_capacity_and_preserves_null_zero() {
fn quota_forecast_projection_formats_capacity_and_preserves_null_zero() {
let now = "2026-09-12T00:00:00Z".parse().unwrap();
let forecast = AccountQuotaForecast {
period: AccountUsagePeriod::Weekly,
@@ -54,16 +54,6 @@ fn quota_forecast_projection_only_exposes_capacity_and_preserves_null_zero() {
assert_eq!(week["estimatedTokensDisplay"], "1M");
assert!(week["estimatedUsd"].is_null());
assert_eq!(week["estimatedUsdDisplay"], "—");
for forecast in value["forecasts"].as_array().unwrap() {
for field in [
"remainingTokens",
"remainingTokensDisplay",
"remainingUsd",
"remainingUsdDisplay",
] {
assert!(forecast.get(field).is_none(), "unexpected field: {field}");
}
}
assert_eq!(
week["source"],
serde_json::json!({
@@ -77,9 +67,6 @@ fn quota_forecast_projection_only_exposes_capacity_and_preserves_null_zero() {
"usdDisplay": "$0.1234"
})
);
assert!(week.get("method").is_none());
assert!(week.get("methodDisplay").is_none());
assert!(value.get("generatedAtDisplay").is_none());
assert_eq!(value["forecasts"][1]["estimatedTokens"], 0);
assert_eq!(value["forecasts"][1]["estimatedTokensDisplay"], "0");
assert_eq!(value["forecasts"][1]["estimatedUsd"], 0.0);
@@ -87,7 +74,6 @@ fn quota_forecast_projection_only_exposes_capacity_and_preserves_null_zero() {
assert_eq!(value["forecasts"][1]["period"], "monthly");
assert_eq!(value["forecasts"][1]["targetDays"], 30.0);
assert_eq!(value["forecasts"][1]["extrapolated"], true);
assert!(value.get("account").is_none());
}
#[test]
@@ -375,18 +375,17 @@ fn client_key_mutations_should_validate_text_limits_and_unknown_fields() {
assert!(
serde_json::from_value::<UpdateClientKeyRequest>(json!({
"id": "key_1",
"expectedConfigRevision": 1,
"name": "key",
"groupIds": [],
"maxConcurrency": 0,
"requestsPerMinute": 0,
"tokensPerMinute": 0
"unknownField": true
}))
.is_err()
);
assert!(
serde_json::from_value::<CreateClientKeyRequest>(json!({
"expectedConfigRevision": 7,
"unknownField": true,
"name": "terminal key",
"groupIds": [],
"maxConcurrency": 2,
@@ -458,7 +457,6 @@ fn client_key_responses_should_keep_shape_and_redact_creation_debug() {
));
let list =
serde_json::to_value(ClientKeyListData::new(vec![view], None, 1)).expect("serialize list");
assert!(list.get("configRevision").is_none());
assert_eq!(list["total"], 1);
assert_eq!(list["items"][0]["id"], "key_visible");
assert_eq!(
@@ -468,12 +466,9 @@ fn client_key_responses_should_keep_shape_and_redact_creation_debug() {
assert_eq!(list["items"][0]["routingScope"], "all");
assert_eq!(list["items"][0]["maxConcurrency"], 2);
assert_eq!(list["items"][0]["requestsPerMinute"], 60);
assert!(list["items"][0].get("tokensPerMinute").is_none());
assert!(list["items"][0].get("policyJson").is_none());
assert_eq!(list["items"][0]["dailyLimitUsd"], "0");
assert_eq!(list["items"][0]["weeklyLimitUsd"], "0");
assert_eq!(list["items"][0]["dailyUsedUsd"], "0");
assert!(list["items"][0].get("unresolvedRequests").is_none());
assert_eq!(
DateTime::parse_from_rfc3339(
list["items"][0]["lastUsedAt"]
@@ -7,7 +7,7 @@ use gateway_api::admin::observability::{
use serde_json::json;
#[test]
fn dashboard_query_should_parse_terminal_trend_kinds() {
fn dashboard_query_should_parse_error_trends() {
let query: DashboardQuery = serde_json::from_value(json!({"kind": "errors"})).unwrap();
assert_eq!(query.trend_kind().unwrap(), TrendKind::Errors);
}
@@ -19,7 +19,7 @@ fn dashboard_query_should_reject_unknown_trend_kind() {
}
#[test]
fn usage_query_should_accept_element_plus_pagination() {
fn usage_query_should_accept_page_number_and_size() {
let query: UsageQuery = serde_json::from_value(json!({
"currentPage": 129,
"pageSize": 10
@@ -29,32 +29,20 @@ fn usage_query_should_accept_element_plus_pagination() {
}
#[test]
fn usage_query_should_reject_removed_total_toggle() {
assert!(serde_json::from_value::<UsageQuery>(json!({"includeTotal": true})).is_err());
fn observability_queries_should_reject_unknown_fields() {
let query = json!({"unknownFilter": true});
assert!(serde_json::from_value::<DashboardQuery>(query.clone()).is_err());
assert!(serde_json::from_value::<UsageQuery>(query.clone()).is_err());
assert!(serde_json::from_value::<OpsQuery>(query.clone()).is_err());
assert!(serde_json::from_value::<DiagnosticsQuery>(query).is_err());
}
#[test]
fn usage_query_should_reject_removed_cursor_contract() {
assert!(serde_json::from_value::<UsageQuery>(json!({"cursor": "opaque"})).is_err());
}
#[test]
fn usage_query_should_reject_nonstandard_page_name() {
assert!(serde_json::from_value::<UsageQuery>(json!({"page": 1})).is_err());
}
#[test]
fn ops_query_should_reject_page_size_above_terminal_limit() {
fn ops_query_should_reject_page_size_above_maximum() {
let query: OpsQuery = serde_json::from_value(json!({"pageSize": 101})).unwrap();
assert_eq!(query.validate_pagination().unwrap_err().field(), "pageSize");
}
#[test]
fn ops_query_should_reject_removed_filters() {
assert!(serde_json::from_value::<OpsQuery>(json!({"route": "/v1/responses"})).is_err());
assert!(serde_json::from_value::<OpsQuery>(json!({"failureClass": "rate_limited"})).is_err());
}
#[test]
fn diagnostics_query_should_keep_wire_dimension_name() {
let query: DiagnosticsQuery =
@@ -1,4 +1,4 @@
//! 响应 DTO 与固定 wire 形状的合同测试
//! 验证观测响应的指标、展示投影与查询时间边界
use chrono::Utc;
use serde_json::json;
@@ -288,10 +288,6 @@ async fn usage_detail_should_keep_attempt_snapshot_contract() {
record.upstream_response_ms = Some(1_000);
record.upstream_api_overhead_ms = Some(120.25);
record.upstream_engine_ms = Some(6400.0);
record.upstream_engine_iapi_ttft_ms = Some(650.5);
record.upstream_engine_service_ttft_ms = Some(720.25);
record.upstream_engine_iapi_tbt_ms = Some(18.45);
record.upstream_engine_service_tbt_ms = Some(20.12);
fixture
.usage_detail
.lock()
@@ -362,29 +358,6 @@ async fn usage_detail_should_keep_attempt_snapshot_contract() {
value["data"]["latencyDetails"]["upstreamEngineMs"],
json!(6400.0)
);
assert_eq!(
value["data"]["latencyDetails"]["upstreamEngineIapiTtftMs"],
json!(650.5)
);
assert_eq!(
value["data"]["latencyDetails"]["upstreamEngineServiceTtftMs"],
json!(720.25)
);
assert_eq!(
value["data"]["latencyDetails"]["upstreamEngineIapiTbtMs"],
json!(18.45)
);
assert_eq!(
value["data"]["latencyDetails"]["upstreamEngineServiceTbtMs"],
json!(20.12)
);
assert!(value["data"].get("firstTokenMs").is_none());
assert!(value["data"].get("firstTokenLatencyMsDisplay").is_none());
assert!(
value["data"]["latencyDetails"]
.get("firstTokenMs")
.is_none()
);
assert_eq!(
serde_json::json!({
"accountId": value["data"]["accountId"],
@@ -490,8 +463,6 @@ async fn zero_attempt_failure_detail_omits_trace_and_keeps_missing_upstream_fact
assert_eq!(data["clientStatusCode"], 503);
assert_eq!(data["message"], "no upstream account is available");
assert_eq!(data["model"], "coding");
assert!(data.get("upstreamStatusCode").is_none());
assert!(data.get("httpVersion").is_none());
for field in [
"provider",
"accountId",
@@ -642,7 +613,6 @@ async fn diagnostics_should_keep_stable_key_and_display_name_contract() {
.await
.expect("diagnostics body");
let value: serde_json::Value = serde_json::from_slice(&body).expect("diagnostics JSON");
assert!(value["data"]["items"][0].get("impactScore").is_none());
assert_eq!(value["data"]["items"][0]["requestShare"], 0.5);
assert_eq!(value["data"]["items"][0]["retryCount"], 3);
assert_eq!(value["data"]["items"][0]["retryRate"], 0.5);
@@ -734,10 +704,6 @@ pub(super) fn usage_record_with_account(
upstream_response_ms: None,
upstream_api_overhead_ms: None,
upstream_engine_ms: None,
upstream_engine_iapi_ttft_ms: None,
upstream_engine_service_ttft_ms: None,
upstream_engine_iapi_tbt_ms: None,
upstream_engine_service_tbt_ms: None,
latency_ms: None,
admission_decision_ms: None,
account_selection_wait_ms: None,
@@ -877,10 +843,6 @@ async fn usage_route_should_expose_table_facts_without_detail_payload() {
upstream_response_ms: Some(2_000),
upstream_api_overhead_ms: None,
upstream_engine_ms: None,
upstream_engine_iapi_ttft_ms: None,
upstream_engine_service_ttft_ms: None,
upstream_engine_iapi_tbt_ms: None,
upstream_engine_service_tbt_ms: None,
latency_ms: Some(31),
admission_decision_ms: Some(1),
account_selection_wait_ms: Some(2),
@@ -35,7 +35,7 @@ use std::{collections::BTreeMap, sync::Arc};
mod management;
#[test]
fn artifact_metadata_wire_exposes_v2_identity_and_contributions() {
fn artifact_metadata_wire_preserves_identity_icons_and_contributions() {
use gateway_admin::model::plugins::{
PluginArtifactIcon, PluginArtifactIconVariants, PluginArtifactMetadata, PluginContribution,
};
@@ -70,7 +70,7 @@ fn artifact_metadata_wire_exposes_v2_identity_and_contributions() {
secret_fields: vec![],
state_namespaces: vec![],
};
let value = serde_json::to_value(metadata).expect("v2 artifact metadata JSON");
let value = serde_json::to_value(metadata).expect("artifact metadata JSON");
assert_eq!(value["displayName"], "Example");
assert_eq!(value["publisher"], "test");
assert!(value["author"].is_null());
@@ -80,33 +80,29 @@ fn artifact_metadata_wire_exposes_v2_identity_and_contributions() {
value["contributes"]["middleware"]["id"],
"test.example.middleware"
);
assert!(value.get("capabilities").is_none());
assert!(value.get("permissionDescriptions").is_none());
}
#[test]
fn instance_binding_wire_uses_contribution_without_capability_alias() {
fn instance_binding_requires_a_contribution_and_rejects_unknown_fields() {
use gateway_admin::model::plugins::instances::PluginCapabilityBinding;
let binding = serde_json::from_value::<PluginCapabilityBinding>(serde_json::json!({
let input = serde_json::json!({
"contribution": "test.example.usage",
"stage": "observation",
"order": 0,
"failurePolicy": "observe"
}))
.expect("v2 contribution binding");
});
let binding = serde_json::from_value::<PluginCapabilityBinding>(input.clone())
.expect("contribution binding");
assert_eq!(binding.contribution, "test.example.usage");
assert!(
serde_json::from_value::<PluginCapabilityBinding>(serde_json::json!({
"capability": "usage",
"stage": "observation",
"order": 0,
"failurePolicy": "observe"
}))
.is_err(),
"legacy capability binding must be rejected"
);
let mut missing = input.clone();
missing.as_object_mut().unwrap().remove("contribution");
assert!(serde_json::from_value::<PluginCapabilityBinding>(missing).is_err());
let mut unknown = input;
unknown["unknownField"] = serde_json::json!(true);
assert!(serde_json::from_value::<PluginCapabilityBinding>(unknown).is_err());
}
#[tokio::test]
@@ -381,7 +377,7 @@ async fn plugin_upload_endpoints_require_admin_and_bound_raw_bodies() {
}
#[tokio::test]
async fn artifact_acceptance_returns_the_flat_install_contract() {
async fn artifact_acceptance_returns_installed_artifact_and_configuration_status() {
use axum::{
body::{Body, to_bytes},
http::{Request, StatusCode},
@@ -418,22 +414,11 @@ async fn artifact_acceptance_returns_the_flat_install_contract() {
assert_eq!(result["defaultInstanceId"], serde_json::Value::Null);
assert_eq!(result["configurationRequired"], false);
assert_eq!(result["artifact"]["metadata"]["sha256"], "a".repeat(64));
assert!(
result["artifact"]["metadata"]
.get("permissionDescriptions")
.is_none()
);
assert!(
result["artifact"]["metadata"]
.get("requestedPermissions")
.is_none()
);
assert!(result["artifact"]["acceptedAt"].is_string());
assert!(result.get("mutation").is_none());
}
#[tokio::test]
async fn removed_instance_policy_fields_and_extra_accept_fields_are_rejected() {
async fn plugin_mutations_reject_unknown_fields() {
use axum::{
body::Body,
http::{Request, StatusCode},
@@ -442,24 +427,25 @@ async fn removed_instance_policy_fields_and_extra_accept_fields_are_rejected() {
let fixture = super::AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session");
for (path, body) in [
(
"/api/admin/plugins/instances",
serde_json::json!({
"name":"Example",
"artifactSha256":"a".repeat(64),
"enabled":false,
"trustedProcess":true,
"configuration":{},
"grants":[],
"bindings":[]
}),
),
let instance = serde_json::json!({
"name":"Example",
"artifactSha256":"a".repeat(64),
"enabled":false,
"configuration":{},
"bindings":[]
});
serde_json::from_value::<gateway_admin::model::plugins::instances::ConfigurePluginInstance>(
instance.clone(),
)
.expect("valid instance request");
for (path, mut body) in [
("/api/admin/plugins/instances", instance),
(
"/api/admin/plugins/artifacts/accept",
serde_json::json!({"sha256":"a".repeat(64),"trusted":true}),
serde_json::json!({"sha256":"a".repeat(64)}),
),
] {
body["unknownField"] = serde_json::json!(true);
let response = gateway_api::admin::router::<super::AdminTestState>()
.with_state(fixture.state())
.oneshot(
@@ -181,7 +181,7 @@ fn privacy_policy_is_required_and_rejects_unknown_contract_fields() {
.remove("codexPrivacyPolicy");
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(missing).is_err());
let mut unknown = update_body();
unknown["codexPrivacyPolicy"]["protectedFields"] = json!([]);
unknown["codexPrivacyPolicy"]["unknownField"] = json!(true);
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(unknown).is_err());
}
@@ -471,110 +471,6 @@ fn settings_response_should_cover_the_full_runtime_settings_contract() {
);
}
#[test]
fn settings_request_and_response_fields_should_stay_in_lockstep() {
use std::collections::{BTreeMap, BTreeSet};
use gateway_admin::model::Revision;
use gateway_admin::model::settings::RuntimeSettings;
use gateway_api::admin::settings::RuntimeSettingsView;
use gateway_core::account::RotationStrategy;
use gateway_core::routing::{PublicModelId, UpstreamModelId};
let request: UpdateRuntimeSettingsRequest =
serde_json::from_value(update_body()).expect("decode settings");
request.validate().expect("fixture settings must validate");
let request_fields: BTreeSet<String> = update_body()
.as_object()
.expect("request body object")
.keys()
.cloned()
.collect();
let settings = RuntimeSettings {
request_profiles: Default::default(),
config_revision: Revision::new(7).expect("revision"),
model_mappings: request
.model_mappings
.iter()
.map(|(public, upstream)| {
Ok((
PublicModelId::new(public.clone())?,
UpstreamModelId::new(upstream.clone())?,
))
})
.collect::<Result<BTreeMap<_, _>, gateway_core::error::IdentifierError>>()
.expect("valid model mappings"),
rotation_strategy: RotationStrategy::parse(&request.rotation_strategy)
.expect("fixture rotation strategy"),
updated_at: chrono::Utc::now(),
values: gateway_admin::model::settings::RuntimeSettingsValues {
codex_privacy_policy: Default::default(),
request_location_enabled: false,
request_location: Default::default(),
refresh_margin_seconds: request.values.refresh_margin_seconds,
refresh_concurrency: u32::try_from(request.values.refresh_concurrency).expect("u32"),
max_concurrent_per_account: u32::try_from(request.values.max_concurrent_per_account)
.expect("u32"),
max_waiting_per_key: 0,
max_waiting_per_account: 0,
concurrency_wait_timeout_seconds: 30,
openai_guardian_reserved_concurrency: 0,
openai_account_affinity: gateway_core::account::AccountAffinity::Relaxed,
max_account_rotations: 3,
openai_session_affinity_ttl_hours: 24,
responses_max_decompressed_body_bytes: 64 * 1024 * 1024,
smart_scheduling: request.values.smart_scheduling,
min_codex_desktop_version: request.values.min_codex_desktop_version,
min_codex_cli_version: request.values.min_codex_cli_version,
usage_retention_days: u32::try_from(request.values.usage_retention_days).expect("u32"),
ops_event_retention_days: u32::try_from(request.values.ops_event_retention_days)
.expect("u32"),
audit_retention_days: u32::try_from(request.values.audit_retention_days).expect("u32"),
account_auto_freeze_enabled: true,
account_auto_freeze_threshold: 12,
account_auto_freeze_window_seconds: 600,
account_auto_freeze_duration_seconds: 7_200,
account_auto_freeze_probe_enabled: true,
account_auto_freeze_probe_model: None,
account_auto_freeze_adaptive_concurrency: true,
account_warmup_enabled: false,
account_warmup_schedule_time: "08:00".to_owned(),
account_warmup_model: None,
reset_credit_lead_minutes: 10,
reset_credit_max_weekly_remaining_percent: None,
},
};
let response_fields: BTreeSet<String> = serde_json::to_value(RuntimeSettingsView::from((
settings,
gateway_api::TimePresenter::new(Default::default()),
)))
.expect("serialize view")
.as_object()
.expect("view object")
.keys()
.cloned()
.collect();
let mut expected_fields = request_fields;
expected_fields.insert("providerRequestProfiles".to_owned());
expected_fields.insert("openaiClientProfile".to_owned());
expected_fields.insert("xaiClientProfile".to_owned());
expected_fields.insert("updatedAt".to_owned());
expected_fields.insert("updatedAtDisplay".to_owned());
expected_fields.insert("smartSchedulingDefaults".to_owned());
assert_eq!(response_fields, expected_fields);
}
#[test]
fn settings_request_accepts_json_bytes_with_flattened_fields() {
serde_json::from_slice::<UpdateRuntimeSettingsRequest>(
&serde_json::to_vec(&update_body()).unwrap(),
)
.expect("settings JSON bytes must decode like the HTTP request body");
}
#[test]
fn settings_service_contract_stays_flat_and_rejects_unknown_fields() {
use gateway_admin::model::settings::{ReplaceRuntimeSettings, RuntimeSettings};
@@ -610,17 +506,9 @@ fn settings_service_contract_stays_flat_and_rejects_unknown_fields() {
}
#[test]
fn settings_request_should_reject_unknown_revision_field() {
fn settings_request_should_reject_unknown_fields() {
let mut body = update_body();
body["expectedConfigRevision"] = json!(7);
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(body).is_err());
}
#[test]
fn settings_request_should_reject_removed_bucket_retention() {
let mut body = update_body();
body["bucketRetentionDays"] = json!(365);
body["unknownSetting"] = json!(true);
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(body).is_err());
}
@@ -960,36 +848,23 @@ fn decompression_setting_should_reject_invalid_values() {
}
#[tokio::test]
async fn settings_reject_removed_global_fast_policy() {
async fn settings_update_rejects_unknown_fields_without_writing() {
let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session");
let app = app(fixture.state());
let initial = fixture.settings.settings.lock().unwrap().clone();
for value in [json!(true), json!(false), Value::Null] {
let mut body = update_body();
body["disableFast"] = value;
let response = app
.clone()
.oneshot(request(
Method::POST,
"/api/admin/settings/update",
Some(body),
))
.await
.unwrap();
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
assert_eq!(*fixture.settings.settings.lock().unwrap(), initial);
}
let mut body = update_body();
body["unknownSetting"] = json!(true);
let response = app
.oneshot(request(Method::GET, "/api/admin/settings", None))
.oneshot(request(
Method::POST,
"/api/admin/settings/update",
Some(body),
))
.await
.unwrap();
assert_eq!(response.status(), StatusCode::OK);
assert!(
response_json(response).await["data"]
.get("disableFast")
.is_none()
);
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
assert_eq!(*fixture.settings.settings.lock().unwrap(), initial);
}
#[test]
@@ -86,7 +86,7 @@ async fn update_event_stream_should_preserve_event_id_and_download_progress_perc
}
#[tokio::test]
async fn update_should_return_accepted_and_report_restart_only_through_status() {
async fn update_should_return_accepted_and_expose_restart_status() {
let fixture = AdminTestFixture::with_system(Arc::new(ProgressSystem::default())).await;
fixture.auth.insert_session("valid-session");
let router = app(fixture.state());
@@ -110,7 +110,6 @@ async fn update_should_return_accepted_and_report_restart_only_through_status()
.expect("JSON");
assert_eq!(body["data"]["operationId"], "update-1");
assert_eq!(body["data"]["targetVersion"], "0.2.0");
assert!(body["data"].get("needRestart").is_none());
let response = router
.oneshot(
Request::builder()
+22 -22
View File
@@ -297,15 +297,15 @@ async fn unified_login_returns_server_identity_and_rotates_the_previous_session(
}
#[tokio::test]
async fn login_rejects_legacy_type_without_replacing_the_current_session() {
async fn login_rejects_missing_mode_and_unknown_fields_without_replacing_the_session() {
let (app, _) = auth_app().await;
let cookie =
session_cookie(&login(&app, json!({"mode": "key", "apiKey": RAW_KEY}), None).await);
for body in [
json!({"type": "key", "apiKey": RAW_KEY}),
json!({"type": "admin", "password": "strong-admin-password"}),
json!({"mode": "key", "type": "key", "apiKey": RAW_KEY}),
json!({"mode": "admin", "type": "admin", "password": "strong-admin-password"}),
json!({"apiKey": RAW_KEY}),
json!({"password": "strong-admin-password"}),
json!({"mode": "key", "unknownField": true, "apiKey": RAW_KEY}),
json!({"mode": "admin", "unknownField": true, "password": "strong-admin-password"}),
] {
let response = login(&app, body, Some(&cookie)).await;
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
@@ -458,26 +458,26 @@ async fn key_store_outage_is_unavailable_not_expired_or_authenticated() {
}
#[tokio::test]
async fn removed_auth_routes_do_not_fall_back_to_spa_or_accept_old_cookies() {
async fn unknown_auth_routes_return_json_not_found() {
let (app, _) = auth_app().await;
let response = app
.oneshot(empty_request(Method::GET, "/api/auth/unknown"))
.await
.unwrap();
assert_eq!(response.status(), StatusCode::NOT_FOUND);
assert_eq!(response_json(response).await["code"], 40401);
}
#[tokio::test]
async fn session_authentication_requires_the_configured_cookie_name() {
let (app, _) = auth_app().await;
for path in ["/api/admin/auth/login", "/api/auth/unknown"] {
let response = app
.clone()
.oneshot(empty_request(Method::GET, path))
.await
.unwrap();
assert_eq!(response.status(), StatusCode::NOT_FOUND);
assert_eq!(response_json(response).await["code"], 40401);
}
let cookie =
session_cookie(&login(&app, json!({"mode": "key", "apiKey": RAW_KEY}), None).await);
for old_name in ["cpr_admin_session", "cpr_client_session"] {
let old_cookie = cookie.replace("cpr_session", old_name);
assert_eq!(
response_json(get(&app, "/api/auth/status", &old_cookie).await).await["data"]["authenticated"],
false
);
}
let unrelated_cookie = cookie.replace("cpr_session", "unrelated_session");
assert_eq!(
response_json(get(&app, "/api/auth/status", &unrelated_cookie).await).await["data"]["authenticated"],
false
);
}
#[test]
@@ -140,11 +140,7 @@ fn usage_record() -> UsageListRecord {
provider_processing_ms: None,
upstream_response_ms: Some(1_000),
upstream_api_overhead_ms: None,
upstream_engine_ms: None,
upstream_engine_iapi_ttft_ms: None,
upstream_engine_service_ttft_ms: None,
upstream_engine_iapi_tbt_ms: Some(2.450638),
upstream_engine_service_tbt_ms: None,
upstream_engine_ms: Some(2.450638),
latency_ms: Some(1250),
admission_decision_ms: None,
account_selection_wait_ms: None,
@@ -320,7 +320,7 @@ async fn records_keep_pagination_and_hide_admin_and_upstream_data() {
&record["latencyDetails"],
&[
"upstreamResponseMs",
"upstreamEngineIapiTbtMs",
"upstreamEngineMs",
"firstEventMs",
"firstReasoningMs",
"firstTextMs",
@@ -328,10 +328,7 @@ async fn records_keep_pagination_and_hide_admin_and_upstream_data() {
);
assert_eq!(record["latencyDetails"]["firstEventMs"], 100);
assert_eq!(record["latencyDetails"]["upstreamResponseMs"], 1000);
assert_eq!(
record["latencyDetails"]["upstreamEngineIapiTbtMs"],
2.450638
);
assert_eq!(record["latencyDetails"]["upstreamEngineMs"], 2.450638);
assert!(record["statusCode"].is_null());
} else {
assert_eq!(record["statusCode"], 502);
@@ -799,7 +799,7 @@ async fn websocket_failure_before_first_event_should_return_observed_metadata_th
}
#[tokio::test]
async fn websocket_response_create_should_not_have_a_private_16_mib_frame_limit() {
async fn large_websocket_response_create_frames_should_reach_execution() {
let trace = Arc::new(AtomicFailureTrace::default());
let execution = Arc::new(AtomicFailureExecution {
client: authenticated_client("sk_ws_atomic"),
@@ -829,15 +829,14 @@ async fn websocket_response_create_should_not_have_a_private_16_mib_frame_limit(
let payload = json!({
"type": "response.create",
"model": "model-a",
"input": "x".repeat(16 * 1024 * 1024 + 1),
"input": "x".repeat(17 * 1024 * 1024),
})
.to_string();
assert!(payload.len() > 16 * 1024 * 1024);
socket
.send(ClientMessage::Text(payload.into()))
.await
.expect("send response.create above the former frame limit");
.expect("send large response.create frame");
let mut saw_upstream_event = false;
for _ in 0..3 {
@@ -8,6 +8,26 @@ use serde_json::json;
use super::decode_response_create;
#[test]
fn response_create_should_preserve_field_order_when_removing_frame_type() {
let expected = r#"{"model":"smart-code","stream":true,"input":"hello","future_field":{"z":1,"a":2},"client_metadata":{"thread_id":"thread","turn_id":"turn"}}"#;
for payload in [
r#"{"type":"response.create","model":"smart-code","stream":true,"input":"hello","future_field":{"z":1,"a":2},"client_metadata":{"thread_id":"thread","turn_id":"turn"}}"#,
r#"{"model":"smart-code","stream":true,"type":"response.create","input":"hello","future_field":{"z":1,"a":2},"client_metadata":{"thread_id":"thread","turn_id":"turn"}}"#,
r#"{"model":"smart-code","stream":true,"input":"hello","future_field":{"z":1,"a":2},"client_metadata":{"thread_id":"thread","turn_id":"turn"},"type":"response.create"}"#,
] {
let decoded = decode_response_create(payload).expect("decode response.create");
let body = decoded
.operation()
.middleware_body()
.expect("encode transparent body");
assert_eq!(
std::str::from_utf8(&body).expect("UTF-8 JSON body"),
expected,
);
}
}
#[test]
fn client_retry_identity_should_follow_each_frame_instead_of_handshake_metadata() {
let mut headers = HeaderMap::new();
@@ -2,14 +2,14 @@
use axum::{
body::Body,
http::{Method, Request, StatusCode, header::AUTHORIZATION},
http::{Method, Request, StatusCode},
};
use tower::ServiceExt;
use super::api_router_with_origins;
use super::models::ModelsExecution;
const REMOVED_BODY_LIMIT_BYTES: usize = 16 * 1024 * 1024;
const LARGE_BODY_BYTES: usize = 17 * 1024 * 1024;
#[tokio::test]
async fn browser_origin_controls_http_admin_sessions_without_configuration() {
@@ -117,64 +117,16 @@ async fn browser_origin_controls_http_admin_sessions_without_configuration() {
}
#[tokio::test]
async fn removed_responses_review_route_should_not_reach_the_responses_handler() {
let response = api_router_with_origins(ModelsExecution::new(), Vec::new())
.await
.oneshot(
Request::post("/v1/responses/review")
.header(AUTHORIZATION, "Bearer sk_models_test")
.body(Body::empty())
.expect("build removed review request"),
)
.await
.expect("route removed review request");
assert_eq!(response.status(), StatusCode::METHOD_NOT_ALLOWED);
}
#[tokio::test]
async fn removed_models_catalog_extension_should_use_standard_model_lookup() {
let response = api_router_with_origins(ModelsExecution::new(), Vec::new())
.await
.oneshot(
Request::get("/v1/models/catalog")
.header(AUTHORIZATION, "Bearer sk_models_test")
.body(Body::empty())
.expect("build removed model catalog request"),
)
.await
.expect("route removed model catalog request");
assert_eq!(response.status(), StatusCode::NOT_FOUND);
}
#[tokio::test]
async fn removed_model_info_route_should_return_not_found() {
let response = api_router_with_origins(ModelsExecution::new(), Vec::new())
.await
.oneshot(
Request::get("/v1/models/model-a/info")
.header(AUTHORIZATION, "Bearer sk_models_test")
.body(Body::empty())
.expect("build removed model info request"),
)
.await
.expect("route removed model info request");
assert_eq!(response.status(), StatusCode::NOT_FOUND);
}
#[tokio::test]
async fn responses_body_should_accept_payload_above_the_removed_private_limit() {
async fn large_responses_bodies_should_reach_authentication() {
let router = api_router_with_origins(ModelsExecution::new(), Vec::new()).await;
let response = router
.oneshot(
Request::post("/v1/responses")
.body(Body::from(vec![b'a'; REMOVED_BODY_LIMIT_BYTES + 1]))
.expect("build request above the removed limit"),
.body(Body::from(vec![b'a'; LARGE_BODY_BYTES]))
.expect("build large request"),
)
.await
.expect("route request above the removed limit");
.expect("route large request");
// 请求进入处理器后应因缺少凭据而被拒绝
// 若提前施加正文大小限制,则会在认证前返回 413
@@ -955,10 +955,6 @@ pub struct ModelRequestTimings {
pub upstream_response_ms: Option<u64>,
pub upstream_api_overhead_ms: Option<f64>,
pub upstream_engine_ms: Option<f64>,
pub upstream_engine_iapi_ttft_ms: Option<f64>,
pub upstream_engine_service_ttft_ms: Option<f64>,
pub upstream_engine_iapi_tbt_ms: Option<f64>,
pub upstream_engine_service_tbt_ms: Option<f64>,
pub latency_ms: Option<u64>,
}
@@ -807,10 +807,6 @@ impl ResponseObservation {
self.timings.upstream_response_ms = None;
self.timings.upstream_api_overhead_ms = None;
self.timings.upstream_engine_ms = None;
self.timings.upstream_engine_iapi_ttft_ms = None;
self.timings.upstream_engine_service_ttft_ms = None;
self.timings.upstream_engine_iapi_tbt_ms = None;
self.timings.upstream_engine_service_tbt_ms = None;
}
pub(super) fn observe_response(&mut self, observation: &ProviderResponseObservation) {
@@ -848,17 +844,5 @@ impl ResponseObservation {
if let Some(value) = observed.upstream_engine_ms {
self.timings.upstream_engine_ms = Some(value);
}
if let Some(value) = observed.upstream_engine_iapi_ttft_ms {
self.timings.upstream_engine_iapi_ttft_ms = Some(value);
}
if let Some(value) = observed.upstream_engine_service_ttft_ms {
self.timings.upstream_engine_service_ttft_ms = Some(value);
}
if let Some(value) = observed.upstream_engine_iapi_tbt_ms {
self.timings.upstream_engine_iapi_tbt_ms = Some(value);
}
if let Some(value) = observed.upstream_engine_service_tbt_ms {
self.timings.upstream_engine_service_tbt_ms = Some(value);
}
}
}
+1 -5
View File
@@ -536,12 +536,8 @@ pub struct ProviderResponseTimings {
pub upstream_response_ms: Option<u64>,
/// 上游排除引擎与客户端工具时间后的 API 耗时
pub upstream_api_overhead_ms: Option<f64>,
/// 上游响应级引擎耗时;以下专项指标保留毫秒小数及各自内部计量层级
/// 上游响应级引擎耗时,保留毫秒小数
pub upstream_engine_ms: Option<f64>,
pub upstream_engine_iapi_ttft_ms: Option<f64>,
pub upstream_engine_service_ttft_ms: Option<f64>,
pub upstream_engine_iapi_tbt_ms: Option<f64>,
pub upstream_engine_service_tbt_ms: Option<f64>,
}
/// Provider 已筛选的响应观测 JSON
@@ -505,10 +505,6 @@ struct FinalState {
upstream_response_ms: Option<u64>,
upstream_api_overhead_ms: Option<f64>,
upstream_engine_ms: Option<f64>,
upstream_engine_iapi_ttft_ms: Option<f64>,
upstream_engine_service_ttft_ms: Option<f64>,
upstream_engine_iapi_tbt_ms: Option<f64>,
upstream_engine_service_tbt_ms: Option<f64>,
provider_metadata_json: Option<String>,
cost_source: CostSource,
cost_ticks: Option<u128>,
@@ -684,12 +680,6 @@ impl ExecutionStore for FakeStore {
upstream_response_ms: finalization.timings.upstream_response_ms,
upstream_api_overhead_ms: finalization.timings.upstream_api_overhead_ms,
upstream_engine_ms: finalization.timings.upstream_engine_ms,
upstream_engine_iapi_ttft_ms: finalization.timings.upstream_engine_iapi_ttft_ms,
upstream_engine_service_ttft_ms: finalization
.timings
.upstream_engine_service_ttft_ms,
upstream_engine_iapi_tbt_ms: finalization.timings.upstream_engine_iapi_tbt_ms,
upstream_engine_service_tbt_ms: finalization.timings.upstream_engine_service_tbt_ms,
provider_metadata_json: finalization.provider_metadata_json,
cost_source: finalization.cost.source(),
cost_ticks: finalization
@@ -2067,10 +2057,6 @@ fn discarded_attempt_observation_does_not_leak_into_retry_result() {
upstream_response_ms: Some(7_000),
upstream_api_overhead_ms: Some(120.25),
upstream_engine_ms: Some(6400.0),
upstream_engine_iapi_ttft_ms: Some(650.5),
upstream_engine_service_ttft_ms: Some(720.25),
upstream_engine_iapi_tbt_ms: Some(18.45),
upstream_engine_service_tbt_ms: Some(20.12),
..ProviderResponseTimings::default()
});
let second_observation = ProviderResponseObservation::new(
@@ -2164,10 +2150,6 @@ fn discarded_attempt_observation_does_not_leak_into_retry_result() {
assert_eq!(finalization.upstream_response_ms, None);
assert_eq!(finalization.upstream_api_overhead_ms, None);
assert_eq!(finalization.upstream_engine_ms, None);
assert_eq!(finalization.upstream_engine_iapi_ttft_ms, None);
assert_eq!(finalization.upstream_engine_service_ttft_ms, None);
assert_eq!(finalization.upstream_engine_iapi_tbt_ms, None);
assert_eq!(finalization.upstream_engine_service_tbt_ms, None);
let contexts = provider.contexts.lock().unwrap();
assert_eq!(contexts.len(), 2);
assert_eq!(
+1 -3
View File
@@ -10,9 +10,7 @@ use gateway_core::upstream::UpstreamTransport;
use serde_json::json;
#[test]
fn provider_event_keeps_large_wire_and_observation_payloads_behind_one_indirection() {
assert!(std::mem::size_of::<ProviderEvent>() <= 64);
fn provider_event_should_distinguish_client_delivery_from_observation() {
let wire = ProviderEvent::wire(
ProtocolWireEvent::json("openai", None, json!({"type": "response.created"}))
.expect("wire event"),
+3 -8
View File
@@ -162,20 +162,15 @@ fn logging_rejects_zero_retention_windows() {
}
#[test]
fn logging_defaults_to_time_retention_and_ignores_removed_count_retention() {
let mut value = serde_json::json!({
fn logging_defaults_to_time_retention() {
let value = serde_json::json!({
"level": "info", "stdout": true,
"file": { "enabled": true, "directory": "logs", "max_file_size_mb": 20 }
});
let config: LoggingConfig = serde_json::from_value(value.clone()).unwrap();
let config: LoggingConfig = serde_json::from_value(value).unwrap();
assert_eq!(config.request_dump_retention_days, 1.0);
assert_eq!(config.file.retention_days, 7);
assert!(!config.oauth_recovery);
value["file"]["max_files"] = serde_json::json!(20);
assert_eq!(
serde_json::from_value::<LoggingConfig>(value).unwrap(),
config
);
}
#[test]
@@ -543,10 +543,6 @@ fn wire_timings(observation: &RequestObservation) -> RequestTimings {
upstream_response_ms: timings.upstream_response_ms,
upstream_api_overhead_ms: timings.upstream_api_overhead_ms,
upstream_engine_ms: timings.upstream_engine_ms,
upstream_engine_iapi_ttft_ms: timings.upstream_engine_iapi_ttft_ms,
upstream_engine_service_ttft_ms: timings.upstream_engine_service_ttft_ms,
upstream_engine_iapi_tbt_ms: timings.upstream_engine_iapi_tbt_ms,
upstream_engine_service_tbt_ms: timings.upstream_engine_service_tbt_ms,
latency_ms: timings.latency_ms,
}
}
@@ -132,10 +132,6 @@ fn observation(value: &ProviderResponseObservation) -> facts::ResponseObservatio
upstream_response_ms: timings.upstream_response_ms,
upstream_api_overhead_ms: timings.upstream_api_overhead_ms,
upstream_engine_ms: timings.upstream_engine_ms,
upstream_engine_iapi_ttft_ms: timings.upstream_engine_iapi_ttft_ms,
upstream_engine_service_ttft_ms: timings.upstream_engine_service_ttft_ms,
upstream_engine_iapi_tbt_ms: timings.upstream_engine_iapi_tbt_ms,
upstream_engine_service_tbt_ms: timings.upstream_engine_service_tbt_ms,
latency_ms: None,
},
client_headers: value
@@ -817,7 +817,7 @@ async fn completed_observation_contains_terminal_cost_timings_and_failure() {
.with_timings(ModelRequestTimings {
first_text_ms: Some(12),
upstream_response_ms: Some(7_000),
upstream_engine_iapi_tbt_ms: Some(2.450638),
upstream_engine_ms: Some(2.450638),
latency_ms: Some(34),
..ModelRequestTimings::default()
}),
@@ -831,10 +831,7 @@ async fn completed_observation_contains_terminal_cost_timings_and_failure() {
assert_eq!(observed["usage"]["cost"]["total"]["currency"], "USD");
assert_eq!(observed["usage"]["timings"]["first_text_ms"], 12);
assert_eq!(observed["usage"]["timings"]["upstream_response_ms"], 7_000);
assert_eq!(
observed["usage"]["timings"]["upstream_engine_iapi_tbt_ms"],
2.450638
);
assert_eq!(observed["usage"]["timings"]["upstream_engine_ms"], 2.450638);
assert_eq!(observed["usage"]["timings"]["latency_ms"], 34);
assert_eq!(observed["usage"]["failure"]["outcome"], "failed");
assert_eq!(observed["usage"]["failure"]["attempt_count"], 2);
@@ -103,10 +103,24 @@ async fn middleware_preserves_three_state_fast_and_sources() {
Some(90_000)
);
let sources = settings.inspect();
assert!(sources["execution"].get("disable_fast").is_none());
if mode != FastMode::Default && mode == expected {
assert_eq!(sources["execution"]["fast_mode"]["instance_id"], "seed");
let fast_source = &sources["execution"]["fast_mode"];
if mode == FastMode::Default && expected == FastMode::Default {
assert!(fast_source.is_null());
} else {
assert_eq!(
fast_source["instance_id"],
if mode == expected {
"seed"
} else {
"fast-settings"
}
);
assert_eq!(fast_source["value"], expected.as_str());
}
assert_eq!(
sources["execution"]["timeout_ms"]["instance_id"],
"fast-settings"
);
next.run(request).await
})
}),
@@ -186,7 +186,7 @@ async fn facts_are_readable_without_replacing_or_duplicating_the_original_execut
.with_timings(ProviderResponseTimings {
headers_ms: Some(19),
upstream_response_ms: Some(7_000),
upstream_engine_iapi_tbt_ms: Some(2.450638),
upstream_engine_ms: Some(2.450638),
..Default::default()
}),
);
@@ -273,7 +273,7 @@ async fn facts_are_readable_without_replacing_or_duplicating_the_original_execut
);
assert_eq!(facts["observation"]["timings"]["headers_ms"], 19);
assert_eq!(
facts["observation"]["timings"]["upstream_engine_iapi_tbt_ms"],
facts["observation"]["timings"]["upstream_engine_ms"],
2.450638
);
assert_eq!(
@@ -278,7 +278,7 @@ fn icon_content_must_fully_decode_match_its_type_and_fit_dimensions() {
}
#[test]
fn obsolete_manifest_fields_are_rejected_without_aliases() {
fn manifest_rejects_unknown_fields_before_package_validation() {
let manifest = serde_json::json!({
"manifestVersion":2,
"name": "example",
@@ -290,7 +290,8 @@ fn obsolete_manifest_fields_are_rejected_without_aliases() {
"engines": {"codex-proxy-rs": ">=1.0.0, <2.0.0"},
"main": "bin/worker",
"runtime": "trustedProcess",
"capabilities": [{"capability":"request_normalizer"}]
"contributes": {},
"unknownField": true
});
let archive = crate::support::archive(BTreeMap::from([(
"plugin.json".to_owned(),
@@ -110,14 +110,6 @@ pub struct RequestTimings {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub upstream_engine_ms: Option<f64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub upstream_engine_iapi_ttft_ms: Option<f64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub upstream_engine_service_ttft_ms: Option<f64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub upstream_engine_iapi_tbt_ms: Option<f64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub upstream_engine_service_tbt_ms: Option<f64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub latency_ms: Option<u64>,
}
@@ -204,13 +204,13 @@ fn cost_timings_and_failure_keep_unknown_facts_explicit_and_safe() {
let timings = RequestTimings {
first_text_ms: Some(12),
upstream_response_ms: Some(7_000),
upstream_engine_iapi_tbt_ms: Some(2.450638),
upstream_engine_ms: Some(2.450638),
latency_ms: Some(34),
..RequestTimings::default()
};
assert_eq!(
serde_json::to_value(&timings).unwrap(),
json!({"first_text_ms":12,"upstream_response_ms":7000,"upstream_engine_iapi_tbt_ms":2.450638,"latency_ms":34})
json!({"first_text_ms":12,"upstream_response_ms":7000,"upstream_engine_ms":2.450638,"latency_ms":34})
);
let failure = RequestFailure {
@@ -132,52 +132,40 @@ fn installation_checks_protocol_engine_and_target() {
#[test]
fn capability_contract_versions_are_checked_independently_of_rpc_version() {
let mut manifest = packaged_manifest();
assert!(manifest.validate().is_ok());
for version in [0, 1, 2, 3, 4, 6, u32::MAX] {
manifest
.contributes
.get_mut(&Capability::Middleware)
.unwrap()
.version = version;
assert_eq!(manifest.validate(), Err(ManifestError::Invalid));
for (capability, stage, supported, needs_formats) in [
(Capability::Middleware, Stage::Request, 5, true),
(Capability::Management, Stage::Management, 2, false),
(Capability::UpstreamAdapter, Stage::Upstream, 3, true),
(Capability::Scheduler, Stage::Scheduling, 3, false),
] {
for mut manifest in [source_manifest(), packaged_manifest()] {
let formats = if needs_formats {
vec!["openai".into()]
} else {
vec![]
};
manifest.contributes.clear();
manifest.contributes.insert(
capability,
ContributionDeclaration {
id: "9acme.request-tags.contract".into(),
version: supported,
stages: vec![stage],
input_formats: formats.clone(),
output_formats: formats,
},
);
assert!(manifest.validate().is_ok(), "{capability:?} v{supported}");
for version in (0..supported).chain([supported + 1, u32::MAX]) {
manifest.contributes.get_mut(&capability).unwrap().version = version;
assert_eq!(
manifest.validate(),
Err(ManifestError::Invalid),
"{capability:?} v{version}"
);
}
}
}
manifest.contributes.clear();
manifest.contributes.insert(
Capability::Management,
ContributionDeclaration {
id: "9acme.request-tags.management".into(),
version: 3,
stages: vec![Stage::Management],
input_formats: vec![],
output_formats: vec![],
},
);
assert_eq!(manifest.validate(), Err(ManifestError::Invalid));
}
#[test]
fn retired_upstream_contract_requires_an_author_upgrade() {
let mut manifest = source_manifest();
manifest.contributes.clear();
manifest.contributes.insert(
Capability::UpstreamAdapter,
ContributionDeclaration {
id: "9acme.request-tags.upstream".into(),
version: 3,
stages: vec![Stage::Upstream],
input_formats: vec!["openai".into()],
output_formats: vec!["openai".into()],
},
);
assert!(manifest.validate().is_ok());
manifest
.contributes
.get_mut(&Capability::UpstreamAdapter)
.unwrap()
.version = 1;
assert_eq!(manifest.validate(), Err(ManifestError::Invalid));
}
#[test]
@@ -371,18 +359,19 @@ fn state_namespaces_are_base_infrastructure_with_bounded_versions() {
}
#[test]
fn public_manifest_fields_use_only_the_v3_camel_case_shape() {
let value = serde_json::to_value(source_manifest()).unwrap();
assert!(value.get("manifestVersion").is_some());
assert!(value.get("displayName").is_some());
assert!(value.get("configurationSchema").is_some());
assert!(value.get("schema_version").is_none());
fn manifest_round_trips_public_fields_and_rejects_unknown_fields() {
let manifest = source_manifest();
let mut value = serde_json::to_value(&manifest).unwrap();
assert_eq!(value["manifestVersion"], MANIFEST_VERSION);
assert_eq!(value["displayName"], manifest.display_name);
assert_eq!(value["configurationSchema"], manifest.configuration_schema);
assert_eq!(
serde_json::from_value::<Manifest>(value.clone()).unwrap(),
manifest
);
let mut old = value;
old.as_object_mut()
.unwrap()
.insert("schema_version".into(), json!(3));
assert!(serde_json::from_value::<Manifest>(old).is_err());
value["unknownField"] = json!(true);
assert!(serde_json::from_value::<Manifest>(value).is_err());
}
#[test]
@@ -439,26 +428,3 @@ fn contribution_declaration_uses_camel_case_description_fields() {
})
);
}
#[test]
fn scheduler_requires_the_current_contract() {
let mut manifest = packaged_manifest();
manifest.contributes.clear();
manifest.contributes.insert(
Capability::Scheduler,
ContributionDeclaration {
id: "9acme.request-tags.scheduler".into(),
version: 3,
stages: vec![Stage::Scheduling],
input_formats: vec![],
output_formats: vec![],
},
);
assert!(manifest.validate().is_ok());
manifest
.contributes
.get_mut(&Capability::Scheduler)
.unwrap()
.version = 1;
assert_eq!(manifest.validate(), Err(ManifestError::Invalid));
}
+1
View File
@@ -37,6 +37,7 @@ url = "2.5.8"
uuid = { version = "1.23.3", features = ["v4", "v7"] }
[dev-dependencies]
percent-encoding = "2.3"
tempfile = "3.23"
tokio = { version = "1.52.3", default-features = false, features = ["macros", "rt-multi-thread", "fs", "io-util", "net", "process", "time", "test-util"] }
+9 -3
View File
@@ -184,12 +184,17 @@ pub(crate) struct StoreConnectionConfig {
impl StoreConnectionConfig {
fn validate(&self, field: &'static str) -> StoreResult<()> {
require_nonempty("store config", field, &self.url)?;
if self.password.len() != 48 || !self.password.bytes().all(|byte| byte.is_ascii_hexdigit())
// 托管存储的具体密码策略由服务端约束,客户端共用长度与控制字符边界
if !(16..=128).contains(&self.password.len())
|| self.password.chars().any(char::is_control)
|| self.password.trim().is_empty()
{
return Err(StoreError::InvalidData {
source: None,
entity: "store config",
message: format!("{field}.password must be exactly 48 hexadecimal characters"),
message: format!(
"{field}.password must be 16 to 128 bytes, not blank, without control characters"
),
});
}
self.connection_url(field).map(|_| ())
@@ -208,7 +213,8 @@ impl StoreConnectionConfig {
message: format!("{field}.url must not contain a password"),
});
}
url.set_password(Some(&self.password))
// set_password 保留已有百分号序列,先转义字面量 %,避免驱动解码后改变原始密码
url.set_password(Some(&self.password.replace('%', "%25")))
.map_err(|()| StoreError::InvalidData {
source: None,
entity: "store config",
+1 -1
View File
@@ -46,4 +46,4 @@ pub use bundle::*;
pub use config::*;
pub use value::*;
pub(crate) use workers::*;
pub use workers::{CommandStoreDrainError, PostgresHealthProbe, run_request_analytics_cycle};
pub use workers::{CommandStoreDrainError, PostgresHealthProbe, RequestAnalyticsCycle};
@@ -107,14 +107,6 @@ struct UpstreamTimings {
api_overhead_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
engine_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
engine_iapi_ttft_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
engine_service_ttft_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
engine_iapi_tbt_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
engine_service_tbt_ms: Option<f64>,
}
#[derive(Default, Serialize)]
@@ -229,16 +221,9 @@ impl RequestObservation {
value.retry_after_ms,
])?;
// serde_json 会把非有限浮点数写成 null,先拒绝以免把非法指标伪装成缺失
for value in [
timings.upstream_api_overhead_ms,
timings.upstream_engine_ms,
timings.upstream_engine_iapi_ttft_ms,
timings.upstream_engine_service_ttft_ms,
timings.upstream_engine_iapi_tbt_ms,
timings.upstream_engine_service_tbt_ms,
]
.into_iter()
.flatten()
for value in [timings.upstream_api_overhead_ms, timings.upstream_engine_ms]
.into_iter()
.flatten()
{
if !value.is_finite() || value < 0.0 || value >= i64::MAX as f64 {
return Err(invalid(
@@ -263,10 +248,6 @@ impl RequestObservation {
response_ms: timings.upstream_response_ms,
api_overhead_ms: timings.upstream_api_overhead_ms,
engine_ms: timings.upstream_engine_ms,
engine_iapi_ttft_ms: timings.upstream_engine_iapi_ttft_ms,
engine_service_ttft_ms: timings.upstream_engine_service_ttft_ms,
engine_iapi_tbt_ms: timings.upstream_engine_iapi_tbt_ms,
engine_service_tbt_ms: timings.upstream_engine_service_tbt_ms,
}),
}),
transport: Some(TransportObservation {
@@ -192,6 +192,8 @@ async fn apply_changes(
}
let mut buckets =
BTreeMap::<DateTime<Utc>, BTreeMap<(String, Vec<String>, String), MetricState>>::new();
let mut empty_groups = Vec::new();
let mut rebuild = Vec::new();
for (key, change) in changes {
if change.previous.encode()? == change.current.encode()?
&& change.previous.last_used_at() == change.current.last_used_at()
@@ -202,21 +204,17 @@ async fn apply_changes(
let needs_rebuild = subtract(&mut state, &change.previous, &change.current)?;
state.merge(&change.current)?;
if state.count("request_count") == 0 {
let statement = format!(
"delete from {table} where bucket_start=$1 and collection=$2 and dimensions=$3 and outcome=$4"
);
sqlx::query(sqlx::AssertSqlSafe(statement))
.bind(key.0)
.bind(&key.1)
.bind(&key.2)
.bind(&key.3)
.execute(&mut **tx)
.await
.map_err(|error| postgres_unavailable("remove empty analytics group", error))?;
empty_groups.push(serde_json::json!({
"bucket_start": key.0,
"collection": key.1,
"dimensions": key.2,
"outcome": key.3,
}));
} else {
// 预测状态只有可加计数和美元总额,零金额无需修复币种存在性
if needs_rebuild && key.1 != forecast::COLLECTION {
state = rebuild_group(tx, hourly, progress, &key).await?;
rebuild.push(key);
continue;
}
buckets
.entry(key.0)
@@ -224,6 +222,24 @@ async fn apply_changes(
.insert((key.1, key.2, key.3), state);
}
}
for (key, state) in rebuild_groups(tx, hourly, progress, rebuild).await? {
buckets
.entry(key.0)
.or_default()
.insert((key.1, key.2, key.3), state);
}
// 保留期缩短会同时清空大量分组,集合删除避免 SQL 往返随分组数放大
for groups in empty_groups.chunks(1_000) {
let statement = format!(
"delete from {table} a using jsonb_to_recordset($1::jsonb) k(bucket_start timestamptz,collection text,dimensions text[],outcome text)
where a.bucket_start=k.bucket_start and a.collection=k.collection and a.dimensions=k.dimensions and a.outcome=k.outcome"
);
sqlx::query(sqlx::AssertSqlSafe(statement))
.bind(sqlx::types::Json(groups))
.execute(&mut **tx)
.await
.map_err(|error| postgres_unavailable("remove empty analytics groups", error))?;
}
write_bucket_groups(tx, table, buckets).await
}
@@ -311,61 +327,91 @@ pub(super) fn subtract(
Ok(rebuild)
}
async fn rebuild_group(
async fn rebuild_groups(
tx: &mut sqlx::Transaction<'_, Postgres>,
hourly: bool,
progress: &BTreeMap<DateTime<Utc>, Progress>,
key: &GroupKey,
) -> StoreResult<MetricState> {
let mut end = key.0 + TimeDelta::seconds(if hourly { 3600 } else { 900 });
let mut cursor = None;
if hourly {
if let Some((_, owner)) = progress.range(key.0..end).find(|(_, p)| p.phase == 4)
&& let Some(time) = owner.started_at
{
let candidate = (time, key.1.clone(), key.2.clone(), key.3.clone());
end = if owner.contains_quarter(&candidate) {
time + TimeDelta::minutes(15)
} else {
time
};
keys: Vec<GroupKey>,
) -> StoreResult<BTreeMap<GroupKey, MetricState>> {
let mut by_collection = BTreeMap::<&str, Vec<&GroupKey>>::new();
for key in &keys {
by_collection.entry(&key.1).or_default().push(key);
}
let mut rebuilt = BTreeMap::new();
for (collection, keys) in by_collection {
let columns = worker::COLLECTION_COLUMNS
.iter()
.find_map(|(name, columns)| (*name == collection).then_some(*columns))
.ok_or_else(|| invalid("unknown incremental analytics collection"))?;
// 同类分组共用维度匹配和聚合 SQL,各组仍独立遵守已发布的构建前缀
for keys in keys.chunks(1_000) {
let mut input = Vec::with_capacity(keys.len());
for (index, key) in keys.iter().enumerate() {
let mut end = key.0 + TimeDelta::seconds(if hourly { 3600 } else { 900 });
let mut cursor = None;
if hourly {
if let Some((_, owner)) = progress.range(key.0..end).find(|(_, p)| p.phase == 4)
&& let Some(time) = owner.started_at
{
let candidate = (time, key.1.clone(), key.2.clone(), key.3.clone());
end = if owner.contains_quarter(&candidate) {
time + TimeDelta::minutes(15)
} else {
time
};
}
} else if let Some(stage) = progress.get(&key.0)
&& stage.phase == 1
{
cursor = stage.started_at.zip(stage.id.as_deref());
}
input.push(serde_json::json!({
"bucket_start": key.0,
"end_at": end,
"cursor_time": cursor.map(|(time, _)| time),
"cursor_id": cursor.map(|(_, id)| id),
"dimensions": key.2,
"outcome": key.3,
"name": index.to_string(),
}));
rebuilt.insert((*key).clone(), MetricState::default());
}
// 等值桶键避免范围连接反复比较其他时间桶,范围和游标仍限定已发布前缀
let mut query = QueryBuilder::<Postgres>::new(
"with matching as materialized (select mr.*,k.bucket_start as output_bucket,k.name,coalesce((",
);
query.push(completed_usage_fact_predicate("mr"))
.push("),false) as is_completed_usage from jsonb_to_recordset(")
.push_bind(serde_json::Value::Array(input))
.push("::jsonb) k(bucket_start timestamptz,end_at timestamptz,cursor_time timestamptz,cursor_id text,dimensions text[],outcome text,name text)
join request_analytics_facts mr on date_bin(make_interval(secs => ")
.push_bind(if hourly { 3600_f64 } else { 900_f64 })
.push("),mr.started_at,timestamptz '2000-01-01 UTC')=k.bucket_start
and mr.started_at >= k.bucket_start and mr.started_at < k.end_at
and mr.recovered_at is null and mr.outcome=k.outcome
and (k.cursor_time is null or (mr.started_at,mr.id collate \"C\") <= (k.cursor_time,k.cursor_id collate \"C\"))");
for (index, column) in columns.iter().enumerate() {
query.push(format!(
" and coalesce(mr.{column}::text,'')=k.dimensions[{}]",
index + 1
));
}
query.push(")");
aggregate::push_fact_aggregation(&mut query, false, true);
let rows = query.build().fetch_all(&mut **tx).await.map_err(|error| {
postgres_unavailable("rebuild changed analytics extrema", error)
})?;
for row in rows {
let name: String = get(&row, "name")?;
let index = name
.parse::<usize>()
.map_err(|_| invalid("invalid analytics rebuild group"))?;
let key = keys
.get(index)
.ok_or_else(|| invalid("unknown analytics rebuild group"))?;
rebuilt.insert((*key).clone(), MetricState::decode(&row)?);
}
}
} else if let Some(stage) = progress.get(&key.0)
&& stage.phase == 1
{
cursor = stage.started_at.zip(stage.id.as_deref());
}
let mut query = QueryBuilder::<Postgres>::new("with matching as materialized (select mr.*,");
query.push_bind(key.0).push("::timestamptz as output_bucket,''::text as name,coalesce((")
.push(completed_usage_fact_predicate("mr")).push("),false) as is_completed_usage from request_analytics_facts mr where mr.started_at >= ")
.push_bind(key.0).push(" and mr.started_at < ").push_bind(end)
.push(" and mr.recovered_at is null and mr.outcome=").push_bind(&key.3);
if let Some((time, id)) = cursor {
query
.push(" and (mr.started_at,mr.id collate \"C\") <= (")
.push_bind(time)
.push(",")
.push_bind(id)
.push(")");
}
let columns = worker::COLLECTION_COLUMNS
.iter()
.find_map(|(collection, columns)| (*collection == key.1).then_some(*columns))
.ok_or_else(|| invalid("unknown incremental analytics collection"))?;
for (column, value) in columns.iter().zip(&key.2) {
query
.push(format!(" and coalesce(mr.{column}::text,'')="))
.push_bind(value);
}
query.push(")");
aggregate::push_fact_aggregation(&mut query, false, true);
let row = query
.build()
.fetch_optional(&mut **tx)
.await
.map_err(|error| postgres_unavailable("rebuild changed analytics extrema", error))?;
row.as_ref()
.map(MetricState::decode)
.transpose()
.map(Option::unwrap_or_default)
Ok(rebuilt)
}
@@ -83,7 +83,17 @@ impl PgRequestAnalyticsRepository {
return Ok(AnalyticsMaintenance { reset_progress: true, ..Default::default() });
}
let mut result = AnalyticsMaintenance::default();
let rows = sqlx::query("select request_id, change_version from request_analytics_pending_requests order by oldest_pending_at, request_id limit $1")
// 历史删除可能一次排入大量旧任务,给仍存在的请求预留半批名额,避免实时统计饥饿
// 其余名额仍按最早入队消费,两类任务都能前进;同一快照内选版本,沿用精确版本 ACK
let rows = sqlx::query("with live as materialized (
select pending.request_id, pending.change_version from request_analytics_pending_requests pending
join model_requests source on source.id=pending.request_id
order by pending.oldest_pending_at, pending.request_id limit ($1::bigint / 2)
), oldest as (
select pending.request_id, pending.change_version from request_analytics_pending_requests pending
where not exists (select 1 from live where live.request_id=pending.request_id)
order by pending.oldest_pending_at, pending.request_id limit ($1::bigint - (select count(*) from live))
) select request_id, change_version from live union all select request_id, change_version from oldest")
.bind(i64::from(batch_size)).fetch_all(&mut *transaction).await.map_err(|error| postgres_unavailable("load analytics request queue", error))?;
let mut jobs = Vec::with_capacity(rows.len());
for row in rows {
@@ -350,10 +350,6 @@ pub(crate) fn usage_list_record_from_row(
upstream_response_ms: optional_unsigned(row, "upstream_response_ms")?,
upstream_api_overhead_ms: get(row, "upstream_api_overhead_ms")?,
upstream_engine_ms: get(row, "upstream_engine_ms")?,
upstream_engine_iapi_ttft_ms: get(row, "upstream_engine_iapi_ttft_ms")?,
upstream_engine_service_ttft_ms: get(row, "upstream_engine_service_ttft_ms")?,
upstream_engine_iapi_tbt_ms: get(row, "upstream_engine_iapi_tbt_ms")?,
upstream_engine_service_tbt_ms: get(row, "upstream_engine_service_tbt_ms")?,
latency_ms: optional_unsigned(row, "latency_ms")?,
admission_decision_ms: optional_unsigned(row, "admission_decision_ms")?,
account_selection_wait_ms: optional_unsigned(row, "account_selection_wait_ms")?,
@@ -446,10 +442,6 @@ pub(crate) fn usage_record_from_row(row: &sqlx::postgres::PgRow) -> StoreResult<
upstream_response_ms: optional_unsigned(row, "upstream_response_ms")?,
upstream_api_overhead_ms: get(row, "upstream_api_overhead_ms")?,
upstream_engine_ms: get(row, "upstream_engine_ms")?,
upstream_engine_iapi_ttft_ms: get(row, "upstream_engine_iapi_ttft_ms")?,
upstream_engine_service_ttft_ms: get(row, "upstream_engine_service_ttft_ms")?,
upstream_engine_iapi_tbt_ms: get(row, "upstream_engine_iapi_tbt_ms")?,
upstream_engine_service_tbt_ms: get(row, "upstream_engine_service_tbt_ms")?,
latency_ms: optional_unsigned(row, "latency_ms")?,
admission_decision_ms: optional_unsigned(row, "admission_decision_ms")?,
account_selection_wait_ms: optional_unsigned(row, "account_selection_wait_ms")?,
@@ -128,7 +128,7 @@ pub(crate) const USAGE_LIST_RECORD_SELECT: &str =
mr.transport_decision_wait_ms, mr.connect_ms, mr.headers_ms,
mr.first_event_ms, mr.first_reasoning_ms, mr.first_text_ms, mr.first_token_ms,
mr.provider_processing_ms, mr.upstream_response_ms,
mr.upstream_api_overhead_ms, mr.upstream_engine_ms, mr.upstream_engine_iapi_ttft_ms, mr.upstream_engine_service_ttft_ms, mr.upstream_engine_iapi_tbt_ms, mr.upstream_engine_service_tbt_ms, mr.latency_ms, mr.admission_decision_ms,
mr.upstream_api_overhead_ms, mr.upstream_engine_ms, mr.latency_ms, mr.admission_decision_ms,
mr.account_selection_wait_ms, mr.capacity_used_slots, mr.capacity_total_slots,
host(mr.client_ip) as client_ip, mr.user_agent,
mr.reasoning_effort, mr.reasoning_preset, mr.subagent_kind, mr.compact,
@@ -157,7 +157,7 @@ pub(crate) const USAGE_RECORD_DETAIL_SELECT: &str =
mr.cost_currency, mr.transport_decision_wait_ms, mr.connect_ms, mr.headers_ms,
mr.first_event_ms, mr.first_reasoning_ms, mr.first_text_ms, mr.first_token_ms,
mr.provider_processing_ms, mr.upstream_response_ms,
mr.upstream_api_overhead_ms, mr.upstream_engine_ms, mr.upstream_engine_iapi_ttft_ms, mr.upstream_engine_service_ttft_ms, mr.upstream_engine_iapi_tbt_ms, mr.upstream_engine_service_tbt_ms, mr.latency_ms, mr.admission_decision_ms,
mr.upstream_api_overhead_ms, mr.upstream_engine_ms, mr.latency_ms, mr.admission_decision_ms,
mr.account_selection_wait_ms, mr.capacity_used_slots, mr.capacity_total_slots,
host(mr.client_ip) as client_ip,
mr.user_agent, mr.reasoning_effort, mr.reasoning_preset, mr.request_kind,
+108 -39
View File
@@ -2,6 +2,7 @@
use super::*;
use gateway_core::task::DaemonTask;
use std::sync::atomic::{AtomicU32, Ordering};
const COMMAND_DRAIN_TIMEOUT: Duration = Duration::from_secs(4);
@@ -325,12 +326,16 @@ pub(crate) fn request_analytics_worker(
Ok(WorkerContribution::Registration(scheduled_worker(
id,
Duration::from_secs(5),
Box::new(RequestAnalyticsTask { repository }),
Box::new(RequestAnalyticsTask {
repository,
cycle: RequestAnalyticsCycle::default(),
}),
)?))
}
struct RequestAnalyticsTask {
repository: postgres::PgRequestAnalyticsRepository,
cycle: RequestAnalyticsCycle,
}
impl ScheduledTask for RequestAnalyticsTask {
@@ -341,7 +346,7 @@ impl ScheduledTask for RequestAnalyticsTask {
Box::pin(async move {
tokio::select! {
() = context.cancellation().cancelled() => Ok(()),
result = run_request_analytics_cycle(&self.repository) => result.map(|_| ()).map_err(|source| {
result = self.cycle.run(&self.repository) => result.map(|_| ()).map_err(|source| {
WorkerTaskError::safe("request analytics maintenance failed").with_source(source)
}),
}
@@ -349,43 +354,107 @@ impl ScheduledTask for RequestAnalyticsTask {
}
}
/// 生产与隔离压测共用的单周期推进预算;各批独立提交并归还观测槽位
pub async fn run_request_analytics_cycle(
repository: &postgres::PgRequestAnalyticsRepository,
) -> StoreResult<postgres::AnalyticsMaintenance> {
let deadline = tokio::time::Instant::now() + Duration::from_secs(10);
let mut total = postgres::AnalyticsMaintenance::default();
loop {
let result = match tokio::time::timeout_at(deadline, repository.maintain(1_000)).await {
Ok(result) => result?,
Err(source) => {
if total.projected_requests == 0
&& total.backfilled_requests == 0
&& total.rebuilt_buckets == 0
&& total.advanced_buckets == 0
&& !total.reset_progress
{
return Err(StoreError::Unavailable {
backend: StoreBackend::PostgreSql,
message: "request analytics cycle timed out without progress".to_owned(),
source: Some(gateway_core::error::ErrorSource::new(source)),
});
}
return Ok(total);
}
};
let idle = result.projected_requests == 0
&& result.backfilled_requests == 0
&& result.rebuilt_buckets == 0
&& result.advanced_buckets == 0
&& !result.reset_progress;
total.projected_requests += result.projected_requests;
total.backfilled_requests += result.backfilled_requests;
total.rebuilt_buckets += result.rebuilt_buckets;
total.advanced_buckets += result.advanced_buckets;
total.reset_progress |= result.reset_progress;
if idle || tokio::time::Instant::now() >= deadline {
return Ok(total);
/// 统计消费周期预算,跨轮保留批量上限以便超时后的下一轮提交进展
pub struct RequestAnalyticsCycle {
batch_size: AtomicU32,
}
impl Default for RequestAnalyticsCycle {
fn default() -> Self {
Self {
batch_size: AtomicU32::new(1_000),
}
}
}
impl RequestAnalyticsCycle {
/// 生产与隔离压测共用的推进路径;各批独立提交并归还观测槽位
pub async fn run(
&self,
repository: &postgres::PgRequestAnalyticsRepository,
) -> StoreResult<postgres::AnalyticsMaintenance> {
let deadline = tokio::time::Instant::now() + Duration::from_secs(10);
let mut total = postgres::AnalyticsMaintenance::default();
loop {
let batch_size = self.batch_size.load(Ordering::Relaxed);
let started = tokio::time::Instant::now();
let result =
match tokio::time::timeout_at(deadline, repository.maintain(batch_size)).await {
Ok(Ok(result)) => result,
Ok(Err(error)) => {
if analytics_maintenance_timed_out(&error) {
self.reduce_batch_size(batch_size);
}
return Err(error);
}
Err(source) => {
if total.projected_requests == 0
&& total.backfilled_requests == 0
&& total.rebuilt_buckets == 0
&& total.advanced_buckets == 0
&& !total.reset_progress
{
// 只在整轮没有返回进展时退让,避免尾批剩余时间短导致持续缩批
self.reduce_batch_size(batch_size);
return Err(StoreError::Unavailable {
backend: StoreBackend::PostgreSql,
message: "request analytics cycle timed out without progress"
.to_owned(),
source: Some(gateway_core::error::ErrorSource::new(source)),
});
}
return Ok(total);
}
};
let idle = result.projected_requests == 0
&& result.backfilled_requests == 0
&& result.rebuilt_buckets == 0
&& result.advanced_buckets == 0
&& !result.reset_progress;
total.projected_requests += result.projected_requests;
total.backfilled_requests += result.backfilled_requests;
total.rebuilt_buckets += result.rebuilt_buckets;
total.advanced_buckets += result.advanced_buckets;
total.reset_progress |= result.reset_progress;
// 满批快速完成才恢复吞吐,空结果也可能只是其他实例持有维护锁
if (result.projected_requests >= u64::from(batch_size)
|| result.backfilled_requests >= u64::from(batch_size))
&& started.elapsed() < Duration::from_secs(2)
{
self.batch_size
.store((batch_size * 2).min(1_000), Ordering::Relaxed);
}
if idle {
return Ok(total);
}
if tokio::time::Instant::now() >= deadline {
return Ok(total);
}
}
}
fn reduce_batch_size(&self, batch_size: u32) {
let next_batch_size = (batch_size / 2).max(1);
self.batch_size.store(next_batch_size, Ordering::Relaxed);
tracing::warn!(
batch_size,
next_batch_size,
"request analytics batch timed out"
);
}
}
fn analytics_maintenance_timed_out(error: &StoreError) -> bool {
let StoreError::Unavailable {
source: Some(source),
..
} = error
else {
return false;
};
source.is::<tokio::time::error::Elapsed>()
|| source
.downcast_ref::<sqlx::Error>()
.and_then(sqlx::Error::as_database_error)
.is_some_and(|error| error.code().as_deref() == Some("57014"))
}
+134 -6
View File
@@ -1,4 +1,4 @@
//! 验证只读检查 Bundle 不执行迁移且拒绝业务写入
//! 验证 Store 认证、凭据编码与只读检查 Bundle 的写入边界
use gateway_core::{
account::{
@@ -33,11 +33,6 @@ async fn inspection_bundle_does_not_migrate_and_rejects_business_writes() {
database
.query_pairs_mut()
.append_pair("options", &format!("-csearch_path={schema}"));
let config_connection = |mut url: url::Url| {
let password = url.password().unwrap().to_owned();
url.set_password(None).unwrap();
json!({"url":url.as_str(),"password":password})
};
let mut config: StoreConfig = serde_json::from_value(json!({
"database":config_connection(database), "redis":config_connection(url::Url::parse(&redis).unwrap()),
})).unwrap();
@@ -109,3 +104,136 @@ async fn inspection_bundle_does_not_migrate_and_rejects_business_writes() {
.unwrap();
admin.close().await;
}
#[tokio::test]
async fn store_bundle_authenticates_configured_and_redis_acl_credentials() {
let (Some(database), Some(redis_url)) = (
crate::support::test_env("CPR_TEST_DATABASE_URL"),
crate::support::test_env("CPR_TEST_REDIS_URL"),
) else {
return;
};
let document = json!({
"database": config_connection(url::Url::parse(&database).unwrap()),
"redis": config_connection(url::Url::parse(&redis_url).unwrap()),
});
let mut admin = redis::Client::open(redis_url.as_str())
.unwrap()
.get_connection_manager()
.await
.unwrap();
let username = format!("cpr_password_{}", uuid::Uuid::new_v4().simple());
let password = "Managed%41%25%FF@:/?#%&+$'\"\\ secret9!";
redis::cmd("ACL")
.arg("SETUSER")
.arg(&username)
.arg("on")
.arg(format!(">{password}"))
.arg("~*")
.arg("&*")
.arg("+@all")
.query_async::<()>(&mut admin)
.await
.unwrap();
let mut acl_url = url::Url::parse(&redis_url).unwrap();
acl_url.set_username(&username).unwrap();
acl_url.set_password(None).unwrap();
let mut acl_document = document.clone();
acl_document["redis"] = json!({"url": acl_url.as_str(), "password": password});
let directory = tempfile::tempdir().unwrap();
let mut results = Vec::new();
// 同时走 Store 的配置解析、URL 构造与真实连接路径,断言前先清理临时 ACL 用户
for document in [document, acl_document] {
let mut config: StoreConfig = serde_json::from_value(document).unwrap();
let result = async {
config.resolve_and_validate(directory.path())?;
let bundle = gateway_store::initialize_read_only(config).await?;
let mut states = Vec::new();
for probe in bundle.health_probes() {
states.push(probe.check().await);
}
Ok::<_, gateway_store::StoreError>(states)
}
.await;
results.push(result);
}
redis::cmd("ACL")
.arg("DELUSER")
.arg(&username)
.query_async::<i64>(&mut admin)
.await
.unwrap();
for result in results {
let states = result.expect("Store credentials authenticate without double encoding");
assert!(!states.is_empty());
assert!(
states
.iter()
.all(|state| *state == gateway_core::health::HealthState::Healthy)
);
}
}
#[tokio::test]
async fn store_bundle_authenticates_postgres_password_with_percent_sequences() {
let (Some(database), Some(redis_url)) = (
crate::support::test_env("CPR_TEST_DATABASE_URL"),
crate::support::test_env("CPR_TEST_REDIS_URL"),
) else {
return;
};
let admin = PgPoolOptions::new()
.max_connections(1)
.connect(&database)
.await
.unwrap();
let username = format!("cpr_password_{}", uuid::Uuid::new_v4().simple());
let password = "Managed%41%25%FFPassword9!";
sqlx::raw_sql(sqlx::AssertSqlSafe(format!(
"CREATE ROLE {username} LOGIN PASSWORD '{password}'"
)))
.execute(&admin)
.await
.unwrap();
let mut database_url = url::Url::parse(&database).unwrap();
database_url.set_username(&username).unwrap();
database_url.set_password(None).unwrap();
let mut config: StoreConfig = serde_json::from_value(json!({
"database": {"url": database_url.as_str(), "password": password},
"redis": config_connection(url::Url::parse(&redis_url).unwrap()),
}))
.unwrap();
let directory = tempfile::tempdir().unwrap();
let result = async {
config.resolve_and_validate(directory.path())?;
let bundle = gateway_store::initialize_read_only(config).await?;
let mut states = Vec::new();
for probe in bundle.health_probes() {
states.push(probe.check().await);
}
Ok::<_, gateway_store::StoreError>(states)
}
.await;
// 连接失败时也先清理临时角色,避免回归断言留下测试账号
sqlx::raw_sql(sqlx::AssertSqlSafe(format!("DROP ROLE {username}")))
.execute(&admin)
.await
.unwrap();
admin.close().await;
let states = result.expect("PostgreSQL password percent sequences remain literal");
assert!(!states.is_empty());
assert!(
states
.iter()
.all(|state| *state == gateway_core::health::HealthState::Healthy)
);
}
fn config_connection(mut url: url::Url) -> serde_json::Value {
let password = percent_encoding::percent_decode_str(url.password().unwrap())
.decode_utf8()
.expect("UTF-8 fixture password")
.into_owned();
url.set_password(None).unwrap();
json!({"url": url.as_str(), "password": password})
}
+1 -8
View File
@@ -20,7 +20,7 @@ fn store_config_should_derive_backup_staging_from_runtime_data_dir() {
}
#[test]
fn store_pool_config_ignores_unknown_options_and_keeps_connection_pool_limits() {
fn store_pool_config_defaults_reserve_capacity_for_observability() {
let pool: StorePoolConfig =
serde_json::from_value(serde_json::json!({})).expect("default pool configuration");
@@ -37,13 +37,6 @@ fn store_pool_config_ignores_unknown_options_and_keeps_connection_pool_limits()
.observability_max_connections(),
40
);
assert_eq!(
serde_json::from_value::<StorePoolConfig>(serde_json::json!({
"statement_timeout_seconds": 60,
}))
.unwrap(),
pool,
);
}
#[test]
@@ -27,36 +27,6 @@ use super::TestDatabase;
const MIXED_GROUP: &str = "grp_00000000000000000000000000000001";
const EMPTY_GROUP: &str = "grp_00000000000000000000000000000002";
#[tokio::test]
async fn fast_mode_migration_preserves_existing_group_choices() {
let Some(database) = TestDatabase::create_through("group_fast_mode_upgrade", 21).await else {
return;
};
for (id, disabled) in [(MIXED_GROUP, true), (EMPTY_GROUP, false)] {
sqlx::query("insert into account_groups (id, name, color, disable_fast, created_at, updated_at) values ($1, $1, '#2563EBFF', $2, now(), now())")
.bind(id)
.bind(disabled)
.execute(&database.pool)
.await
.unwrap();
}
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
let groups = sqlx::query_as::<_, (String, String)>(
"select id, fast_mode from account_groups order by id",
)
.fetch_all(&database.pool)
.await
.unwrap();
assert_eq!(
groups,
vec![
(MIXED_GROUP.to_owned(), "disabled".to_owned()),
(EMPTY_GROUP.to_owned(), "default".to_owned())
]
);
database.close().await;
}
#[tokio::test]
async fn groups_aggregate_cross_provider_members_and_key_bindings_without_multiplication() {
let Some(database) = TestDatabase::create("account_group_aggregate").await else {
@@ -840,20 +810,19 @@ async fn image_generation_group_updates_preserve_omitted_values_and_publish_snap
}
#[tokio::test]
async fn image_generation_migration_allows_existing_and_default_groups() {
let Some(database) = TestDatabase::create_through("group_image_upgrade", 31).await else {
async fn image_generation_defaults_to_allowed_for_new_groups() {
let Some(database) = TestDatabase::create("group_image_default").await else {
return;
};
sqlx::query("insert into account_groups (id, name, color, created_at, updated_at) values ($1, $1, '#2563EBFF', now(), now())")
.bind(MIXED_GROUP).execute(&database.pool).await.unwrap();
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
sqlx::query("insert into account_groups (id, name, color, created_at, updated_at) values ($1, $1, '#2563EBFF', now(), now())")
.bind(EMPTY_GROUP).execute(&database.pool).await.unwrap();
let allowed =
sqlx::query_scalar::<_, bool>("select allow_image_generation from account_groups")
.fetch_all(&database.pool)
.await
.unwrap();
assert_eq!(allowed, vec![true, true]);
let allowed = sqlx::query_scalar::<_, bool>(
"select allow_image_generation from account_groups where id = $1",
)
.bind(EMPTY_GROUP)
.fetch_one(&database.pool)
.await
.unwrap();
assert!(allowed);
database.close().await;
}
@@ -1,87 +0,0 @@
//! 验证统计升级保留历史事实与回填进度,并重建已有时间桶
use chrono::{DateTime, Utc};
use serde_json::Value;
use super::{TEST_MIGRATOR, TestDatabase};
#[tokio::test]
async fn analytics_upgrade_preserves_facts_and_backfill_progress_and_requeues_buckets() {
let Some(database) = TestDatabase::create_through("analytics_upgrade", 29).await else {
return;
};
sqlx::raw_sql(
"insert into request_analytics_facts(id,started_at,source_version) values
('first','2026-10-01 10:01:00+00',1),
('same_bucket','2026-10-01 10:02:00+00',2),
('next_bucket','2026-10-01 10:16:00+00',3);
insert into request_analytics_pending_buckets(bucket_start,oldest_pending_at)
values ('2026-10-01 10:00:00+00','2026-10-01 09:00:00+00');
update request_analytics_build_state set
backfill_complete=true, backfill_cursor_id='next_bucket',
backfill_cursor_started_at='2026-10-01 10:16:00+00',
coverage_start='2026-10-01 10:00:00+00',
coverage_end='2026-10-01 10:30:00+00', generation=7;",
)
.execute(&database.pool)
.await
.unwrap();
let facts_before: Vec<Value> =
sqlx::query_scalar("select to_jsonb(f) from request_analytics_facts f order by id")
.fetch_all(&database.pool)
.await
.unwrap();
let progress_before: Value =
sqlx::query_scalar("select to_jsonb(s) from request_analytics_build_state s")
.fetch_one(&database.pool)
.await
.unwrap();
let pending_before: (i64, DateTime<Utc>) = sqlx::query_as(
"select change_version,oldest_pending_at from request_analytics_pending_buckets",
)
.fetch_one(&database.pool)
.await
.unwrap();
TEST_MIGRATOR.run(&database.pool).await.unwrap();
let facts_after: Vec<Value> =
sqlx::query_scalar("select to_jsonb(f) from request_analytics_facts f order by id")
.fetch_all(&database.pool)
.await
.unwrap();
let progress_after: Value =
sqlx::query_scalar("select to_jsonb(s) from request_analytics_build_state s")
.fetch_one(&database.pool)
.await
.unwrap();
assert_eq!(facts_after, facts_before);
assert_eq!(progress_after, progress_before);
let pending: Vec<(DateTime<Utc>, i64, DateTime<Utc>)> = sqlx::query_as(
"select bucket_start,change_version,oldest_pending_at
from request_analytics_pending_buckets order by bucket_start",
)
.fetch_all(&database.pool)
.await
.unwrap();
let expected_buckets = ["2026-10-01T10:00:00Z", "2026-10-01T10:15:00Z"]
.map(|value| value.parse::<DateTime<Utc>>().unwrap());
assert_eq!(
pending.iter().map(|row| row.0).collect::<Vec<_>>(),
expected_buckets
);
assert!(pending[0].1 > pending_before.0);
assert_eq!(pending[0].2, pending_before.1);
// 重启不会再次入队,也不会覆盖最早待处理时间
TEST_MIGRATOR.run(&database.pool).await.unwrap();
let pending_after_restart: Vec<(DateTime<Utc>, i64, DateTime<Utc>)> = sqlx::query_as(
"select bucket_start,change_version,oldest_pending_at
from request_analytics_pending_buckets order by bucket_start",
)
.fetch_all(&database.pool)
.await
.unwrap();
assert_eq!(pending_after_restart, pending);
database.close().await;
}
@@ -1,4 +1,4 @@
//! PostgreSQL 连接错误、迁移应用与事务归属测试
//! PostgreSQL 连接初始化、会话设置与事务取消测试
use std::error::Error as _;
@@ -109,32 +109,32 @@ async fn invalid_postgres_options_preserve_the_sqlx_source() {
}
#[tokio::test]
async fn connect_and_migrate_should_apply_all_migrations_once_and_reopen_cleanly() {
async fn connection_initialization_preserves_settings_when_reopened() {
let Some(database_url) = crate::support::test_env("CPR_TEST_DATABASE_URL") else {
return;
};
let database = format!("cpr_store_migrator_{}", Uuid::new_v4().simple());
let database = format!("cpr_store_connection_{}", Uuid::new_v4().simple());
let admin = PgPoolOptions::new()
.max_connections(1)
.connect(&database_url)
.await
.expect("connect migration test PostgreSQL");
.expect("connect test PostgreSQL");
sqlx::raw_sql(sqlx::AssertSqlSafe(format!(
"create database \"{database}\""
)))
.execute(&admin)
.await
.expect("create migration test database");
.expect("create connection test database");
let isolated_url = PgConnectOptions::from_str(&database_url)
.expect("parse migration test PostgreSQL URL")
.expect("parse test PostgreSQL URL")
.database(&database)
.to_url_lossy()
.to_string();
let pool_config = gateway_store::StorePoolConfig::default();
let first = connect_and_migrate(&isolated_url, pool_config)
.await
.expect("apply migrations through production migrator");
.expect("initialize production connection");
let session_settings = sqlx::query_as::<_, (String, i64, i64, i64)>(
"select current_setting('application_name'),
extract(epoch from current_setting('statement_timeout')::interval)::bigint,
@@ -154,70 +154,20 @@ async fn connect_and_migrate_should_apply_all_migrations_once_and_reopen_cleanly
timezone, "UTC",
"runtime sessions use an explicit technical time basis"
);
let first_tables = sqlx::query_scalar::<_, String>(
"select table_name
from information_schema.tables
where table_schema = 'public'
order by table_name",
)
.fetch_all(&first)
.await
.expect("load migrated tables");
sqlx::query("update runtime_settings set refresh_margin_seconds = 1800 where id = 1")
.execute(&first)
.await
.expect("save custom runtime setting");
first.close().await;
let second = connect_and_migrate(&isolated_url, pool_config)
.await
.expect("reopen database through production migrator");
let migration_count =
sqlx::query_scalar::<_, i64>("select count(*) from _sqlx_migrations where success")
.expect("reopen production connection");
let refresh_margin: i64 =
sqlx::query_scalar("select refresh_margin_seconds from runtime_settings where id = 1")
.fetch_one(&second)
.await
.expect("count successful migrations");
let response_id_types = sqlx::query_scalar::<_, String>(
"select data_type
from information_schema.columns
where table_schema = 'public'
and table_name = 'model_requests'
and column_name in ('client_response_id', 'upstream_response_id')
order by column_name",
)
.fetch_all(&second)
.await
.expect("load opaque response ID column types");
let raw_response_id_index_exists = sqlx::query_scalar::<_, bool>(
"select exists (
select 1
from pg_indexes
where schemaname = 'public'
and indexname = 'model_requests_client_response_uq'
)",
)
.fetch_one(&second)
.await
.expect("check removed raw response ID index");
let legacy_key_provider_column_exists = sqlx::query_scalar::<_, bool>(
"select exists (
select 1 from information_schema.columns
where table_schema = 'public'
and table_name = 'client_api_keys'
and column_name = 'provider_kind'
)",
)
.fetch_one(&second)
.await
.expect("check removed client key provider column");
let routing_history_columns = sqlx::query_scalar::<_, String>(
"select column_name from information_schema.columns
where table_schema = 'public'
and table_name = 'model_requests'
and column_name in (
'routing_scope', 'routing_group_refs', 'routing_group_names_snapshot'
)
order by column_name",
)
.fetch_all(&second)
.await
.expect("load routing history columns");
.expect("load saved runtime setting");
second.close().await;
sqlx::raw_sql(sqlx::AssertSqlSafe(format!(
@@ -225,72 +175,9 @@ async fn connect_and_migrate_should_apply_all_migrations_once_and_reopen_cleanly
)))
.execute(&admin)
.await
.expect("drop migration test database");
.expect("drop connection test database");
admin.close().await;
assert_eq!(
first_tables,
[
"_sqlx_migrations",
"account_group_accounts",
"account_groups",
"admin_audit_events",
"admin_users",
"backup_records",
"backup_settings",
"client_api_key_budget_windows",
"client_api_key_charge_events",
"client_api_key_groups",
"client_api_keys",
"model_request_leases",
"model_request_observations",
"model_requests",
"ops_events",
"outbound_proxies",
"plugin_account_group_resources",
"plugin_artifact_credentials",
"plugin_artifact_platforms",
"plugin_artifacts",
"plugin_client_api_key_resources",
"plugin_instance_secrets",
"plugin_instances",
"plugin_source_credentials",
"plugin_state_generations",
"plugin_state_records",
"plugin_update_sources",
"plugin_version_configurations",
"provider_accounts",
"provider_authorization_receipts",
"request_analytics_15m",
"request_analytics_1d_counts",
"request_analytics_1h",
"request_analytics_build_state",
"request_analytics_facts",
"request_analytics_pending_buckets",
"request_analytics_pending_requests",
"runtime_settings",
]
);
assert_eq!(session_settings, ("codex-proxy-rs".to_owned(), 30, 5, 30));
assert_eq!(
migration_count,
i64::try_from(TEST_MIGRATOR.iter().count())
.expect("migration count fits PostgreSQL bigint")
);
assert_eq!(response_id_types, ["bytea", "bytea"]);
assert!(!raw_response_id_index_exists);
assert!(!legacy_key_provider_column_exists);
assert!(routing_history_columns.is_empty());
}
#[test]
fn migrations_should_leave_transaction_ownership_to_sqlx() {
let transaction_statements = TEST_MIGRATOR
.iter()
.flat_map(|migration| migration.sql.as_str().lines())
.map(str::trim)
.filter(|line| matches!(*line, "begin;" | "commit;"))
.count();
assert_eq!(transaction_statements, 0);
assert_eq!(refresh_margin, 1800, "reopening preserves custom settings");
}
@@ -2168,10 +2168,6 @@ async fn shared_core_usage_and_timings_preserve_fields_and_reject_invalid_phases
upstream_response_ms: Some(1_000),
upstream_api_overhead_ms: Some(120.25),
upstream_engine_ms: Some(6400.0),
upstream_engine_iapi_ttft_ms: Some(650.5),
upstream_engine_service_ttft_ms: Some(720.25),
upstream_engine_iapi_tbt_ms: Some(18.45),
upstream_engine_service_tbt_ms: Some(20.12),
latency_ms: Some(9),
};
ExecutionStore::finalize_model_request(&store, finalization)
@@ -2202,22 +2198,6 @@ async fn shared_core_usage_and_timings_preserve_fields_and_reject_invalid_phases
}
assert_eq!(persisted["upstream_api_overhead_ms"].as_f64(), Some(120.25));
assert_eq!(persisted["upstream_engine_ms"].as_f64(), Some(6400.0));
assert_eq!(
persisted["upstream_engine_iapi_ttft_ms"].as_f64(),
Some(650.5)
);
assert_eq!(
persisted["upstream_engine_service_ttft_ms"].as_f64(),
Some(720.25)
);
assert_eq!(
persisted["upstream_engine_iapi_tbt_ms"].as_f64(),
Some(18.45)
);
assert_eq!(
persisted["upstream_engine_service_tbt_ms"].as_f64(),
Some(20.12)
);
let invalid_id = "req_invalid_phase";
seed_running_request(&database.pool, invalid_id)
.await
@@ -2233,7 +2213,7 @@ async fn shared_core_usage_and_timings_preserve_fields_and_reject_invalid_phases
assert_eq!(stored_row(&database.pool, invalid_id).await, before);
for invalid_ms in [-1.0, f64::NAN, f64::INFINITY, i64::MAX as f64] {
let mut invalid = successful_core_finalization(invalid_id);
invalid.timings.upstream_engine_iapi_tbt_ms = Some(invalid_ms);
invalid.timings.upstream_engine_ms = Some(invalid_ms);
let error = ExecutionStore::finalize_model_request(&store, invalid)
.await
.expect_err("invalid official timing must not silently become missing");
@@ -18,7 +18,6 @@ mod admission_recovery;
mod analytics;
mod analytics_buckets;
mod analytics_reset;
mod analytics_upgrade;
mod backup;
mod client_budgets;
mod client_keys;
@@ -69,10 +68,6 @@ pub(super) fn admin_account_store(pool: &PgPool) -> PgAdminAccountStore {
impl TestDatabase {
pub(super) async fn create(label: &str) -> Option<Self> {
Self::create_through(label, i64::MAX).await
}
pub(super) async fn create_through(label: &str, migration_version: i64) -> Option<Self> {
let database_url = crate::support::test_env("CPR_TEST_DATABASE_URL")?;
let schema = format!("cpr_store_{label}_{}", Uuid::new_v4().simple());
// 临时 schema 验证事务可见性与回滚,不模拟 PostgreSQL 掉电恢复
@@ -95,7 +90,7 @@ impl TestDatabase {
.await
.expect("connect isolated test schema");
TEST_MIGRATOR
.run_to(migration_version, &pool)
.run(&pool)
.await
.expect("apply test migrations");
Some(Self {
@@ -110,7 +110,7 @@ async fn output_throughput_uses_official_response_duration_without_local_fallbac
'firstTokenMs', $2::bigint,
'latencyMs', $3::bigint),
'upstream', coalesce(request_observation_json #> '{timings,upstream}', '{}'::jsonb) || jsonb_build_object(
'responseMs', $4::bigint, 'engineIapiTbtMs', 2.450638))))
'responseMs', $4::bigint, 'engineMs', 2.450638))))
where id = 'req_observe_success'",
)
.bind(output)
@@ -150,8 +150,6 @@ async fn output_throughput_uses_official_response_duration_without_local_fallbac
detail.request.upstream_response_ms,
record.upstream_response_ms
);
assert_eq!(record.upstream_engine_iapi_tbt_ms, Some(2.450638));
assert_eq!(detail.request.upstream_engine_iapi_tbt_ms, Some(2.450638));
super::drain_analytics(&database.pool).await;
let summary = repository
@@ -1,58 +1,10 @@
//! 观测视图升级兼容性与普通聚合的覆盖索引规划回归测试
//! 验证观测视图普通聚合能够使用覆盖索引
use chrono::Utc;
use serde_json::Value;
use super::{TestDatabase, seed_observability_facts};
#[tokio::test]
async fn projection_upgrade_preserves_observation_values() {
let Some(db) = TestDatabase::create_through("observation_projection_upgrade", 26).await else {
return;
};
// 此测试固定覆盖 26 → 27 的冻结视图合同,不依赖最新版本的写入夹具
sqlx::query("insert into model_requests(id,client_api_key_ref,operation,client_transport,outcome,client_status_code,started_at,deadline_at,completed_at,request_observation_json) values('req_observe_success','historical_key','responses','http_sse','succeeded',200,now()-interval '1 hour',now(),now(),'{\"request\":{\"configRevision\":1,\"protocol\":\"openai\",\"endpoint\":\"/v1/responses\",\"compact\":false},\"routing\":{\"scope\":\"all\",\"groupRefs\":[],\"groupNamesSnapshot\":[]},\"timings\":{\"local\":{\"latencyMs\":800}}}'::jsonb)")
.execute(&db.pool).await.unwrap();
sqlx::query(
"update model_requests
set request_observation_json = jsonb_set(request_observation_json,
'{timings,upstream}', '{\"responseMs\":700,\"engineIapiTbtMs\":2.450638}')
where id = 'req_observe_success'",
)
.execute(&db.pool)
.await
.unwrap();
let before: Vec<Value> =
sqlx::query_scalar("select to_jsonb(mr) from model_request_observations mr order by id")
.fetch_all(&db.pool)
.await
.unwrap();
super::super::TEST_MIGRATOR
.run_to(27, &db.pool)
.await
.unwrap();
let after: Vec<Value> =
sqlx::query_scalar("select to_jsonb(mr) from model_request_observations mr order by id")
.fetch_all(&db.pool)
.await
.unwrap();
assert_eq!(
after, before,
"projection upgrade must preserve every field"
);
let updatable: String = sqlx::query_scalar(
"select is_updatable from information_schema.views
where table_schema = current_schema() and table_name = 'model_request_observations'",
)
.fetch_one(&db.pool)
.await
.unwrap();
assert_eq!(updatable, "NO", "observation projection remains read-only");
db.close().await;
}
#[tokio::test]
async fn plain_observation_aggregates_can_use_a_covering_index() {
let Some(db) = TestDatabase::create("observation_covering_index").await else {
@@ -1,4 +1,4 @@
//! 验证插件制品身份、信任接受、迁移与安装幂等性
//! 验证插件制品身份、信任接受与安装幂等性
use std::collections::BTreeMap;
@@ -83,12 +83,7 @@ async fn unknown_metadata_fields_do_not_change_artifact_identity() {
.fetch_one(&database.pool)
.await
.unwrap();
assert!(stored.get("permissionDescriptions").is_none());
assert!(stored.get("requestedPermissions").is_none());
let mut extended = stored;
extended["permissionDescriptions"] =
serde_json::json!([{"permission": "models", "label": "旧文案"}]);
extended["requestedPermissions"] = serde_json::json!(["models"]);
extended["futureMetadata"] = serde_json::json!({"enabled": true});
extended["contributes"]["middleware"]["futureField"] = serde_json::json!(true);
sqlx::query("update plugin_artifacts set metadata_json=$2 where sha256=$1")
@@ -136,94 +131,6 @@ async fn unknown_metadata_fields_do_not_change_artifact_identity() {
database.close().await;
}
#[tokio::test]
async fn trust_migration_removes_only_obsolete_metadata_fields() {
let Some(database) = TestDatabase::create_through("plugin_trust_upgrade", 19).await else {
return;
};
initialize_revision(&database).await;
let store = PgPluginStore::new(database.pool.clone());
for (digest, extra) in [
('a', serde_json::json!({"requestedPermissions": ["models"]})),
(
'b',
serde_json::json!({"permissionDescriptions": ["旧文案"]}),
),
(
'c',
serde_json::json!({"requestedPermissions": [], "permissionDescriptions": []}),
),
('d', serde_json::json!({})),
] {
// 每个制品使用独立平台,保留同一版本的平台不可变约束
let fixture = artifact(digest, &[&format!("test-{digest}")]);
let installed = store
.install_artifact(fixture.clone(), PluginSource::Upload, &context())
.await
.unwrap();
if digest == 'a' {
store
.accept_artifact(&fixture.metadata.sha256, &context())
.await
.unwrap();
}
sqlx::query("update plugin_artifacts set metadata_json=metadata_json || $2 || '{\"futureMetadata\":true}'::jsonb where sha256=$1")
.bind(&fixture.metadata.sha256).bind(extra).execute(&database.pool).await.unwrap();
sqlx::query("insert into plugin_instances(id,artifact_sha256,name,enabled,configuration_json,bindings_json,revision) values ($1,$2,'Example',false,'{}','[]',$3)")
.bind(uuid::Uuid::now_v7()).bind(&fixture.metadata.sha256)
.bind(i64::try_from(installed.config_revision.get()).unwrap())
.execute(&database.pool).await.unwrap();
}
let mut expected: Vec<serde_json::Value> =
sqlx::query_scalar("select to_jsonb(a) from plugin_artifacts a order by sha256")
.fetch_all(&database.pool)
.await
.unwrap();
for row in &mut expected {
let metadata = row["metadata_json"].as_object_mut().unwrap();
metadata.remove("requestedPermissions");
metadata.remove("permissionDescriptions");
}
let instances: Vec<serde_json::Value> =
sqlx::query_scalar("select to_jsonb(i) from plugin_instances i order by id")
.fetch_all(&database.pool)
.await
.unwrap();
let revision: i64 =
sqlx::query_scalar("select config_revision from runtime_settings where id=1")
.fetch_one(&database.pool)
.await
.unwrap();
for _ in 0..2 {
super::super::TEST_MIGRATOR
.run(&database.pool)
.await
.unwrap();
let actual: Vec<serde_json::Value> =
sqlx::query_scalar("select to_jsonb(a) from plugin_artifacts a order by sha256")
.fetch_all(&database.pool)
.await
.unwrap();
assert_eq!(
actual, expected,
"只清理两个旧字段,保留包体、接受事实与其他元数据"
);
let current_instances: Vec<serde_json::Value> =
sqlx::query_scalar("select to_jsonb(i) from plugin_instances i order by id")
.fetch_all(&database.pool)
.await
.unwrap();
assert_eq!(current_instances, instances);
let snapshot = store.load_instances().await.unwrap();
assert_eq!(snapshot.instances.len(), 4);
assert_eq!(
snapshot.config_revision.get(),
u64::try_from(revision).unwrap()
);
}
database.close().await;
}
#[tokio::test]
async fn artifact_acceptance_is_exact_and_idempotent() {
let Some(database) = TestDatabase::create("plugin_artifact_acceptance").await else {
@@ -262,51 +169,6 @@ async fn artifact_acceptance_is_exact_and_idempotent() {
database.close().await;
}
#[tokio::test]
async fn plugin_migration_preserves_main_settings_and_accepts_new_installations() {
let Some(database) = TestDatabase::create_through("plugin_upgrade_from_main", 17).await else {
return;
};
initialize_revision(&database).await;
// 0022 会把仍为旧默认 3600 的 margin 收敛到 300;预设自定义值,
// 让本测试继续验证「所有已自定义设置都原样保留」。
sqlx::query("update runtime_settings set max_concurrent_per_account=0, refresh_margin_seconds=1800 where id=1")
.execute(&database.pool)
.await
.unwrap();
let before: serde_json::Value =
sqlx::query_scalar("select to_jsonb(s) from runtime_settings s where id=1")
.fetch_one(&database.pool)
.await
.unwrap();
super::super::TEST_MIGRATOR
.run(&database.pool)
.await
.unwrap();
let after: serde_json::Value =
sqlx::query_scalar("select to_jsonb(s) from runtime_settings s where id=1")
.fetch_one(&database.pool)
.await
.unwrap();
for (key, value) in before.as_object().unwrap() {
if key == "request_interval_ms" {
assert!(after.get(key).is_none(), "removed setting {key}");
continue;
}
assert_eq!(after.get(key), Some(value), "existing setting {key}");
}
let store = PgPluginStore::new(database.pool.clone());
store
.install_artifact(
artifact('a', &["linux-x86_64"]),
PluginSource::Upload,
&context(),
)
.await
.unwrap();
database.close().await;
}
#[tokio::test]
async fn source_identity_requires_publisher_and_name() {
let Some(database) = TestDatabase::create("plugin_source_identity").await else {
@@ -1193,63 +1193,6 @@ async fn legacy_urls_join_one_catalog_entry_and_invalid_batch_rolls_back() {
database.close().await;
}
#[tokio::test]
async fn migration_backfills_shared_proxies_without_changing_credentials() {
let Some(database) = TestDatabase::create_through("proxy_backfill", 4).await else {
return;
};
// 从真实旧 schema 正向升级,不拆卸最新结构,避免遗漏后续增加的引用约束
for id in ["acct_one", "acct_two", "acct_direct"] {
let proxy_url = (id != "acct_direct").then_some("http://user:secret@127.0.0.1:8080/");
sqlx::query(
"insert into provider_accounts (
id,provider_kind,name,authentication_kind,provider_credentials_json,
has_refresh_token,credential_observed_at,created_at,updated_at,outbound_proxy_url
) values ($1,'openai',$1,'oauth','{}',false,now(),now(),now(),$2)",
)
.bind(id)
.bind(proxy_url)
.execute(&database.pool)
.await
.unwrap();
}
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
assert!(
sqlx::query_scalar::<_, bool>("select bool_and(enabled) from outbound_proxies")
.fetch_one(&database.pool)
.await
.unwrap()
);
let accounts = PgProviderAccountRepository::new(database.pool.clone());
let inherited = gateway_core::account::ProviderAccountStore::list_accounts(&accounts)
.await
.unwrap();
assert_eq!(inherited.len(), 3);
assert!(
inherited
.iter()
.all(|account| account.request_location().is_none())
);
assert_eq!(
sqlx::query_scalar::<_, i64>("select count(*) from outbound_proxies")
.fetch_one(&database.pool)
.await
.unwrap(),
1
);
assert_eq!(sqlx::query_scalar::<_, i64>("select count(*) from provider_accounts a join outbound_proxies p on a.outbound_proxy_id = p.id where a.outbound_proxy_url = p.proxy_url and a.credential_revision = 1").fetch_one(&database.pool).await.unwrap(), 2);
assert!(
sqlx::query_scalar::<_, Option<String>>(
"select outbound_proxy_id from provider_accounts where id = 'acct_direct'"
)
.fetch_one(&database.pool)
.await
.unwrap()
.is_none()
);
database.close().await;
}
#[tokio::test]
async fn automatic_proxy_location_projects_to_accounts_and_guards_refresh_results() {
use gateway_core::account::{ProviderAccountStore, RequestLocation};
@@ -1,14 +1,18 @@
//! 验证短保留期升级、分批清理边界与统计异步收敛期间的查询可用性
use std::num::NonZeroU32;
use std::{num::NonZeroU32, time::Duration};
use chrono::{Duration as ChronoDuration, Utc};
use gateway_admin::{
model::retention::{RetentionPolicy, RetentionTarget},
ports::retention::RetentionStore as _,
ports::{retention::RetentionStore as _, store::ObservabilityStore as _},
};
use gateway_store::postgres::{
ObservabilityRange, PgRequestAnalyticsRepository, PgRetentionRepository,
use gateway_store::{
RequestAnalyticsCycle,
postgres::{
ObservabilityQueryBudget, ObservabilityRange, PgRequestAnalyticsRepository,
PgRetentionRepository,
},
};
use super::{
@@ -18,10 +22,7 @@ use super::{
#[tokio::test]
async fn short_retention_keeps_statistics_readable_until_each_deletion_batch_is_projected() {
use gateway_admin::{
model::observability::{AnalyticsState, DashboardQuery, TimeRange},
ports::store::ObservabilityStore as _,
};
use gateway_admin::model::observability::{AnalyticsState, DashboardQuery, TimeRange};
let Some(database) = TestDatabase::create("short_retention_readable").await else {
return;
@@ -280,3 +281,271 @@ async fn empty_request_cleanup_does_not_conflict_with_analytics_publication() {
publication.commit().await.unwrap();
database.close().await;
}
#[tokio::test]
async fn scattered_retention_deletions_commit_atomically_and_allow_new_requests_to_advance() {
let Some(database) = TestDatabase::create("retention_scattered_groups").await else {
return;
};
let now = Utc::now();
for number in 0..600 {
seed(
&database.pool,
&format!("expired_{number:04}"),
now - ChronoDuration::days(30) + ChronoDuration::minutes(number * 20),
&[],
)
.await;
}
drain_analytics(&database.pool).await;
let worker = PgRequestAnalyticsRepository::new(
database.pool.clone(),
ObservabilityQueryBudget::try_new(2, Duration::from_secs(5), Duration::from_secs(120))
.unwrap(),
);
let before = worker.status().await.unwrap();
let retention = PgRetentionRepository::new(database.pool.clone());
assert_eq!(
retention
.purge_batch(
RetentionTarget::ModelRequests,
now,
RetentionPolicy::try_new(10, 30, 90).unwrap(),
NonZeroU32::new(1000).unwrap(),
)
.await
.unwrap(),
600
);
seed(
&database.pool,
"new_request",
now - ChronoDuration::minutes(2),
&[],
)
.await;
// 小时层失败时,已经执行的多个刻钟删除批次与事实删除也必须一起回滚
sqlx::raw_sql("create function reject_hour_deletion() returns trigger language plpgsql as $$begin raise exception 'test hour deletion failure'; end$$;
create trigger reject_hour_deletion before delete on request_analytics_1h for each statement execute function reject_hour_deletion()")
.execute(&database.pool).await.unwrap();
let cycle = RequestAnalyticsCycle::default();
assert!(cycle.run(&worker).await.is_err());
let failed = worker.status().await.unwrap();
assert_eq!(failed.generation, before.generation);
assert_eq!(failed.pending_requests, 601);
let statistics = observability_repository(&database.pool);
let range = ObservabilityRange::new(now - ChronoDuration::days(31), now).unwrap();
assert_eq!(
statistics
.usage_summary(range, Default::default())
.await
.unwrap()
.requests
.request_count,
600
);
sqlx::raw_sql("drop trigger reject_hour_deletion on request_analytics_1h; drop function reject_hour_deletion();
create function slow_summary_statement() returns trigger language plpgsql as $$begin perform pg_sleep(0.002); return null; end$$;
create trigger slow_quarter_delete before delete on request_analytics_15m for each statement execute function slow_summary_statement();
create trigger slow_hour_delete before delete on request_analytics_1h for each statement execute function slow_summary_statement()")
.execute(&database.pool).await.unwrap();
// 按语句注入延迟,覆盖大量分组的 SQL 往返成本及多个集合删除批次
cycle
.run(&worker)
.await
.expect("retention backlog must advance within the cycle budget");
let after = worker.status().await.unwrap();
assert_eq!(after.pending_requests, 0);
assert_eq!(after.pending_buckets, 0);
assert!(after.generation > before.generation);
let actual = statistics
.usage_summary(range, Default::default())
.await
.unwrap();
assert_eq!(actual.requests.request_count, 1);
assert_eq!(actual.requests.total_tokens, 120);
let timeline = admin_observability_store(&database.pool)
.dashboard_summary(
gateway_admin::model::observability::DashboardQuery::new(
gateway_admin::model::observability::TimeRange {
start: now - ChronoDuration::days(1),
end: now,
},
Default::default(),
)
.unwrap(),
now,
)
.await
.unwrap();
assert_eq!(
timeline
.trend
.iter()
.map(|point| point.metrics.request_count)
.sum::<u64>(),
1
);
let remaining: (i64, i64, i64, i64) = sqlx::query_as(
"select
(select count(*) from request_analytics_facts where id like 'expired_%'),
(select count(*) from request_analytics_15m where bucket_start < $1),
(select count(*) from request_analytics_1h where bucket_start < $1),
(select count(*) from request_analytics_1d_counts where bucket_start < $1)",
)
.bind(now - ChronoDuration::days(10))
.fetch_one(&database.pool)
.await
.unwrap();
assert_eq!(remaining, (0, 0, 0, 0));
database.close().await;
}
#[tokio::test]
async fn fresh_requests_and_retention_deletions_both_advance_within_each_batch() {
let Some(database) = TestDatabase::create("retention_fair_projection").await else {
return;
};
let now = Utc::now();
for number in 0..20 {
seed(
&database.pool,
&format!("expired_{number:04}"),
now - ChronoDuration::days(30),
&[],
)
.await;
}
drain_analytics(&database.pool).await;
PgRetentionRepository::new(database.pool.clone())
.purge_batch(
RetentionTarget::ModelRequests,
now,
RetentionPolicy::try_new(10, 30, 90).unwrap(),
NonZeroU32::new(1000).unwrap(),
)
.await
.unwrap();
let worker =
PgRequestAnalyticsRepository::new(database.pool.clone(), observability_query_budget());
for wave in 1..=3 {
for number in 0..2 {
seed(
&database.pool,
&format!("new_{wave}_{number}"),
now - ChronoDuration::minutes(2),
&[],
)
.await;
}
assert_eq!(worker.maintain(4).await.unwrap().projected_requests, 4);
let facts: (i64, i64) = sqlx::query_as(
"select count(*) filter (where id like 'expired_%'), count(*) filter (where id like 'new_%') from request_analytics_facts",
)
.fetch_one(&database.pool)
.await
.unwrap();
// 历史队列尚未排空时,每批都须撤销旧贡献并发布本轮的新请求
assert_eq!(facts, (20 - wave * 2, wave * 2));
let current = observability_repository(&database.pool)
.usage_summary(
ObservabilityRange::new(now - ChronoDuration::days(1), now).unwrap(),
Default::default(),
)
.await
.unwrap();
assert_eq!(current.requests.request_count, wave as u64 * 2);
}
drain_analytics(&database.pool).await;
let final_summary = observability_repository(&database.pool)
.usage_summary(
ObservabilityRange::new(now - ChronoDuration::days(31), now).unwrap(),
Default::default(),
)
.await
.unwrap();
assert_eq!(final_summary.requests.request_count, 6);
assert_eq!(final_summary.requests.total_tokens, 720);
database.close().await;
}
#[tokio::test]
async fn timed_out_retention_batches_shrink_across_cycles_and_lock_contention() {
let Some(database) = TestDatabase::create("retention_batch_timeout").await else {
return;
};
let now = Utc::now();
for number in 0..600 {
seed(
&database.pool,
&format!("expired_{number:04}"),
now - ChronoDuration::days(30),
&[],
)
.await;
}
drain_analytics(&database.pool).await;
sqlx::query("delete from model_requests")
.execute(&database.pool)
.await
.unwrap();
seed(
&database.pool,
"new_request",
now - ChronoDuration::minutes(2),
&[],
)
.await;
// 数据库延迟与实际批量相关,小批量无需等待,以可观察进展验证跨轮退让
sqlx::raw_sql("create function slow_large_fact_deletion() returns trigger language plpgsql as $$begin
if (select count(*) from removed_facts) > 250 then perform pg_sleep(2); end if;
return null; end$$;
create trigger slow_large_fact_deletion after delete on request_analytics_facts referencing old table as removed_facts for each statement execute function slow_large_fact_deletion()")
.execute(&database.pool).await.unwrap();
let worker = PgRequestAnalyticsRepository::new(
database.pool.clone(),
ObservabilityQueryBudget::try_new(2, Duration::from_secs(1), Duration::from_secs(1))
.unwrap(),
);
let cycle = RequestAnalyticsCycle::default();
for _ in 0..2 {
assert!(cycle.run(&worker).await.is_err());
// 空轮可能来自别的实例持锁,不能因此恢复会反复超时的大批量
let mut lock = database.pool.begin().await.unwrap();
sqlx::query(
"select pg_advisory_xact_lock(hashtextextended(current_schema()::text,382032))",
)
.execute(&mut *lock)
.await
.unwrap();
let idle = cycle.run(&worker).await.unwrap();
assert_eq!(idle.projected_requests, 0);
lock.rollback().await.unwrap();
}
let pending = worker.status().await.unwrap().pending_requests;
assert_eq!(pending, 601);
// 此轮可以在后续批次超时前已有提交,下一轮重放仍须保持计数正确
let _ = cycle.run(&worker).await;
assert!(worker.status().await.unwrap().pending_requests < pending);
for _ in 0..5 {
// 与调度器错误重试一样留出退避时间,让已取消语句完成回滚并释放维护锁
tokio::time::sleep(Duration::from_secs(1)).await;
let _ = cycle.run(&worker).await;
if worker.status().await.unwrap().pending_requests == 0 {
break;
}
}
assert_eq!(worker.status().await.unwrap().pending_requests, 0);
let actual = observability_repository(&database.pool)
.usage_summary(
ObservabilityRange::new(now - ChronoDuration::days(31), now).unwrap(),
Default::default(),
)
.await
.unwrap();
assert_eq!(actual.requests.request_count, 1);
assert_eq!(actual.requests.total_tokens, 120);
database.close().await;
}
@@ -1,4 +1,4 @@
//! 验证运行设置的数据库约束、升级保留与快照发布
//! 验证运行设置的读写、数据库约束与快照发布
use std::collections::BTreeMap;
@@ -61,56 +61,12 @@ fn runtime_settings_keep_account_rotation_global() {
}
#[tokio::test]
async fn removing_request_interval_preserves_settings_and_snapshot() {
use gateway_store::postgres::{PgRuntimeSnapshotRepository, RuntimeSnapshotRepository};
let Some(database) = TestDatabase::create_through("drop_interval", 29).await else {
return;
};
sqlx::query("update runtime_settings set request_interval_ms = 60000, max_concurrent_per_account = 7, refresh_concurrency = 12, refresh_margin_seconds = 1234, config_revision = 70 where id = 1")
.execute(&database.pool).await.unwrap();
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
let column_exists: bool = sqlx::query_scalar("select exists(select 1 from information_schema.columns where table_schema = current_schema() and table_name = 'runtime_settings' and column_name = 'request_interval_ms')")
.fetch_one(&database.pool).await.unwrap();
assert!(!column_exists);
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
let settings = repository.load_runtime_settings().await.unwrap();
assert_eq!(settings.values.max_concurrent_per_account, 7);
assert_eq!(settings.values.refresh_concurrency, 12);
assert_eq!(settings.values.refresh_margin_seconds, 1234);
let snapshot = PgRuntimeSnapshotRepository::new(database.pool.clone())
.load_runtime_snapshot()
.await
.unwrap();
assert!(
serde_json::to_value(snapshot.settings)
.unwrap()
.get("request_interval_ms")
.is_none()
);
repository
.update_runtime_settings(settings_with_margin(600))
.await
.unwrap();
assert_eq!(
repository
.load_runtime_settings()
.await
.unwrap()
.values
.refresh_margin_seconds,
600
);
database.close().await;
}
#[tokio::test]
async fn privacy_policy_upgrade_defaults_off_and_round_trips_in_snapshot() {
async fn privacy_policy_defaults_off_and_round_trips_in_snapshot() {
use gateway_core::settings::privacy::*;
use gateway_store::postgres::{PgRuntimeSnapshotRepository, RuntimeSnapshotRepository};
let Some(database) = TestDatabase::create_through("privacy_policy", 25).await else {
let Some(database) = TestDatabase::create("privacy_policy").await else {
return;
};
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
let before = repository.load_runtime_settings().await.unwrap();
assert_eq!(
@@ -151,167 +107,18 @@ async fn privacy_policy_upgrade_defaults_off_and_round_trips_in_snapshot() {
serde_json::to_value(snapshot.settings).unwrap()["codex_privacy_policy"],
serde_json::to_value(expected).unwrap()
);
assert_eq!(
before.values.codex_privacy_policy,
CodexPrivacyPolicy::default()
);
database.close().await;
}
#[tokio::test]
async fn privacy_steps_migration_preserves_rules_and_other_settings() {
let Some(database) = TestDatabase::create_through("privacy_steps", 30).await else {
return;
};
let legacy = serde_json::json!({
"enabled":true, "onError":"skip_rule", "rules":[
{"id":"first","name":"旧规则一","enabled":true,"scope":"request_header","selector":"x-private",
"action":"remove_field","pattern":null,"replacement":"","value":null,
"replaceAll":true,"caseInsensitive":false,"multiLine":false},
{"id":"second","name":"旧规则二","enabled":false,"scope":"request_body","selector":"$.owner",
"action":"regex_replace","pattern":"alice","replacement":"user","value":null,
"replaceAll":false,"caseInsensitive":true,"multiLine":true}
]
});
let mut expected = serde_json::json!({
"enabled":true, "onError":"skip_rule", "rules":[
{"id":"first","name":"旧规则一","enabled":true,"steps":[
{"scope":"request_header","selector":"x-private","action":"remove_field",
"pattern":null,"replacement":"","value":null,
"replaceAll":true,"caseInsensitive":false,"multiLine":false}]},
{"id":"second","name":"旧规则二","enabled":false,"steps":[
{"scope":"request_body","selector":"$.owner","action":"regex_replace",
"pattern":"alice","replacement":"user","value":null,
"replaceAll":false,"caseInsensitive":true,"multiLine":true}]}
]
});
// 同时包含聚合规则,迁移不能嵌套包装或拆散现有步骤
let canonical = serde_json::json!({
"id":"aggregate","name":"通用聚合规则","enabled":true,"steps":[
{"scope":"request_body","selector":"$.metadata","action":"set_value",
"pattern":null,"replacement":"","value":{"owner":"user","items":[1,2]},
"replaceAll":true,"caseInsensitive":false,"multiLine":false},
{"scope":"request_header","selector":"x-private","action":"remove_field",
"pattern":null,"replacement":"","value":null,
"replaceAll":true,"caseInsensitive":false,"multiLine":false}
]
});
let mut mixed = legacy;
mixed["rules"]
.as_array_mut()
.unwrap()
.insert(1, canonical.clone());
expected["rules"]
.as_array_mut()
.unwrap()
.insert(1, canonical);
sqlx::query("update runtime_settings set codex_privacy_policy_json = $1, config_revision = 70, refresh_margin_seconds = 1234 where id = 1")
.bind(&mixed).execute(&database.pool).await.unwrap();
let mut expected_row: serde_json::Value =
sqlx::query_scalar("select to_jsonb(runtime_settings) from runtime_settings where id = 1")
.fetch_one(&database.pool)
.await
.unwrap();
expected_row["codex_privacy_policy_json"] = expected.clone();
expected_row["reset_credit_lead_minutes"] = serde_json::json!(10);
expected_row["reset_credit_max_weekly_remaining_percent"] = serde_json::Value::Null;
// 完整升级还会清理自动回切字段,其余运行设置必须原样保留
expected_row["smart_scheduling_json"]
.as_object_mut()
.unwrap()
.remove("preferHigherWeight");
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
let stored: serde_json::Value =
sqlx::query_scalar("select to_jsonb(runtime_settings) from runtime_settings where id = 1")
.fetch_one(&database.pool)
.await
.unwrap();
assert_eq!(stored, expected_row);
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
let loaded = repository.load_runtime_settings().await.unwrap();
assert_eq!(
serde_json::to_value(&loaded.values.codex_privacy_policy).unwrap(),
expected
);
// 服务重启后再次检查迁移,不产生写入或额外嵌套
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
let stored: serde_json::Value =
sqlx::query_scalar("select to_jsonb(runtime_settings) from runtime_settings where id = 1")
.fetch_one(&database.pool)
.await
.unwrap();
assert_eq!(stored, expected_row);
let snapshot = gateway_store::postgres::RuntimeSnapshotRepository::load_runtime_snapshot(
&gateway_store::postgres::PgRuntimeSnapshotRepository::new(database.pool.clone()),
)
.await
.unwrap();
assert_eq!(
serde_json::to_value(snapshot.settings).unwrap()["codex_privacy_policy"],
expected
);
database.close().await;
}
#[tokio::test]
async fn privacy_steps_migration_rejects_non_object_without_losing_data() {
let Some(database) = TestDatabase::create_through("privacy_invalid", 30).await else {
return;
};
let invalid = serde_json::json!({
"enabled":false,"onError":"reject_request","rules":[{},42]
});
sqlx::query("update runtime_settings set codex_privacy_policy_json = $1 where id = 1")
.bind(&invalid)
.execute(&database.pool)
.await
.unwrap();
assert!(super::TEST_MIGRATOR.run(&database.pool).await.is_err());
let stored: serde_json::Value =
sqlx::query_scalar("select codex_privacy_policy_json from runtime_settings where id = 1")
.fetch_one(&database.pool)
.await
.unwrap();
assert_eq!(stored, invalid);
let applied: bool =
sqlx::query_scalar("select exists(select 1 from _sqlx_migrations where version = 31)")
.fetch_one(&database.pool)
.await
.unwrap();
assert!(!applied);
sqlx::query("update runtime_settings set codex_privacy_policy_json = jsonb_set(codex_privacy_policy_json, '{rules}', '[]') where id = 1")
.execute(&database.pool).await.unwrap();
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
let loaded = PgRuntimeSettingsRepository::new(database.pool.clone())
.load_runtime_settings()
.await
.unwrap();
assert!(!loaded.values.codex_privacy_policy.enabled);
assert!(loaded.values.codex_privacy_policy.rules.is_empty());
assert_eq!(
loaded.values.codex_privacy_policy.on_error,
gateway_core::settings::privacy::PrivacyFailureMode::RejectRequest
);
database.close().await;
}
#[tokio::test]
async fn smart_settings_upgrade_preserves_selection_and_publishes_custom_config() {
async fn smart_settings_round_trip_publish_and_restore_database_defaults() {
use gateway_core::account::SmartSchedulingConfig;
use gateway_store::postgres::{PgRuntimeSnapshotRepository, RuntimeSnapshotRepository};
let Some(database) = TestDatabase::create_through("smart_config", 18).await else {
let Some(database) = TestDatabase::create("smart_config").await else {
return;
};
// 升级前不能用包含新列的 Repository,直接写入旧版本已有字段
sqlx::query("update runtime_settings set rotation_strategy = 'sticky', refresh_margin_seconds = 3600 where id = 1")
.execute(&database.pool).await.unwrap();
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
let before = repository.load_runtime_settings().await.unwrap();
assert_eq!(before.rotation_strategy, "sticky");
// 0022 将仍为旧默认 3600 的行迁移到 300;管理员自定义值才会原样保留。
assert_eq!(before.values.refresh_margin_seconds, 300);
assert_eq!(
before.values.smart_scheduling,
SmartSchedulingConfig::default()
@@ -333,8 +140,17 @@ async fn smart_settings_upgrade_preserves_selection_and_publishes_custom_config(
serde_json::to_value(expected).unwrap()
);
assert!(snapshot.config_revision > before.config_revision);
sqlx::query("update runtime_settings set smart_scheduling_json = default where id = 1")
.execute(&database.pool)
.await
.unwrap();
assert_eq!(
before.values.smart_scheduling,
repository
.load_runtime_settings()
.await
.unwrap()
.values
.smart_scheduling,
SmartSchedulingConfig::default()
);
database.close().await;
@@ -434,52 +250,33 @@ async fn unlimited_default_account_concurrency_round_trips_without_relaxing_othe
}
#[tokio::test]
async fn migrations_should_narrow_the_default_refresh_margin_to_the_codex_baseline() {
async fn refresh_margin_defaults_apply_to_initial_settings_and_database_resets() {
let Some(database) = TestDatabase::create("refresh_margin_default").await else {
return;
};
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
// 新装实例:0001 种子行携带旧默认 3600,0022 数据迁移统一收敛到 300。
let settings = repository
.load_runtime_settings()
.await
.expect("load settings");
assert_eq!(settings.values.refresh_margin_seconds, 300);
// 列默认值同步收窄,重建行不会回退到旧值;目录查询限定本测试 schema,
// 同库并行测试里停留在旧迁移版本的 schema 仍保留 3600 旧默认,不能被读到。
let column_default: String = sqlx::query_scalar(
"select column_default from information_schema.columns \
where table_schema = current_schema() \
and table_name = 'runtime_settings' \
and column_name = 'refresh_margin_seconds'",
)
.fetch_one(&database.pool)
.await
.expect("read column default");
assert_eq!(column_default, "300");
database.close().await;
}
#[tokio::test]
async fn refresh_margin_migration_should_preserve_admin_customized_values() {
// 升级语义:0021 之前部署的实例经 0022 迁移后,仅旧默认 3600 收敛到 300,
// 管理员自定义值原样保留。
let Some(database) = TestDatabase::create_through("refresh_margin_custom", 21).await else {
return;
};
sqlx::query("update runtime_settings set refresh_margin_seconds = 1800 where id = 1")
.execute(&database.pool)
.await
.expect("persist customized margin");
super::TEST_MIGRATOR
.run(&database.pool)
.unwrap();
sqlx::query("update runtime_settings set refresh_margin_seconds = default where id = 1")
.execute(&database.pool)
.await
.expect("apply remaining migrations");
let settings = PgRuntimeSettingsRepository::new(database.pool.clone())
.load_runtime_settings()
.await
.expect("load settings");
assert_eq!(settings.values.refresh_margin_seconds, 1_800);
.expect("reset refresh margin to the database default");
assert_eq!(
repository
.load_runtime_settings()
.await
.unwrap()
.values
.refresh_margin_seconds,
300
);
database.close().await;
}
@@ -714,7 +511,9 @@ async fn concurrency_queue_settings_round_trip_into_the_runtime_snapshot() {
),
(0, 0, 30, 0)
);
let mut update = settings_with_margin(3600);
let mut update = settings_with_margin(1234);
update.values.max_concurrent_per_account = 7;
update.values.refresh_concurrency = 12;
update.values.max_waiting_per_key = 5;
update.values.max_waiting_per_account = 7;
update.values.concurrency_wait_timeout_seconds = 12;
@@ -723,12 +522,15 @@ async fn concurrency_queue_settings_round_trip_into_the_runtime_snapshot() {
let settings = repository.load_runtime_settings().await.unwrap();
assert_eq!(
(
settings.values.max_concurrent_per_account,
settings.values.refresh_concurrency,
settings.values.refresh_margin_seconds,
settings.values.max_waiting_per_key,
settings.values.max_waiting_per_account,
settings.values.concurrency_wait_timeout_seconds,
settings.values.openai_guardian_reserved_concurrency,
),
(5, 7, 12, 1)
(7, 12, 1234, 5, 7, 12, 1)
);
let snapshot = PgRuntimeSnapshotRepository::new(database.pool.clone())
.load_runtime_snapshot()
@@ -737,6 +539,9 @@ async fn concurrency_queue_settings_round_trip_into_the_runtime_snapshot() {
let snapshot_values = serde_json::to_value(&snapshot.settings).unwrap();
assert_eq!(
(
snapshot_values["max_concurrent_per_account"]
.as_u64()
.unwrap(),
snapshot_values["max_waiting_per_key"].as_u64().unwrap(),
snapshot_values["max_waiting_per_account"].as_u64().unwrap(),
snapshot_values["concurrency_wait_timeout_seconds"]
@@ -746,7 +551,7 @@ async fn concurrency_queue_settings_round_trip_into_the_runtime_snapshot() {
.as_u64()
.unwrap(),
),
(5, 7, 12, 1)
(7, 5, 7, 12, 1)
);
assert!(snapshot.config_revision > before.config_revision);
database.close().await;
@@ -1235,48 +1040,6 @@ async fn warmup_cursor_survives_restart_and_settings_updates_without_rewinding()
database.close().await;
}
#[tokio::test]
async fn runtime_scheduling_upgrade_preserves_settings_without_a_warmup_table() {
let Some(database) = TestDatabase::create_through("runtime_scheduling_upgrade", 20).await
else {
return;
};
sqlx::query(
"update runtime_settings set rotation_strategy = 'sticky',
account_warmup_enabled = true, account_warmup_schedule_time = '08:00,13:00',
account_warmup_model = 'test-model' where id = 1",
)
.execute(&database.pool)
.await
.unwrap();
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
let warmup_table: Option<String> =
sqlx::query_scalar("select to_regclass('account_warmup_slots')::text")
.fetch_one(&database.pool)
.await
.unwrap();
assert!(warmup_table.is_none());
let cursor: Option<DateTime<Utc>> =
sqlx::query_scalar("select account_warmup_cursor from runtime_settings where id = 1")
.fetch_one(&database.pool)
.await
.unwrap();
assert!(cursor.is_none());
let settings = PgRuntimeSettingsRepository::new(database.pool.clone())
.load_runtime_settings()
.await
.unwrap();
assert_eq!(settings.rotation_strategy, "sticky");
assert!(settings.values.account_warmup_enabled);
assert_eq!(settings.values.account_warmup_schedule_time, "08:00,13:00");
assert_eq!(
settings.values.account_warmup_model.as_deref(),
Some("test-model")
);
assert_eq!(settings.values.openai_guardian_reserved_concurrency, 0);
database.close().await;
}
#[tokio::test]
async fn warmup_cursor_resolves_dst_and_deduplicates_across_timezones() {
use gateway_core::{provider_ports::ProviderRuntimePolicyPort, time::DeploymentTimeZone};
@@ -1320,23 +1083,17 @@ async fn warmup_cursor_resolves_dst_and_deduplicates_across_timezones() {
}
#[tokio::test]
async fn account_affinity_upgrade_defaults_and_updates_reach_the_snapshot() {
async fn account_affinity_defaults_and_updates_reach_the_snapshot() {
use gateway_core::account::AccountAffinity;
use gateway_store::postgres::{PgRuntimeSnapshotRepository, RuntimeSnapshotRepository};
let Some(database) = TestDatabase::create_through("account_affinity_upgrade", 22).await else {
let Some(database) = TestDatabase::create("account_affinity").await else {
return;
};
sqlx::query("update runtime_settings set rotation_strategy = 'sticky', config_revision = config_revision + 1 where id = 1")
.execute(&database.pool)
.await
.unwrap();
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
let before = repository.load_runtime_settings().await.unwrap();
assert_eq!(before.rotation_strategy, "sticky");
assert_eq!(
before.values.openai_account_affinity,
AccountAffinity::Relaxed
AccountAffinity::Strict
);
assert_eq!(before.values.max_account_rotations, 3);
assert_eq!(before.values.openai_session_affinity_ttl_hours, 24);
@@ -1402,105 +1159,12 @@ async fn account_affinity_upgrade_defaults_and_updates_reach_the_snapshot() {
}
#[tokio::test]
async fn preferred_affinity_migration_defaults_to_strict_and_preserves_saved_modes() {
use gateway_core::account::AccountAffinity;
let Some(fresh) = TestDatabase::create("affinity_default").await else {
async fn reset_credit_weekly_threshold_defaults_unset_and_round_trips() {
let Some(database) = TestDatabase::create("reset_weekly_threshold").await else {
return;
};
let repository = PgRuntimeSettingsRepository::new(fresh.pool.clone());
assert_eq!(
repository
.load_runtime_settings()
.await
.unwrap()
.values
.openai_account_affinity,
AccountAffinity::Strict
);
fresh.close().await;
for mode in [AccountAffinity::Relaxed, AccountAffinity::Strict] {
let database = TestDatabase::create_through("affinity_preserve", 23)
.await
.unwrap();
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
// 升级前只写当时已有列,现行 Repository 依赖完整迁移后的 schema
sqlx::query("update runtime_settings set openai_account_affinity = $1, config_revision = config_revision + 1, updated_at = now() where id = 1")
.bind(match mode {
AccountAffinity::Relaxed => "relaxed",
_ => "strict",
})
.execute(&database.pool)
.await
.unwrap();
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
assert_eq!(
repository
.load_runtime_settings()
.await
.unwrap()
.values
.openai_account_affinity,
mode
);
database.close().await;
}
}
#[tokio::test]
async fn removing_switchback_preserves_custom_weights_and_replaces_database_default() {
let Some(database) = TestDatabase::create_through("remove_switchback", 31).await else {
return;
};
let expected =
gateway_core::account::SmartSchedulingConfig::new([0.0, 2.0, 1.0, 0.5, 1.2, 2.3]).unwrap();
let mut legacy = serde_json::to_value(expected).unwrap();
legacy["preferHigherWeight"] = serde_json::json!(true);
sqlx::query("update runtime_settings set smart_scheduling_json = $1 where id = 1")
.bind(legacy)
.execute(&database.pool)
.await
.unwrap();
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
assert_eq!(
repository
.load_runtime_settings()
.await
.unwrap()
.values
.smart_scheduling,
expected
);
sqlx::query("update runtime_settings set smart_scheduling_json = default where id = 1")
.execute(&database.pool)
.await
.unwrap();
assert_eq!(
repository
.load_runtime_settings()
.await
.unwrap()
.values
.smart_scheduling,
gateway_core::account::SmartSchedulingConfig::default()
);
database.close().await;
}
#[tokio::test]
async fn reset_credit_weekly_threshold_upgrade_defaults_unset_and_round_trips() {
let Some(database) = TestDatabase::create_through("reset_weekly_threshold", 34).await else {
return;
};
sqlx::query("update runtime_settings set reset_credit_lead_minutes = 20 where id = 1")
.execute(&database.pool)
.await
.unwrap();
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
let settings = repository.load_runtime_settings().await.unwrap();
assert_eq!(settings.values.reset_credit_lead_minutes, 20);
assert_eq!(
settings.values.reset_credit_max_weekly_remaining_percent,
None
@@ -2,59 +2,6 @@
use super::TestDatabase;
#[tokio::test]
async fn error_details_migration_preserves_existing_request_and_event_text() {
let Some(db) = TestDatabase::create_through("error_details_upgrade", 22).await else {
return;
};
seed_legacy_request(&db.pool).await;
let raw = "{ \"error\": {\"code\":\"Vendor.Unknown\",\"message\":\"原始错误\"} }";
sqlx::query("update model_requests set raw_upstream_error = $1 where id = 'req_integrity'")
.bind(raw)
.execute(&db.pool)
.await
.unwrap();
sqlx::query(
"insert into ops_events (id, level, component, operation, failure_kind, message, created_at, raw_upstream_error)
values ('error_details_upgrade', 'error', 'provider', 'generate', 'unavailable', 'safe message', now(), $1)",
).bind(raw).execute(&db.pool).await.unwrap();
super::TEST_MIGRATOR.run(&db.pool).await.unwrap();
let details: Vec<String> = sqlx::query_scalar(
"select error_details from model_requests where id = 'req_integrity'
union all select error_details from ops_events where id = 'error_details_upgrade'",
)
.fetch_all(&db.pool)
.await
.unwrap();
assert_eq!(details, vec![raw.to_owned(), raw.to_owned()]);
let old_columns: i64 = sqlx::query_scalar(
"select count(*) from information_schema.columns
where table_schema = current_schema() and table_name in ('model_requests', 'ops_events')
and column_name = 'raw_upstream_error'",
)
.fetch_one(&db.pool)
.await
.unwrap();
assert_eq!(old_columns, 0);
db.close().await;
}
async fn seed_legacy_request(pool: &sqlx::PgPool) {
sqlx::query(
"insert into model_requests (
id, client_api_key_ref, config_revision, protocol, operation, endpoint,
client_transport, started_at, deadline_at, outcome, completed_at, routing_scope
) values (
'req_integrity', 'deleted_key', 1, 'openai', 'responses', '/v1/responses',
'http_sse', now(), now() + interval '1 hour', 'failed', now(), 'all'
)",
)
.execute(pool)
.await
.expect("seed request without optional facts");
}
async fn seed_request(pool: &sqlx::PgPool) {
sqlx::query(
"insert into model_requests (
@@ -191,11 +138,6 @@ async fn historical_references_are_validated_without_live_foreign_keys() {
.await
.unwrap();
assert_eq!(references, "already_deleted_account");
let live_columns: i64 = sqlx::query_scalar("select count(*) from information_schema.columns where table_schema = current_schema()
and ((table_name = 'model_requests' and column_name in ('client_api_key_id','provider_account_id'))
or (table_name = 'ops_events' and column_name = 'provider_account_id'))")
.fetch_one(&db.pool).await.unwrap();
assert_eq!(live_columns, 0);
db.close().await;
}
@@ -252,89 +194,6 @@ async fn backup_completion_cannot_precede_its_start() {
db.close().await;
}
#[tokio::test]
async fn request_observation_migration_preserves_every_historical_field() {
let Some(db) = TestDatabase::create_through("request_observation_upgrade", 24).await else {
return;
};
seed_legacy_request(&db.pool).await;
sqlx::raw_sql(
"insert into model_requests
select (jsonb_populate_record(null::model_requests,
to_jsonb(mr) || jsonb_build_object('id', 'req_empty'))).*
from model_requests mr;
update model_requests set config_revision = 7,
provider_account_name_snapshot = 'historical account',
provider_account_email_snapshot = 'history@example.invalid',
provider_account_authentication_kind_snapshot = 'oauth',
routing_scope = 'groups', routing_group_refs = array['deleted_group'],
routing_group_names_snapshot = '[\"历史组\"]',
provider_error_code = 'rate_limit', error_message = 'safe error', retry_after_ms = 0,
client_ip = '192.0.2.10/24', user_agent = 'fixture', reasoning_effort = 'high',
reasoning_preset = 'balanced', subagent_kind = 'review', compact = true,
transport_decision_wait_ms = 0, connect_ms = 2, headers_ms = 3,
first_event_ms = 4, first_reasoning_ms = 5, first_text_ms = 6,
first_token_ms = 5, latency_ms = 1000, provider_processing_ms = 7,
admission_decision_ms = 0, account_selection_wait_ms = 8,
capacity_used_slots = 0, capacity_total_slots = 20,
http_version = 'HTTP/2', websocket_pool = 'reuse',
upstream_connection_id = 'connection', upstream_connection_exit_reason = 'peer_closed',
upstream_connection_age_ms = 10, upstream_connection_idle_ms = 0,
upstream_response_model = 'response-model', continuation_requested = true,
continuation_previous_response_id_hash = repeat('a', 64),
continuation_unavailable_reason = 'missing_response',
recovery_request_id = 'recovery', recovered_at = completed_at + interval '1 second',
recovery_attempt_count = 1, recovery_retry_delay_ms = 0, recovery_total_latency_ms = 1000,
input_tokens = 10, output_tokens = 20, total_tokens = 30,
cost_source = 'provider_reported', cost_amount = 0.0123456789, cost_currency = 'USD',
error_details = '{ \"opaque\": true }', provider_observation_json = '{\"vendor\":true}',
diagnostic_trace_json = '{\"attempts\":[]}'
where id = 'req_integrity'",
).execute(&db.pool).await.expect("populate all moved fields in the released schema");
let before: Vec<serde_json::Value> =
sqlx::query_scalar("select to_jsonb(mr) from model_requests mr order by id")
.fetch_all(&db.pool)
.await
.unwrap();
super::TEST_MIGRATOR.run_to(27, &db.pool).await.unwrap();
let after: Vec<serde_json::Value> = sqlx::query_scalar(
"select to_jsonb(mr) - 'request_observation_json' - 'upstream_response_ms' - 'upstream_api_overhead_ms' - 'upstream_engine_ms' - 'upstream_engine_iapi_ttft_ms' - 'upstream_engine_service_ttft_ms' - 'upstream_engine_iapi_tbt_ms' - 'upstream_engine_service_tbt_ms'
from model_request_observations mr order by id",
)
.fetch_all(&db.pool)
.await
.unwrap();
assert_eq!(
after, before,
"all 90 historical facts survive the migration"
);
let columns: i64 = sqlx::query_scalar(
"select count(*) from information_schema.columns
where table_schema = current_schema() and table_name = 'model_requests'",
)
.fetch_one(&db.pool)
.await
.unwrap();
assert_eq!(columns, 49);
let observation: serde_json::Value = sqlx::query_scalar(
"select request_observation_json from model_requests where id = 'req_integrity'",
)
.fetch_one(&db.pool)
.await
.unwrap();
assert_eq!(
observation["timings"]["local"]["transportDecisionWaitMs"],
0
);
assert_eq!(observation["scheduling"]["capacityUsedSlots"], 0);
assert!(
observation["timings"]["upstream"]
.get("responseMs")
.is_none()
);
db.close().await;
}
#[tokio::test]
async fn request_observation_reject_malformed_documents_and_keep_upstream_clock_independent() {
use serde_json::json;
@@ -8,7 +8,7 @@ use gateway_store::postgres::{
use super::TestDatabase;
#[test]
fn snapshot_client_policy_contains_only_common_limits() {
fn client_key_snapshot_debug_redacts_plaintext_key() {
let policy = ClientApiKeySnapshot {
name_snapshot: None,
request_profiles: Default::default(),
@@ -20,8 +20,6 @@ fn snapshot_client_policy_contains_only_common_limits() {
requests_per_minute: 60,
},
};
assert_eq!(policy.limits.max_concurrency, 3);
assert!(policy.group_ids.is_empty());
assert!(!format!("{policy:?}").contains("sk_snapshot_secret"));
}
@@ -50,6 +48,8 @@ async fn runtime_snapshot_loads_enabled_plaintext_key_without_debug_exposure() {
Some("snapshot")
);
assert!(snapshot.client_api_keys[0].group_ids.is_empty());
assert_eq!(snapshot.client_api_keys[0].limits.max_concurrency, 2);
assert_eq!(snapshot.client_api_keys[0].limits.requests_per_minute, 60);
assert_eq!(
snapshot.client_api_keys[0].plaintext_key.expose_for_auth(),
plaintext
@@ -776,12 +776,12 @@ fn authorization_url(
.append_pair("response_type", "code")
.append_pair("client_id", oauth_client_id)
.append_pair("redirect_uri", OFFICIAL_CODEX_REDIRECT_URI)
.append_pair("scope", AUTHORIZATION_SCOPE)
.append_pair("code_challenge", challenge.as_str())
.append_pair("code_challenge_method", "S256")
.append_pair("state", pending.state.expose_secret())
.append_pair("scope", AUTHORIZATION_SCOPE)
.append_pair("id_token_add_organizations", "true")
.append_pair("codex_cli_simplified_flow", "true")
.append_pair("state", pending.state.expose_secret())
.append_pair("originator", AUTHORIZATION_ORIGINATOR)
.append_pair("codex_app_version", desktop_version)
.append_pair("source_surface_stable_id", &stable_id)
@@ -73,6 +73,10 @@ const DEFAULT_RATE_LIMIT_COOLDOWN: Duration = Duration::from_secs(60);
pub(crate) const QUOTA_SCHEDULING_TTL: Duration = Duration::from_secs(10 * 60);
const QUOTA_HYDRATION_FAILURE_TTL: Duration = Duration::from_secs(5);
const PERIODIC_QUOTA_REFRESH_RETRY_INTERVAL: Duration = Duration::from_secs(30 * 60);
const QUOTA_RESET_FAST_REFRESH_WINDOW: Duration = Duration::from_secs(5 * 60);
const QUOTA_RESET_FAST_REFRESH_INTERVAL: Duration = Duration::from_secs(30);
const QUOTA_RESET_REFRESH_RETRY_INTERVAL: Duration = Duration::from_secs(5 * 60);
const EXHAUSTED_QUOTA_RESET_GRACE: Duration = Duration::from_secs(10);
const QUOTA_RESET_GRACE: Duration = Duration::from_secs(2 * 60);
/// 首次 OAuth 异步观察失败时,由既有 quota worker 兜底重试的单轮上限
/// 随机启动延迟已把候选摊到多轮,该上限仅作单轮安全边界
@@ -4,7 +4,7 @@ use std::collections::BTreeMap;
use std::time::SystemTime;
use chrono::{DateTime, Utc};
use gateway_core::account::{QuotaEvidence, QuotaState};
use gateway_core::account::{QuotaAccessState, QuotaEvidence, QuotaState};
use serde::{Deserialize, Serialize};
use serde_json::{Map, Value};
@@ -127,11 +127,8 @@ fn window_reset_recovered(window: &CodexQuotaWindow, previous: Option<DateTime<U
}
/// 统一手动刷新和 worker 的恢复规则
/// 已耗尽窗口的解除有两条证据路径:
/// reset 前进且用量低于低用量阈值(窗口已滚动),或同一窗口内连续两次观测
/// 都未触顶(滑动窗口的用量回落,reset 不滚动)
/// 后者防止账号被钉在旧
/// reset 日期上;连续两次的要求排除耗尽后立刻拉到的旧快照
/// 到期前仍要求窗口滚动且低用量,或同窗口连续两次未触顶
/// 已记录的耗尽窗口全部到期后,允许新的明确放行观察解除,兼容用量百分比滞后
pub(super) fn reconcile_refresh(
current: QuotaState,
refreshed: &mut CodexAccountQuotaSnapshot,
@@ -139,6 +136,11 @@ pub(super) fn reconcile_refresh(
document: &mut Map<String, Value>,
) -> Result<QuotaState, CodexCredentialQuotaError> {
document.remove(RECOVERY_FIELD);
if refreshed.credits_usable() {
// 点数独立于套餐窗口,恢复后不能继续携带已失效的窗口恢复进度
refreshed.recovery = None;
return Ok(QuotaState::allowed(refreshed.observed_at()));
}
let state = if current.is_exhausted() {
current
} else {
@@ -155,7 +157,18 @@ pub(super) fn reconcile_refresh(
if recovery.pending.is_empty() {
recovery = QuotaRecovery::new(state, refreshed);
}
if current.is_exhausted() && recovery.observe(refreshed) {
// 只信任本次耗尽之后的边界,防止再次拒绝后复用已经过期的 reset 反复解封
// 多个耗尽窗口必须全部到期,未知 reset 仍走原有恢复规则
let allowed_after_reset = refreshed.quota().access() == QuotaAccessState::Allowed
&& !recovery.pending.is_empty()
&& recovery.pending.values().all(|reset| {
reset.is_some_and(|reset| {
reset.timestamp_micros() > recovery.exhausted_at_micros
&& SystemTime::from(reset) <= refreshed.observed_at()
})
});
if current.is_exhausted() && (allowed_after_reset || recovery.observe(refreshed)) {
refreshed.recovery = None;
return Ok(QuotaState::allowed(refreshed.observed_at()));
}
let reset_at = recovery
@@ -1,6 +1,7 @@
//! 请求级额度投影、首查与周期刷新排程
use super::*;
use tokio::time::Instant as TokioInstant;
impl CodexCredentialQuotaService {
/// 批量预热请求级额度投影;持久层或 Provider JSON 异常只退化为未知额度
@@ -67,7 +68,7 @@ struct CodexQuotaProjectionState {
}
struct CodexQuotaRefreshAttempt {
monotonic_at: Instant,
monotonic_at: TokioInstant,
wall_at: SystemTime,
}
@@ -230,7 +231,7 @@ impl CodexQuotaSchedulingProjection {
.iter()
.map(|(account, _)| account.id().clone())
.collect::<BTreeSet<_>>();
let refreshed_at = Instant::now();
let refreshed_at = TokioInstant::now();
let mut state = self
.state
.write()
@@ -239,12 +240,11 @@ impl CodexQuotaSchedulingProjection {
.last_periodic_refresh_at
.retain(|account_id, _| candidate_ids.contains(account_id));
// 已耗尽账号首次立即复核,之后每 30 分钟复核,以发现官方提前重置
// reset + 2 分钟额外触发一次复核,给上游重置留出传播时间
// 已耗尽账号到达 reset 后短周期复核,其余时间保留 30 分钟兜底
// 正常账号仅在非零用量窗口经过宽限期后参与,未更新时复用周期节流
let mut reserved = Vec::new();
for (account, target_reset) in candidates {
if !periodic_quota_refresh_due(&state, account.id(), target_reset, now, refreshed_at) {
if !periodic_quota_refresh_due(&state, &account, target_reset, now, refreshed_at) {
continue;
}
state.last_periodic_refresh_at.insert(
@@ -364,22 +364,34 @@ fn quota_refresh_candidate(
fn periodic_quota_refresh_due(
state: &CodexQuotaProjectionState,
account_id: &ProviderAccountId,
account: &ProviderAccount,
reset_at: Option<SystemTime>,
now: SystemTime,
monotonic_now: Instant,
monotonic_now: TokioInstant,
) -> bool {
state
.last_periodic_refresh_at
.get(account_id)
.is_none_or(|last| {
monotonic_now.saturating_duration_since(last.monotonic_at)
>= PERIODIC_QUOTA_REFRESH_RETRY_INTERVAL
|| reset_at
.and_then(|reset| reset.checked_add(QUOTA_RESET_GRACE))
// 已在该边界之后复核过时回到周期重试,避免过期 reset 每轮触发
.is_some_and(|due_at| last.wall_at < due_at && due_at <= now)
})
let mut interval = PERIODIC_QUOTA_REFRESH_RETRY_INTERVAL;
let due_at = if account.quota().is_exhausted() {
if let Some(since_reset) = reset_at.and_then(|reset| now.duration_since(reset).ok()) {
// 上游边界附近可能仍返回旧额度,首次复核也保留传播余量
if since_reset < EXHAUSTED_QUOTA_RESET_GRACE {
return false;
}
interval = if since_reset < QUOTA_RESET_FAST_REFRESH_WINDOW {
QUOTA_RESET_FAST_REFRESH_INTERVAL
} else {
QUOTA_RESET_REFRESH_RETRY_INTERVAL
};
}
reset_at.and_then(|reset| reset.checked_add(EXHAUSTED_QUOTA_RESET_GRACE))
} else {
reset_at.and_then(|reset| reset.checked_add(QUOTA_RESET_GRACE))
};
let Some(last) = state.last_periodic_refresh_at.get(account.id()) else {
return true;
};
monotonic_now.saturating_duration_since(last.monotonic_at) >= interval
// 跨越 reset 宽限期的首轮复核不受旧间隔阻挡,之后按对应间隔节流
|| due_at.is_some_and(|due_at| last.wall_at < due_at && due_at <= now)
}
fn insert_projection_entry(
@@ -515,7 +515,7 @@ impl CodexCredentialQuotaService {
.await
}
/// 真实限额失败后的异步刷新只补齐展示快照,不允许 usage 快照撤销已确认的失败状态
/// 真实限额失败后的异步回读仅允许明确可用的点数恢复访问,套餐状态仍保留失败保护
pub(crate) async fn refresh_account_after_failure(
&self,
account_id: &ProviderAccountId,
@@ -589,9 +589,10 @@ impl CodexCredentialQuotaService {
} else {
None
};
if account.quota().is_exhausted()
&& matches!(authority, QuotaRefreshAuthority::PreserveAccess)
{
// 失败后回读只允许明确可用的点数恢复访问,套餐 allowed 仍可能滞后
let observe_access =
matches!(authority, QuotaRefreshAuthority::ObserveAccess) || snapshot.credits_usable();
if account.quota().is_exhausted() && !observe_access {
if self
.store
.touch_quota_observation(QuotaObservationTouch {
@@ -615,14 +616,15 @@ impl CodexCredentialQuotaService {
.with_observed_at(observed_at),
});
}
let state = match authority {
QuotaRefreshAuthority::ObserveAccess => reconcile_refresh(
let state = if observe_access {
reconcile_refresh(
account.quota(),
&mut snapshot,
previous.as_ref(),
&mut object,
)?,
QuotaRefreshAuthority::PreserveAccess => account.quota(),
)?
} else {
account.quota()
};
let snapshot = snapshot.with_quota_state(state);
if self
@@ -116,6 +116,7 @@ pub struct CodexAccountQuotaSnapshot {
observed_at: SystemTime,
plan_type: Option<String>,
credits: Option<CreditsSnapshot>,
credits_usable: bool,
fact: CodexQuotaFact,
quota: QuotaState,
windows: Vec<CodexQuotaWindow>,
@@ -144,12 +145,16 @@ impl CodexAccountQuotaSnapshot {
self.plan_type.as_deref()
}
/// 点数只用于展示,不改变套餐限额或账号可用性
/// 上游明确返回的点数事实;点数可用性独立于套餐窗口
#[must_use]
pub fn credits(&self) -> Option<&CreditsSnapshot> {
self.credits.as_ref()
}
pub(super) const fn credits_usable(&self) -> bool {
self.credits_usable
}
#[must_use]
pub const fn fact(&self) -> CodexQuotaFact {
self.fact
@@ -209,7 +214,7 @@ fn parse_codex_quota_object(
for limit in canonical_rate_limits(object)? {
aggregate.observe_rate_limit(limit.rate_limit)?;
}
// credits / spend_control 只证明文档结构已识别,不参与账号级访问结论
// 点数与支出控制不换算成套餐剩余百分比,访问结论由账号快照统一判断
if let Some(spend_control) = object.get("spend_control") {
aggregate.observe_metadata_object(spend_control, "reached")?;
}
@@ -345,6 +350,12 @@ pub(crate) fn parse_account_quota_snapshot(
.then_with(|| left.key.cmp(&right.key))
});
let quota = authoritative_quota_state(&object, &windows, observed_at)?;
let credits_usable = has_usable_credits(&object);
let quota = if credits_usable {
QuotaState::allowed(observed_at)
} else {
quota
};
let recovery = object
.get(RECOVERY_FIELD)
.map(|value| serde_json::from_value(value.clone()))
@@ -359,6 +370,7 @@ pub(crate) fn parse_account_quota_snapshot(
.and_then(Value::as_str)
.map(str::to_owned),
credits: object.get("credits").and_then(parse_credits_from_object),
credits_usable,
fact,
quota,
windows,
@@ -374,7 +386,27 @@ fn quota_source_order(source: &str) -> u8 {
}
}
/// 默认 `codex` 桶是唯一可改变账号额度访问结论的 quota 响应字段
fn has_usable_credits(usage: &Map<String, Value>) -> bool {
let Some(credits) = usage.get("credits") else {
return false;
};
let available = credits.get("has_credits").and_then(Value::as_bool) == Some(true)
|| credits.get("unlimited").and_then(Value::as_bool) == Some(true);
// 套餐耗尽不阻止点数支出,但不能绕过显式支出上限或工作区限制
let spend_reached = usage
.get("spend_control")
.and_then(|spend| spend.get("reached"))
.and_then(Value::as_bool)
== Some(true);
let other_limit_reached = usage
.get("rate_limit_reached_type")
.and_then(|reason| reason.get("type"))
.and_then(Value::as_str)
.is_some_and(|reason| reason != "rate_limit_reached");
available && !spend_reached && !other_limit_reached
}
/// 只有默认 `codex` 桶能提供账号级套餐访问结论,具名桶只保留展示事实
fn authoritative_quota_state(
usage: &Map<String, Value>,
windows: &[CodexQuotaWindow],
@@ -199,18 +199,6 @@ impl OpenAiResponseObservationState {
if metrics.upstream_engine_ms.is_some() {
self.timings.upstream_engine_ms = metrics.upstream_engine_ms;
}
if metrics.upstream_engine_iapi_ttft_ms.is_some() {
self.timings.upstream_engine_iapi_ttft_ms = metrics.upstream_engine_iapi_ttft_ms;
}
if metrics.upstream_engine_service_ttft_ms.is_some() {
self.timings.upstream_engine_service_ttft_ms = metrics.upstream_engine_service_ttft_ms;
}
if metrics.upstream_engine_iapi_tbt_ms.is_some() {
self.timings.upstream_engine_iapi_tbt_ms = metrics.upstream_engine_iapi_tbt_ms;
}
if metrics.upstream_engine_service_tbt_ms.is_some() {
self.timings.upstream_engine_service_tbt_ms = metrics.upstream_engine_service_tbt_ms;
}
self.timings != previous
}
@@ -165,12 +165,6 @@ impl CodexBackendClient {
context: CodexRequestContext<'_>,
) -> CodexClientResult<HeaderMap> {
let mut headers = self.response_headers(profile, request, context)?;
if !headers.contains_key("x-responsesapi-include-timing-metrics") {
headers.insert(
HeaderName::from_static("x-responsesapi-include-timing-metrics"),
HeaderValue::from_static("true"),
);
}
headers.insert(
HeaderName::from_static("openai-beta"),
HeaderValue::from_static("responses_websockets=2026-02-06"),
@@ -255,10 +255,6 @@ pub(crate) fn response_duration_ms(value: &Value) -> Option<u64> {
pub(crate) struct ResponseTimingMetrics {
pub upstream_api_overhead_ms: Option<f64>,
pub upstream_engine_ms: Option<f64>,
pub upstream_engine_iapi_ttft_ms: Option<f64>,
pub upstream_engine_service_ttft_ms: Option<f64>,
pub upstream_engine_iapi_tbt_ms: Option<f64>,
pub upstream_engine_service_tbt_ms: Option<f64>,
}
impl ResponseTimingMetrics {
@@ -287,7 +283,6 @@ impl ResponseTimingMetrics {
return Self {
upstream_api_overhead_ms: overhead,
upstream_engine_ms: milliseconds(path, "engine_wall_ms"),
..Self::default()
};
}
// 直接返回的专项指标保留原口径,明确标为整轮累计的值不能归入单次请求
@@ -303,16 +298,6 @@ impl ResponseTimingMetrics {
"responses_duration_excl_engine_and_client_tool_time_ms",
),
upstream_engine_ms: milliseconds(metrics, "engine_service_total_ms"),
upstream_engine_iapi_ttft_ms: milliseconds(metrics, "engine_iapi_ttft_total_ms"),
upstream_engine_service_ttft_ms: milliseconds(metrics, "engine_service_ttft_total_ms"),
upstream_engine_iapi_tbt_ms: milliseconds(
metrics,
"engine_iapi_tbt_across_engine_calls_ms",
),
upstream_engine_service_tbt_ms: milliseconds(
metrics,
"engine_service_tbt_across_engine_calls_ms",
),
}
}
@@ -323,18 +308,6 @@ impl ResponseTimingMetrics {
if incoming.upstream_engine_ms.is_some() {
self.upstream_engine_ms = incoming.upstream_engine_ms;
}
if incoming.upstream_engine_iapi_ttft_ms.is_some() {
self.upstream_engine_iapi_ttft_ms = incoming.upstream_engine_iapi_ttft_ms;
}
if incoming.upstream_engine_service_ttft_ms.is_some() {
self.upstream_engine_service_ttft_ms = incoming.upstream_engine_service_ttft_ms;
}
if incoming.upstream_engine_iapi_tbt_ms.is_some() {
self.upstream_engine_iapi_tbt_ms = incoming.upstream_engine_iapi_tbt_ms;
}
if incoming.upstream_engine_service_tbt_ms.is_some() {
self.upstream_engine_service_tbt_ms = incoming.upstream_engine_service_tbt_ms;
}
}
}
@@ -4,14 +4,6 @@ use std::path::Path;
use provider_openai::config::{DEFAULT_STREAM_MAX_RETRIES, MAX_STREAM_MAX_RETRIES, OpenAiConfig};
#[test]
fn openai_config_ignores_removed_yaml_identity_fields() {
let config: OpenAiConfig = serde_json::from_value(serde_json::json!({
"wire_profile": {"originator": "legacy-client", "codex_version": "invalid", "residency": "us"}
})).unwrap();
assert_eq!(config, OpenAiConfig::default());
}
#[test]
fn openai_config_keeps_residency_separate_from_client_identity() {
use provider_openai::transport::profile::CodexResidency;
@@ -98,10 +90,6 @@ fn openai_config_defaults_to_the_provider_owned_operating_values() {
u32::try_from(DEFAULT_STREAM_MAX_RETRIES).expect("default retry budget fits u32"),
)
);
assert_eq!(
provider_openai::transport::profile::CodexWireProfile::default().user_agent(),
"Codex Desktop/0.153.4 (Mac OS 15.7.1; arm64) unknown (Codex Desktop; 26.901.51231)"
);
}
#[test]
@@ -265,12 +265,12 @@ async fn authorize_url_matches_the_official_desktop_parameter_contract() {
"response_type",
"client_id",
"redirect_uri",
"scope",
"code_challenge",
"code_challenge_method",
"state",
"scope",
"id_token_add_organizations",
"codex_cli_simplified_flow",
"state",
"originator",
"codex_app_version",
"source_surface_stable_id",
@@ -0,0 +1,215 @@
//! 点数独立可用性、耗尽恢复和过期查询隔离
use super::*;
fn exhausted_usage() -> serde_json::Value {
json!({"rate_limit": {
"allowed": false,
"limit_reached": true,
"primary_window": {"used_percent": 100, "reset_at": 1_900_000_000},
"secondary_window": {"used_percent": 100, "reset_at": 1_900_604_800}
}})
}
async fn mount_usage(server: &MockServer, usage: serde_json::Value) {
server.reset().await;
Mock::given(method("GET"))
.and(path("/api/codex/usage"))
.respond_with(ResponseTemplate::new(200).set_body_json(usage))
.mount(server)
.await;
}
#[tokio::test]
async fn credits_allow_exhausted_subscription_without_bypassing_spend_restrictions() {
for (metadata, allowed) in [
(
json!({"credits": {"has_credits": true, "unlimited": false}}),
true,
),
(
json!({"credits": {"has_credits": false, "unlimited": true}}),
true,
),
(json!({"credits": {"has_credits": true}}), true),
(json!({"credits": {"unlimited": true}}), true),
(
json!({"credits": {"has_credits": false, "unlimited": false, "balance": "500"}}),
false,
),
(json!({"credits": null}), false),
(
json!({"credits": {"has_credits": true}, "spend_control": {"reached": true}}),
false,
),
(
json!({"credits": {"has_credits": true}, "rate_limit_reached_type": {"type": "spend_limit_reached"}}),
false,
),
(
json!({"credits": {"has_credits": true}, "rate_limit_reached_type": {"type": "rate_limit_reached"}}),
true,
),
] {
let store = Arc::new(MemoryAccountStore::default());
create_account_with_enabled(&store, "acct_credits", false).await;
let account = store.account("acct_credits").expect("account");
persist_credential_state(&store, &account, CredentialState::Expired).await;
let server = MockServer::start().await;
let service = quota_service_with_base_url(&store, reqwest::Client::new(), server.uri());
let mut usage = exhausted_usage();
usage
.as_object_mut()
.expect("usage")
.extend(metadata.as_object().expect("metadata").clone());
mount_usage(&server, usage).await;
let snapshot = service
.refresh_account(account.id())
.await
.expect("refresh credits");
assert_eq!(
snapshot.quota().access() == QuotaAccessState::Allowed,
allowed,
"{metadata}"
);
assert_eq!(snapshot.fact().remaining_percent(), Some(0));
let current = store.account("acct_credits").expect("current account");
assert_eq!(current.quota().is_exhausted(), !allowed, "{metadata}");
assert!(!current.enabled());
assert_eq!(current.credential_state(), CredentialState::Expired);
}
}
#[tokio::test]
async fn credits_refresh_clears_old_recovery_and_locks_again_only_when_spent() {
for periodic in [false, true] {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_credits_recovery").await;
let account = store.account("acct_credits_recovery").expect("account");
let server = MockServer::start().await;
let service = quota_service_with_base_url(&store, reqwest::Client::new(), server.uri());
mount_usage(&server, exhausted_usage()).await;
service
.refresh_account(account.id())
.await
.expect("seed exhaustion");
assert!(
store
.quota_json("acct_credits_recovery")
.expect("quota")
.get("_quota_recovery")
.is_some()
);
let mut usage = exhausted_usage();
usage["credits"] =
json!({"has_credits": !periodic, "unlimited": periodic, "balance": "20"});
mount_usage(&server, usage.clone()).await;
if periodic {
assert_eq!(
service
.synchronize()
.await
.expect("periodic recovery")
.updated,
1
);
} else {
service
.refresh_account(account.id())
.await
.expect("manual recovery");
}
let snapshot = service
.read_account(account.id())
.await
.expect("persisted snapshot")
.expect("snapshot");
assert_eq!(snapshot.quota().access(), QuotaAccessState::Allowed);
assert_eq!(snapshot.fact().remaining_percent(), Some(0));
assert_eq!(
snapshot.credits().expect("credits").balance.as_deref(),
Some("20")
);
assert!(
store
.quota_json("acct_credits_recovery")
.expect("quota")
.get("_quota_recovery")
.is_none()
);
usage["credits"] = json!({"has_credits": false, "unlimited": false, "balance": "0"});
mount_usage(&server, usage).await;
let snapshot = service
.refresh_account(account.id())
.await
.expect("spent credits");
assert!(snapshot.quota().is_exhausted());
assert!(
store
.account("acct_credits_recovery")
.expect("account")
.quota()
.is_exhausted()
);
}
}
#[tokio::test]
async fn credits_query_started_before_new_exhaustion_cannot_restore_access() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_stale_credits").await;
let account = store.account("acct_stale_credits").expect("account");
let server = MockServer::start().await;
let service = quota_service_with_base_url(&store, reqwest::Client::new(), server.uri());
let mut usage = exhausted_usage();
usage["credits"] = json!({"has_credits": true, "unlimited": false});
Mock::given(method("GET"))
.and(path("/api/codex/usage"))
.respond_with(
ResponseTemplate::new(200)
.set_delay(Duration::from_millis(200))
.set_body_json(usage),
)
.mount(&server)
.await;
let (result, ()) = tokio::join!(service.refresh_account(account.id()), async {
tokio::time::timeout(Duration::from_secs(2), async {
while server
.received_requests()
.await
.expect("requests")
.is_empty()
{
tokio::time::sleep(Duration::from_millis(1)).await;
}
})
.await
.expect("query started");
persist_quota_state(
&store,
&account,
QuotaState::exhausted(QuotaEvidence::UsageLimitReached, SystemTime::now(), None),
)
.await;
});
result.expect("old display observation can persist independently of newer access state");
assert!(
store
.account("acct_stale_credits")
.expect("account")
.quota()
.is_exhausted()
);
assert!(
service
.read_account(account.id())
.await
.expect("persisted quota")
.expect("snapshot")
.quota()
.is_exhausted()
);
}
@@ -1,10 +1,12 @@
//! OpenAI 额度事实边界与展示快照回归
mod capacity_freeze;
mod credits;
mod initial_sync;
mod recovery;
mod refresh_timing;
mod reset_credits;
mod reset_recovery;
mod scheduling;
mod slots;
mod snapshot;
@@ -1,4 +1,4 @@
//! 通过 quota 服务验证周期复核与 reset 宽限期,不依赖内部调度状态
//! 通过 quota 服务验证耗尽复核节奏、恢复与正常窗口宽限期
use std::time::Duration;
@@ -17,8 +17,17 @@ fn reset_grace(reset: i64) -> SystemTime {
SystemTime::UNIX_EPOCH + Duration::from_secs((reset + 120) as u64)
}
async fn advance_clock(now: &mut SystemTime, seconds: u64) {
let duration = Duration::from_secs(seconds);
// 只在推进时暂停 Tokio 时钟,HTTP 请求期间保持真实计时
tokio::time::pause();
tokio::time::advance(duration).await;
tokio::time::resume();
*now += duration;
}
#[tokio::test]
async fn reset_grace_bypasses_periodic_throttle_once_then_allows_the_next_window() {
async fn exhausted_reset_bypasses_periodic_throttle_and_rechecks_each_pending_window() {
let mut now = SystemTime::now();
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_quota_timing").await;
@@ -30,8 +39,8 @@ async fn reset_grace_bypasses_periodic_throttle_once_then_allows_the_next_window
.duration_since(SystemTime::UNIX_EPOCH)
.unwrap()
.as_secs() as i64
- 115;
let week_reset = short_reset + 3;
+ 5;
let week_reset = short_reset + 30;
let usage = |short_used, short_reset, week_used, week_reset| {
json!({"rate_limit": {
"allowed": false,
@@ -55,20 +64,28 @@ async fn reset_grace_bypasses_periodic_throttle_once_then_allows_the_next_window
let requests = server.received_requests().await.expect("requests").len();
assert!(
now < reset_grace(short_reset),
"fixture must precede grace deadline"
now < SystemTime::UNIX_EPOCH + Duration::from_secs(short_reset as u64),
"fixture must precede reset"
);
service
.synchronize_at(now)
.await
.expect("before grace deadline");
service.synchronize_at(now).await.expect("before reset");
assert_eq!(
server.received_requests().await.expect("requests").len(),
requests
);
mount_usage(&server, usage(0, short_reset + 18_000, 100, week_reset)).await;
now = reset_grace(short_reset);
for delay in [0, 9] {
now = SystemTime::UNIX_EPOCH + Duration::from_secs(short_reset as u64 + delay);
service.synchronize_at(now).await.expect("reset grace");
assert!(
server
.received_requests()
.await
.expect("requests")
.is_empty()
);
}
now = SystemTime::UNIX_EPOCH + Duration::from_secs(short_reset as u64 + 10);
let partial = service
.synchronize_at(now)
.await
@@ -85,11 +102,11 @@ async fn reset_grace_bypasses_periodic_throttle_once_then_allows_the_next_window
service
.synchronize_at(now)
.await
.expect("next scan before weekly grace deadline");
.expect("next scan before weekly reset");
assert_eq!(server.received_requests().await.expect("requests").len(), 1);
// 周窗口的到期复核仍未恢复时,也不能每轮扫描重复请求
now = reset_grace(week_reset);
now = SystemTime::UNIX_EPOCH + Duration::from_secs(week_reset as u64 + 10);
assert_eq!(
service
.synchronize_at(now)
@@ -127,19 +144,139 @@ async fn periodic_checks_continue_when_reset_is_unknown_or_far_in_the_future() {
.expect("seed exhaustion");
mount_usage(&server, usage).await;
let mut now = SystemTime::now();
assert_eq!(
service
.synchronize()
.synchronize_at(now)
.await
.expect("periodic check")
.exhausted,
1
);
service.synchronize().await.expect("throttled repeat check");
advance_clock(&mut now, 29 * 60).await;
service
.synchronize_at(now)
.await
.expect("throttled repeat check");
assert_eq!(server.received_requests().await.expect("requests").len(), 1);
advance_clock(&mut now, 60).await;
service.synchronize_at(now).await.expect("periodic retry");
assert_eq!(server.received_requests().await.expect("requests").len(), 2);
}
}
#[tokio::test]
async fn exhausted_reset_refreshes_quickly_then_backs_off_and_recovers_from_usage() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_reset_retry").await;
let account = store.account("acct_reset_retry").expect("account");
let server = MockServer::start().await;
let service = quota_service_with_base_url(&store, reqwest::Client::new(), server.uri());
let reset = Utc::now().timestamp();
let mut now = SystemTime::UNIX_EPOCH + Duration::from_secs(reset as u64);
let usage = |allowed, used, reset| {
json!({"rate_limit": {
"allowed": allowed,
"primary_window": {"used_percent": used, "reset_at": reset}
}})
};
mount_usage(&server, usage(false, 100, reset)).await;
service
.refresh_account(account.id())
.await
.expect("seed exhaustion");
service
.synchronize_at(now)
.await
.expect("reset grace without periodic history");
advance_clock(&mut now, 9).await;
service
.synchronize_at(now)
.await
.expect("still in reset grace");
assert_eq!(server.received_requests().await.expect("requests").len(), 1);
advance_clock(&mut now, 1).await;
service
.synchronize_at(now)
.await
.expect("initial reset check");
assert_eq!(server.received_requests().await.expect("requests").len(), 2);
// 本次拒绝发生在 reset 之后,滞后的 allowed 不能复用旧边界解封
mount_usage(&server, usage(true, 100, reset)).await;
for (elapsed, requests) in [
(29, 0),
(1, 1),
(29, 1),
(1, 2),
(229, 3),
(1, 3),
(298, 3),
(1, 4),
] {
advance_clock(&mut now, elapsed).await;
service.synchronize_at(now).await.expect("scheduled check");
assert_eq!(
server.received_requests().await.expect("requests").len(),
requests
);
assert!(
store
.account("acct_reset_retry")
.expect("account")
.quota()
.is_exhausted()
);
}
// 查询失败也占用一次尝试,避免网络故障时取消节流
server.reset().await;
Mock::given(method("GET"))
.and(path("/api/codex/usage"))
.respond_with(ResponseTemplate::new(429))
.mount(&server)
.await;
advance_clock(&mut now, 300).await;
assert_eq!(
service
.synchronize_at(now)
.await
.expect("failed check")
.transient,
1
);
advance_clock(&mut now, 299).await;
service
.synchronize_at(now)
.await
.expect("throttled failed check");
assert_eq!(server.received_requests().await.expect("requests").len(), 1);
mount_usage(&server, usage(true, 0, reset + 18_000)).await;
advance_clock(&mut now, 1).await;
assert_eq!(
service
.synchronize_at(now)
.await
.expect("recovery check")
.updated,
1
);
assert!(
!store
.account("acct_reset_retry")
.expect("account")
.quota()
.is_exhausted()
);
advance_clock(&mut now, 300).await;
service
.synchronize_at(now)
.await
.expect("recovered account leaves retry schedule");
assert_eq!(server.received_requests().await.expect("requests").len(), 1);
}
#[tokio::test]
async fn allowed_account_with_expired_window_synchronizes_at_reset_grace() {
let mut now = SystemTime::now();
@@ -0,0 +1,160 @@
//! 到期后的明确放行恢复,保留到期前、多窗口与再次拒绝的保护
use gateway_core::account::{OpaqueProviderData, QuotaObservation};
use super::*;
fn usage(now: i64, resets: &[Option<i64>], allowed: Option<bool>) -> serde_json::Value {
let mut value = json!({"rate_limit": {}});
if let Some(allowed) = allowed {
value["rate_limit"]["allowed"] = json!(allowed);
}
for ((role, seconds), reset) in [("primary_window", 18_000), ("secondary_window", 604_800)]
.into_iter()
.zip(resets)
{
let mut window = json!({"used_percent": 100, "limit_window_seconds": seconds});
if let Some(offset) = reset {
window["reset_at"] = json!(now + offset);
}
value["rate_limit"][role] = window;
}
value
}
#[tokio::test]
async fn allowed_refresh_recovers_full_windows_only_after_the_confirmed_resets() {
for (name, resets, failure_offset, allowed, recovered) in [
("before_reset", vec![Some(3600)], -120, Some(true), false),
("after_reset", vec![Some(-60)], -120, Some(true), true),
("still_denied", vec![Some(-60)], -120, Some(false), false),
("unknown_access", vec![Some(-60)], -120, None, false),
("unknown_reset", vec![None], -120, Some(true), false),
("new_failure", vec![Some(-60)], -30, Some(true), false),
(
"weekly_not_due",
vec![Some(-60), Some(3600)],
-120,
Some(true),
false,
),
(
"both_due",
vec![Some(-90), Some(-60)],
-120,
Some(true),
true,
),
(
"weekly_unknown",
vec![Some(-60), None],
-120,
Some(true),
false,
),
] {
for worker in [false, true] {
let store = Arc::new(MemoryAccountStore::default());
let account_id = format!("acct_reset_{name}_{worker}");
create_account(&store, &account_id).await;
let account = store.account(&account_id).expect("account");
let now = Utc::now().timestamp();
let observed_at =
SystemTime::UNIX_EPOCH + Duration::from_secs((now + failure_offset) as u64);
let reset_at = resets
.iter()
.filter_map(|reset| *reset)
.min()
.map(|offset| SystemTime::UNIX_EPOCH + Duration::from_secs((now + offset) as u64));
let old_usage = usage(now, &resets, Some(false));
store
.compare_and_swap_quota(QuotaObservation {
plan_type: None,
account_id: account.id().clone(),
expected_revision: account.revision(),
quota: OpaqueProviderData::new(old_usage.as_object().expect("usage").clone()),
observed_at,
state: QuotaState::exhausted(
QuotaEvidence::UsageLimitReached,
observed_at,
reset_at,
),
})
.await
.expect("persist confirmed exhaustion before refresh");
let server = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/api/codex/usage"))
.respond_with(
ResponseTemplate::new(200).set_body_json(usage(now, &resets, allowed)),
)
.mount(&server)
.await;
let service = quota_service_with_base_url(&store, reqwest::Client::new(), server.uri());
if worker {
let summary = service.synchronize().await.expect("worker refresh");
assert_eq!(summary.updated, u64::from(recovered), "{name}");
} else {
service
.refresh_account(account.id())
.await
.expect("manual refresh");
}
let refreshed = service
.read_account(account.id())
.await
.expect("read")
.expect("snapshot");
assert_eq!(
refreshed.quota().is_exhausted(),
!recovered,
"{name}, worker={worker}"
);
assert!(
refreshed
.windows()
.iter()
.all(|window| window.used_percent() == Some(100.0))
);
assert_eq!(
store
.account(&account_id)
.expect("account")
.quota()
.is_exhausted(),
!recovered
);
assert_eq!(
store
.quota_json(&account_id)
.expect("quota")
.get("_quota_recovery")
.is_none(),
recovered
);
if recovered {
service
.refresh_account(account.id())
.await
.expect("allowed remains allowed");
assert!(
!store
.account(&account_id)
.expect("account")
.quota()
.is_exhausted()
);
// 新拒绝不能沿用刚刚解除时的旧 reset,即使查询仍然 allowed=true
persist_quota_state(&store, &account, exhausted_quota(reset_at)).await;
for _ in 0..2 {
let snapshot = service
.refresh_account(account.id())
.await
.expect("refresh after new refusal");
assert!(snapshot.quota().is_exhausted(), "{name}");
}
}
}
}
}
@@ -8421,6 +8421,82 @@ async fn official_usage_limit_failure_persists_fact_without_fabricating_usage()
);
}
#[tokio::test]
async fn usage_limit_failure_recovers_with_credits_without_replaying_the_failed_request() {
let store = Arc::new(MemoryAccountStore::default());
let account_id = "acct_provider_contract";
create_account(&store, account_id).await;
let server = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/api/codex/usage"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"rate_limit": {
"allowed": false,
"limit_reached": true,
"primary_window": {"used_percent": 100, "reset_at": 1_900_000_000}
},
"credits": {"has_credits": true, "unlimited": false, "balance": "20"}
})))
.expect(1)
.mount(&server)
.await;
Mock::given(method("POST"))
.and(path("/codex/responses"))
.respond_with(ResponseTemplate::new(429).set_body_json(json!({
"error": {"type": "usage_limit_reached", "message": "usage limit reached", "resets_at": 1_900_000_000}
})))
.expect(1)
.mount(&server)
.await;
let mut stream = provider_with_base_url(&store, server.uri())
.execute(
planned_request("openai", http_generate_operation()),
context("req_usage_limit_credits", CancellationToken::new()),
)
.await
.expect("provider stream");
let failure = loop {
match stream.next().await {
Some(Ok(_)) => {}
Some(Err(error)) => break error,
None => panic!("quota failure must reach the caller"),
}
};
assert_eq!(failure.kind(), ProviderErrorKind::QuotaExhausted);
assert!(
store
.account(account_id)
.expect("account")
.quota()
.is_exhausted()
);
timeout(Duration::from_secs(5), async {
while store
.account(account_id)
.expect("account")
.quota()
.is_exhausted()
{
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await
.expect("credits restore access after background refresh");
assert!(stream.next().await.is_none());
let account = store.account(account_id).expect("recovered account");
assert_eq!(account.quota().access(), QuotaAccessState::Allowed);
assert_eq!(account.credential_state(), CredentialState::Ready);
let quota = store.quota_json(account_id).expect("persisted credits");
assert_eq!(quota.pointer("/credits/has_credits"), Some(&json!(true)));
assert_eq!(
quota.pointer("/rate_limits_by_limit_id/codex/primary_window/used_percent"),
Some(&json!(100))
);
assert!(quota.get("_quota_recovery").is_none());
server.verify().await;
}
#[tokio::test]
async fn capacity_http_and_websocket_opening_rejections_use_bounded_business_retry() {
for use_websocket in [false, true] {
@@ -66,9 +66,10 @@ async fn observe_timing_events(websocket: bool, events: Vec<Value>) -> ProviderR
let mut ws = crate::transport::accept_codex_test_websocket_with(
socket,
|request: &tokio_tungstenite::tungstenite::handshake::server::Request, response| {
assert_eq!(
request.headers()["x-responsesapi-include-timing-metrics"],
"true"
assert!(
!request
.headers()
.contains_key("x-responsesapi-include-timing-metrics")
);
response.headers_mut().insert(
"sec-websocket-extensions",
@@ -125,19 +126,11 @@ async fn official_response_duration_is_observed_independently_of_the_request_clo
let metrics = json!({"type":"responsesapi.websocket_timing","timing_metrics":{
"responses_duration_excl_engine_and_client_tool_time_ms":120.25,
"engine_service_total_ms":6400.0,
"engine_iapi_ttft_total_ms":650.5,
"engine_service_ttft_total_ms":720.25,
"engine_iapi_tbt_across_engine_calls_ms":18.45,
"engine_service_tbt_across_engine_calls_ms":20.12
}});
let wrong_id = json!({"type":"responsesapi.websocket_timing","response_id":"resp_other","timing_metrics":{"engine_service_total_ms":999.0}});
let invalid = json!({"type":"responsesapi.websocket_timing","timing_metrics":{
"responses_duration_excl_engine_and_client_tool_time_ms":null,
"engine_service_total_ms":"100",
"engine_iapi_ttft_total_ms":-1,
"engine_service_ttft_total_ms":1e99,
"engine_iapi_tbt_across_engine_calls_ms":{},
"engine_service_tbt_across_engine_calls_ms":[]
}});
let stale = json!({"type":"responsesapi.websocket_timing","timing_metrics":{"engine_service_total_ms":999.0}});
let events = [stale, created, metrics, wrong_id, invalid, completed];
@@ -145,10 +138,6 @@ async fn official_response_duration_is_observed_independently_of_the_request_clo
assert_eq!(timings.upstream_response_ms, Some(7_000));
assert_eq!(timings.upstream_api_overhead_ms, Some(120.25));
assert_eq!(timings.upstream_engine_ms, Some(6400.0));
assert_eq!(timings.upstream_engine_iapi_ttft_ms, Some(650.5));
assert_eq!(timings.upstream_engine_service_ttft_ms, Some(720.25));
assert_eq!(timings.upstream_engine_iapi_tbt_ms, Some(18.45));
assert_eq!(timings.upstream_engine_service_tbt_ms, Some(20.12));
assert_eq!(timings.first_token_ms, None);
assert!(timings.first_event_ms.is_some_and(|value| value >= 20_000));
}
@@ -197,7 +186,6 @@ async fn response_critical_path_is_observed_without_logical_turn_totals_on_both_
let mut metrics = response_critical_path_event();
// 同帧即使携带整轮专项值,也不能与响应级 critical path 混算
metrics["timing_metrics"]["engine_service_total_ms"] = json!(25_000);
metrics["timing_metrics"]["engine_iapi_ttft_total_ms"] = json!(500);
let mut events = critical_path_response_events(metrics);
let mut wrong_id = response_critical_path_event();
wrong_id["timing_metrics"]["response_id"] = json!("resp_other");
@@ -208,10 +196,6 @@ async fn response_critical_path_is_observed_without_logical_turn_totals_on_both_
assert!((timings.upstream_api_overhead_ms.unwrap() - 81.409594).abs() < 0.000001);
assert_eq!(timings.upstream_response_ms, Some(7000));
assert_eq!(timings.first_token_ms, None);
assert_eq!(timings.upstream_engine_iapi_ttft_ms, None);
assert_eq!(timings.upstream_engine_service_ttft_ms, None);
assert_eq!(timings.upstream_engine_iapi_tbt_ms, None);
assert_eq!(timings.upstream_engine_service_tbt_ms, None);
}
}
@@ -252,12 +236,10 @@ async fn critical_path_rejects_mismatched_identity_and_incomplete_or_cumulative_
"timing_scope":"logical_turn",
"engine_service_total_ms":25_000,
"responses_duration_excl_engine_and_client_tool_time_ms":500,
"engine_iapi_ttft_total_ms":200
}});
let timings = observe_timing_events(false, critical_path_response_events(metrics)).await;
assert_eq!(timings.upstream_engine_ms, None);
assert_eq!(timings.upstream_api_overhead_ms, None);
assert_eq!(timings.upstream_engine_iapi_ttft_ms, None);
}
#[tokio::test]
@@ -288,10 +270,6 @@ async fn timing_frames_outside_the_active_response_do_not_enter_request_metrics(
let timing = json!({"type":"responsesapi.websocket_timing","timing_metrics":{
"responses_duration_excl_engine_and_client_tool_time_ms":0,
"engine_service_total_ms":450,
"engine_iapi_ttft_total_ms":211,
"engine_service_ttft_total_ms":233,
"engine_iapi_tbt_across_engine_calls_ms":2.450638,
"engine_service_tbt_across_engine_calls_ms":5.267279
}});
let created = json!({"type":"response.created","response":{"id":"resp_boundary"}});
let completed = json!({"type":"response.completed","response":{"id":"resp_boundary","status":"completed","output":[],"usage":{"input_tokens":1,"output_tokens":1,"total_tokens":2}}});
@@ -311,10 +289,6 @@ async fn timing_frames_outside_the_active_response_do_not_enter_request_metrics(
let timings = finish_timings(&mut stream, ProviderResponseTimings::default()).await;
assert_eq!(timings.upstream_api_overhead_ms, None);
assert_eq!(timings.upstream_engine_ms, None);
assert_eq!(timings.upstream_engine_iapi_ttft_ms, None);
assert_eq!(timings.upstream_engine_service_ttft_ms, None);
assert_eq!(timings.upstream_engine_iapi_tbt_ms, None);
assert_eq!(timings.upstream_engine_service_tbt_ms, None);
assert_eq!(timings.first_token_ms, None);
server.await.unwrap();
}
@@ -395,9 +369,10 @@ async fn structural_frames_start_ttft_before_content_on_http_and_websocket() {
socket,
|request: &tokio_tungstenite::tungstenite::handshake::server::Request,
response| {
assert_eq!(
request.headers()["x-responsesapi-include-timing-metrics"],
"true"
assert!(
!request
.headers()
.contains_key("x-responsesapi-include-timing-metrics")
);
response.headers_mut().insert(
"sec-websocket-extensions",
@@ -430,7 +430,7 @@ impl ProviderAccountStore for MemoryAccountStore {
async fn compare_and_swap_quota(
&self,
observation: QuotaObservation,
mut observation: QuotaObservation,
) -> Result<QuotaWriteOutcome, StoreError> {
let mut accounts = self.accounts.lock().expect("account store lock");
let stored = accounts
@@ -446,7 +446,18 @@ impl ProviderAccountStore for MemoryAccountStore {
{
return Ok(QuotaWriteOutcome::Conflict);
}
let quota = observation.state;
// 与 PostgreSQL 一致,展示快照与访问事实分别按各自观察时间隔离
let current = stored.account.quota();
let quota = if observation
.state
.observed_at()
.is_some_and(|observed| current.observed_at().is_none_or(|time| time <= observed))
{
observation.state
} else {
current
};
observation.state = quota;
let mut replacement = AccountRebuild::preserving(&stored.account).with_quota(quota);
if let Some(plan_type) = &observation.plan_type {
replacement.profile = Some((
@@ -119,24 +119,23 @@ fn raw_identity_rejects_injection_conflicting_headers_and_unknown_modes() {
}
#[test]
fn old_configuration_and_old_frozen_profile_keep_their_existing_identity() {
fn preset_identity_round_trip_preserves_request_headers_and_official_preview() {
let state = CodexWireProfileState::new(CodexWireProfile::default());
let legacy = ClientProfileSelection::default();
let resolved = legacy.resolve(&state).unwrap();
let configuration = ClientProfileSelection::default().document().unwrap();
let resolved = RequestProfileSelection::parse(&configuration)
.unwrap()
.resolve(&state)
.unwrap();
let restored: CodexWireProfile =
serde_json::from_value(serde_json::to_value(&resolved).unwrap()).unwrap();
assert_eq!(
RequestProfileSelection::parse(&legacy.document().unwrap())
.unwrap()
.resolve(&state)
.unwrap(),
resolved
build_codex_model_headers(&restored, "Bearer fixture", None).unwrap(),
build_codex_model_headers(&resolved, "Bearer fixture", None).unwrap()
);
let old_document = serde_json::to_value(&resolved).unwrap();
assert!(old_document.get("exact_user_agent").is_none());
let restored: CodexWireProfile = serde_json::from_value(old_document).unwrap();
assert_eq!(restored.user_agent(), resolved.user_agent());
assert_eq!(
state
.preview_selection(&legacy.document().unwrap())
.preview_selection(&configuration)
.unwrap()
.expose_to_provider()["versionSource"],
"official"
@@ -529,98 +529,30 @@ fn encoder_should_restore_conversation_fallback_after_body_and_context_values()
}
#[test]
fn encoder_should_accept_metadata_session_and_thread_ids_but_ignore_legacy_context_aliases() {
fn encoder_should_extract_websocket_context_from_client_metadata() {
let metadata = json!({
"session_id": "metadata-session",
"thread_id": "metadata-thread",
"x-codex-turn-state": "metadata-turn-state",
"turn_id": "metadata-turn-id",
"future_context": {"keep": true}
});
let request = request(Map::from_iter([
("model".to_owned(), json!("client-model")),
("input".to_owned(), json!("prompt")),
("turn_state".to_owned(), json!("legacy-turn-state")),
(
"x-codex-turn-state".to_owned(),
json!("legacy-header-turn-state"),
),
("turn_metadata".to_owned(), json!("legacy-turn-metadata")),
("beta_features".to_owned(), json!("legacy-beta")),
("include_timing_metrics".to_owned(), json!("legacy-timing")),
("codex_window_id".to_owned(), json!("legacy-window")),
(
"x-codex-window-id".to_owned(),
json!("legacy-header-window"),
),
("parent_thread_id".to_owned(), json!("legacy-parent")),
(
"x-codex-parent-thread-id".to_owned(),
json!("legacy-header-parent"),
),
("conversationId".to_owned(), json!("legacy-conversation")),
("sessionId".to_owned(), json!("legacy-session")),
("threadId".to_owned(), json!("legacy-thread")),
("client_request_id".to_owned(), json!("legacy-request")),
("clientRequestId".to_owned(), json!("legacy-camel-request")),
("turnId".to_owned(), json!("legacy-turn")),
("x-codex-turn-id".to_owned(), json!("legacy-header-turn")),
(
"client_metadata".to_owned(),
json!({
"turnState": "metadata-turn-state",
"turnMetadata": "metadata-turn-metadata",
"conversation_id": "metadata-conversation",
"session_id": "metadata-session",
"thread_id": "metadata-thread",
"x-codex-window-id": "metadata-window"
}),
),
("client_metadata".to_owned(), metadata.clone()),
]));
let encoded = encode_generate_request(&request, "gpt-test", None).expect("encode");
assert_eq!(
(
encoded.turn_state,
encoded.turn_metadata,
encoded.beta_features,
encoded.include_timing_metrics,
encoded.codex_window_id,
encoded.parent_thread_id,
encoded.client_conversation_id,
encoded.client_session_id,
encoded.client_thread_id,
encoded.client_request_id,
encoded.client_turn_id,
),
(
None,
None,
None,
None,
None,
None,
None,
Some("metadata-session".to_owned()),
Some("metadata-thread".to_owned()),
None,
None,
)
encoded.client_session_id.as_deref(),
Some("metadata-session")
);
}
#[test]
fn encoder_should_extract_official_websocket_context_projection() {
let request = request(Map::from_iter([
("model".to_owned(), json!("client-model")),
("input".to_owned(), json!("prompt")),
(
"client_metadata".to_owned(),
json!({
"x-codex-turn-state": "metadata-turn-state",
"turn_id": "metadata-turn-id"
}),
),
]));
let encoded = encode_generate_request(&request, "gpt-test", None).expect("encode");
assert_eq!(encoded.client_thread_id.as_deref(), Some("metadata-thread"));
assert_eq!(encoded.turn_state.as_deref(), Some("metadata-turn-state"));
assert_eq!(encoded.client_turn_id.as_deref(), Some("metadata-turn-id"));
assert_eq!(encoded.body().get("client_metadata"), Some(&metadata));
}
#[test]
@@ -110,8 +110,7 @@ async fn official_websocket_wire_fingerprint_should_remain_aligned() {
opening.lines().next(),
Some("GET /backend-api/codex/responses HTTP/1.1")
);
// 固定身份字段沿用官方画像,专项计时头按官方客户端在 openai-beta 后插入
// HeaderMap 序列化会随新增头调整迭代顺序,锁定启用计时后的完整 opening
// 固定身份字段沿用官方画像,锁定默认不请求专项计时的完整 opening
assert_eq!(
read_header_names(&opening),
vec![
@@ -124,7 +123,7 @@ async fn official_websocket_wire_fingerprint_should_remain_aligned() {
"authorization",
"user-agent",
"originator",
"x-responsesapi-include-timing-metrics",
"openai-beta",
"version",
"x-codex-beta-features",
"x-client-request-id",
@@ -133,13 +132,12 @@ async fn official_websocket_wire_fingerprint_should_remain_aligned() {
"x-codex-window-id",
"x-codex-turn-metadata",
"x-codex-routing-hint",
"openai-beta",
"sec-websocket-extensions",
]
);
assert_eq!(
read_header_value(&opening, "x-responsesapi-include-timing-metrics"),
Some("true"),
None,
);
assert_eq!(
read_header_value(&opening, "sec-websocket-extensions"),
@@ -365,8 +365,8 @@ async fn websocket_execute_response_create_request_should_collect_completed_sse(
}
#[tokio::test]
async fn websocket_exchange_should_accept_upstream_frame_above_removed_private_limit() {
const REMOVED_FRAME_LIMIT_BYTES: usize = 16 * 1024 * 1024;
async fn websocket_exchange_should_deliver_large_upstream_frames() {
const LARGE_DELTA_BYTES: usize = 17 * 1024 * 1024;
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
@@ -376,14 +376,13 @@ async fn websocket_exchange_should_accept_upstream_frame_above_removed_private_l
let mut websocket =
accept_codex_test_websocket_with(stream, |_request, _response| {}).await;
let _message = websocket.next().await.unwrap().unwrap();
let oversized_event = json!({
let large_event = json!({
"type": "response.output_text.delta",
"delta": "x".repeat(REMOVED_FRAME_LIMIT_BYTES + 1),
"delta": "x".repeat(LARGE_DELTA_BYTES),
})
.to_string();
assert!(oversized_event.len() > REMOVED_FRAME_LIMIT_BYTES);
websocket
.send(Message::Text(oversized_event.into()))
.send(Message::Text(large_event.into()))
.await
.unwrap();
websocket
@@ -391,7 +390,7 @@ async fn websocket_exchange_should_accept_upstream_frame_above_removed_private_l
json!({
"type": "response.completed",
"response": {
"id": "resp_after_oversized_frame",
"id": "resp_after_large_frame",
"object": "response",
"output": [],
"usage": {
@@ -421,11 +420,11 @@ async fn websocket_exchange_should_accept_upstream_frame_above_removed_private_l
let response = execute_response_create_request(&prepared)
.await
.expect("upstream frame above the former private limit should remain forwardable");
.expect("large upstream frame should remain forwardable");
server.await.unwrap();
assert!(response.body.len() > REMOVED_FRAME_LIMIT_BYTES);
assert!(response.body.contains("resp_after_oversized_frame"));
assert!(response.body.len() > LARGE_DELTA_BYTES);
assert!(response.body.contains("resp_after_large_frame"));
}
#[tokio::test]
@@ -228,39 +228,16 @@ async fn quota_refresh_persists_subscription_without_user_profile_and_reads_it_f
assert!(document.get("userId").is_none());
}
#[tokio::test]
async fn billing_percent_does_not_rewrite_quota_access_fact() {
let (store, repository) =
repository_with_accounts(&[("still-exhausted", "still-exhausted")]).await;
let id = account_id("still-exhausted");
store
.apply_quota_access(QuotaAccessChange {
account_id: id.clone(),
expected_revision: CredentialRevision::new(1).expect("revision"),
state: QuotaState::exhausted(QuotaEvidence::ProviderDenied, SystemTime::now(), None),
})
.await
.expect("mark quota exhausted");
crate::support::grok_quota_service(
repository,
QueueBillingTransport::success(br#"{"config":{"creditUsagePercent":100}}"#),
)
.refresh_account(&id)
.await
.expect("refresh exhausted quota");
assert_eq!(
store.account(&id).expect("account").quota().access(),
gateway_core::account::QuotaAccessState::Exhausted
);
}
#[tokio::test]
async fn billing_access_and_scheduling_use_current_quota_fields() {
use gateway_core::account::QuotaAccessState::{Allowed, Exhausted};
for (config, expected_percent, expected_access) in [
(
serde_json::json!({"creditUsagePercent": 0}),
Some(0.0),
Allowed,
),
(
serde_json::json!({"creditUsagePercent": 100}),
Some(100.0),
@@ -306,6 +283,10 @@ async fn billing_access_and_scheduling_use_current_quota_fields() {
let account = store.account(&id).expect("refreshed account");
assert_eq!(snapshot.billing().used_percent(), expected_percent);
assert!(
snapshot.billing().has_authoritative_quota(),
"config: {config}"
);
assert_eq!(
account.quota().access(),
expected_access,
@@ -526,66 +507,10 @@ async fn quota_refresh_rejects_a_concurrent_credential_revision() {
}
#[tokio::test]
async fn legacy_billing_fields_do_not_supply_quota_evidence_or_period() {
for (limit, used) in [
(
serde_json::json!({"val": 10000}),
serde_json::json!({"val": 2500}),
),
(
serde_json::json!("uninterpreted"),
serde_json::json!({"val": -1}),
),
] {
let (store, repository) = repository_with_accounts(&[("legacy", "legacy")]).await;
let id = account_id("legacy");
store
.apply_quota_access(QuotaAccessChange {
account_id: id.clone(),
expected_revision: CredentialRevision::new(1).expect("revision"),
state: QuotaState::exhausted(
QuotaEvidence::ProviderDenied,
SystemTime::now(),
None,
),
})
.await
.expect("mark quota exhausted");
let body = serde_json::to_vec(&serde_json::json!({"config": {
"monthlyLimit": limit, "used": used,
"billingPeriodStart": "2026-07-01T00:00:00Z",
"billingPeriodEnd": "2099-08-01T00:00:00Z"
}}))
.expect("billing document");
let service =
crate::support::grok_quota_service(repository, QueueBillingTransport::success(&body));
let snapshot = service
.refresh_account(&id)
.await
.expect("refresh opaque billing");
let account = store.account(&id).expect("account");
assert_eq!(snapshot.billing().used_percent(), None);
assert!(!snapshot.billing().has_authoritative_quota());
assert_eq!(
snapshot.billing().period_kind(),
provider_xai::GrokQuotaPeriodKind::Other
);
assert_eq!(snapshot.billing().period_start(), None);
assert_eq!(snapshot.billing().period_end(), None);
assert_eq!(service.scheduling_signals(&account), None);
assert_eq!(
account.quota().access(),
gateway_core::account::QuotaAccessState::Exhausted
);
}
}
#[tokio::test]
async fn current_quota_period_does_not_fill_missing_dates_from_legacy_fields() {
async fn quota_period_dates_remain_absent_when_not_reported() {
let (_, repository) = repository_with_accounts(&[("current-period", "current-period")]).await;
let transport = QueueBillingTransport::success(
br#"{"config":{"creditUsagePercent":25,"currentPeriod":{"type":"USAGE_PERIOD_TYPE_WEEKLY"},"billingPeriodStart":"2026-07-01T00:00:00Z","billingPeriodEnd":"2099-08-01T00:00:00Z"}}"#,
br#"{"config":{"creditUsagePercent":25,"currentPeriod":{"type":"USAGE_PERIOD_TYPE_WEEKLY"}}}"#,
);
let snapshot = crate::support::grok_quota_service(repository, transport)
.refresh_account(&account_id("current-period"))
@@ -658,61 +583,59 @@ async fn expired_billing_window_does_not_participate_in_scheduling_rank() {
}
#[tokio::test]
async fn non_authoritative_quota_refresh_does_not_clear_existing_quota_exhaustion() {
let (store, repository) =
repository_with_accounts(&[("free-quota", "subject-free-quota")]).await;
let id = account_id("free-quota");
store
.apply_quota_access(QuotaAccessChange {
account_id: id.clone(),
expected_revision: CredentialRevision::new(1).expect("revision"),
state: QuotaState::exhausted(QuotaEvidence::ProviderDenied, SystemTime::now(), None),
})
.await
.expect("mark quota exhausted");
let transport = QueueBillingTransport::success(
br#"{"config":{"currentPeriod":{"type":"USAGE_PERIOD_TYPE_WEEKLY","start":"2026-07-15T00:00:00Z","end":"2026-07-22T00:00:00Z"},"onDemandCap":{"val":0},"onDemandUsed":{"val":0},"prepaidBalance":{"val":0}}}"#,
);
let snapshot = crate::support::grok_quota_service(repository, transport)
.refresh_account(&id)
.await
.expect("refresh Free quota");
async fn non_authoritative_quota_refresh_preserves_exhaustion_without_scheduling_evidence() {
use provider_xai::GrokQuotaPeriodKind::{Other, Weekly};
assert!(!snapshot.billing().has_authoritative_quota());
assert_eq!(
store.account(&id).expect("account").quota().access(),
gateway_core::account::QuotaAccessState::Exhausted
);
}
for (config, expected_period) in [
(
serde_json::json!({"future_quota": {"remaining": 100}}),
Other,
),
(
serde_json::json!({
"currentPeriod": {"type": "USAGE_PERIOD_TYPE_WEEKLY"},
"onDemandCap": {"val": 0},
"onDemandUsed": {"val": 0},
"prepaidBalance": {"val": 0}
}),
Weekly,
),
] {
let (store, repository) =
repository_with_accounts(&[("free-quota", "subject-free-quota")]).await;
let id = account_id("free-quota");
store
.apply_quota_access(QuotaAccessChange {
account_id: id.clone(),
expected_revision: CredentialRevision::new(1).expect("revision"),
state: QuotaState::exhausted(
QuotaEvidence::ProviderDenied,
SystemTime::now(),
None,
),
})
.await
.expect("mark quota exhausted");
let body = serde_json::to_vec(&serde_json::json!({"config": config})).expect("billing");
let service =
crate::support::grok_quota_service(repository, QueueBillingTransport::success(&body));
let snapshot = service
.refresh_account(&id)
.await
.expect("refresh Free quota");
let account = store.account(&id).expect("account");
#[tokio::test]
async fn reported_zero_percent_is_authoritative_quota() {
let (_, repository) =
repository_with_accounts(&[("zero-percent", "subject-zero-percent")]).await;
let transport = QueueBillingTransport::success(
br#"{"config":{"creditUsagePercent":0,"currentPeriod":{"type":"USAGE_PERIOD_TYPE_WEEKLY","start":"2026-07-15T00:00:00Z","end":"2026-07-22T00:00:00Z"}}}"#,
);
let snapshot = crate::support::grok_quota_service(repository, transport)
.refresh_account(&account_id("zero-percent"))
.await
.expect("refresh reported quota");
assert!(snapshot.billing().has_authoritative_quota());
}
#[tokio::test]
async fn positive_prepaid_balance_is_authoritative_quota() {
let (_, repository) =
repository_with_accounts(&[("prepaid-quota", "subject-prepaid-quota")]).await;
let transport = QueueBillingTransport::success(
br#"{"config":{"currentPeriod":{"type":"USAGE_PERIOD_TYPE_WEEKLY","start":"2026-07-15T00:00:00Z","end":"2026-07-22T00:00:00Z"},"prepaidBalance":{"val":500}}}"#,
);
let snapshot = crate::support::grok_quota_service(repository, transport)
.refresh_account(&account_id("prepaid-quota"))
.await
.expect("refresh prepaid quota");
assert!(snapshot.billing().has_authoritative_quota());
assert!(!snapshot.billing().has_authoritative_quota());
assert_eq!(snapshot.billing().used_percent(), None);
assert_eq!(snapshot.billing().period_kind(), expected_period);
assert_eq!(snapshot.billing().period_start(), None);
assert_eq!(snapshot.billing().period_end(), None);
assert_eq!(service.scheduling_signals(&account), None);
assert_eq!(
account.quota().access(),
gateway_core::account::QuotaAccessState::Exhausted
);
}
}
#[tokio::test]
@@ -177,109 +177,40 @@ fn encoder_should_strip_all_client_metadata_before_grok_build() {
}
#[test]
fn encoder_should_apply_build_defaults_and_normalize_reasoning_effort() {
let request = raw_request(json!({
"model": "client-model",
"input": "hello",
"include": ["web_search_call.action.sources"],
"reasoning": {"effort": "XHIGH", "summary": "auto"}
}));
fn encoder_should_apply_build_defaults_and_preserve_requested_includes() {
for (fields, expected_include) in [
(json!({}), json!(["reasoning.encrypted_content"])),
(
json!({"store": null, "include": null}),
json!(["reasoning.encrypted_content"]),
),
(
json!({"include": ["web_search_call.action.sources"]}),
json!([
"web_search_call.action.sources",
"reasoning.encrypted_content"
]),
),
] {
let mut body = fields;
body["input"] = json!("hello");
let encoded = GrokResponsesRequest::encode(&raw_request(body), "grok-4.7", &client_key())
.expect("build defaults");
let encoded = GrokResponsesRequest::encode(&request, "grok-4.5", &client_key())
.expect("normalized request");
let body = Value::Object(encoded.body().clone());
assert_eq!(body.pointer("/store"), Some(&json!(false)));
assert_eq!(
body.pointer("/include"),
Some(&json!([
"web_search_call.action.sources",
"reasoning.encrypted_content"
]))
);
assert_eq!(body.pointer("/reasoning/effort"), Some(&json!("xhigh")));
assert_eq!(body.pointer("/reasoning/summary"), Some(&json!("auto")));
let request = raw_request(json!({
"model": "client-model",
"input": "hello",
"reasoning": {"effort": "xhigh"}
}));
let encoded = GrokResponsesRequest::encode(&request, "xai/grok-4.6-latest", &client_key())
.expect("routed model request");
let body = Value::Object(encoded.body().clone());
assert_eq!(body.pointer("/model"), Some(&json!("xai/grok-4.6-latest")));
assert_eq!(body.pointer("/reasoning/effort"), Some(&json!("xhigh")));
assert_eq!(body.pointer("/reasoning_effort"), None);
assert_eq!(body.pointer("/reasoningEffort"), None);
let request = raw_request(json!({
"model": "client-model",
"input": "hello",
"reasoning": {"effort": "high", "summary": "auto"}
}));
let encoded = GrokResponsesRequest::encode(&request, "grok-composer-2.5-fast", &client_key())
.expect("Composer request");
let body = Value::Object(encoded.body().clone());
assert_eq!(
body.pointer("/reasoning"),
Some(&json!({"effort":"high", "summary":"auto"}))
);
let request = raw_request(json!({
"model": "client-model",
"input": "hello",
"store": null,
"include": null,
"reasoning": {"effort": "max"}
}));
let encoded = GrokResponsesRequest::encode(&request, "grok-composer-2.5-fast", &client_key())
.expect("empty Composer reasoning");
let body = Value::Object(encoded.body().clone());
assert_eq!(body.pointer("/store"), Some(&json!(false)));
assert_eq!(
body.pointer("/include"),
Some(&json!(["reasoning.encrypted_content"]))
);
assert_eq!(body.pointer("/reasoning"), Some(&json!({"effort":"max"})));
let request = raw_request(json!({
"model": "client-model",
"input": "hello",
"reasoning": {}
}));
let encoded = GrokResponsesRequest::encode(&request, "grok-composer-2.5-fast", &client_key())
.expect("Composer reasoning without effort");
assert_eq!(
Value::Object(encoded.body().clone()).pointer("/reasoning"),
Some(&json!({}))
);
let request = raw_request(json!({
"model": "client-model",
"input": "hello",
"reasoning": {"effort": "max"}
}));
let encoded = GrokResponsesRequest::encode(&request, "tenant/foo/grok-4.6", &client_key())
.expect("unknown provider prefix");
assert_eq!(
Value::Object(encoded.body().clone()).pointer("/reasoning/effort"),
Some(&json!("max"))
);
assert_eq!(encoded.body()["store"], json!(false));
assert_eq!(encoded.body()["include"], expected_include);
}
}
#[test]
fn encoder_should_strip_codex_only_fields_without_model_guesses() {
fn encoder_should_strip_codex_only_fields_and_preserve_business_extensions() {
let request = raw_request(json!({
"model": "client-model",
"input": "hello",
"prompt_cache_retention": "24h",
"safety_identifier": "user-1",
"presence_penalty": 0.1,
"presencePenalty": 0.2,
"frequency_penalty": 0.3,
"frequencyPenalty": 0.4,
"stop": ["done"],
"provider_options": {"providers": {"xai": {"transport": "websocket"}}},
"future_official_field": {"keep": true},
"external_web_access": true,
"metadata": {"external_web_access": false},
"tools": [{
@@ -299,6 +230,7 @@ fn encoder_should_strip_codex_only_fields_without_model_guesses() {
for pointer in [
"/prompt_cache_retention",
"/safety_identifier",
"/provider_options",
"/external_web_access",
"/metadata/external_web_access",
"/tools/0/external_web_access",
@@ -306,18 +238,10 @@ fn encoder_should_strip_codex_only_fields_without_model_guesses() {
assert_eq!(body.pointer(pointer), None, "field survived at {pointer}");
}
let request = raw_request(json!({
"model": "client-model",
"input": "hello",
"logprobs": true,
"top_logprobs": 5
}));
let encoded = GrokResponsesRequest::encode(&request, "grok-4.20-reasoning", &client_key())
.expect("sanitized 4.20 request");
let body = Value::Object(encoded.body().clone());
assert_eq!(body.pointer("/model"), Some(&json!("grok-4.20-reasoning")));
assert_eq!(body.pointer("/logprobs"), Some(&json!(true)));
assert_eq!(body.pointer("/top_logprobs"), Some(&json!(5)));
assert_eq!(
body.pointer("/future_official_field"),
Some(&json!({"keep": true}))
);
}
#[test]
@@ -444,31 +368,6 @@ fn request_debug_should_not_expose_prompt_or_unknown_values() {
assert!(!debug.contains("must-not-leak"));
}
#[test]
fn encoder_should_remove_retired_gateway_provider_options_before_xai_conversion() {
let request = raw_request(Value::Object(Map::from_iter([
("model".to_owned(), json!("client")),
("input".to_owned(), json!("hello")),
(
"provider_options".to_owned(),
json!({
"version": "future-version",
"providers": {"xai": {"transport": "websocket", "turn_index": "7"}}
}),
),
("future_official_field".to_owned(), json!({"keep": true})),
])));
let encoded =
GrokResponsesRequest::encode(&request, "grok-routed", &client_key()).expect("request");
assert!(encoded.body().get("provider_options").is_none());
assert_eq!(
encoded.body().get("future_official_field"),
Some(&json!({"keep": true}))
);
}
#[test]
fn explicit_session_should_be_tenant_isolated_and_stable_across_turns() {
let first = raw_request(json!({
@@ -1,52 +1,32 @@
//! 路由已选定真实模型,xAI 编码器不再解释旧别名或 Provider 前缀
//! 验证路由模型标识与业务参数在 xAI 请求编码后保持原值
use super::*;
#[test]
fn routed_model_identity_is_preserved_without_provider_local_aliases() {
let request = raw_request(json!({"model": "client-model", "input": "hello"}));
fn routed_model_and_business_parameters_are_preserved() {
let fields = json!({
"presence_penalty": 0.1,
"frequency_penalty": 0.2,
"stop": ["done"],
"logprobs": true,
"top_logprobs": 5
});
let mut body = fields.clone();
body["model"] = json!("client-model");
body["input"] = json!("hello");
let request = raw_request(body);
for model in [
"grok",
"grok-latest",
"grok-build-latest",
"grok-4.5-latest",
"grok-4.6-latest",
"grok-4.3-latest",
"grok-build",
"grok-composer",
"composer-2.5",
"grok-4.20-reasoning",
"grok-4.20-non-reasoning",
"grok-4.20-multi-agent",
"grok-4.20-multi-agent-latest",
"grok-4.7",
"Grok-4.6",
"grok-latest",
"grok-4.20-reasoning",
"Grok-4.7",
"xai/grok-4.7",
"x-ai/grok-4.7",
"grok/grok-4.7",
"tenant/custom-model",
] {
let encoded = GrokResponsesRequest::encode(&request, model, &client_key())
.expect("routed upstream model");
assert_eq!(encoded.body()["model"], model);
}
}
#[test]
fn routed_model_identity_does_not_infer_unsupported_fields_from_old_slugs() {
let fields = json!({"presence_penalty": 0.1, "frequency_penalty": 0.2,
"stop": ["done"], "logprobs": true, "top_logprobs": 5});
let request = raw_request(fields.clone());
for model in [
"grok-4.5",
"grok-4.6",
"grok-4.3",
"grok-4.20-reasoning",
"tenant/grok-4.6",
"grok-4.7",
] {
let encoded =
GrokResponsesRequest::encode(&request, model, &client_key()).expect("request");
for (key, value) in fields.as_object().expect("fields") {
assert_eq!(encoded.body().get(key), Some(value), "{model} {key}");
}
@@ -4,14 +4,7 @@ use super::*;
#[test]
fn official_reasoning_efforts_are_preserved_for_current_and_future_models() {
for model in [
"grok-4.7",
"grok-future",
"grok-4.5",
"grok-4.6",
"grok-3-mini",
"grok-composer-2.5-fast",
] {
for model in ["grok-4.7", "grok-future", "tenant/custom-model"] {
for effort in ["none", "minimal", "low", "medium", "high", "xhigh", "max"] {
let request = raw_request(json!({
"input": "hello",
+3 -2
View File
@@ -48,7 +48,8 @@ export CPR_TEST_REDIS_URL='redis://:<password>@127.0.0.1:6379'
PostgreSQL 的临时 schema 测试连接使用异步提交,保留事务可见性、回滚和完整迁移检查
需要验证数据库崩溃后的持久性时,使用独立数据库与生产连接配置,不复用这类 fixture
运行包含 `StoreBundle` 初始化的完整集成测试时,两条测试 URL 都须包含密码,且密码满足 Store
启动配置的 48 位十六进制要求;仅能连接数据库并不代表该初始化合同通过。专用服务使用对应测试密码,
运行包含 `StoreBundle` 初始化的完整集成测试时,两条测试 URL 都须包含密码,原始密码须满足
[Store 密码要求](../../deploy/README.md#手动安装),在测试 URL 中按 URL 规则编码。
仅能连接数据库并不代表该初始化合同通过。专用服务使用对应测试密码,
并在测试进程中清除 `CPR_DATABASE_URL`、`CPR_REDIS_URL`、`CPR_DATABASE_PASSWORD` 和
`CPR_REDIS_PASSWORD`,避免启动配置被部署环境覆盖
+8 -3
View File
@@ -159,8 +159,13 @@ openssl rand -hex 24
推荐使用模板中的 7 天不活动期限和 30 天最长登录期限。修改配置需重启,不能延长已建立会话的最长有效期。
浏览器会话与续期规则见[认证 API](../docs/api.md#统一登录与会话),独立于 `/v1/*` 鉴权和限额
PostgreSQL 与 Redis 密码必须是 48 位十六进制字符。Compose 通过 `config.yaml` 的凭据桥接区
引用同一密码;三个值都不需要额外导出为环境变量,数据库和 Redis 密码也不能嵌入连接 URL
PostgreSQL 与 Redis 密码统一接受 16–128 字节,不能全为空白或包含控制字符;仍须满足
各自服务端的密码策略。托管服务可使用服务商允许的高强度密码,默认安装继续生成 48 位随机十六进制密码
Compose 通过 `config.yaml` 的凭据桥接区引用同一密码;三个值都不需要额外导出为环境变量,
数据库和 Redis 密码不能嵌入连接 URL。通过环境变量提供密码时分别使用
`CPR_DATABASE_PASSWORD` 与 `CPR_REDIS_PASSWORD`,原始密码由连接 URL 构造器编码,不要预先做 URL 编码。Compose 凭据桥接建议沿用上面的
随机十六进制密码,避免 `$` 的变量插值语义
Linux 上应用容器以 `10001:10001` 运行。上述命令将应用数据和日志目录设为 `0770`,
配置设为 `0640`,均由当前用户持有、容器组 `10001` 访问。
@@ -174,7 +179,7 @@ Linux 上应用容器以 `10001:10001` 运行。上述命令将应用数据和
store:
redis:
url: 'redis://u1@redis-host:6379/0'
password: &redis_password '<u1 的 48 位十六进制密码>'
password: &redis_password '<u1 的密码>'
```
将这两项合并到已有配置,保留其他配置和密码锚点;Redis 服务端需事先创建并授权该用户。
+2
View File
@@ -44,11 +44,13 @@ store:
database:
# Docker 内由 CPR_DATABASE_URL 覆盖;密码不要写进 URL。
url: 'postgres://codex_proxy@127.0.0.1:5432/codex_proxy'
# 16–128 字节,不能全为空白或含控制字符,并满足服务端密码策略
password: &postgres_password ''
redis:
# Docker 内由 CPR_REDIS_URL 覆盖;密码不要写进 URL。
# ACL 用户写在 URL 中,例如 redis://u1@redis-host:6379/0;配置步骤见 deploy/README.md。
url: 'redis://127.0.0.1:6379/'
# 与 PostgreSQL 共用密码格式要求,并满足 Redis 服务端密码策略
password: &redis_password ''
# 可选:PostgreSQL 连接池预算;下列值与当前推荐默认一致。
# pool:
+13 -11
View File
@@ -754,7 +754,10 @@ Codex 的 `model_catalog_json` 配置。账号设置保存不等待上游模型
`quota.credits` 返回上游点数余额,未提供点数信息时为 `null`。对象包含 `hasCredits`、`unlimited` 和
`balance`,余额为保留上游精度的十进制字符串,未提供余额时为 `null`,明确的零余额为 `"0"`。
OpenAI 点数随现有额度刷新和正常请求的额度信息同步,不与主动重置卡库存或百分比限额合并,
不参与账号可用性判断
不根据 `balance` 单独推导可用性。主动查询明确 `hasCredits=true` 或 `unlimited=true`,且无阻止
点数支出的上限或工作区限制时,即使套餐窗口耗尽也可使用点数,并解除已有额度耗尽状态。
真实额度拒绝后的后台回读同样支持此恢复路径,但不会自动重发本次失败请求;点数不能清除
凭据错误、停用或冷却状态。上游点数信息也可能延迟,后续真实拒绝仍会记录额度耗尽
账号列表和详情返回 `notes`(无备注时为 `null`)。编辑时省略或 `null` 保留原备注;字符串最多 500 个 Unicode
字符,允许换行和制表符,保存时去除首尾空白,空字符串清空备注。备注独立于上游身份,导入时未显式提供备注、
@@ -1118,11 +1121,14 @@ OAuth start 使用:
但上游 authorization code 本身通常只能交换一次;已完成过 token exchange 时应重新创建 OAuth flow
- `GET /accounts/quota` 只读取最后一次落库快照;`POST /accounts/quota/refresh` 才访问上游。access token
已过期时,额度刷新要求先走 credential 刷新或重新授权,不会拿过期 token 探测额度
- OpenAI 已耗尽账号每 30 分钟主动复核一次,也会在最早未恢复窗口的 `resetAt + 2 分钟` 到期后
提前复核;周期复核不等待旧重置时间,因此也能发现官方提前重置。正常账号有非零用量或触顶窗口时,
在该窗口的 `resetAt + 2 分钟` 后主动复核。后台每 30 秒检查触发条件;同一重置边界复核后仍未更新时
回到 30 分钟重试,避免旧 reset 持续触发请求。各窗口独立确认恢复,时间到期本身不会直接解除账号
耗尽或将展示用量归零
- OpenAI 已耗尽账号首次可复核,在 `resetAt` 尚未到达或未知时按 30 分钟间隔兜底,也可发现
官方提前重置。到达最早未恢复窗口的 `resetAt` 后,先等待 10 秒传播余量,再在下一轮后台检查
复核;reset 后前 5 分钟最小查询间隔为 30 秒,之后仍耗尽则改为 5 分钟。正常账号有非零用量或触顶窗口时,仍在
`resetAt + 2 分钟` 后复核,未更新则按 30 分钟间隔重试。后台每 30 秒检查条件,实际恢复时间还受
排队、查询耗时和上游传播影响;查询失败也遵守间隔。无可用点数时,到期前仍按原窗口证据恢复;
本次耗尽之后的已记录窗口 reset 全部到期后,新查询明确 `allowed=true` 可解除限制,即使用量仍为
100%。未知 reset 或再次拒绝后残留的旧 reset 仍走原规则,不复用旧边界解封;时间到期本身不会
解除限制,也不会将展示用量归零
- 账号 `enabled` 控制是否参与请求调度,不控制 OAuth 凭据续期。OpenAI 与 xAI 的停用账号仍按各自刷新
策略维护 Token;刷新结果更新凭据状态,但不会启用调度。无 refresh token 或凭据已进入失效、无效、
封禁状态的账号不参加后台自动续期
@@ -2055,7 +2061,7 @@ OpenAI 仅使用同一条 `response.completed` 中的 `created_at` 与 `complete
`latencyDetails` 还可包含以下上游性能指标,单位为毫秒,保留上游返回的小数。
OpenAI 从当前响应的 `responsesapi.websocket_timing.timing_metrics` 读取,
WebSocket 握手默认请求专项计时,客户端显式提供的开关保留原值。
WebSocket 握手不主动添加专项计时请求头,客户端显式提供的开关保留原值。
各字段独立可选;响应开始前、终态后的计时和明确属于其他响应的事件不写入当前请求,重试时清空
`critical_path` 仅采集 `scope=response`、`coverage=complete` 且边界为
@@ -2067,10 +2073,6 @@ WebSocket 握手默认请求专项计时,客户端显式提供的开关保留
| --- | --- |
| `upstreamApiOverheadMs` | API 排除引擎与客户端工具时间后的耗时 |
| `upstreamEngineMs` | 响应级引擎耗时,取完整 critical path 的墙钟时间或直接返回的 Engine Service 时间 |
| `upstreamEngineIapiTtftMs` | Engine IAPI TTFT 总计 |
| `upstreamEngineServiceTtftMs` | Engine Service TTFT 总计 |
| `upstreamEngineIapiTbtMs` | 跨引擎调用的 IAPI Token 间隔 |
| `upstreamEngineServiceTbtMs` | 跨引擎调用的 Service Token 间隔 |
以上指标按上游内部口径展示,不相加重建总耗时,不替换请求级 `firstTokenLatencyMs`。
它们仅补充性能诊断,不改变列表与聚合统计的速率分母

Some files were not shown because too many files have changed in this diff Show More