mirror of
https://github.com/zyycn/codex-proxy-rs.git
synced 2026-11-08 00:05:59 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
bad2a245ab | ||
|
|
b49b4528e7 | ||
|
|
d678f350c1 | ||
|
|
17b05d4671 | ||
|
|
7148303ad6 | ||
|
|
f6538d433d | ||
|
|
406256bcc0 | ||
|
|
d912fa0c59 | ||
|
|
a166580aed | ||
|
|
d1aec7311c | ||
|
|
16d8a3f873 | ||
|
|
ac6033f6fa | ||
|
|
cef9f0b21e | ||
|
|
591f95cc18 | ||
|
|
99ba5b1120 |
@@ -15,6 +15,7 @@ description: Codex Proxy RS 前后端开发、排障、代码审查和文档修
|
||||
| 修改服务端接口、路由、调度、账号与用量规则、持久化、后台任务或协议行为;目标是 `backend/` 中的实现、配置、迁移,或 Rust 源码 | [Workspace 边界](../../../docs/architecture.md#3-workspace-边界)、所属模块、[后端自审](../../../docs/architecture.md#后端自审);涉及 Rust 时使用 `$rust-best-practices` 和下方后端注释要求,验证时读[命令](../../../docs/architecture.md#验证命令) |
|
||||
| 修改页面接口调用,或开发、排查、审查面向管理端或模型客户端的 HTTP/WS API 合同 | [API](../../../docs/api.md)对应章节;优先复用已有接口,核对调用方 → 服务端解析 → 实际查询或执行,不能只凭前端类型认定参数生效;按实际改动叠加前后端规范 |
|
||||
| 模块或调用链变更 | [Workspace 边界](../../../docs/architecture.md#3-workspace-边界)及所属模块章节,不通读整份架构 |
|
||||
| 新增、调整、清理或审查测试 | 下方[测试维护](#测试维护),按当前行为、职责归属与已有覆盖判断保留、合并或删除 |
|
||||
| 修改部署行为、`deploy/` 配置或数据库迁移 | 按范围读取[部署](../../../deploy/README.md)、[迁移](../../../backend/migrations/README.md)对应章节 |
|
||||
| 开发、排查或审查宿主 SDK、Runtime、扩展接口或独立插件;目标在 `backend/crates/gateway-plugin/` | [插件职责边界](references/plugin-boundaries.md)及所用 [SDK](../../../backend/crates/gateway-plugin/sdk/README.md) 类型与能力合同;宿主守住自身合同、资源与稳定性,不猜测或补偿插件业务错误;仅独立插件创作、排查另用 [cpr-plugin-dev](../cpr-plugin-dev/SKILL.md) |
|
||||
| 修改或审查文档 | [文档检查](references/documentation.md)及目标章节,不默认读取代码开发、页面验收或发布流程 |
|
||||
@@ -35,6 +36,16 @@ description: Codex Proxy RS 前后端开发、排障、代码审查和文档修
|
||||
4. 按变更范围执行[验证](../../../CONTRIBUTING.md#验证)及其停止条件,沿用本轮有效证据与用户已确定的验收分工,区分通过、失败、跳过与未执行;提交时补读[提交约定](../../../CONTRIBUTING.md#项目约定)
|
||||
5. 交付实际结果与验证缺口;前端按[前端流程](references/frontend.md)核对最终差异和运行证据,后端按上表检查生命周期,不把构建通过当作行为验收
|
||||
|
||||
## 测试维护
|
||||
|
||||
- 测试随当前功能维护,名称、夹具和断言描述当前输入、行为与结果;功能变更时优先更新所属模块的现有用例,不为每次改动叠加一组记录变更经过的测试
|
||||
- 清理只证明旧字段、旧路由或旧实现已经消失的用例与附带断言;未知字段校验使用合法请求基线加无业务含义的未知字段,避免因缺少必填项等其他错误而误通过
|
||||
- 同一层对同一行为的重复覆盖合并为清晰的场景矩阵,复用相邻测试夹具;跨层测试确有不同职责或失败路径时保留,不为统一而扩大测试辅助抽象
|
||||
- 断言可观察的行为与业务不变量,避免复述结构体赋值、罗列表列或字段清单、匹配 SQL 文本与源码布局;项目明确要求的架构边界检查继续维护
|
||||
- 当前协议兼容、数据读取兼容、安全边界、并发和资源生命周期仍需覆盖,不按 `old`、`legacy` 等名称机械删除,也不能用改名代替无效用例清理
|
||||
- 不在仓库中新增或保留迁移 SQL 专项测试,包括迁移文本、schema 演变与指定版本升级用例;需要验证迁移执行、数据保留或回滚时,在临时目录编写一次性验证并使用隔离数据库,测试通过即可,临时脚本不纳入维护测试集
|
||||
- 持久化功能测试使用当前完整 schema,验证现行读写、事务与数据约束;插件运行时状态迁移等业务能力按当前功能测试,不归为迁移 SQL 专项测试
|
||||
|
||||
## 后端注释
|
||||
|
||||
- 每个 Rust 文件顶层使用简短的中文 `//!` 注释说明文件职责,覆盖生产源码、测试与构建脚本;测试文件说明测试范围或辅助用途,生成文件同步维护生成器
|
||||
|
||||
@@ -52,7 +52,7 @@ updates:
|
||||
- patch
|
||||
|
||||
- package-ecosystem: npm
|
||||
directory: /
|
||||
directory: /frontend
|
||||
labels: []
|
||||
schedule:
|
||||
interval: weekly
|
||||
|
||||
@@ -56,7 +56,7 @@ jobs:
|
||||
const marker = '<!-- agents-md-guard -->';
|
||||
const body = [
|
||||
marker,
|
||||
'佬友,我看到这个 PR 涉及受保护的协作规则文件,按仓库约定需要先关闭',
|
||||
'我看到这个 PR 涉及受保护的协作规则文件,按仓库约定需要先关闭',
|
||||
'',
|
||||
'保护范围包括根目录或子目录的 `AGENTS.md`、根目录的 `CONTRIBUTING.md`,以及 `.agents/skills/` 和 `.github/` 中的所有文件,涵盖新增、修改、删除和重命名',
|
||||
'',
|
||||
|
||||
+2
-1
@@ -23,7 +23,8 @@
|
||||
|
||||
AI 草拟或发布 Issue、PR 讨论及审查意见的回复时:
|
||||
|
||||
- 用第一人称“我”,自然称呼对方为“佬友”,不逐句重复,也不用“我们”“本助手”代称自己
|
||||
- 用第一人称“我”,不用“我们”“本助手”代称自己
|
||||
- 普通回复直接回应问题,不加问候语或称呼;向对方提问时称呼“佬”,同一条回复不重复称呼
|
||||
- 客气、委婉地回应具体问题,补充信息或提出不同意见时说明缘由,避免命令式、责备式表达
|
||||
- 使用自然短句,简单回复用短段,并列信息较多时再分点;少用句号,避免客服套话、机械模板和空泛保证
|
||||
- 只把实际完成的排查、复现、修改和验证写成“我已……”;未采纳的意见说明依据,未确认、待处理或未验证的内容如实交代,说清结论、限制和下一步
|
||||
|
||||
Generated
+21
-19
@@ -198,7 +198,7 @@ checksum = "82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.9",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -856,7 +856,7 @@ dependencies = [
|
||||
"heck",
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.9",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -910,7 +910,7 @@ dependencies = [
|
||||
"provider-xai",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.9",
|
||||
"tempfile",
|
||||
"thiserror",
|
||||
"tokio",
|
||||
@@ -1657,7 +1657,7 @@ checksum = "9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.9",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1720,7 +1720,7 @@ dependencies = [
|
||||
"serde_json",
|
||||
"sha2",
|
||||
"subtle",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.9",
|
||||
"tempfile",
|
||||
"thiserror",
|
||||
"tokio",
|
||||
@@ -1915,6 +1915,7 @@ dependencies = [
|
||||
"gateway-admin",
|
||||
"gateway-core",
|
||||
"hex",
|
||||
"percent-encoding",
|
||||
"redis",
|
||||
"secrecy",
|
||||
"serde",
|
||||
@@ -3266,7 +3267,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2bfe0f4c752e450fc2faf62654f1c134747922825d5b04ca717b8874f41a40c0"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.9",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -3626,7 +3627,7 @@ checksum = "92ecd8964f8453721699a1ed72037b0db49ce2f5a5138486ee89bed6f67cdf3a"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.9",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -3999,7 +4000,7 @@ checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.9",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -4014,10 +4015,11 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "serde_json"
|
||||
version = "1.0.151"
|
||||
version = "1.0.154"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
|
||||
checksum = "e7e9cc8b1b85264074fbcc02a88680c4096b1e47df8f739dceb03bf482f04bd6"
|
||||
dependencies = [
|
||||
"foldhash",
|
||||
"indexmap",
|
||||
"itoa",
|
||||
"memchr",
|
||||
@@ -4098,9 +4100,9 @@ checksum = "bbfa15b3dddfee50a0fff136974b3e1bde555604ba463834a7eb7deb6417705d"
|
||||
|
||||
[[package]]
|
||||
name = "sha2"
|
||||
version = "0.11.0"
|
||||
version = "0.11.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4"
|
||||
checksum = "47d7069beb7d6ac7b9acd1039986e73443f24234f41074da099d6f994ac9ad19"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"cpufeatures 0.3.0",
|
||||
@@ -4480,9 +4482,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "syn"
|
||||
version = "3.0.6"
|
||||
version = "3.0.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8593e8e72159ed2257d083c7a454a85cbf854f37a0966d8d483aff8c8a3ebcee"
|
||||
checksum = "d78c8dee4c7bf0e14673097256fed6142ce9d3b85a408189d07482442145823b"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
@@ -4580,7 +4582,7 @@ checksum = "fe5197923287db20a58125f0bc85c062f7f2c892de97b18c356f9efb14b28524"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
"syn 3.0.6",
|
||||
"syn 3.0.9",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -4734,9 +4736,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "tokio-util"
|
||||
version = "0.7.19"
|
||||
version = "0.7.20"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "494815d09bf52b5548659851081238f0ca39ff638363907596da739561c62c52"
|
||||
checksum = "e464cf451ba96ebfc6f9b6542f17ee8b8956e33f1e40d9690624e59d7a7f8a4b"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"futures-core",
|
||||
@@ -5051,9 +5053,9 @@ checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
|
||||
|
||||
[[package]]
|
||||
name = "uuid"
|
||||
version = "1.27.0"
|
||||
version = "1.28.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "97277d36b9c3ace13e58fa6e753f8b0bbbf302a18dd193240d70f9e29681059a"
|
||||
checksum = "7cc1186384beb7dd8eedea376413fd654937285ea6c9cfbb928dc3043ea4b606"
|
||||
dependencies = [
|
||||
"getrandom 0.4.3",
|
||||
"js-sys",
|
||||
|
||||
+1
-1
@@ -43,7 +43,7 @@ opt-level = 2
|
||||
[profile.dev.package.blake2]
|
||||
opt-level = 2
|
||||
|
||||
[profile.dev.package."sha2@0.11.0"]
|
||||
[profile.dev.package."sha2@0.11.1"]
|
||||
opt-level = 2
|
||||
|
||||
[profile.dev.package."miniz_oxide@0.9.1"]
|
||||
|
||||
@@ -106,8 +106,11 @@ fn config_loader_should_apply_only_explicit_topology_overrides() {
|
||||
"postgres://codex_proxy@127.0.0.1:5432/codex_proxy",
|
||||
)
|
||||
.env("CPR_REDIS_URL", "redis://127.0.0.1:6379/")
|
||||
.env("CPR_DATABASE_PASSWORD", POSTGRES_PASSWORD)
|
||||
.env("CPR_REDIS_PASSWORD", REDIS_PASSWORD)
|
||||
.env(
|
||||
"CPR_DATABASE_PASSWORD",
|
||||
"RdsTest9!abcdefGhRdsTest9!abcdefGh",
|
||||
)
|
||||
.env("CPR_REDIS_PASSWORD", "ManagedRedis9!abcdefGh")
|
||||
.status()
|
||||
.expect("run isolated environment override test");
|
||||
assert!(status.success());
|
||||
@@ -152,14 +155,6 @@ fn config_loader_should_ignore_unknown_fields_in_configuration_sections() {
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn config_loader_should_ignore_removed_tls_and_fingerprint_sections() {
|
||||
let mut document = valid_config_document();
|
||||
document["openai"]["tls"] = serde_json::json!({});
|
||||
document["openai"]["fingerprint"] = serde_json::json!({"browser": "removed"});
|
||||
parse_config(&document.to_string()).expect("removed provider settings are ignored");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn config_loader_should_reject_invalid_known_fields_alongside_unknown_fields() {
|
||||
for port in [serde_json::json!(0), serde_json::json!("not-a-port")] {
|
||||
@@ -189,7 +184,7 @@ fn config_loader_should_report_startup_configuration_diagnostics() {
|
||||
] {
|
||||
let mut document = valid_config_document();
|
||||
if case == "unused" {
|
||||
document["openai"]["wire_profile"]["location"] = serde_json::json!(UNUSED_SECRET);
|
||||
document["openai"]["unused_setting"] = serde_json::json!(UNUSED_SECRET);
|
||||
} else if case == "missing" {
|
||||
document["host"]["listen"]
|
||||
.as_object_mut()
|
||||
@@ -229,7 +224,7 @@ fn config_loader_should_report_startup_configuration_diagnostics() {
|
||||
);
|
||||
match case {
|
||||
"unused" => assert!(
|
||||
stderr.contains("openai.wire_profile") && stderr.contains("已忽略"),
|
||||
stderr.contains("openai.unused_setting") && stderr.contains("已忽略"),
|
||||
"{stderr}"
|
||||
),
|
||||
"missing" => assert!(stderr.contains("host.listen.port"), "{stderr}"),
|
||||
@@ -253,19 +248,7 @@ fn config_loader_should_allow_missing_provider_configuration() {
|
||||
let mut document = valid_config_document();
|
||||
document.as_object_mut().unwrap().remove("openai");
|
||||
document.as_object_mut().unwrap().remove("xai");
|
||||
parse_config(&document.to_string()).expect("provider defaults require no YAML identity");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn config_loader_should_ignore_removed_provider_identity_sections() {
|
||||
let mut document = valid_config_document();
|
||||
document["openai"]["wire_profile"] = serde_json::json!({"codex_version":"legacy-invalid"});
|
||||
document["xai"] = serde_json::json!({"wire_profile": {"client_identifier": "legacy-client"}});
|
||||
let (config, _directory) =
|
||||
parse_config(&document.to_string()).expect("removed identities ignored");
|
||||
let debug = format!("{config:?}");
|
||||
assert!(!debug.contains("legacy-invalid"));
|
||||
assert!(!debug.contains("legacy-client"));
|
||||
parse_config(&document.to_string()).expect("providers use defaults when omitted");
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -282,13 +265,53 @@ fn config_loader_should_reject_embedded_database_password() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn config_loader_should_reject_non_hex_postgres_password() {
|
||||
assert_rejected(valid_config().replace(POSTGRES_PASSWORD, &"g".repeat(48)));
|
||||
fn config_loader_should_accept_managed_store_passwords() {
|
||||
for password in [
|
||||
"RdsTest9!abcdefGh".to_owned(),
|
||||
"RdsTest9!abcdefGhRdsTest9!abcdefGh".to_owned(),
|
||||
POSTGRES_PASSWORD.to_owned(),
|
||||
"x".repeat(128),
|
||||
"Managed@:/?#%&+Password9!".to_owned(),
|
||||
"数据库密码".repeat(4),
|
||||
] {
|
||||
for field in ["database", "redis"] {
|
||||
let mut document = valid_config_document();
|
||||
document["store"][field]["password"] = serde_json::json!(password);
|
||||
let (config, _directory) =
|
||||
parse_config(&document.to_string()).expect("managed store password");
|
||||
assert!(!format!("{config:?}").contains(&password));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn config_loader_should_reject_wrong_length_redis_password() {
|
||||
assert_rejected(valid_config().replace(REDIS_PASSWORD, "1234"));
|
||||
fn config_loader_should_reject_invalid_store_passwords() {
|
||||
for password in [
|
||||
String::new(),
|
||||
"x".repeat(15),
|
||||
"x".repeat(129),
|
||||
" ".repeat(32),
|
||||
"\u{3000}".repeat(6),
|
||||
"数据库密码".repeat(9),
|
||||
"RdsTest9!abcdefGh\u{0000}".to_owned(),
|
||||
"RdsTest9!abcdefGh\n".to_owned(),
|
||||
"RdsTest9!abcdefGh\t".to_owned(),
|
||||
"RdsTest9!abcdefGh\u{007f}".to_owned(),
|
||||
] {
|
||||
for field in ["database", "redis"] {
|
||||
let mut document = valid_config_document();
|
||||
document["store"][field]["password"] = serde_json::json!(password);
|
||||
assert_rejected(document.to_string());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn config_loader_should_reject_embedded_redis_password() {
|
||||
assert_rejected(valid_config().replace(
|
||||
"redis://127.0.0.1:6379/",
|
||||
"redis://default:embedded@127.0.0.1:6379/",
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -722,10 +722,6 @@ pub struct UsageListRecord {
|
||||
pub upstream_response_ms: Option<u64>,
|
||||
pub upstream_api_overhead_ms: Option<f64>,
|
||||
pub upstream_engine_ms: Option<f64>,
|
||||
pub upstream_engine_iapi_ttft_ms: Option<f64>,
|
||||
pub upstream_engine_service_ttft_ms: Option<f64>,
|
||||
pub upstream_engine_iapi_tbt_ms: Option<f64>,
|
||||
pub upstream_engine_service_tbt_ms: Option<f64>,
|
||||
pub latency_ms: Option<u64>,
|
||||
pub admission_decision_ms: Option<u64>,
|
||||
pub account_selection_wait_ms: Option<u64>,
|
||||
@@ -803,10 +799,6 @@ pub struct UsageRecord {
|
||||
pub upstream_response_ms: Option<u64>,
|
||||
pub upstream_api_overhead_ms: Option<f64>,
|
||||
pub upstream_engine_ms: Option<f64>,
|
||||
pub upstream_engine_iapi_ttft_ms: Option<f64>,
|
||||
pub upstream_engine_service_ttft_ms: Option<f64>,
|
||||
pub upstream_engine_iapi_tbt_ms: Option<f64>,
|
||||
pub upstream_engine_service_tbt_ms: Option<f64>,
|
||||
pub latency_ms: Option<u64>,
|
||||
pub admission_decision_ms: Option<u64>,
|
||||
pub account_selection_wait_ms: Option<u64>,
|
||||
|
||||
@@ -5,20 +5,16 @@ use std::str::FromStr as _;
|
||||
use chrono::{TimeDelta, Utc};
|
||||
use gateway_core::account::OpaqueProviderData;
|
||||
|
||||
use gateway_admin::{
|
||||
model::{
|
||||
PageSize, Revision,
|
||||
accounts::AccountUsage,
|
||||
auth::LoginCommand,
|
||||
client_keys::ClientKeyPageSize,
|
||||
observability::{DecimalAmount, RequestOutcome, TimeRange},
|
||||
provider_credentials::{
|
||||
CredentialCommitGuard, ProviderDocument, ProviderQuota, ProviderQuotaWindow,
|
||||
QuotaLocalUsageAttribution,
|
||||
},
|
||||
settings::AdminApiKey,
|
||||
use gateway_admin::model::{
|
||||
PageSize, Revision,
|
||||
accounts::AccountUsage,
|
||||
auth::LoginCommand,
|
||||
client_keys::ClientKeyPageSize,
|
||||
observability::{DecimalAmount, RequestOutcome, TimeRange},
|
||||
provider_credentials::{
|
||||
ProviderDocument, ProviderQuota, ProviderQuotaWindow, QuotaLocalUsageAttribution,
|
||||
},
|
||||
ports::store::{AccountStore, AuthStore, ClientKeyStore, ObservabilityStore, SettingsStore},
|
||||
settings::AdminApiKey,
|
||||
};
|
||||
|
||||
mod backup;
|
||||
@@ -99,20 +95,6 @@ fn login_command_debug_should_redact_password() {
|
||||
assert!(!format!("{command:?}").contains("secret-password"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn all_store_capability_traits_should_be_object_safe() {
|
||||
fn assert_object_safe<T: ?Sized + Send + Sync>() {}
|
||||
|
||||
assert_object_safe::<dyn AccountStore>();
|
||||
assert_object_safe::<dyn AuthStore>();
|
||||
assert_object_safe::<dyn ClientKeyStore>();
|
||||
assert_object_safe::<dyn ObservabilityStore>();
|
||||
assert_object_safe::<dyn SettingsStore>();
|
||||
|
||||
fn assert_send_object_safe<T: ?Sized + Send>() {}
|
||||
assert_send_object_safe::<dyn CredentialCommitGuard>();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn provider_document_debug_should_not_expose_opaque_material() {
|
||||
let document = ProviderDocument::new(OpaqueProviderData::new(Default::default()));
|
||||
|
||||
@@ -1390,10 +1390,6 @@ fn total_record(
|
||||
upstream_response_ms: None,
|
||||
upstream_api_overhead_ms: None,
|
||||
upstream_engine_ms: None,
|
||||
upstream_engine_iapi_ttft_ms: None,
|
||||
upstream_engine_service_ttft_ms: None,
|
||||
upstream_engine_iapi_tbt_ms: None,
|
||||
upstream_engine_service_tbt_ms: None,
|
||||
latency_ms: Some(100),
|
||||
admission_decision_ms: None,
|
||||
account_selection_wait_ms: None,
|
||||
|
||||
@@ -21,7 +21,7 @@ fn counters_preserve_integer_precision_and_reject_json_numbers() {
|
||||
}
|
||||
}
|
||||
#[tokio::test]
|
||||
async fn usage_mutations_require_trusted_actor_and_old_workflow_operations_are_absent() {
|
||||
async fn usage_mutations_require_a_trusted_actor() {
|
||||
let admin = super::AdminHarness::new().build().await;
|
||||
let registry = admin.public_services();
|
||||
let denied = registry
|
||||
@@ -38,21 +38,6 @@ async fn usage_mutations_require_trusted_actor_and_old_workflow_operations_are_a
|
||||
.await
|
||||
.is_err()
|
||||
);
|
||||
let ops = registry
|
||||
.operations()
|
||||
.filter(|op| op.starts_with("usage.") || *op == "pricing.evaluate")
|
||||
.collect::<std::collections::BTreeSet<_>>();
|
||||
assert_eq!(
|
||||
ops,
|
||||
std::collections::BTreeSet::from([
|
||||
"usage.records.get",
|
||||
"usage.records.list",
|
||||
"usage.records.write",
|
||||
"usage.aggregates.list",
|
||||
"usage.aggregates.write",
|
||||
"pricing.evaluate"
|
||||
])
|
||||
);
|
||||
}
|
||||
#[test]
|
||||
fn sdk_wire_types_match_host_contracts() {
|
||||
|
||||
@@ -304,10 +304,6 @@ pub(crate) fn usage_list_record_view(
|
||||
upstream_response_ms: record.upstream_response_ms,
|
||||
upstream_api_overhead_ms: record.upstream_api_overhead_ms,
|
||||
upstream_engine_ms: record.upstream_engine_ms,
|
||||
upstream_engine_iapi_ttft_ms: record.upstream_engine_iapi_ttft_ms,
|
||||
upstream_engine_service_ttft_ms: record.upstream_engine_service_ttft_ms,
|
||||
upstream_engine_iapi_tbt_ms: record.upstream_engine_iapi_tbt_ms,
|
||||
upstream_engine_service_tbt_ms: record.upstream_engine_service_tbt_ms,
|
||||
},
|
||||
first_token_latency_ms: record.first_token_ms,
|
||||
latency_ms: record.latency_ms,
|
||||
@@ -411,10 +407,6 @@ pub(crate) fn usage_record_view(
|
||||
upstream_response_ms: record.upstream_response_ms,
|
||||
upstream_api_overhead_ms: record.upstream_api_overhead_ms,
|
||||
upstream_engine_ms: record.upstream_engine_ms,
|
||||
upstream_engine_iapi_ttft_ms: record.upstream_engine_iapi_ttft_ms,
|
||||
upstream_engine_service_ttft_ms: record.upstream_engine_service_ttft_ms,
|
||||
upstream_engine_iapi_tbt_ms: record.upstream_engine_iapi_tbt_ms,
|
||||
upstream_engine_service_tbt_ms: record.upstream_engine_service_tbt_ms,
|
||||
},
|
||||
attempt_index: None,
|
||||
attempt_count: u64::from(record.attempt_count),
|
||||
|
||||
@@ -230,14 +230,6 @@ pub struct UsageLatencyDetailsView {
|
||||
pub upstream_api_overhead_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub upstream_engine_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub upstream_engine_iapi_ttft_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub upstream_engine_service_ttft_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub upstream_engine_iapi_tbt_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub upstream_engine_service_tbt_ms: Option<f64>,
|
||||
}
|
||||
|
||||
/// 单次上游尝试展示
|
||||
|
||||
@@ -213,14 +213,6 @@ struct OutputTimingView {
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
upstream_engine_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
upstream_engine_iapi_ttft_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
upstream_engine_service_ttft_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
upstream_engine_iapi_tbt_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
upstream_engine_service_tbt_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
first_event_ms: Option<u64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
first_reasoning_ms: Option<u64>,
|
||||
@@ -260,10 +252,6 @@ fn usage_record(value: UsageListRecord, time: crate::time::TimePresenter) -> Rec
|
||||
upstream_response_ms: value.upstream_response_ms,
|
||||
upstream_api_overhead_ms: value.upstream_api_overhead_ms,
|
||||
upstream_engine_ms: value.upstream_engine_ms,
|
||||
upstream_engine_iapi_ttft_ms: value.upstream_engine_iapi_ttft_ms,
|
||||
upstream_engine_service_ttft_ms: value.upstream_engine_service_ttft_ms,
|
||||
upstream_engine_iapi_tbt_ms: value.upstream_engine_iapi_tbt_ms,
|
||||
upstream_engine_service_tbt_ms: value.upstream_engine_service_tbt_ms,
|
||||
first_event_ms: value.first_event_ms,
|
||||
first_reasoning_ms: value.first_reasoning_ms,
|
||||
first_text_ms: value.first_text_ms,
|
||||
|
||||
@@ -58,7 +58,8 @@ fn decode_response_create_inner(
|
||||
else {
|
||||
return Err(ResponseCreateFrameError::ExpectedObject);
|
||||
};
|
||||
match body.remove("type") {
|
||||
// 保持剩余字段顺序,避免末尾元数据交换到模型等路由字段之前
|
||||
match body.shift_remove("type") {
|
||||
Some(Value::String(message_type)) if message_type == "response.create" => {}
|
||||
_ => return Err(ResponseCreateFrameError::UnsupportedType),
|
||||
}
|
||||
|
||||
@@ -46,7 +46,6 @@ async fn list_route_should_keep_camel_case_group_and_page_wire() {
|
||||
value["data"]["items"][0]["usage"]["retainedTotalUsd"],
|
||||
"5.5"
|
||||
);
|
||||
assert!(value["data"]["items"][0].get("member_count").is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
||||
@@ -149,7 +149,7 @@ fn account_page_item(
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn account_usage_returns_distinct_cache_read_and_write_tokens_without_aliases() {
|
||||
async fn account_usage_preserves_cache_read_and_write_counts_and_missing_values() {
|
||||
use chrono::Utc;
|
||||
use gateway_admin::model::accounts::AccountUsage;
|
||||
use gateway_core::account::{AccountStatusFacts, CredentialState, QuotaState};
|
||||
@@ -215,37 +215,9 @@ async fn account_usage_returns_distinct_cache_read_and_write_tokens_without_alia
|
||||
usage["cacheWriteInputTokensDisplay"],
|
||||
if write.is_some() { "7" } else { "—" }
|
||||
);
|
||||
for alias in [
|
||||
"createdTokens",
|
||||
"createdTokensDisplay",
|
||||
"readTokens",
|
||||
"readTokensDisplay",
|
||||
] {
|
||||
assert!(usage.get(alias).is_none(), "obsolete token alias: {alias}");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn standalone_credential_rotation_route_is_not_exposed() {
|
||||
let fixture = AdminTestFixture::new().await;
|
||||
fixture.auth.insert_session("valid-session");
|
||||
let response = admin::router::<AdminTestState>()
|
||||
.with_state(fixture.state())
|
||||
.oneshot(
|
||||
Request::builder()
|
||||
.method("POST")
|
||||
.uri("/api/admin/accounts/rotate")
|
||||
.header(header::COOKIE, "cpr_session=valid-session")
|
||||
.header(header::CONTENT_TYPE, "application/json")
|
||||
.body(Body::from("{}"))
|
||||
.unwrap(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(response.status(), StatusCode::NOT_FOUND);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn connection_update_requires_admin_and_validates_before_calling_the_service() {
|
||||
let fixture = AdminTestFixture::new().await;
|
||||
|
||||
@@ -248,9 +248,8 @@ mod profile_statistics {
|
||||
value["activityInsights"]["invocations"][0]["usageCount"],
|
||||
29
|
||||
);
|
||||
assert!(value.get("cycle").is_none());
|
||||
assert!(value.get("models").is_none());
|
||||
assert!(value.get("estimatedCost").is_none());
|
||||
assert_eq!(value["activityInsights"]["fastModePercent"], 0.0);
|
||||
assert!(value["activityInsights"]["invocations"][0]["skillId"].is_null());
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -375,7 +374,7 @@ mod batch_update {
|
||||
assert!(request.concurrency_limit.is_none());
|
||||
assert!(
|
||||
serde_json::from_value::<BatchUpdateAccountsRequest>(json!({
|
||||
"accountIds": ["acct_test"], "enabled": true, "legacy": true
|
||||
"accountIds": ["acct_test"], "enabled": true, "unknownField": true
|
||||
}))
|
||||
.is_err()
|
||||
);
|
||||
@@ -583,12 +582,12 @@ mod actions {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn account_actions_should_require_frozen_account_ids_and_reject_unknown_revision() {
|
||||
fn account_actions_should_validate_account_ids_and_reject_unknown_fields() {
|
||||
let id: AccountIdQuery =
|
||||
serde_json::from_value(json!({ "accountId": "acct_1" })).expect("decode ID query");
|
||||
assert!(id.validate().is_ok());
|
||||
let action: AccountActionRequest =
|
||||
serde_json::from_value(json!({ "accountId": "legacy-id" })).expect("decode action");
|
||||
serde_json::from_value(json!({ "accountId": "invalid-id" })).expect("decode action");
|
||||
assert_eq!(action.validate().unwrap_err().field(), "accountId");
|
||||
let refresh: AccountRefreshRequest = serde_json::from_value(json!({
|
||||
"accountId": "acct_1"
|
||||
@@ -598,7 +597,7 @@ mod actions {
|
||||
assert!(
|
||||
serde_json::from_value::<AccountRefreshRequest>(json!({
|
||||
"accountId": "acct_1",
|
||||
"expectedConfigRevision": 0
|
||||
"unknownField": true
|
||||
}))
|
||||
.is_err()
|
||||
);
|
||||
@@ -772,15 +771,14 @@ mod actions {
|
||||
assert_eq!(invalid.validate().unwrap_err().field(), "data");
|
||||
assert!(
|
||||
serde_json::from_value::<AccountImportRequest>(json!({
|
||||
"provider": "openai",
|
||||
"document": {}
|
||||
"provider": "openai"
|
||||
}))
|
||||
.is_err()
|
||||
);
|
||||
assert!(
|
||||
serde_json::from_value::<AccountImportRequest>(json!({
|
||||
"provider": "openai",
|
||||
"expectedConfigRevision": 7,
|
||||
"unknownField": true,
|
||||
"data": {}
|
||||
}))
|
||||
.is_err()
|
||||
|
||||
@@ -11,7 +11,7 @@ use gateway_api::admin::accounts::{
|
||||
use gateway_core::operation::RawJsonPayload;
|
||||
|
||||
#[test]
|
||||
fn quota_forecast_projection_only_exposes_capacity_and_preserves_null_zero() {
|
||||
fn quota_forecast_projection_formats_capacity_and_preserves_null_zero() {
|
||||
let now = "2026-09-12T00:00:00Z".parse().unwrap();
|
||||
let forecast = AccountQuotaForecast {
|
||||
period: AccountUsagePeriod::Weekly,
|
||||
@@ -54,16 +54,6 @@ fn quota_forecast_projection_only_exposes_capacity_and_preserves_null_zero() {
|
||||
assert_eq!(week["estimatedTokensDisplay"], "1M");
|
||||
assert!(week["estimatedUsd"].is_null());
|
||||
assert_eq!(week["estimatedUsdDisplay"], "—");
|
||||
for forecast in value["forecasts"].as_array().unwrap() {
|
||||
for field in [
|
||||
"remainingTokens",
|
||||
"remainingTokensDisplay",
|
||||
"remainingUsd",
|
||||
"remainingUsdDisplay",
|
||||
] {
|
||||
assert!(forecast.get(field).is_none(), "unexpected field: {field}");
|
||||
}
|
||||
}
|
||||
assert_eq!(
|
||||
week["source"],
|
||||
serde_json::json!({
|
||||
@@ -77,9 +67,6 @@ fn quota_forecast_projection_only_exposes_capacity_and_preserves_null_zero() {
|
||||
"usdDisplay": "$0.1234"
|
||||
})
|
||||
);
|
||||
assert!(week.get("method").is_none());
|
||||
assert!(week.get("methodDisplay").is_none());
|
||||
assert!(value.get("generatedAtDisplay").is_none());
|
||||
assert_eq!(value["forecasts"][1]["estimatedTokens"], 0);
|
||||
assert_eq!(value["forecasts"][1]["estimatedTokensDisplay"], "0");
|
||||
assert_eq!(value["forecasts"][1]["estimatedUsd"], 0.0);
|
||||
@@ -87,7 +74,6 @@ fn quota_forecast_projection_only_exposes_capacity_and_preserves_null_zero() {
|
||||
assert_eq!(value["forecasts"][1]["period"], "monthly");
|
||||
assert_eq!(value["forecasts"][1]["targetDays"], 30.0);
|
||||
assert_eq!(value["forecasts"][1]["extrapolated"], true);
|
||||
assert!(value.get("account").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -375,18 +375,17 @@ fn client_key_mutations_should_validate_text_limits_and_unknown_fields() {
|
||||
assert!(
|
||||
serde_json::from_value::<UpdateClientKeyRequest>(json!({
|
||||
"id": "key_1",
|
||||
"expectedConfigRevision": 1,
|
||||
"name": "key",
|
||||
"groupIds": [],
|
||||
"maxConcurrency": 0,
|
||||
"requestsPerMinute": 0,
|
||||
"tokensPerMinute": 0
|
||||
"unknownField": true
|
||||
}))
|
||||
.is_err()
|
||||
);
|
||||
assert!(
|
||||
serde_json::from_value::<CreateClientKeyRequest>(json!({
|
||||
"expectedConfigRevision": 7,
|
||||
"unknownField": true,
|
||||
"name": "terminal key",
|
||||
"groupIds": [],
|
||||
"maxConcurrency": 2,
|
||||
@@ -458,7 +457,6 @@ fn client_key_responses_should_keep_shape_and_redact_creation_debug() {
|
||||
));
|
||||
let list =
|
||||
serde_json::to_value(ClientKeyListData::new(vec![view], None, 1)).expect("serialize list");
|
||||
assert!(list.get("configRevision").is_none());
|
||||
assert_eq!(list["total"], 1);
|
||||
assert_eq!(list["items"][0]["id"], "key_visible");
|
||||
assert_eq!(
|
||||
@@ -468,12 +466,9 @@ fn client_key_responses_should_keep_shape_and_redact_creation_debug() {
|
||||
assert_eq!(list["items"][0]["routingScope"], "all");
|
||||
assert_eq!(list["items"][0]["maxConcurrency"], 2);
|
||||
assert_eq!(list["items"][0]["requestsPerMinute"], 60);
|
||||
assert!(list["items"][0].get("tokensPerMinute").is_none());
|
||||
assert!(list["items"][0].get("policyJson").is_none());
|
||||
assert_eq!(list["items"][0]["dailyLimitUsd"], "0");
|
||||
assert_eq!(list["items"][0]["weeklyLimitUsd"], "0");
|
||||
assert_eq!(list["items"][0]["dailyUsedUsd"], "0");
|
||||
assert!(list["items"][0].get("unresolvedRequests").is_none());
|
||||
assert_eq!(
|
||||
DateTime::parse_from_rfc3339(
|
||||
list["items"][0]["lastUsedAt"]
|
||||
|
||||
@@ -7,7 +7,7 @@ use gateway_api::admin::observability::{
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn dashboard_query_should_parse_terminal_trend_kinds() {
|
||||
fn dashboard_query_should_parse_error_trends() {
|
||||
let query: DashboardQuery = serde_json::from_value(json!({"kind": "errors"})).unwrap();
|
||||
assert_eq!(query.trend_kind().unwrap(), TrendKind::Errors);
|
||||
}
|
||||
@@ -19,7 +19,7 @@ fn dashboard_query_should_reject_unknown_trend_kind() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn usage_query_should_accept_element_plus_pagination() {
|
||||
fn usage_query_should_accept_page_number_and_size() {
|
||||
let query: UsageQuery = serde_json::from_value(json!({
|
||||
"currentPage": 129,
|
||||
"pageSize": 10
|
||||
@@ -29,32 +29,20 @@ fn usage_query_should_accept_element_plus_pagination() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn usage_query_should_reject_removed_total_toggle() {
|
||||
assert!(serde_json::from_value::<UsageQuery>(json!({"includeTotal": true})).is_err());
|
||||
fn observability_queries_should_reject_unknown_fields() {
|
||||
let query = json!({"unknownFilter": true});
|
||||
assert!(serde_json::from_value::<DashboardQuery>(query.clone()).is_err());
|
||||
assert!(serde_json::from_value::<UsageQuery>(query.clone()).is_err());
|
||||
assert!(serde_json::from_value::<OpsQuery>(query.clone()).is_err());
|
||||
assert!(serde_json::from_value::<DiagnosticsQuery>(query).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn usage_query_should_reject_removed_cursor_contract() {
|
||||
assert!(serde_json::from_value::<UsageQuery>(json!({"cursor": "opaque"})).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn usage_query_should_reject_nonstandard_page_name() {
|
||||
assert!(serde_json::from_value::<UsageQuery>(json!({"page": 1})).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ops_query_should_reject_page_size_above_terminal_limit() {
|
||||
fn ops_query_should_reject_page_size_above_maximum() {
|
||||
let query: OpsQuery = serde_json::from_value(json!({"pageSize": 101})).unwrap();
|
||||
assert_eq!(query.validate_pagination().unwrap_err().field(), "pageSize");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ops_query_should_reject_removed_filters() {
|
||||
assert!(serde_json::from_value::<OpsQuery>(json!({"route": "/v1/responses"})).is_err());
|
||||
assert!(serde_json::from_value::<OpsQuery>(json!({"failureClass": "rate_limited"})).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn diagnostics_query_should_keep_wire_dimension_name() {
|
||||
let query: DiagnosticsQuery =
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
//! 响应 DTO 与固定 wire 形状的合同测试
|
||||
//! 验证观测响应的指标、展示投影与查询时间边界
|
||||
|
||||
use chrono::Utc;
|
||||
use serde_json::json;
|
||||
@@ -288,10 +288,6 @@ async fn usage_detail_should_keep_attempt_snapshot_contract() {
|
||||
record.upstream_response_ms = Some(1_000);
|
||||
record.upstream_api_overhead_ms = Some(120.25);
|
||||
record.upstream_engine_ms = Some(6400.0);
|
||||
record.upstream_engine_iapi_ttft_ms = Some(650.5);
|
||||
record.upstream_engine_service_ttft_ms = Some(720.25);
|
||||
record.upstream_engine_iapi_tbt_ms = Some(18.45);
|
||||
record.upstream_engine_service_tbt_ms = Some(20.12);
|
||||
fixture
|
||||
.usage_detail
|
||||
.lock()
|
||||
@@ -362,29 +358,6 @@ async fn usage_detail_should_keep_attempt_snapshot_contract() {
|
||||
value["data"]["latencyDetails"]["upstreamEngineMs"],
|
||||
json!(6400.0)
|
||||
);
|
||||
assert_eq!(
|
||||
value["data"]["latencyDetails"]["upstreamEngineIapiTtftMs"],
|
||||
json!(650.5)
|
||||
);
|
||||
assert_eq!(
|
||||
value["data"]["latencyDetails"]["upstreamEngineServiceTtftMs"],
|
||||
json!(720.25)
|
||||
);
|
||||
assert_eq!(
|
||||
value["data"]["latencyDetails"]["upstreamEngineIapiTbtMs"],
|
||||
json!(18.45)
|
||||
);
|
||||
assert_eq!(
|
||||
value["data"]["latencyDetails"]["upstreamEngineServiceTbtMs"],
|
||||
json!(20.12)
|
||||
);
|
||||
assert!(value["data"].get("firstTokenMs").is_none());
|
||||
assert!(value["data"].get("firstTokenLatencyMsDisplay").is_none());
|
||||
assert!(
|
||||
value["data"]["latencyDetails"]
|
||||
.get("firstTokenMs")
|
||||
.is_none()
|
||||
);
|
||||
assert_eq!(
|
||||
serde_json::json!({
|
||||
"accountId": value["data"]["accountId"],
|
||||
@@ -490,8 +463,6 @@ async fn zero_attempt_failure_detail_omits_trace_and_keeps_missing_upstream_fact
|
||||
assert_eq!(data["clientStatusCode"], 503);
|
||||
assert_eq!(data["message"], "no upstream account is available");
|
||||
assert_eq!(data["model"], "coding");
|
||||
assert!(data.get("upstreamStatusCode").is_none());
|
||||
assert!(data.get("httpVersion").is_none());
|
||||
for field in [
|
||||
"provider",
|
||||
"accountId",
|
||||
@@ -642,7 +613,6 @@ async fn diagnostics_should_keep_stable_key_and_display_name_contract() {
|
||||
.await
|
||||
.expect("diagnostics body");
|
||||
let value: serde_json::Value = serde_json::from_slice(&body).expect("diagnostics JSON");
|
||||
assert!(value["data"]["items"][0].get("impactScore").is_none());
|
||||
assert_eq!(value["data"]["items"][0]["requestShare"], 0.5);
|
||||
assert_eq!(value["data"]["items"][0]["retryCount"], 3);
|
||||
assert_eq!(value["data"]["items"][0]["retryRate"], 0.5);
|
||||
@@ -734,10 +704,6 @@ pub(super) fn usage_record_with_account(
|
||||
upstream_response_ms: None,
|
||||
upstream_api_overhead_ms: None,
|
||||
upstream_engine_ms: None,
|
||||
upstream_engine_iapi_ttft_ms: None,
|
||||
upstream_engine_service_ttft_ms: None,
|
||||
upstream_engine_iapi_tbt_ms: None,
|
||||
upstream_engine_service_tbt_ms: None,
|
||||
latency_ms: None,
|
||||
admission_decision_ms: None,
|
||||
account_selection_wait_ms: None,
|
||||
@@ -877,10 +843,6 @@ async fn usage_route_should_expose_table_facts_without_detail_payload() {
|
||||
upstream_response_ms: Some(2_000),
|
||||
upstream_api_overhead_ms: None,
|
||||
upstream_engine_ms: None,
|
||||
upstream_engine_iapi_ttft_ms: None,
|
||||
upstream_engine_service_ttft_ms: None,
|
||||
upstream_engine_iapi_tbt_ms: None,
|
||||
upstream_engine_service_tbt_ms: None,
|
||||
latency_ms: Some(31),
|
||||
admission_decision_ms: Some(1),
|
||||
account_selection_wait_ms: Some(2),
|
||||
|
||||
@@ -35,7 +35,7 @@ use std::{collections::BTreeMap, sync::Arc};
|
||||
mod management;
|
||||
|
||||
#[test]
|
||||
fn artifact_metadata_wire_exposes_v2_identity_and_contributions() {
|
||||
fn artifact_metadata_wire_preserves_identity_icons_and_contributions() {
|
||||
use gateway_admin::model::plugins::{
|
||||
PluginArtifactIcon, PluginArtifactIconVariants, PluginArtifactMetadata, PluginContribution,
|
||||
};
|
||||
@@ -70,7 +70,7 @@ fn artifact_metadata_wire_exposes_v2_identity_and_contributions() {
|
||||
secret_fields: vec![],
|
||||
state_namespaces: vec![],
|
||||
};
|
||||
let value = serde_json::to_value(metadata).expect("v2 artifact metadata JSON");
|
||||
let value = serde_json::to_value(metadata).expect("artifact metadata JSON");
|
||||
assert_eq!(value["displayName"], "Example");
|
||||
assert_eq!(value["publisher"], "test");
|
||||
assert!(value["author"].is_null());
|
||||
@@ -80,33 +80,29 @@ fn artifact_metadata_wire_exposes_v2_identity_and_contributions() {
|
||||
value["contributes"]["middleware"]["id"],
|
||||
"test.example.middleware"
|
||||
);
|
||||
assert!(value.get("capabilities").is_none());
|
||||
assert!(value.get("permissionDescriptions").is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn instance_binding_wire_uses_contribution_without_capability_alias() {
|
||||
fn instance_binding_requires_a_contribution_and_rejects_unknown_fields() {
|
||||
use gateway_admin::model::plugins::instances::PluginCapabilityBinding;
|
||||
|
||||
let binding = serde_json::from_value::<PluginCapabilityBinding>(serde_json::json!({
|
||||
let input = serde_json::json!({
|
||||
"contribution": "test.example.usage",
|
||||
"stage": "observation",
|
||||
"order": 0,
|
||||
"failurePolicy": "observe"
|
||||
}))
|
||||
.expect("v2 contribution binding");
|
||||
});
|
||||
let binding = serde_json::from_value::<PluginCapabilityBinding>(input.clone())
|
||||
.expect("contribution binding");
|
||||
assert_eq!(binding.contribution, "test.example.usage");
|
||||
|
||||
assert!(
|
||||
serde_json::from_value::<PluginCapabilityBinding>(serde_json::json!({
|
||||
"capability": "usage",
|
||||
"stage": "observation",
|
||||
"order": 0,
|
||||
"failurePolicy": "observe"
|
||||
}))
|
||||
.is_err(),
|
||||
"legacy capability binding must be rejected"
|
||||
);
|
||||
let mut missing = input.clone();
|
||||
missing.as_object_mut().unwrap().remove("contribution");
|
||||
assert!(serde_json::from_value::<PluginCapabilityBinding>(missing).is_err());
|
||||
|
||||
let mut unknown = input;
|
||||
unknown["unknownField"] = serde_json::json!(true);
|
||||
assert!(serde_json::from_value::<PluginCapabilityBinding>(unknown).is_err());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -381,7 +377,7 @@ async fn plugin_upload_endpoints_require_admin_and_bound_raw_bodies() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn artifact_acceptance_returns_the_flat_install_contract() {
|
||||
async fn artifact_acceptance_returns_installed_artifact_and_configuration_status() {
|
||||
use axum::{
|
||||
body::{Body, to_bytes},
|
||||
http::{Request, StatusCode},
|
||||
@@ -418,22 +414,11 @@ async fn artifact_acceptance_returns_the_flat_install_contract() {
|
||||
assert_eq!(result["defaultInstanceId"], serde_json::Value::Null);
|
||||
assert_eq!(result["configurationRequired"], false);
|
||||
assert_eq!(result["artifact"]["metadata"]["sha256"], "a".repeat(64));
|
||||
assert!(
|
||||
result["artifact"]["metadata"]
|
||||
.get("permissionDescriptions")
|
||||
.is_none()
|
||||
);
|
||||
assert!(
|
||||
result["artifact"]["metadata"]
|
||||
.get("requestedPermissions")
|
||||
.is_none()
|
||||
);
|
||||
assert!(result["artifact"]["acceptedAt"].is_string());
|
||||
assert!(result.get("mutation").is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn removed_instance_policy_fields_and_extra_accept_fields_are_rejected() {
|
||||
async fn plugin_mutations_reject_unknown_fields() {
|
||||
use axum::{
|
||||
body::Body,
|
||||
http::{Request, StatusCode},
|
||||
@@ -442,24 +427,25 @@ async fn removed_instance_policy_fields_and_extra_accept_fields_are_rejected() {
|
||||
|
||||
let fixture = super::AdminTestFixture::new().await;
|
||||
fixture.auth.insert_session("valid-session");
|
||||
for (path, body) in [
|
||||
(
|
||||
"/api/admin/plugins/instances",
|
||||
serde_json::json!({
|
||||
"name":"Example",
|
||||
"artifactSha256":"a".repeat(64),
|
||||
"enabled":false,
|
||||
"trustedProcess":true,
|
||||
"configuration":{},
|
||||
"grants":[],
|
||||
"bindings":[]
|
||||
}),
|
||||
),
|
||||
let instance = serde_json::json!({
|
||||
"name":"Example",
|
||||
"artifactSha256":"a".repeat(64),
|
||||
"enabled":false,
|
||||
"configuration":{},
|
||||
"bindings":[]
|
||||
});
|
||||
serde_json::from_value::<gateway_admin::model::plugins::instances::ConfigurePluginInstance>(
|
||||
instance.clone(),
|
||||
)
|
||||
.expect("valid instance request");
|
||||
for (path, mut body) in [
|
||||
("/api/admin/plugins/instances", instance),
|
||||
(
|
||||
"/api/admin/plugins/artifacts/accept",
|
||||
serde_json::json!({"sha256":"a".repeat(64),"trusted":true}),
|
||||
serde_json::json!({"sha256":"a".repeat(64)}),
|
||||
),
|
||||
] {
|
||||
body["unknownField"] = serde_json::json!(true);
|
||||
let response = gateway_api::admin::router::<super::AdminTestState>()
|
||||
.with_state(fixture.state())
|
||||
.oneshot(
|
||||
|
||||
@@ -181,7 +181,7 @@ fn privacy_policy_is_required_and_rejects_unknown_contract_fields() {
|
||||
.remove("codexPrivacyPolicy");
|
||||
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(missing).is_err());
|
||||
let mut unknown = update_body();
|
||||
unknown["codexPrivacyPolicy"]["protectedFields"] = json!([]);
|
||||
unknown["codexPrivacyPolicy"]["unknownField"] = json!(true);
|
||||
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(unknown).is_err());
|
||||
}
|
||||
|
||||
@@ -471,110 +471,6 @@ fn settings_response_should_cover_the_full_runtime_settings_contract() {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn settings_request_and_response_fields_should_stay_in_lockstep() {
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
|
||||
use gateway_admin::model::Revision;
|
||||
use gateway_admin::model::settings::RuntimeSettings;
|
||||
use gateway_api::admin::settings::RuntimeSettingsView;
|
||||
use gateway_core::account::RotationStrategy;
|
||||
use gateway_core::routing::{PublicModelId, UpstreamModelId};
|
||||
|
||||
let request: UpdateRuntimeSettingsRequest =
|
||||
serde_json::from_value(update_body()).expect("decode settings");
|
||||
request.validate().expect("fixture settings must validate");
|
||||
|
||||
let request_fields: BTreeSet<String> = update_body()
|
||||
.as_object()
|
||||
.expect("request body object")
|
||||
.keys()
|
||||
.cloned()
|
||||
.collect();
|
||||
let settings = RuntimeSettings {
|
||||
request_profiles: Default::default(),
|
||||
config_revision: Revision::new(7).expect("revision"),
|
||||
model_mappings: request
|
||||
.model_mappings
|
||||
.iter()
|
||||
.map(|(public, upstream)| {
|
||||
Ok((
|
||||
PublicModelId::new(public.clone())?,
|
||||
UpstreamModelId::new(upstream.clone())?,
|
||||
))
|
||||
})
|
||||
.collect::<Result<BTreeMap<_, _>, gateway_core::error::IdentifierError>>()
|
||||
.expect("valid model mappings"),
|
||||
rotation_strategy: RotationStrategy::parse(&request.rotation_strategy)
|
||||
.expect("fixture rotation strategy"),
|
||||
updated_at: chrono::Utc::now(),
|
||||
values: gateway_admin::model::settings::RuntimeSettingsValues {
|
||||
codex_privacy_policy: Default::default(),
|
||||
request_location_enabled: false,
|
||||
request_location: Default::default(),
|
||||
refresh_margin_seconds: request.values.refresh_margin_seconds,
|
||||
refresh_concurrency: u32::try_from(request.values.refresh_concurrency).expect("u32"),
|
||||
max_concurrent_per_account: u32::try_from(request.values.max_concurrent_per_account)
|
||||
.expect("u32"),
|
||||
max_waiting_per_key: 0,
|
||||
max_waiting_per_account: 0,
|
||||
concurrency_wait_timeout_seconds: 30,
|
||||
openai_guardian_reserved_concurrency: 0,
|
||||
openai_account_affinity: gateway_core::account::AccountAffinity::Relaxed,
|
||||
max_account_rotations: 3,
|
||||
openai_session_affinity_ttl_hours: 24,
|
||||
responses_max_decompressed_body_bytes: 64 * 1024 * 1024,
|
||||
smart_scheduling: request.values.smart_scheduling,
|
||||
min_codex_desktop_version: request.values.min_codex_desktop_version,
|
||||
min_codex_cli_version: request.values.min_codex_cli_version,
|
||||
usage_retention_days: u32::try_from(request.values.usage_retention_days).expect("u32"),
|
||||
ops_event_retention_days: u32::try_from(request.values.ops_event_retention_days)
|
||||
.expect("u32"),
|
||||
audit_retention_days: u32::try_from(request.values.audit_retention_days).expect("u32"),
|
||||
account_auto_freeze_enabled: true,
|
||||
account_auto_freeze_threshold: 12,
|
||||
account_auto_freeze_window_seconds: 600,
|
||||
account_auto_freeze_duration_seconds: 7_200,
|
||||
account_auto_freeze_probe_enabled: true,
|
||||
account_auto_freeze_probe_model: None,
|
||||
account_auto_freeze_adaptive_concurrency: true,
|
||||
account_warmup_enabled: false,
|
||||
account_warmup_schedule_time: "08:00".to_owned(),
|
||||
account_warmup_model: None,
|
||||
reset_credit_lead_minutes: 10,
|
||||
reset_credit_max_weekly_remaining_percent: None,
|
||||
},
|
||||
};
|
||||
|
||||
let response_fields: BTreeSet<String> = serde_json::to_value(RuntimeSettingsView::from((
|
||||
settings,
|
||||
gateway_api::TimePresenter::new(Default::default()),
|
||||
)))
|
||||
.expect("serialize view")
|
||||
.as_object()
|
||||
.expect("view object")
|
||||
.keys()
|
||||
.cloned()
|
||||
.collect();
|
||||
let mut expected_fields = request_fields;
|
||||
expected_fields.insert("providerRequestProfiles".to_owned());
|
||||
expected_fields.insert("openaiClientProfile".to_owned());
|
||||
expected_fields.insert("xaiClientProfile".to_owned());
|
||||
expected_fields.insert("updatedAt".to_owned());
|
||||
expected_fields.insert("updatedAtDisplay".to_owned());
|
||||
expected_fields.insert("smartSchedulingDefaults".to_owned());
|
||||
|
||||
assert_eq!(response_fields, expected_fields);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn settings_request_accepts_json_bytes_with_flattened_fields() {
|
||||
serde_json::from_slice::<UpdateRuntimeSettingsRequest>(
|
||||
&serde_json::to_vec(&update_body()).unwrap(),
|
||||
)
|
||||
.expect("settings JSON bytes must decode like the HTTP request body");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn settings_service_contract_stays_flat_and_rejects_unknown_fields() {
|
||||
use gateway_admin::model::settings::{ReplaceRuntimeSettings, RuntimeSettings};
|
||||
@@ -610,17 +506,9 @@ fn settings_service_contract_stays_flat_and_rejects_unknown_fields() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn settings_request_should_reject_unknown_revision_field() {
|
||||
fn settings_request_should_reject_unknown_fields() {
|
||||
let mut body = update_body();
|
||||
body["expectedConfigRevision"] = json!(7);
|
||||
|
||||
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(body).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn settings_request_should_reject_removed_bucket_retention() {
|
||||
let mut body = update_body();
|
||||
body["bucketRetentionDays"] = json!(365);
|
||||
body["unknownSetting"] = json!(true);
|
||||
|
||||
assert!(serde_json::from_value::<UpdateRuntimeSettingsRequest>(body).is_err());
|
||||
}
|
||||
@@ -960,36 +848,23 @@ fn decompression_setting_should_reject_invalid_values() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn settings_reject_removed_global_fast_policy() {
|
||||
async fn settings_update_rejects_unknown_fields_without_writing() {
|
||||
let fixture = AdminTestFixture::new().await;
|
||||
fixture.auth.insert_session("valid-session");
|
||||
let app = app(fixture.state());
|
||||
let initial = fixture.settings.settings.lock().unwrap().clone();
|
||||
for value in [json!(true), json!(false), Value::Null] {
|
||||
let mut body = update_body();
|
||||
body["disableFast"] = value;
|
||||
let response = app
|
||||
.clone()
|
||||
.oneshot(request(
|
||||
Method::POST,
|
||||
"/api/admin/settings/update",
|
||||
Some(body),
|
||||
))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
|
||||
assert_eq!(*fixture.settings.settings.lock().unwrap(), initial);
|
||||
}
|
||||
let mut body = update_body();
|
||||
body["unknownSetting"] = json!(true);
|
||||
let response = app
|
||||
.oneshot(request(Method::GET, "/api/admin/settings", None))
|
||||
.oneshot(request(
|
||||
Method::POST,
|
||||
"/api/admin/settings/update",
|
||||
Some(body),
|
||||
))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(response.status(), StatusCode::OK);
|
||||
assert!(
|
||||
response_json(response).await["data"]
|
||||
.get("disableFast")
|
||||
.is_none()
|
||||
);
|
||||
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
|
||||
assert_eq!(*fixture.settings.settings.lock().unwrap(), initial);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -86,7 +86,7 @@ async fn update_event_stream_should_preserve_event_id_and_download_progress_perc
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn update_should_return_accepted_and_report_restart_only_through_status() {
|
||||
async fn update_should_return_accepted_and_expose_restart_status() {
|
||||
let fixture = AdminTestFixture::with_system(Arc::new(ProgressSystem::default())).await;
|
||||
fixture.auth.insert_session("valid-session");
|
||||
let router = app(fixture.state());
|
||||
@@ -110,7 +110,6 @@ async fn update_should_return_accepted_and_report_restart_only_through_status()
|
||||
.expect("JSON");
|
||||
assert_eq!(body["data"]["operationId"], "update-1");
|
||||
assert_eq!(body["data"]["targetVersion"], "0.2.0");
|
||||
assert!(body["data"].get("needRestart").is_none());
|
||||
let response = router
|
||||
.oneshot(
|
||||
Request::builder()
|
||||
|
||||
@@ -297,15 +297,15 @@ async fn unified_login_returns_server_identity_and_rotates_the_previous_session(
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn login_rejects_legacy_type_without_replacing_the_current_session() {
|
||||
async fn login_rejects_missing_mode_and_unknown_fields_without_replacing_the_session() {
|
||||
let (app, _) = auth_app().await;
|
||||
let cookie =
|
||||
session_cookie(&login(&app, json!({"mode": "key", "apiKey": RAW_KEY}), None).await);
|
||||
for body in [
|
||||
json!({"type": "key", "apiKey": RAW_KEY}),
|
||||
json!({"type": "admin", "password": "strong-admin-password"}),
|
||||
json!({"mode": "key", "type": "key", "apiKey": RAW_KEY}),
|
||||
json!({"mode": "admin", "type": "admin", "password": "strong-admin-password"}),
|
||||
json!({"apiKey": RAW_KEY}),
|
||||
json!({"password": "strong-admin-password"}),
|
||||
json!({"mode": "key", "unknownField": true, "apiKey": RAW_KEY}),
|
||||
json!({"mode": "admin", "unknownField": true, "password": "strong-admin-password"}),
|
||||
] {
|
||||
let response = login(&app, body, Some(&cookie)).await;
|
||||
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
|
||||
@@ -458,26 +458,26 @@ async fn key_store_outage_is_unavailable_not_expired_or_authenticated() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn removed_auth_routes_do_not_fall_back_to_spa_or_accept_old_cookies() {
|
||||
async fn unknown_auth_routes_return_json_not_found() {
|
||||
let (app, _) = auth_app().await;
|
||||
let response = app
|
||||
.oneshot(empty_request(Method::GET, "/api/auth/unknown"))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(response.status(), StatusCode::NOT_FOUND);
|
||||
assert_eq!(response_json(response).await["code"], 40401);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn session_authentication_requires_the_configured_cookie_name() {
|
||||
let (app, _) = auth_app().await;
|
||||
for path in ["/api/admin/auth/login", "/api/auth/unknown"] {
|
||||
let response = app
|
||||
.clone()
|
||||
.oneshot(empty_request(Method::GET, path))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(response.status(), StatusCode::NOT_FOUND);
|
||||
assert_eq!(response_json(response).await["code"], 40401);
|
||||
}
|
||||
let cookie =
|
||||
session_cookie(&login(&app, json!({"mode": "key", "apiKey": RAW_KEY}), None).await);
|
||||
for old_name in ["cpr_admin_session", "cpr_client_session"] {
|
||||
let old_cookie = cookie.replace("cpr_session", old_name);
|
||||
assert_eq!(
|
||||
response_json(get(&app, "/api/auth/status", &old_cookie).await).await["data"]["authenticated"],
|
||||
false
|
||||
);
|
||||
}
|
||||
let unrelated_cookie = cookie.replace("cpr_session", "unrelated_session");
|
||||
assert_eq!(
|
||||
response_json(get(&app, "/api/auth/status", &unrelated_cookie).await).await["data"]["authenticated"],
|
||||
false
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -140,11 +140,7 @@ fn usage_record() -> UsageListRecord {
|
||||
provider_processing_ms: None,
|
||||
upstream_response_ms: Some(1_000),
|
||||
upstream_api_overhead_ms: None,
|
||||
upstream_engine_ms: None,
|
||||
upstream_engine_iapi_ttft_ms: None,
|
||||
upstream_engine_service_ttft_ms: None,
|
||||
upstream_engine_iapi_tbt_ms: Some(2.450638),
|
||||
upstream_engine_service_tbt_ms: None,
|
||||
upstream_engine_ms: Some(2.450638),
|
||||
latency_ms: Some(1250),
|
||||
admission_decision_ms: None,
|
||||
account_selection_wait_ms: None,
|
||||
|
||||
@@ -320,7 +320,7 @@ async fn records_keep_pagination_and_hide_admin_and_upstream_data() {
|
||||
&record["latencyDetails"],
|
||||
&[
|
||||
"upstreamResponseMs",
|
||||
"upstreamEngineIapiTbtMs",
|
||||
"upstreamEngineMs",
|
||||
"firstEventMs",
|
||||
"firstReasoningMs",
|
||||
"firstTextMs",
|
||||
@@ -328,10 +328,7 @@ async fn records_keep_pagination_and_hide_admin_and_upstream_data() {
|
||||
);
|
||||
assert_eq!(record["latencyDetails"]["firstEventMs"], 100);
|
||||
assert_eq!(record["latencyDetails"]["upstreamResponseMs"], 1000);
|
||||
assert_eq!(
|
||||
record["latencyDetails"]["upstreamEngineIapiTbtMs"],
|
||||
2.450638
|
||||
);
|
||||
assert_eq!(record["latencyDetails"]["upstreamEngineMs"], 2.450638);
|
||||
assert!(record["statusCode"].is_null());
|
||||
} else {
|
||||
assert_eq!(record["statusCode"], 502);
|
||||
|
||||
@@ -799,7 +799,7 @@ async fn websocket_failure_before_first_event_should_return_observed_metadata_th
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn websocket_response_create_should_not_have_a_private_16_mib_frame_limit() {
|
||||
async fn large_websocket_response_create_frames_should_reach_execution() {
|
||||
let trace = Arc::new(AtomicFailureTrace::default());
|
||||
let execution = Arc::new(AtomicFailureExecution {
|
||||
client: authenticated_client("sk_ws_atomic"),
|
||||
@@ -829,15 +829,14 @@ async fn websocket_response_create_should_not_have_a_private_16_mib_frame_limit(
|
||||
let payload = json!({
|
||||
"type": "response.create",
|
||||
"model": "model-a",
|
||||
"input": "x".repeat(16 * 1024 * 1024 + 1),
|
||||
"input": "x".repeat(17 * 1024 * 1024),
|
||||
})
|
||||
.to_string();
|
||||
assert!(payload.len() > 16 * 1024 * 1024);
|
||||
|
||||
socket
|
||||
.send(ClientMessage::Text(payload.into()))
|
||||
.await
|
||||
.expect("send response.create above the former frame limit");
|
||||
.expect("send large response.create frame");
|
||||
|
||||
let mut saw_upstream_event = false;
|
||||
for _ in 0..3 {
|
||||
|
||||
@@ -8,6 +8,26 @@ use serde_json::json;
|
||||
|
||||
use super::decode_response_create;
|
||||
|
||||
#[test]
|
||||
fn response_create_should_preserve_field_order_when_removing_frame_type() {
|
||||
let expected = r#"{"model":"smart-code","stream":true,"input":"hello","future_field":{"z":1,"a":2},"client_metadata":{"thread_id":"thread","turn_id":"turn"}}"#;
|
||||
for payload in [
|
||||
r#"{"type":"response.create","model":"smart-code","stream":true,"input":"hello","future_field":{"z":1,"a":2},"client_metadata":{"thread_id":"thread","turn_id":"turn"}}"#,
|
||||
r#"{"model":"smart-code","stream":true,"type":"response.create","input":"hello","future_field":{"z":1,"a":2},"client_metadata":{"thread_id":"thread","turn_id":"turn"}}"#,
|
||||
r#"{"model":"smart-code","stream":true,"input":"hello","future_field":{"z":1,"a":2},"client_metadata":{"thread_id":"thread","turn_id":"turn"},"type":"response.create"}"#,
|
||||
] {
|
||||
let decoded = decode_response_create(payload).expect("decode response.create");
|
||||
let body = decoded
|
||||
.operation()
|
||||
.middleware_body()
|
||||
.expect("encode transparent body");
|
||||
assert_eq!(
|
||||
std::str::from_utf8(&body).expect("UTF-8 JSON body"),
|
||||
expected,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn client_retry_identity_should_follow_each_frame_instead_of_handshake_metadata() {
|
||||
let mut headers = HeaderMap::new();
|
||||
|
||||
@@ -2,14 +2,14 @@
|
||||
|
||||
use axum::{
|
||||
body::Body,
|
||||
http::{Method, Request, StatusCode, header::AUTHORIZATION},
|
||||
http::{Method, Request, StatusCode},
|
||||
};
|
||||
use tower::ServiceExt;
|
||||
|
||||
use super::api_router_with_origins;
|
||||
use super::models::ModelsExecution;
|
||||
|
||||
const REMOVED_BODY_LIMIT_BYTES: usize = 16 * 1024 * 1024;
|
||||
const LARGE_BODY_BYTES: usize = 17 * 1024 * 1024;
|
||||
|
||||
#[tokio::test]
|
||||
async fn browser_origin_controls_http_admin_sessions_without_configuration() {
|
||||
@@ -117,64 +117,16 @@ async fn browser_origin_controls_http_admin_sessions_without_configuration() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn removed_responses_review_route_should_not_reach_the_responses_handler() {
|
||||
let response = api_router_with_origins(ModelsExecution::new(), Vec::new())
|
||||
.await
|
||||
.oneshot(
|
||||
Request::post("/v1/responses/review")
|
||||
.header(AUTHORIZATION, "Bearer sk_models_test")
|
||||
.body(Body::empty())
|
||||
.expect("build removed review request"),
|
||||
)
|
||||
.await
|
||||
.expect("route removed review request");
|
||||
|
||||
assert_eq!(response.status(), StatusCode::METHOD_NOT_ALLOWED);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn removed_models_catalog_extension_should_use_standard_model_lookup() {
|
||||
let response = api_router_with_origins(ModelsExecution::new(), Vec::new())
|
||||
.await
|
||||
.oneshot(
|
||||
Request::get("/v1/models/catalog")
|
||||
.header(AUTHORIZATION, "Bearer sk_models_test")
|
||||
.body(Body::empty())
|
||||
.expect("build removed model catalog request"),
|
||||
)
|
||||
.await
|
||||
.expect("route removed model catalog request");
|
||||
|
||||
assert_eq!(response.status(), StatusCode::NOT_FOUND);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn removed_model_info_route_should_return_not_found() {
|
||||
let response = api_router_with_origins(ModelsExecution::new(), Vec::new())
|
||||
.await
|
||||
.oneshot(
|
||||
Request::get("/v1/models/model-a/info")
|
||||
.header(AUTHORIZATION, "Bearer sk_models_test")
|
||||
.body(Body::empty())
|
||||
.expect("build removed model info request"),
|
||||
)
|
||||
.await
|
||||
.expect("route removed model info request");
|
||||
|
||||
assert_eq!(response.status(), StatusCode::NOT_FOUND);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn responses_body_should_accept_payload_above_the_removed_private_limit() {
|
||||
async fn large_responses_bodies_should_reach_authentication() {
|
||||
let router = api_router_with_origins(ModelsExecution::new(), Vec::new()).await;
|
||||
let response = router
|
||||
.oneshot(
|
||||
Request::post("/v1/responses")
|
||||
.body(Body::from(vec![b'a'; REMOVED_BODY_LIMIT_BYTES + 1]))
|
||||
.expect("build request above the removed limit"),
|
||||
.body(Body::from(vec![b'a'; LARGE_BODY_BYTES]))
|
||||
.expect("build large request"),
|
||||
)
|
||||
.await
|
||||
.expect("route request above the removed limit");
|
||||
.expect("route large request");
|
||||
|
||||
// 请求进入处理器后应因缺少凭据而被拒绝
|
||||
// 若提前施加正文大小限制,则会在认证前返回 413
|
||||
|
||||
@@ -955,10 +955,6 @@ pub struct ModelRequestTimings {
|
||||
pub upstream_response_ms: Option<u64>,
|
||||
pub upstream_api_overhead_ms: Option<f64>,
|
||||
pub upstream_engine_ms: Option<f64>,
|
||||
pub upstream_engine_iapi_ttft_ms: Option<f64>,
|
||||
pub upstream_engine_service_ttft_ms: Option<f64>,
|
||||
pub upstream_engine_iapi_tbt_ms: Option<f64>,
|
||||
pub upstream_engine_service_tbt_ms: Option<f64>,
|
||||
pub latency_ms: Option<u64>,
|
||||
}
|
||||
|
||||
|
||||
@@ -807,10 +807,6 @@ impl ResponseObservation {
|
||||
self.timings.upstream_response_ms = None;
|
||||
self.timings.upstream_api_overhead_ms = None;
|
||||
self.timings.upstream_engine_ms = None;
|
||||
self.timings.upstream_engine_iapi_ttft_ms = None;
|
||||
self.timings.upstream_engine_service_ttft_ms = None;
|
||||
self.timings.upstream_engine_iapi_tbt_ms = None;
|
||||
self.timings.upstream_engine_service_tbt_ms = None;
|
||||
}
|
||||
|
||||
pub(super) fn observe_response(&mut self, observation: &ProviderResponseObservation) {
|
||||
@@ -848,17 +844,5 @@ impl ResponseObservation {
|
||||
if let Some(value) = observed.upstream_engine_ms {
|
||||
self.timings.upstream_engine_ms = Some(value);
|
||||
}
|
||||
if let Some(value) = observed.upstream_engine_iapi_ttft_ms {
|
||||
self.timings.upstream_engine_iapi_ttft_ms = Some(value);
|
||||
}
|
||||
if let Some(value) = observed.upstream_engine_service_ttft_ms {
|
||||
self.timings.upstream_engine_service_ttft_ms = Some(value);
|
||||
}
|
||||
if let Some(value) = observed.upstream_engine_iapi_tbt_ms {
|
||||
self.timings.upstream_engine_iapi_tbt_ms = Some(value);
|
||||
}
|
||||
if let Some(value) = observed.upstream_engine_service_tbt_ms {
|
||||
self.timings.upstream_engine_service_tbt_ms = Some(value);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -536,12 +536,8 @@ pub struct ProviderResponseTimings {
|
||||
pub upstream_response_ms: Option<u64>,
|
||||
/// 上游排除引擎与客户端工具时间后的 API 耗时
|
||||
pub upstream_api_overhead_ms: Option<f64>,
|
||||
/// 上游响应级引擎耗时;以下专项指标保留毫秒小数及各自内部计量层级
|
||||
/// 上游响应级引擎耗时,保留毫秒小数
|
||||
pub upstream_engine_ms: Option<f64>,
|
||||
pub upstream_engine_iapi_ttft_ms: Option<f64>,
|
||||
pub upstream_engine_service_ttft_ms: Option<f64>,
|
||||
pub upstream_engine_iapi_tbt_ms: Option<f64>,
|
||||
pub upstream_engine_service_tbt_ms: Option<f64>,
|
||||
}
|
||||
|
||||
/// Provider 已筛选的响应观测 JSON
|
||||
|
||||
@@ -505,10 +505,6 @@ struct FinalState {
|
||||
upstream_response_ms: Option<u64>,
|
||||
upstream_api_overhead_ms: Option<f64>,
|
||||
upstream_engine_ms: Option<f64>,
|
||||
upstream_engine_iapi_ttft_ms: Option<f64>,
|
||||
upstream_engine_service_ttft_ms: Option<f64>,
|
||||
upstream_engine_iapi_tbt_ms: Option<f64>,
|
||||
upstream_engine_service_tbt_ms: Option<f64>,
|
||||
provider_metadata_json: Option<String>,
|
||||
cost_source: CostSource,
|
||||
cost_ticks: Option<u128>,
|
||||
@@ -684,12 +680,6 @@ impl ExecutionStore for FakeStore {
|
||||
upstream_response_ms: finalization.timings.upstream_response_ms,
|
||||
upstream_api_overhead_ms: finalization.timings.upstream_api_overhead_ms,
|
||||
upstream_engine_ms: finalization.timings.upstream_engine_ms,
|
||||
upstream_engine_iapi_ttft_ms: finalization.timings.upstream_engine_iapi_ttft_ms,
|
||||
upstream_engine_service_ttft_ms: finalization
|
||||
.timings
|
||||
.upstream_engine_service_ttft_ms,
|
||||
upstream_engine_iapi_tbt_ms: finalization.timings.upstream_engine_iapi_tbt_ms,
|
||||
upstream_engine_service_tbt_ms: finalization.timings.upstream_engine_service_tbt_ms,
|
||||
provider_metadata_json: finalization.provider_metadata_json,
|
||||
cost_source: finalization.cost.source(),
|
||||
cost_ticks: finalization
|
||||
@@ -2067,10 +2057,6 @@ fn discarded_attempt_observation_does_not_leak_into_retry_result() {
|
||||
upstream_response_ms: Some(7_000),
|
||||
upstream_api_overhead_ms: Some(120.25),
|
||||
upstream_engine_ms: Some(6400.0),
|
||||
upstream_engine_iapi_ttft_ms: Some(650.5),
|
||||
upstream_engine_service_ttft_ms: Some(720.25),
|
||||
upstream_engine_iapi_tbt_ms: Some(18.45),
|
||||
upstream_engine_service_tbt_ms: Some(20.12),
|
||||
..ProviderResponseTimings::default()
|
||||
});
|
||||
let second_observation = ProviderResponseObservation::new(
|
||||
@@ -2164,10 +2150,6 @@ fn discarded_attempt_observation_does_not_leak_into_retry_result() {
|
||||
assert_eq!(finalization.upstream_response_ms, None);
|
||||
assert_eq!(finalization.upstream_api_overhead_ms, None);
|
||||
assert_eq!(finalization.upstream_engine_ms, None);
|
||||
assert_eq!(finalization.upstream_engine_iapi_ttft_ms, None);
|
||||
assert_eq!(finalization.upstream_engine_service_ttft_ms, None);
|
||||
assert_eq!(finalization.upstream_engine_iapi_tbt_ms, None);
|
||||
assert_eq!(finalization.upstream_engine_service_tbt_ms, None);
|
||||
let contexts = provider.contexts.lock().unwrap();
|
||||
assert_eq!(contexts.len(), 2);
|
||||
assert_eq!(
|
||||
|
||||
@@ -10,9 +10,7 @@ use gateway_core::upstream::UpstreamTransport;
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn provider_event_keeps_large_wire_and_observation_payloads_behind_one_indirection() {
|
||||
assert!(std::mem::size_of::<ProviderEvent>() <= 64);
|
||||
|
||||
fn provider_event_should_distinguish_client_delivery_from_observation() {
|
||||
let wire = ProviderEvent::wire(
|
||||
ProtocolWireEvent::json("openai", None, json!({"type": "response.created"}))
|
||||
.expect("wire event"),
|
||||
|
||||
@@ -162,20 +162,15 @@ fn logging_rejects_zero_retention_windows() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn logging_defaults_to_time_retention_and_ignores_removed_count_retention() {
|
||||
let mut value = serde_json::json!({
|
||||
fn logging_defaults_to_time_retention() {
|
||||
let value = serde_json::json!({
|
||||
"level": "info", "stdout": true,
|
||||
"file": { "enabled": true, "directory": "logs", "max_file_size_mb": 20 }
|
||||
});
|
||||
let config: LoggingConfig = serde_json::from_value(value.clone()).unwrap();
|
||||
let config: LoggingConfig = serde_json::from_value(value).unwrap();
|
||||
assert_eq!(config.request_dump_retention_days, 1.0);
|
||||
assert_eq!(config.file.retention_days, 7);
|
||||
assert!(!config.oauth_recovery);
|
||||
value["file"]["max_files"] = serde_json::json!(20);
|
||||
assert_eq!(
|
||||
serde_json::from_value::<LoggingConfig>(value).unwrap(),
|
||||
config
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -543,10 +543,6 @@ fn wire_timings(observation: &RequestObservation) -> RequestTimings {
|
||||
upstream_response_ms: timings.upstream_response_ms,
|
||||
upstream_api_overhead_ms: timings.upstream_api_overhead_ms,
|
||||
upstream_engine_ms: timings.upstream_engine_ms,
|
||||
upstream_engine_iapi_ttft_ms: timings.upstream_engine_iapi_ttft_ms,
|
||||
upstream_engine_service_ttft_ms: timings.upstream_engine_service_ttft_ms,
|
||||
upstream_engine_iapi_tbt_ms: timings.upstream_engine_iapi_tbt_ms,
|
||||
upstream_engine_service_tbt_ms: timings.upstream_engine_service_tbt_ms,
|
||||
latency_ms: timings.latency_ms,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -132,10 +132,6 @@ fn observation(value: &ProviderResponseObservation) -> facts::ResponseObservatio
|
||||
upstream_response_ms: timings.upstream_response_ms,
|
||||
upstream_api_overhead_ms: timings.upstream_api_overhead_ms,
|
||||
upstream_engine_ms: timings.upstream_engine_ms,
|
||||
upstream_engine_iapi_ttft_ms: timings.upstream_engine_iapi_ttft_ms,
|
||||
upstream_engine_service_ttft_ms: timings.upstream_engine_service_ttft_ms,
|
||||
upstream_engine_iapi_tbt_ms: timings.upstream_engine_iapi_tbt_ms,
|
||||
upstream_engine_service_tbt_ms: timings.upstream_engine_service_tbt_ms,
|
||||
latency_ms: None,
|
||||
},
|
||||
client_headers: value
|
||||
|
||||
@@ -817,7 +817,7 @@ async fn completed_observation_contains_terminal_cost_timings_and_failure() {
|
||||
.with_timings(ModelRequestTimings {
|
||||
first_text_ms: Some(12),
|
||||
upstream_response_ms: Some(7_000),
|
||||
upstream_engine_iapi_tbt_ms: Some(2.450638),
|
||||
upstream_engine_ms: Some(2.450638),
|
||||
latency_ms: Some(34),
|
||||
..ModelRequestTimings::default()
|
||||
}),
|
||||
@@ -831,10 +831,7 @@ async fn completed_observation_contains_terminal_cost_timings_and_failure() {
|
||||
assert_eq!(observed["usage"]["cost"]["total"]["currency"], "USD");
|
||||
assert_eq!(observed["usage"]["timings"]["first_text_ms"], 12);
|
||||
assert_eq!(observed["usage"]["timings"]["upstream_response_ms"], 7_000);
|
||||
assert_eq!(
|
||||
observed["usage"]["timings"]["upstream_engine_iapi_tbt_ms"],
|
||||
2.450638
|
||||
);
|
||||
assert_eq!(observed["usage"]["timings"]["upstream_engine_ms"], 2.450638);
|
||||
assert_eq!(observed["usage"]["timings"]["latency_ms"], 34);
|
||||
assert_eq!(observed["usage"]["failure"]["outcome"], "failed");
|
||||
assert_eq!(observed["usage"]["failure"]["attempt_count"], 2);
|
||||
|
||||
@@ -103,10 +103,24 @@ async fn middleware_preserves_three_state_fast_and_sources() {
|
||||
Some(90_000)
|
||||
);
|
||||
let sources = settings.inspect();
|
||||
assert!(sources["execution"].get("disable_fast").is_none());
|
||||
if mode != FastMode::Default && mode == expected {
|
||||
assert_eq!(sources["execution"]["fast_mode"]["instance_id"], "seed");
|
||||
let fast_source = &sources["execution"]["fast_mode"];
|
||||
if mode == FastMode::Default && expected == FastMode::Default {
|
||||
assert!(fast_source.is_null());
|
||||
} else {
|
||||
assert_eq!(
|
||||
fast_source["instance_id"],
|
||||
if mode == expected {
|
||||
"seed"
|
||||
} else {
|
||||
"fast-settings"
|
||||
}
|
||||
);
|
||||
assert_eq!(fast_source["value"], expected.as_str());
|
||||
}
|
||||
assert_eq!(
|
||||
sources["execution"]["timeout_ms"]["instance_id"],
|
||||
"fast-settings"
|
||||
);
|
||||
next.run(request).await
|
||||
})
|
||||
}),
|
||||
|
||||
@@ -186,7 +186,7 @@ async fn facts_are_readable_without_replacing_or_duplicating_the_original_execut
|
||||
.with_timings(ProviderResponseTimings {
|
||||
headers_ms: Some(19),
|
||||
upstream_response_ms: Some(7_000),
|
||||
upstream_engine_iapi_tbt_ms: Some(2.450638),
|
||||
upstream_engine_ms: Some(2.450638),
|
||||
..Default::default()
|
||||
}),
|
||||
);
|
||||
@@ -273,7 +273,7 @@ async fn facts_are_readable_without_replacing_or_duplicating_the_original_execut
|
||||
);
|
||||
assert_eq!(facts["observation"]["timings"]["headers_ms"], 19);
|
||||
assert_eq!(
|
||||
facts["observation"]["timings"]["upstream_engine_iapi_tbt_ms"],
|
||||
facts["observation"]["timings"]["upstream_engine_ms"],
|
||||
2.450638
|
||||
);
|
||||
assert_eq!(
|
||||
|
||||
@@ -278,7 +278,7 @@ fn icon_content_must_fully_decode_match_its_type_and_fit_dimensions() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn obsolete_manifest_fields_are_rejected_without_aliases() {
|
||||
fn manifest_rejects_unknown_fields_before_package_validation() {
|
||||
let manifest = serde_json::json!({
|
||||
"manifestVersion":2,
|
||||
"name": "example",
|
||||
@@ -290,7 +290,8 @@ fn obsolete_manifest_fields_are_rejected_without_aliases() {
|
||||
"engines": {"codex-proxy-rs": ">=1.0.0, <2.0.0"},
|
||||
"main": "bin/worker",
|
||||
"runtime": "trustedProcess",
|
||||
"capabilities": [{"capability":"request_normalizer"}]
|
||||
"contributes": {},
|
||||
"unknownField": true
|
||||
});
|
||||
let archive = crate::support::archive(BTreeMap::from([(
|
||||
"plugin.json".to_owned(),
|
||||
|
||||
@@ -110,14 +110,6 @@ pub struct RequestTimings {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub upstream_engine_ms: Option<f64>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub upstream_engine_iapi_ttft_ms: Option<f64>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub upstream_engine_service_ttft_ms: Option<f64>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub upstream_engine_iapi_tbt_ms: Option<f64>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub upstream_engine_service_tbt_ms: Option<f64>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub latency_ms: Option<u64>,
|
||||
}
|
||||
|
||||
|
||||
@@ -204,13 +204,13 @@ fn cost_timings_and_failure_keep_unknown_facts_explicit_and_safe() {
|
||||
let timings = RequestTimings {
|
||||
first_text_ms: Some(12),
|
||||
upstream_response_ms: Some(7_000),
|
||||
upstream_engine_iapi_tbt_ms: Some(2.450638),
|
||||
upstream_engine_ms: Some(2.450638),
|
||||
latency_ms: Some(34),
|
||||
..RequestTimings::default()
|
||||
};
|
||||
assert_eq!(
|
||||
serde_json::to_value(&timings).unwrap(),
|
||||
json!({"first_text_ms":12,"upstream_response_ms":7000,"upstream_engine_iapi_tbt_ms":2.450638,"latency_ms":34})
|
||||
json!({"first_text_ms":12,"upstream_response_ms":7000,"upstream_engine_ms":2.450638,"latency_ms":34})
|
||||
);
|
||||
|
||||
let failure = RequestFailure {
|
||||
|
||||
@@ -132,52 +132,40 @@ fn installation_checks_protocol_engine_and_target() {
|
||||
|
||||
#[test]
|
||||
fn capability_contract_versions_are_checked_independently_of_rpc_version() {
|
||||
let mut manifest = packaged_manifest();
|
||||
assert!(manifest.validate().is_ok());
|
||||
for version in [0, 1, 2, 3, 4, 6, u32::MAX] {
|
||||
manifest
|
||||
.contributes
|
||||
.get_mut(&Capability::Middleware)
|
||||
.unwrap()
|
||||
.version = version;
|
||||
assert_eq!(manifest.validate(), Err(ManifestError::Invalid));
|
||||
for (capability, stage, supported, needs_formats) in [
|
||||
(Capability::Middleware, Stage::Request, 5, true),
|
||||
(Capability::Management, Stage::Management, 2, false),
|
||||
(Capability::UpstreamAdapter, Stage::Upstream, 3, true),
|
||||
(Capability::Scheduler, Stage::Scheduling, 3, false),
|
||||
] {
|
||||
for mut manifest in [source_manifest(), packaged_manifest()] {
|
||||
let formats = if needs_formats {
|
||||
vec!["openai".into()]
|
||||
} else {
|
||||
vec![]
|
||||
};
|
||||
manifest.contributes.clear();
|
||||
manifest.contributes.insert(
|
||||
capability,
|
||||
ContributionDeclaration {
|
||||
id: "9acme.request-tags.contract".into(),
|
||||
version: supported,
|
||||
stages: vec![stage],
|
||||
input_formats: formats.clone(),
|
||||
output_formats: formats,
|
||||
},
|
||||
);
|
||||
assert!(manifest.validate().is_ok(), "{capability:?} v{supported}");
|
||||
for version in (0..supported).chain([supported + 1, u32::MAX]) {
|
||||
manifest.contributes.get_mut(&capability).unwrap().version = version;
|
||||
assert_eq!(
|
||||
manifest.validate(),
|
||||
Err(ManifestError::Invalid),
|
||||
"{capability:?} v{version}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
manifest.contributes.clear();
|
||||
manifest.contributes.insert(
|
||||
Capability::Management,
|
||||
ContributionDeclaration {
|
||||
id: "9acme.request-tags.management".into(),
|
||||
version: 3,
|
||||
stages: vec![Stage::Management],
|
||||
input_formats: vec![],
|
||||
output_formats: vec![],
|
||||
},
|
||||
);
|
||||
assert_eq!(manifest.validate(), Err(ManifestError::Invalid));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn retired_upstream_contract_requires_an_author_upgrade() {
|
||||
let mut manifest = source_manifest();
|
||||
manifest.contributes.clear();
|
||||
manifest.contributes.insert(
|
||||
Capability::UpstreamAdapter,
|
||||
ContributionDeclaration {
|
||||
id: "9acme.request-tags.upstream".into(),
|
||||
version: 3,
|
||||
stages: vec![Stage::Upstream],
|
||||
input_formats: vec!["openai".into()],
|
||||
output_formats: vec!["openai".into()],
|
||||
},
|
||||
);
|
||||
assert!(manifest.validate().is_ok());
|
||||
manifest
|
||||
.contributes
|
||||
.get_mut(&Capability::UpstreamAdapter)
|
||||
.unwrap()
|
||||
.version = 1;
|
||||
assert_eq!(manifest.validate(), Err(ManifestError::Invalid));
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -371,18 +359,19 @@ fn state_namespaces_are_base_infrastructure_with_bounded_versions() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn public_manifest_fields_use_only_the_v3_camel_case_shape() {
|
||||
let value = serde_json::to_value(source_manifest()).unwrap();
|
||||
assert!(value.get("manifestVersion").is_some());
|
||||
assert!(value.get("displayName").is_some());
|
||||
assert!(value.get("configurationSchema").is_some());
|
||||
assert!(value.get("schema_version").is_none());
|
||||
fn manifest_round_trips_public_fields_and_rejects_unknown_fields() {
|
||||
let manifest = source_manifest();
|
||||
let mut value = serde_json::to_value(&manifest).unwrap();
|
||||
assert_eq!(value["manifestVersion"], MANIFEST_VERSION);
|
||||
assert_eq!(value["displayName"], manifest.display_name);
|
||||
assert_eq!(value["configurationSchema"], manifest.configuration_schema);
|
||||
assert_eq!(
|
||||
serde_json::from_value::<Manifest>(value.clone()).unwrap(),
|
||||
manifest
|
||||
);
|
||||
|
||||
let mut old = value;
|
||||
old.as_object_mut()
|
||||
.unwrap()
|
||||
.insert("schema_version".into(), json!(3));
|
||||
assert!(serde_json::from_value::<Manifest>(old).is_err());
|
||||
value["unknownField"] = json!(true);
|
||||
assert!(serde_json::from_value::<Manifest>(value).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -439,26 +428,3 @@ fn contribution_declaration_uses_camel_case_description_fields() {
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn scheduler_requires_the_current_contract() {
|
||||
let mut manifest = packaged_manifest();
|
||||
manifest.contributes.clear();
|
||||
manifest.contributes.insert(
|
||||
Capability::Scheduler,
|
||||
ContributionDeclaration {
|
||||
id: "9acme.request-tags.scheduler".into(),
|
||||
version: 3,
|
||||
stages: vec![Stage::Scheduling],
|
||||
input_formats: vec![],
|
||||
output_formats: vec![],
|
||||
},
|
||||
);
|
||||
assert!(manifest.validate().is_ok());
|
||||
manifest
|
||||
.contributes
|
||||
.get_mut(&Capability::Scheduler)
|
||||
.unwrap()
|
||||
.version = 1;
|
||||
assert_eq!(manifest.validate(), Err(ManifestError::Invalid));
|
||||
}
|
||||
|
||||
@@ -37,6 +37,7 @@ url = "2.5.8"
|
||||
uuid = { version = "1.23.3", features = ["v4", "v7"] }
|
||||
|
||||
[dev-dependencies]
|
||||
percent-encoding = "2.3"
|
||||
tempfile = "3.23"
|
||||
tokio = { version = "1.52.3", default-features = false, features = ["macros", "rt-multi-thread", "fs", "io-util", "net", "process", "time", "test-util"] }
|
||||
|
||||
|
||||
@@ -184,12 +184,17 @@ pub(crate) struct StoreConnectionConfig {
|
||||
impl StoreConnectionConfig {
|
||||
fn validate(&self, field: &'static str) -> StoreResult<()> {
|
||||
require_nonempty("store config", field, &self.url)?;
|
||||
if self.password.len() != 48 || !self.password.bytes().all(|byte| byte.is_ascii_hexdigit())
|
||||
// 托管存储的具体密码策略由服务端约束,客户端共用长度与控制字符边界
|
||||
if !(16..=128).contains(&self.password.len())
|
||||
|| self.password.chars().any(char::is_control)
|
||||
|| self.password.trim().is_empty()
|
||||
{
|
||||
return Err(StoreError::InvalidData {
|
||||
source: None,
|
||||
entity: "store config",
|
||||
message: format!("{field}.password must be exactly 48 hexadecimal characters"),
|
||||
message: format!(
|
||||
"{field}.password must be 16 to 128 bytes, not blank, without control characters"
|
||||
),
|
||||
});
|
||||
}
|
||||
self.connection_url(field).map(|_| ())
|
||||
@@ -208,7 +213,8 @@ impl StoreConnectionConfig {
|
||||
message: format!("{field}.url must not contain a password"),
|
||||
});
|
||||
}
|
||||
url.set_password(Some(&self.password))
|
||||
// set_password 保留已有百分号序列,先转义字面量 %,避免驱动解码后改变原始密码
|
||||
url.set_password(Some(&self.password.replace('%', "%25")))
|
||||
.map_err(|()| StoreError::InvalidData {
|
||||
source: None,
|
||||
entity: "store config",
|
||||
|
||||
@@ -46,4 +46,4 @@ pub use bundle::*;
|
||||
pub use config::*;
|
||||
pub use value::*;
|
||||
pub(crate) use workers::*;
|
||||
pub use workers::{CommandStoreDrainError, PostgresHealthProbe, run_request_analytics_cycle};
|
||||
pub use workers::{CommandStoreDrainError, PostgresHealthProbe, RequestAnalyticsCycle};
|
||||
|
||||
@@ -107,14 +107,6 @@ struct UpstreamTimings {
|
||||
api_overhead_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
engine_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
engine_iapi_ttft_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
engine_service_ttft_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
engine_iapi_tbt_ms: Option<f64>,
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
engine_service_tbt_ms: Option<f64>,
|
||||
}
|
||||
|
||||
#[derive(Default, Serialize)]
|
||||
@@ -229,16 +221,9 @@ impl RequestObservation {
|
||||
value.retry_after_ms,
|
||||
])?;
|
||||
// serde_json 会把非有限浮点数写成 null,先拒绝以免把非法指标伪装成缺失
|
||||
for value in [
|
||||
timings.upstream_api_overhead_ms,
|
||||
timings.upstream_engine_ms,
|
||||
timings.upstream_engine_iapi_ttft_ms,
|
||||
timings.upstream_engine_service_ttft_ms,
|
||||
timings.upstream_engine_iapi_tbt_ms,
|
||||
timings.upstream_engine_service_tbt_ms,
|
||||
]
|
||||
.into_iter()
|
||||
.flatten()
|
||||
for value in [timings.upstream_api_overhead_ms, timings.upstream_engine_ms]
|
||||
.into_iter()
|
||||
.flatten()
|
||||
{
|
||||
if !value.is_finite() || value < 0.0 || value >= i64::MAX as f64 {
|
||||
return Err(invalid(
|
||||
@@ -263,10 +248,6 @@ impl RequestObservation {
|
||||
response_ms: timings.upstream_response_ms,
|
||||
api_overhead_ms: timings.upstream_api_overhead_ms,
|
||||
engine_ms: timings.upstream_engine_ms,
|
||||
engine_iapi_ttft_ms: timings.upstream_engine_iapi_ttft_ms,
|
||||
engine_service_ttft_ms: timings.upstream_engine_service_ttft_ms,
|
||||
engine_iapi_tbt_ms: timings.upstream_engine_iapi_tbt_ms,
|
||||
engine_service_tbt_ms: timings.upstream_engine_service_tbt_ms,
|
||||
}),
|
||||
}),
|
||||
transport: Some(TransportObservation {
|
||||
|
||||
@@ -192,6 +192,8 @@ async fn apply_changes(
|
||||
}
|
||||
let mut buckets =
|
||||
BTreeMap::<DateTime<Utc>, BTreeMap<(String, Vec<String>, String), MetricState>>::new();
|
||||
let mut empty_groups = Vec::new();
|
||||
let mut rebuild = Vec::new();
|
||||
for (key, change) in changes {
|
||||
if change.previous.encode()? == change.current.encode()?
|
||||
&& change.previous.last_used_at() == change.current.last_used_at()
|
||||
@@ -202,21 +204,17 @@ async fn apply_changes(
|
||||
let needs_rebuild = subtract(&mut state, &change.previous, &change.current)?;
|
||||
state.merge(&change.current)?;
|
||||
if state.count("request_count") == 0 {
|
||||
let statement = format!(
|
||||
"delete from {table} where bucket_start=$1 and collection=$2 and dimensions=$3 and outcome=$4"
|
||||
);
|
||||
sqlx::query(sqlx::AssertSqlSafe(statement))
|
||||
.bind(key.0)
|
||||
.bind(&key.1)
|
||||
.bind(&key.2)
|
||||
.bind(&key.3)
|
||||
.execute(&mut **tx)
|
||||
.await
|
||||
.map_err(|error| postgres_unavailable("remove empty analytics group", error))?;
|
||||
empty_groups.push(serde_json::json!({
|
||||
"bucket_start": key.0,
|
||||
"collection": key.1,
|
||||
"dimensions": key.2,
|
||||
"outcome": key.3,
|
||||
}));
|
||||
} else {
|
||||
// 预测状态只有可加计数和美元总额,零金额无需修复币种存在性
|
||||
if needs_rebuild && key.1 != forecast::COLLECTION {
|
||||
state = rebuild_group(tx, hourly, progress, &key).await?;
|
||||
rebuild.push(key);
|
||||
continue;
|
||||
}
|
||||
buckets
|
||||
.entry(key.0)
|
||||
@@ -224,6 +222,24 @@ async fn apply_changes(
|
||||
.insert((key.1, key.2, key.3), state);
|
||||
}
|
||||
}
|
||||
for (key, state) in rebuild_groups(tx, hourly, progress, rebuild).await? {
|
||||
buckets
|
||||
.entry(key.0)
|
||||
.or_default()
|
||||
.insert((key.1, key.2, key.3), state);
|
||||
}
|
||||
// 保留期缩短会同时清空大量分组,集合删除避免 SQL 往返随分组数放大
|
||||
for groups in empty_groups.chunks(1_000) {
|
||||
let statement = format!(
|
||||
"delete from {table} a using jsonb_to_recordset($1::jsonb) k(bucket_start timestamptz,collection text,dimensions text[],outcome text)
|
||||
where a.bucket_start=k.bucket_start and a.collection=k.collection and a.dimensions=k.dimensions and a.outcome=k.outcome"
|
||||
);
|
||||
sqlx::query(sqlx::AssertSqlSafe(statement))
|
||||
.bind(sqlx::types::Json(groups))
|
||||
.execute(&mut **tx)
|
||||
.await
|
||||
.map_err(|error| postgres_unavailable("remove empty analytics groups", error))?;
|
||||
}
|
||||
write_bucket_groups(tx, table, buckets).await
|
||||
}
|
||||
|
||||
@@ -311,61 +327,91 @@ pub(super) fn subtract(
|
||||
Ok(rebuild)
|
||||
}
|
||||
|
||||
async fn rebuild_group(
|
||||
async fn rebuild_groups(
|
||||
tx: &mut sqlx::Transaction<'_, Postgres>,
|
||||
hourly: bool,
|
||||
progress: &BTreeMap<DateTime<Utc>, Progress>,
|
||||
key: &GroupKey,
|
||||
) -> StoreResult<MetricState> {
|
||||
let mut end = key.0 + TimeDelta::seconds(if hourly { 3600 } else { 900 });
|
||||
let mut cursor = None;
|
||||
if hourly {
|
||||
if let Some((_, owner)) = progress.range(key.0..end).find(|(_, p)| p.phase == 4)
|
||||
&& let Some(time) = owner.started_at
|
||||
{
|
||||
let candidate = (time, key.1.clone(), key.2.clone(), key.3.clone());
|
||||
end = if owner.contains_quarter(&candidate) {
|
||||
time + TimeDelta::minutes(15)
|
||||
} else {
|
||||
time
|
||||
};
|
||||
keys: Vec<GroupKey>,
|
||||
) -> StoreResult<BTreeMap<GroupKey, MetricState>> {
|
||||
let mut by_collection = BTreeMap::<&str, Vec<&GroupKey>>::new();
|
||||
for key in &keys {
|
||||
by_collection.entry(&key.1).or_default().push(key);
|
||||
}
|
||||
let mut rebuilt = BTreeMap::new();
|
||||
for (collection, keys) in by_collection {
|
||||
let columns = worker::COLLECTION_COLUMNS
|
||||
.iter()
|
||||
.find_map(|(name, columns)| (*name == collection).then_some(*columns))
|
||||
.ok_or_else(|| invalid("unknown incremental analytics collection"))?;
|
||||
// 同类分组共用维度匹配和聚合 SQL,各组仍独立遵守已发布的构建前缀
|
||||
for keys in keys.chunks(1_000) {
|
||||
let mut input = Vec::with_capacity(keys.len());
|
||||
for (index, key) in keys.iter().enumerate() {
|
||||
let mut end = key.0 + TimeDelta::seconds(if hourly { 3600 } else { 900 });
|
||||
let mut cursor = None;
|
||||
if hourly {
|
||||
if let Some((_, owner)) = progress.range(key.0..end).find(|(_, p)| p.phase == 4)
|
||||
&& let Some(time) = owner.started_at
|
||||
{
|
||||
let candidate = (time, key.1.clone(), key.2.clone(), key.3.clone());
|
||||
end = if owner.contains_quarter(&candidate) {
|
||||
time + TimeDelta::minutes(15)
|
||||
} else {
|
||||
time
|
||||
};
|
||||
}
|
||||
} else if let Some(stage) = progress.get(&key.0)
|
||||
&& stage.phase == 1
|
||||
{
|
||||
cursor = stage.started_at.zip(stage.id.as_deref());
|
||||
}
|
||||
input.push(serde_json::json!({
|
||||
"bucket_start": key.0,
|
||||
"end_at": end,
|
||||
"cursor_time": cursor.map(|(time, _)| time),
|
||||
"cursor_id": cursor.map(|(_, id)| id),
|
||||
"dimensions": key.2,
|
||||
"outcome": key.3,
|
||||
"name": index.to_string(),
|
||||
}));
|
||||
rebuilt.insert((*key).clone(), MetricState::default());
|
||||
}
|
||||
// 等值桶键避免范围连接反复比较其他时间桶,范围和游标仍限定已发布前缀
|
||||
let mut query = QueryBuilder::<Postgres>::new(
|
||||
"with matching as materialized (select mr.*,k.bucket_start as output_bucket,k.name,coalesce((",
|
||||
);
|
||||
query.push(completed_usage_fact_predicate("mr"))
|
||||
.push("),false) as is_completed_usage from jsonb_to_recordset(")
|
||||
.push_bind(serde_json::Value::Array(input))
|
||||
.push("::jsonb) k(bucket_start timestamptz,end_at timestamptz,cursor_time timestamptz,cursor_id text,dimensions text[],outcome text,name text)
|
||||
join request_analytics_facts mr on date_bin(make_interval(secs => ")
|
||||
.push_bind(if hourly { 3600_f64 } else { 900_f64 })
|
||||
.push("),mr.started_at,timestamptz '2000-01-01 UTC')=k.bucket_start
|
||||
and mr.started_at >= k.bucket_start and mr.started_at < k.end_at
|
||||
and mr.recovered_at is null and mr.outcome=k.outcome
|
||||
and (k.cursor_time is null or (mr.started_at,mr.id collate \"C\") <= (k.cursor_time,k.cursor_id collate \"C\"))");
|
||||
for (index, column) in columns.iter().enumerate() {
|
||||
query.push(format!(
|
||||
" and coalesce(mr.{column}::text,'')=k.dimensions[{}]",
|
||||
index + 1
|
||||
));
|
||||
}
|
||||
query.push(")");
|
||||
aggregate::push_fact_aggregation(&mut query, false, true);
|
||||
let rows = query.build().fetch_all(&mut **tx).await.map_err(|error| {
|
||||
postgres_unavailable("rebuild changed analytics extrema", error)
|
||||
})?;
|
||||
for row in rows {
|
||||
let name: String = get(&row, "name")?;
|
||||
let index = name
|
||||
.parse::<usize>()
|
||||
.map_err(|_| invalid("invalid analytics rebuild group"))?;
|
||||
let key = keys
|
||||
.get(index)
|
||||
.ok_or_else(|| invalid("unknown analytics rebuild group"))?;
|
||||
rebuilt.insert((*key).clone(), MetricState::decode(&row)?);
|
||||
}
|
||||
}
|
||||
} else if let Some(stage) = progress.get(&key.0)
|
||||
&& stage.phase == 1
|
||||
{
|
||||
cursor = stage.started_at.zip(stage.id.as_deref());
|
||||
}
|
||||
let mut query = QueryBuilder::<Postgres>::new("with matching as materialized (select mr.*,");
|
||||
query.push_bind(key.0).push("::timestamptz as output_bucket,''::text as name,coalesce((")
|
||||
.push(completed_usage_fact_predicate("mr")).push("),false) as is_completed_usage from request_analytics_facts mr where mr.started_at >= ")
|
||||
.push_bind(key.0).push(" and mr.started_at < ").push_bind(end)
|
||||
.push(" and mr.recovered_at is null and mr.outcome=").push_bind(&key.3);
|
||||
if let Some((time, id)) = cursor {
|
||||
query
|
||||
.push(" and (mr.started_at,mr.id collate \"C\") <= (")
|
||||
.push_bind(time)
|
||||
.push(",")
|
||||
.push_bind(id)
|
||||
.push(")");
|
||||
}
|
||||
let columns = worker::COLLECTION_COLUMNS
|
||||
.iter()
|
||||
.find_map(|(collection, columns)| (*collection == key.1).then_some(*columns))
|
||||
.ok_or_else(|| invalid("unknown incremental analytics collection"))?;
|
||||
for (column, value) in columns.iter().zip(&key.2) {
|
||||
query
|
||||
.push(format!(" and coalesce(mr.{column}::text,'')="))
|
||||
.push_bind(value);
|
||||
}
|
||||
query.push(")");
|
||||
aggregate::push_fact_aggregation(&mut query, false, true);
|
||||
let row = query
|
||||
.build()
|
||||
.fetch_optional(&mut **tx)
|
||||
.await
|
||||
.map_err(|error| postgres_unavailable("rebuild changed analytics extrema", error))?;
|
||||
row.as_ref()
|
||||
.map(MetricState::decode)
|
||||
.transpose()
|
||||
.map(Option::unwrap_or_default)
|
||||
Ok(rebuilt)
|
||||
}
|
||||
|
||||
@@ -83,7 +83,17 @@ impl PgRequestAnalyticsRepository {
|
||||
return Ok(AnalyticsMaintenance { reset_progress: true, ..Default::default() });
|
||||
}
|
||||
let mut result = AnalyticsMaintenance::default();
|
||||
let rows = sqlx::query("select request_id, change_version from request_analytics_pending_requests order by oldest_pending_at, request_id limit $1")
|
||||
// 历史删除可能一次排入大量旧任务,给仍存在的请求预留半批名额,避免实时统计饥饿
|
||||
// 其余名额仍按最早入队消费,两类任务都能前进;同一快照内选版本,沿用精确版本 ACK
|
||||
let rows = sqlx::query("with live as materialized (
|
||||
select pending.request_id, pending.change_version from request_analytics_pending_requests pending
|
||||
join model_requests source on source.id=pending.request_id
|
||||
order by pending.oldest_pending_at, pending.request_id limit ($1::bigint / 2)
|
||||
), oldest as (
|
||||
select pending.request_id, pending.change_version from request_analytics_pending_requests pending
|
||||
where not exists (select 1 from live where live.request_id=pending.request_id)
|
||||
order by pending.oldest_pending_at, pending.request_id limit ($1::bigint - (select count(*) from live))
|
||||
) select request_id, change_version from live union all select request_id, change_version from oldest")
|
||||
.bind(i64::from(batch_size)).fetch_all(&mut *transaction).await.map_err(|error| postgres_unavailable("load analytics request queue", error))?;
|
||||
let mut jobs = Vec::with_capacity(rows.len());
|
||||
for row in rows {
|
||||
|
||||
@@ -350,10 +350,6 @@ pub(crate) fn usage_list_record_from_row(
|
||||
upstream_response_ms: optional_unsigned(row, "upstream_response_ms")?,
|
||||
upstream_api_overhead_ms: get(row, "upstream_api_overhead_ms")?,
|
||||
upstream_engine_ms: get(row, "upstream_engine_ms")?,
|
||||
upstream_engine_iapi_ttft_ms: get(row, "upstream_engine_iapi_ttft_ms")?,
|
||||
upstream_engine_service_ttft_ms: get(row, "upstream_engine_service_ttft_ms")?,
|
||||
upstream_engine_iapi_tbt_ms: get(row, "upstream_engine_iapi_tbt_ms")?,
|
||||
upstream_engine_service_tbt_ms: get(row, "upstream_engine_service_tbt_ms")?,
|
||||
latency_ms: optional_unsigned(row, "latency_ms")?,
|
||||
admission_decision_ms: optional_unsigned(row, "admission_decision_ms")?,
|
||||
account_selection_wait_ms: optional_unsigned(row, "account_selection_wait_ms")?,
|
||||
@@ -446,10 +442,6 @@ pub(crate) fn usage_record_from_row(row: &sqlx::postgres::PgRow) -> StoreResult<
|
||||
upstream_response_ms: optional_unsigned(row, "upstream_response_ms")?,
|
||||
upstream_api_overhead_ms: get(row, "upstream_api_overhead_ms")?,
|
||||
upstream_engine_ms: get(row, "upstream_engine_ms")?,
|
||||
upstream_engine_iapi_ttft_ms: get(row, "upstream_engine_iapi_ttft_ms")?,
|
||||
upstream_engine_service_ttft_ms: get(row, "upstream_engine_service_ttft_ms")?,
|
||||
upstream_engine_iapi_tbt_ms: get(row, "upstream_engine_iapi_tbt_ms")?,
|
||||
upstream_engine_service_tbt_ms: get(row, "upstream_engine_service_tbt_ms")?,
|
||||
latency_ms: optional_unsigned(row, "latency_ms")?,
|
||||
admission_decision_ms: optional_unsigned(row, "admission_decision_ms")?,
|
||||
account_selection_wait_ms: optional_unsigned(row, "account_selection_wait_ms")?,
|
||||
|
||||
@@ -128,7 +128,7 @@ pub(crate) const USAGE_LIST_RECORD_SELECT: &str =
|
||||
mr.transport_decision_wait_ms, mr.connect_ms, mr.headers_ms,
|
||||
mr.first_event_ms, mr.first_reasoning_ms, mr.first_text_ms, mr.first_token_ms,
|
||||
mr.provider_processing_ms, mr.upstream_response_ms,
|
||||
mr.upstream_api_overhead_ms, mr.upstream_engine_ms, mr.upstream_engine_iapi_ttft_ms, mr.upstream_engine_service_ttft_ms, mr.upstream_engine_iapi_tbt_ms, mr.upstream_engine_service_tbt_ms, mr.latency_ms, mr.admission_decision_ms,
|
||||
mr.upstream_api_overhead_ms, mr.upstream_engine_ms, mr.latency_ms, mr.admission_decision_ms,
|
||||
mr.account_selection_wait_ms, mr.capacity_used_slots, mr.capacity_total_slots,
|
||||
host(mr.client_ip) as client_ip, mr.user_agent,
|
||||
mr.reasoning_effort, mr.reasoning_preset, mr.subagent_kind, mr.compact,
|
||||
@@ -157,7 +157,7 @@ pub(crate) const USAGE_RECORD_DETAIL_SELECT: &str =
|
||||
mr.cost_currency, mr.transport_decision_wait_ms, mr.connect_ms, mr.headers_ms,
|
||||
mr.first_event_ms, mr.first_reasoning_ms, mr.first_text_ms, mr.first_token_ms,
|
||||
mr.provider_processing_ms, mr.upstream_response_ms,
|
||||
mr.upstream_api_overhead_ms, mr.upstream_engine_ms, mr.upstream_engine_iapi_ttft_ms, mr.upstream_engine_service_ttft_ms, mr.upstream_engine_iapi_tbt_ms, mr.upstream_engine_service_tbt_ms, mr.latency_ms, mr.admission_decision_ms,
|
||||
mr.upstream_api_overhead_ms, mr.upstream_engine_ms, mr.latency_ms, mr.admission_decision_ms,
|
||||
mr.account_selection_wait_ms, mr.capacity_used_slots, mr.capacity_total_slots,
|
||||
host(mr.client_ip) as client_ip,
|
||||
mr.user_agent, mr.reasoning_effort, mr.reasoning_preset, mr.request_kind,
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
use super::*;
|
||||
use gateway_core::task::DaemonTask;
|
||||
use std::sync::atomic::{AtomicU32, Ordering};
|
||||
|
||||
const COMMAND_DRAIN_TIMEOUT: Duration = Duration::from_secs(4);
|
||||
|
||||
@@ -325,12 +326,16 @@ pub(crate) fn request_analytics_worker(
|
||||
Ok(WorkerContribution::Registration(scheduled_worker(
|
||||
id,
|
||||
Duration::from_secs(5),
|
||||
Box::new(RequestAnalyticsTask { repository }),
|
||||
Box::new(RequestAnalyticsTask {
|
||||
repository,
|
||||
cycle: RequestAnalyticsCycle::default(),
|
||||
}),
|
||||
)?))
|
||||
}
|
||||
|
||||
struct RequestAnalyticsTask {
|
||||
repository: postgres::PgRequestAnalyticsRepository,
|
||||
cycle: RequestAnalyticsCycle,
|
||||
}
|
||||
|
||||
impl ScheduledTask for RequestAnalyticsTask {
|
||||
@@ -341,7 +346,7 @@ impl ScheduledTask for RequestAnalyticsTask {
|
||||
Box::pin(async move {
|
||||
tokio::select! {
|
||||
() = context.cancellation().cancelled() => Ok(()),
|
||||
result = run_request_analytics_cycle(&self.repository) => result.map(|_| ()).map_err(|source| {
|
||||
result = self.cycle.run(&self.repository) => result.map(|_| ()).map_err(|source| {
|
||||
WorkerTaskError::safe("request analytics maintenance failed").with_source(source)
|
||||
}),
|
||||
}
|
||||
@@ -349,43 +354,107 @@ impl ScheduledTask for RequestAnalyticsTask {
|
||||
}
|
||||
}
|
||||
|
||||
/// 生产与隔离压测共用的单周期推进预算;各批独立提交并归还观测槽位
|
||||
pub async fn run_request_analytics_cycle(
|
||||
repository: &postgres::PgRequestAnalyticsRepository,
|
||||
) -> StoreResult<postgres::AnalyticsMaintenance> {
|
||||
let deadline = tokio::time::Instant::now() + Duration::from_secs(10);
|
||||
let mut total = postgres::AnalyticsMaintenance::default();
|
||||
loop {
|
||||
let result = match tokio::time::timeout_at(deadline, repository.maintain(1_000)).await {
|
||||
Ok(result) => result?,
|
||||
Err(source) => {
|
||||
if total.projected_requests == 0
|
||||
&& total.backfilled_requests == 0
|
||||
&& total.rebuilt_buckets == 0
|
||||
&& total.advanced_buckets == 0
|
||||
&& !total.reset_progress
|
||||
{
|
||||
return Err(StoreError::Unavailable {
|
||||
backend: StoreBackend::PostgreSql,
|
||||
message: "request analytics cycle timed out without progress".to_owned(),
|
||||
source: Some(gateway_core::error::ErrorSource::new(source)),
|
||||
});
|
||||
}
|
||||
return Ok(total);
|
||||
}
|
||||
};
|
||||
let idle = result.projected_requests == 0
|
||||
&& result.backfilled_requests == 0
|
||||
&& result.rebuilt_buckets == 0
|
||||
&& result.advanced_buckets == 0
|
||||
&& !result.reset_progress;
|
||||
total.projected_requests += result.projected_requests;
|
||||
total.backfilled_requests += result.backfilled_requests;
|
||||
total.rebuilt_buckets += result.rebuilt_buckets;
|
||||
total.advanced_buckets += result.advanced_buckets;
|
||||
total.reset_progress |= result.reset_progress;
|
||||
if idle || tokio::time::Instant::now() >= deadline {
|
||||
return Ok(total);
|
||||
/// 统计消费周期预算,跨轮保留批量上限以便超时后的下一轮提交进展
|
||||
pub struct RequestAnalyticsCycle {
|
||||
batch_size: AtomicU32,
|
||||
}
|
||||
|
||||
impl Default for RequestAnalyticsCycle {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
batch_size: AtomicU32::new(1_000),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl RequestAnalyticsCycle {
|
||||
/// 生产与隔离压测共用的推进路径;各批独立提交并归还观测槽位
|
||||
pub async fn run(
|
||||
&self,
|
||||
repository: &postgres::PgRequestAnalyticsRepository,
|
||||
) -> StoreResult<postgres::AnalyticsMaintenance> {
|
||||
let deadline = tokio::time::Instant::now() + Duration::from_secs(10);
|
||||
let mut total = postgres::AnalyticsMaintenance::default();
|
||||
loop {
|
||||
let batch_size = self.batch_size.load(Ordering::Relaxed);
|
||||
let started = tokio::time::Instant::now();
|
||||
let result =
|
||||
match tokio::time::timeout_at(deadline, repository.maintain(batch_size)).await {
|
||||
Ok(Ok(result)) => result,
|
||||
Ok(Err(error)) => {
|
||||
if analytics_maintenance_timed_out(&error) {
|
||||
self.reduce_batch_size(batch_size);
|
||||
}
|
||||
return Err(error);
|
||||
}
|
||||
Err(source) => {
|
||||
if total.projected_requests == 0
|
||||
&& total.backfilled_requests == 0
|
||||
&& total.rebuilt_buckets == 0
|
||||
&& total.advanced_buckets == 0
|
||||
&& !total.reset_progress
|
||||
{
|
||||
// 只在整轮没有返回进展时退让,避免尾批剩余时间短导致持续缩批
|
||||
self.reduce_batch_size(batch_size);
|
||||
return Err(StoreError::Unavailable {
|
||||
backend: StoreBackend::PostgreSql,
|
||||
message: "request analytics cycle timed out without progress"
|
||||
.to_owned(),
|
||||
source: Some(gateway_core::error::ErrorSource::new(source)),
|
||||
});
|
||||
}
|
||||
return Ok(total);
|
||||
}
|
||||
};
|
||||
let idle = result.projected_requests == 0
|
||||
&& result.backfilled_requests == 0
|
||||
&& result.rebuilt_buckets == 0
|
||||
&& result.advanced_buckets == 0
|
||||
&& !result.reset_progress;
|
||||
total.projected_requests += result.projected_requests;
|
||||
total.backfilled_requests += result.backfilled_requests;
|
||||
total.rebuilt_buckets += result.rebuilt_buckets;
|
||||
total.advanced_buckets += result.advanced_buckets;
|
||||
total.reset_progress |= result.reset_progress;
|
||||
// 满批快速完成才恢复吞吐,空结果也可能只是其他实例持有维护锁
|
||||
if (result.projected_requests >= u64::from(batch_size)
|
||||
|| result.backfilled_requests >= u64::from(batch_size))
|
||||
&& started.elapsed() < Duration::from_secs(2)
|
||||
{
|
||||
self.batch_size
|
||||
.store((batch_size * 2).min(1_000), Ordering::Relaxed);
|
||||
}
|
||||
if idle {
|
||||
return Ok(total);
|
||||
}
|
||||
if tokio::time::Instant::now() >= deadline {
|
||||
return Ok(total);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn reduce_batch_size(&self, batch_size: u32) {
|
||||
let next_batch_size = (batch_size / 2).max(1);
|
||||
self.batch_size.store(next_batch_size, Ordering::Relaxed);
|
||||
tracing::warn!(
|
||||
batch_size,
|
||||
next_batch_size,
|
||||
"request analytics batch timed out"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
fn analytics_maintenance_timed_out(error: &StoreError) -> bool {
|
||||
let StoreError::Unavailable {
|
||||
source: Some(source),
|
||||
..
|
||||
} = error
|
||||
else {
|
||||
return false;
|
||||
};
|
||||
source.is::<tokio::time::error::Elapsed>()
|
||||
|| source
|
||||
.downcast_ref::<sqlx::Error>()
|
||||
.and_then(sqlx::Error::as_database_error)
|
||||
.is_some_and(|error| error.code().as_deref() == Some("57014"))
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
//! 验证只读检查 Bundle 不执行迁移且拒绝业务写入
|
||||
//! 验证 Store 认证、凭据编码与只读检查 Bundle 的写入边界
|
||||
|
||||
use gateway_core::{
|
||||
account::{
|
||||
@@ -33,11 +33,6 @@ async fn inspection_bundle_does_not_migrate_and_rejects_business_writes() {
|
||||
database
|
||||
.query_pairs_mut()
|
||||
.append_pair("options", &format!("-csearch_path={schema}"));
|
||||
let config_connection = |mut url: url::Url| {
|
||||
let password = url.password().unwrap().to_owned();
|
||||
url.set_password(None).unwrap();
|
||||
json!({"url":url.as_str(),"password":password})
|
||||
};
|
||||
let mut config: StoreConfig = serde_json::from_value(json!({
|
||||
"database":config_connection(database), "redis":config_connection(url::Url::parse(&redis).unwrap()),
|
||||
})).unwrap();
|
||||
@@ -109,3 +104,136 @@ async fn inspection_bundle_does_not_migrate_and_rejects_business_writes() {
|
||||
.unwrap();
|
||||
admin.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn store_bundle_authenticates_configured_and_redis_acl_credentials() {
|
||||
let (Some(database), Some(redis_url)) = (
|
||||
crate::support::test_env("CPR_TEST_DATABASE_URL"),
|
||||
crate::support::test_env("CPR_TEST_REDIS_URL"),
|
||||
) else {
|
||||
return;
|
||||
};
|
||||
let document = json!({
|
||||
"database": config_connection(url::Url::parse(&database).unwrap()),
|
||||
"redis": config_connection(url::Url::parse(&redis_url).unwrap()),
|
||||
});
|
||||
let mut admin = redis::Client::open(redis_url.as_str())
|
||||
.unwrap()
|
||||
.get_connection_manager()
|
||||
.await
|
||||
.unwrap();
|
||||
let username = format!("cpr_password_{}", uuid::Uuid::new_v4().simple());
|
||||
let password = "Managed%41%25%FF@:/?#%&+$'\"\\ secret9!";
|
||||
redis::cmd("ACL")
|
||||
.arg("SETUSER")
|
||||
.arg(&username)
|
||||
.arg("on")
|
||||
.arg(format!(">{password}"))
|
||||
.arg("~*")
|
||||
.arg("&*")
|
||||
.arg("+@all")
|
||||
.query_async::<()>(&mut admin)
|
||||
.await
|
||||
.unwrap();
|
||||
let mut acl_url = url::Url::parse(&redis_url).unwrap();
|
||||
acl_url.set_username(&username).unwrap();
|
||||
acl_url.set_password(None).unwrap();
|
||||
let mut acl_document = document.clone();
|
||||
acl_document["redis"] = json!({"url": acl_url.as_str(), "password": password});
|
||||
let directory = tempfile::tempdir().unwrap();
|
||||
let mut results = Vec::new();
|
||||
// 同时走 Store 的配置解析、URL 构造与真实连接路径,断言前先清理临时 ACL 用户
|
||||
for document in [document, acl_document] {
|
||||
let mut config: StoreConfig = serde_json::from_value(document).unwrap();
|
||||
let result = async {
|
||||
config.resolve_and_validate(directory.path())?;
|
||||
let bundle = gateway_store::initialize_read_only(config).await?;
|
||||
let mut states = Vec::new();
|
||||
for probe in bundle.health_probes() {
|
||||
states.push(probe.check().await);
|
||||
}
|
||||
Ok::<_, gateway_store::StoreError>(states)
|
||||
}
|
||||
.await;
|
||||
results.push(result);
|
||||
}
|
||||
redis::cmd("ACL")
|
||||
.arg("DELUSER")
|
||||
.arg(&username)
|
||||
.query_async::<i64>(&mut admin)
|
||||
.await
|
||||
.unwrap();
|
||||
for result in results {
|
||||
let states = result.expect("Store credentials authenticate without double encoding");
|
||||
assert!(!states.is_empty());
|
||||
assert!(
|
||||
states
|
||||
.iter()
|
||||
.all(|state| *state == gateway_core::health::HealthState::Healthy)
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn store_bundle_authenticates_postgres_password_with_percent_sequences() {
|
||||
let (Some(database), Some(redis_url)) = (
|
||||
crate::support::test_env("CPR_TEST_DATABASE_URL"),
|
||||
crate::support::test_env("CPR_TEST_REDIS_URL"),
|
||||
) else {
|
||||
return;
|
||||
};
|
||||
let admin = PgPoolOptions::new()
|
||||
.max_connections(1)
|
||||
.connect(&database)
|
||||
.await
|
||||
.unwrap();
|
||||
let username = format!("cpr_password_{}", uuid::Uuid::new_v4().simple());
|
||||
let password = "Managed%41%25%FFPassword9!";
|
||||
sqlx::raw_sql(sqlx::AssertSqlSafe(format!(
|
||||
"CREATE ROLE {username} LOGIN PASSWORD '{password}'"
|
||||
)))
|
||||
.execute(&admin)
|
||||
.await
|
||||
.unwrap();
|
||||
let mut database_url = url::Url::parse(&database).unwrap();
|
||||
database_url.set_username(&username).unwrap();
|
||||
database_url.set_password(None).unwrap();
|
||||
let mut config: StoreConfig = serde_json::from_value(json!({
|
||||
"database": {"url": database_url.as_str(), "password": password},
|
||||
"redis": config_connection(url::Url::parse(&redis_url).unwrap()),
|
||||
}))
|
||||
.unwrap();
|
||||
let directory = tempfile::tempdir().unwrap();
|
||||
let result = async {
|
||||
config.resolve_and_validate(directory.path())?;
|
||||
let bundle = gateway_store::initialize_read_only(config).await?;
|
||||
let mut states = Vec::new();
|
||||
for probe in bundle.health_probes() {
|
||||
states.push(probe.check().await);
|
||||
}
|
||||
Ok::<_, gateway_store::StoreError>(states)
|
||||
}
|
||||
.await;
|
||||
// 连接失败时也先清理临时角色,避免回归断言留下测试账号
|
||||
sqlx::raw_sql(sqlx::AssertSqlSafe(format!("DROP ROLE {username}")))
|
||||
.execute(&admin)
|
||||
.await
|
||||
.unwrap();
|
||||
admin.close().await;
|
||||
let states = result.expect("PostgreSQL password percent sequences remain literal");
|
||||
assert!(!states.is_empty());
|
||||
assert!(
|
||||
states
|
||||
.iter()
|
||||
.all(|state| *state == gateway_core::health::HealthState::Healthy)
|
||||
);
|
||||
}
|
||||
|
||||
fn config_connection(mut url: url::Url) -> serde_json::Value {
|
||||
let password = percent_encoding::percent_decode_str(url.password().unwrap())
|
||||
.decode_utf8()
|
||||
.expect("UTF-8 fixture password")
|
||||
.into_owned();
|
||||
url.set_password(None).unwrap();
|
||||
json!({"url": url.as_str(), "password": password})
|
||||
}
|
||||
|
||||
@@ -20,7 +20,7 @@ fn store_config_should_derive_backup_staging_from_runtime_data_dir() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn store_pool_config_ignores_unknown_options_and_keeps_connection_pool_limits() {
|
||||
fn store_pool_config_defaults_reserve_capacity_for_observability() {
|
||||
let pool: StorePoolConfig =
|
||||
serde_json::from_value(serde_json::json!({})).expect("default pool configuration");
|
||||
|
||||
@@ -37,13 +37,6 @@ fn store_pool_config_ignores_unknown_options_and_keeps_connection_pool_limits()
|
||||
.observability_max_connections(),
|
||||
40
|
||||
);
|
||||
assert_eq!(
|
||||
serde_json::from_value::<StorePoolConfig>(serde_json::json!({
|
||||
"statement_timeout_seconds": 60,
|
||||
}))
|
||||
.unwrap(),
|
||||
pool,
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -27,36 +27,6 @@ use super::TestDatabase;
|
||||
const MIXED_GROUP: &str = "grp_00000000000000000000000000000001";
|
||||
const EMPTY_GROUP: &str = "grp_00000000000000000000000000000002";
|
||||
|
||||
#[tokio::test]
|
||||
async fn fast_mode_migration_preserves_existing_group_choices() {
|
||||
let Some(database) = TestDatabase::create_through("group_fast_mode_upgrade", 21).await else {
|
||||
return;
|
||||
};
|
||||
for (id, disabled) in [(MIXED_GROUP, true), (EMPTY_GROUP, false)] {
|
||||
sqlx::query("insert into account_groups (id, name, color, disable_fast, created_at, updated_at) values ($1, $1, '#2563EBFF', $2, now(), now())")
|
||||
.bind(id)
|
||||
.bind(disabled)
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
let groups = sqlx::query_as::<_, (String, String)>(
|
||||
"select id, fast_mode from account_groups order by id",
|
||||
)
|
||||
.fetch_all(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
groups,
|
||||
vec![
|
||||
(MIXED_GROUP.to_owned(), "disabled".to_owned()),
|
||||
(EMPTY_GROUP.to_owned(), "default".to_owned())
|
||||
]
|
||||
);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn groups_aggregate_cross_provider_members_and_key_bindings_without_multiplication() {
|
||||
let Some(database) = TestDatabase::create("account_group_aggregate").await else {
|
||||
@@ -840,20 +810,19 @@ async fn image_generation_group_updates_preserve_omitted_values_and_publish_snap
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn image_generation_migration_allows_existing_and_default_groups() {
|
||||
let Some(database) = TestDatabase::create_through("group_image_upgrade", 31).await else {
|
||||
async fn image_generation_defaults_to_allowed_for_new_groups() {
|
||||
let Some(database) = TestDatabase::create("group_image_default").await else {
|
||||
return;
|
||||
};
|
||||
sqlx::query("insert into account_groups (id, name, color, created_at, updated_at) values ($1, $1, '#2563EBFF', now(), now())")
|
||||
.bind(MIXED_GROUP).execute(&database.pool).await.unwrap();
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
sqlx::query("insert into account_groups (id, name, color, created_at, updated_at) values ($1, $1, '#2563EBFF', now(), now())")
|
||||
.bind(EMPTY_GROUP).execute(&database.pool).await.unwrap();
|
||||
let allowed =
|
||||
sqlx::query_scalar::<_, bool>("select allow_image_generation from account_groups")
|
||||
.fetch_all(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(allowed, vec![true, true]);
|
||||
let allowed = sqlx::query_scalar::<_, bool>(
|
||||
"select allow_image_generation from account_groups where id = $1",
|
||||
)
|
||||
.bind(EMPTY_GROUP)
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(allowed);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
@@ -1,87 +0,0 @@
|
||||
//! 验证统计升级保留历史事实与回填进度,并重建已有时间桶
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
use serde_json::Value;
|
||||
|
||||
use super::{TEST_MIGRATOR, TestDatabase};
|
||||
|
||||
#[tokio::test]
|
||||
async fn analytics_upgrade_preserves_facts_and_backfill_progress_and_requeues_buckets() {
|
||||
let Some(database) = TestDatabase::create_through("analytics_upgrade", 29).await else {
|
||||
return;
|
||||
};
|
||||
sqlx::raw_sql(
|
||||
"insert into request_analytics_facts(id,started_at,source_version) values
|
||||
('first','2026-10-01 10:01:00+00',1),
|
||||
('same_bucket','2026-10-01 10:02:00+00',2),
|
||||
('next_bucket','2026-10-01 10:16:00+00',3);
|
||||
insert into request_analytics_pending_buckets(bucket_start,oldest_pending_at)
|
||||
values ('2026-10-01 10:00:00+00','2026-10-01 09:00:00+00');
|
||||
update request_analytics_build_state set
|
||||
backfill_complete=true, backfill_cursor_id='next_bucket',
|
||||
backfill_cursor_started_at='2026-10-01 10:16:00+00',
|
||||
coverage_start='2026-10-01 10:00:00+00',
|
||||
coverage_end='2026-10-01 10:30:00+00', generation=7;",
|
||||
)
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
let facts_before: Vec<Value> =
|
||||
sqlx::query_scalar("select to_jsonb(f) from request_analytics_facts f order by id")
|
||||
.fetch_all(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
let progress_before: Value =
|
||||
sqlx::query_scalar("select to_jsonb(s) from request_analytics_build_state s")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
let pending_before: (i64, DateTime<Utc>) = sqlx::query_as(
|
||||
"select change_version,oldest_pending_at from request_analytics_pending_buckets",
|
||||
)
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
|
||||
let facts_after: Vec<Value> =
|
||||
sqlx::query_scalar("select to_jsonb(f) from request_analytics_facts f order by id")
|
||||
.fetch_all(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
let progress_after: Value =
|
||||
sqlx::query_scalar("select to_jsonb(s) from request_analytics_build_state s")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(facts_after, facts_before);
|
||||
assert_eq!(progress_after, progress_before);
|
||||
let pending: Vec<(DateTime<Utc>, i64, DateTime<Utc>)> = sqlx::query_as(
|
||||
"select bucket_start,change_version,oldest_pending_at
|
||||
from request_analytics_pending_buckets order by bucket_start",
|
||||
)
|
||||
.fetch_all(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
let expected_buckets = ["2026-10-01T10:00:00Z", "2026-10-01T10:15:00Z"]
|
||||
.map(|value| value.parse::<DateTime<Utc>>().unwrap());
|
||||
assert_eq!(
|
||||
pending.iter().map(|row| row.0).collect::<Vec<_>>(),
|
||||
expected_buckets
|
||||
);
|
||||
assert!(pending[0].1 > pending_before.0);
|
||||
assert_eq!(pending[0].2, pending_before.1);
|
||||
|
||||
// 重启不会再次入队,也不会覆盖最早待处理时间
|
||||
TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
let pending_after_restart: Vec<(DateTime<Utc>, i64, DateTime<Utc>)> = sqlx::query_as(
|
||||
"select bucket_start,change_version,oldest_pending_at
|
||||
from request_analytics_pending_buckets order by bucket_start",
|
||||
)
|
||||
.fetch_all(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(pending_after_restart, pending);
|
||||
database.close().await;
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
//! PostgreSQL 连接错误、迁移应用与事务归属测试
|
||||
//! PostgreSQL 连接初始化、会话设置与事务取消测试
|
||||
|
||||
use std::error::Error as _;
|
||||
|
||||
@@ -109,32 +109,32 @@ async fn invalid_postgres_options_preserve_the_sqlx_source() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn connect_and_migrate_should_apply_all_migrations_once_and_reopen_cleanly() {
|
||||
async fn connection_initialization_preserves_settings_when_reopened() {
|
||||
let Some(database_url) = crate::support::test_env("CPR_TEST_DATABASE_URL") else {
|
||||
return;
|
||||
};
|
||||
let database = format!("cpr_store_migrator_{}", Uuid::new_v4().simple());
|
||||
let database = format!("cpr_store_connection_{}", Uuid::new_v4().simple());
|
||||
let admin = PgPoolOptions::new()
|
||||
.max_connections(1)
|
||||
.connect(&database_url)
|
||||
.await
|
||||
.expect("connect migration test PostgreSQL");
|
||||
.expect("connect test PostgreSQL");
|
||||
sqlx::raw_sql(sqlx::AssertSqlSafe(format!(
|
||||
"create database \"{database}\""
|
||||
)))
|
||||
.execute(&admin)
|
||||
.await
|
||||
.expect("create migration test database");
|
||||
.expect("create connection test database");
|
||||
|
||||
let isolated_url = PgConnectOptions::from_str(&database_url)
|
||||
.expect("parse migration test PostgreSQL URL")
|
||||
.expect("parse test PostgreSQL URL")
|
||||
.database(&database)
|
||||
.to_url_lossy()
|
||||
.to_string();
|
||||
let pool_config = gateway_store::StorePoolConfig::default();
|
||||
let first = connect_and_migrate(&isolated_url, pool_config)
|
||||
.await
|
||||
.expect("apply migrations through production migrator");
|
||||
.expect("initialize production connection");
|
||||
let session_settings = sqlx::query_as::<_, (String, i64, i64, i64)>(
|
||||
"select current_setting('application_name'),
|
||||
extract(epoch from current_setting('statement_timeout')::interval)::bigint,
|
||||
@@ -154,70 +154,20 @@ async fn connect_and_migrate_should_apply_all_migrations_once_and_reopen_cleanly
|
||||
timezone, "UTC",
|
||||
"runtime sessions use an explicit technical time basis"
|
||||
);
|
||||
let first_tables = sqlx::query_scalar::<_, String>(
|
||||
"select table_name
|
||||
from information_schema.tables
|
||||
where table_schema = 'public'
|
||||
order by table_name",
|
||||
)
|
||||
.fetch_all(&first)
|
||||
.await
|
||||
.expect("load migrated tables");
|
||||
sqlx::query("update runtime_settings set refresh_margin_seconds = 1800 where id = 1")
|
||||
.execute(&first)
|
||||
.await
|
||||
.expect("save custom runtime setting");
|
||||
first.close().await;
|
||||
|
||||
let second = connect_and_migrate(&isolated_url, pool_config)
|
||||
.await
|
||||
.expect("reopen database through production migrator");
|
||||
let migration_count =
|
||||
sqlx::query_scalar::<_, i64>("select count(*) from _sqlx_migrations where success")
|
||||
.expect("reopen production connection");
|
||||
let refresh_margin: i64 =
|
||||
sqlx::query_scalar("select refresh_margin_seconds from runtime_settings where id = 1")
|
||||
.fetch_one(&second)
|
||||
.await
|
||||
.expect("count successful migrations");
|
||||
let response_id_types = sqlx::query_scalar::<_, String>(
|
||||
"select data_type
|
||||
from information_schema.columns
|
||||
where table_schema = 'public'
|
||||
and table_name = 'model_requests'
|
||||
and column_name in ('client_response_id', 'upstream_response_id')
|
||||
order by column_name",
|
||||
)
|
||||
.fetch_all(&second)
|
||||
.await
|
||||
.expect("load opaque response ID column types");
|
||||
let raw_response_id_index_exists = sqlx::query_scalar::<_, bool>(
|
||||
"select exists (
|
||||
select 1
|
||||
from pg_indexes
|
||||
where schemaname = 'public'
|
||||
and indexname = 'model_requests_client_response_uq'
|
||||
)",
|
||||
)
|
||||
.fetch_one(&second)
|
||||
.await
|
||||
.expect("check removed raw response ID index");
|
||||
let legacy_key_provider_column_exists = sqlx::query_scalar::<_, bool>(
|
||||
"select exists (
|
||||
select 1 from information_schema.columns
|
||||
where table_schema = 'public'
|
||||
and table_name = 'client_api_keys'
|
||||
and column_name = 'provider_kind'
|
||||
)",
|
||||
)
|
||||
.fetch_one(&second)
|
||||
.await
|
||||
.expect("check removed client key provider column");
|
||||
let routing_history_columns = sqlx::query_scalar::<_, String>(
|
||||
"select column_name from information_schema.columns
|
||||
where table_schema = 'public'
|
||||
and table_name = 'model_requests'
|
||||
and column_name in (
|
||||
'routing_scope', 'routing_group_refs', 'routing_group_names_snapshot'
|
||||
)
|
||||
order by column_name",
|
||||
)
|
||||
.fetch_all(&second)
|
||||
.await
|
||||
.expect("load routing history columns");
|
||||
.expect("load saved runtime setting");
|
||||
second.close().await;
|
||||
|
||||
sqlx::raw_sql(sqlx::AssertSqlSafe(format!(
|
||||
@@ -225,72 +175,9 @@ async fn connect_and_migrate_should_apply_all_migrations_once_and_reopen_cleanly
|
||||
)))
|
||||
.execute(&admin)
|
||||
.await
|
||||
.expect("drop migration test database");
|
||||
.expect("drop connection test database");
|
||||
admin.close().await;
|
||||
|
||||
assert_eq!(
|
||||
first_tables,
|
||||
[
|
||||
"_sqlx_migrations",
|
||||
"account_group_accounts",
|
||||
"account_groups",
|
||||
"admin_audit_events",
|
||||
"admin_users",
|
||||
"backup_records",
|
||||
"backup_settings",
|
||||
"client_api_key_budget_windows",
|
||||
"client_api_key_charge_events",
|
||||
"client_api_key_groups",
|
||||
"client_api_keys",
|
||||
"model_request_leases",
|
||||
"model_request_observations",
|
||||
"model_requests",
|
||||
"ops_events",
|
||||
"outbound_proxies",
|
||||
"plugin_account_group_resources",
|
||||
"plugin_artifact_credentials",
|
||||
"plugin_artifact_platforms",
|
||||
"plugin_artifacts",
|
||||
"plugin_client_api_key_resources",
|
||||
"plugin_instance_secrets",
|
||||
"plugin_instances",
|
||||
"plugin_source_credentials",
|
||||
"plugin_state_generations",
|
||||
"plugin_state_records",
|
||||
"plugin_update_sources",
|
||||
"plugin_version_configurations",
|
||||
"provider_accounts",
|
||||
"provider_authorization_receipts",
|
||||
"request_analytics_15m",
|
||||
"request_analytics_1d_counts",
|
||||
"request_analytics_1h",
|
||||
"request_analytics_build_state",
|
||||
"request_analytics_facts",
|
||||
"request_analytics_pending_buckets",
|
||||
"request_analytics_pending_requests",
|
||||
"runtime_settings",
|
||||
]
|
||||
);
|
||||
assert_eq!(session_settings, ("codex-proxy-rs".to_owned(), 30, 5, 30));
|
||||
assert_eq!(
|
||||
migration_count,
|
||||
i64::try_from(TEST_MIGRATOR.iter().count())
|
||||
.expect("migration count fits PostgreSQL bigint")
|
||||
);
|
||||
assert_eq!(response_id_types, ["bytea", "bytea"]);
|
||||
assert!(!raw_response_id_index_exists);
|
||||
assert!(!legacy_key_provider_column_exists);
|
||||
assert!(routing_history_columns.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn migrations_should_leave_transaction_ownership_to_sqlx() {
|
||||
let transaction_statements = TEST_MIGRATOR
|
||||
.iter()
|
||||
.flat_map(|migration| migration.sql.as_str().lines())
|
||||
.map(str::trim)
|
||||
.filter(|line| matches!(*line, "begin;" | "commit;"))
|
||||
.count();
|
||||
|
||||
assert_eq!(transaction_statements, 0);
|
||||
assert_eq!(refresh_margin, 1800, "reopening preserves custom settings");
|
||||
}
|
||||
|
||||
@@ -2168,10 +2168,6 @@ async fn shared_core_usage_and_timings_preserve_fields_and_reject_invalid_phases
|
||||
upstream_response_ms: Some(1_000),
|
||||
upstream_api_overhead_ms: Some(120.25),
|
||||
upstream_engine_ms: Some(6400.0),
|
||||
upstream_engine_iapi_ttft_ms: Some(650.5),
|
||||
upstream_engine_service_ttft_ms: Some(720.25),
|
||||
upstream_engine_iapi_tbt_ms: Some(18.45),
|
||||
upstream_engine_service_tbt_ms: Some(20.12),
|
||||
latency_ms: Some(9),
|
||||
};
|
||||
ExecutionStore::finalize_model_request(&store, finalization)
|
||||
@@ -2202,22 +2198,6 @@ async fn shared_core_usage_and_timings_preserve_fields_and_reject_invalid_phases
|
||||
}
|
||||
assert_eq!(persisted["upstream_api_overhead_ms"].as_f64(), Some(120.25));
|
||||
assert_eq!(persisted["upstream_engine_ms"].as_f64(), Some(6400.0));
|
||||
assert_eq!(
|
||||
persisted["upstream_engine_iapi_ttft_ms"].as_f64(),
|
||||
Some(650.5)
|
||||
);
|
||||
assert_eq!(
|
||||
persisted["upstream_engine_service_ttft_ms"].as_f64(),
|
||||
Some(720.25)
|
||||
);
|
||||
assert_eq!(
|
||||
persisted["upstream_engine_iapi_tbt_ms"].as_f64(),
|
||||
Some(18.45)
|
||||
);
|
||||
assert_eq!(
|
||||
persisted["upstream_engine_service_tbt_ms"].as_f64(),
|
||||
Some(20.12)
|
||||
);
|
||||
let invalid_id = "req_invalid_phase";
|
||||
seed_running_request(&database.pool, invalid_id)
|
||||
.await
|
||||
@@ -2233,7 +2213,7 @@ async fn shared_core_usage_and_timings_preserve_fields_and_reject_invalid_phases
|
||||
assert_eq!(stored_row(&database.pool, invalid_id).await, before);
|
||||
for invalid_ms in [-1.0, f64::NAN, f64::INFINITY, i64::MAX as f64] {
|
||||
let mut invalid = successful_core_finalization(invalid_id);
|
||||
invalid.timings.upstream_engine_iapi_tbt_ms = Some(invalid_ms);
|
||||
invalid.timings.upstream_engine_ms = Some(invalid_ms);
|
||||
let error = ExecutionStore::finalize_model_request(&store, invalid)
|
||||
.await
|
||||
.expect_err("invalid official timing must not silently become missing");
|
||||
|
||||
@@ -18,7 +18,6 @@ mod admission_recovery;
|
||||
mod analytics;
|
||||
mod analytics_buckets;
|
||||
mod analytics_reset;
|
||||
mod analytics_upgrade;
|
||||
mod backup;
|
||||
mod client_budgets;
|
||||
mod client_keys;
|
||||
@@ -69,10 +68,6 @@ pub(super) fn admin_account_store(pool: &PgPool) -> PgAdminAccountStore {
|
||||
|
||||
impl TestDatabase {
|
||||
pub(super) async fn create(label: &str) -> Option<Self> {
|
||||
Self::create_through(label, i64::MAX).await
|
||||
}
|
||||
|
||||
pub(super) async fn create_through(label: &str, migration_version: i64) -> Option<Self> {
|
||||
let database_url = crate::support::test_env("CPR_TEST_DATABASE_URL")?;
|
||||
let schema = format!("cpr_store_{label}_{}", Uuid::new_v4().simple());
|
||||
// 临时 schema 验证事务可见性与回滚,不模拟 PostgreSQL 掉电恢复
|
||||
@@ -95,7 +90,7 @@ impl TestDatabase {
|
||||
.await
|
||||
.expect("connect isolated test schema");
|
||||
TEST_MIGRATOR
|
||||
.run_to(migration_version, &pool)
|
||||
.run(&pool)
|
||||
.await
|
||||
.expect("apply test migrations");
|
||||
Some(Self {
|
||||
|
||||
@@ -110,7 +110,7 @@ async fn output_throughput_uses_official_response_duration_without_local_fallbac
|
||||
'firstTokenMs', $2::bigint,
|
||||
'latencyMs', $3::bigint),
|
||||
'upstream', coalesce(request_observation_json #> '{timings,upstream}', '{}'::jsonb) || jsonb_build_object(
|
||||
'responseMs', $4::bigint, 'engineIapiTbtMs', 2.450638))))
|
||||
'responseMs', $4::bigint, 'engineMs', 2.450638))))
|
||||
where id = 'req_observe_success'",
|
||||
)
|
||||
.bind(output)
|
||||
@@ -150,8 +150,6 @@ async fn output_throughput_uses_official_response_duration_without_local_fallbac
|
||||
detail.request.upstream_response_ms,
|
||||
record.upstream_response_ms
|
||||
);
|
||||
assert_eq!(record.upstream_engine_iapi_tbt_ms, Some(2.450638));
|
||||
assert_eq!(detail.request.upstream_engine_iapi_tbt_ms, Some(2.450638));
|
||||
|
||||
super::drain_analytics(&database.pool).await;
|
||||
let summary = repository
|
||||
|
||||
@@ -1,58 +1,10 @@
|
||||
//! 观测视图升级兼容性与普通聚合的覆盖索引规划回归测试
|
||||
//! 验证观测视图普通聚合能够使用覆盖索引
|
||||
|
||||
use chrono::Utc;
|
||||
use serde_json::Value;
|
||||
|
||||
use super::{TestDatabase, seed_observability_facts};
|
||||
|
||||
#[tokio::test]
|
||||
async fn projection_upgrade_preserves_observation_values() {
|
||||
let Some(db) = TestDatabase::create_through("observation_projection_upgrade", 26).await else {
|
||||
return;
|
||||
};
|
||||
// 此测试固定覆盖 26 → 27 的冻结视图合同,不依赖最新版本的写入夹具
|
||||
sqlx::query("insert into model_requests(id,client_api_key_ref,operation,client_transport,outcome,client_status_code,started_at,deadline_at,completed_at,request_observation_json) values('req_observe_success','historical_key','responses','http_sse','succeeded',200,now()-interval '1 hour',now(),now(),'{\"request\":{\"configRevision\":1,\"protocol\":\"openai\",\"endpoint\":\"/v1/responses\",\"compact\":false},\"routing\":{\"scope\":\"all\",\"groupRefs\":[],\"groupNamesSnapshot\":[]},\"timings\":{\"local\":{\"latencyMs\":800}}}'::jsonb)")
|
||||
.execute(&db.pool).await.unwrap();
|
||||
sqlx::query(
|
||||
"update model_requests
|
||||
set request_observation_json = jsonb_set(request_observation_json,
|
||||
'{timings,upstream}', '{\"responseMs\":700,\"engineIapiTbtMs\":2.450638}')
|
||||
where id = 'req_observe_success'",
|
||||
)
|
||||
.execute(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
let before: Vec<Value> =
|
||||
sqlx::query_scalar("select to_jsonb(mr) from model_request_observations mr order by id")
|
||||
.fetch_all(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
super::super::TEST_MIGRATOR
|
||||
.run_to(27, &db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
let after: Vec<Value> =
|
||||
sqlx::query_scalar("select to_jsonb(mr) from model_request_observations mr order by id")
|
||||
.fetch_all(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
after, before,
|
||||
"projection upgrade must preserve every field"
|
||||
);
|
||||
let updatable: String = sqlx::query_scalar(
|
||||
"select is_updatable from information_schema.views
|
||||
where table_schema = current_schema() and table_name = 'model_request_observations'",
|
||||
)
|
||||
.fetch_one(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(updatable, "NO", "observation projection remains read-only");
|
||||
db.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn plain_observation_aggregates_can_use_a_covering_index() {
|
||||
let Some(db) = TestDatabase::create("observation_covering_index").await else {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
//! 验证插件制品身份、信任接受、迁移与安装幂等性
|
||||
//! 验证插件制品身份、信任接受与安装幂等性
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
@@ -83,12 +83,7 @@ async fn unknown_metadata_fields_do_not_change_artifact_identity() {
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(stored.get("permissionDescriptions").is_none());
|
||||
assert!(stored.get("requestedPermissions").is_none());
|
||||
let mut extended = stored;
|
||||
extended["permissionDescriptions"] =
|
||||
serde_json::json!([{"permission": "models", "label": "旧文案"}]);
|
||||
extended["requestedPermissions"] = serde_json::json!(["models"]);
|
||||
extended["futureMetadata"] = serde_json::json!({"enabled": true});
|
||||
extended["contributes"]["middleware"]["futureField"] = serde_json::json!(true);
|
||||
sqlx::query("update plugin_artifacts set metadata_json=$2 where sha256=$1")
|
||||
@@ -136,94 +131,6 @@ async fn unknown_metadata_fields_do_not_change_artifact_identity() {
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn trust_migration_removes_only_obsolete_metadata_fields() {
|
||||
let Some(database) = TestDatabase::create_through("plugin_trust_upgrade", 19).await else {
|
||||
return;
|
||||
};
|
||||
initialize_revision(&database).await;
|
||||
let store = PgPluginStore::new(database.pool.clone());
|
||||
for (digest, extra) in [
|
||||
('a', serde_json::json!({"requestedPermissions": ["models"]})),
|
||||
(
|
||||
'b',
|
||||
serde_json::json!({"permissionDescriptions": ["旧文案"]}),
|
||||
),
|
||||
(
|
||||
'c',
|
||||
serde_json::json!({"requestedPermissions": [], "permissionDescriptions": []}),
|
||||
),
|
||||
('d', serde_json::json!({})),
|
||||
] {
|
||||
// 每个制品使用独立平台,保留同一版本的平台不可变约束
|
||||
let fixture = artifact(digest, &[&format!("test-{digest}")]);
|
||||
let installed = store
|
||||
.install_artifact(fixture.clone(), PluginSource::Upload, &context())
|
||||
.await
|
||||
.unwrap();
|
||||
if digest == 'a' {
|
||||
store
|
||||
.accept_artifact(&fixture.metadata.sha256, &context())
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
sqlx::query("update plugin_artifacts set metadata_json=metadata_json || $2 || '{\"futureMetadata\":true}'::jsonb where sha256=$1")
|
||||
.bind(&fixture.metadata.sha256).bind(extra).execute(&database.pool).await.unwrap();
|
||||
sqlx::query("insert into plugin_instances(id,artifact_sha256,name,enabled,configuration_json,bindings_json,revision) values ($1,$2,'Example',false,'{}','[]',$3)")
|
||||
.bind(uuid::Uuid::now_v7()).bind(&fixture.metadata.sha256)
|
||||
.bind(i64::try_from(installed.config_revision.get()).unwrap())
|
||||
.execute(&database.pool).await.unwrap();
|
||||
}
|
||||
let mut expected: Vec<serde_json::Value> =
|
||||
sqlx::query_scalar("select to_jsonb(a) from plugin_artifacts a order by sha256")
|
||||
.fetch_all(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
for row in &mut expected {
|
||||
let metadata = row["metadata_json"].as_object_mut().unwrap();
|
||||
metadata.remove("requestedPermissions");
|
||||
metadata.remove("permissionDescriptions");
|
||||
}
|
||||
let instances: Vec<serde_json::Value> =
|
||||
sqlx::query_scalar("select to_jsonb(i) from plugin_instances i order by id")
|
||||
.fetch_all(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
let revision: i64 =
|
||||
sqlx::query_scalar("select config_revision from runtime_settings where id=1")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
for _ in 0..2 {
|
||||
super::super::TEST_MIGRATOR
|
||||
.run(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
let actual: Vec<serde_json::Value> =
|
||||
sqlx::query_scalar("select to_jsonb(a) from plugin_artifacts a order by sha256")
|
||||
.fetch_all(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
actual, expected,
|
||||
"只清理两个旧字段,保留包体、接受事实与其他元数据"
|
||||
);
|
||||
let current_instances: Vec<serde_json::Value> =
|
||||
sqlx::query_scalar("select to_jsonb(i) from plugin_instances i order by id")
|
||||
.fetch_all(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(current_instances, instances);
|
||||
let snapshot = store.load_instances().await.unwrap();
|
||||
assert_eq!(snapshot.instances.len(), 4);
|
||||
assert_eq!(
|
||||
snapshot.config_revision.get(),
|
||||
u64::try_from(revision).unwrap()
|
||||
);
|
||||
}
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn artifact_acceptance_is_exact_and_idempotent() {
|
||||
let Some(database) = TestDatabase::create("plugin_artifact_acceptance").await else {
|
||||
@@ -262,51 +169,6 @@ async fn artifact_acceptance_is_exact_and_idempotent() {
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn plugin_migration_preserves_main_settings_and_accepts_new_installations() {
|
||||
let Some(database) = TestDatabase::create_through("plugin_upgrade_from_main", 17).await else {
|
||||
return;
|
||||
};
|
||||
initialize_revision(&database).await;
|
||||
// 0022 会把仍为旧默认 3600 的 margin 收敛到 300;预设自定义值,
|
||||
// 让本测试继续验证「所有已自定义设置都原样保留」。
|
||||
sqlx::query("update runtime_settings set max_concurrent_per_account=0, refresh_margin_seconds=1800 where id=1")
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
let before: serde_json::Value =
|
||||
sqlx::query_scalar("select to_jsonb(s) from runtime_settings s where id=1")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
super::super::TEST_MIGRATOR
|
||||
.run(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
let after: serde_json::Value =
|
||||
sqlx::query_scalar("select to_jsonb(s) from runtime_settings s where id=1")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
for (key, value) in before.as_object().unwrap() {
|
||||
if key == "request_interval_ms" {
|
||||
assert!(after.get(key).is_none(), "removed setting {key}");
|
||||
continue;
|
||||
}
|
||||
assert_eq!(after.get(key), Some(value), "existing setting {key}");
|
||||
}
|
||||
let store = PgPluginStore::new(database.pool.clone());
|
||||
store
|
||||
.install_artifact(
|
||||
artifact('a', &["linux-x86_64"]),
|
||||
PluginSource::Upload,
|
||||
&context(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn source_identity_requires_publisher_and_name() {
|
||||
let Some(database) = TestDatabase::create("plugin_source_identity").await else {
|
||||
|
||||
@@ -1193,63 +1193,6 @@ async fn legacy_urls_join_one_catalog_entry_and_invalid_batch_rolls_back() {
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn migration_backfills_shared_proxies_without_changing_credentials() {
|
||||
let Some(database) = TestDatabase::create_through("proxy_backfill", 4).await else {
|
||||
return;
|
||||
};
|
||||
// 从真实旧 schema 正向升级,不拆卸最新结构,避免遗漏后续增加的引用约束
|
||||
for id in ["acct_one", "acct_two", "acct_direct"] {
|
||||
let proxy_url = (id != "acct_direct").then_some("http://user:secret@127.0.0.1:8080/");
|
||||
sqlx::query(
|
||||
"insert into provider_accounts (
|
||||
id,provider_kind,name,authentication_kind,provider_credentials_json,
|
||||
has_refresh_token,credential_observed_at,created_at,updated_at,outbound_proxy_url
|
||||
) values ($1,'openai',$1,'oauth','{}',false,now(),now(),now(),$2)",
|
||||
)
|
||||
.bind(id)
|
||||
.bind(proxy_url)
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
assert!(
|
||||
sqlx::query_scalar::<_, bool>("select bool_and(enabled) from outbound_proxies")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap()
|
||||
);
|
||||
let accounts = PgProviderAccountRepository::new(database.pool.clone());
|
||||
let inherited = gateway_core::account::ProviderAccountStore::list_accounts(&accounts)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(inherited.len(), 3);
|
||||
assert!(
|
||||
inherited
|
||||
.iter()
|
||||
.all(|account| account.request_location().is_none())
|
||||
);
|
||||
assert_eq!(
|
||||
sqlx::query_scalar::<_, i64>("select count(*) from outbound_proxies")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap(),
|
||||
1
|
||||
);
|
||||
assert_eq!(sqlx::query_scalar::<_, i64>("select count(*) from provider_accounts a join outbound_proxies p on a.outbound_proxy_id = p.id where a.outbound_proxy_url = p.proxy_url and a.credential_revision = 1").fetch_one(&database.pool).await.unwrap(), 2);
|
||||
assert!(
|
||||
sqlx::query_scalar::<_, Option<String>>(
|
||||
"select outbound_proxy_id from provider_accounts where id = 'acct_direct'"
|
||||
)
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap()
|
||||
.is_none()
|
||||
);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn automatic_proxy_location_projects_to_accounts_and_guards_refresh_results() {
|
||||
use gateway_core::account::{ProviderAccountStore, RequestLocation};
|
||||
|
||||
@@ -1,14 +1,18 @@
|
||||
//! 验证短保留期升级、分批清理边界与统计异步收敛期间的查询可用性
|
||||
|
||||
use std::num::NonZeroU32;
|
||||
use std::{num::NonZeroU32, time::Duration};
|
||||
|
||||
use chrono::{Duration as ChronoDuration, Utc};
|
||||
use gateway_admin::{
|
||||
model::retention::{RetentionPolicy, RetentionTarget},
|
||||
ports::retention::RetentionStore as _,
|
||||
ports::{retention::RetentionStore as _, store::ObservabilityStore as _},
|
||||
};
|
||||
use gateway_store::postgres::{
|
||||
ObservabilityRange, PgRequestAnalyticsRepository, PgRetentionRepository,
|
||||
use gateway_store::{
|
||||
RequestAnalyticsCycle,
|
||||
postgres::{
|
||||
ObservabilityQueryBudget, ObservabilityRange, PgRequestAnalyticsRepository,
|
||||
PgRetentionRepository,
|
||||
},
|
||||
};
|
||||
|
||||
use super::{
|
||||
@@ -18,10 +22,7 @@ use super::{
|
||||
|
||||
#[tokio::test]
|
||||
async fn short_retention_keeps_statistics_readable_until_each_deletion_batch_is_projected() {
|
||||
use gateway_admin::{
|
||||
model::observability::{AnalyticsState, DashboardQuery, TimeRange},
|
||||
ports::store::ObservabilityStore as _,
|
||||
};
|
||||
use gateway_admin::model::observability::{AnalyticsState, DashboardQuery, TimeRange};
|
||||
|
||||
let Some(database) = TestDatabase::create("short_retention_readable").await else {
|
||||
return;
|
||||
@@ -280,3 +281,271 @@ async fn empty_request_cleanup_does_not_conflict_with_analytics_publication() {
|
||||
publication.commit().await.unwrap();
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn scattered_retention_deletions_commit_atomically_and_allow_new_requests_to_advance() {
|
||||
let Some(database) = TestDatabase::create("retention_scattered_groups").await else {
|
||||
return;
|
||||
};
|
||||
let now = Utc::now();
|
||||
for number in 0..600 {
|
||||
seed(
|
||||
&database.pool,
|
||||
&format!("expired_{number:04}"),
|
||||
now - ChronoDuration::days(30) + ChronoDuration::minutes(number * 20),
|
||||
&[],
|
||||
)
|
||||
.await;
|
||||
}
|
||||
drain_analytics(&database.pool).await;
|
||||
let worker = PgRequestAnalyticsRepository::new(
|
||||
database.pool.clone(),
|
||||
ObservabilityQueryBudget::try_new(2, Duration::from_secs(5), Duration::from_secs(120))
|
||||
.unwrap(),
|
||||
);
|
||||
let before = worker.status().await.unwrap();
|
||||
let retention = PgRetentionRepository::new(database.pool.clone());
|
||||
assert_eq!(
|
||||
retention
|
||||
.purge_batch(
|
||||
RetentionTarget::ModelRequests,
|
||||
now,
|
||||
RetentionPolicy::try_new(10, 30, 90).unwrap(),
|
||||
NonZeroU32::new(1000).unwrap(),
|
||||
)
|
||||
.await
|
||||
.unwrap(),
|
||||
600
|
||||
);
|
||||
seed(
|
||||
&database.pool,
|
||||
"new_request",
|
||||
now - ChronoDuration::minutes(2),
|
||||
&[],
|
||||
)
|
||||
.await;
|
||||
|
||||
// 小时层失败时,已经执行的多个刻钟删除批次与事实删除也必须一起回滚
|
||||
sqlx::raw_sql("create function reject_hour_deletion() returns trigger language plpgsql as $$begin raise exception 'test hour deletion failure'; end$$;
|
||||
create trigger reject_hour_deletion before delete on request_analytics_1h for each statement execute function reject_hour_deletion()")
|
||||
.execute(&database.pool).await.unwrap();
|
||||
let cycle = RequestAnalyticsCycle::default();
|
||||
assert!(cycle.run(&worker).await.is_err());
|
||||
let failed = worker.status().await.unwrap();
|
||||
assert_eq!(failed.generation, before.generation);
|
||||
assert_eq!(failed.pending_requests, 601);
|
||||
let statistics = observability_repository(&database.pool);
|
||||
let range = ObservabilityRange::new(now - ChronoDuration::days(31), now).unwrap();
|
||||
assert_eq!(
|
||||
statistics
|
||||
.usage_summary(range, Default::default())
|
||||
.await
|
||||
.unwrap()
|
||||
.requests
|
||||
.request_count,
|
||||
600
|
||||
);
|
||||
sqlx::raw_sql("drop trigger reject_hour_deletion on request_analytics_1h; drop function reject_hour_deletion();
|
||||
create function slow_summary_statement() returns trigger language plpgsql as $$begin perform pg_sleep(0.002); return null; end$$;
|
||||
create trigger slow_quarter_delete before delete on request_analytics_15m for each statement execute function slow_summary_statement();
|
||||
create trigger slow_hour_delete before delete on request_analytics_1h for each statement execute function slow_summary_statement()")
|
||||
.execute(&database.pool).await.unwrap();
|
||||
|
||||
// 按语句注入延迟,覆盖大量分组的 SQL 往返成本及多个集合删除批次
|
||||
cycle
|
||||
.run(&worker)
|
||||
.await
|
||||
.expect("retention backlog must advance within the cycle budget");
|
||||
let after = worker.status().await.unwrap();
|
||||
assert_eq!(after.pending_requests, 0);
|
||||
assert_eq!(after.pending_buckets, 0);
|
||||
assert!(after.generation > before.generation);
|
||||
let actual = statistics
|
||||
.usage_summary(range, Default::default())
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(actual.requests.request_count, 1);
|
||||
assert_eq!(actual.requests.total_tokens, 120);
|
||||
let timeline = admin_observability_store(&database.pool)
|
||||
.dashboard_summary(
|
||||
gateway_admin::model::observability::DashboardQuery::new(
|
||||
gateway_admin::model::observability::TimeRange {
|
||||
start: now - ChronoDuration::days(1),
|
||||
end: now,
|
||||
},
|
||||
Default::default(),
|
||||
)
|
||||
.unwrap(),
|
||||
now,
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
timeline
|
||||
.trend
|
||||
.iter()
|
||||
.map(|point| point.metrics.request_count)
|
||||
.sum::<u64>(),
|
||||
1
|
||||
);
|
||||
let remaining: (i64, i64, i64, i64) = sqlx::query_as(
|
||||
"select
|
||||
(select count(*) from request_analytics_facts where id like 'expired_%'),
|
||||
(select count(*) from request_analytics_15m where bucket_start < $1),
|
||||
(select count(*) from request_analytics_1h where bucket_start < $1),
|
||||
(select count(*) from request_analytics_1d_counts where bucket_start < $1)",
|
||||
)
|
||||
.bind(now - ChronoDuration::days(10))
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(remaining, (0, 0, 0, 0));
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn fresh_requests_and_retention_deletions_both_advance_within_each_batch() {
|
||||
let Some(database) = TestDatabase::create("retention_fair_projection").await else {
|
||||
return;
|
||||
};
|
||||
let now = Utc::now();
|
||||
for number in 0..20 {
|
||||
seed(
|
||||
&database.pool,
|
||||
&format!("expired_{number:04}"),
|
||||
now - ChronoDuration::days(30),
|
||||
&[],
|
||||
)
|
||||
.await;
|
||||
}
|
||||
drain_analytics(&database.pool).await;
|
||||
PgRetentionRepository::new(database.pool.clone())
|
||||
.purge_batch(
|
||||
RetentionTarget::ModelRequests,
|
||||
now,
|
||||
RetentionPolicy::try_new(10, 30, 90).unwrap(),
|
||||
NonZeroU32::new(1000).unwrap(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let worker =
|
||||
PgRequestAnalyticsRepository::new(database.pool.clone(), observability_query_budget());
|
||||
for wave in 1..=3 {
|
||||
for number in 0..2 {
|
||||
seed(
|
||||
&database.pool,
|
||||
&format!("new_{wave}_{number}"),
|
||||
now - ChronoDuration::minutes(2),
|
||||
&[],
|
||||
)
|
||||
.await;
|
||||
}
|
||||
assert_eq!(worker.maintain(4).await.unwrap().projected_requests, 4);
|
||||
let facts: (i64, i64) = sqlx::query_as(
|
||||
"select count(*) filter (where id like 'expired_%'), count(*) filter (where id like 'new_%') from request_analytics_facts",
|
||||
)
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
// 历史队列尚未排空时,每批都须撤销旧贡献并发布本轮的新请求
|
||||
assert_eq!(facts, (20 - wave * 2, wave * 2));
|
||||
let current = observability_repository(&database.pool)
|
||||
.usage_summary(
|
||||
ObservabilityRange::new(now - ChronoDuration::days(1), now).unwrap(),
|
||||
Default::default(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(current.requests.request_count, wave as u64 * 2);
|
||||
}
|
||||
drain_analytics(&database.pool).await;
|
||||
let final_summary = observability_repository(&database.pool)
|
||||
.usage_summary(
|
||||
ObservabilityRange::new(now - ChronoDuration::days(31), now).unwrap(),
|
||||
Default::default(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(final_summary.requests.request_count, 6);
|
||||
assert_eq!(final_summary.requests.total_tokens, 720);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn timed_out_retention_batches_shrink_across_cycles_and_lock_contention() {
|
||||
let Some(database) = TestDatabase::create("retention_batch_timeout").await else {
|
||||
return;
|
||||
};
|
||||
let now = Utc::now();
|
||||
for number in 0..600 {
|
||||
seed(
|
||||
&database.pool,
|
||||
&format!("expired_{number:04}"),
|
||||
now - ChronoDuration::days(30),
|
||||
&[],
|
||||
)
|
||||
.await;
|
||||
}
|
||||
drain_analytics(&database.pool).await;
|
||||
sqlx::query("delete from model_requests")
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
seed(
|
||||
&database.pool,
|
||||
"new_request",
|
||||
now - ChronoDuration::minutes(2),
|
||||
&[],
|
||||
)
|
||||
.await;
|
||||
// 数据库延迟与实际批量相关,小批量无需等待,以可观察进展验证跨轮退让
|
||||
sqlx::raw_sql("create function slow_large_fact_deletion() returns trigger language plpgsql as $$begin
|
||||
if (select count(*) from removed_facts) > 250 then perform pg_sleep(2); end if;
|
||||
return null; end$$;
|
||||
create trigger slow_large_fact_deletion after delete on request_analytics_facts referencing old table as removed_facts for each statement execute function slow_large_fact_deletion()")
|
||||
.execute(&database.pool).await.unwrap();
|
||||
let worker = PgRequestAnalyticsRepository::new(
|
||||
database.pool.clone(),
|
||||
ObservabilityQueryBudget::try_new(2, Duration::from_secs(1), Duration::from_secs(1))
|
||||
.unwrap(),
|
||||
);
|
||||
let cycle = RequestAnalyticsCycle::default();
|
||||
for _ in 0..2 {
|
||||
assert!(cycle.run(&worker).await.is_err());
|
||||
// 空轮可能来自别的实例持锁,不能因此恢复会反复超时的大批量
|
||||
let mut lock = database.pool.begin().await.unwrap();
|
||||
sqlx::query(
|
||||
"select pg_advisory_xact_lock(hashtextextended(current_schema()::text,382032))",
|
||||
)
|
||||
.execute(&mut *lock)
|
||||
.await
|
||||
.unwrap();
|
||||
let idle = cycle.run(&worker).await.unwrap();
|
||||
assert_eq!(idle.projected_requests, 0);
|
||||
lock.rollback().await.unwrap();
|
||||
}
|
||||
let pending = worker.status().await.unwrap().pending_requests;
|
||||
assert_eq!(pending, 601);
|
||||
// 此轮可以在后续批次超时前已有提交,下一轮重放仍须保持计数正确
|
||||
let _ = cycle.run(&worker).await;
|
||||
assert!(worker.status().await.unwrap().pending_requests < pending);
|
||||
for _ in 0..5 {
|
||||
// 与调度器错误重试一样留出退避时间,让已取消语句完成回滚并释放维护锁
|
||||
tokio::time::sleep(Duration::from_secs(1)).await;
|
||||
let _ = cycle.run(&worker).await;
|
||||
if worker.status().await.unwrap().pending_requests == 0 {
|
||||
break;
|
||||
}
|
||||
}
|
||||
assert_eq!(worker.status().await.unwrap().pending_requests, 0);
|
||||
let actual = observability_repository(&database.pool)
|
||||
.usage_summary(
|
||||
ObservabilityRange::new(now - ChronoDuration::days(31), now).unwrap(),
|
||||
Default::default(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(actual.requests.request_count, 1);
|
||||
assert_eq!(actual.requests.total_tokens, 120);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
//! 验证运行设置的数据库约束、升级保留与快照发布
|
||||
//! 验证运行设置的读写、数据库约束与快照发布
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
@@ -61,56 +61,12 @@ fn runtime_settings_keep_account_rotation_global() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn removing_request_interval_preserves_settings_and_snapshot() {
|
||||
use gateway_store::postgres::{PgRuntimeSnapshotRepository, RuntimeSnapshotRepository};
|
||||
let Some(database) = TestDatabase::create_through("drop_interval", 29).await else {
|
||||
return;
|
||||
};
|
||||
sqlx::query("update runtime_settings set request_interval_ms = 60000, max_concurrent_per_account = 7, refresh_concurrency = 12, refresh_margin_seconds = 1234, config_revision = 70 where id = 1")
|
||||
.execute(&database.pool).await.unwrap();
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
let column_exists: bool = sqlx::query_scalar("select exists(select 1 from information_schema.columns where table_schema = current_schema() and table_name = 'runtime_settings' and column_name = 'request_interval_ms')")
|
||||
.fetch_one(&database.pool).await.unwrap();
|
||||
assert!(!column_exists);
|
||||
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
|
||||
let settings = repository.load_runtime_settings().await.unwrap();
|
||||
assert_eq!(settings.values.max_concurrent_per_account, 7);
|
||||
assert_eq!(settings.values.refresh_concurrency, 12);
|
||||
assert_eq!(settings.values.refresh_margin_seconds, 1234);
|
||||
let snapshot = PgRuntimeSnapshotRepository::new(database.pool.clone())
|
||||
.load_runtime_snapshot()
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(
|
||||
serde_json::to_value(snapshot.settings)
|
||||
.unwrap()
|
||||
.get("request_interval_ms")
|
||||
.is_none()
|
||||
);
|
||||
repository
|
||||
.update_runtime_settings(settings_with_margin(600))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
repository
|
||||
.load_runtime_settings()
|
||||
.await
|
||||
.unwrap()
|
||||
.values
|
||||
.refresh_margin_seconds,
|
||||
600
|
||||
);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn privacy_policy_upgrade_defaults_off_and_round_trips_in_snapshot() {
|
||||
async fn privacy_policy_defaults_off_and_round_trips_in_snapshot() {
|
||||
use gateway_core::settings::privacy::*;
|
||||
use gateway_store::postgres::{PgRuntimeSnapshotRepository, RuntimeSnapshotRepository};
|
||||
let Some(database) = TestDatabase::create_through("privacy_policy", 25).await else {
|
||||
let Some(database) = TestDatabase::create("privacy_policy").await else {
|
||||
return;
|
||||
};
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
|
||||
let before = repository.load_runtime_settings().await.unwrap();
|
||||
assert_eq!(
|
||||
@@ -151,167 +107,18 @@ async fn privacy_policy_upgrade_defaults_off_and_round_trips_in_snapshot() {
|
||||
serde_json::to_value(snapshot.settings).unwrap()["codex_privacy_policy"],
|
||||
serde_json::to_value(expected).unwrap()
|
||||
);
|
||||
assert_eq!(
|
||||
before.values.codex_privacy_policy,
|
||||
CodexPrivacyPolicy::default()
|
||||
);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn privacy_steps_migration_preserves_rules_and_other_settings() {
|
||||
let Some(database) = TestDatabase::create_through("privacy_steps", 30).await else {
|
||||
return;
|
||||
};
|
||||
let legacy = serde_json::json!({
|
||||
"enabled":true, "onError":"skip_rule", "rules":[
|
||||
{"id":"first","name":"旧规则一","enabled":true,"scope":"request_header","selector":"x-private",
|
||||
"action":"remove_field","pattern":null,"replacement":"","value":null,
|
||||
"replaceAll":true,"caseInsensitive":false,"multiLine":false},
|
||||
{"id":"second","name":"旧规则二","enabled":false,"scope":"request_body","selector":"$.owner",
|
||||
"action":"regex_replace","pattern":"alice","replacement":"user","value":null,
|
||||
"replaceAll":false,"caseInsensitive":true,"multiLine":true}
|
||||
]
|
||||
});
|
||||
let mut expected = serde_json::json!({
|
||||
"enabled":true, "onError":"skip_rule", "rules":[
|
||||
{"id":"first","name":"旧规则一","enabled":true,"steps":[
|
||||
{"scope":"request_header","selector":"x-private","action":"remove_field",
|
||||
"pattern":null,"replacement":"","value":null,
|
||||
"replaceAll":true,"caseInsensitive":false,"multiLine":false}]},
|
||||
{"id":"second","name":"旧规则二","enabled":false,"steps":[
|
||||
{"scope":"request_body","selector":"$.owner","action":"regex_replace",
|
||||
"pattern":"alice","replacement":"user","value":null,
|
||||
"replaceAll":false,"caseInsensitive":true,"multiLine":true}]}
|
||||
]
|
||||
});
|
||||
// 同时包含聚合规则,迁移不能嵌套包装或拆散现有步骤
|
||||
let canonical = serde_json::json!({
|
||||
"id":"aggregate","name":"通用聚合规则","enabled":true,"steps":[
|
||||
{"scope":"request_body","selector":"$.metadata","action":"set_value",
|
||||
"pattern":null,"replacement":"","value":{"owner":"user","items":[1,2]},
|
||||
"replaceAll":true,"caseInsensitive":false,"multiLine":false},
|
||||
{"scope":"request_header","selector":"x-private","action":"remove_field",
|
||||
"pattern":null,"replacement":"","value":null,
|
||||
"replaceAll":true,"caseInsensitive":false,"multiLine":false}
|
||||
]
|
||||
});
|
||||
let mut mixed = legacy;
|
||||
mixed["rules"]
|
||||
.as_array_mut()
|
||||
.unwrap()
|
||||
.insert(1, canonical.clone());
|
||||
expected["rules"]
|
||||
.as_array_mut()
|
||||
.unwrap()
|
||||
.insert(1, canonical);
|
||||
sqlx::query("update runtime_settings set codex_privacy_policy_json = $1, config_revision = 70, refresh_margin_seconds = 1234 where id = 1")
|
||||
.bind(&mixed).execute(&database.pool).await.unwrap();
|
||||
let mut expected_row: serde_json::Value =
|
||||
sqlx::query_scalar("select to_jsonb(runtime_settings) from runtime_settings where id = 1")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
expected_row["codex_privacy_policy_json"] = expected.clone();
|
||||
expected_row["reset_credit_lead_minutes"] = serde_json::json!(10);
|
||||
expected_row["reset_credit_max_weekly_remaining_percent"] = serde_json::Value::Null;
|
||||
// 完整升级还会清理自动回切字段,其余运行设置必须原样保留
|
||||
expected_row["smart_scheduling_json"]
|
||||
.as_object_mut()
|
||||
.unwrap()
|
||||
.remove("preferHigherWeight");
|
||||
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
let stored: serde_json::Value =
|
||||
sqlx::query_scalar("select to_jsonb(runtime_settings) from runtime_settings where id = 1")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(stored, expected_row);
|
||||
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
|
||||
let loaded = repository.load_runtime_settings().await.unwrap();
|
||||
assert_eq!(
|
||||
serde_json::to_value(&loaded.values.codex_privacy_policy).unwrap(),
|
||||
expected
|
||||
);
|
||||
// 服务重启后再次检查迁移,不产生写入或额外嵌套
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
let stored: serde_json::Value =
|
||||
sqlx::query_scalar("select to_jsonb(runtime_settings) from runtime_settings where id = 1")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(stored, expected_row);
|
||||
let snapshot = gateway_store::postgres::RuntimeSnapshotRepository::load_runtime_snapshot(
|
||||
&gateway_store::postgres::PgRuntimeSnapshotRepository::new(database.pool.clone()),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
serde_json::to_value(snapshot.settings).unwrap()["codex_privacy_policy"],
|
||||
expected
|
||||
);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn privacy_steps_migration_rejects_non_object_without_losing_data() {
|
||||
let Some(database) = TestDatabase::create_through("privacy_invalid", 30).await else {
|
||||
return;
|
||||
};
|
||||
let invalid = serde_json::json!({
|
||||
"enabled":false,"onError":"reject_request","rules":[{},42]
|
||||
});
|
||||
sqlx::query("update runtime_settings set codex_privacy_policy_json = $1 where id = 1")
|
||||
.bind(&invalid)
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(super::TEST_MIGRATOR.run(&database.pool).await.is_err());
|
||||
let stored: serde_json::Value =
|
||||
sqlx::query_scalar("select codex_privacy_policy_json from runtime_settings where id = 1")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(stored, invalid);
|
||||
let applied: bool =
|
||||
sqlx::query_scalar("select exists(select 1 from _sqlx_migrations where version = 31)")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(!applied);
|
||||
sqlx::query("update runtime_settings set codex_privacy_policy_json = jsonb_set(codex_privacy_policy_json, '{rules}', '[]') where id = 1")
|
||||
.execute(&database.pool).await.unwrap();
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
let loaded = PgRuntimeSettingsRepository::new(database.pool.clone())
|
||||
.load_runtime_settings()
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(!loaded.values.codex_privacy_policy.enabled);
|
||||
assert!(loaded.values.codex_privacy_policy.rules.is_empty());
|
||||
assert_eq!(
|
||||
loaded.values.codex_privacy_policy.on_error,
|
||||
gateway_core::settings::privacy::PrivacyFailureMode::RejectRequest
|
||||
);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn smart_settings_upgrade_preserves_selection_and_publishes_custom_config() {
|
||||
async fn smart_settings_round_trip_publish_and_restore_database_defaults() {
|
||||
use gateway_core::account::SmartSchedulingConfig;
|
||||
use gateway_store::postgres::{PgRuntimeSnapshotRepository, RuntimeSnapshotRepository};
|
||||
let Some(database) = TestDatabase::create_through("smart_config", 18).await else {
|
||||
let Some(database) = TestDatabase::create("smart_config").await else {
|
||||
return;
|
||||
};
|
||||
// 升级前不能用包含新列的 Repository,直接写入旧版本已有字段
|
||||
sqlx::query("update runtime_settings set rotation_strategy = 'sticky', refresh_margin_seconds = 3600 where id = 1")
|
||||
.execute(&database.pool).await.unwrap();
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
|
||||
let before = repository.load_runtime_settings().await.unwrap();
|
||||
assert_eq!(before.rotation_strategy, "sticky");
|
||||
// 0022 将仍为旧默认 3600 的行迁移到 300;管理员自定义值才会原样保留。
|
||||
assert_eq!(before.values.refresh_margin_seconds, 300);
|
||||
assert_eq!(
|
||||
before.values.smart_scheduling,
|
||||
SmartSchedulingConfig::default()
|
||||
@@ -333,8 +140,17 @@ async fn smart_settings_upgrade_preserves_selection_and_publishes_custom_config(
|
||||
serde_json::to_value(expected).unwrap()
|
||||
);
|
||||
assert!(snapshot.config_revision > before.config_revision);
|
||||
sqlx::query("update runtime_settings set smart_scheduling_json = default where id = 1")
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
before.values.smart_scheduling,
|
||||
repository
|
||||
.load_runtime_settings()
|
||||
.await
|
||||
.unwrap()
|
||||
.values
|
||||
.smart_scheduling,
|
||||
SmartSchedulingConfig::default()
|
||||
);
|
||||
database.close().await;
|
||||
@@ -434,52 +250,33 @@ async fn unlimited_default_account_concurrency_round_trips_without_relaxing_othe
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn migrations_should_narrow_the_default_refresh_margin_to_the_codex_baseline() {
|
||||
async fn refresh_margin_defaults_apply_to_initial_settings_and_database_resets() {
|
||||
let Some(database) = TestDatabase::create("refresh_margin_default").await else {
|
||||
return;
|
||||
};
|
||||
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
|
||||
// 新装实例:0001 种子行携带旧默认 3600,0022 数据迁移统一收敛到 300。
|
||||
let settings = repository
|
||||
.load_runtime_settings()
|
||||
.await
|
||||
.expect("load settings");
|
||||
assert_eq!(settings.values.refresh_margin_seconds, 300);
|
||||
// 列默认值同步收窄,重建行不会回退到旧值;目录查询限定本测试 schema,
|
||||
// 同库并行测试里停留在旧迁移版本的 schema 仍保留 3600 旧默认,不能被读到。
|
||||
let column_default: String = sqlx::query_scalar(
|
||||
"select column_default from information_schema.columns \
|
||||
where table_schema = current_schema() \
|
||||
and table_name = 'runtime_settings' \
|
||||
and column_name = 'refresh_margin_seconds'",
|
||||
)
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.expect("read column default");
|
||||
assert_eq!(column_default, "300");
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn refresh_margin_migration_should_preserve_admin_customized_values() {
|
||||
// 升级语义:0021 之前部署的实例经 0022 迁移后,仅旧默认 3600 收敛到 300,
|
||||
// 管理员自定义值原样保留。
|
||||
let Some(database) = TestDatabase::create_through("refresh_margin_custom", 21).await else {
|
||||
return;
|
||||
};
|
||||
sqlx::query("update runtime_settings set refresh_margin_seconds = 1800 where id = 1")
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.expect("persist customized margin");
|
||||
super::TEST_MIGRATOR
|
||||
.run(&database.pool)
|
||||
.unwrap();
|
||||
sqlx::query("update runtime_settings set refresh_margin_seconds = default where id = 1")
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.expect("apply remaining migrations");
|
||||
let settings = PgRuntimeSettingsRepository::new(database.pool.clone())
|
||||
.load_runtime_settings()
|
||||
.await
|
||||
.expect("load settings");
|
||||
assert_eq!(settings.values.refresh_margin_seconds, 1_800);
|
||||
.expect("reset refresh margin to the database default");
|
||||
assert_eq!(
|
||||
repository
|
||||
.load_runtime_settings()
|
||||
.await
|
||||
.unwrap()
|
||||
.values
|
||||
.refresh_margin_seconds,
|
||||
300
|
||||
);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
@@ -714,7 +511,9 @@ async fn concurrency_queue_settings_round_trip_into_the_runtime_snapshot() {
|
||||
),
|
||||
(0, 0, 30, 0)
|
||||
);
|
||||
let mut update = settings_with_margin(3600);
|
||||
let mut update = settings_with_margin(1234);
|
||||
update.values.max_concurrent_per_account = 7;
|
||||
update.values.refresh_concurrency = 12;
|
||||
update.values.max_waiting_per_key = 5;
|
||||
update.values.max_waiting_per_account = 7;
|
||||
update.values.concurrency_wait_timeout_seconds = 12;
|
||||
@@ -723,12 +522,15 @@ async fn concurrency_queue_settings_round_trip_into_the_runtime_snapshot() {
|
||||
let settings = repository.load_runtime_settings().await.unwrap();
|
||||
assert_eq!(
|
||||
(
|
||||
settings.values.max_concurrent_per_account,
|
||||
settings.values.refresh_concurrency,
|
||||
settings.values.refresh_margin_seconds,
|
||||
settings.values.max_waiting_per_key,
|
||||
settings.values.max_waiting_per_account,
|
||||
settings.values.concurrency_wait_timeout_seconds,
|
||||
settings.values.openai_guardian_reserved_concurrency,
|
||||
),
|
||||
(5, 7, 12, 1)
|
||||
(7, 12, 1234, 5, 7, 12, 1)
|
||||
);
|
||||
let snapshot = PgRuntimeSnapshotRepository::new(database.pool.clone())
|
||||
.load_runtime_snapshot()
|
||||
@@ -737,6 +539,9 @@ async fn concurrency_queue_settings_round_trip_into_the_runtime_snapshot() {
|
||||
let snapshot_values = serde_json::to_value(&snapshot.settings).unwrap();
|
||||
assert_eq!(
|
||||
(
|
||||
snapshot_values["max_concurrent_per_account"]
|
||||
.as_u64()
|
||||
.unwrap(),
|
||||
snapshot_values["max_waiting_per_key"].as_u64().unwrap(),
|
||||
snapshot_values["max_waiting_per_account"].as_u64().unwrap(),
|
||||
snapshot_values["concurrency_wait_timeout_seconds"]
|
||||
@@ -746,7 +551,7 @@ async fn concurrency_queue_settings_round_trip_into_the_runtime_snapshot() {
|
||||
.as_u64()
|
||||
.unwrap(),
|
||||
),
|
||||
(5, 7, 12, 1)
|
||||
(7, 5, 7, 12, 1)
|
||||
);
|
||||
assert!(snapshot.config_revision > before.config_revision);
|
||||
database.close().await;
|
||||
@@ -1235,48 +1040,6 @@ async fn warmup_cursor_survives_restart_and_settings_updates_without_rewinding()
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn runtime_scheduling_upgrade_preserves_settings_without_a_warmup_table() {
|
||||
let Some(database) = TestDatabase::create_through("runtime_scheduling_upgrade", 20).await
|
||||
else {
|
||||
return;
|
||||
};
|
||||
sqlx::query(
|
||||
"update runtime_settings set rotation_strategy = 'sticky',
|
||||
account_warmup_enabled = true, account_warmup_schedule_time = '08:00,13:00',
|
||||
account_warmup_model = 'test-model' where id = 1",
|
||||
)
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
let warmup_table: Option<String> =
|
||||
sqlx::query_scalar("select to_regclass('account_warmup_slots')::text")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(warmup_table.is_none());
|
||||
let cursor: Option<DateTime<Utc>> =
|
||||
sqlx::query_scalar("select account_warmup_cursor from runtime_settings where id = 1")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(cursor.is_none());
|
||||
let settings = PgRuntimeSettingsRepository::new(database.pool.clone())
|
||||
.load_runtime_settings()
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(settings.rotation_strategy, "sticky");
|
||||
assert!(settings.values.account_warmup_enabled);
|
||||
assert_eq!(settings.values.account_warmup_schedule_time, "08:00,13:00");
|
||||
assert_eq!(
|
||||
settings.values.account_warmup_model.as_deref(),
|
||||
Some("test-model")
|
||||
);
|
||||
assert_eq!(settings.values.openai_guardian_reserved_concurrency, 0);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn warmup_cursor_resolves_dst_and_deduplicates_across_timezones() {
|
||||
use gateway_core::{provider_ports::ProviderRuntimePolicyPort, time::DeploymentTimeZone};
|
||||
@@ -1320,23 +1083,17 @@ async fn warmup_cursor_resolves_dst_and_deduplicates_across_timezones() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn account_affinity_upgrade_defaults_and_updates_reach_the_snapshot() {
|
||||
async fn account_affinity_defaults_and_updates_reach_the_snapshot() {
|
||||
use gateway_core::account::AccountAffinity;
|
||||
use gateway_store::postgres::{PgRuntimeSnapshotRepository, RuntimeSnapshotRepository};
|
||||
let Some(database) = TestDatabase::create_through("account_affinity_upgrade", 22).await else {
|
||||
let Some(database) = TestDatabase::create("account_affinity").await else {
|
||||
return;
|
||||
};
|
||||
sqlx::query("update runtime_settings set rotation_strategy = 'sticky', config_revision = config_revision + 1 where id = 1")
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
|
||||
let before = repository.load_runtime_settings().await.unwrap();
|
||||
assert_eq!(before.rotation_strategy, "sticky");
|
||||
assert_eq!(
|
||||
before.values.openai_account_affinity,
|
||||
AccountAffinity::Relaxed
|
||||
AccountAffinity::Strict
|
||||
);
|
||||
assert_eq!(before.values.max_account_rotations, 3);
|
||||
assert_eq!(before.values.openai_session_affinity_ttl_hours, 24);
|
||||
@@ -1402,105 +1159,12 @@ async fn account_affinity_upgrade_defaults_and_updates_reach_the_snapshot() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn preferred_affinity_migration_defaults_to_strict_and_preserves_saved_modes() {
|
||||
use gateway_core::account::AccountAffinity;
|
||||
let Some(fresh) = TestDatabase::create("affinity_default").await else {
|
||||
async fn reset_credit_weekly_threshold_defaults_unset_and_round_trips() {
|
||||
let Some(database) = TestDatabase::create("reset_weekly_threshold").await else {
|
||||
return;
|
||||
};
|
||||
let repository = PgRuntimeSettingsRepository::new(fresh.pool.clone());
|
||||
assert_eq!(
|
||||
repository
|
||||
.load_runtime_settings()
|
||||
.await
|
||||
.unwrap()
|
||||
.values
|
||||
.openai_account_affinity,
|
||||
AccountAffinity::Strict
|
||||
);
|
||||
fresh.close().await;
|
||||
|
||||
for mode in [AccountAffinity::Relaxed, AccountAffinity::Strict] {
|
||||
let database = TestDatabase::create_through("affinity_preserve", 23)
|
||||
.await
|
||||
.unwrap();
|
||||
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
|
||||
// 升级前只写当时已有列,现行 Repository 依赖完整迁移后的 schema
|
||||
sqlx::query("update runtime_settings set openai_account_affinity = $1, config_revision = config_revision + 1, updated_at = now() where id = 1")
|
||||
.bind(match mode {
|
||||
AccountAffinity::Relaxed => "relaxed",
|
||||
_ => "strict",
|
||||
})
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
assert_eq!(
|
||||
repository
|
||||
.load_runtime_settings()
|
||||
.await
|
||||
.unwrap()
|
||||
.values
|
||||
.openai_account_affinity,
|
||||
mode
|
||||
);
|
||||
database.close().await;
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn removing_switchback_preserves_custom_weights_and_replaces_database_default() {
|
||||
let Some(database) = TestDatabase::create_through("remove_switchback", 31).await else {
|
||||
return;
|
||||
};
|
||||
let expected =
|
||||
gateway_core::account::SmartSchedulingConfig::new([0.0, 2.0, 1.0, 0.5, 1.2, 2.3]).unwrap();
|
||||
let mut legacy = serde_json::to_value(expected).unwrap();
|
||||
legacy["preferHigherWeight"] = serde_json::json!(true);
|
||||
sqlx::query("update runtime_settings set smart_scheduling_json = $1 where id = 1")
|
||||
.bind(legacy)
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
|
||||
assert_eq!(
|
||||
repository
|
||||
.load_runtime_settings()
|
||||
.await
|
||||
.unwrap()
|
||||
.values
|
||||
.smart_scheduling,
|
||||
expected
|
||||
);
|
||||
sqlx::query("update runtime_settings set smart_scheduling_json = default where id = 1")
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
repository
|
||||
.load_runtime_settings()
|
||||
.await
|
||||
.unwrap()
|
||||
.values
|
||||
.smart_scheduling,
|
||||
gateway_core::account::SmartSchedulingConfig::default()
|
||||
);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn reset_credit_weekly_threshold_upgrade_defaults_unset_and_round_trips() {
|
||||
let Some(database) = TestDatabase::create_through("reset_weekly_threshold", 34).await else {
|
||||
return;
|
||||
};
|
||||
sqlx::query("update runtime_settings set reset_credit_lead_minutes = 20 where id = 1")
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
super::TEST_MIGRATOR.run(&database.pool).await.unwrap();
|
||||
let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
|
||||
let settings = repository.load_runtime_settings().await.unwrap();
|
||||
assert_eq!(settings.values.reset_credit_lead_minutes, 20);
|
||||
assert_eq!(
|
||||
settings.values.reset_credit_max_weekly_remaining_percent,
|
||||
None
|
||||
|
||||
@@ -2,59 +2,6 @@
|
||||
|
||||
use super::TestDatabase;
|
||||
|
||||
#[tokio::test]
|
||||
async fn error_details_migration_preserves_existing_request_and_event_text() {
|
||||
let Some(db) = TestDatabase::create_through("error_details_upgrade", 22).await else {
|
||||
return;
|
||||
};
|
||||
seed_legacy_request(&db.pool).await;
|
||||
let raw = "{ \"error\": {\"code\":\"Vendor.Unknown\",\"message\":\"原始错误\"} }";
|
||||
sqlx::query("update model_requests set raw_upstream_error = $1 where id = 'req_integrity'")
|
||||
.bind(raw)
|
||||
.execute(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
sqlx::query(
|
||||
"insert into ops_events (id, level, component, operation, failure_kind, message, created_at, raw_upstream_error)
|
||||
values ('error_details_upgrade', 'error', 'provider', 'generate', 'unavailable', 'safe message', now(), $1)",
|
||||
).bind(raw).execute(&db.pool).await.unwrap();
|
||||
|
||||
super::TEST_MIGRATOR.run(&db.pool).await.unwrap();
|
||||
let details: Vec<String> = sqlx::query_scalar(
|
||||
"select error_details from model_requests where id = 'req_integrity'
|
||||
union all select error_details from ops_events where id = 'error_details_upgrade'",
|
||||
)
|
||||
.fetch_all(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(details, vec![raw.to_owned(), raw.to_owned()]);
|
||||
let old_columns: i64 = sqlx::query_scalar(
|
||||
"select count(*) from information_schema.columns
|
||||
where table_schema = current_schema() and table_name in ('model_requests', 'ops_events')
|
||||
and column_name = 'raw_upstream_error'",
|
||||
)
|
||||
.fetch_one(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(old_columns, 0);
|
||||
db.close().await;
|
||||
}
|
||||
|
||||
async fn seed_legacy_request(pool: &sqlx::PgPool) {
|
||||
sqlx::query(
|
||||
"insert into model_requests (
|
||||
id, client_api_key_ref, config_revision, protocol, operation, endpoint,
|
||||
client_transport, started_at, deadline_at, outcome, completed_at, routing_scope
|
||||
) values (
|
||||
'req_integrity', 'deleted_key', 1, 'openai', 'responses', '/v1/responses',
|
||||
'http_sse', now(), now() + interval '1 hour', 'failed', now(), 'all'
|
||||
)",
|
||||
)
|
||||
.execute(pool)
|
||||
.await
|
||||
.expect("seed request without optional facts");
|
||||
}
|
||||
|
||||
async fn seed_request(pool: &sqlx::PgPool) {
|
||||
sqlx::query(
|
||||
"insert into model_requests (
|
||||
@@ -191,11 +138,6 @@ async fn historical_references_are_validated_without_live_foreign_keys() {
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(references, "already_deleted_account");
|
||||
let live_columns: i64 = sqlx::query_scalar("select count(*) from information_schema.columns where table_schema = current_schema()
|
||||
and ((table_name = 'model_requests' and column_name in ('client_api_key_id','provider_account_id'))
|
||||
or (table_name = 'ops_events' and column_name = 'provider_account_id'))")
|
||||
.fetch_one(&db.pool).await.unwrap();
|
||||
assert_eq!(live_columns, 0);
|
||||
db.close().await;
|
||||
}
|
||||
|
||||
@@ -252,89 +194,6 @@ async fn backup_completion_cannot_precede_its_start() {
|
||||
db.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn request_observation_migration_preserves_every_historical_field() {
|
||||
let Some(db) = TestDatabase::create_through("request_observation_upgrade", 24).await else {
|
||||
return;
|
||||
};
|
||||
seed_legacy_request(&db.pool).await;
|
||||
sqlx::raw_sql(
|
||||
"insert into model_requests
|
||||
select (jsonb_populate_record(null::model_requests,
|
||||
to_jsonb(mr) || jsonb_build_object('id', 'req_empty'))).*
|
||||
from model_requests mr;
|
||||
update model_requests set config_revision = 7,
|
||||
provider_account_name_snapshot = 'historical account',
|
||||
provider_account_email_snapshot = 'history@example.invalid',
|
||||
provider_account_authentication_kind_snapshot = 'oauth',
|
||||
routing_scope = 'groups', routing_group_refs = array['deleted_group'],
|
||||
routing_group_names_snapshot = '[\"历史组\"]',
|
||||
provider_error_code = 'rate_limit', error_message = 'safe error', retry_after_ms = 0,
|
||||
client_ip = '192.0.2.10/24', user_agent = 'fixture', reasoning_effort = 'high',
|
||||
reasoning_preset = 'balanced', subagent_kind = 'review', compact = true,
|
||||
transport_decision_wait_ms = 0, connect_ms = 2, headers_ms = 3,
|
||||
first_event_ms = 4, first_reasoning_ms = 5, first_text_ms = 6,
|
||||
first_token_ms = 5, latency_ms = 1000, provider_processing_ms = 7,
|
||||
admission_decision_ms = 0, account_selection_wait_ms = 8,
|
||||
capacity_used_slots = 0, capacity_total_slots = 20,
|
||||
http_version = 'HTTP/2', websocket_pool = 'reuse',
|
||||
upstream_connection_id = 'connection', upstream_connection_exit_reason = 'peer_closed',
|
||||
upstream_connection_age_ms = 10, upstream_connection_idle_ms = 0,
|
||||
upstream_response_model = 'response-model', continuation_requested = true,
|
||||
continuation_previous_response_id_hash = repeat('a', 64),
|
||||
continuation_unavailable_reason = 'missing_response',
|
||||
recovery_request_id = 'recovery', recovered_at = completed_at + interval '1 second',
|
||||
recovery_attempt_count = 1, recovery_retry_delay_ms = 0, recovery_total_latency_ms = 1000,
|
||||
input_tokens = 10, output_tokens = 20, total_tokens = 30,
|
||||
cost_source = 'provider_reported', cost_amount = 0.0123456789, cost_currency = 'USD',
|
||||
error_details = '{ \"opaque\": true }', provider_observation_json = '{\"vendor\":true}',
|
||||
diagnostic_trace_json = '{\"attempts\":[]}'
|
||||
where id = 'req_integrity'",
|
||||
).execute(&db.pool).await.expect("populate all moved fields in the released schema");
|
||||
let before: Vec<serde_json::Value> =
|
||||
sqlx::query_scalar("select to_jsonb(mr) from model_requests mr order by id")
|
||||
.fetch_all(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
super::TEST_MIGRATOR.run_to(27, &db.pool).await.unwrap();
|
||||
let after: Vec<serde_json::Value> = sqlx::query_scalar(
|
||||
"select to_jsonb(mr) - 'request_observation_json' - 'upstream_response_ms' - 'upstream_api_overhead_ms' - 'upstream_engine_ms' - 'upstream_engine_iapi_ttft_ms' - 'upstream_engine_service_ttft_ms' - 'upstream_engine_iapi_tbt_ms' - 'upstream_engine_service_tbt_ms'
|
||||
from model_request_observations mr order by id",
|
||||
)
|
||||
.fetch_all(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
after, before,
|
||||
"all 90 historical facts survive the migration"
|
||||
);
|
||||
let columns: i64 = sqlx::query_scalar(
|
||||
"select count(*) from information_schema.columns
|
||||
where table_schema = current_schema() and table_name = 'model_requests'",
|
||||
)
|
||||
.fetch_one(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(columns, 49);
|
||||
let observation: serde_json::Value = sqlx::query_scalar(
|
||||
"select request_observation_json from model_requests where id = 'req_integrity'",
|
||||
)
|
||||
.fetch_one(&db.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
observation["timings"]["local"]["transportDecisionWaitMs"],
|
||||
0
|
||||
);
|
||||
assert_eq!(observation["scheduling"]["capacityUsedSlots"], 0);
|
||||
assert!(
|
||||
observation["timings"]["upstream"]
|
||||
.get("responseMs")
|
||||
.is_none()
|
||||
);
|
||||
db.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn request_observation_reject_malformed_documents_and_keep_upstream_clock_independent() {
|
||||
use serde_json::json;
|
||||
|
||||
@@ -8,7 +8,7 @@ use gateway_store::postgres::{
|
||||
use super::TestDatabase;
|
||||
|
||||
#[test]
|
||||
fn snapshot_client_policy_contains_only_common_limits() {
|
||||
fn client_key_snapshot_debug_redacts_plaintext_key() {
|
||||
let policy = ClientApiKeySnapshot {
|
||||
name_snapshot: None,
|
||||
request_profiles: Default::default(),
|
||||
@@ -20,8 +20,6 @@ fn snapshot_client_policy_contains_only_common_limits() {
|
||||
requests_per_minute: 60,
|
||||
},
|
||||
};
|
||||
assert_eq!(policy.limits.max_concurrency, 3);
|
||||
assert!(policy.group_ids.is_empty());
|
||||
assert!(!format!("{policy:?}").contains("sk_snapshot_secret"));
|
||||
}
|
||||
|
||||
@@ -50,6 +48,8 @@ async fn runtime_snapshot_loads_enabled_plaintext_key_without_debug_exposure() {
|
||||
Some("snapshot")
|
||||
);
|
||||
assert!(snapshot.client_api_keys[0].group_ids.is_empty());
|
||||
assert_eq!(snapshot.client_api_keys[0].limits.max_concurrency, 2);
|
||||
assert_eq!(snapshot.client_api_keys[0].limits.requests_per_minute, 60);
|
||||
assert_eq!(
|
||||
snapshot.client_api_keys[0].plaintext_key.expose_for_auth(),
|
||||
plaintext
|
||||
|
||||
@@ -776,12 +776,12 @@ fn authorization_url(
|
||||
.append_pair("response_type", "code")
|
||||
.append_pair("client_id", oauth_client_id)
|
||||
.append_pair("redirect_uri", OFFICIAL_CODEX_REDIRECT_URI)
|
||||
.append_pair("scope", AUTHORIZATION_SCOPE)
|
||||
.append_pair("code_challenge", challenge.as_str())
|
||||
.append_pair("code_challenge_method", "S256")
|
||||
.append_pair("state", pending.state.expose_secret())
|
||||
.append_pair("scope", AUTHORIZATION_SCOPE)
|
||||
.append_pair("id_token_add_organizations", "true")
|
||||
.append_pair("codex_cli_simplified_flow", "true")
|
||||
.append_pair("state", pending.state.expose_secret())
|
||||
.append_pair("originator", AUTHORIZATION_ORIGINATOR)
|
||||
.append_pair("codex_app_version", desktop_version)
|
||||
.append_pair("source_surface_stable_id", &stable_id)
|
||||
|
||||
@@ -73,6 +73,10 @@ const DEFAULT_RATE_LIMIT_COOLDOWN: Duration = Duration::from_secs(60);
|
||||
pub(crate) const QUOTA_SCHEDULING_TTL: Duration = Duration::from_secs(10 * 60);
|
||||
const QUOTA_HYDRATION_FAILURE_TTL: Duration = Duration::from_secs(5);
|
||||
const PERIODIC_QUOTA_REFRESH_RETRY_INTERVAL: Duration = Duration::from_secs(30 * 60);
|
||||
const QUOTA_RESET_FAST_REFRESH_WINDOW: Duration = Duration::from_secs(5 * 60);
|
||||
const QUOTA_RESET_FAST_REFRESH_INTERVAL: Duration = Duration::from_secs(30);
|
||||
const QUOTA_RESET_REFRESH_RETRY_INTERVAL: Duration = Duration::from_secs(5 * 60);
|
||||
const EXHAUSTED_QUOTA_RESET_GRACE: Duration = Duration::from_secs(10);
|
||||
const QUOTA_RESET_GRACE: Duration = Duration::from_secs(2 * 60);
|
||||
/// 首次 OAuth 异步观察失败时,由既有 quota worker 兜底重试的单轮上限
|
||||
/// 随机启动延迟已把候选摊到多轮,该上限仅作单轮安全边界
|
||||
|
||||
@@ -4,7 +4,7 @@ use std::collections::BTreeMap;
|
||||
use std::time::SystemTime;
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
use gateway_core::account::{QuotaEvidence, QuotaState};
|
||||
use gateway_core::account::{QuotaAccessState, QuotaEvidence, QuotaState};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::{Map, Value};
|
||||
|
||||
@@ -127,11 +127,8 @@ fn window_reset_recovered(window: &CodexQuotaWindow, previous: Option<DateTime<U
|
||||
}
|
||||
|
||||
/// 统一手动刷新和 worker 的恢复规则
|
||||
/// 已耗尽窗口的解除有两条证据路径:
|
||||
/// reset 前进且用量低于低用量阈值(窗口已滚动),或同一窗口内连续两次观测
|
||||
/// 都未触顶(滑动窗口的用量回落,reset 不滚动)
|
||||
/// 后者防止账号被钉在旧
|
||||
/// reset 日期上;连续两次的要求排除耗尽后立刻拉到的旧快照
|
||||
/// 到期前仍要求窗口滚动且低用量,或同窗口连续两次未触顶
|
||||
/// 已记录的耗尽窗口全部到期后,允许新的明确放行观察解除,兼容用量百分比滞后
|
||||
pub(super) fn reconcile_refresh(
|
||||
current: QuotaState,
|
||||
refreshed: &mut CodexAccountQuotaSnapshot,
|
||||
@@ -139,6 +136,11 @@ pub(super) fn reconcile_refresh(
|
||||
document: &mut Map<String, Value>,
|
||||
) -> Result<QuotaState, CodexCredentialQuotaError> {
|
||||
document.remove(RECOVERY_FIELD);
|
||||
if refreshed.credits_usable() {
|
||||
// 点数独立于套餐窗口,恢复后不能继续携带已失效的窗口恢复进度
|
||||
refreshed.recovery = None;
|
||||
return Ok(QuotaState::allowed(refreshed.observed_at()));
|
||||
}
|
||||
let state = if current.is_exhausted() {
|
||||
current
|
||||
} else {
|
||||
@@ -155,7 +157,18 @@ pub(super) fn reconcile_refresh(
|
||||
if recovery.pending.is_empty() {
|
||||
recovery = QuotaRecovery::new(state, refreshed);
|
||||
}
|
||||
if current.is_exhausted() && recovery.observe(refreshed) {
|
||||
// 只信任本次耗尽之后的边界,防止再次拒绝后复用已经过期的 reset 反复解封
|
||||
// 多个耗尽窗口必须全部到期,未知 reset 仍走原有恢复规则
|
||||
let allowed_after_reset = refreshed.quota().access() == QuotaAccessState::Allowed
|
||||
&& !recovery.pending.is_empty()
|
||||
&& recovery.pending.values().all(|reset| {
|
||||
reset.is_some_and(|reset| {
|
||||
reset.timestamp_micros() > recovery.exhausted_at_micros
|
||||
&& SystemTime::from(reset) <= refreshed.observed_at()
|
||||
})
|
||||
});
|
||||
if current.is_exhausted() && (allowed_after_reset || recovery.observe(refreshed)) {
|
||||
refreshed.recovery = None;
|
||||
return Ok(QuotaState::allowed(refreshed.observed_at()));
|
||||
}
|
||||
let reset_at = recovery
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
//! 请求级额度投影、首查与周期刷新排程
|
||||
|
||||
use super::*;
|
||||
use tokio::time::Instant as TokioInstant;
|
||||
|
||||
impl CodexCredentialQuotaService {
|
||||
/// 批量预热请求级额度投影;持久层或 Provider JSON 异常只退化为未知额度
|
||||
@@ -67,7 +68,7 @@ struct CodexQuotaProjectionState {
|
||||
}
|
||||
|
||||
struct CodexQuotaRefreshAttempt {
|
||||
monotonic_at: Instant,
|
||||
monotonic_at: TokioInstant,
|
||||
wall_at: SystemTime,
|
||||
}
|
||||
|
||||
@@ -230,7 +231,7 @@ impl CodexQuotaSchedulingProjection {
|
||||
.iter()
|
||||
.map(|(account, _)| account.id().clone())
|
||||
.collect::<BTreeSet<_>>();
|
||||
let refreshed_at = Instant::now();
|
||||
let refreshed_at = TokioInstant::now();
|
||||
let mut state = self
|
||||
.state
|
||||
.write()
|
||||
@@ -239,12 +240,11 @@ impl CodexQuotaSchedulingProjection {
|
||||
.last_periodic_refresh_at
|
||||
.retain(|account_id, _| candidate_ids.contains(account_id));
|
||||
|
||||
// 已耗尽账号首次立即复核,之后每 30 分钟复核,以发现官方提前重置
|
||||
// reset + 2 分钟额外触发一次复核,给上游重置留出传播时间
|
||||
// 已耗尽账号到达 reset 后短周期复核,其余时间保留 30 分钟兜底
|
||||
// 正常账号仅在非零用量窗口经过宽限期后参与,未更新时复用周期节流
|
||||
let mut reserved = Vec::new();
|
||||
for (account, target_reset) in candidates {
|
||||
if !periodic_quota_refresh_due(&state, account.id(), target_reset, now, refreshed_at) {
|
||||
if !periodic_quota_refresh_due(&state, &account, target_reset, now, refreshed_at) {
|
||||
continue;
|
||||
}
|
||||
state.last_periodic_refresh_at.insert(
|
||||
@@ -364,22 +364,34 @@ fn quota_refresh_candidate(
|
||||
|
||||
fn periodic_quota_refresh_due(
|
||||
state: &CodexQuotaProjectionState,
|
||||
account_id: &ProviderAccountId,
|
||||
account: &ProviderAccount,
|
||||
reset_at: Option<SystemTime>,
|
||||
now: SystemTime,
|
||||
monotonic_now: Instant,
|
||||
monotonic_now: TokioInstant,
|
||||
) -> bool {
|
||||
state
|
||||
.last_periodic_refresh_at
|
||||
.get(account_id)
|
||||
.is_none_or(|last| {
|
||||
monotonic_now.saturating_duration_since(last.monotonic_at)
|
||||
>= PERIODIC_QUOTA_REFRESH_RETRY_INTERVAL
|
||||
|| reset_at
|
||||
.and_then(|reset| reset.checked_add(QUOTA_RESET_GRACE))
|
||||
// 已在该边界之后复核过时回到周期重试,避免过期 reset 每轮触发
|
||||
.is_some_and(|due_at| last.wall_at < due_at && due_at <= now)
|
||||
})
|
||||
let mut interval = PERIODIC_QUOTA_REFRESH_RETRY_INTERVAL;
|
||||
let due_at = if account.quota().is_exhausted() {
|
||||
if let Some(since_reset) = reset_at.and_then(|reset| now.duration_since(reset).ok()) {
|
||||
// 上游边界附近可能仍返回旧额度,首次复核也保留传播余量
|
||||
if since_reset < EXHAUSTED_QUOTA_RESET_GRACE {
|
||||
return false;
|
||||
}
|
||||
interval = if since_reset < QUOTA_RESET_FAST_REFRESH_WINDOW {
|
||||
QUOTA_RESET_FAST_REFRESH_INTERVAL
|
||||
} else {
|
||||
QUOTA_RESET_REFRESH_RETRY_INTERVAL
|
||||
};
|
||||
}
|
||||
reset_at.and_then(|reset| reset.checked_add(EXHAUSTED_QUOTA_RESET_GRACE))
|
||||
} else {
|
||||
reset_at.and_then(|reset| reset.checked_add(QUOTA_RESET_GRACE))
|
||||
};
|
||||
let Some(last) = state.last_periodic_refresh_at.get(account.id()) else {
|
||||
return true;
|
||||
};
|
||||
monotonic_now.saturating_duration_since(last.monotonic_at) >= interval
|
||||
// 跨越 reset 宽限期的首轮复核不受旧间隔阻挡,之后按对应间隔节流
|
||||
|| due_at.is_some_and(|due_at| last.wall_at < due_at && due_at <= now)
|
||||
}
|
||||
|
||||
fn insert_projection_entry(
|
||||
|
||||
@@ -515,7 +515,7 @@ impl CodexCredentialQuotaService {
|
||||
.await
|
||||
}
|
||||
|
||||
/// 真实限额失败后的异步刷新只补齐展示快照,不允许 usage 快照撤销已确认的失败状态
|
||||
/// 真实限额失败后的异步回读仅允许明确可用的点数恢复访问,套餐状态仍保留失败保护
|
||||
pub(crate) async fn refresh_account_after_failure(
|
||||
&self,
|
||||
account_id: &ProviderAccountId,
|
||||
@@ -589,9 +589,10 @@ impl CodexCredentialQuotaService {
|
||||
} else {
|
||||
None
|
||||
};
|
||||
if account.quota().is_exhausted()
|
||||
&& matches!(authority, QuotaRefreshAuthority::PreserveAccess)
|
||||
{
|
||||
// 失败后回读只允许明确可用的点数恢复访问,套餐 allowed 仍可能滞后
|
||||
let observe_access =
|
||||
matches!(authority, QuotaRefreshAuthority::ObserveAccess) || snapshot.credits_usable();
|
||||
if account.quota().is_exhausted() && !observe_access {
|
||||
if self
|
||||
.store
|
||||
.touch_quota_observation(QuotaObservationTouch {
|
||||
@@ -615,14 +616,15 @@ impl CodexCredentialQuotaService {
|
||||
.with_observed_at(observed_at),
|
||||
});
|
||||
}
|
||||
let state = match authority {
|
||||
QuotaRefreshAuthority::ObserveAccess => reconcile_refresh(
|
||||
let state = if observe_access {
|
||||
reconcile_refresh(
|
||||
account.quota(),
|
||||
&mut snapshot,
|
||||
previous.as_ref(),
|
||||
&mut object,
|
||||
)?,
|
||||
QuotaRefreshAuthority::PreserveAccess => account.quota(),
|
||||
)?
|
||||
} else {
|
||||
account.quota()
|
||||
};
|
||||
let snapshot = snapshot.with_quota_state(state);
|
||||
if self
|
||||
|
||||
@@ -116,6 +116,7 @@ pub struct CodexAccountQuotaSnapshot {
|
||||
observed_at: SystemTime,
|
||||
plan_type: Option<String>,
|
||||
credits: Option<CreditsSnapshot>,
|
||||
credits_usable: bool,
|
||||
fact: CodexQuotaFact,
|
||||
quota: QuotaState,
|
||||
windows: Vec<CodexQuotaWindow>,
|
||||
@@ -144,12 +145,16 @@ impl CodexAccountQuotaSnapshot {
|
||||
self.plan_type.as_deref()
|
||||
}
|
||||
|
||||
/// 点数只用于展示,不改变套餐限额或账号可用性
|
||||
/// 上游明确返回的点数事实;点数可用性独立于套餐窗口
|
||||
#[must_use]
|
||||
pub fn credits(&self) -> Option<&CreditsSnapshot> {
|
||||
self.credits.as_ref()
|
||||
}
|
||||
|
||||
pub(super) const fn credits_usable(&self) -> bool {
|
||||
self.credits_usable
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub const fn fact(&self) -> CodexQuotaFact {
|
||||
self.fact
|
||||
@@ -209,7 +214,7 @@ fn parse_codex_quota_object(
|
||||
for limit in canonical_rate_limits(object)? {
|
||||
aggregate.observe_rate_limit(limit.rate_limit)?;
|
||||
}
|
||||
// credits / spend_control 只证明文档结构已识别,不参与账号级访问结论
|
||||
// 点数与支出控制不换算成套餐剩余百分比,访问结论由账号快照统一判断
|
||||
if let Some(spend_control) = object.get("spend_control") {
|
||||
aggregate.observe_metadata_object(spend_control, "reached")?;
|
||||
}
|
||||
@@ -345,6 +350,12 @@ pub(crate) fn parse_account_quota_snapshot(
|
||||
.then_with(|| left.key.cmp(&right.key))
|
||||
});
|
||||
let quota = authoritative_quota_state(&object, &windows, observed_at)?;
|
||||
let credits_usable = has_usable_credits(&object);
|
||||
let quota = if credits_usable {
|
||||
QuotaState::allowed(observed_at)
|
||||
} else {
|
||||
quota
|
||||
};
|
||||
let recovery = object
|
||||
.get(RECOVERY_FIELD)
|
||||
.map(|value| serde_json::from_value(value.clone()))
|
||||
@@ -359,6 +370,7 @@ pub(crate) fn parse_account_quota_snapshot(
|
||||
.and_then(Value::as_str)
|
||||
.map(str::to_owned),
|
||||
credits: object.get("credits").and_then(parse_credits_from_object),
|
||||
credits_usable,
|
||||
fact,
|
||||
quota,
|
||||
windows,
|
||||
@@ -374,7 +386,27 @@ fn quota_source_order(source: &str) -> u8 {
|
||||
}
|
||||
}
|
||||
|
||||
/// 默认 `codex` 桶是唯一可改变账号额度访问结论的 quota 响应字段
|
||||
fn has_usable_credits(usage: &Map<String, Value>) -> bool {
|
||||
let Some(credits) = usage.get("credits") else {
|
||||
return false;
|
||||
};
|
||||
let available = credits.get("has_credits").and_then(Value::as_bool) == Some(true)
|
||||
|| credits.get("unlimited").and_then(Value::as_bool) == Some(true);
|
||||
// 套餐耗尽不阻止点数支出,但不能绕过显式支出上限或工作区限制
|
||||
let spend_reached = usage
|
||||
.get("spend_control")
|
||||
.and_then(|spend| spend.get("reached"))
|
||||
.and_then(Value::as_bool)
|
||||
== Some(true);
|
||||
let other_limit_reached = usage
|
||||
.get("rate_limit_reached_type")
|
||||
.and_then(|reason| reason.get("type"))
|
||||
.and_then(Value::as_str)
|
||||
.is_some_and(|reason| reason != "rate_limit_reached");
|
||||
available && !spend_reached && !other_limit_reached
|
||||
}
|
||||
|
||||
/// 只有默认 `codex` 桶能提供账号级套餐访问结论,具名桶只保留展示事实
|
||||
fn authoritative_quota_state(
|
||||
usage: &Map<String, Value>,
|
||||
windows: &[CodexQuotaWindow],
|
||||
|
||||
@@ -199,18 +199,6 @@ impl OpenAiResponseObservationState {
|
||||
if metrics.upstream_engine_ms.is_some() {
|
||||
self.timings.upstream_engine_ms = metrics.upstream_engine_ms;
|
||||
}
|
||||
if metrics.upstream_engine_iapi_ttft_ms.is_some() {
|
||||
self.timings.upstream_engine_iapi_ttft_ms = metrics.upstream_engine_iapi_ttft_ms;
|
||||
}
|
||||
if metrics.upstream_engine_service_ttft_ms.is_some() {
|
||||
self.timings.upstream_engine_service_ttft_ms = metrics.upstream_engine_service_ttft_ms;
|
||||
}
|
||||
if metrics.upstream_engine_iapi_tbt_ms.is_some() {
|
||||
self.timings.upstream_engine_iapi_tbt_ms = metrics.upstream_engine_iapi_tbt_ms;
|
||||
}
|
||||
if metrics.upstream_engine_service_tbt_ms.is_some() {
|
||||
self.timings.upstream_engine_service_tbt_ms = metrics.upstream_engine_service_tbt_ms;
|
||||
}
|
||||
self.timings != previous
|
||||
}
|
||||
|
||||
|
||||
@@ -165,12 +165,6 @@ impl CodexBackendClient {
|
||||
context: CodexRequestContext<'_>,
|
||||
) -> CodexClientResult<HeaderMap> {
|
||||
let mut headers = self.response_headers(profile, request, context)?;
|
||||
if !headers.contains_key("x-responsesapi-include-timing-metrics") {
|
||||
headers.insert(
|
||||
HeaderName::from_static("x-responsesapi-include-timing-metrics"),
|
||||
HeaderValue::from_static("true"),
|
||||
);
|
||||
}
|
||||
headers.insert(
|
||||
HeaderName::from_static("openai-beta"),
|
||||
HeaderValue::from_static("responses_websockets=2026-02-06"),
|
||||
|
||||
@@ -255,10 +255,6 @@ pub(crate) fn response_duration_ms(value: &Value) -> Option<u64> {
|
||||
pub(crate) struct ResponseTimingMetrics {
|
||||
pub upstream_api_overhead_ms: Option<f64>,
|
||||
pub upstream_engine_ms: Option<f64>,
|
||||
pub upstream_engine_iapi_ttft_ms: Option<f64>,
|
||||
pub upstream_engine_service_ttft_ms: Option<f64>,
|
||||
pub upstream_engine_iapi_tbt_ms: Option<f64>,
|
||||
pub upstream_engine_service_tbt_ms: Option<f64>,
|
||||
}
|
||||
|
||||
impl ResponseTimingMetrics {
|
||||
@@ -287,7 +283,6 @@ impl ResponseTimingMetrics {
|
||||
return Self {
|
||||
upstream_api_overhead_ms: overhead,
|
||||
upstream_engine_ms: milliseconds(path, "engine_wall_ms"),
|
||||
..Self::default()
|
||||
};
|
||||
}
|
||||
// 直接返回的专项指标保留原口径,明确标为整轮累计的值不能归入单次请求
|
||||
@@ -303,16 +298,6 @@ impl ResponseTimingMetrics {
|
||||
"responses_duration_excl_engine_and_client_tool_time_ms",
|
||||
),
|
||||
upstream_engine_ms: milliseconds(metrics, "engine_service_total_ms"),
|
||||
upstream_engine_iapi_ttft_ms: milliseconds(metrics, "engine_iapi_ttft_total_ms"),
|
||||
upstream_engine_service_ttft_ms: milliseconds(metrics, "engine_service_ttft_total_ms"),
|
||||
upstream_engine_iapi_tbt_ms: milliseconds(
|
||||
metrics,
|
||||
"engine_iapi_tbt_across_engine_calls_ms",
|
||||
),
|
||||
upstream_engine_service_tbt_ms: milliseconds(
|
||||
metrics,
|
||||
"engine_service_tbt_across_engine_calls_ms",
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -323,18 +308,6 @@ impl ResponseTimingMetrics {
|
||||
if incoming.upstream_engine_ms.is_some() {
|
||||
self.upstream_engine_ms = incoming.upstream_engine_ms;
|
||||
}
|
||||
if incoming.upstream_engine_iapi_ttft_ms.is_some() {
|
||||
self.upstream_engine_iapi_ttft_ms = incoming.upstream_engine_iapi_ttft_ms;
|
||||
}
|
||||
if incoming.upstream_engine_service_ttft_ms.is_some() {
|
||||
self.upstream_engine_service_ttft_ms = incoming.upstream_engine_service_ttft_ms;
|
||||
}
|
||||
if incoming.upstream_engine_iapi_tbt_ms.is_some() {
|
||||
self.upstream_engine_iapi_tbt_ms = incoming.upstream_engine_iapi_tbt_ms;
|
||||
}
|
||||
if incoming.upstream_engine_service_tbt_ms.is_some() {
|
||||
self.upstream_engine_service_tbt_ms = incoming.upstream_engine_service_tbt_ms;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -4,14 +4,6 @@ use std::path::Path;
|
||||
|
||||
use provider_openai::config::{DEFAULT_STREAM_MAX_RETRIES, MAX_STREAM_MAX_RETRIES, OpenAiConfig};
|
||||
|
||||
#[test]
|
||||
fn openai_config_ignores_removed_yaml_identity_fields() {
|
||||
let config: OpenAiConfig = serde_json::from_value(serde_json::json!({
|
||||
"wire_profile": {"originator": "legacy-client", "codex_version": "invalid", "residency": "us"}
|
||||
})).unwrap();
|
||||
assert_eq!(config, OpenAiConfig::default());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn openai_config_keeps_residency_separate_from_client_identity() {
|
||||
use provider_openai::transport::profile::CodexResidency;
|
||||
@@ -98,10 +90,6 @@ fn openai_config_defaults_to_the_provider_owned_operating_values() {
|
||||
u32::try_from(DEFAULT_STREAM_MAX_RETRIES).expect("default retry budget fits u32"),
|
||||
)
|
||||
);
|
||||
assert_eq!(
|
||||
provider_openai::transport::profile::CodexWireProfile::default().user_agent(),
|
||||
"Codex Desktop/0.153.4 (Mac OS 15.7.1; arm64) unknown (Codex Desktop; 26.901.51231)"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -265,12 +265,12 @@ async fn authorize_url_matches_the_official_desktop_parameter_contract() {
|
||||
"response_type",
|
||||
"client_id",
|
||||
"redirect_uri",
|
||||
"scope",
|
||||
"code_challenge",
|
||||
"code_challenge_method",
|
||||
"state",
|
||||
"scope",
|
||||
"id_token_add_organizations",
|
||||
"codex_cli_simplified_flow",
|
||||
"state",
|
||||
"originator",
|
||||
"codex_app_version",
|
||||
"source_surface_stable_id",
|
||||
|
||||
@@ -0,0 +1,215 @@
|
||||
//! 点数独立可用性、耗尽恢复和过期查询隔离
|
||||
|
||||
use super::*;
|
||||
|
||||
fn exhausted_usage() -> serde_json::Value {
|
||||
json!({"rate_limit": {
|
||||
"allowed": false,
|
||||
"limit_reached": true,
|
||||
"primary_window": {"used_percent": 100, "reset_at": 1_900_000_000},
|
||||
"secondary_window": {"used_percent": 100, "reset_at": 1_900_604_800}
|
||||
}})
|
||||
}
|
||||
|
||||
async fn mount_usage(server: &MockServer, usage: serde_json::Value) {
|
||||
server.reset().await;
|
||||
Mock::given(method("GET"))
|
||||
.and(path("/api/codex/usage"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(usage))
|
||||
.mount(server)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn credits_allow_exhausted_subscription_without_bypassing_spend_restrictions() {
|
||||
for (metadata, allowed) in [
|
||||
(
|
||||
json!({"credits": {"has_credits": true, "unlimited": false}}),
|
||||
true,
|
||||
),
|
||||
(
|
||||
json!({"credits": {"has_credits": false, "unlimited": true}}),
|
||||
true,
|
||||
),
|
||||
(json!({"credits": {"has_credits": true}}), true),
|
||||
(json!({"credits": {"unlimited": true}}), true),
|
||||
(
|
||||
json!({"credits": {"has_credits": false, "unlimited": false, "balance": "500"}}),
|
||||
false,
|
||||
),
|
||||
(json!({"credits": null}), false),
|
||||
(
|
||||
json!({"credits": {"has_credits": true}, "spend_control": {"reached": true}}),
|
||||
false,
|
||||
),
|
||||
(
|
||||
json!({"credits": {"has_credits": true}, "rate_limit_reached_type": {"type": "spend_limit_reached"}}),
|
||||
false,
|
||||
),
|
||||
(
|
||||
json!({"credits": {"has_credits": true}, "rate_limit_reached_type": {"type": "rate_limit_reached"}}),
|
||||
true,
|
||||
),
|
||||
] {
|
||||
let store = Arc::new(MemoryAccountStore::default());
|
||||
create_account_with_enabled(&store, "acct_credits", false).await;
|
||||
let account = store.account("acct_credits").expect("account");
|
||||
persist_credential_state(&store, &account, CredentialState::Expired).await;
|
||||
let server = MockServer::start().await;
|
||||
let service = quota_service_with_base_url(&store, reqwest::Client::new(), server.uri());
|
||||
let mut usage = exhausted_usage();
|
||||
usage
|
||||
.as_object_mut()
|
||||
.expect("usage")
|
||||
.extend(metadata.as_object().expect("metadata").clone());
|
||||
mount_usage(&server, usage).await;
|
||||
|
||||
let snapshot = service
|
||||
.refresh_account(account.id())
|
||||
.await
|
||||
.expect("refresh credits");
|
||||
assert_eq!(
|
||||
snapshot.quota().access() == QuotaAccessState::Allowed,
|
||||
allowed,
|
||||
"{metadata}"
|
||||
);
|
||||
assert_eq!(snapshot.fact().remaining_percent(), Some(0));
|
||||
let current = store.account("acct_credits").expect("current account");
|
||||
assert_eq!(current.quota().is_exhausted(), !allowed, "{metadata}");
|
||||
assert!(!current.enabled());
|
||||
assert_eq!(current.credential_state(), CredentialState::Expired);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn credits_refresh_clears_old_recovery_and_locks_again_only_when_spent() {
|
||||
for periodic in [false, true] {
|
||||
let store = Arc::new(MemoryAccountStore::default());
|
||||
create_account(&store, "acct_credits_recovery").await;
|
||||
let account = store.account("acct_credits_recovery").expect("account");
|
||||
let server = MockServer::start().await;
|
||||
let service = quota_service_with_base_url(&store, reqwest::Client::new(), server.uri());
|
||||
mount_usage(&server, exhausted_usage()).await;
|
||||
service
|
||||
.refresh_account(account.id())
|
||||
.await
|
||||
.expect("seed exhaustion");
|
||||
assert!(
|
||||
store
|
||||
.quota_json("acct_credits_recovery")
|
||||
.expect("quota")
|
||||
.get("_quota_recovery")
|
||||
.is_some()
|
||||
);
|
||||
|
||||
let mut usage = exhausted_usage();
|
||||
usage["credits"] =
|
||||
json!({"has_credits": !periodic, "unlimited": periodic, "balance": "20"});
|
||||
mount_usage(&server, usage.clone()).await;
|
||||
if periodic {
|
||||
assert_eq!(
|
||||
service
|
||||
.synchronize()
|
||||
.await
|
||||
.expect("periodic recovery")
|
||||
.updated,
|
||||
1
|
||||
);
|
||||
} else {
|
||||
service
|
||||
.refresh_account(account.id())
|
||||
.await
|
||||
.expect("manual recovery");
|
||||
}
|
||||
let snapshot = service
|
||||
.read_account(account.id())
|
||||
.await
|
||||
.expect("persisted snapshot")
|
||||
.expect("snapshot");
|
||||
assert_eq!(snapshot.quota().access(), QuotaAccessState::Allowed);
|
||||
assert_eq!(snapshot.fact().remaining_percent(), Some(0));
|
||||
assert_eq!(
|
||||
snapshot.credits().expect("credits").balance.as_deref(),
|
||||
Some("20")
|
||||
);
|
||||
assert!(
|
||||
store
|
||||
.quota_json("acct_credits_recovery")
|
||||
.expect("quota")
|
||||
.get("_quota_recovery")
|
||||
.is_none()
|
||||
);
|
||||
|
||||
usage["credits"] = json!({"has_credits": false, "unlimited": false, "balance": "0"});
|
||||
mount_usage(&server, usage).await;
|
||||
let snapshot = service
|
||||
.refresh_account(account.id())
|
||||
.await
|
||||
.expect("spent credits");
|
||||
assert!(snapshot.quota().is_exhausted());
|
||||
assert!(
|
||||
store
|
||||
.account("acct_credits_recovery")
|
||||
.expect("account")
|
||||
.quota()
|
||||
.is_exhausted()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn credits_query_started_before_new_exhaustion_cannot_restore_access() {
|
||||
let store = Arc::new(MemoryAccountStore::default());
|
||||
create_account(&store, "acct_stale_credits").await;
|
||||
let account = store.account("acct_stale_credits").expect("account");
|
||||
let server = MockServer::start().await;
|
||||
let service = quota_service_with_base_url(&store, reqwest::Client::new(), server.uri());
|
||||
let mut usage = exhausted_usage();
|
||||
usage["credits"] = json!({"has_credits": true, "unlimited": false});
|
||||
Mock::given(method("GET"))
|
||||
.and(path("/api/codex/usage"))
|
||||
.respond_with(
|
||||
ResponseTemplate::new(200)
|
||||
.set_delay(Duration::from_millis(200))
|
||||
.set_body_json(usage),
|
||||
)
|
||||
.mount(&server)
|
||||
.await;
|
||||
let (result, ()) = tokio::join!(service.refresh_account(account.id()), async {
|
||||
tokio::time::timeout(Duration::from_secs(2), async {
|
||||
while server
|
||||
.received_requests()
|
||||
.await
|
||||
.expect("requests")
|
||||
.is_empty()
|
||||
{
|
||||
tokio::time::sleep(Duration::from_millis(1)).await;
|
||||
}
|
||||
})
|
||||
.await
|
||||
.expect("query started");
|
||||
persist_quota_state(
|
||||
&store,
|
||||
&account,
|
||||
QuotaState::exhausted(QuotaEvidence::UsageLimitReached, SystemTime::now(), None),
|
||||
)
|
||||
.await;
|
||||
});
|
||||
result.expect("old display observation can persist independently of newer access state");
|
||||
assert!(
|
||||
store
|
||||
.account("acct_stale_credits")
|
||||
.expect("account")
|
||||
.quota()
|
||||
.is_exhausted()
|
||||
);
|
||||
assert!(
|
||||
service
|
||||
.read_account(account.id())
|
||||
.await
|
||||
.expect("persisted quota")
|
||||
.expect("snapshot")
|
||||
.quota()
|
||||
.is_exhausted()
|
||||
);
|
||||
}
|
||||
@@ -1,10 +1,12 @@
|
||||
//! OpenAI 额度事实边界与展示快照回归
|
||||
|
||||
mod capacity_freeze;
|
||||
mod credits;
|
||||
mod initial_sync;
|
||||
mod recovery;
|
||||
mod refresh_timing;
|
||||
mod reset_credits;
|
||||
mod reset_recovery;
|
||||
mod scheduling;
|
||||
mod slots;
|
||||
mod snapshot;
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
//! 通过 quota 服务验证周期复核与 reset 宽限期,不依赖内部调度状态
|
||||
//! 通过 quota 服务验证耗尽复核节奏、恢复与正常窗口宽限期
|
||||
|
||||
use std::time::Duration;
|
||||
|
||||
@@ -17,8 +17,17 @@ fn reset_grace(reset: i64) -> SystemTime {
|
||||
SystemTime::UNIX_EPOCH + Duration::from_secs((reset + 120) as u64)
|
||||
}
|
||||
|
||||
async fn advance_clock(now: &mut SystemTime, seconds: u64) {
|
||||
let duration = Duration::from_secs(seconds);
|
||||
// 只在推进时暂停 Tokio 时钟,HTTP 请求期间保持真实计时
|
||||
tokio::time::pause();
|
||||
tokio::time::advance(duration).await;
|
||||
tokio::time::resume();
|
||||
*now += duration;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn reset_grace_bypasses_periodic_throttle_once_then_allows_the_next_window() {
|
||||
async fn exhausted_reset_bypasses_periodic_throttle_and_rechecks_each_pending_window() {
|
||||
let mut now = SystemTime::now();
|
||||
let store = Arc::new(MemoryAccountStore::default());
|
||||
create_account(&store, "acct_quota_timing").await;
|
||||
@@ -30,8 +39,8 @@ async fn reset_grace_bypasses_periodic_throttle_once_then_allows_the_next_window
|
||||
.duration_since(SystemTime::UNIX_EPOCH)
|
||||
.unwrap()
|
||||
.as_secs() as i64
|
||||
- 115;
|
||||
let week_reset = short_reset + 3;
|
||||
+ 5;
|
||||
let week_reset = short_reset + 30;
|
||||
let usage = |short_used, short_reset, week_used, week_reset| {
|
||||
json!({"rate_limit": {
|
||||
"allowed": false,
|
||||
@@ -55,20 +64,28 @@ async fn reset_grace_bypasses_periodic_throttle_once_then_allows_the_next_window
|
||||
let requests = server.received_requests().await.expect("requests").len();
|
||||
|
||||
assert!(
|
||||
now < reset_grace(short_reset),
|
||||
"fixture must precede grace deadline"
|
||||
now < SystemTime::UNIX_EPOCH + Duration::from_secs(short_reset as u64),
|
||||
"fixture must precede reset"
|
||||
);
|
||||
service
|
||||
.synchronize_at(now)
|
||||
.await
|
||||
.expect("before grace deadline");
|
||||
service.synchronize_at(now).await.expect("before reset");
|
||||
assert_eq!(
|
||||
server.received_requests().await.expect("requests").len(),
|
||||
requests
|
||||
);
|
||||
|
||||
mount_usage(&server, usage(0, short_reset + 18_000, 100, week_reset)).await;
|
||||
now = reset_grace(short_reset);
|
||||
for delay in [0, 9] {
|
||||
now = SystemTime::UNIX_EPOCH + Duration::from_secs(short_reset as u64 + delay);
|
||||
service.synchronize_at(now).await.expect("reset grace");
|
||||
assert!(
|
||||
server
|
||||
.received_requests()
|
||||
.await
|
||||
.expect("requests")
|
||||
.is_empty()
|
||||
);
|
||||
}
|
||||
now = SystemTime::UNIX_EPOCH + Duration::from_secs(short_reset as u64 + 10);
|
||||
let partial = service
|
||||
.synchronize_at(now)
|
||||
.await
|
||||
@@ -85,11 +102,11 @@ async fn reset_grace_bypasses_periodic_throttle_once_then_allows_the_next_window
|
||||
service
|
||||
.synchronize_at(now)
|
||||
.await
|
||||
.expect("next scan before weekly grace deadline");
|
||||
.expect("next scan before weekly reset");
|
||||
assert_eq!(server.received_requests().await.expect("requests").len(), 1);
|
||||
|
||||
// 周窗口的到期复核仍未恢复时,也不能每轮扫描重复请求
|
||||
now = reset_grace(week_reset);
|
||||
now = SystemTime::UNIX_EPOCH + Duration::from_secs(week_reset as u64 + 10);
|
||||
assert_eq!(
|
||||
service
|
||||
.synchronize_at(now)
|
||||
@@ -127,19 +144,139 @@ async fn periodic_checks_continue_when_reset_is_unknown_or_far_in_the_future() {
|
||||
.expect("seed exhaustion");
|
||||
mount_usage(&server, usage).await;
|
||||
|
||||
let mut now = SystemTime::now();
|
||||
assert_eq!(
|
||||
service
|
||||
.synchronize()
|
||||
.synchronize_at(now)
|
||||
.await
|
||||
.expect("periodic check")
|
||||
.exhausted,
|
||||
1
|
||||
);
|
||||
service.synchronize().await.expect("throttled repeat check");
|
||||
advance_clock(&mut now, 29 * 60).await;
|
||||
service
|
||||
.synchronize_at(now)
|
||||
.await
|
||||
.expect("throttled repeat check");
|
||||
assert_eq!(server.received_requests().await.expect("requests").len(), 1);
|
||||
advance_clock(&mut now, 60).await;
|
||||
service.synchronize_at(now).await.expect("periodic retry");
|
||||
assert_eq!(server.received_requests().await.expect("requests").len(), 2);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn exhausted_reset_refreshes_quickly_then_backs_off_and_recovers_from_usage() {
|
||||
let store = Arc::new(MemoryAccountStore::default());
|
||||
create_account(&store, "acct_reset_retry").await;
|
||||
let account = store.account("acct_reset_retry").expect("account");
|
||||
let server = MockServer::start().await;
|
||||
let service = quota_service_with_base_url(&store, reqwest::Client::new(), server.uri());
|
||||
let reset = Utc::now().timestamp();
|
||||
let mut now = SystemTime::UNIX_EPOCH + Duration::from_secs(reset as u64);
|
||||
let usage = |allowed, used, reset| {
|
||||
json!({"rate_limit": {
|
||||
"allowed": allowed,
|
||||
"primary_window": {"used_percent": used, "reset_at": reset}
|
||||
}})
|
||||
};
|
||||
mount_usage(&server, usage(false, 100, reset)).await;
|
||||
service
|
||||
.refresh_account(account.id())
|
||||
.await
|
||||
.expect("seed exhaustion");
|
||||
service
|
||||
.synchronize_at(now)
|
||||
.await
|
||||
.expect("reset grace without periodic history");
|
||||
advance_clock(&mut now, 9).await;
|
||||
service
|
||||
.synchronize_at(now)
|
||||
.await
|
||||
.expect("still in reset grace");
|
||||
assert_eq!(server.received_requests().await.expect("requests").len(), 1);
|
||||
advance_clock(&mut now, 1).await;
|
||||
service
|
||||
.synchronize_at(now)
|
||||
.await
|
||||
.expect("initial reset check");
|
||||
assert_eq!(server.received_requests().await.expect("requests").len(), 2);
|
||||
|
||||
// 本次拒绝发生在 reset 之后,滞后的 allowed 不能复用旧边界解封
|
||||
mount_usage(&server, usage(true, 100, reset)).await;
|
||||
for (elapsed, requests) in [
|
||||
(29, 0),
|
||||
(1, 1),
|
||||
(29, 1),
|
||||
(1, 2),
|
||||
(229, 3),
|
||||
(1, 3),
|
||||
(298, 3),
|
||||
(1, 4),
|
||||
] {
|
||||
advance_clock(&mut now, elapsed).await;
|
||||
service.synchronize_at(now).await.expect("scheduled check");
|
||||
assert_eq!(
|
||||
server.received_requests().await.expect("requests").len(),
|
||||
requests
|
||||
);
|
||||
assert!(
|
||||
store
|
||||
.account("acct_reset_retry")
|
||||
.expect("account")
|
||||
.quota()
|
||||
.is_exhausted()
|
||||
);
|
||||
}
|
||||
|
||||
// 查询失败也占用一次尝试,避免网络故障时取消节流
|
||||
server.reset().await;
|
||||
Mock::given(method("GET"))
|
||||
.and(path("/api/codex/usage"))
|
||||
.respond_with(ResponseTemplate::new(429))
|
||||
.mount(&server)
|
||||
.await;
|
||||
advance_clock(&mut now, 300).await;
|
||||
assert_eq!(
|
||||
service
|
||||
.synchronize_at(now)
|
||||
.await
|
||||
.expect("failed check")
|
||||
.transient,
|
||||
1
|
||||
);
|
||||
advance_clock(&mut now, 299).await;
|
||||
service
|
||||
.synchronize_at(now)
|
||||
.await
|
||||
.expect("throttled failed check");
|
||||
assert_eq!(server.received_requests().await.expect("requests").len(), 1);
|
||||
|
||||
mount_usage(&server, usage(true, 0, reset + 18_000)).await;
|
||||
advance_clock(&mut now, 1).await;
|
||||
assert_eq!(
|
||||
service
|
||||
.synchronize_at(now)
|
||||
.await
|
||||
.expect("recovery check")
|
||||
.updated,
|
||||
1
|
||||
);
|
||||
assert!(
|
||||
!store
|
||||
.account("acct_reset_retry")
|
||||
.expect("account")
|
||||
.quota()
|
||||
.is_exhausted()
|
||||
);
|
||||
advance_clock(&mut now, 300).await;
|
||||
service
|
||||
.synchronize_at(now)
|
||||
.await
|
||||
.expect("recovered account leaves retry schedule");
|
||||
assert_eq!(server.received_requests().await.expect("requests").len(), 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn allowed_account_with_expired_window_synchronizes_at_reset_grace() {
|
||||
let mut now = SystemTime::now();
|
||||
|
||||
@@ -0,0 +1,160 @@
|
||||
//! 到期后的明确放行恢复,保留到期前、多窗口与再次拒绝的保护
|
||||
|
||||
use gateway_core::account::{OpaqueProviderData, QuotaObservation};
|
||||
|
||||
use super::*;
|
||||
|
||||
fn usage(now: i64, resets: &[Option<i64>], allowed: Option<bool>) -> serde_json::Value {
|
||||
let mut value = json!({"rate_limit": {}});
|
||||
if let Some(allowed) = allowed {
|
||||
value["rate_limit"]["allowed"] = json!(allowed);
|
||||
}
|
||||
for ((role, seconds), reset) in [("primary_window", 18_000), ("secondary_window", 604_800)]
|
||||
.into_iter()
|
||||
.zip(resets)
|
||||
{
|
||||
let mut window = json!({"used_percent": 100, "limit_window_seconds": seconds});
|
||||
if let Some(offset) = reset {
|
||||
window["reset_at"] = json!(now + offset);
|
||||
}
|
||||
value["rate_limit"][role] = window;
|
||||
}
|
||||
value
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn allowed_refresh_recovers_full_windows_only_after_the_confirmed_resets() {
|
||||
for (name, resets, failure_offset, allowed, recovered) in [
|
||||
("before_reset", vec![Some(3600)], -120, Some(true), false),
|
||||
("after_reset", vec![Some(-60)], -120, Some(true), true),
|
||||
("still_denied", vec![Some(-60)], -120, Some(false), false),
|
||||
("unknown_access", vec![Some(-60)], -120, None, false),
|
||||
("unknown_reset", vec![None], -120, Some(true), false),
|
||||
("new_failure", vec![Some(-60)], -30, Some(true), false),
|
||||
(
|
||||
"weekly_not_due",
|
||||
vec![Some(-60), Some(3600)],
|
||||
-120,
|
||||
Some(true),
|
||||
false,
|
||||
),
|
||||
(
|
||||
"both_due",
|
||||
vec![Some(-90), Some(-60)],
|
||||
-120,
|
||||
Some(true),
|
||||
true,
|
||||
),
|
||||
(
|
||||
"weekly_unknown",
|
||||
vec![Some(-60), None],
|
||||
-120,
|
||||
Some(true),
|
||||
false,
|
||||
),
|
||||
] {
|
||||
for worker in [false, true] {
|
||||
let store = Arc::new(MemoryAccountStore::default());
|
||||
let account_id = format!("acct_reset_{name}_{worker}");
|
||||
create_account(&store, &account_id).await;
|
||||
let account = store.account(&account_id).expect("account");
|
||||
let now = Utc::now().timestamp();
|
||||
let observed_at =
|
||||
SystemTime::UNIX_EPOCH + Duration::from_secs((now + failure_offset) as u64);
|
||||
let reset_at = resets
|
||||
.iter()
|
||||
.filter_map(|reset| *reset)
|
||||
.min()
|
||||
.map(|offset| SystemTime::UNIX_EPOCH + Duration::from_secs((now + offset) as u64));
|
||||
let old_usage = usage(now, &resets, Some(false));
|
||||
store
|
||||
.compare_and_swap_quota(QuotaObservation {
|
||||
plan_type: None,
|
||||
account_id: account.id().clone(),
|
||||
expected_revision: account.revision(),
|
||||
quota: OpaqueProviderData::new(old_usage.as_object().expect("usage").clone()),
|
||||
observed_at,
|
||||
state: QuotaState::exhausted(
|
||||
QuotaEvidence::UsageLimitReached,
|
||||
observed_at,
|
||||
reset_at,
|
||||
),
|
||||
})
|
||||
.await
|
||||
.expect("persist confirmed exhaustion before refresh");
|
||||
let server = MockServer::start().await;
|
||||
Mock::given(method("GET"))
|
||||
.and(path("/api/codex/usage"))
|
||||
.respond_with(
|
||||
ResponseTemplate::new(200).set_body_json(usage(now, &resets, allowed)),
|
||||
)
|
||||
.mount(&server)
|
||||
.await;
|
||||
let service = quota_service_with_base_url(&store, reqwest::Client::new(), server.uri());
|
||||
if worker {
|
||||
let summary = service.synchronize().await.expect("worker refresh");
|
||||
assert_eq!(summary.updated, u64::from(recovered), "{name}");
|
||||
} else {
|
||||
service
|
||||
.refresh_account(account.id())
|
||||
.await
|
||||
.expect("manual refresh");
|
||||
}
|
||||
let refreshed = service
|
||||
.read_account(account.id())
|
||||
.await
|
||||
.expect("read")
|
||||
.expect("snapshot");
|
||||
assert_eq!(
|
||||
refreshed.quota().is_exhausted(),
|
||||
!recovered,
|
||||
"{name}, worker={worker}"
|
||||
);
|
||||
assert!(
|
||||
refreshed
|
||||
.windows()
|
||||
.iter()
|
||||
.all(|window| window.used_percent() == Some(100.0))
|
||||
);
|
||||
assert_eq!(
|
||||
store
|
||||
.account(&account_id)
|
||||
.expect("account")
|
||||
.quota()
|
||||
.is_exhausted(),
|
||||
!recovered
|
||||
);
|
||||
assert_eq!(
|
||||
store
|
||||
.quota_json(&account_id)
|
||||
.expect("quota")
|
||||
.get("_quota_recovery")
|
||||
.is_none(),
|
||||
recovered
|
||||
);
|
||||
|
||||
if recovered {
|
||||
service
|
||||
.refresh_account(account.id())
|
||||
.await
|
||||
.expect("allowed remains allowed");
|
||||
assert!(
|
||||
!store
|
||||
.account(&account_id)
|
||||
.expect("account")
|
||||
.quota()
|
||||
.is_exhausted()
|
||||
);
|
||||
// 新拒绝不能沿用刚刚解除时的旧 reset,即使查询仍然 allowed=true
|
||||
persist_quota_state(&store, &account, exhausted_quota(reset_at)).await;
|
||||
for _ in 0..2 {
|
||||
let snapshot = service
|
||||
.refresh_account(account.id())
|
||||
.await
|
||||
.expect("refresh after new refusal");
|
||||
assert!(snapshot.quota().is_exhausted(), "{name}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -8421,6 +8421,82 @@ async fn official_usage_limit_failure_persists_fact_without_fabricating_usage()
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn usage_limit_failure_recovers_with_credits_without_replaying_the_failed_request() {
|
||||
let store = Arc::new(MemoryAccountStore::default());
|
||||
let account_id = "acct_provider_contract";
|
||||
create_account(&store, account_id).await;
|
||||
let server = MockServer::start().await;
|
||||
Mock::given(method("GET"))
|
||||
.and(path("/api/codex/usage"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
|
||||
"rate_limit": {
|
||||
"allowed": false,
|
||||
"limit_reached": true,
|
||||
"primary_window": {"used_percent": 100, "reset_at": 1_900_000_000}
|
||||
},
|
||||
"credits": {"has_credits": true, "unlimited": false, "balance": "20"}
|
||||
})))
|
||||
.expect(1)
|
||||
.mount(&server)
|
||||
.await;
|
||||
Mock::given(method("POST"))
|
||||
.and(path("/codex/responses"))
|
||||
.respond_with(ResponseTemplate::new(429).set_body_json(json!({
|
||||
"error": {"type": "usage_limit_reached", "message": "usage limit reached", "resets_at": 1_900_000_000}
|
||||
})))
|
||||
.expect(1)
|
||||
.mount(&server)
|
||||
.await;
|
||||
let mut stream = provider_with_base_url(&store, server.uri())
|
||||
.execute(
|
||||
planned_request("openai", http_generate_operation()),
|
||||
context("req_usage_limit_credits", CancellationToken::new()),
|
||||
)
|
||||
.await
|
||||
.expect("provider stream");
|
||||
let failure = loop {
|
||||
match stream.next().await {
|
||||
Some(Ok(_)) => {}
|
||||
Some(Err(error)) => break error,
|
||||
None => panic!("quota failure must reach the caller"),
|
||||
}
|
||||
};
|
||||
assert_eq!(failure.kind(), ProviderErrorKind::QuotaExhausted);
|
||||
assert!(
|
||||
store
|
||||
.account(account_id)
|
||||
.expect("account")
|
||||
.quota()
|
||||
.is_exhausted()
|
||||
);
|
||||
|
||||
timeout(Duration::from_secs(5), async {
|
||||
while store
|
||||
.account(account_id)
|
||||
.expect("account")
|
||||
.quota()
|
||||
.is_exhausted()
|
||||
{
|
||||
tokio::time::sleep(Duration::from_millis(10)).await;
|
||||
}
|
||||
})
|
||||
.await
|
||||
.expect("credits restore access after background refresh");
|
||||
assert!(stream.next().await.is_none());
|
||||
let account = store.account(account_id).expect("recovered account");
|
||||
assert_eq!(account.quota().access(), QuotaAccessState::Allowed);
|
||||
assert_eq!(account.credential_state(), CredentialState::Ready);
|
||||
let quota = store.quota_json(account_id).expect("persisted credits");
|
||||
assert_eq!(quota.pointer("/credits/has_credits"), Some(&json!(true)));
|
||||
assert_eq!(
|
||||
quota.pointer("/rate_limits_by_limit_id/codex/primary_window/used_percent"),
|
||||
Some(&json!(100))
|
||||
);
|
||||
assert!(quota.get("_quota_recovery").is_none());
|
||||
server.verify().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn capacity_http_and_websocket_opening_rejections_use_bounded_business_retry() {
|
||||
for use_websocket in [false, true] {
|
||||
|
||||
@@ -66,9 +66,10 @@ async fn observe_timing_events(websocket: bool, events: Vec<Value>) -> ProviderR
|
||||
let mut ws = crate::transport::accept_codex_test_websocket_with(
|
||||
socket,
|
||||
|request: &tokio_tungstenite::tungstenite::handshake::server::Request, response| {
|
||||
assert_eq!(
|
||||
request.headers()["x-responsesapi-include-timing-metrics"],
|
||||
"true"
|
||||
assert!(
|
||||
!request
|
||||
.headers()
|
||||
.contains_key("x-responsesapi-include-timing-metrics")
|
||||
);
|
||||
response.headers_mut().insert(
|
||||
"sec-websocket-extensions",
|
||||
@@ -125,19 +126,11 @@ async fn official_response_duration_is_observed_independently_of_the_request_clo
|
||||
let metrics = json!({"type":"responsesapi.websocket_timing","timing_metrics":{
|
||||
"responses_duration_excl_engine_and_client_tool_time_ms":120.25,
|
||||
"engine_service_total_ms":6400.0,
|
||||
"engine_iapi_ttft_total_ms":650.5,
|
||||
"engine_service_ttft_total_ms":720.25,
|
||||
"engine_iapi_tbt_across_engine_calls_ms":18.45,
|
||||
"engine_service_tbt_across_engine_calls_ms":20.12
|
||||
}});
|
||||
let wrong_id = json!({"type":"responsesapi.websocket_timing","response_id":"resp_other","timing_metrics":{"engine_service_total_ms":999.0}});
|
||||
let invalid = json!({"type":"responsesapi.websocket_timing","timing_metrics":{
|
||||
"responses_duration_excl_engine_and_client_tool_time_ms":null,
|
||||
"engine_service_total_ms":"100",
|
||||
"engine_iapi_ttft_total_ms":-1,
|
||||
"engine_service_ttft_total_ms":1e99,
|
||||
"engine_iapi_tbt_across_engine_calls_ms":{},
|
||||
"engine_service_tbt_across_engine_calls_ms":[]
|
||||
}});
|
||||
let stale = json!({"type":"responsesapi.websocket_timing","timing_metrics":{"engine_service_total_ms":999.0}});
|
||||
let events = [stale, created, metrics, wrong_id, invalid, completed];
|
||||
@@ -145,10 +138,6 @@ async fn official_response_duration_is_observed_independently_of_the_request_clo
|
||||
assert_eq!(timings.upstream_response_ms, Some(7_000));
|
||||
assert_eq!(timings.upstream_api_overhead_ms, Some(120.25));
|
||||
assert_eq!(timings.upstream_engine_ms, Some(6400.0));
|
||||
assert_eq!(timings.upstream_engine_iapi_ttft_ms, Some(650.5));
|
||||
assert_eq!(timings.upstream_engine_service_ttft_ms, Some(720.25));
|
||||
assert_eq!(timings.upstream_engine_iapi_tbt_ms, Some(18.45));
|
||||
assert_eq!(timings.upstream_engine_service_tbt_ms, Some(20.12));
|
||||
assert_eq!(timings.first_token_ms, None);
|
||||
assert!(timings.first_event_ms.is_some_and(|value| value >= 20_000));
|
||||
}
|
||||
@@ -197,7 +186,6 @@ async fn response_critical_path_is_observed_without_logical_turn_totals_on_both_
|
||||
let mut metrics = response_critical_path_event();
|
||||
// 同帧即使携带整轮专项值,也不能与响应级 critical path 混算
|
||||
metrics["timing_metrics"]["engine_service_total_ms"] = json!(25_000);
|
||||
metrics["timing_metrics"]["engine_iapi_ttft_total_ms"] = json!(500);
|
||||
let mut events = critical_path_response_events(metrics);
|
||||
let mut wrong_id = response_critical_path_event();
|
||||
wrong_id["timing_metrics"]["response_id"] = json!("resp_other");
|
||||
@@ -208,10 +196,6 @@ async fn response_critical_path_is_observed_without_logical_turn_totals_on_both_
|
||||
assert!((timings.upstream_api_overhead_ms.unwrap() - 81.409594).abs() < 0.000001);
|
||||
assert_eq!(timings.upstream_response_ms, Some(7000));
|
||||
assert_eq!(timings.first_token_ms, None);
|
||||
assert_eq!(timings.upstream_engine_iapi_ttft_ms, None);
|
||||
assert_eq!(timings.upstream_engine_service_ttft_ms, None);
|
||||
assert_eq!(timings.upstream_engine_iapi_tbt_ms, None);
|
||||
assert_eq!(timings.upstream_engine_service_tbt_ms, None);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -252,12 +236,10 @@ async fn critical_path_rejects_mismatched_identity_and_incomplete_or_cumulative_
|
||||
"timing_scope":"logical_turn",
|
||||
"engine_service_total_ms":25_000,
|
||||
"responses_duration_excl_engine_and_client_tool_time_ms":500,
|
||||
"engine_iapi_ttft_total_ms":200
|
||||
}});
|
||||
let timings = observe_timing_events(false, critical_path_response_events(metrics)).await;
|
||||
assert_eq!(timings.upstream_engine_ms, None);
|
||||
assert_eq!(timings.upstream_api_overhead_ms, None);
|
||||
assert_eq!(timings.upstream_engine_iapi_ttft_ms, None);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -288,10 +270,6 @@ async fn timing_frames_outside_the_active_response_do_not_enter_request_metrics(
|
||||
let timing = json!({"type":"responsesapi.websocket_timing","timing_metrics":{
|
||||
"responses_duration_excl_engine_and_client_tool_time_ms":0,
|
||||
"engine_service_total_ms":450,
|
||||
"engine_iapi_ttft_total_ms":211,
|
||||
"engine_service_ttft_total_ms":233,
|
||||
"engine_iapi_tbt_across_engine_calls_ms":2.450638,
|
||||
"engine_service_tbt_across_engine_calls_ms":5.267279
|
||||
}});
|
||||
let created = json!({"type":"response.created","response":{"id":"resp_boundary"}});
|
||||
let completed = json!({"type":"response.completed","response":{"id":"resp_boundary","status":"completed","output":[],"usage":{"input_tokens":1,"output_tokens":1,"total_tokens":2}}});
|
||||
@@ -311,10 +289,6 @@ async fn timing_frames_outside_the_active_response_do_not_enter_request_metrics(
|
||||
let timings = finish_timings(&mut stream, ProviderResponseTimings::default()).await;
|
||||
assert_eq!(timings.upstream_api_overhead_ms, None);
|
||||
assert_eq!(timings.upstream_engine_ms, None);
|
||||
assert_eq!(timings.upstream_engine_iapi_ttft_ms, None);
|
||||
assert_eq!(timings.upstream_engine_service_ttft_ms, None);
|
||||
assert_eq!(timings.upstream_engine_iapi_tbt_ms, None);
|
||||
assert_eq!(timings.upstream_engine_service_tbt_ms, None);
|
||||
assert_eq!(timings.first_token_ms, None);
|
||||
server.await.unwrap();
|
||||
}
|
||||
@@ -395,9 +369,10 @@ async fn structural_frames_start_ttft_before_content_on_http_and_websocket() {
|
||||
socket,
|
||||
|request: &tokio_tungstenite::tungstenite::handshake::server::Request,
|
||||
response| {
|
||||
assert_eq!(
|
||||
request.headers()["x-responsesapi-include-timing-metrics"],
|
||||
"true"
|
||||
assert!(
|
||||
!request
|
||||
.headers()
|
||||
.contains_key("x-responsesapi-include-timing-metrics")
|
||||
);
|
||||
response.headers_mut().insert(
|
||||
"sec-websocket-extensions",
|
||||
|
||||
@@ -430,7 +430,7 @@ impl ProviderAccountStore for MemoryAccountStore {
|
||||
|
||||
async fn compare_and_swap_quota(
|
||||
&self,
|
||||
observation: QuotaObservation,
|
||||
mut observation: QuotaObservation,
|
||||
) -> Result<QuotaWriteOutcome, StoreError> {
|
||||
let mut accounts = self.accounts.lock().expect("account store lock");
|
||||
let stored = accounts
|
||||
@@ -446,7 +446,18 @@ impl ProviderAccountStore for MemoryAccountStore {
|
||||
{
|
||||
return Ok(QuotaWriteOutcome::Conflict);
|
||||
}
|
||||
let quota = observation.state;
|
||||
// 与 PostgreSQL 一致,展示快照与访问事实分别按各自观察时间隔离
|
||||
let current = stored.account.quota();
|
||||
let quota = if observation
|
||||
.state
|
||||
.observed_at()
|
||||
.is_some_and(|observed| current.observed_at().is_none_or(|time| time <= observed))
|
||||
{
|
||||
observation.state
|
||||
} else {
|
||||
current
|
||||
};
|
||||
observation.state = quota;
|
||||
let mut replacement = AccountRebuild::preserving(&stored.account).with_quota(quota);
|
||||
if let Some(plan_type) = &observation.plan_type {
|
||||
replacement.profile = Some((
|
||||
|
||||
@@ -119,24 +119,23 @@ fn raw_identity_rejects_injection_conflicting_headers_and_unknown_modes() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn old_configuration_and_old_frozen_profile_keep_their_existing_identity() {
|
||||
fn preset_identity_round_trip_preserves_request_headers_and_official_preview() {
|
||||
let state = CodexWireProfileState::new(CodexWireProfile::default());
|
||||
let legacy = ClientProfileSelection::default();
|
||||
let resolved = legacy.resolve(&state).unwrap();
|
||||
let configuration = ClientProfileSelection::default().document().unwrap();
|
||||
let resolved = RequestProfileSelection::parse(&configuration)
|
||||
.unwrap()
|
||||
.resolve(&state)
|
||||
.unwrap();
|
||||
let restored: CodexWireProfile =
|
||||
serde_json::from_value(serde_json::to_value(&resolved).unwrap()).unwrap();
|
||||
|
||||
assert_eq!(
|
||||
RequestProfileSelection::parse(&legacy.document().unwrap())
|
||||
.unwrap()
|
||||
.resolve(&state)
|
||||
.unwrap(),
|
||||
resolved
|
||||
build_codex_model_headers(&restored, "Bearer fixture", None).unwrap(),
|
||||
build_codex_model_headers(&resolved, "Bearer fixture", None).unwrap()
|
||||
);
|
||||
let old_document = serde_json::to_value(&resolved).unwrap();
|
||||
assert!(old_document.get("exact_user_agent").is_none());
|
||||
let restored: CodexWireProfile = serde_json::from_value(old_document).unwrap();
|
||||
assert_eq!(restored.user_agent(), resolved.user_agent());
|
||||
assert_eq!(
|
||||
state
|
||||
.preview_selection(&legacy.document().unwrap())
|
||||
.preview_selection(&configuration)
|
||||
.unwrap()
|
||||
.expose_to_provider()["versionSource"],
|
||||
"official"
|
||||
|
||||
@@ -529,98 +529,30 @@ fn encoder_should_restore_conversation_fallback_after_body_and_context_values()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn encoder_should_accept_metadata_session_and_thread_ids_but_ignore_legacy_context_aliases() {
|
||||
fn encoder_should_extract_websocket_context_from_client_metadata() {
|
||||
let metadata = json!({
|
||||
"session_id": "metadata-session",
|
||||
"thread_id": "metadata-thread",
|
||||
"x-codex-turn-state": "metadata-turn-state",
|
||||
"turn_id": "metadata-turn-id",
|
||||
"future_context": {"keep": true}
|
||||
});
|
||||
let request = request(Map::from_iter([
|
||||
("model".to_owned(), json!("client-model")),
|
||||
("input".to_owned(), json!("prompt")),
|
||||
("turn_state".to_owned(), json!("legacy-turn-state")),
|
||||
(
|
||||
"x-codex-turn-state".to_owned(),
|
||||
json!("legacy-header-turn-state"),
|
||||
),
|
||||
("turn_metadata".to_owned(), json!("legacy-turn-metadata")),
|
||||
("beta_features".to_owned(), json!("legacy-beta")),
|
||||
("include_timing_metrics".to_owned(), json!("legacy-timing")),
|
||||
("codex_window_id".to_owned(), json!("legacy-window")),
|
||||
(
|
||||
"x-codex-window-id".to_owned(),
|
||||
json!("legacy-header-window"),
|
||||
),
|
||||
("parent_thread_id".to_owned(), json!("legacy-parent")),
|
||||
(
|
||||
"x-codex-parent-thread-id".to_owned(),
|
||||
json!("legacy-header-parent"),
|
||||
),
|
||||
("conversationId".to_owned(), json!("legacy-conversation")),
|
||||
("sessionId".to_owned(), json!("legacy-session")),
|
||||
("threadId".to_owned(), json!("legacy-thread")),
|
||||
("client_request_id".to_owned(), json!("legacy-request")),
|
||||
("clientRequestId".to_owned(), json!("legacy-camel-request")),
|
||||
("turnId".to_owned(), json!("legacy-turn")),
|
||||
("x-codex-turn-id".to_owned(), json!("legacy-header-turn")),
|
||||
(
|
||||
"client_metadata".to_owned(),
|
||||
json!({
|
||||
"turnState": "metadata-turn-state",
|
||||
"turnMetadata": "metadata-turn-metadata",
|
||||
"conversation_id": "metadata-conversation",
|
||||
"session_id": "metadata-session",
|
||||
"thread_id": "metadata-thread",
|
||||
"x-codex-window-id": "metadata-window"
|
||||
}),
|
||||
),
|
||||
("client_metadata".to_owned(), metadata.clone()),
|
||||
]));
|
||||
|
||||
let encoded = encode_generate_request(&request, "gpt-test", None).expect("encode");
|
||||
|
||||
assert_eq!(
|
||||
(
|
||||
encoded.turn_state,
|
||||
encoded.turn_metadata,
|
||||
encoded.beta_features,
|
||||
encoded.include_timing_metrics,
|
||||
encoded.codex_window_id,
|
||||
encoded.parent_thread_id,
|
||||
encoded.client_conversation_id,
|
||||
encoded.client_session_id,
|
||||
encoded.client_thread_id,
|
||||
encoded.client_request_id,
|
||||
encoded.client_turn_id,
|
||||
),
|
||||
(
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
None,
|
||||
Some("metadata-session".to_owned()),
|
||||
Some("metadata-thread".to_owned()),
|
||||
None,
|
||||
None,
|
||||
)
|
||||
encoded.client_session_id.as_deref(),
|
||||
Some("metadata-session")
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn encoder_should_extract_official_websocket_context_projection() {
|
||||
let request = request(Map::from_iter([
|
||||
("model".to_owned(), json!("client-model")),
|
||||
("input".to_owned(), json!("prompt")),
|
||||
(
|
||||
"client_metadata".to_owned(),
|
||||
json!({
|
||||
"x-codex-turn-state": "metadata-turn-state",
|
||||
"turn_id": "metadata-turn-id"
|
||||
}),
|
||||
),
|
||||
]));
|
||||
|
||||
let encoded = encode_generate_request(&request, "gpt-test", None).expect("encode");
|
||||
|
||||
assert_eq!(encoded.client_thread_id.as_deref(), Some("metadata-thread"));
|
||||
assert_eq!(encoded.turn_state.as_deref(), Some("metadata-turn-state"));
|
||||
assert_eq!(encoded.client_turn_id.as_deref(), Some("metadata-turn-id"));
|
||||
assert_eq!(encoded.body().get("client_metadata"), Some(&metadata));
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -110,8 +110,7 @@ async fn official_websocket_wire_fingerprint_should_remain_aligned() {
|
||||
opening.lines().next(),
|
||||
Some("GET /backend-api/codex/responses HTTP/1.1")
|
||||
);
|
||||
// 固定身份字段沿用官方画像,专项计时头按官方客户端在 openai-beta 后插入
|
||||
// HeaderMap 序列化会随新增头调整迭代顺序,锁定启用计时后的完整 opening
|
||||
// 固定身份字段沿用官方画像,锁定默认不请求专项计时的完整 opening
|
||||
assert_eq!(
|
||||
read_header_names(&opening),
|
||||
vec![
|
||||
@@ -124,7 +123,7 @@ async fn official_websocket_wire_fingerprint_should_remain_aligned() {
|
||||
"authorization",
|
||||
"user-agent",
|
||||
"originator",
|
||||
"x-responsesapi-include-timing-metrics",
|
||||
"openai-beta",
|
||||
"version",
|
||||
"x-codex-beta-features",
|
||||
"x-client-request-id",
|
||||
@@ -133,13 +132,12 @@ async fn official_websocket_wire_fingerprint_should_remain_aligned() {
|
||||
"x-codex-window-id",
|
||||
"x-codex-turn-metadata",
|
||||
"x-codex-routing-hint",
|
||||
"openai-beta",
|
||||
"sec-websocket-extensions",
|
||||
]
|
||||
);
|
||||
assert_eq!(
|
||||
read_header_value(&opening, "x-responsesapi-include-timing-metrics"),
|
||||
Some("true"),
|
||||
None,
|
||||
);
|
||||
assert_eq!(
|
||||
read_header_value(&opening, "sec-websocket-extensions"),
|
||||
|
||||
@@ -365,8 +365,8 @@ async fn websocket_execute_response_create_request_should_collect_completed_sse(
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn websocket_exchange_should_accept_upstream_frame_above_removed_private_limit() {
|
||||
const REMOVED_FRAME_LIMIT_BYTES: usize = 16 * 1024 * 1024;
|
||||
async fn websocket_exchange_should_deliver_large_upstream_frames() {
|
||||
const LARGE_DELTA_BYTES: usize = 17 * 1024 * 1024;
|
||||
|
||||
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
|
||||
let addr = listener.local_addr().unwrap();
|
||||
@@ -376,14 +376,13 @@ async fn websocket_exchange_should_accept_upstream_frame_above_removed_private_l
|
||||
let mut websocket =
|
||||
accept_codex_test_websocket_with(stream, |_request, _response| {}).await;
|
||||
let _message = websocket.next().await.unwrap().unwrap();
|
||||
let oversized_event = json!({
|
||||
let large_event = json!({
|
||||
"type": "response.output_text.delta",
|
||||
"delta": "x".repeat(REMOVED_FRAME_LIMIT_BYTES + 1),
|
||||
"delta": "x".repeat(LARGE_DELTA_BYTES),
|
||||
})
|
||||
.to_string();
|
||||
assert!(oversized_event.len() > REMOVED_FRAME_LIMIT_BYTES);
|
||||
websocket
|
||||
.send(Message::Text(oversized_event.into()))
|
||||
.send(Message::Text(large_event.into()))
|
||||
.await
|
||||
.unwrap();
|
||||
websocket
|
||||
@@ -391,7 +390,7 @@ async fn websocket_exchange_should_accept_upstream_frame_above_removed_private_l
|
||||
json!({
|
||||
"type": "response.completed",
|
||||
"response": {
|
||||
"id": "resp_after_oversized_frame",
|
||||
"id": "resp_after_large_frame",
|
||||
"object": "response",
|
||||
"output": [],
|
||||
"usage": {
|
||||
@@ -421,11 +420,11 @@ async fn websocket_exchange_should_accept_upstream_frame_above_removed_private_l
|
||||
|
||||
let response = execute_response_create_request(&prepared)
|
||||
.await
|
||||
.expect("upstream frame above the former private limit should remain forwardable");
|
||||
.expect("large upstream frame should remain forwardable");
|
||||
server.await.unwrap();
|
||||
|
||||
assert!(response.body.len() > REMOVED_FRAME_LIMIT_BYTES);
|
||||
assert!(response.body.contains("resp_after_oversized_frame"));
|
||||
assert!(response.body.len() > LARGE_DELTA_BYTES);
|
||||
assert!(response.body.contains("resp_after_large_frame"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
||||
@@ -228,39 +228,16 @@ async fn quota_refresh_persists_subscription_without_user_profile_and_reads_it_f
|
||||
assert!(document.get("userId").is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn billing_percent_does_not_rewrite_quota_access_fact() {
|
||||
let (store, repository) =
|
||||
repository_with_accounts(&[("still-exhausted", "still-exhausted")]).await;
|
||||
let id = account_id("still-exhausted");
|
||||
store
|
||||
.apply_quota_access(QuotaAccessChange {
|
||||
account_id: id.clone(),
|
||||
expected_revision: CredentialRevision::new(1).expect("revision"),
|
||||
state: QuotaState::exhausted(QuotaEvidence::ProviderDenied, SystemTime::now(), None),
|
||||
})
|
||||
.await
|
||||
.expect("mark quota exhausted");
|
||||
|
||||
crate::support::grok_quota_service(
|
||||
repository,
|
||||
QueueBillingTransport::success(br#"{"config":{"creditUsagePercent":100}}"#),
|
||||
)
|
||||
.refresh_account(&id)
|
||||
.await
|
||||
.expect("refresh exhausted quota");
|
||||
|
||||
assert_eq!(
|
||||
store.account(&id).expect("account").quota().access(),
|
||||
gateway_core::account::QuotaAccessState::Exhausted
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn billing_access_and_scheduling_use_current_quota_fields() {
|
||||
use gateway_core::account::QuotaAccessState::{Allowed, Exhausted};
|
||||
|
||||
for (config, expected_percent, expected_access) in [
|
||||
(
|
||||
serde_json::json!({"creditUsagePercent": 0}),
|
||||
Some(0.0),
|
||||
Allowed,
|
||||
),
|
||||
(
|
||||
serde_json::json!({"creditUsagePercent": 100}),
|
||||
Some(100.0),
|
||||
@@ -306,6 +283,10 @@ async fn billing_access_and_scheduling_use_current_quota_fields() {
|
||||
let account = store.account(&id).expect("refreshed account");
|
||||
|
||||
assert_eq!(snapshot.billing().used_percent(), expected_percent);
|
||||
assert!(
|
||||
snapshot.billing().has_authoritative_quota(),
|
||||
"config: {config}"
|
||||
);
|
||||
assert_eq!(
|
||||
account.quota().access(),
|
||||
expected_access,
|
||||
@@ -526,66 +507,10 @@ async fn quota_refresh_rejects_a_concurrent_credential_revision() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn legacy_billing_fields_do_not_supply_quota_evidence_or_period() {
|
||||
for (limit, used) in [
|
||||
(
|
||||
serde_json::json!({"val": 10000}),
|
||||
serde_json::json!({"val": 2500}),
|
||||
),
|
||||
(
|
||||
serde_json::json!("uninterpreted"),
|
||||
serde_json::json!({"val": -1}),
|
||||
),
|
||||
] {
|
||||
let (store, repository) = repository_with_accounts(&[("legacy", "legacy")]).await;
|
||||
let id = account_id("legacy");
|
||||
store
|
||||
.apply_quota_access(QuotaAccessChange {
|
||||
account_id: id.clone(),
|
||||
expected_revision: CredentialRevision::new(1).expect("revision"),
|
||||
state: QuotaState::exhausted(
|
||||
QuotaEvidence::ProviderDenied,
|
||||
SystemTime::now(),
|
||||
None,
|
||||
),
|
||||
})
|
||||
.await
|
||||
.expect("mark quota exhausted");
|
||||
let body = serde_json::to_vec(&serde_json::json!({"config": {
|
||||
"monthlyLimit": limit, "used": used,
|
||||
"billingPeriodStart": "2026-07-01T00:00:00Z",
|
||||
"billingPeriodEnd": "2099-08-01T00:00:00Z"
|
||||
}}))
|
||||
.expect("billing document");
|
||||
let service =
|
||||
crate::support::grok_quota_service(repository, QueueBillingTransport::success(&body));
|
||||
let snapshot = service
|
||||
.refresh_account(&id)
|
||||
.await
|
||||
.expect("refresh opaque billing");
|
||||
let account = store.account(&id).expect("account");
|
||||
|
||||
assert_eq!(snapshot.billing().used_percent(), None);
|
||||
assert!(!snapshot.billing().has_authoritative_quota());
|
||||
assert_eq!(
|
||||
snapshot.billing().period_kind(),
|
||||
provider_xai::GrokQuotaPeriodKind::Other
|
||||
);
|
||||
assert_eq!(snapshot.billing().period_start(), None);
|
||||
assert_eq!(snapshot.billing().period_end(), None);
|
||||
assert_eq!(service.scheduling_signals(&account), None);
|
||||
assert_eq!(
|
||||
account.quota().access(),
|
||||
gateway_core::account::QuotaAccessState::Exhausted
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn current_quota_period_does_not_fill_missing_dates_from_legacy_fields() {
|
||||
async fn quota_period_dates_remain_absent_when_not_reported() {
|
||||
let (_, repository) = repository_with_accounts(&[("current-period", "current-period")]).await;
|
||||
let transport = QueueBillingTransport::success(
|
||||
br#"{"config":{"creditUsagePercent":25,"currentPeriod":{"type":"USAGE_PERIOD_TYPE_WEEKLY"},"billingPeriodStart":"2026-07-01T00:00:00Z","billingPeriodEnd":"2099-08-01T00:00:00Z"}}"#,
|
||||
br#"{"config":{"creditUsagePercent":25,"currentPeriod":{"type":"USAGE_PERIOD_TYPE_WEEKLY"}}}"#,
|
||||
);
|
||||
let snapshot = crate::support::grok_quota_service(repository, transport)
|
||||
.refresh_account(&account_id("current-period"))
|
||||
@@ -658,61 +583,59 @@ async fn expired_billing_window_does_not_participate_in_scheduling_rank() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn non_authoritative_quota_refresh_does_not_clear_existing_quota_exhaustion() {
|
||||
let (store, repository) =
|
||||
repository_with_accounts(&[("free-quota", "subject-free-quota")]).await;
|
||||
let id = account_id("free-quota");
|
||||
store
|
||||
.apply_quota_access(QuotaAccessChange {
|
||||
account_id: id.clone(),
|
||||
expected_revision: CredentialRevision::new(1).expect("revision"),
|
||||
state: QuotaState::exhausted(QuotaEvidence::ProviderDenied, SystemTime::now(), None),
|
||||
})
|
||||
.await
|
||||
.expect("mark quota exhausted");
|
||||
let transport = QueueBillingTransport::success(
|
||||
br#"{"config":{"currentPeriod":{"type":"USAGE_PERIOD_TYPE_WEEKLY","start":"2026-07-15T00:00:00Z","end":"2026-07-22T00:00:00Z"},"onDemandCap":{"val":0},"onDemandUsed":{"val":0},"prepaidBalance":{"val":0}}}"#,
|
||||
);
|
||||
let snapshot = crate::support::grok_quota_service(repository, transport)
|
||||
.refresh_account(&id)
|
||||
.await
|
||||
.expect("refresh Free quota");
|
||||
async fn non_authoritative_quota_refresh_preserves_exhaustion_without_scheduling_evidence() {
|
||||
use provider_xai::GrokQuotaPeriodKind::{Other, Weekly};
|
||||
|
||||
assert!(!snapshot.billing().has_authoritative_quota());
|
||||
assert_eq!(
|
||||
store.account(&id).expect("account").quota().access(),
|
||||
gateway_core::account::QuotaAccessState::Exhausted
|
||||
);
|
||||
}
|
||||
for (config, expected_period) in [
|
||||
(
|
||||
serde_json::json!({"future_quota": {"remaining": 100}}),
|
||||
Other,
|
||||
),
|
||||
(
|
||||
serde_json::json!({
|
||||
"currentPeriod": {"type": "USAGE_PERIOD_TYPE_WEEKLY"},
|
||||
"onDemandCap": {"val": 0},
|
||||
"onDemandUsed": {"val": 0},
|
||||
"prepaidBalance": {"val": 0}
|
||||
}),
|
||||
Weekly,
|
||||
),
|
||||
] {
|
||||
let (store, repository) =
|
||||
repository_with_accounts(&[("free-quota", "subject-free-quota")]).await;
|
||||
let id = account_id("free-quota");
|
||||
store
|
||||
.apply_quota_access(QuotaAccessChange {
|
||||
account_id: id.clone(),
|
||||
expected_revision: CredentialRevision::new(1).expect("revision"),
|
||||
state: QuotaState::exhausted(
|
||||
QuotaEvidence::ProviderDenied,
|
||||
SystemTime::now(),
|
||||
None,
|
||||
),
|
||||
})
|
||||
.await
|
||||
.expect("mark quota exhausted");
|
||||
let body = serde_json::to_vec(&serde_json::json!({"config": config})).expect("billing");
|
||||
let service =
|
||||
crate::support::grok_quota_service(repository, QueueBillingTransport::success(&body));
|
||||
let snapshot = service
|
||||
.refresh_account(&id)
|
||||
.await
|
||||
.expect("refresh Free quota");
|
||||
let account = store.account(&id).expect("account");
|
||||
|
||||
#[tokio::test]
|
||||
async fn reported_zero_percent_is_authoritative_quota() {
|
||||
let (_, repository) =
|
||||
repository_with_accounts(&[("zero-percent", "subject-zero-percent")]).await;
|
||||
let transport = QueueBillingTransport::success(
|
||||
br#"{"config":{"creditUsagePercent":0,"currentPeriod":{"type":"USAGE_PERIOD_TYPE_WEEKLY","start":"2026-07-15T00:00:00Z","end":"2026-07-22T00:00:00Z"}}}"#,
|
||||
);
|
||||
let snapshot = crate::support::grok_quota_service(repository, transport)
|
||||
.refresh_account(&account_id("zero-percent"))
|
||||
.await
|
||||
.expect("refresh reported quota");
|
||||
|
||||
assert!(snapshot.billing().has_authoritative_quota());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn positive_prepaid_balance_is_authoritative_quota() {
|
||||
let (_, repository) =
|
||||
repository_with_accounts(&[("prepaid-quota", "subject-prepaid-quota")]).await;
|
||||
let transport = QueueBillingTransport::success(
|
||||
br#"{"config":{"currentPeriod":{"type":"USAGE_PERIOD_TYPE_WEEKLY","start":"2026-07-15T00:00:00Z","end":"2026-07-22T00:00:00Z"},"prepaidBalance":{"val":500}}}"#,
|
||||
);
|
||||
let snapshot = crate::support::grok_quota_service(repository, transport)
|
||||
.refresh_account(&account_id("prepaid-quota"))
|
||||
.await
|
||||
.expect("refresh prepaid quota");
|
||||
|
||||
assert!(snapshot.billing().has_authoritative_quota());
|
||||
assert!(!snapshot.billing().has_authoritative_quota());
|
||||
assert_eq!(snapshot.billing().used_percent(), None);
|
||||
assert_eq!(snapshot.billing().period_kind(), expected_period);
|
||||
assert_eq!(snapshot.billing().period_start(), None);
|
||||
assert_eq!(snapshot.billing().period_end(), None);
|
||||
assert_eq!(service.scheduling_signals(&account), None);
|
||||
assert_eq!(
|
||||
account.quota().access(),
|
||||
gateway_core::account::QuotaAccessState::Exhausted
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
||||
@@ -177,109 +177,40 @@ fn encoder_should_strip_all_client_metadata_before_grok_build() {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn encoder_should_apply_build_defaults_and_normalize_reasoning_effort() {
|
||||
let request = raw_request(json!({
|
||||
"model": "client-model",
|
||||
"input": "hello",
|
||||
"include": ["web_search_call.action.sources"],
|
||||
"reasoning": {"effort": "XHIGH", "summary": "auto"}
|
||||
}));
|
||||
fn encoder_should_apply_build_defaults_and_preserve_requested_includes() {
|
||||
for (fields, expected_include) in [
|
||||
(json!({}), json!(["reasoning.encrypted_content"])),
|
||||
(
|
||||
json!({"store": null, "include": null}),
|
||||
json!(["reasoning.encrypted_content"]),
|
||||
),
|
||||
(
|
||||
json!({"include": ["web_search_call.action.sources"]}),
|
||||
json!([
|
||||
"web_search_call.action.sources",
|
||||
"reasoning.encrypted_content"
|
||||
]),
|
||||
),
|
||||
] {
|
||||
let mut body = fields;
|
||||
body["input"] = json!("hello");
|
||||
let encoded = GrokResponsesRequest::encode(&raw_request(body), "grok-4.7", &client_key())
|
||||
.expect("build defaults");
|
||||
|
||||
let encoded = GrokResponsesRequest::encode(&request, "grok-4.5", &client_key())
|
||||
.expect("normalized request");
|
||||
let body = Value::Object(encoded.body().clone());
|
||||
|
||||
assert_eq!(body.pointer("/store"), Some(&json!(false)));
|
||||
assert_eq!(
|
||||
body.pointer("/include"),
|
||||
Some(&json!([
|
||||
"web_search_call.action.sources",
|
||||
"reasoning.encrypted_content"
|
||||
]))
|
||||
);
|
||||
assert_eq!(body.pointer("/reasoning/effort"), Some(&json!("xhigh")));
|
||||
assert_eq!(body.pointer("/reasoning/summary"), Some(&json!("auto")));
|
||||
|
||||
let request = raw_request(json!({
|
||||
"model": "client-model",
|
||||
"input": "hello",
|
||||
"reasoning": {"effort": "xhigh"}
|
||||
}));
|
||||
let encoded = GrokResponsesRequest::encode(&request, "xai/grok-4.6-latest", &client_key())
|
||||
.expect("routed model request");
|
||||
let body = Value::Object(encoded.body().clone());
|
||||
assert_eq!(body.pointer("/model"), Some(&json!("xai/grok-4.6-latest")));
|
||||
assert_eq!(body.pointer("/reasoning/effort"), Some(&json!("xhigh")));
|
||||
assert_eq!(body.pointer("/reasoning_effort"), None);
|
||||
assert_eq!(body.pointer("/reasoningEffort"), None);
|
||||
|
||||
let request = raw_request(json!({
|
||||
"model": "client-model",
|
||||
"input": "hello",
|
||||
"reasoning": {"effort": "high", "summary": "auto"}
|
||||
}));
|
||||
let encoded = GrokResponsesRequest::encode(&request, "grok-composer-2.5-fast", &client_key())
|
||||
.expect("Composer request");
|
||||
let body = Value::Object(encoded.body().clone());
|
||||
assert_eq!(
|
||||
body.pointer("/reasoning"),
|
||||
Some(&json!({"effort":"high", "summary":"auto"}))
|
||||
);
|
||||
|
||||
let request = raw_request(json!({
|
||||
"model": "client-model",
|
||||
"input": "hello",
|
||||
"store": null,
|
||||
"include": null,
|
||||
"reasoning": {"effort": "max"}
|
||||
}));
|
||||
let encoded = GrokResponsesRequest::encode(&request, "grok-composer-2.5-fast", &client_key())
|
||||
.expect("empty Composer reasoning");
|
||||
let body = Value::Object(encoded.body().clone());
|
||||
assert_eq!(body.pointer("/store"), Some(&json!(false)));
|
||||
assert_eq!(
|
||||
body.pointer("/include"),
|
||||
Some(&json!(["reasoning.encrypted_content"]))
|
||||
);
|
||||
assert_eq!(body.pointer("/reasoning"), Some(&json!({"effort":"max"})));
|
||||
|
||||
let request = raw_request(json!({
|
||||
"model": "client-model",
|
||||
"input": "hello",
|
||||
"reasoning": {}
|
||||
}));
|
||||
let encoded = GrokResponsesRequest::encode(&request, "grok-composer-2.5-fast", &client_key())
|
||||
.expect("Composer reasoning without effort");
|
||||
assert_eq!(
|
||||
Value::Object(encoded.body().clone()).pointer("/reasoning"),
|
||||
Some(&json!({}))
|
||||
);
|
||||
|
||||
let request = raw_request(json!({
|
||||
"model": "client-model",
|
||||
"input": "hello",
|
||||
"reasoning": {"effort": "max"}
|
||||
}));
|
||||
let encoded = GrokResponsesRequest::encode(&request, "tenant/foo/grok-4.6", &client_key())
|
||||
.expect("unknown provider prefix");
|
||||
assert_eq!(
|
||||
Value::Object(encoded.body().clone()).pointer("/reasoning/effort"),
|
||||
Some(&json!("max"))
|
||||
);
|
||||
assert_eq!(encoded.body()["store"], json!(false));
|
||||
assert_eq!(encoded.body()["include"], expected_include);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn encoder_should_strip_codex_only_fields_without_model_guesses() {
|
||||
fn encoder_should_strip_codex_only_fields_and_preserve_business_extensions() {
|
||||
let request = raw_request(json!({
|
||||
"model": "client-model",
|
||||
"input": "hello",
|
||||
"prompt_cache_retention": "24h",
|
||||
"safety_identifier": "user-1",
|
||||
"presence_penalty": 0.1,
|
||||
"presencePenalty": 0.2,
|
||||
"frequency_penalty": 0.3,
|
||||
"frequencyPenalty": 0.4,
|
||||
"stop": ["done"],
|
||||
"provider_options": {"providers": {"xai": {"transport": "websocket"}}},
|
||||
"future_official_field": {"keep": true},
|
||||
"external_web_access": true,
|
||||
"metadata": {"external_web_access": false},
|
||||
"tools": [{
|
||||
@@ -299,6 +230,7 @@ fn encoder_should_strip_codex_only_fields_without_model_guesses() {
|
||||
for pointer in [
|
||||
"/prompt_cache_retention",
|
||||
"/safety_identifier",
|
||||
"/provider_options",
|
||||
"/external_web_access",
|
||||
"/metadata/external_web_access",
|
||||
"/tools/0/external_web_access",
|
||||
@@ -306,18 +238,10 @@ fn encoder_should_strip_codex_only_fields_without_model_guesses() {
|
||||
assert_eq!(body.pointer(pointer), None, "field survived at {pointer}");
|
||||
}
|
||||
|
||||
let request = raw_request(json!({
|
||||
"model": "client-model",
|
||||
"input": "hello",
|
||||
"logprobs": true,
|
||||
"top_logprobs": 5
|
||||
}));
|
||||
let encoded = GrokResponsesRequest::encode(&request, "grok-4.20-reasoning", &client_key())
|
||||
.expect("sanitized 4.20 request");
|
||||
let body = Value::Object(encoded.body().clone());
|
||||
assert_eq!(body.pointer("/model"), Some(&json!("grok-4.20-reasoning")));
|
||||
assert_eq!(body.pointer("/logprobs"), Some(&json!(true)));
|
||||
assert_eq!(body.pointer("/top_logprobs"), Some(&json!(5)));
|
||||
assert_eq!(
|
||||
body.pointer("/future_official_field"),
|
||||
Some(&json!({"keep": true}))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -444,31 +368,6 @@ fn request_debug_should_not_expose_prompt_or_unknown_values() {
|
||||
assert!(!debug.contains("must-not-leak"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn encoder_should_remove_retired_gateway_provider_options_before_xai_conversion() {
|
||||
let request = raw_request(Value::Object(Map::from_iter([
|
||||
("model".to_owned(), json!("client")),
|
||||
("input".to_owned(), json!("hello")),
|
||||
(
|
||||
"provider_options".to_owned(),
|
||||
json!({
|
||||
"version": "future-version",
|
||||
"providers": {"xai": {"transport": "websocket", "turn_index": "7"}}
|
||||
}),
|
||||
),
|
||||
("future_official_field".to_owned(), json!({"keep": true})),
|
||||
])));
|
||||
|
||||
let encoded =
|
||||
GrokResponsesRequest::encode(&request, "grok-routed", &client_key()).expect("request");
|
||||
|
||||
assert!(encoded.body().get("provider_options").is_none());
|
||||
assert_eq!(
|
||||
encoded.body().get("future_official_field"),
|
||||
Some(&json!({"keep": true}))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn explicit_session_should_be_tenant_isolated_and_stable_across_turns() {
|
||||
let first = raw_request(json!({
|
||||
|
||||
@@ -1,52 +1,32 @@
|
||||
//! 路由已选定真实模型,xAI 编码器不再解释旧别名或 Provider 前缀
|
||||
//! 验证路由模型标识与业务参数在 xAI 请求编码后保持原值
|
||||
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn routed_model_identity_is_preserved_without_provider_local_aliases() {
|
||||
let request = raw_request(json!({"model": "client-model", "input": "hello"}));
|
||||
fn routed_model_and_business_parameters_are_preserved() {
|
||||
let fields = json!({
|
||||
"presence_penalty": 0.1,
|
||||
"frequency_penalty": 0.2,
|
||||
"stop": ["done"],
|
||||
"logprobs": true,
|
||||
"top_logprobs": 5
|
||||
});
|
||||
let mut body = fields.clone();
|
||||
body["model"] = json!("client-model");
|
||||
body["input"] = json!("hello");
|
||||
let request = raw_request(body);
|
||||
|
||||
for model in [
|
||||
"grok",
|
||||
"grok-latest",
|
||||
"grok-build-latest",
|
||||
"grok-4.5-latest",
|
||||
"grok-4.6-latest",
|
||||
"grok-4.3-latest",
|
||||
"grok-build",
|
||||
"grok-composer",
|
||||
"composer-2.5",
|
||||
"grok-4.20-reasoning",
|
||||
"grok-4.20-non-reasoning",
|
||||
"grok-4.20-multi-agent",
|
||||
"grok-4.20-multi-agent-latest",
|
||||
"grok-4.7",
|
||||
"Grok-4.6",
|
||||
"grok-latest",
|
||||
"grok-4.20-reasoning",
|
||||
"Grok-4.7",
|
||||
"xai/grok-4.7",
|
||||
"x-ai/grok-4.7",
|
||||
"grok/grok-4.7",
|
||||
"tenant/custom-model",
|
||||
] {
|
||||
let encoded = GrokResponsesRequest::encode(&request, model, &client_key())
|
||||
.expect("routed upstream model");
|
||||
assert_eq!(encoded.body()["model"], model);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn routed_model_identity_does_not_infer_unsupported_fields_from_old_slugs() {
|
||||
let fields = json!({"presence_penalty": 0.1, "frequency_penalty": 0.2,
|
||||
"stop": ["done"], "logprobs": true, "top_logprobs": 5});
|
||||
let request = raw_request(fields.clone());
|
||||
for model in [
|
||||
"grok-4.5",
|
||||
"grok-4.6",
|
||||
"grok-4.3",
|
||||
"grok-4.20-reasoning",
|
||||
"tenant/grok-4.6",
|
||||
"grok-4.7",
|
||||
] {
|
||||
let encoded =
|
||||
GrokResponsesRequest::encode(&request, model, &client_key()).expect("request");
|
||||
for (key, value) in fields.as_object().expect("fields") {
|
||||
assert_eq!(encoded.body().get(key), Some(value), "{model} {key}");
|
||||
}
|
||||
|
||||
@@ -4,14 +4,7 @@ use super::*;
|
||||
|
||||
#[test]
|
||||
fn official_reasoning_efforts_are_preserved_for_current_and_future_models() {
|
||||
for model in [
|
||||
"grok-4.7",
|
||||
"grok-future",
|
||||
"grok-4.5",
|
||||
"grok-4.6",
|
||||
"grok-3-mini",
|
||||
"grok-composer-2.5-fast",
|
||||
] {
|
||||
for model in ["grok-4.7", "grok-future", "tenant/custom-model"] {
|
||||
for effort in ["none", "minimal", "low", "medium", "high", "xhigh", "max"] {
|
||||
let request = raw_request(json!({
|
||||
"input": "hello",
|
||||
|
||||
@@ -48,7 +48,8 @@ export CPR_TEST_REDIS_URL='redis://:<password>@127.0.0.1:6379'
|
||||
PostgreSQL 的临时 schema 测试连接使用异步提交,保留事务可见性、回滚和完整迁移检查
|
||||
需要验证数据库崩溃后的持久性时,使用独立数据库与生产连接配置,不复用这类 fixture
|
||||
|
||||
运行包含 `StoreBundle` 初始化的完整集成测试时,两条测试 URL 都须包含密码,且密码满足 Store
|
||||
启动配置的 48 位十六进制要求;仅能连接数据库并不代表该初始化合同通过。专用服务使用对应测试密码,
|
||||
运行包含 `StoreBundle` 初始化的完整集成测试时,两条测试 URL 都须包含密码,原始密码须满足
|
||||
[Store 密码要求](../../deploy/README.md#手动安装),在测试 URL 中按 URL 规则编码。
|
||||
仅能连接数据库并不代表该初始化合同通过。专用服务使用对应测试密码,
|
||||
并在测试进程中清除 `CPR_DATABASE_URL`、`CPR_REDIS_URL`、`CPR_DATABASE_PASSWORD` 和
|
||||
`CPR_REDIS_PASSWORD`,避免启动配置被部署环境覆盖
|
||||
|
||||
+8
-3
@@ -159,8 +159,13 @@ openssl rand -hex 24
|
||||
推荐使用模板中的 7 天不活动期限和 30 天最长登录期限。修改配置需重启,不能延长已建立会话的最长有效期。
|
||||
浏览器会话与续期规则见[认证 API](../docs/api.md#统一登录与会话),独立于 `/v1/*` 鉴权和限额
|
||||
|
||||
PostgreSQL 与 Redis 密码必须是 48 位十六进制字符。Compose 通过 `config.yaml` 的凭据桥接区
|
||||
引用同一密码;三个值都不需要额外导出为环境变量,数据库和 Redis 密码也不能嵌入连接 URL
|
||||
PostgreSQL 与 Redis 密码统一接受 16–128 字节,不能全为空白或包含控制字符;仍须满足
|
||||
各自服务端的密码策略。托管服务可使用服务商允许的高强度密码,默认安装继续生成 48 位随机十六进制密码
|
||||
|
||||
Compose 通过 `config.yaml` 的凭据桥接区引用同一密码;三个值都不需要额外导出为环境变量,
|
||||
数据库和 Redis 密码不能嵌入连接 URL。通过环境变量提供密码时分别使用
|
||||
`CPR_DATABASE_PASSWORD` 与 `CPR_REDIS_PASSWORD`,原始密码由连接 URL 构造器编码,不要预先做 URL 编码。Compose 凭据桥接建议沿用上面的
|
||||
随机十六进制密码,避免 `$` 的变量插值语义
|
||||
|
||||
Linux 上应用容器以 `10001:10001` 运行。上述命令将应用数据和日志目录设为 `0770`,
|
||||
配置设为 `0640`,均由当前用户持有、容器组 `10001` 访问。
|
||||
@@ -174,7 +179,7 @@ Linux 上应用容器以 `10001:10001` 运行。上述命令将应用数据和
|
||||
store:
|
||||
redis:
|
||||
url: 'redis://u1@redis-host:6379/0'
|
||||
password: &redis_password '<u1 的 48 位十六进制密码>'
|
||||
password: &redis_password '<u1 的密码>'
|
||||
```
|
||||
|
||||
将这两项合并到已有配置,保留其他配置和密码锚点;Redis 服务端需事先创建并授权该用户。
|
||||
|
||||
@@ -44,11 +44,13 @@ store:
|
||||
database:
|
||||
# Docker 内由 CPR_DATABASE_URL 覆盖;密码不要写进 URL。
|
||||
url: 'postgres://codex_proxy@127.0.0.1:5432/codex_proxy'
|
||||
# 16–128 字节,不能全为空白或含控制字符,并满足服务端密码策略
|
||||
password: &postgres_password ''
|
||||
redis:
|
||||
# Docker 内由 CPR_REDIS_URL 覆盖;密码不要写进 URL。
|
||||
# ACL 用户写在 URL 中,例如 redis://u1@redis-host:6379/0;配置步骤见 deploy/README.md。
|
||||
url: 'redis://127.0.0.1:6379/'
|
||||
# 与 PostgreSQL 共用密码格式要求,并满足 Redis 服务端密码策略
|
||||
password: &redis_password ''
|
||||
# 可选:PostgreSQL 连接池预算;下列值与当前推荐默认一致。
|
||||
# pool:
|
||||
|
||||
+13
-11
@@ -754,7 +754,10 @@ Codex 的 `model_catalog_json` 配置。账号设置保存不等待上游模型
|
||||
`quota.credits` 返回上游点数余额,未提供点数信息时为 `null`。对象包含 `hasCredits`、`unlimited` 和
|
||||
`balance`,余额为保留上游精度的十进制字符串,未提供余额时为 `null`,明确的零余额为 `"0"`。
|
||||
OpenAI 点数随现有额度刷新和正常请求的额度信息同步,不与主动重置卡库存或百分比限额合并,
|
||||
不参与账号可用性判断
|
||||
不根据 `balance` 单独推导可用性。主动查询明确 `hasCredits=true` 或 `unlimited=true`,且无阻止
|
||||
点数支出的上限或工作区限制时,即使套餐窗口耗尽也可使用点数,并解除已有额度耗尽状态。
|
||||
真实额度拒绝后的后台回读同样支持此恢复路径,但不会自动重发本次失败请求;点数不能清除
|
||||
凭据错误、停用或冷却状态。上游点数信息也可能延迟,后续真实拒绝仍会记录额度耗尽
|
||||
|
||||
账号列表和详情返回 `notes`(无备注时为 `null`)。编辑时省略或 `null` 保留原备注;字符串最多 500 个 Unicode
|
||||
字符,允许换行和制表符,保存时去除首尾空白,空字符串清空备注。备注独立于上游身份,导入时未显式提供备注、
|
||||
@@ -1118,11 +1121,14 @@ OAuth start 使用:
|
||||
但上游 authorization code 本身通常只能交换一次;已完成过 token exchange 时应重新创建 OAuth flow
|
||||
- `GET /accounts/quota` 只读取最后一次落库快照;`POST /accounts/quota/refresh` 才访问上游。access token
|
||||
已过期时,额度刷新要求先走 credential 刷新或重新授权,不会拿过期 token 探测额度
|
||||
- OpenAI 已耗尽账号每 30 分钟主动复核一次,也会在最早未恢复窗口的 `resetAt + 2 分钟` 到期后
|
||||
提前复核;周期复核不等待旧重置时间,因此也能发现官方提前重置。正常账号有非零用量或触顶窗口时,
|
||||
在该窗口的 `resetAt + 2 分钟` 后主动复核。后台每 30 秒检查触发条件;同一重置边界复核后仍未更新时
|
||||
回到 30 分钟重试,避免旧 reset 持续触发请求。各窗口独立确认恢复,时间到期本身不会直接解除账号
|
||||
耗尽或将展示用量归零
|
||||
- OpenAI 已耗尽账号首次可复核,在 `resetAt` 尚未到达或未知时按 30 分钟间隔兜底,也可发现
|
||||
官方提前重置。到达最早未恢复窗口的 `resetAt` 后,先等待 10 秒传播余量,再在下一轮后台检查
|
||||
复核;reset 后前 5 分钟最小查询间隔为 30 秒,之后仍耗尽则改为 5 分钟。正常账号有非零用量或触顶窗口时,仍在
|
||||
`resetAt + 2 分钟` 后复核,未更新则按 30 分钟间隔重试。后台每 30 秒检查条件,实际恢复时间还受
|
||||
排队、查询耗时和上游传播影响;查询失败也遵守间隔。无可用点数时,到期前仍按原窗口证据恢复;
|
||||
本次耗尽之后的已记录窗口 reset 全部到期后,新查询明确 `allowed=true` 可解除限制,即使用量仍为
|
||||
100%。未知 reset 或再次拒绝后残留的旧 reset 仍走原规则,不复用旧边界解封;时间到期本身不会
|
||||
解除限制,也不会将展示用量归零
|
||||
- 账号 `enabled` 控制是否参与请求调度,不控制 OAuth 凭据续期。OpenAI 与 xAI 的停用账号仍按各自刷新
|
||||
策略维护 Token;刷新结果更新凭据状态,但不会启用调度。无 refresh token 或凭据已进入失效、无效、
|
||||
封禁状态的账号不参加后台自动续期
|
||||
@@ -2055,7 +2061,7 @@ OpenAI 仅使用同一条 `response.completed` 中的 `created_at` 与 `complete
|
||||
|
||||
`latencyDetails` 还可包含以下上游性能指标,单位为毫秒,保留上游返回的小数。
|
||||
OpenAI 从当前响应的 `responsesapi.websocket_timing.timing_metrics` 读取,
|
||||
WebSocket 握手默认请求专项计时,客户端显式提供的开关保留原值。
|
||||
WebSocket 握手不主动添加专项计时请求头,客户端显式提供的开关保留原值。
|
||||
各字段独立可选;响应开始前、终态后的计时和明确属于其他响应的事件不写入当前请求,重试时清空
|
||||
|
||||
`critical_path` 仅采集 `scope=response`、`coverage=complete` 且边界为
|
||||
@@ -2067,10 +2073,6 @@ WebSocket 握手默认请求专项计时,客户端显式提供的开关保留
|
||||
| --- | --- |
|
||||
| `upstreamApiOverheadMs` | API 排除引擎与客户端工具时间后的耗时 |
|
||||
| `upstreamEngineMs` | 响应级引擎耗时,取完整 critical path 的墙钟时间或直接返回的 Engine Service 时间 |
|
||||
| `upstreamEngineIapiTtftMs` | Engine IAPI TTFT 总计 |
|
||||
| `upstreamEngineServiceTtftMs` | Engine Service TTFT 总计 |
|
||||
| `upstreamEngineIapiTbtMs` | 跨引擎调用的 IAPI Token 间隔 |
|
||||
| `upstreamEngineServiceTbtMs` | 跨引擎调用的 Service Token 间隔 |
|
||||
|
||||
以上指标按上游内部口径展示,不相加重建总耗时,不替换请求级 `firstTokenLatencyMs`。
|
||||
它们仅补充性能诊断,不改变列表与聚合统计的速率分母
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user