Compare commits

...
30 Commits
Author SHA1 Message Date
Zyy 0f5bf03e13 fix(frontend): keep content below the title bar when opening the PWA 2026-11-14 22:11:03 +08:00
github-actions[bot] 28c8c536c5 chore(release): update release channels to 3.49.0 2026-11-14 13:06:03 +00:00
ZyyandCodex 4ee9abf9d2 chore: bump version to 3.49.0
Co-authored-by: Codex <noreply@openai.com>
2026-11-14 20:46:27 +08:00
Zyy 779ccf4e6c feat(frontend): support a draggable title bar in the desktop PWA 2026-11-14 19:44:12 +08:00
d13a4a2138 feat(scheduling): add independent quota reset window weights (#713)
Rank quota scheduling strictly by configured reset windows and persist its settings independently from smart scheduling.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-14 18:13:33 +08:00
a0e7f38f7f fix(openai): preserve Claude Messages session affinity (#712)
Resolve explicit Claude session identities before account selection and
revalidate them after attempt middleware using the existing affinity policy.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-14 17:22:03 +08:00
David Lee 41f97ac836 feat(openai): check client profile releases hourly (#711) 2026-11-14 16:34:39 +08:00
github-actions[bot] 975b73b33b chore(release): update release channels to 3.48.0 2026-11-14 03:07:56 +00:00
ZyyandCodex d787195f4e chore: bump version to 3.48.0
Co-authored-by: Codex <noreply@openai.com>
2026-11-14 10:48:12 +08:00
ZyyandCodex 8681d6851e fix(deps): update Quinn and rustix to patched releases
Co-authored-by: Codex <noreply@openai.com>
2026-11-14 10:43:50 +08:00
d99ecf9aaf fix(openai): align model operations with global client profiles (#708)
Apply the global request profile to account catalog discovery, warmup,
and diagnostic routing. Freeze each operation's identity through the
existing provider profile resolver.

Refresh verified offline releases and centralize their version, artifact,
and environment defaults. Cover ZIP64 MSIX metadata and keep release
update tests independent of the current bundled versions.

Fixes #703

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-14 10:40:12 +08:00
ZyyandZyy 8f40bfda7c fix(openai): preserve Responses identity semantics (#707)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-11-14 10:36:23 +08:00
ZyyandCodex 72215fe62a feat(web): enable PWA installation with SVG icon
Add an installable web app manifest with standalone launch behavior and document deployment requirements.

Co-authored-by: Codex <noreply@openai.com>
2026-11-13 17:50:05 +08:00
f61995e61d feat(settings): remove request history retention cap (#702)
* feat(settings): remove request history retention cap

Allow positive retention days beyond 31 throughout the form, domain rules,
and database constraint. Keep the existing default and saved settings.
Bound cleanup and account usage queries by stored data so large retention
windows remain safe without expanding empty history.

Co-authored-by: Codex <noreply@openai.com>

* test(admin): align retention policy coverage with positive windows

Accept long request retention windows while preserving zero rejection
for each target and independent event and audit retention settings.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-13 17:29:21 +08:00
ZyyandCodex f85d80d756 fix(openai): release websocket resources under backpressure
Make event delivery respond to downstream cancellation and pool shutdown. Return or discard sockets before publishing terminal events while preserving normal connection reuse.

Co-authored-by: Codex <noreply@openai.com>
2026-11-13 15:38:22 +08:00
github-actions[bot] e1f84c210b chore(release): update release channels to 3.47.0 2026-11-13 05:01:40 +00:00
ZyyandCodex 0e02a056b5 chore: bump version to 3.47.0
Co-authored-by: Codex <noreply@openai.com>
2026-11-13 12:40:31 +08:00
1eac929e9d perf(store): reduce temporary writes in analytics projections (#698)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-13 12:37:40 +08:00
9b4a042be5 feat(settings): support multiple account warmup cron expressions (#697)
Match independently scheduled hour and minute pairs while claiming each local minute once. Keep the existing string setting and database constraint, and edit the rules as tags using UI v0.11.0.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-13 12:17:50 +08:00
ZyyandCodex 3bd042181e fix(openai): preserve model exclusions during catalog refresh
Use the last successful catalog for the same account, credential revision, and plan until a refresh replaces it. Apply the same exclusions in memory and in PostgreSQL while preserving explicit allowlists and accounts with no catalog.

Co-authored-by: Codex <noreply@openai.com>
2026-11-13 10:16:12 +08:00
ZyyandCodex 2cae28d7f4 fix(ui): restore themed surfaces in the admin console
Use UI v0.10.1 so built-in presets tint pages and cards while preserving explicit background and semantic color settings.

Co-authored-by: Codex <noreply@openai.com>
2026-11-12 22:29:24 +08:00
github-actions[bot] bab781cbe0 chore(release): update release channels to 3.46.0 2026-11-12 13:56:39 +00:00
ZyyandCodex 28ad8fb1c4 chore: bump version to 3.46.0
Co-authored-by: Codex <noreply@openai.com>
2026-11-12 21:12:25 +08:00
ZyyandZyy 4888681a04 feat(scheduling): weight quota resets by window in smart scheduling (#694)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-11-12 18:56:30 +08:00
9739bb08be fix(egress): honor deployment proxies for unbound outbound requests (#693)
* fix(egress): honor deployment proxies for unbound outbound requests

Prefer the proxy selected for the request. Otherwise apply the deployment proxy and bypass rules consistently to HTTP and WebSocket connections, including background downloads. Keep explicit direct probes separate.

Co-authored-by: Codex <noreply@openai.com>

* fix(openai): preserve WebSocket dial errors

Recover the OS error or I/O kind from Hyper without retaining the endpoint. Keep refused connections distinguishable for both direct and proxied WebSocket dials.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-12 17:29:35 +08:00
fac37ec563 fix(update): preserve permissions when moving read-only directories (#688)
* fix(update): unlock official plugin dirs before rename on macOS

macOS rejects renaming a 0555 official plugin directory during backup
and rollback, while Linux only needs the parent writable.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(update): preserve directory permissions during macOS moves

Fall back to the existing permission-preserving copy path when macOS
rejects a directory rename. Keep chmod limited to source cleanup so
backups and restored releases retain their original directory modes.

Cover read-only staged assets, rollback, failed replacement, and
cancelled or rejected rollback with permission assertions.

Co-authored-by: Codex <noreply@openai.com>

* fix(update): handle read-only directory moves across Unix platforms

Linux also requires source directory write permission when rename moves
it between parents. Apply the same copy fallback to Unix while retaining
the cross-platform permission regression test.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: CzsGit <czsde@outlook.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-12 13:20:16 +08:00
9f0d712671 fix(openai): honor advertised WebSocket message limits (#691)
Use each upstream handshake's message-size limit to select HTTP before
uploading oversized response.create frames. Remove fixed thresholds and
thread size hysteresis while preserving continuation replay and cleanup.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-12 13:19:56 +08:00
ZyyandCodex 75be1287ef fix(openai): omit output token limits from Responses requests
Co-authored-by: Codex <noreply@openai.com>
2026-11-12 10:24:27 +08:00
ZyyandCodex dad91fe9f7 fix(ui): preserve semantic and neutral colors across theme presets
Co-authored-by: Codex <noreply@openai.com>
2026-11-11 20:48:41 +08:00
github-actions[bot] b22b0cfa41 chore(release): update release channels to 3.45.2 2026-11-11 09:28:26 +00:00
219 changed files with 5189 additions and 1772 deletions
+20 -6
View File
@@ -1807,6 +1807,17 @@ dependencies = [
"uuid", "uuid",
] ]
[[package]]
name = "gateway-egress"
version = "0.0.0"
dependencies = [
"gateway-core",
"http",
"hyper-util",
"reqwest",
"url",
]
[[package]] [[package]]
name = "gateway-host" name = "gateway-host"
version = "0.0.0" version = "0.0.0"
@@ -1822,6 +1833,7 @@ dependencies = [
"futures", "futures",
"gateway-admin", "gateway-admin",
"gateway-core", "gateway-core",
"gateway-egress",
"gateway-protocol", "gateway-protocol",
"hex", "hex",
"http", "http",
@@ -3353,6 +3365,7 @@ dependencies = [
"futures", "futures",
"gateway-admin", "gateway-admin",
"gateway-core", "gateway-core",
"gateway-egress",
"gateway-host", "gateway-host",
"gateway-protocol", "gateway-protocol",
"getrandom 0.4.3", "getrandom 0.4.3",
@@ -3410,6 +3423,7 @@ dependencies = [
"futures", "futures",
"gateway-admin", "gateway-admin",
"gateway-core", "gateway-core",
"gateway-egress",
"gateway-protocol", "gateway-protocol",
"getrandom 0.4.3", "getrandom 0.4.3",
"jsonwebtoken", "jsonwebtoken",
@@ -3478,9 +3492,9 @@ dependencies = [
[[package]] [[package]]
name = "quinn-proto" name = "quinn-proto"
version = "0.11.18" version = "0.11.19"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a9746dbde176634f4f2f1faf2404e30a31b2bc1e9cafb5329c95d8177a18c9fc" checksum = "0e750cca55fe4f0439a15d0bb529da9651e79993e8e72c61a899a36d462befbe"
dependencies = [ dependencies = [
"bytes", "bytes",
"getrandom 0.4.3", "getrandom 0.4.3",
@@ -3500,9 +3514,9 @@ dependencies = [
[[package]] [[package]]
name = "quinn-udp" name = "quinn-udp"
version = "0.5.15" version = "0.5.16"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "35a133f956daabe89a61a685c2649f13d82d5aa4bd5d12d1277e1072a21c0694" checksum = "af66907df18639dcf4db56ca65490cabc4b27a97dbadd96f2926cca73298f016"
dependencies = [ dependencies = [
"cfg_aliases", "cfg_aliases",
"libc", "libc",
@@ -3813,9 +3827,9 @@ dependencies = [
[[package]] [[package]]
name = "rustix" name = "rustix"
version = "1.1.4" version = "1.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" checksum = "e5e76742e5294153d4dde1207dbec5bdd81eef6fabdc0d2118babadabfd858aa"
dependencies = [ dependencies = [
"bitflags", "bitflags",
"errno", "errno",
+1
View File
@@ -5,6 +5,7 @@ members = [
"crates/gateway-admin", "crates/gateway-admin",
"crates/gateway-api", "crates/gateway-api",
"crates/gateway-core", "crates/gateway-core",
"crates/gateway-egress",
"crates/gateway-host", "crates/gateway-host",
"crates/gateway-plugin/runtime", "crates/gateway-plugin/runtime",
"crates/gateway-plugin/sdk", "crates/gateway-plugin/sdk",
@@ -16,6 +16,7 @@ pub(super) const WORKSPACE_MEMBERS: &[&str] = &[
"crates/gateway-admin", "crates/gateway-admin",
"crates/gateway-api", "crates/gateway-api",
"crates/gateway-core", "crates/gateway-core",
"crates/gateway-egress",
"crates/gateway-host", "crates/gateway-host",
"crates/gateway-plugin/runtime", "crates/gateway-plugin/runtime",
"crates/gateway-plugin/sdk", "crates/gateway-plugin/sdk",
@@ -447,6 +448,7 @@ const PACKAGE_TO_MEMBER: &[(&str, &str)] = &[
("gateway-admin", "crates/gateway-admin"), ("gateway-admin", "crates/gateway-admin"),
("gateway-api", "crates/gateway-api"), ("gateway-api", "crates/gateway-api"),
("gateway-core", "crates/gateway-core"), ("gateway-core", "crates/gateway-core"),
("gateway-egress", "crates/gateway-egress"),
("gateway-host", "crates/gateway-host"), ("gateway-host", "crates/gateway-host"),
("gateway-plugin-runtime", "crates/gateway-plugin/runtime"), ("gateway-plugin-runtime", "crates/gateway-plugin/runtime"),
("gateway-plugin-sdk", "crates/gateway-plugin/sdk"), ("gateway-plugin-sdk", "crates/gateway-plugin/sdk"),
@@ -510,8 +512,10 @@ const ALLOWED_INTERNAL_EDGES: &[(&str, &str)] = &[
("gateway-api", "gateway-core"), ("gateway-api", "gateway-core"),
("gateway-api", "gateway-protocol"), ("gateway-api", "gateway-protocol"),
("gateway-core", "gateway-protocol"), ("gateway-core", "gateway-protocol"),
("gateway-egress", "gateway-core"),
("gateway-host", "gateway-admin"), ("gateway-host", "gateway-admin"),
("gateway-host", "gateway-core"), ("gateway-host", "gateway-core"),
("gateway-host", "gateway-egress"),
("gateway-host", "gateway-protocol"), ("gateway-host", "gateway-protocol"),
("gateway-plugin-runtime", "gateway-host"), ("gateway-plugin-runtime", "gateway-host"),
("gateway-plugin-runtime", "gateway-core"), ("gateway-plugin-runtime", "gateway-core"),
@@ -522,9 +526,11 @@ const ALLOWED_INTERNAL_EDGES: &[(&str, &str)] = &[
("gateway-store", "gateway-core"), ("gateway-store", "gateway-core"),
("provider-openai", "gateway-admin"), ("provider-openai", "gateway-admin"),
("provider-openai", "gateway-core"), ("provider-openai", "gateway-core"),
("provider-openai", "gateway-egress"),
("provider-openai", "gateway-protocol"), ("provider-openai", "gateway-protocol"),
("provider-xai", "gateway-admin"), ("provider-xai", "gateway-admin"),
("provider-xai", "gateway-core"), ("provider-xai", "gateway-core"),
("provider-xai", "gateway-egress"),
("provider-xai", "gateway-protocol"), ("provider-xai", "gateway-protocol"),
]; ];
@@ -78,7 +78,12 @@ async fn proxy_probe_should_use_provider_custom_ca_for_https_proxies() {
}); });
let probe = HttpProxyProbe::new("http://unresolvable.invalid/ip") let probe = HttpProxyProbe::new("http://unresolvable.invalid/ip")
.with_client_builder(provider_openai::build_reqwest_client_with_custom_ca); .with_client_builder(provider_openai::build_reqwest_client_with_custom_ca);
let result = probe.test(Some(&proxy), false).await; let result = probe
.test(
gateway_admin::ports::proxy::ProbeRoute::Proxy(&proxy),
false,
)
.await;
assert!(result.success, "{}", result.message); assert!(result.success, "{}", result.message);
assert_eq!(result.exit_ip.unwrap().to_string(), "203.0.113.8"); assert_eq!(result.exit_ip.unwrap().to_string(), "203.0.113.8");
tokio::time::timeout(Duration::from_secs(5), server) tokio::time::timeout(Duration::from_secs(5), server)
@@ -146,7 +146,7 @@ pub struct AccountRecord {
} }
impl AccountRecord { impl AccountRecord {
/// 配置保留绑定,执行请求时禁用代理按直连处理 /// 配置保留绑定,执行请求时禁用代理使用部署默认出口
#[must_use] #[must_use]
pub fn effective_outbound_proxy(&self) -> Option<&gateway_core::account::OutboundProxy> { pub fn effective_outbound_proxy(&self) -> Option<&gateway_core::account::OutboundProxy> {
self.outbound_proxy self.outbound_proxy
@@ -24,7 +24,7 @@ impl RetentionPolicy {
audit_days: u32, audit_days: u32,
) -> Result<(), &'static str> { ) -> Result<(), &'static str> {
for (valid, field) in [ for (valid, field) in [
((1..=31).contains(&usage_days), "usage_retention_days"), (usage_days > 0, "usage_retention_days"),
(ops_days > 0, "ops_event_retention_days"), (ops_days > 0, "ops_event_retention_days"),
(audit_days > 0, "audit_retention_days"), (audit_days > 0, "audit_retention_days"),
] { ] {
@@ -38,7 +38,7 @@ impl RetentionPolicy {
pub fn try_new(usage_days: u32, ops_days: u32, audit_days: u32) -> Result<Self, AdminError> { pub fn try_new(usage_days: u32, ops_days: u32, audit_days: u32) -> Result<Self, AdminError> {
if Self::validate_values(usage_days, ops_days, audit_days).is_err() { if Self::validate_values(usage_days, ops_days, audit_days).is_err() {
return Err(AdminError::invalid( return Err(AdminError::invalid(
"请求保留期必须为 1~31 天,事件与审计保留期必须为正数", "请求、事件与审计保留期必须为正整数天数",
)); ));
} }
Ok(Self { Ok(Self {
@@ -63,6 +63,7 @@ pub struct RuntimeSettingsValues {
pub openai_session_affinity_ttl_hours: u32, pub openai_session_affinity_ttl_hours: u32,
pub responses_max_decompressed_body_bytes: u64, pub responses_max_decompressed_body_bytes: u64,
pub smart_scheduling: gateway_core::account::SmartSchedulingConfig, pub smart_scheduling: gateway_core::account::SmartSchedulingConfig,
pub quota_reset_window_weights: gateway_core::account::ResetWindowWeights,
pub min_codex_desktop_version: Option<String>, pub min_codex_desktop_version: Option<String>,
pub min_codex_cli_version: Option<String>, pub min_codex_cli_version: Option<String>,
pub usage_retention_days: u32, pub usage_retention_days: u32,
@@ -69,8 +69,14 @@ pub struct ProxyImportReservation {
pub guard: Box<dyn ProxyImportGuard>, pub guard: Box<dyn ProxyImportGuard>,
} }
#[derive(Clone, Copy)]
pub enum ProbeRoute<'a> {
Direct,
Deployment,
Proxy(&'a OutboundProxy),
}
#[async_trait] #[async_trait]
pub trait ProxyProbe: Send + Sync { pub trait ProxyProbe: Send + Sync {
/// 未指定代理时探测服务器直连出口,不继承进程代理环境变量 async fn test(&self, proxy: ProbeRoute<'_>, detect_location: bool) -> ProxyTestResult;
async fn test(&self, proxy: Option<&OutboundProxy>, detect_location: bool) -> ProxyTestResult;
} }
@@ -306,7 +306,9 @@ impl PluginsService {
.await .await
.map_err(|error| map_store_error(error, "proxy"))?; .map_err(|error| map_store_error(error, "proxy"))?;
if !record.enabled { if !record.enabled {
return Err(AdminError::invalid("下载代理已禁用,请启用或改为直连")); return Err(AdminError::invalid(
"下载代理已禁用,请启用或改用部署默认出口",
));
} }
Ok(Some(PluginDistributionEgress::new( Ok(Some(PluginDistributionEgress::new(
record.id, record.id,
@@ -23,7 +23,7 @@ use crate::{
}, },
ports::{ ports::{
provider::ProviderAdminRegistry, provider::ProviderAdminRegistry,
proxy::{PluginProxyAccess, ProxyProbe, ProxyStore}, proxy::{PluginProxyAccess, ProbeRoute, ProxyProbe, ProxyStore},
store::AdminStoreErrorKind, store::AdminStoreErrorKind,
}, },
}; };
@@ -47,7 +47,7 @@ impl PluginProxyAccess for DefaultPluginProxyAccess {
.get(id) .get(id)
.await .await
.map_err(|error| map_store_error(error, "proxy"))?; .map_err(|error| map_store_error(error, "proxy"))?;
// 显式选择代理时必须有可用配置,不能沿用账号停用代理后直连的语义 // 显式选择代理时必须有可用配置,不能沿用账号停用代理后使用部署默认出口的语义
if !record.enabled { if !record.enabled {
return Err(AdminError::invalid("代理已禁用,请启用后再选择")); return Err(AdminError::invalid("代理已禁用,请启用后再选择"));
} }
@@ -114,10 +114,10 @@ pub trait ProxiesService: Send + Sync {
detect_location: bool, detect_location: bool,
context: &MutationContext, context: &MutationContext,
) -> Result<ProxyRecord, AdminError>; ) -> Result<ProxyRecord, AdminError>;
/// 探测未保存的连接地址或服务器直连出口,不写入配置或修改账号绑定 /// 探测未保存的连接地址、部署默认出口或服务器直连出口,不写入配置或修改账号绑定
async fn probe( async fn probe(
&self, &self,
proxy: Option<&OutboundProxy>, proxy: ProbeRoute<'_>,
detect_location: bool, detect_location: bool,
) -> Result<ProxyTestResult, AdminError>; ) -> Result<ProxyTestResult, AdminError>;
} }
@@ -323,7 +323,7 @@ impl ProxiesService for DefaultProxiesService {
.transpose() .transpose()
.map_err(|_| AdminError::invalid("代理位置不合法"))?; .map_err(|_| AdminError::invalid("代理位置不合法"))?;
command.test = if command.auto_location { command.test = if command.auto_location {
Some(self.probe(Some(&command.proxy), true).await?) Some(self.probe(ProbeRoute::Proxy(&command.proxy), true).await?)
} else { } else {
None None
}; };
@@ -362,7 +362,7 @@ impl ProxiesService for DefaultProxiesService {
if command.auto_location.unwrap_or(current.auto_location) if command.auto_location.unwrap_or(current.auto_location)
&& (!current.auto_location || proxy != &current.proxy) && (!current.auto_location || proxy != &current.proxy)
{ {
command.test = Some(self.probe(Some(proxy), true).await?); command.test = Some(self.probe(ProbeRoute::Proxy(proxy), true).await?);
} }
} }
let result = self let result = self
@@ -391,7 +391,7 @@ impl ProxiesService for DefaultProxiesService {
async fn probe( async fn probe(
&self, &self,
proxy: Option<&OutboundProxy>, proxy: ProbeRoute<'_>,
detect_location: bool, detect_location: bool,
) -> Result<ProxyTestResult, AdminError> { ) -> Result<ProxyTestResult, AdminError> {
let _permit = self let _permit = self
@@ -423,7 +423,10 @@ impl ProxiesService for DefaultProxiesService {
// 手动解析只为本次测试请求位置,不改变自动跟随的持久配置 // 手动解析只为本次测试请求位置,不改变自动跟随的持久配置
let result = self let result = self
.probe .probe
.test(Some(&record.proxy), record.auto_location || detect_location) .test(
ProbeRoute::Proxy(&record.proxy),
record.auto_location || detect_location,
)
.await; .await;
let mutation = self let mutation = self
.store .store
@@ -3,25 +3,15 @@
use gateway_admin::model::retention::{RetentionPolicy, RetentionTarget}; use gateway_admin::model::retention::{RetentionPolicy, RetentionTarget};
#[test] #[test]
fn retention_policy_rejects_invalid_windows_and_preserves_independent_windows() { fn retention_policy_rejects_zero_for_each_target() {
for (usage, ops, audit) in [ for (usage, ops, audit) in [(0, 1, 1), (1, 0, 1), (1, 1, 0)] {
(0, 1, 1),
(32, 1, 1),
(u32::MAX, 1, 1),
(1, 0, 1),
(1, 1, 0),
] {
assert!(RetentionPolicy::try_new(usage, ops, audit).is_err()); assert!(RetentionPolicy::try_new(usage, ops, audit).is_err());
} }
let policy = RetentionPolicy::try_new(31, 7, 90).unwrap();
assert_eq!(policy.days(RetentionTarget::ModelRequests), 31);
assert_eq!(policy.days(RetentionTarget::OpsEvents), 7);
assert_eq!(policy.days(RetentionTarget::AdminAuditEvents), 90);
} }
#[test] #[test]
fn request_retention_accepts_short_windows_without_changing_other_targets() { fn request_retention_accepts_positive_windows_without_changing_other_targets() {
for days in [1, 3, 7, 15, 30, 31] { for days in [1, 31, 32, 365, 3650, u32::MAX] {
let policy = RetentionPolicy::try_new(days, 30, 90).unwrap(); let policy = RetentionPolicy::try_new(days, 30, 90).unwrap();
assert_eq!(policy.days(RetentionTarget::ModelRequests), days); assert_eq!(policy.days(RetentionTarget::ModelRequests), days);
assert_eq!(policy.days(RetentionTarget::OpsEvents), 30); assert_eq!(policy.days(RetentionTarget::OpsEvents), 30);
@@ -1180,6 +1180,7 @@ impl SettingsStore for StaticSettingsStore {
openai_session_affinity_ttl_hours: 24, openai_session_affinity_ttl_hours: 24,
responses_max_decompressed_body_bytes: 64 * 1024 * 1024, responses_max_decompressed_body_bytes: 64 * 1024 * 1024,
smart_scheduling: gateway_core::account::SmartSchedulingConfig::default(), smart_scheduling: gateway_core::account::SmartSchedulingConfig::default(),
quota_reset_window_weights: gateway_core::account::ResetWindowWeights::default(),
min_codex_desktop_version: None, min_codex_desktop_version: None,
min_codex_cli_version: None, min_codex_cli_version: None,
usage_retention_days: 30, usage_retention_days: 30,
@@ -51,6 +51,7 @@ fn runtime_settings(enabled: bool, probe_enabled: bool) -> RuntimeSettings {
openai_session_affinity_ttl_hours: 24, openai_session_affinity_ttl_hours: 24,
responses_max_decompressed_body_bytes: 64 * 1024 * 1024, responses_max_decompressed_body_bytes: 64 * 1024 * 1024,
smart_scheduling: gateway_core::account::SmartSchedulingConfig::default(), smart_scheduling: gateway_core::account::SmartSchedulingConfig::default(),
quota_reset_window_weights: gateway_core::account::ResetWindowWeights::default(),
min_codex_desktop_version: None, min_codex_desktop_version: None,
min_codex_cli_version: None, min_codex_cli_version: None,
usage_retention_days: 31, usage_retention_days: 31,
@@ -1432,6 +1432,7 @@ impl SettingsStore for FixtureSettingsStore {
openai_session_affinity_ttl_hours: 24, openai_session_affinity_ttl_hours: 24,
responses_max_decompressed_body_bytes: 64 * 1024 * 1024, responses_max_decompressed_body_bytes: 64 * 1024 * 1024,
smart_scheduling: gateway_core::account::SmartSchedulingConfig::default(), smart_scheduling: gateway_core::account::SmartSchedulingConfig::default(),
quota_reset_window_weights: gateway_core::account::ResetWindowWeights::default(),
min_codex_desktop_version: None, min_codex_desktop_version: None,
min_codex_cli_version: None, min_codex_cli_version: None,
usage_retention_days: 31, usage_retention_days: 31,
@@ -4,7 +4,7 @@ use async_trait::async_trait;
use gateway_admin::{ use gateway_admin::{
model::{MutationContext, Revision, proxies::*}, model::{MutationContext, Revision, proxies::*},
ports::{ ports::{
proxy::{ProxyProbe, ProxyStore}, proxy::{ProbeRoute, ProxyProbe, ProxyStore},
store::AdminStoreResult, store::AdminStoreResult,
}, },
}; };
@@ -194,7 +194,7 @@ async fn plugin_proxy_resolution_requires_enabled_configuration_without_account_
#[async_trait] #[async_trait]
impl ProxyProbe for TestProxies { impl ProxyProbe for TestProxies {
async fn test(&self, _: Option<&OutboundProxy>, _: bool) -> ProxyTestResult { async fn test(&self, _: ProbeRoute<'_>, _: bool) -> ProxyTestResult {
panic!("unexpected proxy probe") panic!("unexpected proxy probe")
} }
} }
@@ -456,7 +456,7 @@ impl Drop for ActiveProbe<'_> {
#[async_trait] #[async_trait]
impl ProxyProbe for GatedProxyProbe { impl ProxyProbe for GatedProxyProbe {
async fn test(&self, _: Option<&OutboundProxy>, _: bool) -> ProxyTestResult { async fn test(&self, _: ProbeRoute<'_>, _: bool) -> ProxyTestResult {
self.active self.active
.fetch_add(1, std::sync::atomic::Ordering::Relaxed); .fetch_add(1, std::sync::atomic::Ordering::Relaxed);
let _active = ActiveProbe(&self.active); let _active = ActiveProbe(&self.active);
@@ -526,8 +526,8 @@ async fn batch_tests_share_capacity_preserve_order_and_release_slots_on_cancella
_ = async { receiver.recv().await.unwrap(); receiver.recv().await.unwrap(); } => {} _ = async { receiver.recv().await.unwrap(); receiver.recv().await.unwrap(); } => {}
} }
assert_eq!(probe.active.load(Ordering::Relaxed), 2); assert_eq!(probe.active.load(Ordering::Relaxed), 2);
let mut single_a = Box::pin(services.proxies().probe(None, false)); let mut single_a = Box::pin(services.proxies().probe(ProbeRoute::Direct, false));
let mut single_b = Box::pin(services.proxies().probe(None, false)); let mut single_b = Box::pin(services.proxies().probe(ProbeRoute::Direct, false));
tokio::select! { tokio::select! {
_ = &mut batch => panic!("batch unexpectedly finished"), _ = &mut batch => panic!("batch unexpectedly finished"),
_ = &mut single_a => panic!("single probe unexpectedly finished"), _ = &mut single_a => panic!("single probe unexpectedly finished"),
@@ -538,7 +538,7 @@ async fn batch_tests_share_capacity_preserve_order_and_release_slots_on_cancella
assert_eq!( assert_eq!(
services services
.proxies() .proxies()
.probe(None, false) .probe(ProbeRoute::Direct, false)
.await .await
.unwrap_err() .unwrap_err()
.kind(), .kind(),
@@ -100,7 +100,9 @@ fn contract_type(mut item: Item, name: &str) -> Item {
attrs.retain(|attr| attr.path().is_ident("serde")); attrs.retain(|attr| attr.path().is_ident("serde"));
attrs.insert( attrs.insert(
0, 0,
if name == "SmartSchedulingConfig" { if name == "ResetWindowWeights" {
parse_quote!(#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)])
} else if name == "SmartSchedulingConfig" {
parse_quote!(#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]) parse_quote!(#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)])
} else if comparable { } else if comparable {
parse_quote!(#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]) parse_quote!(#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)])
@@ -255,8 +257,8 @@ fn sdk_settings_contract_matches_host_declarations() {
&["RequestLocation"][..], &["RequestLocation"][..],
), ),
( (
"../gateway-core/src/account/smart_scheduling.rs", "../gateway-core/src/account/scheduling.rs",
&["SmartSchedulingValues"][..], &["SmartSchedulingValues", "ResetWindowValues"][..],
), ),
( (
"../gateway-core/src/metering/pricing.rs", "../gateway-core/src/metering/pricing.rs",
@@ -324,10 +326,10 @@ fn sdk_settings_contract_matches_host_declarations() {
} }
types.push(contract_type( types.push(contract_type(
item, item,
if *name == "SmartSchedulingValues" { match *name {
"SmartSchedulingConfig" "SmartSchedulingValues" => "SmartSchedulingConfig",
} else { "ResetWindowValues" => "ResetWindowWeights",
name _ => name,
}, },
)); ));
} }
@@ -343,6 +343,8 @@ async fn settings_should_reject_zero_refresh_margin_before_store_call() {
openai_session_affinity_ttl_hours: 24, openai_session_affinity_ttl_hours: 24,
responses_max_decompressed_body_bytes: 64 * 1024 * 1024, responses_max_decompressed_body_bytes: 64 * 1024 * 1024,
smart_scheduling: gateway_core::account::SmartSchedulingConfig::default(), smart_scheduling: gateway_core::account::SmartSchedulingConfig::default(),
quota_reset_window_weights: gateway_core::account::ResetWindowWeights::default(
),
min_codex_desktop_version: None, min_codex_desktop_version: None,
min_codex_cli_version: None, min_codex_cli_version: None,
usage_retention_days: 31, usage_retention_days: 31,
@@ -101,6 +101,8 @@ struct TestRequest {
#[derive(Debug, Deserialize)] #[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)] #[serde(rename_all = "camelCase", deny_unknown_fields)]
struct ProbeRequest { struct ProbeRequest {
#[serde(default)]
use_deployment_proxy: bool,
#[serde(default)] #[serde(default)]
detect_location: bool, detect_location: bool,
proxy_url: AccountProxyUpdate, proxy_url: AccountProxyUpdate,
@@ -537,10 +539,21 @@ async fn probe<S>(
where where
S: SessionState + Send + Sync, S: SessionState + Send + Sync,
{ {
use gateway_admin::ports::proxy::ProbeRoute;
let route = match (request.proxy_url.0.as_ref(), request.use_deployment_proxy) {
(Some(_), true) => {
return Err(AdminError::bad_request(
"指定代理地址时不能同时选择部署默认出口",
));
}
(Some(proxy), false) => ProbeRoute::Proxy(proxy),
(None, true) => ProbeRoute::Deployment,
(None, false) => ProbeRoute::Direct,
};
let result = state let result = state
.admin_services() .admin_services()
.proxies() .proxies()
.probe(request.proxy_url.0.as_ref(), request.detect_location) .probe(route, request.detect_location)
.await .await
.map_err(map_error)?; .map_err(map_error)?;
Ok(AdminResponse::new( Ok(AdminResponse::new(
@@ -54,6 +54,7 @@ pub struct RuntimeSettingsFields {
pub openai_session_affinity_ttl_hours: u32, pub openai_session_affinity_ttl_hours: u32,
pub responses_max_decompressed_body_bytes: u64, pub responses_max_decompressed_body_bytes: u64,
pub smart_scheduling: gateway_core::account::SmartSchedulingConfig, pub smart_scheduling: gateway_core::account::SmartSchedulingConfig,
pub quota_reset_window_weights: gateway_core::account::ResetWindowWeights,
pub min_codex_desktop_version: Option<String>, pub min_codex_desktop_version: Option<String>,
pub min_codex_cli_version: Option<String>, pub min_codex_cli_version: Option<String>,
pub usage_retention_days: u64, pub usage_retention_days: u64,
@@ -77,6 +78,7 @@ pub struct RuntimeSettingsView {
pub values: RuntimeSettingsFields, pub values: RuntimeSettingsFields,
pub config_revision: u64, pub config_revision: u64,
pub smart_scheduling_defaults: gateway_core::account::SmartSchedulingConfig, pub smart_scheduling_defaults: gateway_core::account::SmartSchedulingConfig,
pub quota_reset_window_weights_defaults: gateway_core::account::ResetWindowWeights,
pub provider_request_profiles: ProviderRequestProfiles, pub provider_request_profiles: ProviderRequestProfiles,
/// 固定兼容字段;值始终从 provider_request_profiles 派生 /// 固定兼容字段;值始终从 provider_request_profiles 派生
pub openai_client_profile: Option<serde_json::Map<String, serde_json::Value>>, pub openai_client_profile: Option<serde_json::Map<String, serde_json::Value>>,
@@ -150,6 +152,8 @@ impl From<(RuntimeSettings, crate::time::TimePresenter)> for RuntimeSettingsView
provider_request_profiles, provider_request_profiles,
model_mappings: wire_model_mappings(settings.model_mappings), model_mappings: wire_model_mappings(settings.model_mappings),
smart_scheduling_defaults: gateway_core::account::SmartSchedulingConfig::default(), smart_scheduling_defaults: gateway_core::account::SmartSchedulingConfig::default(),
quota_reset_window_weights_defaults: gateway_core::account::ResetWindowWeights::default(
),
rotation_strategy: settings.rotation_strategy.as_str().to_owned(), rotation_strategy: settings.rotation_strategy.as_str().to_owned(),
updated_at_display: time.datetime(&settings.updated_at), updated_at_display: time.datetime(&settings.updated_at),
updated_at: settings.updated_at, updated_at: settings.updated_at,
@@ -658,7 +662,7 @@ where
let result = state let result = state
.admin_services() .admin_services()
.proxies() .proxies()
.probe(None, true) .probe(gateway_admin::ports::proxy::ProbeRoute::Direct, true)
.await .await
.map_err(map_service_error)?; .map_err(map_service_error)?;
Ok(AdminResponse::new( Ok(AdminResponse::new(
@@ -765,6 +769,7 @@ impl RuntimeSettingsFields {
openai_session_affinity_ttl_hours: self.openai_session_affinity_ttl_hours, openai_session_affinity_ttl_hours: self.openai_session_affinity_ttl_hours,
responses_max_decompressed_body_bytes: self.responses_max_decompressed_body_bytes, responses_max_decompressed_body_bytes: self.responses_max_decompressed_body_bytes,
smart_scheduling: self.smart_scheduling, smart_scheduling: self.smart_scheduling,
quota_reset_window_weights: self.quota_reset_window_weights,
min_codex_desktop_version: self.min_codex_desktop_version, min_codex_desktop_version: self.min_codex_desktop_version,
min_codex_cli_version: self.min_codex_cli_version, min_codex_cli_version: self.min_codex_cli_version,
usage_retention_days: u32::try_from(self.usage_retention_days) usage_retention_days: u32::try_from(self.usage_retention_days)
@@ -808,6 +813,7 @@ impl From<gateway_admin::model::settings::RuntimeSettingsValues> for RuntimeSett
openai_session_affinity_ttl_hours: settings.openai_session_affinity_ttl_hours, openai_session_affinity_ttl_hours: settings.openai_session_affinity_ttl_hours,
responses_max_decompressed_body_bytes: settings.responses_max_decompressed_body_bytes, responses_max_decompressed_body_bytes: settings.responses_max_decompressed_body_bytes,
smart_scheduling: settings.smart_scheduling, smart_scheduling: settings.smart_scheduling,
quota_reset_window_weights: settings.quota_reset_window_weights,
min_codex_desktop_version: settings.min_codex_desktop_version, min_codex_desktop_version: settings.min_codex_desktop_version,
min_codex_cli_version: settings.min_codex_cli_version, min_codex_cli_version: settings.min_codex_cli_version,
usage_retention_days: u64::from(settings.usage_retention_days), usage_retention_days: u64::from(settings.usage_retention_days),
@@ -664,6 +664,7 @@ impl SettingsStore for MemorySettingsStore {
.values .values
.responses_max_decompressed_body_bytes, .responses_max_decompressed_body_bytes,
smart_scheduling: command.values.smart_scheduling, smart_scheduling: command.values.smart_scheduling,
quota_reset_window_weights: command.values.quota_reset_window_weights,
min_codex_desktop_version: command.values.min_codex_desktop_version, min_codex_desktop_version: command.values.min_codex_desktop_version,
min_codex_cli_version: command.values.min_codex_cli_version, min_codex_cli_version: command.values.min_codex_cli_version,
usage_retention_days: command.values.usage_retention_days, usage_retention_days: command.values.usage_retention_days,
@@ -1940,6 +1941,7 @@ fn test_runtime_settings() -> RuntimeSettings {
openai_session_affinity_ttl_hours: 24, openai_session_affinity_ttl_hours: 24,
responses_max_decompressed_body_bytes: 64 * 1024 * 1024, responses_max_decompressed_body_bytes: 64 * 1024 * 1024,
smart_scheduling: gateway_core::account::SmartSchedulingConfig::default(), smart_scheduling: gateway_core::account::SmartSchedulingConfig::default(),
quota_reset_window_weights: gateway_core::account::ResetWindowWeights::default(),
min_codex_desktop_version: None, min_codex_desktop_version: None,
min_codex_cli_version: None, min_codex_cli_version: None,
usage_retention_days: 31, usage_retention_days: 31,
@@ -11,11 +11,11 @@ use chrono::Utc;
use gateway_admin::{ use gateway_admin::{
model::{MutationContext, Revision, proxies::*}, model::{MutationContext, Revision, proxies::*},
ports::{ ports::{
proxy::{ProxyProbe, ProxyStore}, proxy::{ProbeRoute, ProxyProbe, ProxyStore},
store::{AdminStoreError, AdminStoreErrorKind, AdminStoreResult}, store::{AdminStoreError, AdminStoreErrorKind, AdminStoreResult},
}, },
}; };
use gateway_core::account::{OutboundProxy, ProviderAccountId}; use gateway_core::account::ProviderAccountId;
use serde_json::{Value, json}; use serde_json::{Value, json};
use tower::ServiceExt as _; use tower::ServiceExt as _;
@@ -348,17 +348,17 @@ pub(super) struct SuccessfulProbe;
#[async_trait] #[async_trait]
impl ProxyProbe for SuccessfulProbe { impl ProxyProbe for SuccessfulProbe {
async fn test(&self, proxy: Option<&OutboundProxy>, detect_location: bool) -> ProxyTestResult { async fn test(&self, proxy: ProbeRoute<'_>, detect_location: bool) -> ProxyTestResult {
if let Some(proxy) = proxy { if let ProbeRoute::Proxy(proxy) = proxy {
assert_eq!( assert_eq!(
proxy.expose_url(), proxy.expose_url(),
"http://test-user:private-password@proxy.example:8080/" "http://test-user:private-password@proxy.example:8080/"
); );
} }
let exit_ip = if proxy.is_some() { let exit_ip = match proxy {
"203.0.113.2" ProbeRoute::Proxy(_) => "203.0.113.2",
} else { ProbeRoute::Direct => "203.0.113.3",
"203.0.113.3" ProbeRoute::Deployment => "203.0.113.4",
}; };
ProxyTestResult { ProxyTestResult {
location: if detect_location { location: if detect_location {
@@ -600,6 +600,15 @@ async fn proxy_probe_checks_direct_egress_without_changing_saved_proxies() {
} }
); );
} }
let (status, deployed) = request(
&fixture,
"/api/admin/proxies/probe",
Some(json!({"proxyUrl": "", "useDeploymentProxy": true})),
true,
)
.await;
assert_eq!(status, StatusCode::OK);
assert_eq!(deployed["data"]["exitIpv4"], "203.0.113.4");
let (_, listed) = request(&fixture, "/api/admin/proxies", None, true).await; let (_, listed) = request(&fixture, "/api/admin/proxies", None, true).await;
assert_eq!(listed["data"]["page"]["total"], 1); assert_eq!(listed["data"]["page"]["total"], 1);
assert_eq!( assert_eq!(
@@ -613,6 +622,10 @@ async fn proxy_probe_rejects_missing_or_invalid_addresses() {
let fixture = AdminTestFixture::new().await; let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session"); fixture.auth.insert_session("valid-session");
for (body, expected) in [ for (body, expected) in [
(
json!({"proxyUrl": "http://proxy.example:8080", "useDeploymentProxy": true}),
StatusCode::BAD_REQUEST,
),
(json!({"proxyUrl": " "}), StatusCode::UNPROCESSABLE_ENTITY), (json!({"proxyUrl": " "}), StatusCode::UNPROCESSABLE_ENTITY),
(json!({"proxyUrl": null}), StatusCode::UNPROCESSABLE_ENTITY), (json!({"proxyUrl": null}), StatusCode::UNPROCESSABLE_ENTITY),
(json!({}), StatusCode::UNPROCESSABLE_ENTITY), (json!({}), StatusCode::UNPROCESSABLE_ENTITY),
@@ -63,6 +63,7 @@ fn update_body() -> Value {
"responsesMaxDecompressedBodyBytes": 67108864, "responsesMaxDecompressedBodyBytes": 67108864,
"rotationStrategy": "round_robin", "rotationStrategy": "round_robin",
"smartScheduling": gateway_core::account::SmartSchedulingConfig::default(), "smartScheduling": gateway_core::account::SmartSchedulingConfig::default(),
"quotaResetWindowWeights": gateway_core::account::ResetWindowWeights::default(),
"minCodexDesktopVersion": "26.825.6671", "minCodexDesktopVersion": "26.825.6671",
"minCodexCliVersion": "0.40.0", "minCodexCliVersion": "0.40.0",
"usageRetentionDays": 31, "usageRetentionDays": 31,
@@ -264,12 +265,14 @@ fn error_protection_requires_an_explicit_policy_in_settings_replacement() {
} }
#[tokio::test] #[tokio::test]
async fn smart_settings_round_trip_and_invalid_updates_leave_the_saved_value_intact() { async fn scheduling_settings_round_trip_and_invalid_updates_leave_the_saved_value_intact() {
let fixture = AdminTestFixture::new().await; let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session"); fixture.auth.insert_session("valid-session");
let mut body = update_body(); let mut body = update_body();
let custom = json!({"loadWeight": 2.0, "quotaWeight": 0.0, "healthWeight": 1.0, "latencyWeight": 0.5, "resetWeight": 1.2, "queueWeight": 2.3}); let custom = json!({"loadWeight": 2.0, "quotaWeight": 0.0, "healthWeight": 1.0, "latencyWeight": 0.5, "resetWeight": 1.2, "queueWeight": 2.3, "resetWindowWeights":{"fiveHour":0.0,"weekly":1.5,"monthly":0.0}});
body["smartScheduling"] = custom.clone(); body["smartScheduling"] = custom.clone();
let windows = json!({"fiveHour":2.5,"weekly":0.0,"monthly":1.0});
body["quotaResetWindowWeights"] = windows.clone();
let response = app(fixture.state()) let response = app(fixture.state())
.oneshot(request( .oneshot(request(
Method::POST, Method::POST,
@@ -281,6 +284,7 @@ async fn smart_settings_round_trip_and_invalid_updates_leave_the_saved_value_int
assert_eq!(response.status(), StatusCode::OK); assert_eq!(response.status(), StatusCode::OK);
let response = response_json(response).await; let response = response_json(response).await;
assert_eq!(response["data"]["smartScheduling"], custom); assert_eq!(response["data"]["smartScheduling"], custom);
assert_eq!(response["data"]["quotaResetWindowWeights"], windows);
assert_eq!( assert_eq!(
response["data"]["smartSchedulingDefaults"], response["data"]["smartSchedulingDefaults"],
json!(gateway_core::account::SmartSchedulingConfig::default()) json!(gateway_core::account::SmartSchedulingConfig::default())
@@ -301,36 +305,61 @@ async fn smart_settings_round_trip_and_invalid_updates_leave_the_saved_value_int
missing.as_object_mut().unwrap().remove(field); missing.as_object_mut().unwrap().remove(field);
invalid_values.push(missing); invalid_values.push(missing);
} }
for invalid in invalid_values { let invalid_windows = [
body["smartScheduling"] = invalid; json!({}),
json!({"weekly":1}),
json!(null),
json!({"fiveHour":0,"weekly":1.23,"monthly":0}),
json!({"fiveHour":-1,"weekly":1,"monthly":0}),
json!({"fiveHour":0,"weekly":1,"monthly":10.1}),
json!({"fiveHour":0,"weekly":1,"monthly":0,"daily":1}),
];
for windows in &invalid_windows {
let mut invalid = custom.clone();
invalid["resetWindowWeights"] = windows.clone();
invalid_values.push(invalid);
}
for (field, invalid) in invalid_values
.into_iter()
.map(|value| ("smartScheduling", value))
.chain(
invalid_windows
.into_iter()
.map(|value| ("quotaResetWindowWeights", value)),
)
{
let mut invalid_body = body.clone();
invalid_body[field] = invalid;
let response = app(fixture.state()) let response = app(fixture.state())
.oneshot(request( .oneshot(request(
Method::POST, Method::POST,
"/api/admin/settings/update", "/api/admin/settings/update",
Some(body.clone()), Some(invalid_body),
))
.await
.unwrap();
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
}
for field in ["smartScheduling", "quotaResetWindowWeights"] {
let mut missing = body.clone();
missing.as_object_mut().unwrap().remove(field);
let response = app(fixture.state())
.oneshot(request(
Method::POST,
"/api/admin/settings/update",
Some(missing),
)) ))
.await .await
.unwrap(); .unwrap();
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY); assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
} }
body.as_object_mut().unwrap().remove("smartScheduling");
let response = app(fixture.state())
.oneshot(request(
Method::POST,
"/api/admin/settings/update",
Some(body),
))
.await
.unwrap();
assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY);
let response = app(fixture.state()) let response = app(fixture.state())
.oneshot(request(Method::GET, "/api/admin/settings", None)) .oneshot(request(Method::GET, "/api/admin/settings", None))
.await .await
.unwrap(); .unwrap();
assert_eq!( let saved = response_json(response).await;
response_json(response).await["data"]["smartScheduling"], assert_eq!(saved["data"]["smartScheduling"], custom);
custom assert_eq!(saved["data"]["quotaResetWindowWeights"], windows);
);
} }
#[test] #[test]
@@ -344,11 +373,11 @@ fn settings_request_should_reject_unknown_rotation_strategy() {
} }
#[tokio::test] #[tokio::test]
async fn short_request_retention_round_trips_and_invalid_days_do_not_replace_it() { async fn request_retention_round_trips_and_invalid_days_do_not_replace_it() {
let fixture = AdminTestFixture::new().await; let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session"); fixture.auth.insert_session("valid-session");
let mut body = update_body(); let mut body = update_body();
for days in [1, 3, 7, 15, 30, 31] { for days in [1, 31, 32, 365, 3650, u32::MAX] {
body["usageRetentionDays"] = json!(days); body["usageRetentionDays"] = json!(days);
let response = app(fixture.state()) let response = app(fixture.state())
.oneshot(request( .oneshot(request(
@@ -365,8 +394,7 @@ async fn short_request_retention_round_trips_and_invalid_days_do_not_replace_it(
} }
for (days, expected_status) in [ for (days, expected_status) in [
(json!(0), StatusCode::BAD_REQUEST), (json!(0), StatusCode::BAD_REQUEST),
(json!(32), StatusCode::BAD_REQUEST), (json!(u64::from(u32::MAX) + 1), StatusCode::BAD_REQUEST),
(json!(u32::MAX), StatusCode::BAD_REQUEST),
(json!(-1), StatusCode::UNPROCESSABLE_ENTITY), (json!(-1), StatusCode::UNPROCESSABLE_ENTITY),
(json!(1.5), StatusCode::UNPROCESSABLE_ENTITY), (json!(1.5), StatusCode::UNPROCESSABLE_ENTITY),
(json!(null), StatusCode::UNPROCESSABLE_ENTITY), (json!(null), StatusCode::UNPROCESSABLE_ENTITY),
@@ -389,7 +417,7 @@ async fn short_request_retention_round_trips_and_invalid_days_do_not_replace_it(
assert_eq!(response.status(), StatusCode::OK); assert_eq!(response.status(), StatusCode::OK);
assert_eq!( assert_eq!(
response_json(response).await["data"]["usageRetentionDays"], response_json(response).await["data"]["usageRetentionDays"],
31 u32::MAX
); );
} }
@@ -410,7 +438,12 @@ fn settings_request_validates_cron_and_allows_disabled_empty_schedule() {
(false, "08:00", false), (false, "08:00", false),
(true, "0 8 * * MON-FRI", true), (true, "0 8 * * MON-FRI", true),
(true, "60 8 * * *", false), (true, "60 8 * * *", false),
(true, "0 8 * * *;30 13 * * *", true),
(true, "0 8 * * *\n30 13 * * *", false), (true, "0 8 * * *\n30 13 * * *", false),
(true, "0 8 * * *;60 13 * * *", false),
(false, "0 8 * * *;", false),
(true, "0 8 * * *;;30 13 * * *", false),
(true, "0 8 * * *\r\n30 13 * * *", false),
] { ] {
let mut body = update_body(); let mut body = update_body();
body["accountWarmupEnabled"] = json!(enabled); body["accountWarmupEnabled"] = json!(enabled);
@@ -423,6 +456,11 @@ fn settings_request_validates_cron_and_allows_disabled_empty_schedule() {
assert_eq!(error.field(), "accountWarmupCron"); assert_eq!(error.field(), "accountWarmupCron");
} }
} }
let mut oversized = update_body();
oversized["accountWarmupCron"] = json!(["* * * * *"; 26].join(";"));
let request: UpdateRuntimeSettingsRequest = serde_json::from_value(oversized).unwrap();
assert_eq!(request.validate().unwrap_err().field(), "accountWarmupCron");
let mut legacy = update_body(); let mut legacy = update_body();
legacy.as_object_mut().unwrap().remove("accountWarmupCron"); legacy.as_object_mut().unwrap().remove("accountWarmupCron");
legacy["accountWarmupScheduleTime"] = json!("08:00"); legacy["accountWarmupScheduleTime"] = json!("08:00");
@@ -4,6 +4,7 @@ use axum::{
body::{Body, to_bytes}, body::{Body, to_bytes},
http::{Request, StatusCode}, http::{Request, StatusCode},
}; };
use base64::{Engine as _, engine::general_purpose::STANDARD};
use bytes::Bytes; use bytes::Bytes;
use futures::{StreamExt as _, future::BoxFuture}; use futures::{StreamExt as _, future::BoxFuture};
use gateway_core::{ use gateway_core::{
@@ -122,10 +123,12 @@ async fn native_json_preserves_extensions_and_uses_client_key_authentication() {
)]); )]);
let trace = session.trace.clone(); let trace = session.trace.clone();
let (router, execution) = fixture(session).await; let (router, execution) = fixture(session).await;
let response = router let mut incoming = request("/v1/messages", false);
.oneshot(request("/v1/messages", false)) incoming.headers_mut().insert(
.await "x-claude-code-session-id",
.unwrap(); "claude-session".parse().unwrap(),
);
let response = router.oneshot(incoming).await.unwrap();
assert_eq!(response.status(), StatusCode::OK); assert_eq!(response.status(), StatusCode::OK);
let bytes = to_bytes(response.into_body(), usize::MAX).await.unwrap(); let bytes = to_bytes(response.into_body(), usize::MAX).await.unwrap();
assert_eq!(bytes, raw); assert_eq!(bytes, raw);
@@ -136,6 +139,15 @@ async fn native_json_preserves_extensions_and_uses_client_key_authentication() {
panic!("generate") panic!("generate")
}; };
assert_eq!(generation.protocol_payload().protocol(), "anthropic"); assert_eq!(generation.protocol_payload().protocol(), "anthropic");
assert!(
generation.protocol_payload().context()["opaque_request_headers"]
.as_array()
.unwrap()
.contains(&json!([
"x-claude-code-session-id",
STANDARD.encode("claude-session")
]))
);
assert_eq!( assert_eq!(
generation.protocol_payload().body()["future_extension"]["preserved"], generation.protocol_payload().body()["future_extension"]["preserved"],
true true
@@ -17,17 +17,19 @@ mod quota_limits;
pub use quota_limits::{AccountQuotaLimits, QuotaMergeBase, QuotaPauseDecision}; pub use quota_limits::{AccountQuotaLimits, QuotaMergeBase, QuotaPauseDecision};
mod proxy; mod proxy;
pub use proxy::{InvalidOutboundProxy, OutboundProxy}; pub use proxy::{InvalidOutboundProxy, OutboundProxy};
mod scheduling;
pub mod scope; pub mod scope;
mod selection; mod selection;
mod smart_scheduling;
mod store; mod store;
mod upstream; mod upstream;
pub use error::CredentialError; pub use error::CredentialError;
pub use model::*; pub use model::*;
pub use scheduling::{
QuotaResetPeriod, ResetWindowWeights, SchedulingConfigError, SmartSchedulingConfig,
};
pub(crate) use selection::smart_score; pub(crate) use selection::smart_score;
pub use selection::*; pub use selection::*;
pub use smart_scheduling::{SmartSchedulingConfig, SmartSchedulingConfigError};
pub use store::{ pub use store::{
AccountCatalogObservation, AccountCatalogOmissions, ProviderAccountStore, StoredAccountCatalog, AccountCatalogObservation, AccountCatalogOmissions, ProviderAccountStore, StoredAccountCatalog,
}; };
@@ -0,0 +1,256 @@
//! 账号调度参数与共享重置窗口评分;固定十分位保证快照可精确比较,浮点数只用于 wire 与评分
use std::time::SystemTime;
use super::AccountRuntimeSignals;
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
#[serde(into = "SmartSchedulingValues")]
pub struct SmartSchedulingConfig {
weights: [u8; 6],
reset_windows: ResetWindowWeights,
}
/// 账号通用额度的周期;Provider 按上游周期事实投影,不依赖窗口名称和槽位
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum QuotaResetPeriod {
FiveHour,
Weekly,
Monthly,
}
impl QuotaResetPeriod {
const fn seconds(self) -> u32 {
match self {
Self::FiveHour => 18_000,
Self::Weekly => 604_800,
Self::Monthly => 2_592_000,
}
}
}
fn weight_tenths(value: f64, field: &'static str) -> Result<u8, SchedulingConfigError> {
if !(0.0..=10.0).contains(&value) || (value * 10.0).round() / 10.0 != value {
return Err(SchedulingConfigError::InvalidWeight(field));
}
Ok((value * 10.0).round() as u8)
}
impl<'de> Deserialize<'de> for SmartSchedulingConfig {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
// flatten 会缓冲字段,arbitrary_precision 的小数需先恢复为 JSON Number
// 随后仍由同一类型检查完整字段、数值范围和十分位精度
let value = serde_json::Value::deserialize(deserializer)?;
let values = serde_json::from_value::<SmartSchedulingValues>(value)
.map_err(serde::de::Error::custom)?;
Self::try_from(values).map_err(serde::de::Error::custom)
}
}
impl Default for SmartSchedulingConfig {
fn default() -> Self {
Self {
weights: [10, 8, 10, 5, 0, 0],
reset_windows: ResetWindowWeights::default(),
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
pub enum SchedulingConfigError {
#[error("{0} must be between 0 and 10 with at most one decimal place")]
InvalidWeight(&'static str),
#[error("at least one smart scheduling weight must be positive")]
EmptyWeights,
}
impl SmartSchedulingConfig {
/// 顺序为负载、剩余额度、健康、首输出延迟、额度重置和排队压力
pub fn new(weights: [f64; 6]) -> Result<Self, SchedulingConfigError> {
let mut tenths = [0; 6];
for (index, field) in [
"loadWeight",
"quotaWeight",
"healthWeight",
"latencyWeight",
"resetWeight",
"queueWeight",
]
.into_iter()
.enumerate()
{
tenths[index] = weight_tenths(weights[index], field)?;
}
if tenths == [0; 6] {
return Err(SchedulingConfigError::EmptyWeights);
}
Ok(Self {
weights: tenths,
..Self::default()
})
}
#[must_use]
pub fn weights(self) -> [f64; 6] {
self.weights.map(|weight| f64::from(weight) / 10.0)
}
fn reset_enabled(self) -> bool {
self.weights[4] > 0 && self.reset_windows.enabled()
}
pub(crate) fn reset_score(self, signals: &AccountRuntimeSignals, now: SystemTime) -> f64 {
if !self.reset_enabled() {
return 0.0;
}
self.reset_windows.score(signals, now)
}
pub(crate) fn score_tolerance(self) -> f64 {
// 与系数同步缩放,避免仅放大相同比例的系数就改变近似最优候选集合
// 排队和未启用的重置窗口不能放大立即选号的容差
let count = if self.reset_enabled() { 5 } else { 4 };
0.05 * f64::from(
self.weights[..count]
.iter()
.copied()
.map(u16::from)
.sum::<u16>(),
) / 33.0
}
}
#[derive(Serialize, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
struct SmartSchedulingValues {
load_weight: f64,
quota_weight: f64,
health_weight: f64,
latency_weight: f64,
reset_weight: f64,
queue_weight: f64,
#[serde(default)]
reset_window_weights: ResetWindowWeights,
}
#[derive(Default, Serialize, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
struct ResetWindowValues {
five_hour: f64,
weekly: f64,
monthly: f64,
}
impl TryFrom<SmartSchedulingValues> for SmartSchedulingConfig {
type Error = SchedulingConfigError;
fn try_from(value: SmartSchedulingValues) -> Result<Self, Self::Error> {
let mut config = Self::new([
value.load_weight,
value.quota_weight,
value.health_weight,
value.latency_weight,
value.reset_weight,
value.queue_weight,
])?;
config.reset_windows = value.reset_window_weights;
Ok(config)
}
}
impl From<SmartSchedulingConfig> for SmartSchedulingValues {
fn from(value: SmartSchedulingConfig) -> Self {
let [
load_weight,
quota_weight,
health_weight,
latency_weight,
reset_weight,
queue_weight,
] = value.weights();
Self {
load_weight,
quota_weight,
health_weight,
latency_weight,
reset_weight,
queue_weight,
reset_window_weights: value.reset_windows,
}
}
}
/// 两种调度策略共用的重置窗口系数,各策略独立持有配置
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize)]
#[serde(into = "ResetWindowValues")]
pub struct ResetWindowWeights {
weights: [u8; 3],
}
impl ResetWindowWeights {
/// 顺序为 5 小时、周、月;全零表示未指定窗口
pub fn new(weights: [f64; 3]) -> Result<Self, SchedulingConfigError> {
Ok(Self {
weights: [
weight_tenths(weights[0], "fiveHour")?,
weight_tenths(weights[1], "weekly")?,
weight_tenths(weights[2], "monthly")?,
],
})
}
pub(crate) fn enabled(self) -> bool {
self.weights != [0; 3]
}
pub(crate) fn score(self, signals: &AccountRuntimeSignals, now: SystemTime) -> f64 {
if !self.enabled() {
return 0.0;
}
let mut scores = [0.0_f64; 3];
for window in signals.quota_reset_windows.as_deref().unwrap_or_default() {
let Ok(remaining) = window.reset_at.duration_since(now) else {
continue;
};
if remaining.is_zero() {
continue;
}
let half = f64::from(window.period.seconds()) / 7.0;
let score = half / (half + remaining.as_secs_f64());
// 同一周期只计最近的有效重置,避免重复额度桶叠加得分
let slot = &mut scores[window.period as usize];
*slot = slot.max(score);
}
// 缺失周期得零分,分母固定,避免少报窗口反而提高排名
let weights = self.weights.map(f64::from);
scores
.into_iter()
.zip(weights)
.map(|(score, w)| score * w)
.sum::<f64>()
/ weights.into_iter().sum::<f64>()
}
}
impl<'de> Deserialize<'de> for ResetWindowWeights {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
// 与智能调度一致,先恢复 flatten 缓冲的小数再校验
let value = serde_json::Value::deserialize(deserializer)?;
let values =
serde_json::from_value::<ResetWindowValues>(value).map_err(serde::de::Error::custom)?;
Self::new([values.five_hour, values.weekly, values.monthly])
.map_err(serde::de::Error::custom)
}
}
impl From<ResetWindowWeights> for ResetWindowValues {
fn from(value: ResetWindowWeights) -> Self {
let [five_hour, weekly, monthly] = value.weights.map(|weight| f64::from(weight) / 10.0);
Self {
five_hour,
weekly,
monthly,
}
}
}
@@ -3,7 +3,7 @@
use std::cmp::Reverse; use std::cmp::Reverse;
use std::collections::{BTreeSet, HashMap}; use std::collections::{BTreeSet, HashMap};
use std::sync::atomic::{AtomicU64, Ordering}; use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::{Mutex, RwLock}; use std::sync::{Arc, Mutex, RwLock};
use std::time::{Duration, Instant, SystemTime}; use std::time::{Duration, Instant, SystemTime};
use futures::FutureExt; use futures::FutureExt;
@@ -12,7 +12,8 @@ use crate::concurrency::{CapacityWait, ConcurrencyQueuePolicy, QueueRejection};
use crate::identity::ProviderKind; use crate::identity::ProviderKind;
use super::{ use super::{
AccountConcurrency, AccountStatus, ProviderAccount, ProviderAccountId, SmartSchedulingConfig, AccountConcurrency, AccountStatus, ProviderAccount, ProviderAccountId, ResetWindowWeights,
SmartSchedulingConfig,
}; };
/// `runtime_settings.rotation_strategy` 的稳定值 /// `runtime_settings.rotation_strategy` 的稳定值
@@ -90,6 +91,7 @@ impl AccountAffinity {
pub struct AccountSelectionPolicy { pub struct AccountSelectionPolicy {
strategy: RotationStrategy, strategy: RotationStrategy,
smart_scheduling: SmartSchedulingConfig, smart_scheduling: SmartSchedulingConfig,
quota_reset_window_weights: ResetWindowWeights,
max_concurrent_per_account: AccountConcurrency, max_concurrent_per_account: AccountConcurrency,
queue_policy: ConcurrencyQueuePolicy, queue_policy: ConcurrencyQueuePolicy,
openai_guardian_reserved_concurrency: u32, openai_guardian_reserved_concurrency: u32,
@@ -108,6 +110,7 @@ impl AccountSelectionPolicy {
Self { Self {
strategy, strategy,
smart_scheduling: SmartSchedulingConfig::default(), smart_scheduling: SmartSchedulingConfig::default(),
quota_reset_window_weights: ResetWindowWeights::default(),
max_concurrent_per_account: max_concurrent_per_account.into(), max_concurrent_per_account: max_concurrent_per_account.into(),
openai_guardian_reserved_concurrency: 0, openai_guardian_reserved_concurrency: 0,
openai_account_affinity: AccountAffinity::default(), openai_account_affinity: AccountAffinity::default(),
@@ -121,6 +124,17 @@ impl AccountSelectionPolicy {
} }
} }
#[must_use]
pub const fn with_quota_reset_window_weights(mut self, weights: ResetWindowWeights) -> Self {
self.quota_reset_window_weights = weights;
self
}
#[must_use]
pub const fn quota_reset_window_weights(self) -> ResetWindowWeights {
self.quota_reset_window_weights
}
#[must_use] #[must_use]
pub const fn with_openai_account_affinity(mut self, affinity: AccountAffinity) -> Self { pub const fn with_openai_account_affinity(mut self, affinity: AccountAffinity) -> Self {
self.openai_account_affinity = affinity; self.openai_account_affinity = affinity;
@@ -245,6 +259,7 @@ pub struct AccountRuntimeSignals {
pub in_flight: u32, pub in_flight: u32,
pub last_started_at: Option<SystemTime>, pub last_started_at: Option<SystemTime>,
pub quota_reset_at: Option<SystemTime>, pub quota_reset_at: Option<SystemTime>,
pub quota_reset_windows: Option<Arc<[AccountQuotaResetWindow]>>,
/// Provider 归一化的剩余额度基点:0 耗尽,10_000 全部可用 /// Provider 归一化的剩余额度基点:0 耗尽,10_000 全部可用
pub quota_remaining_rank: Option<u64>, pub quota_remaining_rank: Option<u64>,
pub cooldown: Option<super::AccountCooldown>, pub cooldown: Option<super::AccountCooldown>,
@@ -459,6 +474,7 @@ impl AccountRuntimeSignals {
pub fn with_provider_quota(mut self, quota: Option<AccountQuotaSignals>) -> Self { pub fn with_provider_quota(mut self, quota: Option<AccountQuotaSignals>) -> Self {
if let Some(quota) = quota { if let Some(quota) = quota {
self.quota_reset_at = quota.reset_at; self.quota_reset_at = quota.reset_at;
self.quota_reset_windows = quota.reset_windows;
self.quota_remaining_rank = quota.remaining_rank; self.quota_remaining_rank = quota.remaining_rank;
} }
self self
@@ -482,9 +498,17 @@ impl AccountRuntimeSignals {
} }
} }
/// Provider 确认为账号通用且属于受支持周期的重置观测
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct AccountQuotaResetWindow {
pub period: super::QuotaResetPeriod,
pub reset_at: SystemTime,
}
/// Provider 从私有 quota JSON 投影出的中立调度事实 /// Provider 从私有 quota JSON 投影出的中立调度事实
#[derive(Debug, Clone, Copy, PartialEq, Eq)] #[derive(Debug, Clone, PartialEq, Eq)]
pub struct AccountQuotaSignals { pub struct AccountQuotaSignals {
reset_windows: Option<Arc<[AccountQuotaResetWindow]>>,
reset_at: Option<SystemTime>, reset_at: Option<SystemTime>,
remaining_rank: Option<u64>, remaining_rank: Option<u64>,
} }
@@ -496,16 +520,23 @@ impl AccountQuotaSignals {
Self { Self {
reset_at, reset_at,
remaining_rank, remaining_rank,
reset_windows: None,
} }
} }
#[must_use] #[must_use]
pub const fn reset_at(self) -> Option<SystemTime> { pub fn with_reset_windows(mut self, windows: Vec<AccountQuotaResetWindow>) -> Self {
self.reset_windows = (!windows.is_empty()).then(|| windows.into());
self
}
#[must_use]
pub const fn reset_at(&self) -> Option<SystemTime> {
self.reset_at self.reset_at
} }
#[must_use] #[must_use]
pub const fn remaining_rank(self) -> Option<u64> { pub const fn remaining_rank(&self) -> Option<u64> {
self.remaining_rank self.remaining_rank
} }
} }
@@ -782,24 +813,31 @@ impl AccountSelector {
let candidate = match context.policy.strategy() { let candidate = match context.policy.strategy() {
RotationStrategy::QuotaResetPriority => { RotationStrategy::QuotaResetPriority => {
eligible.sort_by(|left, right| { eligible.sort_by(|left, right| {
( let windows = context.policy.quota_reset_window_weights();
left.signals.quota_reset_at.is_none(), if windows.enabled() {
left.signals.quota_reset_at, windows
) .score(&right.signals, context.now)
.cmp(&( .total_cmp(&windows.score(&left.signals, context.now))
right.signals.quota_reset_at.is_none(), } else {
right.signals.quota_reset_at, (
)) left.signals.quota_reset_at.is_none(),
.then_with(|| { left.signals.quota_reset_at,
capacity_utilization(left, context) )
.total_cmp(&capacity_utilization(right, context)) .cmp(&(
}) right.signals.quota_reset_at.is_none(),
.then_with(|| { right.signals.quota_reset_at,
left.signals ))
.last_started_at }
.cmp(&right.signals.last_started_at) .then_with(|| {
}) capacity_utilization(left, context)
.then_with(|| left.account.id().cmp(right.account.id())) .total_cmp(&capacity_utilization(right, context))
})
.then_with(|| {
left.signals
.last_started_at
.cmp(&right.signals.last_started_at)
})
.then_with(|| left.account.id().cmp(right.account.id()))
}); });
eligible.first().copied()? eligible.first().copied()?
} }
@@ -940,8 +978,6 @@ impl AccountSelector {
// 首输出 10 秒时延迟得分减半;固定尺度不随其他候选账号变化 // 首输出 10 秒时延迟得分减半;固定尺度不随其他候选账号变化
const SMART_LATENCY_HALF_SCORE_MS: f64 = 10_000.0; const SMART_LATENCY_HALF_SCORE_MS: f64 = 10_000.0;
// 距重置一小时时得分减半;未知和已过期时间不提供重置奖励
const SMART_RESET_HALF_SCORE_SECONDS: f64 = 3_600.0;
fn capacity_utilization(candidate: &AccountCandidate, context: &AccountSelectionContext) -> f64 { fn capacity_utilization(candidate: &AccountCandidate, context: &AccountSelectionContext) -> f64 {
context context
@@ -997,15 +1033,6 @@ pub(crate) fn smart_score(candidate: &AccountCandidate, context: &AccountSelecti
SMART_LATENCY_HALF_SCORE_MS / (SMART_LATENCY_HALF_SCORE_MS + latency as f64) SMART_LATENCY_HALF_SCORE_MS / (SMART_LATENCY_HALF_SCORE_MS + latency as f64)
}); });
let reset = candidate
.signals
.quota_reset_at
.and_then(|reset| reset.duration_since(now).ok())
.filter(|remaining| !remaining.is_zero())
.map_or(0.0, |remaining| {
SMART_RESET_HALF_SCORE_SECONDS
/ (SMART_RESET_HALF_SCORE_SECONDS + remaining.as_secs_f64())
});
let [ let [
load_weight, load_weight,
quota_weight, quota_weight,
@@ -1018,5 +1045,5 @@ pub(crate) fn smart_score(candidate: &AccountCandidate, context: &AccountSelecti
+ quota_weight * quota + quota_weight * quota
+ health_weight * failure + health_weight * failure
+ latency_weight * latency + latency_weight * latency
+ reset_weight * reset + reset_weight * config.reset_score(&candidate.signals, now)
} }
@@ -1,131 +0,0 @@
//! 智能调度参数;固定十分位保证快照可精确比较,浮点数只用于 wire 与评分
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
#[serde(into = "SmartSchedulingValues")]
pub struct SmartSchedulingConfig {
weights: [u8; 6],
}
impl<'de> Deserialize<'de> for SmartSchedulingConfig {
fn deserialize<D: serde::Deserializer<'de>>(deserializer: D) -> Result<Self, D::Error> {
// flatten 会缓冲字段,arbitrary_precision 的小数需先恢复为 JSON Number
// 随后仍由同一类型检查完整字段、数值范围和十分位精度
let value = serde_json::Value::deserialize(deserializer)?;
let values = serde_json::from_value::<SmartSchedulingValues>(value)
.map_err(serde::de::Error::custom)?;
Self::try_from(values).map_err(serde::de::Error::custom)
}
}
impl Default for SmartSchedulingConfig {
fn default() -> Self {
Self {
weights: [10, 8, 10, 5, 0, 0],
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
pub enum SmartSchedulingConfigError {
#[error("{0} must be between 0 and 10 with at most one decimal place")]
InvalidWeight(&'static str),
#[error("at least one smart scheduling weight must be positive")]
EmptyWeights,
}
impl SmartSchedulingConfig {
/// 顺序为负载、剩余额度、健康、首输出延迟、额度重置和排队压力
pub fn new(weights: [f64; 6]) -> Result<Self, SmartSchedulingConfigError> {
let mut tenths = [0; 6];
for (index, field) in [
"loadWeight",
"quotaWeight",
"healthWeight",
"latencyWeight",
"resetWeight",
"queueWeight",
]
.into_iter()
.enumerate()
{
let value = weights[index];
if !value.is_finite()
|| !(0.0..=10.0).contains(&value)
|| (value * 10.0).round() / 10.0 != value
{
return Err(SmartSchedulingConfigError::InvalidWeight(field));
}
tenths[index] = (value * 10.0).round() as u8;
}
if tenths == [0; 6] {
return Err(SmartSchedulingConfigError::EmptyWeights);
}
Ok(Self { weights: tenths })
}
#[must_use]
pub fn weights(self) -> [f64; 6] {
self.weights.map(|weight| f64::from(weight) / 10.0)
}
pub(crate) fn score_tolerance(self) -> f64 {
// 与系数同步缩放,避免仅放大相同比例的系数就改变近似最优候选集合
// 排队系数只参与入队选择,不能放大立即选号的候选容差
0.05 * f64::from(
self.weights[..5]
.iter()
.copied()
.map(u16::from)
.sum::<u16>(),
) / 33.0
}
}
#[derive(Serialize, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
struct SmartSchedulingValues {
load_weight: f64,
quota_weight: f64,
health_weight: f64,
latency_weight: f64,
reset_weight: f64,
queue_weight: f64,
}
impl TryFrom<SmartSchedulingValues> for SmartSchedulingConfig {
type Error = SmartSchedulingConfigError;
fn try_from(value: SmartSchedulingValues) -> Result<Self, Self::Error> {
Self::new([
value.load_weight,
value.quota_weight,
value.health_weight,
value.latency_weight,
value.reset_weight,
value.queue_weight,
])
}
}
impl From<SmartSchedulingConfig> for SmartSchedulingValues {
fn from(value: SmartSchedulingConfig) -> Self {
let [
load_weight,
quota_weight,
health_weight,
latency_weight,
reset_weight,
queue_weight,
] = value.weights();
Self {
load_weight,
quota_weight,
health_weight,
latency_weight,
reset_weight,
queue_weight,
}
}
}
@@ -28,11 +28,10 @@ pub struct StoredAccountCatalog {
pub document: bytes::Bytes, pub document: bytes::Bytes,
} }
/// 批量选号的存储优化:只有账号自己的当前版本、有效时间和目录摘要都匹配才可跳过 /// 批量选号只按账号当前凭据和套餐的目录摘要排除候选
pub struct AccountCatalogOmissions { pub struct AccountCatalogOmissions {
pub documents: Vec<[u8; 32]>, pub documents: Vec<[u8; 32]>,
pub exceptions: Vec<ProviderAccountId>, pub exceptions: Vec<ProviderAccountId>,
pub observed_after: std::time::SystemTime,
pub as_of: std::time::SystemTime, pub as_of: std::time::SystemTime,
} }
@@ -5,6 +5,14 @@ use crate::{account::OpaqueProviderData, identity::ProviderKind, routing::Config
use futures::future::BoxFuture; use futures::future::BoxFuture;
pub trait ProviderRuntimePolicyPort: Send + Sync { pub trait ProviderRuntimePolicyPort: Send + Sync {
/// 后台模型操作读取 Provider 全局画像,不读取单个 Client Key 的覆盖
fn load_request_profile<'a>(
&'a self,
_provider: &'a ProviderKind,
) -> BoxFuture<'a, Result<Option<OpaqueProviderData>, ProviderStoreError>> {
Box::pin(async { Ok(None) })
}
/// 原子推进预热执行游标,重启或时钟回拨后不重复领取已消费的时刻 /// 原子推进预热执行游标,重启或时钟回拨后不重复领取已消费的时刻
fn claim_warmup_slot<'a>( fn claim_warmup_slot<'a>(
&'a self, &'a self,
@@ -52,7 +60,7 @@ pub trait ProviderRuntimePolicyPort: Send + Sync {
} }
} }
/// 校验五段 Cron 预热计划 /// 校验单条五段 Cron 表达式
#[must_use] #[must_use]
pub fn valid_warmup_cron(value: &str) -> bool { pub fn valid_warmup_cron(value: &str) -> bool {
parse_warmup_cron(value).is_some() parse_warmup_cron(value).is_some()
@@ -73,7 +81,7 @@ fn parse_warmup_cron(value: &str) -> Option<cron::Schedule> {
#[derive(Debug, Clone, PartialEq, Eq)] #[derive(Debug, Clone, PartialEq, Eq)]
pub struct ProviderWarmupPolicy { pub struct ProviderWarmupPolicy {
enabled: bool, enabled: bool,
cron: String, schedules: Vec<cron::Schedule>,
model: Option<String>, model: Option<String>,
} }
@@ -84,29 +92,47 @@ impl ProviderWarmupPolicy {
cron: &str, cron: &str,
model: Option<&str>, model: Option<&str>,
) -> Result<(), &'static str> { ) -> Result<(), &'static str> {
if (enabled || !cron.is_empty()) && !valid_warmup_cron(cron) { Self::validate_and_parse(enabled, cron, model).map(|_| ())
}
fn validate_and_parse(
enabled: bool,
cron: &str,
model: Option<&str>,
) -> Result<Vec<cron::Schedule>, &'static str> {
if cron.len() > 255 || (enabled && cron.is_empty()) {
return Err("account_warmup_cron"); return Err("account_warmup_cron");
} }
// 分号分隔独立表达式,逗号保留给 Cron 字段内的列表
let schedules = if cron.is_empty() {
Vec::new()
} else {
cron.split(';')
.map(parse_warmup_cron)
.collect::<Option<Vec<_>>>()
.ok_or("account_warmup_cron")?
};
if (enabled && model.is_none()) || !valid_optional_probe_model(model) { if (enabled && model.is_none()) || !valid_optional_probe_model(model) {
return Err("account_warmup_model"); return Err("account_warmup_model");
} }
Ok(()) Ok(schedules)
} }
pub fn try_new( pub fn try_new(
enabled: bool, enabled: bool,
cron: String, cron: &str,
model: Option<String>, model: Option<String>,
) -> Result<Self, ProviderStoreError> { ) -> Result<Self, ProviderStoreError> {
if Self::validate_values(enabled, &cron, model.as_deref()).is_err() { let schedules =
return Err(ProviderStoreError::new( Self::validate_and_parse(enabled, cron, model.as_deref()).map_err(|_| {
ProviderStoreErrorKind::InvalidData, ProviderStoreError::new(
"validate warmup policy", ProviderStoreErrorKind::InvalidData,
)); "validate warmup policy",
} )
})?;
Ok(Self { Ok(Self {
enabled, enabled,
cron, schedules,
model, model,
}) })
} }
@@ -115,7 +141,7 @@ impl ProviderWarmupPolicy {
pub const fn disabled() -> Self { pub const fn disabled() -> Self {
Self { Self {
enabled: false, enabled: false,
cron: String::new(), schedules: Vec::new(),
model: None, model: None,
} }
} }
@@ -125,11 +151,6 @@ impl ProviderWarmupPolicy {
self.enabled self.enabled
} }
#[must_use]
pub fn cron(&self) -> &str {
&self.cron
}
#[must_use] #[must_use]
pub fn model(&self) -> Option<&str> { pub fn model(&self) -> Option<&str> {
self.model.as_deref() self.model.as_deref()
@@ -138,7 +159,10 @@ impl ProviderWarmupPolicy {
/// 匹配部署时区中的整分钟时刻,时区重叠及执行去重由 Worker 与存储端口负责 /// 匹配部署时区中的整分钟时刻,时区重叠及执行去重由 Worker 与存储端口负责
#[must_use] #[must_use]
pub fn matches_slot(&self, slot: chrono::NaiveDateTime) -> bool { pub fn matches_slot(&self, slot: chrono::NaiveDateTime) -> bool {
parse_warmup_cron(&self.cron).is_some_and(|schedule| schedule.includes(slot.and_utc())) let instant = slot.and_utc();
self.schedules
.iter()
.any(|schedule| schedule.includes(instant))
} }
} }
@@ -1247,13 +1247,13 @@ impl RuntimeSnapshot {
operation: &Operation, operation: &Operation,
context: &RoutingContext, context: &RoutingContext,
) -> Result<RoutingPlan, RoutingError> { ) -> Result<RoutingPlan, RoutingError> {
self.plan_inner( // 探测没有 Client Key,沿用同一发布快照的全局画像
public_model, let scope = self
operation, .all_account_scope()
self.all_account_scope(), .as_ref()
context, .clone()
false, .with_request_profiles(self.settings().request_profiles().clone());
) self.plan_inner(public_model, operation, Arc::new(scope), context, false)
} }
fn plan_inner( fn plan_inner(
@@ -101,6 +101,7 @@ impl CompiledSettings {
u64::from(settings.openai_session_affinity_ttl_hours) * 3600, u64::from(settings.openai_session_affinity_ttl_hours) * 3600,
)) ))
.with_smart_scheduling(settings.smart_scheduling) .with_smart_scheduling(settings.smart_scheduling)
.with_quota_reset_window_weights(settings.quota_reset_window_weights)
.with_queue(ConcurrencyQueuePolicy { .with_queue(ConcurrencyQueuePolicy {
max_waiting: settings.max_waiting_per_account, max_waiting: settings.max_waiting_per_account,
timeout: Duration::from_secs(u64::from(settings.account_queue_timeout_seconds)), timeout: Duration::from_secs(u64::from(settings.account_queue_timeout_seconds)),
@@ -30,6 +30,8 @@ pub struct SettingsValues {
pub(crate) openai_session_affinity_ttl_hours: u32, pub(crate) openai_session_affinity_ttl_hours: u32,
pub(crate) responses_max_decompressed_body_bytes: u64, pub(crate) responses_max_decompressed_body_bytes: u64,
pub(crate) smart_scheduling: crate::account::SmartSchedulingConfig, pub(crate) smart_scheduling: crate::account::SmartSchedulingConfig,
#[serde(default)]
pub(crate) quota_reset_window_weights: crate::account::ResetWindowWeights,
pub(crate) rotation_strategy: String, pub(crate) rotation_strategy: String,
pub(crate) model_mappings: Arc<BTreeMap<String, String>>, pub(crate) model_mappings: Arc<BTreeMap<String, String>>,
pub(crate) min_codex_desktop_version: Option<String>, pub(crate) min_codex_desktop_version: Option<String>,
@@ -111,6 +113,15 @@ impl SettingsValues {
&self.request_profiles &self.request_profiles
} }
#[must_use]
pub const fn with_quota_reset_window_weights(
mut self,
weights: crate::account::ResetWindowWeights,
) -> Self {
self.quota_reset_window_weights = weights;
self
}
#[must_use] #[must_use]
pub const fn with_smart_scheduling( pub const fn with_smart_scheduling(
mut self, mut self,
@@ -201,6 +212,7 @@ impl SettingsValues {
openai_session_affinity_ttl_hours: 24, openai_session_affinity_ttl_hours: 24,
responses_max_decompressed_body_bytes: 64 * 1024 * 1024, responses_max_decompressed_body_bytes: 64 * 1024 * 1024,
smart_scheduling: crate::account::SmartSchedulingConfig::default(), smart_scheduling: crate::account::SmartSchedulingConfig::default(),
quota_reset_window_weights: crate::account::ResetWindowWeights::default(),
rotation_strategy: rotation_strategy.into(), rotation_strategy: rotation_strategy.into(),
model_mappings: Arc::new(model_mappings), model_mappings: Arc::new(model_mappings),
min_codex_desktop_version, min_codex_desktop_version,
@@ -3,6 +3,7 @@
mod location; mod location;
mod model_access; mod model_access;
mod proxy; mod proxy;
mod quota_reset_priority;
mod scope; mod scope;
mod selection; mod selection;
mod selection_diagnostics; mod selection_diagnostics;
@@ -56,6 +57,7 @@ fn candidate(id: &str, in_flight: u32, remaining: Option<u64>) -> AccountCandida
in_flight, in_flight,
last_started_at: None, last_started_at: None,
quota_reset_at: None, quota_reset_at: None,
quota_reset_windows: None,
quota_remaining_rank: remaining, quota_remaining_rank: remaining,
cooldown: None, cooldown: None,
failure_rate_basis_points: None, failure_rate_basis_points: None,
@@ -228,6 +230,7 @@ fn diagnostic_selection_bypasses_all_local_account_eligibility() {
in_flight: 0, in_flight: 0,
last_started_at: None, last_started_at: None,
quota_reset_at: None, quota_reset_at: None,
quota_reset_windows: None,
quota_remaining_rank: None, quota_remaining_rank: None,
cooldown: None, cooldown: None,
failure_rate_basis_points: None, failure_rate_basis_points: None,
@@ -247,6 +250,7 @@ fn diagnostic_selection_bypasses_all_local_account_eligibility() {
in_flight: 0, in_flight: 0,
last_started_at: None, last_started_at: None,
quota_reset_at: None, quota_reset_at: None,
quota_reset_windows: None,
quota_remaining_rank: None, quota_remaining_rank: None,
cooldown: None, cooldown: None,
failure_rate_basis_points: None, failure_rate_basis_points: None,
@@ -641,6 +645,7 @@ fn provider_quota_overlay_should_preserve_store_concurrency_facts() {
in_flight: 2, in_flight: 2,
last_started_at: Some(last_started_at), last_started_at: Some(last_started_at),
quota_reset_at: None, quota_reset_at: None,
quota_reset_windows: None,
quota_remaining_rank: None, quota_remaining_rank: None,
cooldown: None, cooldown: None,
failure_rate_basis_points: None, failure_rate_basis_points: None,
@@ -0,0 +1,135 @@
//! 验证重置窗口独立排序、相对系数及亲和与可用性边界
use std::time::Duration;
use gateway_core::account::{
AccountQuotaResetWindow, AccountSelector, QuotaResetPeriod, ResetWindowWeights,
RotationStrategy,
};
use super::{candidate, context};
#[test]
fn selected_windows_rank_strictly_without_mixing_runtime_scores_or_rotating_near_ties() {
let mut context = context(RotationStrategy::QuotaResetPriority);
let mut early_weekly = candidate("acct_early_weekly", 2, Some(100));
early_weekly.signals.failure_rate_basis_points = Some(9_000);
early_weekly.signals.first_output_latency_ms = Some(20_000);
let mut late_weekly = candidate("acct_late_weekly", 0, Some(9_900));
late_weekly.signals.failure_rate_basis_points = Some(0);
late_weekly.signals.first_output_latency_ms = Some(10);
for (candidate, short, weekly) in [
(&mut early_weekly, 14_400, 6 * 86_400),
(&mut late_weekly, 3_600, 6 * 86_400 + 43_200),
] {
candidate.signals.quota_reset_at = Some(context.now + Duration::from_secs(short));
candidate.signals.quota_reset_windows = Some(
[
(QuotaResetPeriod::FiveHour, short),
(QuotaResetPeriod::Weekly, weekly),
]
.map(|(period, seconds)| AccountQuotaResetWindow {
period,
reset_at: context.now + Duration::from_secs(seconds),
})
.into(),
);
}
let mut candidates = [early_weekly, late_weekly];
for (weights, expected) in [
([0.0, 0.0, 0.0], "acct_late_weekly"),
([1.0, 0.0, 0.0], "acct_late_weekly"),
([0.0, 0.1, 0.0], "acct_early_weekly"),
([0.0, 1.0, 0.0], "acct_early_weekly"),
([0.0, 10.0, 0.0], "acct_early_weekly"),
([1.0, 0.1, 0.0], "acct_late_weekly"),
([10.0, 1.0, 0.0], "acct_late_weekly"),
([0.1, 10.0, 0.0], "acct_early_weekly"),
] {
context.policy = context
.policy
.with_quota_reset_window_weights(ResetWindowWeights::new(weights).unwrap());
for cursor in 0..6 {
context.round_robin_cursor = cursor;
assert_eq!(
AccountSelector
.select(&candidates, &context)
.unwrap()
.candidate()
.account
.id()
.as_str(),
expected,
"weights {weights:?}, cursor {cursor}"
);
}
}
context.preferred_account = Some(candidates[1].account.id().clone());
let selected = AccountSelector.select(&candidates, &context).unwrap();
assert_eq!(
selected.candidate().account.id().as_str(),
"acct_late_weekly"
);
assert_eq!(
selected.preferred(),
gateway_core::account::PreferredAccountSelection::Hit
);
context.preferred_account = None;
candidates[0].signals.in_flight = 3;
assert_eq!(
AccountSelector
.select(&candidates, &context)
.unwrap()
.candidate()
.account
.id()
.as_str(),
"acct_late_weekly"
);
}
#[test]
fn missing_or_elapsed_selected_windows_do_not_fall_back_to_disabled_periods() {
let mut context = context(RotationStrategy::QuotaResetPriority);
context.policy = context
.policy
.with_quota_reset_window_weights(ResetWindowWeights::new([0.0, 0.0, 1.0]).unwrap());
let mut known = candidate("acct_known_monthly", 1, None);
known.signals.quota_reset_windows = Some(
vec![AccountQuotaResetWindow {
period: QuotaResetPeriod::Monthly,
reset_at: context.now + Duration::from_secs(30 * 86_400),
}]
.into(),
);
for reset_at in [
None,
Some(context.now),
Some(context.now - Duration::from_secs(1)),
] {
let mut missing = candidate("acct_earlier_other_period", 0, None);
missing.signals.quota_reset_at = Some(context.now + Duration::from_secs(1));
missing.signals.quota_reset_windows = Some(
reset_at
.into_iter()
.map(|reset_at| AccountQuotaResetWindow {
period: QuotaResetPeriod::Monthly,
reset_at,
})
.collect::<Vec<_>>()
.into(),
);
let candidates = [missing, known.clone()];
assert_eq!(
AccountSelector
.select(&candidates, &context)
.unwrap()
.candidate()
.account
.id()
.as_str(),
"acct_known_monthly"
);
}
}
@@ -1,7 +1,8 @@
//! 验证智能调度权重、软亲和与容量约束的组合行为 //! 验证智能调度权重、软亲和与容量约束的组合行为
use gateway_core::account::{ use gateway_core::account::{
AccountSelector, PreferredAccountSelection, RotationStrategy, SmartSchedulingConfig, AccountQuotaResetWindow, AccountSelector, PreferredAccountSelection, QuotaResetPeriod,
RotationStrategy, SmartSchedulingConfig,
}; };
use serde_json::json; use serde_json::json;
use std::time::{Duration, Instant, SystemTime}; use std::time::{Duration, Instant, SystemTime};
@@ -62,9 +63,10 @@ fn smart_config_rejects_invalid_weights_and_incomplete_wire_values() {
#[test] #[test]
fn reset_weight_rewards_only_future_resets_and_preserves_capacity_checks() { fn reset_weight_rewards_only_future_resets_and_preserves_capacity_checks() {
let mut context = context(RotationStrategy::Smart); let mut context = context(RotationStrategy::Smart);
context.policy = context context.policy = context.policy.with_smart_scheduling(serde_json::from_value(json!({
.policy "loadWeight":0, "quotaWeight":0, "healthWeight":0, "latencyWeight":0, "resetWeight":1, "queueWeight":0,
.with_smart_scheduling(SmartSchedulingConfig::new([0.0, 0.0, 0.0, 0.0, 1.0, 0.0]).unwrap()); "resetWindowWeights":{"fiveHour":0,"weekly":1,"monthly":0}
})).unwrap());
let mut candidates = [ let mut candidates = [
candidate_with_concurrency("acct_unknown", 0, 2), candidate_with_concurrency("acct_unknown", 0, 2),
candidate_with_concurrency("acct_past", 0, 2), candidate_with_concurrency("acct_past", 0, 2),
@@ -72,10 +74,20 @@ fn reset_weight_rewards_only_future_resets_and_preserves_capacity_checks() {
candidate_with_concurrency("acct_near", 0, 2), candidate_with_concurrency("acct_near", 0, 2),
candidate_with_concurrency("acct_far", 0, 2), candidate_with_concurrency("acct_far", 0, 2),
]; ];
candidates[1].signals.quota_reset_at = Some(context.now - Duration::from_secs(1)); for (candidate, reset_at) in candidates[1..].iter_mut().zip([
candidates[2].signals.quota_reset_at = Some(context.now); context.now - Duration::from_secs(1),
candidates[3].signals.quota_reset_at = Some(context.now + Duration::from_secs(3600)); context.now,
candidates[4].signals.quota_reset_at = Some(context.now + Duration::from_secs(86_400)); context.now + Duration::from_secs(3600),
context.now + Duration::from_secs(86_400),
]) {
candidate.signals.quota_reset_windows = Some(
[AccountQuotaResetWindow {
period: QuotaResetPeriod::Weekly,
reset_at,
}]
.into(),
);
}
assert_eq!( assert_eq!(
AccountSelector AccountSelector
.select(&candidates, &context) .select(&candidates, &context)
@@ -92,8 +104,8 @@ fn reset_weight_rewards_only_future_resets_and_preserves_capacity_checks() {
let scores = snapshot["events"][0]["data"]["candidates"] let scores = snapshot["events"][0]["data"]["candidates"]
.as_array() .as_array()
.unwrap(); .unwrap();
for (candidate, expected) in scores.iter().zip([0.0, 0.0, 0.0, 0.5, 0.04]) { for (candidate, expected) in scores.iter().zip([0.0, 0.0, 0.0, 0.96, 0.5]) {
assert_eq!(candidate["smartScore"].as_f64().unwrap(), expected); assert!((candidate["smartScore"].as_f64().unwrap() - expected).abs() < 1e-10);
} }
candidates[3].signals.in_flight = 2; candidates[3].signals.in_flight = 2;
assert_eq!( assert_eq!(
@@ -120,6 +132,36 @@ fn reset_weight_rewards_only_future_resets_and_preserves_capacity_checks() {
); );
} }
#[test]
fn zero_or_absent_window_weights_disable_reset_scores_and_tolerance() {
let mut context = context(RotationStrategy::Smart);
let mut candidate = candidate_with_concurrency("acct_reset", 0, 2);
candidate.signals.quota_reset_windows = Some(
[AccountQuotaResetWindow {
period: QuotaResetPeriod::Monthly,
reset_at: context.now + Duration::from_secs(1),
}]
.into(),
);
let legacy = json!({"loadWeight":0,"quotaWeight":0,"healthWeight":0,"latencyWeight":0,"resetWeight":10,"queueWeight":0});
let mut disabled = legacy.clone();
disabled["resetWindowWeights"] = json!({"fiveHour":0,"weekly":0,"monthly":0});
for wire in [legacy, disabled] {
let config: SmartSchedulingConfig = serde_json::from_value(wire).unwrap();
context.policy = context.policy.with_smart_scheduling(config);
let trace = gateway_core::diagnostics::TraceContext::new("req_zero_windows");
trace.account_selection(std::slice::from_ref(&candidate), &context, None);
let events = trace.snapshot().unwrap();
let data = &events["events"][0]["data"];
assert_eq!(data["candidates"][0]["smartScore"], 0.0);
assert_eq!(data["smartScoreTolerance"], 0.0);
assert_eq!(
serde_json::to_value(config).unwrap()["resetWindowWeights"],
json!({"fiveHour":0.0,"weekly":0.0,"monthly":0.0})
);
}
}
#[test] #[test]
fn queue_weight_balances_live_pressure_and_health_only_when_enabled_for_smart() { fn queue_weight_balances_live_pressure_and_health_only_when_enabled_for_smart() {
for (strategy, queue_weight, expects_shortest) in [ for (strategy, queue_weight, expects_shortest) in [
@@ -349,3 +391,41 @@ fn higher_weight_recovery_preserves_usable_soft_affinity() {
); );
} }
} }
#[test]
fn missing_expired_and_duplicate_periods_do_not_inflate_reset_score() {
let mut context = context(RotationStrategy::Smart);
let mut candidate = candidate_with_concurrency("acct_partial", 0, 2);
candidate.signals.quota_reset_at = Some(context.now + Duration::from_secs(1));
candidate.signals.quota_reset_windows = Some(
[
(QuotaResetPeriod::Weekly, 86400),
(QuotaResetPeriod::Weekly, 90000),
(QuotaResetPeriod::Weekly, 0),
(QuotaResetPeriod::FiveHour, 0),
]
.map(|(period, delta)| AccountQuotaResetWindow {
period,
reset_at: context.now + Duration::from_secs(delta),
})
.into(),
);
for scale in [1.0, 0.5] {
let mut config = serde_json::to_value(
SmartSchedulingConfig::new([0.0, 0.0, 0.0, 0.0, 1.0, 0.0]).unwrap(),
)
.unwrap();
config["resetWindowWeights"] = json!({
"weekly": 2.0*scale, "fiveHour": 4.0*scale, "monthly": 4.0*scale
});
context.policy = context
.policy
.with_smart_scheduling(serde_json::from_value(config).unwrap());
let trace = gateway_core::diagnostics::TraceContext::new("req_partial");
trace.account_selection(std::slice::from_ref(&candidate), &context, None);
assert_eq!(
trace.snapshot().unwrap()["events"][0]["data"]["candidates"][0]["smartScore"],
0.1
);
}
}
@@ -1228,6 +1228,7 @@ fn plan_with_protection(
None, None,
) )
.with_smart_scheduling(account_selection_policy.smart_scheduling()) .with_smart_scheduling(account_selection_policy.smart_scheduling())
.with_quota_reset_window_weights(account_selection_policy.quota_reset_window_weights())
.with_max_account_rotations(account_selection_policy.max_account_rotations()) .with_max_account_rotations(account_selection_policy.max_account_rotations())
.with_openai_account_affinity(account_selection_policy.openai_account_affinity()) .with_openai_account_affinity(account_selection_policy.openai_account_affinity())
.with_openai_session_affinity_ttl_hours( .with_openai_session_affinity_ttl_hours(
@@ -2324,6 +2324,7 @@ impl Provider for PolicySelectingProvider {
in_flight: 0, in_flight: 0,
last_started_at: None, last_started_at: None,
quota_reset_at: None, quota_reset_at: None,
quota_reset_windows: None,
quota_remaining_rank: None, quota_remaining_rank: None,
cooldown: None, cooldown: None,
failure_rate_basis_points: None, failure_rate_basis_points: None,
@@ -3442,7 +3443,7 @@ fn provider_local_probe_failure_should_remain_distinct_from_upstream() {
} }
#[test] #[test]
fn diagnostic_probe_does_not_apply_data_plane_account_model_policy() { fn diagnostic_probe_uses_global_profile_without_data_plane_account_model_policy() {
let provider = ProviderKind::new("openai").expect("provider"); let provider = ProviderKind::new("openai").expect("provider");
let account_id = ProviderAccountId::new("acct_probe").expect("account"); let account_id = ProviderAccountId::new("acct_probe").expect("account");
let snapshot = probe_snapshot().with_account_directory(Arc::new(RuntimeAccountDirectory::new( let snapshot = probe_snapshot().with_account_directory(Arc::new(RuntimeAccountDirectory::new(
@@ -3457,10 +3458,21 @@ fn diagnostic_probe_does_not_apply_data_plane_account_model_policy() {
), ),
)]), )]),
))); )));
let profile = gateway_core::account::OpaqueProviderData::new(
json!({"identity":"global-probe"})
.as_object()
.unwrap()
.clone(),
);
let settings = snapshot
.settings()
.clone()
.with_request_profiles(BTreeMap::from([(provider.clone(), profile.clone())]));
let snapshot = snapshot.with_settings(&settings).unwrap();
let service = DefaultExecutionService::new( let service = DefaultExecutionService::new(
RuntimeSnapshotHandle::new(snapshot), RuntimeSnapshotHandle::new(snapshot),
Arc::new(TrackingExecutionStore::default()), Arc::new(TrackingExecutionStore::default()),
ProviderRegistry::new([Arc::new(LocalFailingProvider) as Arc<dyn Provider>]) ProviderRegistry::new([Arc::new(ProfileCheckingProvider(profile)) as Arc<dyn Provider>])
.expect("providers"), .expect("providers"),
Arc::new(UnusedAdmissions), Arc::new(UnusedAdmissions),
Arc::new(UnusedContinuation), Arc::new(UnusedContinuation),
@@ -3861,6 +3873,36 @@ impl Provider for FailingProvider {
struct LocalFailingProvider; struct LocalFailingProvider;
struct ProfileCheckingProvider(gateway_core::account::OpaqueProviderData);
#[async_trait]
impl Provider for ProfileCheckingProvider {
fn name(&self) -> &'static str {
"openai"
}
fn catalog_generation(&self) -> ProviderCatalogGeneration {
ProviderCatalogGeneration::default()
}
async fn query_model_capabilities(
&self,
) -> Result<Vec<ProviderModelCapabilities>, ProviderError> {
Ok(Vec::new())
}
async fn execute(
self: Arc<Self>,
request: ProviderRequest,
context: AttemptContext,
) -> Result<ProviderStream, ProviderError> {
assert_eq!(context.request_profile(), Some(&self.0));
Arc::new(LocalFailingProvider)
.execute(request, context)
.await
}
}
#[async_trait] #[async_trait]
impl Provider for LocalFailingProvider { impl Provider for LocalFailingProvider {
fn name(&self) -> &'static str { fn name(&self) -> &'static str {
@@ -77,6 +77,7 @@ fn candidate(id: &str) -> AccountCandidate {
in_flight: 0, in_flight: 0,
last_started_at: None, last_started_at: None,
quota_reset_at: None, quota_reset_at: None,
quota_reset_windows: None,
quota_remaining_rank: None, quota_remaining_rank: None,
cooldown: None, cooldown: None,
failure_rate_basis_points: None, failure_rate_basis_points: None,
@@ -200,6 +200,7 @@ fn scheduling_state_preserves_provider_neutral_signals() {
in_flight: 2, in_flight: 2,
last_started_at: None, last_started_at: None,
quota_reset_at: None, quota_reset_at: None,
quota_reset_windows: None,
quota_remaining_rank: Some(7), quota_remaining_rank: Some(7),
cooldown: None, cooldown: None,
failure_rate_basis_points: Some(125), failure_rate_basis_points: Some(125),
@@ -251,14 +252,10 @@ fn warmup_cron_validation_and_calendar_matching() {
] { ] {
assert!(!valid_warmup_cron(value), "{value:?}"); assert!(!valid_warmup_cron(value), "{value:?}");
} }
let policy = ProviderWarmupPolicy::try_new( let policy =
true, ProviderWarmupPolicy::try_new(true, "0 8,13 * * MON-FRI", Some("test-model".to_owned()))
"0 8,13 * * MON-FRI".to_owned(), .unwrap();
Some("test-model".to_owned()),
)
.unwrap();
assert!(policy.enabled()); assert!(policy.enabled());
assert_eq!(policy.cron(), "0 8,13 * * MON-FRI");
assert_eq!(policy.model(), Some("test-model")); assert_eq!(policy.model(), Some("test-model"));
for (day, hour, minute, expected) in [ for (day, hour, minute, expected) in [
(9, 8, 0, true), (9, 8, 0, true),
@@ -274,19 +271,39 @@ fn warmup_cron_validation_and_calendar_matching() {
.unwrap(); .unwrap();
assert_eq!(policy.matches_slot(slot), expected, "{slot}"); assert_eq!(policy.matches_slot(slot), expected, "{slot}");
} }
assert!(ProviderWarmupPolicy::try_new(false, String::new(), None).is_ok()); assert!(ProviderWarmupPolicy::try_new(false, "", None).is_ok());
assert!( assert!(ProviderWarmupPolicy::try_new(true, "", Some("test-model".to_owned())).is_err());
ProviderWarmupPolicy::try_new(true, String::new(), Some("test-model".to_owned())).is_err() assert!(ProviderWarmupPolicy::try_new(false, "08:00", None).is_err());
);
assert!(ProviderWarmupPolicy::try_new(false, "08:00".to_owned(), None).is_err());
assert!(!valid_warmup_cron(&format!("0{}8 * * *", " ".repeat(256)))); assert!(!valid_warmup_cron(&format!("0{}8 * * *", " ".repeat(256))));
let leap = ProviderWarmupPolicy::try_new(false, "0 0 29 FEB *".to_owned(), None).unwrap(); let leap = ProviderWarmupPolicy::try_new(false, "0 0 29 FEB *", None).unwrap();
let slot = NaiveDate::from_ymd_opt(2028, 2, 29) let slot = NaiveDate::from_ymd_opt(2028, 2, 29)
.unwrap() .unwrap()
.and_hms_opt(0, 0, 0) .and_hms_opt(0, 0, 0)
.unwrap(); .unwrap();
assert!(leap.matches_slot(slot)); assert!(leap.matches_slot(slot));
assert!(!leap.matches_slot(slot + chrono::Duration::days(1))); assert!(!leap.matches_slot(slot + chrono::Duration::days(1)));
assert!(ProviderWarmupPolicy::try_new(true, "invalid".to_owned(), None).is_err()); assert!(ProviderWarmupPolicy::try_new(true, "invalid", None).is_err());
assert!(ProviderWarmupPolicy::try_new(true, "0 8 * * *".to_owned(), None).is_err()); assert!(ProviderWarmupPolicy::try_new(true, "0 8 * * *", None).is_err());
}
#[test]
fn warmup_matches_only_the_scheduled_hour_and_minute_pairs() {
use chrono::NaiveDate;
use gateway_core::provider_ports::ProviderWarmupPolicy;
let policy = ProviderWarmupPolicy::try_new(
true,
"58 5 * * *;59 10 * * *;0 16 * * *;1 21 * * *;0 16 * * MON-FRI",
Some("test-model".to_owned()),
)
.unwrap();
let date = NaiveDate::from_ymd_opt(2026, 11, 13).unwrap();
for minute in 0..1440 {
let slot = date.and_hms_opt(minute / 60, minute % 60, 0).unwrap();
assert_eq!(
policy.matches_slot(slot),
[358, 659, 960, 1261].contains(&minute),
"{slot}"
);
}
} }
@@ -877,11 +877,15 @@ fn compiler_accepts_unlimited_default_account_concurrency() {
#[test] #[test]
fn compiled_plans_keep_their_scheduling_config_after_a_new_snapshot_is_built() { fn compiled_plans_keep_their_scheduling_config_after_a_new_snapshot_is_built() {
use gateway_core::account::SmartSchedulingConfig; use gateway_core::account::{ResetWindowWeights, SmartSchedulingConfig};
let configs = [ let configs = [
SmartSchedulingConfig::default(), SmartSchedulingConfig::default(),
SmartSchedulingConfig::new([0.0, 2.0, 1.0, 0.5, 1.2, 2.3]).unwrap(), SmartSchedulingConfig::new([0.0, 2.0, 1.0, 0.5, 1.2, 2.3]).unwrap(),
]; ];
let windows = [
ResetWindowWeights::default(),
ResetWindowWeights::new([0.0, 1.5, 0.0]).unwrap(),
];
let mut plans = Vec::new(); let mut plans = Vec::new();
for (index, config) in configs.into_iter().enumerate() { for (index, config) in configs.into_iter().enumerate() {
let facts = SnapshotFacts::new( let facts = SnapshotFacts::new(
@@ -889,6 +893,7 @@ fn compiled_plans_keep_their_scheduling_config_after_a_new_snapshot_is_built() {
revision(index as u64 + 1), revision(index as u64 + 1),
SettingsValues::new(3, "smart", BTreeMap::new(), None, None) SettingsValues::new(3, "smart", BTreeMap::new(), None, None)
.with_smart_scheduling(config) .with_smart_scheduling(config)
.with_quota_reset_window_weights(windows[index])
.with_openai_guardian_reserved_concurrency(index as u32), .with_openai_guardian_reserved_concurrency(index as u32),
vec![], vec![],
vec![], vec![],
@@ -921,6 +926,10 @@ fn compiled_plans_keep_their_scheduling_config_after_a_new_snapshot_is_built() {
index as u32 index as u32
); );
assert_eq!(plan.account_selection_policy().smart_scheduling(), config); assert_eq!(plan.account_selection_policy().smart_scheduling(), config);
assert_eq!(
plan.account_selection_policy().quota_reset_window_weights(),
windows[index]
);
} }
} }
+17
View File
@@ -0,0 +1,17 @@
[package]
name = "gateway-egress"
version.workspace = true
edition.workspace = true
license.workspace = true
publish.workspace = true
rust-version.workspace = true
[dependencies]
gateway-core = { path = "../gateway-core" }
http = "1.4.0"
hyper-util = { version = "=0.1.21", features = ["client-proxy"] }
reqwest = { version = "=0.12.28", default-features = false, features = ["socks"] }
url = "2.5"
[lints]
workspace = true
+117
View File
@@ -0,0 +1,117 @@
//! 网关 HTTP 与 WebSocket 出口:业务指定的代理优先,否则按目的地址使用部署环境
use std::sync::OnceLock;
use gateway_core::account::{InvalidOutboundProxy, OutboundProxy};
use hyper_util::client::proxy::matcher::Matcher;
use reqwest::{Client, ClientBuilder, NoProxy, Proxy};
use url::Url;
struct Environment {
http: Option<OutboundProxy>,
https: Option<OutboundProxy>,
no_proxy: String,
matcher: Matcher,
}
impl Environment {
fn load() -> Result<Self, InvalidOutboundProxy> {
let var = |name: &str| {
std::env::var(name)
.or_else(|_| std::env::var(name.to_ascii_lowercase()))
.ok()
.filter(|value| !value.is_empty())
};
// CGI 的 HTTP_* 来自入站请求头,不能作为部署配置
let no_proxy = var("NO_PROXY").unwrap_or_default();
let bypass_all = std::env::var_os("REQUEST_METHOD").is_some()
|| no_proxy.split(',').any(|entry| entry.trim() == "*");
let all = var("ALL_PROXY");
let route = |name| {
if bypass_all {
return Ok(None);
}
var(name)
.or_else(|| all.clone())
.map(|value| {
// 部署环境也接受常见的 host:port 写法,服务内保存的代理仍须是完整 URL
if value.contains("://") {
OutboundProxy::parse(&value)
} else {
OutboundProxy::parse(&format!("http://{value}"))
}
})
.transpose()
};
let http = route("HTTP_PROXY")?;
let https = route("HTTPS_PROXY")?;
let matcher = Matcher::builder()
.http(http.as_ref().map_or("", OutboundProxy::expose_url))
.https(https.as_ref().map_or("", OutboundProxy::expose_url))
.no(no_proxy.as_str())
.build();
Ok(Self {
http,
https,
no_proxy,
matcher,
})
}
}
fn environment() -> Result<&'static Environment, InvalidOutboundProxy> {
static ENV: OnceLock<Result<Environment, InvalidOutboundProxy>> = OnceLock::new();
ENV.get_or_init(Environment::load).as_ref().map_err(|e| *e)
}
/// 每个请求(包括重定向)按目标匹配环境代理;显式代理不受 NO_PROXY 影响
pub fn http_client(proxy: Option<&OutboundProxy>) -> Result<ClientBuilder, InvalidOutboundProxy> {
let mut builder = Client::builder().no_proxy();
if let Some(proxy) = proxy {
return Ok(builder.proxy(Proxy::all(proxy.expose_url()).map_err(|_| InvalidOutboundProxy)?));
}
let env = environment()?;
let bypass = NoProxy::from_string(&env.no_proxy);
if let Some(proxy) = &env.http {
builder = builder.proxy(
Proxy::http(proxy.expose_url())
.map_err(|_| InvalidOutboundProxy)?
.no_proxy(bypass.clone()),
);
}
if let Some(proxy) = &env.https {
builder = builder.proxy(
Proxy::https(proxy.expose_url())
.map_err(|_| InvalidOutboundProxy)?
.no_proxy(bypass),
);
}
Ok(builder)
}
/// 自行拨号的传输复用 reqwest 使用的匹配器,但保留原代理的凭据与连接限制
pub fn for_destination<'a>(
proxy: Option<&'a OutboundProxy>,
url: &Url,
) -> Result<Option<&'a OutboundProxy>, InvalidOutboundProxy> {
if proxy.is_some() {
return Ok(proxy);
}
let env = environment()?;
let (scheme, proxy) = match url.scheme() {
"http" | "ws" => ("http", env.http.as_ref()),
"https" | "wss" => ("https", env.https.as_ref()),
_ => return Err(InvalidOutboundProxy),
};
let Some(proxy) = proxy else {
return Ok(None);
};
let authority = &url[url::Position::BeforeHost..url::Position::AfterPort];
let uri = http::Uri::builder()
.scheme(scheme)
.authority(authority)
.path_and_query("/")
.build()
.map_err(|_| InvalidOutboundProxy)?;
Ok(env.matcher.intercept(&uri).map(|_| proxy))
}
+1
View File
@@ -17,6 +17,7 @@ config = { version = "0.15.23", default-features = false, features = ["yaml"] }
flate2 = "1.1.9" flate2 = "1.1.9"
futures = "0.3.32" futures = "0.3.32"
gateway-admin = { path = "../gateway-admin" } gateway-admin = { path = "../gateway-admin" }
gateway-egress = { path = "../gateway-egress" }
gateway-core = { path = "../gateway-core" } gateway-core = { path = "../gateway-core" }
gateway-protocol = { path = "../gateway-protocol" } gateway-protocol = { path = "../gateway-protocol" }
hex = "0.4.3" hex = "0.4.3"
@@ -52,17 +52,19 @@ struct CachedDownloads {
impl RgAdguardClientDistribution { impl RgAdguardClientDistribution {
pub(crate) fn new() -> Self { pub(crate) fn new() -> Self {
let client = Client::builder() let client = gateway_egress::http_client(None).ok().and_then(|builder| {
.connect_timeout(Duration::from_secs(5)) builder
.timeout(Duration::from_secs(30)) .connect_timeout(Duration::from_secs(5))
.redirect(Policy::none()) .timeout(Duration::from_secs(30))
.build() .redirect(Policy::none())
.inspect_err(|_| { .build()
tracing::warn!( .ok()
"download resolver HTTP client initialization failed; official downloads will be used" });
); if client.is_none() {
}) tracing::warn!(
.ok(); "download resolver HTTP client initialization failed; official downloads will be used"
);
}
Self { Self {
client, client,
cache: Mutex::new(None), cache: Mutex::new(None),
@@ -216,6 +216,8 @@ impl HttpClient {
{ {
return Err(HttpError::invalid("URL")); return Err(HttpError::invalid("URL"));
} }
let proxy = gateway_egress::for_destination(proxy, &url)
.map_err(|_| HttpError::invalid("proxy"))?;
let method = Method::from_bytes(request.method.as_bytes()) let method = Method::from_bytes(request.method.as_bytes())
.map_err(|_| HttpError::invalid("method"))?; .map_err(|_| HttpError::invalid("method"))?;
if matches!(method, Method::CONNECT | Method::TRACE) { if matches!(method, Method::CONNECT | Method::TRACE) {
+2 -8
View File
@@ -37,17 +37,11 @@ impl PeerTransport for HttpPeerTransport {
account: &ProviderAccount, account: &ProviderAccount,
authentication: &PeerConnection, authentication: &PeerConnection,
) -> Result<Vec<u8>, PeerFetchFailure> { ) -> Result<Vec<u8>, PeerFetchFailure> {
let mut builder = reqwest::Client::builder() let builder = gateway_egress::http_client(account.outbound_proxy())
.no_proxy() .map_err(|_| UpstreamInfoFailure::Unavailable)?
.redirect(reqwest::redirect::Policy::none()) .redirect(reqwest::redirect::Policy::none())
.connect_timeout(REQUEST_TIMEOUT) .connect_timeout(REQUEST_TIMEOUT)
.timeout(REQUEST_TIMEOUT); .timeout(REQUEST_TIMEOUT);
if let Some(proxy) = account.outbound_proxy() {
builder = builder.proxy(
reqwest::Proxy::all(proxy.expose_url())
.map_err(|_| UpstreamInfoFailure::Unavailable)?,
);
}
let client = (self.build_client)(builder).map_err(|_| UpstreamInfoFailure::Unavailable)?; let client = (self.build_client)(builder).map_err(|_| UpstreamInfoFailure::Unavailable)?;
let response = client let response = client
.get(format!( .get(format!(
+2 -1
View File
@@ -17,7 +17,8 @@ pub struct ModelsDevPricing;
#[async_trait] #[async_trait]
impl PricingSource for ModelsDevPricing { impl PricingSource for ModelsDevPricing {
async fn fetch(&self) -> Result<PricingSyncPreview, AdminError> { async fn fetch(&self) -> Result<PricingSyncPreview, AdminError> {
let client = reqwest::Client::builder() let client = gateway_egress::http_client(None)
.map_err(|_| source_error())?
.https_only(true) .https_only(true)
.redirect(reqwest::redirect::Policy::none()) .redirect(reqwest::redirect::Policy::none())
.connect_timeout(Duration::from_secs(10)) .connect_timeout(Duration::from_secs(10))
+29 -30
View File
@@ -1,4 +1,4 @@
//! 使用与 Provider 请求一致的显式代理或直连路径,执行有超时和响应大小限制的出口测试 //! 按指定出口,执行有超时和响应大小限制的出口测试
use std::{ use std::{
net::{IpAddr, Ipv4Addr, Ipv6Addr}, net::{IpAddr, Ipv4Addr, Ipv6Addr},
@@ -9,11 +9,20 @@ use std::{
use async_trait::async_trait; use async_trait::async_trait;
use gateway_admin::{ use gateway_admin::{
model::proxies::{ProxyLocationDetection, ProxyTestResult}, model::proxies::{ProxyLocationDetection, ProxyTestResult},
ports::proxy::ProxyProbe, ports::proxy::{ProbeRoute, ProxyProbe},
}; };
use gateway_core::account::{OutboundProxy, RequestLocation}; use gateway_core::account::RequestLocation;
use serde::Deserialize; use serde::Deserialize;
fn probe_client(route: ProbeRoute<'_>) -> Result<reqwest::ClientBuilder, &'static str> {
match route {
ProbeRoute::Direct => Ok(reqwest::Client::builder().no_proxy()),
ProbeRoute::Deployment => gateway_egress::http_client(None),
ProbeRoute::Proxy(proxy) => gateway_egress::http_client(Some(proxy)),
}
.map_err(|_| "代理地址不合法")
}
enum ProbeStrategy { enum ProbeStrategy {
Single(String), Single(String),
Dual { Dual {
@@ -75,18 +84,13 @@ impl HttpProxyProbe {
async fn exit_ip_at( async fn exit_ip_at(
&self, &self,
proxy: Option<&OutboundProxy>, proxy: ProbeRoute<'_>,
endpoint: &str, endpoint: &str,
) -> Result<IpAddr, &'static str> { ) -> Result<IpAddr, &'static str> {
let mut builder = reqwest::Client::builder() let builder = probe_client(proxy)?
.no_proxy()
.connect_timeout(Duration::from_secs(5)) .connect_timeout(Duration::from_secs(5))
.timeout(Duration::from_secs(12)) .timeout(Duration::from_secs(12))
.redirect(reqwest::redirect::Policy::none()); .redirect(reqwest::redirect::Policy::none());
if let Some(proxy) = proxy {
builder = builder
.proxy(reqwest::Proxy::all(proxy.expose_url()).map_err(|_| "代理地址不合法")?);
}
let client = (self.build_client)(builder)?; let client = (self.build_client)(builder)?;
let mut response = client.get(endpoint).send().await.map_err(|error| { let mut response = client.get(endpoint).send().await.map_err(|error| {
if error.is_timeout() { if error.is_timeout() {
@@ -125,7 +129,7 @@ impl HttpProxyProbe {
impl HttpProxyProbe { impl HttpProxyProbe {
async fn timed_exit_ip_at( async fn timed_exit_ip_at(
&self, &self,
proxy: Option<&OutboundProxy>, proxy: ProbeRoute<'_>,
endpoint: &str, endpoint: &str,
) -> (Result<IpAddr, &'static str>, u64) { ) -> (Result<IpAddr, &'static str>, u64) {
let started = Instant::now(); let started = Instant::now();
@@ -134,7 +138,7 @@ impl HttpProxyProbe {
(result, latency_ms) (result, latency_ms)
} }
async fn test_connection(&self, proxy: Option<&OutboundProxy>) -> ProxyTestResult { async fn test_connection(&self, proxy: ProbeRoute<'_>) -> ProxyTestResult {
let started = Instant::now(); let started = Instant::now();
let timeout_limit = Duration::from_secs(15); let timeout_limit = Duration::from_secs(15);
@@ -276,39 +280,34 @@ impl HttpProxyProbe {
async fn location_for_ip( async fn location_for_ip(
&self, &self,
proxy: Option<&OutboundProxy>, proxy: ProbeRoute<'_>,
ip: IpAddr, ip: IpAddr,
) -> Result<RequestLocation, &'static str> { ) -> Result<RequestLocation, &'static str> {
// 出口 IP 已由探测确认;按这个固定 IP 从服务端查询位置,避免代理出口屏蔽位置服务 // 查询的是已经验证的 IP,不要求被测代理能访问位置服务。
// 部署环境无法直连位置服务时仍尝试原代理路径 // 服务器直连探测保持直连,其余探测先按部署环境查询,必要时再用被测代理。
match self.lookup_location(ip, None).await { if !matches!(proxy, ProbeRoute::Proxy(_)) {
return self.lookup_location(ip, proxy).await;
}
match self.lookup_location(ip, ProbeRoute::Deployment).await {
Ok(location) => Ok(location), Ok(location) => Ok(location),
Err(direct_error) if proxy.is_some() => { Err(error) => self.lookup_location(ip, proxy).await.or(Err(error)),
self.lookup_location(ip, proxy).await.or(Err(direct_error))
}
Err(error) => Err(error),
} }
} }
async fn lookup_location( async fn lookup_location(
&self, &self,
ip: IpAddr, ip: IpAddr,
proxy: Option<&OutboundProxy>, proxy: ProbeRoute<'_>,
) -> Result<RequestLocation, &'static str> { ) -> Result<RequestLocation, &'static str> {
let timeout = if proxy.is_some() { let timeout = if !matches!(proxy, ProbeRoute::Direct) {
Duration::from_secs(4) Duration::from_secs(4)
} else { } else {
Duration::from_secs(3) Duration::from_secs(3)
}; };
let mut builder = reqwest::Client::builder() let builder = probe_client(proxy)?
.no_proxy()
.connect_timeout(timeout) .connect_timeout(timeout)
.timeout(timeout) .timeout(timeout)
.redirect(reqwest::redirect::Policy::none()); .redirect(reqwest::redirect::Policy::none());
if let Some(proxy) = proxy {
builder = builder
.proxy(reqwest::Proxy::all(proxy.expose_url()).map_err(|_| "代理地址不合法")?);
}
let client = (self.build_client)(builder)?; let client = (self.build_client)(builder)?;
// 查询已检测到的具体出口,不能再次查询“我的 IP”,轮换代理可能换到另一个出口 // 查询已检测到的具体出口,不能再次查询“我的 IP”,轮换代理可能换到另一个出口
let url = format!("{}/{ip}", self.location_endpoint.trim_end_matches('/')); let url = format!("{}/{ip}", self.location_endpoint.trim_end_matches('/'));
@@ -367,7 +366,7 @@ impl HttpProxyProbe {
async fn detect_location( async fn detect_location(
&self, &self,
proxy: Option<&OutboundProxy>, proxy: ProbeRoute<'_>,
result: &ProxyTestResult, result: &ProxyTestResult,
) -> ProxyLocationDetection { ) -> ProxyLocationDetection {
let lookup = async |ip: Option<IpAddr>| match ip { let lookup = async |ip: Option<IpAddr>| match ip {
@@ -397,7 +396,7 @@ impl HttpProxyProbe {
#[async_trait] #[async_trait]
impl ProxyProbe for HttpProxyProbe { impl ProxyProbe for HttpProxyProbe {
async fn test(&self, proxy: Option<&OutboundProxy>, detect_location: bool) -> ProxyTestResult { async fn test(&self, proxy: ProbeRoute<'_>, detect_location: bool) -> ProxyTestResult {
let mut result = self.test_connection(proxy).await; let mut result = self.test_connection(proxy).await;
if detect_location { if detect_location {
result.location = if result.success { result.location = if result.success {
@@ -132,16 +132,17 @@ impl Metadata {
let client = self let client = self
.client .client
.get_or_try_init(|| async { .get_or_try_init(|| async {
Client::builder() gateway_egress::http_client(None)
.map_err(|_| super::error::invalid("部署代理配置不合法"))?
.user_agent("codex-proxy-rs") .user_agent("codex-proxy-rs")
.timeout(Duration::from_secs(10)) .timeout(Duration::from_secs(10))
.connect_timeout(Duration::from_secs(5)) .connect_timeout(Duration::from_secs(5))
.redirect(reqwest::redirect::Policy::none()) .redirect(reqwest::redirect::Policy::none())
.retry(reqwest::retry::never()) .retry(reqwest::retry::never())
.build() .build()
.map_err(|e| super::error::http("无法初始化发行查询连接", e))
}) })
.await .await?;
.map_err(|e| super::error::http("无法初始化发行查询连接", e))?;
let request = client.get(url); let request = client.get(url);
let mut response = if api { let mut response = if api {
request.header("Accept", "application/vnd.github+json") request.header("Accept", "application/vnd.github+json")
@@ -136,7 +136,8 @@ pub(crate) fn download_client(
timeout: Duration, timeout: Duration,
) -> Result<reqwest::Client, OperationError> { ) -> Result<reqwest::Client, OperationError> {
let api_base = api_base.to_owned(); let api_base = api_base.to_owned();
reqwest::Client::builder() gateway_egress::http_client(None)
.map_err(|_| invalid("部署代理配置不合法"))?
.timeout(timeout) .timeout(timeout)
.redirect(reqwest::redirect::Policy::custom(move |attempt| { .redirect(reqwest::redirect::Policy::custom(move |attempt| {
if attempt.previous().len() < 5 && download_url_allowed(attempt.url(), &api_base) { if attempt.previous().len() < 5 && download_url_allowed(attempt.url(), &api_base) {
@@ -268,7 +268,12 @@ fn move_file(from: &Path, to: &Path) -> io::Result<()> {
fn move_dir(from: &Path, to: &Path) -> io::Result<()> { fn move_dir(from: &Path, to: &Path) -> io::Result<()> {
match fs::rename(from, to) { match fs::rename(from, to) {
Ok(()) => Ok(()), Ok(()) => Ok(()),
Err(error) if error.kind() == io::ErrorKind::CrossesDevices => { Err(error)
if error.kind() == io::ErrorKind::CrossesDevices
|| (cfg!(unix) && error.kind() == io::ErrorKind::PermissionDenied) =>
{
// Unix 跨父目录移动只读目录可能被拒绝,复用保留权限的复制路径
// 仅清理源目录时放开写权限,备份与恢复后的目录仍保留原权限
if let Err(copy_error) = copy_dir_all(from, to) { if let Err(copy_error) = copy_dir_all(from, to) {
let _ = remove_dir_all(to); let _ = remove_dir_all(to);
return Err(copy_error); return Err(copy_error);
@@ -354,3 +354,99 @@ async fn managed_http_enforces_the_callers_body_limit_before_sending() {
assert_eq!(requests[0].body.len(), maximum); assert_eq!(requests[0].body.len(), maximum);
assert_eq!(requests[1].body.len(), maximum + 1); assert_eq!(requests[1].body.len(), maximum + 1);
} }
#[tokio::test]
async fn deployment_proxy_routes_managed_http_and_account_probes() {
use gateway_admin::ports::proxy::{ProbeRoute, ProxyProbe};
use gateway_host::proxy_probe::HttpProxyProbe;
if let Ok(origin) = std::env::var("CPR_TEST_EGRESS_ORIGIN") {
let selected =
OutboundProxy::parse(&std::env::var("CPR_TEST_EGRESS_SELECTED").unwrap()).unwrap();
let outside = origin.replace("127.0.0.1", "localhost");
let managed = HttpClient::new().unwrap();
let http = |url: String, proxy| {
let managed = &managed;
async move {
managed
.open(
request(url),
proxy,
&local_network(),
Duration::from_secs(2),
)
.await
.unwrap()
.status
}
};
let probe = HttpProxyProbe::new(&outside);
let observed = |result: gateway_admin::model::proxies::ProxyTestResult| {
result.exit_ipv4.unwrap().octets()[3]
};
assert_eq!(observed(probe.test(ProbeRoute::Direct, false).await), 1);
assert_eq!(http(origin.clone(), Some(&selected)).await, 203);
assert_eq!(http(origin, None).await, 200);
let expect_env = std::env::var_os("HTTP_PROXY").is_some();
assert_eq!(
http(outside, None).await,
if expect_env { 202 } else { 200 }
);
assert_eq!(
observed(probe.test(ProbeRoute::Deployment, false).await),
if expect_env { 2 } else { 1 }
);
println!("checked managed egress");
return;
}
let origin = MockServer::start().await;
let env = MockServer::start().await;
let selected = MockServer::start().await;
for (server, code, ip) in [
(&origin, 200, "192.0.2.1"),
(&env, 202, "192.0.2.2"),
(&selected, 203, "192.0.2.3"),
] {
Mock::given(any())
.respond_with(ResponseTemplate::new(code).set_body_json(serde_json::json!({"ip": ip})))
.mount(server)
.await;
}
for configured in [true, false] {
let mut command = tokio::process::Command::new(std::env::current_exe().unwrap());
for name in [
"HTTP_PROXY",
"HTTPS_PROXY",
"ALL_PROXY",
"NO_PROXY",
"REQUEST_METHOD",
] {
command
.env_remove(name)
.env_remove(name.to_ascii_lowercase());
}
if configured {
command
.env("HTTP_PROXY", env.uri())
.env("NO_PROXY", "127.0.0.0/8");
}
let output = command
.args([
"--exact",
"outbound::http::deployment_proxy_routes_managed_http_and_account_probes",
"--nocapture",
])
.env("CPR_TEST_EGRESS_ORIGIN", origin.uri())
.env("CPR_TEST_EGRESS_SELECTED", selected.uri())
.output()
.await
.unwrap();
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
assert!(String::from_utf8_lossy(&output.stdout).contains("checked managed egress"));
}
}
@@ -1,6 +1,6 @@
//! 验证代理及直连探测的双栈地址、认证、证书与失败边界 //! 验证代理及直连探测的双栈地址、认证、证书与失败边界
use gateway_admin::ports::proxy::ProxyProbe; use gateway_admin::ports::proxy::{ProbeRoute, ProxyProbe};
use gateway_core::account::OutboundProxy; use gateway_core::account::OutboundProxy;
use gateway_host::proxy_probe::HttpProxyProbe; use gateway_host::proxy_probe::HttpProxyProbe;
use serde_json::json; use serde_json::json;
@@ -28,7 +28,7 @@ async fn proxy_probe_supports_ipv4_and_ipv6_proxies_and_exit_addresses() {
OutboundProxy::parse(&format!("http://user:password@{}", proxy_server.address())) OutboundProxy::parse(&format!("http://user:password@{}", proxy_server.address()))
.unwrap(); .unwrap();
let result = HttpProxyProbe::new("http://unresolvable.invalid/ip") let result = HttpProxyProbe::new("http://unresolvable.invalid/ip")
.test(Some(&proxy), false) .test(ProbeRoute::Proxy(&proxy), false)
.await; .await;
assert!( assert!(
result.success, result.success,
@@ -55,7 +55,7 @@ async fn proxy_probe_rejects_auth_errors_redirects_and_invalid_or_oversized_resp
.await; .await;
let result = HttpProxyProbe::new("http://unresolvable.invalid/ip") let result = HttpProxyProbe::new("http://unresolvable.invalid/ip")
.test( .test(
Some(&OutboundProxy::parse(&proxy_server.uri()).unwrap()), ProbeRoute::Proxy(&OutboundProxy::parse(&proxy_server.uri()).unwrap()),
false, false,
) )
.await; .await;
@@ -76,7 +76,7 @@ async fn unavailable_proxy_never_falls_back_to_direct_connection() {
let proxy = OutboundProxy::parse(&format!("http://{}", unused.local_addr().unwrap())).unwrap(); let proxy = OutboundProxy::parse(&format!("http://{}", unused.local_addr().unwrap())).unwrap();
drop(unused); drop(unused);
let result = HttpProxyProbe::new(target.uri()) let result = HttpProxyProbe::new(target.uri())
.test(Some(&proxy), false) .test(ProbeRoute::Proxy(&proxy), false)
.await; .await;
assert!(!result.success); assert!(!result.success);
} }
@@ -91,7 +91,10 @@ async fn invalid_certificate_configuration_should_not_fall_back_or_expose_detail
.await; .await;
let result = HttpProxyProbe::new(target.uri()) let result = HttpProxyProbe::new(target.uri())
.with_client_builder(|_| Err("private-certificate-path")) .with_client_builder(|_| Err("private-certificate-path"))
.test(Some(&OutboundProxy::parse(&target.uri()).unwrap()), false) .test(
ProbeRoute::Proxy(&OutboundProxy::parse(&target.uri()).unwrap()),
false,
)
.await; .await;
assert!(!result.success); assert!(!result.success);
assert!(!result.message.contains("private-certificate-path")); assert!(!result.message.contains("private-certificate-path"));
@@ -117,7 +120,7 @@ async fn dual_stack_proxy_probe_reports_both_addresses_when_available() {
format!("{}/v4", proxy_server.uri()), format!("{}/v4", proxy_server.uri()),
format!("{}/v6", proxy_server.uri()), format!("{}/v6", proxy_server.uri()),
) )
.test(Some(&proxy), false) .test(ProbeRoute::Proxy(&proxy), false)
.await; .await;
assert!(result.success); assert!(result.success);
@@ -146,7 +149,7 @@ async fn dual_stack_proxy_probe_reports_single_stack_when_only_one_succeeds() {
format!("{}/v4", proxy_server.uri()), format!("{}/v4", proxy_server.uri()),
format!("{}/v6", proxy_server.uri()), format!("{}/v6", proxy_server.uri()),
) )
.test(Some(&proxy), false) .test(ProbeRoute::Proxy(&proxy), false)
.await; .await;
assert!(result.success); assert!(result.success);
@@ -178,7 +181,7 @@ async fn dual_stack_latency_measures_the_selected_exit_without_the_other_family_
format!("{}/v4", server.uri()), format!("{}/v4", server.uri()),
format!("{}/v6", server.uri()), format!("{}/v6", server.uri()),
) )
.test(None, false) .test(ProbeRoute::Direct, false)
.await; .await;
assert!(result.success); assert!(result.success);
assert_eq!(result.exit_ipv4.is_some(), working_family == "v4"); assert_eq!(result.exit_ipv4.is_some(), working_family == "v4");
@@ -229,7 +232,7 @@ async fn direct_location_uses_verified_addresses_and_rejects_conflicting_timezon
format!("{}/v6", server.uri()), format!("{}/v6", server.uri()),
) )
.with_location_endpoint(format!("{}/geo", server.uri())) .with_location_endpoint(format!("{}/geo", server.uri()))
.test(None, true) .test(ProbeRoute::Direct, true)
.await; .await;
assert!(result.success); assert!(result.success);
if v6_timezone == Some("America/New_York") { if v6_timezone == Some("America/New_York") {
@@ -261,7 +264,7 @@ async fn direct_location_failure_does_not_retry_the_same_path_or_return_a_locati
.await; .await;
let result = HttpProxyProbe::new(format!("{}/ip", server.uri())) let result = HttpProxyProbe::new(format!("{}/ip", server.uri()))
.with_location_endpoint(format!("{}/geo", server.uri())) .with_location_endpoint(format!("{}/geo", server.uri()))
.test(None, true) .test(ProbeRoute::Direct, true)
.await; .await;
assert_eq!( assert_eq!(
result.location, result.location,
@@ -294,7 +297,7 @@ async fn location_lookup_uses_verified_exit_ip_without_requiring_proxy_access_to
let result = HttpProxyProbe::new("http://unresolvable.invalid/ip") let result = HttpProxyProbe::new("http://unresolvable.invalid/ip")
.with_location_endpoint(format!("{}/geo", location_server.uri())) .with_location_endpoint(format!("{}/geo", location_server.uri()))
.test( .test(
Some(&OutboundProxy::parse(&proxy_server.uri()).unwrap()), ProbeRoute::Proxy(&OutboundProxy::parse(&proxy_server.uri()).unwrap()),
true, true,
) )
.await; .await;
@@ -333,7 +336,7 @@ async fn location_lookup_uses_probed_ips_through_the_proxy_and_checks_both_timez
) )
.with_location_endpoint("http://unresolvable.invalid/geo/") .with_location_endpoint("http://unresolvable.invalid/geo/")
.test( .test(
Some(&OutboundProxy::parse(&proxy_server.uri()).unwrap()), ProbeRoute::Proxy(&OutboundProxy::parse(&proxy_server.uri()).unwrap()),
true, true,
) )
.await; .await;
@@ -379,7 +382,7 @@ async fn geolocation_failure_never_turns_connectivity_into_failure_or_uses_unver
let result = HttpProxyProbe::new("http://unresolvable.invalid/ip") let result = HttpProxyProbe::new("http://unresolvable.invalid/ip")
.with_location_endpoint("http://unresolvable.invalid/geo") .with_location_endpoint("http://unresolvable.invalid/geo")
.test( .test(
Some(&OutboundProxy::parse(&proxy_server.uri()).unwrap()), ProbeRoute::Proxy(&OutboundProxy::parse(&proxy_server.uri()).unwrap()),
true, true,
) )
.await; .await;
@@ -407,7 +410,7 @@ async fn disabled_geolocation_never_calls_the_location_service() {
let result = HttpProxyProbe::new("http://unresolvable.invalid/ip") let result = HttpProxyProbe::new("http://unresolvable.invalid/ip")
.with_location_endpoint("http://unresolvable.invalid/geo") .with_location_endpoint("http://unresolvable.invalid/geo")
.test( .test(
Some(&OutboundProxy::parse(&proxy_server.uri()).unwrap()), ProbeRoute::Proxy(&OutboundProxy::parse(&proxy_server.uri()).unwrap()),
false, false,
) )
.await; .await;
@@ -437,6 +437,13 @@ async fn restart_should_conflict_while_another_system_operation_is_running() {
async fn rollback_should_restore_binary_web_and_version_state() { async fn rollback_should_restore_binary_web_and_version_state() {
let server = MockServer::start().await; let server = MockServer::start().await;
let fixture = Fixture::new(); let fixture = Fixture::new();
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt as _;
fs::set_permissions(fixture.official(), fs::Permissions::from_mode(0o555)).unwrap();
fs::set_permissions(fixture.web(), fs::Permissions::from_mode(0o751)).unwrap();
}
fixture fixture
.mount_release( .mount_release(
&server, &server,
@@ -470,6 +477,14 @@ async fn rollback_should_restore_binary_web_and_version_state() {
& 0o777, & 0o777,
0o555 0o555
); );
assert_eq!(
fs::metadata(fixture.official().with_file_name("official.backup"))
.expect("official backup directory")
.permissions()
.mode()
& 0o777,
0o555
);
for name in ["plugin-release-manifest.json", "new-plugin.tar.gz"] { for name in ["plugin-release-manifest.json", "new-plugin.tar.gz"] {
assert_eq!( assert_eq!(
fs::metadata(fixture.official().join(name)) fs::metadata(fixture.official().join(name))
@@ -487,6 +502,18 @@ async fn rollback_should_restore_binary_web_and_version_state() {
.await .await
.expect("rollback"); .expect("rollback");
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt as _;
for (path, mode) in [(fixture.official(), 0o555), (fixture.web(), 0o751)] {
assert_eq!(
fs::metadata(path).unwrap().permissions().mode() & 0o777,
mode
);
}
}
assert_eq!( assert_eq!(
fs::read(fixture.executable()).expect("binary"), fs::read(fixture.executable()).expect("binary"),
b"old-binary" b"old-binary"
@@ -598,6 +625,8 @@ async fn rollback_should_restore_current_release_when_plugin_revision_changes_du
.expect_err("revision changed after rollback swap"); .expect_err("revision changed after rollback swap");
assert_eq!(error.kind(), SystemOperationErrorKind::Conflict); assert_eq!(error.kind(), SystemOperationErrorKind::Conflict);
#[cfg(unix)]
assert_official_directory_readonly(&fixture);
assert_eq!( assert_eq!(
fs::read(fixture.executable()).expect("binary"), fs::read(fixture.executable()).expect("binary"),
b"new-binary" b"new-binary"
@@ -661,6 +690,8 @@ async fn cancelled_rollback_should_restore_current_release_after_the_swap() {
.expect_err("rollback task cancelled") .expect_err("rollback task cancelled")
.is_cancelled() .is_cancelled()
); );
#[cfg(unix)]
assert_official_directory_readonly(&fixture);
assert_eq!( assert_eq!(
fs::read(fixture.executable()).expect("binary"), fs::read(fixture.executable()).expect("binary"),
@@ -1131,10 +1162,94 @@ async fn update_should_replace_web_assets_across_filesystems() {
); );
} }
#[cfg(unix)]
#[tokio::test]
async fn update_should_preserve_readonly_staged_directory_permissions() {
use std::os::unix::fs::PermissionsExt as _;
struct ReadonlyStagingPreflight(PathBuf);
#[async_trait::async_trait]
impl SystemUpdatePreflight for ReadonlyStagingPreflight {
async fn validate(
&self,
candidate: SystemUpdateCandidate,
) -> Result<Revision, SystemOperationError> {
let staging = fs::read_dir(&self.0)
.unwrap()
.next()
.expect("staging directory")
.unwrap()
.path();
let nested = staging.join("web-dist/assets");
fs::create_dir(&nested).unwrap();
fs::write(nested.join("asset.js"), "asset").unwrap();
for (path, mode) in [
(nested.join("asset.js"), 0o444),
(nested, 0o550),
(staging.join("web-dist"), 0o555),
(staging.join("official-plugins"), 0o555),
] {
fs::set_permissions(path, fs::Permissions::from_mode(mode)).unwrap();
}
AllowingUpdatePreflight.validate(candidate).await
}
async fn confirm_revision(&self, _: Revision) -> Result<(), SystemOperationError> {
Ok(())
}
}
let server = MockServer::start().await;
let fixture = Fixture::new();
fixture
.mount_release(
&server,
TARGET_VERSION,
ArchiveKind::Safe,
ChecksumKind::Valid,
)
.await;
let config = fixture.config(&format!("{}/repos", server.uri()));
let preflight = Arc::new(ReadonlyStagingPreflight(config.update_temp_dir.clone()));
let service = ProcessSystemOperations::new(CancellationToken::new(), config);
service
.perform_update(Some(TARGET_VERSION.to_owned()), None, preflight)
.await
.expect("update accepted");
let status = wait_for_update(&service).await;
assert_eq!(
status.operation.status,
SystemOperationStatus::Succeeded,
"{status:?}"
);
assert_eq!(
fs::read(fixture.web().join("assets/asset.js")).unwrap(),
b"asset"
);
for (path, mode) in [
(fixture.official(), 0o555),
(fixture.web(), 0o555),
(fixture.web().join("assets"), 0o550),
(fixture.web().join("assets/asset.js"), 0o444),
] {
assert_eq!(
fs::metadata(path).unwrap().permissions().mode() & 0o777,
mode
);
}
}
#[tokio::test] #[tokio::test]
async fn update_should_restore_web_assets_when_binary_backup_fails() { async fn update_should_restore_web_assets_when_binary_backup_fails() {
let server = MockServer::start().await; let server = MockServer::start().await;
let fixture = Fixture::new(); let fixture = Fixture::new();
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt as _;
fs::set_permissions(fixture.official(), fs::Permissions::from_mode(0o555)).unwrap();
}
fixture fixture
.mount_release( .mount_release(
&server, &server,
@@ -1162,6 +1277,22 @@ async fn update_should_restore_web_assets_when_binary_backup_fails() {
); );
assert!(fixture.official().join("old-plugin.tar.gz").is_file()); assert!(fixture.official().join("old-plugin.tar.gz").is_file());
assert!(!fixture.official().join("new-plugin.tar.gz").exists()); assert!(!fixture.official().join("new-plugin.tar.gz").exists());
#[cfg(unix)]
assert_official_directory_readonly(&fixture);
}
#[cfg(unix)]
fn assert_official_directory_readonly(fixture: &Fixture) {
use std::os::unix::fs::PermissionsExt as _;
assert_eq!(
fs::metadata(fixture.official())
.unwrap()
.permissions()
.mode()
& 0o777,
0o555
);
} }
#[tokio::test] #[tokio::test]
@@ -630,6 +630,7 @@ fn candidate(id: &str, weight: u16) -> AccountCandidate {
in_flight: 0, in_flight: 0,
last_started_at: None, last_started_at: None,
quota_reset_at: None, quota_reset_at: None,
quota_reset_windows: None,
quota_remaining_rank: None, quota_remaining_rank: None,
cooldown: None, cooldown: None,
failure_rate_basis_points: None, failure_rate_basis_points: None,
@@ -947,7 +947,7 @@ impl PricingSource for UnusedAdminRuntime {
impl ProxyProbe for UnusedAdminRuntime { impl ProxyProbe for UnusedAdminRuntime {
async fn test( async fn test(
&self, &self,
_: Option<&gateway_core::account::OutboundProxy>, _: gateway_admin::ports::proxy::ProbeRoute<'_>,
_: bool, _: bool,
) -> ProxyTestResult { ) -> ProxyTestResult {
panic!("unexpected proxy probe") panic!("unexpected proxy probe")
@@ -279,7 +279,7 @@ HTTP 和 request 中间件中的模型回调沿用有效请求设置;attempt
`HostClient::account_http(AccountHttpRequest { account_id, credential_revision, request }, body)` 使用指定账号的当前代理配置, `HostClient::account_http(AccountHttpRequest { account_id, credential_revision, request }, body)` 使用指定账号的当前代理配置,
经 `host.http.account` 返回同一 `HostHttpBody`。每次发送核对账号存在、启用和凭据版本,版本变化须由插件重新读取后决定下一步。 经 `host.http.account` 返回同一 `HostHttpBody`。每次发送核对账号存在、启用和凭据版本,版本变化须由插件重新读取后决定下一步。
请求 headers 与正文完全由插件提供;宿主不注入账号认证、不申请推理租约,代理不可用时直接失败,不回退直连。 请求 headers 与正文完全由插件提供;宿主不注入账号认证、不申请推理租约。无显式代理时使用部署默认出口,代理不可用时直接失败,不回退直连。
这项能力与上游适配器的账号推理 HTTP 分开,适用于插件自行构造的账号网络请求 这项能力与上游适配器的账号推理 HTTP 分开,适用于插件自行构造的账号网络请求
`HostClient::proxy_http(ProxyHttpRequest { proxy_id, request }, body)` 经 `host.http.proxy` 使用指定代理的当前配置, `HostClient::proxy_http(ProxyHttpRequest { proxy_id, request }, body)` 经 `host.http.proxy` 使用指定代理的当前配置,
@@ -85,7 +85,7 @@ let (message, ()) = tokio::try_join!(
``` ```
宿主先注入已选账号认证,插件显式提供的同名头覆盖默认值。路径支持相对基址或完整 HTTP(S) URL; 宿主先注入已选账号认证,插件显式提供的同名头覆盖默认值。路径支持相对基址或完整 HTTP(S) URL;
声明路径用于标记发送用途,不是访问白名单。出站复用账号代理,代理失败不能回退直连,重定向不自动跟随。 声明路径用于标记发送用途,不是访问白名单。出站复用账号代理,未绑定可用代理时使用部署默认出口,代理失败不能回退直连,重定向不自动跟随。
`auxiliary` 路径的出站计入请求副作用水位,防止附件上传等操作在失败后被透明重复执行 `auxiliary` 路径的出站计入请求副作用水位,防止附件上传等操作在失败后被透明重复执行
## 事件、结算和续接 ## 事件、结算和续接
@@ -154,6 +154,7 @@ pub struct RuntimeSettings {
pub openai_session_affinity_ttl_hours: u32, pub openai_session_affinity_ttl_hours: u32,
pub responses_max_decompressed_body_bytes: u64, pub responses_max_decompressed_body_bytes: u64,
pub smart_scheduling: SmartSchedulingConfig, pub smart_scheduling: SmartSchedulingConfig,
pub quota_reset_window_weights: ResetWindowWeights,
pub min_codex_desktop_version: Option<String>, pub min_codex_desktop_version: Option<String>,
pub min_codex_cli_version: Option<String>, pub min_codex_cli_version: Option<String>,
pub usage_retention_days: u32, pub usage_retention_days: u32,
@@ -194,6 +195,7 @@ pub struct ReplaceRuntimeSettings {
pub openai_session_affinity_ttl_hours: u32, pub openai_session_affinity_ttl_hours: u32,
pub responses_max_decompressed_body_bytes: u64, pub responses_max_decompressed_body_bytes: u64,
pub smart_scheduling: SmartSchedulingConfig, pub smart_scheduling: SmartSchedulingConfig,
pub quota_reset_window_weights: ResetWindowWeights,
pub min_codex_desktop_version: Option<String>, pub min_codex_desktop_version: Option<String>,
pub min_codex_cli_version: Option<String>, pub min_codex_cli_version: Option<String>,
pub usage_retention_days: u32, pub usage_retention_days: u32,
@@ -256,6 +258,15 @@ pub struct SmartSchedulingConfig {
pub latency_weight: f64, pub latency_weight: f64,
pub reset_weight: f64, pub reset_weight: f64,
pub queue_weight: f64, pub queue_weight: f64,
#[serde(default)]
pub reset_window_weights: ResetWindowWeights,
}
#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct ResetWindowWeights {
pub five_hour: f64,
pub weekly: f64,
pub monthly: f64,
} }
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] #[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)] #[serde(rename_all = "camelCase", deny_unknown_fields)]
@@ -396,6 +407,7 @@ impl From<RuntimeSettings> for ReplaceRuntimeSettings {
openai_session_affinity_ttl_hours: settings.openai_session_affinity_ttl_hours, openai_session_affinity_ttl_hours: settings.openai_session_affinity_ttl_hours,
responses_max_decompressed_body_bytes: settings.responses_max_decompressed_body_bytes, responses_max_decompressed_body_bytes: settings.responses_max_decompressed_body_bytes,
smart_scheduling: settings.smart_scheduling, smart_scheduling: settings.smart_scheduling,
quota_reset_window_weights: settings.quota_reset_window_weights,
min_codex_desktop_version: settings.min_codex_desktop_version, min_codex_desktop_version: settings.min_codex_desktop_version,
min_codex_cli_version: settings.min_codex_cli_version, min_codex_cli_version: settings.min_codex_cli_version,
usage_retention_days: settings.usage_retention_days, usage_retention_days: settings.usage_retention_days,
@@ -5,6 +5,7 @@ mod content;
mod error; mod error;
mod request; mod request;
mod response; mod response;
mod session;
mod stream; mod stream;
mod thinking; mod thinking;
mod tools; mod tools;
@@ -20,6 +21,7 @@ pub use request::{
pub use response::{ pub use response::{
messages_to_responses_event, messages_to_responses_response, responses_to_messages_response, messages_to_responses_event, messages_to_responses_response, responses_to_messages_response,
}; };
pub use session::claude_session_id;
pub use stream::{ pub use stream::{
MessagesAccumulator, MessagesToResponsesStream, ResponsesToMessagesStream, MessagesAccumulator, MessagesToResponsesStream, ResponsesToMessagesStream,
message_to_stream_events, message_to_stream_events,
@@ -0,0 +1,36 @@
//! Claude Code 明确会话身份的请求头与 metadata 解析
use serde_json::{Map, Value};
/// 优先使用 Claude Code 会话头,再读取新旧 metadata.user_id 中的会话标识
///
/// 普通用户标识与对话内容不作为会话身份
#[must_use]
pub fn claude_session_id(body: &Map<String, Value>, header: Option<&str>) -> Option<String> {
if let Some(session) = header.and_then(valid_session_id) {
return Some(session.to_owned());
}
let user_id = body.get("metadata")?.get("user_id")?.as_str()?.trim();
if user_id.starts_with('{') {
let metadata: Value = serde_json::from_str(user_id).ok()?;
return metadata
.get("session_id")?
.as_str()
.and_then(valid_session_id)
.map(str::to_owned);
}
let (_, session) = user_id.rsplit_once("_session_")?;
let session = valid_session_id(session)?;
session
.bytes()
.all(|byte| byte.is_ascii_hexdigit() || byte == b'-')
.then(|| session.to_owned())
}
fn valid_session_id(value: &str) -> Option<&str> {
if value.chars().any(char::is_control) {
return None;
}
let value = value.trim();
(!value.is_empty() && value.len() <= 256).then_some(value)
}
@@ -3,5 +3,6 @@
mod capabilities; mod capabilities;
mod request; mod request;
mod response; mod response;
mod session;
mod stream; mod stream;
mod usage; mod usage;
@@ -0,0 +1,69 @@
//! Claude Code 会话来源优先级、格式兼容与无效身份边界
use gateway_protocol::anthropic::claude_session_id;
use serde_json::json;
#[test]
fn explicit_claude_sessions_follow_header_then_metadata() {
let session = "12345678-1234-1234-1234-123456789abc";
for (header, user_id, expected) in [
(
Some(" header-session "),
json!("{\"session_id\":\"body-session\"}"),
Some("header-session"),
),
(
None,
json!("{\"device_id\":\"device\",\"session_id\":\"body-session\"}"),
Some("body-session"),
),
(
None,
json!(format!("user_device_account__session_{session}")),
Some(session),
),
(
Some(""),
json!(format!("user_{}_session_{session}", "x".repeat(300))),
Some(session),
),
(None, json!("ordinary-user-id"), None),
(None, json!("{\"device_id\":\"device\"}"), None),
(None, json!("{invalid}"), None),
(None, json!({"session_id":"not-a-string"}), None),
(None, json!("user_device_session_"), None),
(None, json!("user_device_session_not-a-uuid"), None),
] {
let body =
json!({"metadata":{"user_id":user_id},"messages":[{"role":"user","content":"hello"}]});
assert_eq!(
claude_session_id(body.as_object().unwrap(), header).as_deref(),
expected
);
}
}
#[test]
fn invalid_session_signals_do_not_bind_users_or_message_content() {
for invalid in [String::new(), "x".repeat(257), "bad\nidentity".to_owned()] {
let body = json!({"metadata":{"user_id":json!({"session_id":invalid}).to_string()}});
assert_eq!(
claude_session_id(body.as_object().unwrap(), Some(&invalid)),
None
);
let fallback = json!({"metadata":{"user_id":"{\"session_id\":\"valid-fallback\"}"}});
assert_eq!(
claude_session_id(fallback.as_object().unwrap(), Some(&invalid)).as_deref(),
Some("valid-fallback")
);
}
assert_eq!(
claude_session_id(
json!({"messages":[{"role":"user","content":"same opening"}]})
.as_object()
.unwrap(),
None
),
None
);
}
@@ -114,6 +114,7 @@ impl SettingsStore for AdminSettingsStoreAdapter {
"responses_max_decompressed_body_bytes".to_owned(), "responses_max_decompressed_body_bytes".to_owned(),
"rotation_strategy".to_owned(), "rotation_strategy".to_owned(),
"smart_scheduling_json".to_owned(), "smart_scheduling_json".to_owned(),
"quota_reset_window_weights_json".to_owned(),
"min_codex_desktop_version".to_owned(), "min_codex_desktop_version".to_owned(),
"min_codex_cli_version".to_owned(), "min_codex_cli_version".to_owned(),
"retention".to_owned(), "retention".to_owned(),
+2 -1
View File
@@ -336,7 +336,8 @@ async fn connect(
), ),
}; };
let analytics = let analytics =
postgres::PgRequestAnalyticsRepository::new(pool.clone(), observability_query_budget); postgres::PgRequestAnalyticsRepository::new(pool.clone(), observability_query_budget)
.with_projection_work_mem_mb(config.pool.analytics_projection_work_mem_mb);
Ok(StoreBundle { Ok(StoreBundle {
pool, pool,
analytics, analytics,
@@ -37,6 +37,7 @@ pub struct StorePoolConfig {
pub max_connections: u32, pub max_connections: u32,
pub acquire_timeout_seconds: u64, pub acquire_timeout_seconds: u64,
pub observability_timeout_seconds: u64, pub observability_timeout_seconds: u64,
pub analytics_projection_work_mem_mb: Option<u32>,
} }
impl Default for StorePoolConfig { impl Default for StorePoolConfig {
@@ -45,12 +46,15 @@ impl Default for StorePoolConfig {
max_connections: 20, max_connections: 20,
acquire_timeout_seconds: 5, acquire_timeout_seconds: 5,
observability_timeout_seconds: 120, observability_timeout_seconds: 120,
analytics_projection_work_mem_mb: None,
} }
} }
} }
impl StorePoolConfig { impl StorePoolConfig {
pub(crate) fn validate(&self) -> StoreResult<()> { pub(crate) fn validate(&self) -> StoreResult<()> {
// PostgreSQL 以有符号 32 位的 kB 保存 work_mem,配置按整 MB 填写
const MAX_WORK_MEM_MB: u32 = i32::MAX as u32 / 1024;
if self.max_connections < 2 || self.acquire_timeout_seconds == 0 { if self.max_connections < 2 || self.acquire_timeout_seconds == 0 {
return Err(StoreError::InvalidData { return Err(StoreError::InvalidData {
source: None, source: None,
@@ -69,6 +73,18 @@ impl StorePoolConfig {
message: "pool.observability_timeout_seconds must be positive and fit PostgreSQL statement_timeout".to_owned(), message: "pool.observability_timeout_seconds must be positive and fit PostgreSQL statement_timeout".to_owned(),
}); });
} }
if self
.analytics_projection_work_mem_mb
.is_some_and(|mb| !(1..=MAX_WORK_MEM_MB).contains(&mb))
{
return Err(StoreError::InvalidData {
source: None,
entity: "store config",
message: format!(
"pool.analytics_projection_work_mem_mb must be between 1 and {MAX_WORK_MEM_MB}"
),
});
}
Ok(()) Ok(())
} }
@@ -658,20 +658,41 @@ fn account_costs(state: &MetricState) -> StoreResult<Vec<AccountCost>> {
.collect()) .collect())
} }
async fn retained_account_usage_range(
snapshot: &mut AnalyticsSnapshot<'_>,
now: DateTime<Utc>,
) -> StoreResult<ObservabilityRange> {
// 只为实际存在的数据规划时间桶,超长保留期不展开空白历史或溢出日期
let (days, earliest): (i64, DateTime<Utc>) = sqlx::query_as(
"select usage_retention_days, least(
(select started_at from model_requests order by started_at, id limit 1),
(select started_at from request_analytics_facts order by started_at, id limit 1),
(select bucket_start from request_analytics_1h where collection = 'usage_contribution' order by bucket_start limit 1),
$1::timestamptz - interval '1 day'
) from runtime_settings where id = 1",
)
.bind(now)
.fetch_one(&mut *snapshot.tx)
.await
.map_err(|error| postgres_unavailable("read retained account usage range", error))?;
let duration =
TimeDelta::try_days(days).ok_or_else(|| invalid("invalid account usage retention"))?;
let start = now
.checked_sub_signed(duration)
.unwrap_or(earliest)
.max(earliest);
ObservabilityRange::new(start, now)
}
pub(crate) async fn account_usage_sort_values( pub(crate) async fn account_usage_sort_values(
snapshot: &mut AnalyticsSnapshot<'_>, snapshot: &mut AnalyticsSnapshot<'_>,
now: DateTime<Utc>, now: DateTime<Utc>,
) -> StoreResult<(Vec<String>, Vec<i64>, Vec<Option<DateTime<Utc>>>)> { ) -> StoreResult<(Vec<String>, Vec<i64>, Vec<Option<DateTime<Utc>>>)> {
let days: i32 = let range = retained_account_usage_range(snapshot, now).await?;
sqlx::query_scalar("select usage_retention_days::integer from runtime_settings where id=1")
.fetch_one(&mut *snapshot.tx)
.await
.map_err(|error| postgres_unavailable("read account usage retention", error))?;
let ids: Vec<String> = sqlx::query_scalar("select id from provider_accounts") let ids: Vec<String> = sqlx::query_scalar("select id from provider_accounts")
.fetch_all(&mut *snapshot.tx) .fetch_all(&mut *snapshot.tx)
.await .await
.map_err(|error| postgres_unavailable("read account usage sort identities", error))?; .map_err(|error| postgres_unavailable("read account usage sort identities", error))?;
let range = ObservabilityRange::new(now - TimeDelta::days(i64::from(days)), now)?;
let mut totals = BTreeMap::<String, MetricState>::new(); let mut totals = BTreeMap::<String, MetricState>::new();
for ((account, _), state) in snapshot.account_states(range, &ids, false).await? { for ((account, _), state) in snapshot.account_states(range, &ids, false).await? {
totals.entry(account).or_default().merge(&state)?; totals.entry(account).or_default().merge(&state)?;
@@ -699,11 +720,7 @@ pub(crate) async fn account_group_usage(
} }
let mut snapshot = AnalyticsSnapshot::new_for_accounts(connection).await?; let mut snapshot = AnalyticsSnapshot::new_for_accounts(connection).await?;
let now = Utc::now(); let now = Utc::now();
let days: i32 = let retained = retained_account_usage_range(&mut snapshot, now).await?;
sqlx::query_scalar("select usage_retention_days::integer from runtime_settings where id=1")
.fetch_one(&mut *snapshot.tx)
.await
.map_err(|error| postgres_unavailable("read group usage retention", error))?;
let memberships:Vec<(String,String)>=sqlx::query_as("select account_group_id,provider_account_id from account_group_accounts where account_group_id=any($1::text[])").bind(group_ids).fetch_all(&mut *snapshot.tx).await.map_err(|error|postgres_unavailable("read group usage membership",error))?; let memberships:Vec<(String,String)>=sqlx::query_as("select account_group_id,provider_account_id from account_group_accounts where account_group_id=any($1::text[])").bind(group_ids).fetch_all(&mut *snapshot.tx).await.map_err(|error|postgres_unavailable("read group usage membership",error))?;
let ids = memberships let ids = memberships
.iter() .iter()
@@ -711,7 +728,6 @@ pub(crate) async fn account_group_usage(
.collect::<BTreeSet<_>>() .collect::<BTreeSet<_>>()
.into_iter() .into_iter()
.collect::<Vec<_>>(); .collect::<Vec<_>>();
let retained = ObservabilityRange::new(now - TimeDelta::days(i64::from(days)), now)?;
let today = ObservabilityRange::new( let today = ObservabilityRange::new(
timezone timezone
.day_start(now) .day_start(now)
@@ -2,7 +2,7 @@
use super::*; use super::*;
/// 输入 matching 提供 output_bucket、name 和完整事实列及 is_completed_usage /// 输入 matching 提供分组键、指标输入列及 is_completed_usage
/// 精确分位数在整个结果组内计算,不把局部分位数合并成新的分位数 /// 精确分位数在整个结果组内计算,不把局部分位数合并成新的分位数
pub(super) fn push_fact_aggregation( pub(super) fn push_fact_aggregation(
query: &mut QueryBuilder<Postgres>, query: &mut QueryBuilder<Postgres>,
@@ -379,8 +379,17 @@ async fn rebuild_groups(
rebuilt.insert((*key).clone(), MetricState::default()); rebuilt.insert((*key).clone(), MetricState::default());
} }
// 等值桶键避免范围连接反复比较其他时间桶,范围和游标仍限定已发布前缀 // 等值桶键避免范围连接反复比较其他时间桶,范围和游标仍限定已发布前缀
// 计数、费用和直方图复用匹配结果,只物化这些聚合需要的列
let mut query = QueryBuilder::<Postgres>::new( let mut query = QueryBuilder::<Postgres>::new(
"with matching as materialized (select mr.*,k.bucket_start as output_bucket,k.name,coalesce((", "with matching as materialized (select k.bucket_start as output_bucket,k.name,
mr.started_at,mr.outcome,mr.client_status_code,mr.upstream_status_code,mr.attempt_count,
mr.image_generation_succeeded,mr.error_kind,mr.ops_failure_count,mr.ops_rate_limited_count,
mr.ops_auth_failure_count,mr.ops_provider_5xx_count,mr.input_tokens,mr.output_tokens,
mr.cached_input_tokens,mr.cache_write_input_tokens,mr.reasoning_output_tokens,
mr.image_input_tokens,mr.image_output_tokens,mr.total_tokens,
mr.cost_source,mr.cost_currency,mr.cost_amount,mr.standard_cost_amount,mr.no_cache_cost_amount,
mr.first_token_ms,mr.latency_ms,mr.admission_decision_ms,mr.account_selection_wait_ms,
mr.upstream_response_ms,mr.capacity_used_slots,mr.capacity_total_slots,coalesce((",
); );
query.push(completed_usage_fact_predicate("mr")) query.push(completed_usage_fact_predicate("mr"))
.push("),false) as is_completed_usage from jsonb_to_recordset(") .push("),false) as is_completed_usage from jsonb_to_recordset(")
@@ -32,6 +32,7 @@ pub struct AnalyticsStatus {
pub struct PgRequestAnalyticsRepository { pub struct PgRequestAnalyticsRepository {
pool: PgPool, pool: PgPool,
query_budget: ObservabilityQueryBudget, query_budget: ObservabilityQueryBudget,
projection_work_mem_mb: Option<u32>,
billing_projector: Option<Arc<dyn UsageBillingProjector>>, billing_projector: Option<Arc<dyn UsageBillingProjector>>,
} }
@@ -41,10 +42,17 @@ impl PgRequestAnalyticsRepository {
Self { Self {
pool, pool,
query_budget, query_budget,
projection_work_mem_mb: None,
billing_projector: None, billing_projector: None,
} }
} }
#[must_use]
pub fn with_projection_work_mem_mb(mut self, work_mem_mb: Option<u32>) -> Self {
self.projection_work_mem_mb = work_mem_mb;
self
}
#[must_use] #[must_use]
pub fn with_billing_projector(mut self, projector: Arc<dyn UsageBillingProjector>) -> Self { pub fn with_billing_projector(mut self, projector: Arc<dyn UsageBillingProjector>) -> Self {
self.billing_projector = Some(projector); self.billing_projector = Some(projector);
@@ -82,6 +90,10 @@ impl PgRequestAnalyticsRepository {
transaction.commit().await.map_err(|error| postgres_unavailable("save analytics reset progress", error))?; transaction.commit().await.map_err(|error| postgres_unavailable("save analytics reset progress", error))?;
return Ok(AnalyticsMaintenance { reset_progress: true, ..Default::default() }); return Ok(AnalyticsMaintenance { reset_progress: true, ..Default::default() });
} }
if let Some(mb) = self.projection_work_mem_mb {
sqlx::query("select set_config('work_mem',$1,true)").bind(format!("{mb}MB"))
.execute(&mut *transaction).await.map_err(|error| postgres_unavailable("set analytics projection work_mem", error))?;
}
let mut result = AnalyticsMaintenance::default(); let mut result = AnalyticsMaintenance::default();
// 历史删除可能一次排入大量旧任务,给仍存在的请求预留半批名额,避免实时统计饥饿 // 历史删除可能一次排入大量旧任务,给仍存在的请求预留半批名额,避免实时统计饥饿
// 其余名额仍按最早入队消费,两类任务都能前进;同一快照内选版本,沿用精确版本 ACK // 其余名额仍按最早入队消费,两类任务都能前进;同一快照内选版本,沿用精确版本 ACK
@@ -144,8 +144,7 @@ impl PgProviderAccountRepository {
query.push(" and not exists (select 1 from provider_account_model_catalogs c query.push(" and not exists (select 1 from provider_account_model_catalogs c
where c.account_id=provider_accounts.id and c.credential_revision=provider_accounts.credential_revision where c.account_id=provider_accounts.id and c.credential_revision=provider_accounts.credential_revision
and c.plan_type is not distinct from provider_accounts.plan_type and c.document_hash=any(") and c.plan_type is not distinct from provider_accounts.plan_type and c.document_hash=any(")
.push_bind(hashes).push("::bytea[]) and c.observed_at > ").push_bind(DateTime::<Utc>::from(omit.observed_after)) .push_bind(hashes).push("::bytea[]) and c.observed_at <= ").push_bind(DateTime::<Utc>::from(omit.as_of))
.push(" and c.observed_at <= ").push_bind(DateTime::<Utc>::from(omit.as_of))
.push(" and not (c.account_id=any(").push_bind(exceptions).push("::text[])))"); .push(" and not (c.account_id=any(").push_bind(exceptions).push("::text[])))");
} }
query.push(" order by provider_kind, name, id"); query.push(" order by provider_kind, name, id");
@@ -2,7 +2,7 @@
use crate::{admin_store_error, postgres_unavailable}; use crate::{admin_store_error, postgres_unavailable};
use async_trait::async_trait; use async_trait::async_trait;
use chrono::{DateTime, Utc}; use chrono::{DateTime, TimeDelta, Utc};
use gateway_admin::{ use gateway_admin::{
model::retention::{RetentionPolicy, RetentionTarget}, model::retention::{RetentionPolicy, RetentionTarget},
ports::{ ports::{
@@ -54,15 +54,26 @@ impl RetentionStore for PgRetentionRepository {
policy: RetentionPolicy, policy: RetentionPolicy,
limit: NonZeroU32, limit: NonZeroU32,
) -> AdminStoreResult<u64> { ) -> AdminStoreResult<u64> {
let oldest_sql = match target {
RetentionTarget::ModelRequests => {
"select completed_at from model_requests where outcome <> 'running' order by completed_at limit 1"
}
RetentionTarget::OpsEvents => {
"select created_at from ops_events where model_request_id is null order by created_at limit 1"
}
RetentionTarget::AdminAuditEvents => {
"select created_at from admin_audit_events order by created_at limit 1"
}
};
let sql = match target { let sql = match target {
RetentionTarget::ModelRequests => { RetentionTarget::ModelRequests => {
"delete from model_requests "delete from model_requests
where ctid in ( where ctid in (
select ctid from model_requests select ctid from model_requests
where outcome <> 'running' where outcome <> 'running'
and completed_at < $1 - ($2 * interval '1 day') and completed_at < $1
order by completed_at order by completed_at
limit $3 limit $2
for update skip locked for update skip locked
)" )"
} }
@@ -71,9 +82,9 @@ impl RetentionStore for PgRetentionRepository {
where ctid in ( where ctid in (
select ctid from ops_events select ctid from ops_events
where model_request_id is null where model_request_id is null
and created_at < $1 - ($2 * interval '1 day') and created_at < $1
order by created_at order by created_at
limit $3 limit $2
for update skip locked for update skip locked
)" )"
} }
@@ -81,9 +92,9 @@ impl RetentionStore for PgRetentionRepository {
"delete from admin_audit_events "delete from admin_audit_events
where ctid in ( where ctid in (
select ctid from admin_audit_events select ctid from admin_audit_events
where created_at < $1 - ($2 * interval '1 day') where created_at < $1
order by created_at order by created_at
limit $3 limit $2
for update skip locked for update skip locked
)" )"
} }
@@ -102,14 +113,25 @@ impl RetentionStore for PgRetentionRepository {
.execute(&mut *transaction) .execute(&mut *transaction)
.await .await
.map_err(|source| unavailable("set retention lock timeout", source))?; .map_err(|source| unavailable("set retention lock timeout", source))?;
let deleted = sqlx::query(sql) let oldest: Option<DateTime<Utc>> = sqlx::query_scalar(oldest_sql)
.bind(now) .fetch_optional(&mut *transaction)
.bind(i64::from(policy.days(target)))
.bind(i64::from(limit.get()))
.execute(&mut *transaction)
.await .await
.map(|result| result.rows_affected()) .map_err(|source| unavailable("read oldest retained record", source))?;
.map_err(|source| unavailable("delete expired records", source))?; // 超长窗口覆盖全部现存记录时无需删除,也不向数据库传递越界日期
let cutoff = now
.checked_sub_signed(TimeDelta::days(i64::from(policy.days(target))))
.filter(|cutoff| oldest.is_some_and(|oldest| oldest < *cutoff));
let deleted = if let Some(cutoff) = cutoff {
sqlx::query(sql)
.bind(cutoff)
.bind(i64::from(limit.get()))
.execute(&mut *transaction)
.await
.map(|result| result.rows_affected())
.map_err(|source| unavailable("delete expired records", source))?
} else {
0
};
transaction transaction
.commit() .commit()
.await .await
@@ -153,7 +153,7 @@ pub(crate) async fn load_runtime_settings_from_pool(pool: &PgPool) -> StoreResul
let row = sqlx::query_as::<_, RuntimeSettingsRow>( let row = sqlx::query_as::<_, RuntimeSettingsRow>(
"select provider_request_profiles_json, config_revision, admin_api_key, refresh_margin_seconds, request_location_json, request_location_enabled, codex_privacy_policy_json, "select provider_request_profiles_json, config_revision, admin_api_key, refresh_margin_seconds, request_location_json, request_location_enabled, codex_privacy_policy_json,
refresh_concurrency, max_concurrent_per_account, refresh_concurrency, max_concurrent_per_account,
rotation_strategy, smart_scheduling_json, model_mappings_json, usage_retention_days, ops_event_retention_days, rotation_strategy, smart_scheduling_json, quota_reset_window_weights_json, model_mappings_json, usage_retention_days, ops_event_retention_days,
audit_retention_days, min_codex_desktop_version, audit_retention_days, min_codex_desktop_version,
min_codex_cli_version, updated_at, responses_max_decompressed_body_bytes, max_waiting_per_key, max_waiting_per_account, key_queue_timeout_seconds, account_queue_timeout_seconds, switch_account_on_queue_failure, max_retry_after_seconds, openai_guardian_reserved_concurrency, openai_account_affinity, max_account_rotations, openai_session_affinity_ttl_hours, min_codex_cli_version, updated_at, responses_max_decompressed_body_bytes, max_waiting_per_key, max_waiting_per_account, key_queue_timeout_seconds, account_queue_timeout_seconds, switch_account_on_queue_failure, max_retry_after_seconds, openai_guardian_reserved_concurrency, openai_account_affinity, max_account_rotations, openai_session_affinity_ttl_hours,
account_error_protection_json, account_error_protection_revision, account_error_protection_json, account_error_protection_revision,
@@ -195,6 +195,30 @@ impl ProviderRuntimePolicyPort for PgRuntimeSettingsRepository {
Ok(claimed) Ok(claimed)
}) })
} }
fn load_request_profile<'a>(
&'a self,
provider: &'a gateway_core::routing::ProviderKind,
) -> futures::future::BoxFuture<
'a,
Result<Option<gateway_core::account::OpaqueProviderData>, ProviderStoreError>,
> {
Box::pin(async move {
let document = sqlx::query_scalar::<
_,
Option<sqlx::types::Json<serde_json::Map<String, serde_json::Value>>>,
>(
"select provider_request_profiles_json -> $1 from runtime_settings where id = 1",
)
.bind(provider.as_str())
.fetch_one(&self.pool)
.await
.map_err(|source| {
crate::provider_unavailable("load Provider request profile", source)
})?;
Ok(document.map(|document| gateway_core::account::OpaqueProviderData::new(document.0)))
})
}
fn initialize_request_profile<'a>( fn initialize_request_profile<'a>(
&'a self, &'a self,
provider: &'a gateway_core::routing::ProviderKind, provider: &'a gateway_core::routing::ProviderKind,
@@ -296,7 +320,7 @@ impl ProviderRuntimePolicyPort for PgRuntimeSettingsRepository {
.map_err(|source| crate::provider_unavailable("load warmup policy", source))?; .map_err(|source| crate::provider_unavailable("load warmup policy", source))?;
ProviderWarmupPolicy::try_new( ProviderWarmupPolicy::try_new(
settings.values.account_warmup_enabled, settings.values.account_warmup_enabled,
settings.values.account_warmup_cron, &settings.values.account_warmup_cron,
settings.values.account_warmup_model, settings.values.account_warmup_model,
) )
}) })
@@ -309,7 +333,7 @@ pub(crate) async fn load_runtime_settings_in_transaction(
let row = sqlx::query_as::<_, RuntimeSettingsRow>( let row = sqlx::query_as::<_, RuntimeSettingsRow>(
"select provider_request_profiles_json, config_revision, admin_api_key, refresh_margin_seconds, request_location_json, request_location_enabled, codex_privacy_policy_json, "select provider_request_profiles_json, config_revision, admin_api_key, refresh_margin_seconds, request_location_json, request_location_enabled, codex_privacy_policy_json,
refresh_concurrency, max_concurrent_per_account, refresh_concurrency, max_concurrent_per_account,
rotation_strategy, smart_scheduling_json, model_mappings_json, usage_retention_days, ops_event_retention_days, rotation_strategy, smart_scheduling_json, quota_reset_window_weights_json, model_mappings_json, usage_retention_days, ops_event_retention_days,
audit_retention_days, min_codex_desktop_version, audit_retention_days, min_codex_desktop_version,
min_codex_cli_version, updated_at, responses_max_decompressed_body_bytes, max_waiting_per_key, max_waiting_per_account, key_queue_timeout_seconds, account_queue_timeout_seconds, switch_account_on_queue_failure, max_retry_after_seconds, openai_guardian_reserved_concurrency, openai_account_affinity, max_account_rotations, openai_session_affinity_ttl_hours, min_codex_cli_version, updated_at, responses_max_decompressed_body_bytes, max_waiting_per_key, max_waiting_per_account, key_queue_timeout_seconds, account_queue_timeout_seconds, switch_account_on_queue_failure, max_retry_after_seconds, openai_guardian_reserved_concurrency, openai_account_affinity, max_account_rotations, openai_session_affinity_ttl_hours,
account_error_protection_json, account_error_protection_revision, account_error_protection_json, account_error_protection_revision,
@@ -395,6 +419,7 @@ pub(crate) async fn update_runtime_settings_in_transaction(
account_queue_timeout_seconds = $32, account_queue_timeout_seconds = $32,
switch_account_on_queue_failure = $33, switch_account_on_queue_failure = $33,
reset_credit_on_low_weekly_quota = $34, reset_credit_on_low_weekly_quota = $34,
quota_reset_window_weights_json = $35,
updated_at = now() updated_at = now()
where id = 1 where id = 1
returning config_revision", returning config_revision",
@@ -441,6 +466,7 @@ pub(crate) async fn update_runtime_settings_in_transaction(
.bind(i64::from(update.values.account_queue_timeout_seconds)) .bind(i64::from(update.values.account_queue_timeout_seconds))
.bind(update.values.switch_account_on_queue_failure) .bind(update.values.switch_account_on_queue_failure)
.bind(update.values.reset_credit_on_low_weekly_quota) .bind(update.values.reset_credit_on_low_weekly_quota)
.bind(sqlx::types::Json(update.values.quota_reset_window_weights))
.fetch_optional(&mut **transaction) .fetch_optional(&mut **transaction)
.await .await
.map_err(|source| postgres_unavailable("update runtime settings in transaction", source))? .map_err(|source| postgres_unavailable("update runtime settings in transaction", source))?
@@ -514,6 +540,7 @@ struct RuntimeSettingsRow {
refresh_concurrency: i64, refresh_concurrency: i64,
max_concurrent_per_account: i64, max_concurrent_per_account: i64,
smart_scheduling_json: sqlx::types::Json<gateway_core::account::SmartSchedulingConfig>, smart_scheduling_json: sqlx::types::Json<gateway_core::account::SmartSchedulingConfig>,
quota_reset_window_weights_json: sqlx::types::Json<gateway_core::account::ResetWindowWeights>,
rotation_strategy: String, rotation_strategy: String,
codex_privacy_policy_json: codex_privacy_policy_json:
sqlx::types::Json<gateway_core::settings::privacy::CodexPrivacyPolicy>, sqlx::types::Json<gateway_core::settings::privacy::CodexPrivacyPolicy>,
@@ -576,6 +603,7 @@ fn runtime_settings_from_row(row: RuntimeSettingsRow) -> StoreResult<RuntimeSett
refresh_concurrency: to_u32(row.refresh_concurrency)?, refresh_concurrency: to_u32(row.refresh_concurrency)?,
max_concurrent_per_account: to_u32(row.max_concurrent_per_account)?, max_concurrent_per_account: to_u32(row.max_concurrent_per_account)?,
smart_scheduling: row.smart_scheduling_json.0, smart_scheduling: row.smart_scheduling_json.0,
quota_reset_window_weights: row.quota_reset_window_weights_json.0,
codex_privacy_policy: row.codex_privacy_policy_json.0, codex_privacy_policy: row.codex_privacy_policy_json.0,
request_location_enabled: row.request_location_enabled, request_location_enabled: row.request_location_enabled,
request_location: row request_location: row
@@ -221,6 +221,7 @@ struct SnapshotSettingsRow {
account_error_protection_json: sqlx::types::Json<gateway_core::account::AccountErrorProtection>, account_error_protection_json: sqlx::types::Json<gateway_core::account::AccountErrorProtection>,
max_concurrent_per_account: i64, max_concurrent_per_account: i64,
smart_scheduling_json: sqlx::types::Json<gateway_core::account::SmartSchedulingConfig>, smart_scheduling_json: sqlx::types::Json<gateway_core::account::SmartSchedulingConfig>,
quota_reset_window_weights_json: sqlx::types::Json<gateway_core::account::ResetWindowWeights>,
rotation_strategy: String, rotation_strategy: String,
model_mappings_json: sqlx::types::Json<BTreeMap<String, String>>, model_mappings_json: sqlx::types::Json<BTreeMap<String, String>>,
min_codex_desktop_version: Option<String>, min_codex_desktop_version: Option<String>,
@@ -248,7 +249,7 @@ async fn load_settings(
transaction: &mut Transaction<'_, Postgres>, transaction: &mut Transaction<'_, Postgres>,
) -> StoreResult<(Revision, SettingsValues)> { ) -> StoreResult<(Revision, SettingsValues)> {
let row = sqlx::query_as::<_, SnapshotSettingsRow>( let row = sqlx::query_as::<_, SnapshotSettingsRow>(
"select config_revision, account_error_protection_revision, account_error_protection_json, max_concurrent_per_account, rotation_strategy, smart_scheduling_json, model_mappings_json, min_codex_desktop_version, min_codex_cli_version, max_waiting_per_key, max_waiting_per_account, key_queue_timeout_seconds, account_queue_timeout_seconds, switch_account_on_queue_failure, max_retry_after_seconds, openai_guardian_reserved_concurrency, openai_account_affinity, max_account_rotations, openai_session_affinity_ttl_hours, request_location_json, request_location_enabled, codex_privacy_policy_json, responses_max_decompressed_body_bytes, provider_request_profiles_json, pricing_overrides_json, pricing_synced_json from runtime_settings where id = 1", "select config_revision, account_error_protection_revision, account_error_protection_json, max_concurrent_per_account, rotation_strategy, smart_scheduling_json, quota_reset_window_weights_json, model_mappings_json, min_codex_desktop_version, min_codex_cli_version, max_waiting_per_key, max_waiting_per_account, key_queue_timeout_seconds, account_queue_timeout_seconds, switch_account_on_queue_failure, max_retry_after_seconds, openai_guardian_reserved_concurrency, openai_account_affinity, max_account_rotations, openai_session_affinity_ttl_hours, request_location_json, request_location_enabled, codex_privacy_policy_json, responses_max_decompressed_body_bytes, provider_request_profiles_json, pricing_overrides_json, pricing_synced_json from runtime_settings where id = 1",
) )
.fetch_optional(&mut **transaction) .fetch_optional(&mut **transaction)
.await .await
@@ -281,6 +282,7 @@ async fn load_settings(
.with_max_retry_after_seconds(to_u32(row.max_retry_after_seconds)?) .with_max_retry_after_seconds(to_u32(row.max_retry_after_seconds)?)
.with_openai_session_affinity_ttl_hours(to_u32(row.openai_session_affinity_ttl_hours)?) .with_openai_session_affinity_ttl_hours(to_u32(row.openai_session_affinity_ttl_hours)?)
.with_smart_scheduling(row.smart_scheduling_json.0) .with_smart_scheduling(row.smart_scheduling_json.0)
.with_quota_reset_window_weights(row.quota_reset_window_weights_json.0)
.with_request_profiles(decode_request_profiles( .with_request_profiles(decode_request_profiles(
row.provider_request_profiles_json.0, row.provider_request_profiles_json.0,
)?) )?)
@@ -676,6 +676,7 @@ impl RedisProviderLeaseCoordinator {
in_flight: signal.in_flight, in_flight: signal.in_flight,
last_started_at: signal.last_started_at.map(Into::into), last_started_at: signal.last_started_at.map(Into::into),
quota_reset_at: None, quota_reset_at: None,
quota_reset_windows: None,
quota_remaining_rank: None, quota_remaining_rank: None,
cooldown: None, cooldown: None,
failure_rate_basis_points: None, failure_rate_basis_points: None,
+22 -2
View File
@@ -7,7 +7,7 @@ const PASSWORD: &str = "111111111111111111111111111111111111111111111111";
#[test] #[test]
fn store_config_should_derive_backup_staging_from_runtime_data_dir() { fn store_config_should_derive_backup_staging_from_runtime_data_dir() {
let root = tempfile::tempdir().expect("runtime data root"); let root = tempfile::tempdir().expect("runtime data root");
let mut config = valid_config(); let mut config = config_with_pool(serde_json::json!({}));
config config
.resolve_and_validate(root.path()) .resolve_and_validate(root.path())
@@ -28,6 +28,7 @@ fn store_pool_config_defaults_reserve_capacity_for_observability() {
assert_eq!(pool.max_connections, 20); assert_eq!(pool.max_connections, 20);
assert_eq!(pool.acquire_timeout_seconds, 5); assert_eq!(pool.acquire_timeout_seconds, 5);
assert_eq!(pool.observability_timeout_seconds, 120); assert_eq!(pool.observability_timeout_seconds, 120);
assert_eq!(pool.analytics_projection_work_mem_mb, None);
assert_eq!(pool.observability_max_connections(), 16); assert_eq!(pool.observability_max_connections(), 16);
assert_eq!( assert_eq!(
StorePoolConfig { StorePoolConfig {
@@ -68,7 +69,7 @@ fn store_config_rejects_a_pool_that_cannot_reserve_both_traffic_classes() {
)); ));
} }
fn valid_config() -> StoreConfig { fn config_with_pool(pool: serde_json::Value) -> StoreConfig {
serde_json::from_value(serde_json::json!({ serde_json::from_value(serde_json::json!({
"database": { "database": {
"url": "postgres://codex_proxy@127.0.0.1:5432/codex_proxy", "url": "postgres://codex_proxy@127.0.0.1:5432/codex_proxy",
@@ -78,10 +79,29 @@ fn valid_config() -> StoreConfig {
"url": "redis://127.0.0.1:6379/", "url": "redis://127.0.0.1:6379/",
"password": PASSWORD, "password": PASSWORD,
}, },
"pool": pool,
})) }))
.expect("test Store configuration") .expect("test Store configuration")
} }
#[test]
fn analytics_projection_work_mem_accepts_whole_mb_within_postgres_limits() {
let root = tempfile::tempdir().unwrap();
for mb in [1_u32, 8, 2_097_151] {
config_with_pool(serde_json::json!({"analytics_projection_work_mem_mb": mb}))
.resolve_and_validate(root.path())
.unwrap();
}
for mb in [0_u32, 2_097_152] {
let error = config_with_pool(serde_json::json!({"analytics_projection_work_mem_mb": mb}))
.resolve_and_validate(root.path())
.unwrap_err();
assert!(
matches!(error, StoreError::InvalidData { message, .. } if message.contains("analytics_projection_work_mem_mb"))
);
}
}
#[test] #[test]
fn store_config_accepts_custom_observability_timeout_and_rejects_invalid_values() { fn store_config_accepts_custom_observability_timeout_and_rejects_invalid_values() {
let root = tempfile::tempdir().unwrap(); let root = tempfile::tempdir().unwrap();
@@ -693,9 +693,16 @@ async fn group_costs_follow_current_members_and_pending_source_replacements() {
} }
} }
assert!(ready, "group fixture must publish rollups"); assert!(ready, "group fixture must publish rollups");
let costs = load_retained_group_costs(&groups).await; for days in [31_i64, 365, i64::from(u32::MAX)] {
assert_eq!(costs[MIXED_GROUP], "3.25"); sqlx::query("update runtime_settings set usage_retention_days=$1")
assert_eq!(costs[EMPTY_GROUP], "0"); .bind(days)
.execute(&database.pool)
.await
.unwrap();
let costs = load_retained_group_costs(&groups).await;
assert_eq!(costs[MIXED_GROUP], "3.25");
assert_eq!(costs[EMPTY_GROUP], "0");
}
sqlx::query("update account_group_accounts set account_group_id=$1 where provider_account_id='acct_group_move'") sqlx::query("update account_group_accounts set account_group_id=$1 where provider_account_id='acct_group_move'")
.bind(EMPTY_GROUP).execute(&database.pool).await.unwrap(); .bind(EMPTY_GROUP).execute(&database.pool).await.unwrap();
@@ -316,6 +316,80 @@ async fn incremental_deletions_repair_extrema_last_usage_and_zero_currency_prese
database.close().await; database.close().await;
} }
#[tokio::test]
async fn projection_work_mem_is_local_to_the_worker_transaction() {
let Some(database) = TestDatabase::create("work_mem").await else {
return;
};
let pool = sqlx::postgres::PgPoolOptions::new()
.max_connections(1)
.connect_with(database.pool.connect_options().as_ref().clone())
.await
.unwrap();
// 固定同一会话,直接记录事实写入和队列确认时的值,并在复用连接时检查恢复
sqlx::raw_sql(
"set work_mem='2MB';
create table work_mem_audit(stage text, value text);
create function record_work_mem() returns trigger language plpgsql as $$
begin
insert into work_mem_audit values (tg_argv[0], current_setting('work_mem'));
return null;
end $$;
create trigger audit_fact after insert on request_analytics_facts
for each row execute function record_work_mem('projection');
create trigger audit_ack after delete on request_analytics_pending_requests
for each row execute function record_work_mem('ack')",
)
.execute(&pool)
.await
.unwrap();
let worker = PgRequestAnalyticsRepository::new(pool.clone(), observability_query_budget());
let session = || {
sqlx::query_as::<_, (i32, String)>("select pg_backend_pid(), current_setting('work_mem')")
};
let original = session().fetch_one(&pool).await.unwrap();
for (worker, expected) in [
(worker.clone(), "2MB"),
(worker.with_projection_work_mem_mb(Some(8)), "8MB"),
] {
seed(&pool, expected, hour(), &[]).await;
assert_eq!(worker.maintain(10).await.unwrap().projected_requests, 1);
let audit: Vec<(String, String)> =
sqlx::query_as("select stage, value from work_mem_audit order by stage")
.fetch_all(&pool)
.await
.unwrap();
assert_eq!(
audit,
[
("ack".into(), "2MB".into()),
("projection".into(), expected.into())
]
);
assert_eq!(session().fetch_one(&pool).await.unwrap(), original);
sqlx::query("truncate work_mem_audit")
.execute(&pool)
.await
.unwrap();
}
// 事实写入失败时投影事务回滚,原会话仍可供下一轮使用
sqlx::query("alter table work_mem_audit add check (value <> '8MB')")
.execute(&pool)
.await
.unwrap();
seed(&pool, "rollback", hour(), &[]).await;
let worker = PgRequestAnalyticsRepository::new(pool.clone(), observability_query_budget())
.with_projection_work_mem_mb(Some(8));
let error = worker.maintain(10).await.unwrap_err();
assert!(
error.to_string().contains("project analytics facts"),
"{error}"
);
assert_eq!(session().fetch_one(&pool).await.unwrap(), original);
pool.close().await;
database.close().await;
}
#[tokio::test] #[tokio::test]
async fn incremental_summary_write_failure_rolls_back_facts_and_preserves_queue() { async fn incremental_summary_write_failure_rolls_back_facts_and_preserves_queue() {
let Some(database) = TestDatabase::create("analytics_incremental_rollback").await else { let Some(database) = TestDatabase::create("analytics_incremental_rollback").await else {
@@ -170,7 +170,7 @@ async fn bulk_catalog_restore_keeps_ten_thousand_owners_and_fences_late_writes()
.iter() .iter()
.all(|a| !matches!(a.account_id.as_str(), "acct_1" | "acct_2" | "acct_3")) .all(|a| !matches!(a.account_id.as_str(), "acct_1" | "acct_2" | "acct_3"))
); );
// 数据库只能排除有当前版本、未过期的账号证据;显式例外仍进入选号 // 刷新到期不改变目录排除结果;凭据变化与显式例外仍进入选号
let hash: Vec<u8> = sqlx::query_scalar("select sha256($1::bytea)") let hash: Vec<u8> = sqlx::query_scalar("select sha256($1::bytea)")
.bind(&document) .bind(&document)
.fetch_one(&database.pool) .fetch_one(&database.pool)
@@ -181,7 +181,6 @@ async fn bulk_catalog_restore_keeps_ten_thousand_owners_and_fences_late_writes()
let filter = gateway_core::account::AccountCatalogOmissions { let filter = gateway_core::account::AccountCatalogOmissions {
documents: vec![hash.try_into().unwrap()], documents: vec![hash.try_into().unwrap()],
exceptions: vec![ProviderAccountId::new("acct_5").unwrap()], exceptions: vec![ProviderAccountId::new("acct_5").unwrap()],
observed_after: now - std::time::Duration::from_secs(1800),
as_of: now, as_of: now,
}; };
let ids = (1..=10000) let ids = (1..=10000)
@@ -195,9 +194,6 @@ async fn bulk_catalog_restore_keeps_ten_thousand_owners_and_fences_late_writes()
.iter() .iter()
.map(|a| a.id().as_str()) .map(|a| a.id().as_str())
.collect::<std::collections::BTreeSet<_>>(); .collect::<std::collections::BTreeSet<_>>();
assert_eq!( assert_eq!(ids, std::collections::BTreeSet::from(["acct_1", "acct_5"]));
ids,
std::collections::BTreeSet::from(["acct_1", "acct_4", "acct_5"])
);
database.close().await; database.close().await;
} }
@@ -105,6 +105,11 @@ async fn quota_windows_reconcile_moved_requests_and_pending_deletions_without_wa
assert_eq!(new.usage.reasoning_output_tokens, None); assert_eq!(new.usage.reasoning_output_tokens, None);
assert_eq!(new.usage.models[0].model, "new-model"); assert_eq!(new.usage.models[0].model, "new-model");
assert_eq!(new.usage.costs[0].amount.as_str(), "4.75"); assert_eq!(new.usage.costs[0].amount.as_str(), "4.75");
sqlx::query("update runtime_settings set usage_retention_days=$1")
.bind(i64::from(u32::MAX))
.execute(&database.pool)
.await
.unwrap();
for field in [AccountSortField::Usage, AccountSortField::LastUsedAt] { for field in [AccountSortField::Usage, AccountSortField::LastUsedAt] {
let ranked = store let ranked = store
.list_accounts( .list_accounts(
@@ -1,4 +1,4 @@
//! 验证短保留期升级、分批清理边界与统计异步收敛期间的查询可用性 //! 验证保留期读写、分批清理边界与统计异步收敛期间的查询可用性
use std::{num::NonZeroU32, time::Duration}; use std::{num::NonZeroU32, time::Duration};
@@ -20,6 +20,66 @@ use super::{
observability_query_budget, observability_repository, observability_query_budget, observability_repository,
}; };
#[tokio::test]
async fn long_retention_round_trips_and_cleans_only_expired_requests() {
use gateway_store::postgres::{PgRuntimeSettingsRepository, RuntimeSettingsRepository};
let Some(database) = TestDatabase::create("long_retention").await else {
return;
};
let settings = PgRuntimeSettingsRepository::new(database.pool.clone());
let retention = PgRetentionRepository::new(database.pool.clone());
let now = Utc::now();
for (id, age) in [("old", 400), ("month", 40), ("recent", 1)] {
seed(&database.pool, id, now - ChronoDuration::days(age), &[]).await;
}
for (days, expected_deleted) in [(u32::MAX, 0), (3650, 0), (365, 1), (32, 1), (1, 0)] {
let mut update = super::runtime_settings::settings_with_margin(300);
update.values.usage_retention_days = days;
settings.update_runtime_settings(update).await.unwrap();
assert_eq!(
settings
.load_runtime_settings()
.await
.unwrap()
.values
.usage_retention_days,
days
);
let policy = retention.load_policy().await.unwrap();
assert_eq!(policy.days(RetentionTarget::ModelRequests), days);
assert_eq!(
retention
.purge_batch(
RetentionTarget::ModelRequests,
now,
policy,
NonZeroU32::new(10).unwrap()
)
.await
.unwrap(),
expected_deleted
);
}
let mut invalid = super::runtime_settings::settings_with_margin(300);
invalid.values.usage_retention_days = 0;
assert!(settings.update_runtime_settings(invalid).await.is_err());
assert_eq!(
retention
.load_policy()
.await
.unwrap()
.days(RetentionTarget::ModelRequests),
1
);
let remaining: Vec<String> = sqlx::query_scalar("select id from model_requests order by id")
.fetch_all(&database.pool)
.await
.unwrap();
assert_eq!(remaining, ["recent"]);
database.close().await;
}
#[tokio::test] #[tokio::test]
async fn short_retention_keeps_statistics_readable_until_each_deletion_batch_is_projected() { async fn short_retention_keeps_statistics_readable_until_each_deletion_batch_is_projected() {
use gateway_admin::model::observability::{AnalyticsState, DashboardQuery, TimeRange}; use gateway_admin::model::observability::{AnalyticsState, DashboardQuery, TimeRange};
@@ -40,6 +40,7 @@ pub(super) fn settings_with_margin(refresh_margin_seconds: u64) -> RuntimeSettin
openai_session_affinity_ttl_hours: 24, openai_session_affinity_ttl_hours: 24,
responses_max_decompressed_body_bytes: 64 * 1024 * 1024, responses_max_decompressed_body_bytes: 64 * 1024 * 1024,
smart_scheduling: gateway_core::account::SmartSchedulingConfig::default(), smart_scheduling: gateway_core::account::SmartSchedulingConfig::default(),
quota_reset_window_weights: gateway_core::account::ResetWindowWeights::default(),
min_codex_desktop_version: None, min_codex_desktop_version: None,
min_codex_cli_version: None, min_codex_cli_version: None,
usage_retention_days: 31, usage_retention_days: 31,
@@ -113,7 +114,7 @@ async fn privacy_policy_defaults_off_and_round_trips_in_snapshot() {
} }
#[tokio::test] #[tokio::test]
async fn smart_settings_round_trip_publish_and_restore_database_defaults() { async fn scheduling_settings_round_trip_publish_and_restore_database_defaults() {
use gateway_core::account::SmartSchedulingConfig; use gateway_core::account::SmartSchedulingConfig;
use gateway_store::postgres::{PgRuntimeSnapshotRepository, RuntimeSnapshotRepository}; use gateway_store::postgres::{PgRuntimeSnapshotRepository, RuntimeSnapshotRepository};
let Some(database) = TestDatabase::create("smart_config").await else { let Some(database) = TestDatabase::create("smart_config").await else {
@@ -126,20 +127,29 @@ async fn smart_settings_round_trip_publish_and_restore_database_defaults() {
SmartSchedulingConfig::default() SmartSchedulingConfig::default()
); );
let mut update = settings_with_margin(3600); let mut update = settings_with_margin(3600);
update.values.smart_scheduling = let windows = gateway_core::account::ResetWindowWeights::new([0.0, 1.5, 0.0]).unwrap();
SmartSchedulingConfig::new([0.0, 2.0, 1.0, 0.5, 1.2, 2.3]).unwrap(); update.values.quota_reset_window_weights = windows;
let expected = update.values.smart_scheduling; let expected = serde_json::json!({
"loadWeight": 0.0, "quotaWeight": 2.0, "healthWeight": 1.0, "latencyWeight": 0.5, "resetWeight": 1.2, "queueWeight": 2.3,
"resetWindowWeights": {"fiveHour":2.0,"weekly":8.0,"monthly":0.0}
});
update.values.smart_scheduling = serde_json::from_value(expected.clone()).unwrap();
repository.update_runtime_settings(update).await.unwrap(); repository.update_runtime_settings(update).await.unwrap();
let reloaded = repository.load_runtime_settings().await.unwrap(); let reloaded = repository.load_runtime_settings().await.unwrap();
assert_eq!(reloaded.values.smart_scheduling, expected); assert_eq!(
serde_json::to_value(reloaded.values.smart_scheduling).unwrap(),
expected
);
assert_eq!(reloaded.values.quota_reset_window_weights, windows);
let snapshot = PgRuntimeSnapshotRepository::new(database.pool.clone()) let snapshot = PgRuntimeSnapshotRepository::new(database.pool.clone())
.load_runtime_snapshot() .load_runtime_snapshot()
.await .await
.unwrap(); .unwrap();
let snapshot_values = serde_json::to_value(&snapshot.settings).unwrap(); let snapshot_values = serde_json::to_value(&snapshot.settings).unwrap();
assert_eq!(snapshot_values["smart_scheduling"], expected);
assert_eq!( assert_eq!(
snapshot_values["smart_scheduling"], snapshot_values["quota_reset_window_weights"],
serde_json::to_value(expected).unwrap() serde_json::to_value(windows).unwrap()
); );
assert!(snapshot.config_revision > before.config_revision); assert!(snapshot.config_revision > before.config_revision);
sqlx::query("update runtime_settings set smart_scheduling_json = default where id = 1") sqlx::query("update runtime_settings set smart_scheduling_json = default where id = 1")
@@ -181,7 +191,7 @@ async fn warmup_settings_round_trip_with_database_constraint() {
let repository = PgRuntimeSettingsRepository::new(database.pool.clone()); let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
let mut update = settings_with_margin(3_600); let mut update = settings_with_margin(3_600);
update.values.account_warmup_enabled = true; update.values.account_warmup_enabled = true;
update.values.account_warmup_cron = "0 8,13 * * MON-FRI".to_owned(); update.values.account_warmup_cron = "0 8,13 * * MON-FRI;59 10 * * *".to_owned();
update.values.account_warmup_model = Some("test-model".to_owned()); update.values.account_warmup_model = Some("test-model".to_owned());
repository repository
.update_runtime_settings(update) .update_runtime_settings(update)
@@ -193,7 +203,10 @@ async fn warmup_settings_round_trip_with_database_constraint() {
.await .await
.expect("load warmup settings"); .expect("load warmup settings");
assert!(settings.values.account_warmup_enabled); assert!(settings.values.account_warmup_enabled);
assert_eq!(settings.values.account_warmup_cron, "0 8,13 * * MON-FRI"); assert_eq!(
settings.values.account_warmup_cron,
"0 8,13 * * MON-FRI;59 10 * * *"
);
assert_eq!( assert_eq!(
settings.values.account_warmup_model.as_deref(), settings.values.account_warmup_model.as_deref(),
Some("test-model") Some("test-model")
@@ -1010,6 +1023,13 @@ async fn request_profile_projection_is_revision_consistent_and_includes_key_over
}; };
let repository = PgRuntimeSettingsRepository::new(database.pool.clone()); let repository = PgRuntimeSettingsRepository::new(database.pool.clone());
let provider = ProviderKind::new("xai").unwrap(); let provider = ProviderKind::new("xai").unwrap();
assert!(
repository
.load_request_profile(&provider)
.await
.unwrap()
.is_none()
);
let document = |marker: &str| { let document = |marker: &str| {
OpaqueProviderData::new( OpaqueProviderData::new(
serde_json::json!({"marker":marker}) serde_json::json!({"marker":marker})
@@ -1051,6 +1071,10 @@ async fn request_profile_projection_is_revision_consistent_and_includes_key_over
assert_eq!(profiles.len(), 2); assert_eq!(profiles.len(), 2);
assert!(profiles.contains(&document("global"))); assert!(profiles.contains(&document("global")));
assert!(profiles.contains(&document("key"))); assert!(profiles.contains(&document("key")));
assert_eq!(
repository.load_request_profile(&provider).await.unwrap(),
Some(document("global"))
);
repository repository
.update_runtime_settings(settings_with_margin(7200)) .update_runtime_settings(settings_with_margin(7200))
@@ -19,6 +19,7 @@ crc32fast = "1.5.0"
futures = "0.3.32" futures = "0.3.32"
flate2 = "1.1.9" flate2 = "1.1.9"
gateway-admin = { path = "../../gateway-admin" } gateway-admin = { path = "../../gateway-admin" }
gateway-egress = { path = "../../gateway-egress" }
gateway-core = { path = "../../gateway-core" } gateway-core = { path = "../../gateway-core" }
gateway-protocol = { path = "../../gateway-protocol" } gateway-protocol = { path = "../../gateway-protocol" }
getrandom = "0.4.3" getrandom = "0.4.3"
@@ -1957,6 +1957,11 @@ fn map_quota_error(error: CodexCredentialQuotaError) -> ProviderAdminError {
use CodexCredentialQuotaError as Error; use CodexCredentialQuotaError as Error;
use ProviderAdminErrorKind as Kind; use ProviderAdminErrorKind as Kind;
let (kind, public_message, upstream_message) = match error { let (kind, public_message, upstream_message) = match error {
Error::Profile(source) => {
return provider_admin_error(Kind::Unavailable)
.with_public_message("OpenAI 账号预热客户端身份不可用")
.with_source(source);
}
Error::InvalidCredentialData => ( Error::InvalidCredentialData => (
Kind::Invalid, Kind::Invalid,
"OpenAI 额度查询凭据无效,请检查账号授权", "OpenAI 额度查询凭据无效,请检查账号授权",
@@ -2131,6 +2136,11 @@ fn map_catalog_error(error: CodexCredentialCatalogError) -> ProviderAdminError {
"OpenAI 模型查询失败,请检查出站连接与上游服务", "OpenAI 模型查询失败,请检查出站连接与上游服务",
), ),
Error::Cache => (Kind::Unavailable, "OpenAI 模型目录缓存暂不可用"), Error::Cache => (Kind::Unavailable, "OpenAI 模型目录缓存暂不可用"),
Error::Profile(source) => {
return provider_admin_error(Kind::Unavailable)
.with_public_message("OpenAI 模型目录客户端身份不可用")
.with_source(source);
}
}; };
provider_admin_error(kind).with_public_message(message) provider_admin_error(kind).with_public_message(message)
} }
@@ -72,15 +72,6 @@ fn normalize_codex_oauth_options(body: &mut Map<String, Value>) {
{ {
body.shift_remove("prompt_cache_options"); body.shift_remove("prompt_cache_options");
} }
// 原生上限优先,Chat 别名只补缺,非法值仍交给上游校验
for alias in ["max_completion_tokens", "max_tokens"] {
if let Some(limit) = body.shift_remove(alias)
&& !limit.is_null()
&& body.get("max_output_tokens").is_none_or(Value::is_null)
{
body.insert("max_output_tokens".to_owned(), limit);
}
}
} }
/// 补齐 Codex 请求缺省字段并适配已确认不兼容的请求形状,不递归清洗业务正文 /// 补齐 Codex 请求缺省字段并适配已确认不兼容的请求形状,不递归清洗业务正文
@@ -121,8 +112,15 @@ pub(crate) fn normalize_codex_request_body(body: &mut Map<String, Value>) {
} }
} }
// Codex 拒绝显式采样温度和缓存保留时长,独立的缓存键仍须保留 // Responses 上游统一忽略客户端输出上限,避免请求因上限低于模型下限被拒
for field in ["temperature", "prompt_cache_retention"] { // Codex 也不接受显式采样温度和缓存保留时长
body.remove(field); for field in [
"max_output_tokens",
"max_completion_tokens",
"max_tokens",
"temperature",
"prompt_cache_retention",
] {
body.shift_remove(field);
} }
} }
@@ -1,6 +1,6 @@
//! 逻辑会话账号绑定与线程传输隔离键的单向派生 //! 逻辑会话账号绑定与线程传输隔离键的单向派生
use gateway_core::operation::RawJsonPayload; use gateway_core::operation::{ProtocolPayload, RawJsonPayload};
use gateway_core::policy::ClientApiKeyId; use gateway_core::policy::ClientApiKeyId;
use gateway_core::provider_ports::{ProviderSessionAffinityKey, ProviderSessionAlias}; use gateway_core::provider_ports::{ProviderSessionAffinityKey, ProviderSessionAlias};
use serde_json::{Map, Value}; use serde_json::{Map, Value};
@@ -148,6 +148,38 @@ pub(crate) fn derive_codex_session_affinity(
.map(|affinity| affinity.with_follow_only(request.client_account_follow_only)) .map(|affinity| affinity.with_follow_only(request.client_account_follow_only))
} }
/// Messages 的会话只参与账号绑定,不解释为 OpenAI 轮次或原生续接身份
pub(crate) fn derive_messages_session_affinity(
payload: &ProtocolPayload,
headers: &[gateway_core::engine::middleware::MiddlewareHeader],
client_api_key_id: &ClientApiKeyId,
) -> Option<CodexSessionAffinity> {
if payload.protocol() != gateway_protocol::anthropic::ANTHROPIC_PROTOCOL {
return None;
}
let passthrough = crate::transport::request::decode_passthrough_headers(payload.context());
let header = headers
.iter()
.find(|header| {
header
.name()
.eq_ignore_ascii_case("x-claude-code-session-id")
})
.map(|header| std::str::from_utf8(header.value()).ok())
.unwrap_or_else(|| {
passthrough
.get("x-claude-code-session-id")
.and_then(|value| value.to_str().ok())
});
let session = gateway_protocol::anthropic::claude_session_id(payload.body(), header)?;
session_affinity(
"claude-session",
&session,
Some(&session),
client_api_key_id,
)
}
/// 仅供选号使用。缓存键与输入正文均不作为会话、轮次或传输身份。 /// 仅供选号使用。缓存键与输入正文均不作为会话、轮次或传输身份。
pub(crate) fn derive_cache_preference( pub(crate) fn derive_cache_preference(
request: &CodexResponsesRequest, request: &CodexResponsesRequest,
@@ -15,6 +15,7 @@ use gateway_core::account::{
}; };
use gateway_core::provider_ports::{ use gateway_core::provider_ports::{
ProviderCatalogCacheKey, ProviderCatalogCachePort, ProviderCatalogScope, ProviderCatalogCacheKey, ProviderCatalogCachePort, ProviderCatalogScope,
ProviderRuntimePolicyPort,
}; };
use gateway_core::routing::{ use gateway_core::routing::{
FrozenAccountScope, ModelPresentation, ProviderCatalogGeneration, ProviderKind, FrozenAccountScope, ModelPresentation, ProviderCatalogGeneration, ProviderKind,
@@ -32,8 +33,7 @@ use crate::transport::{CodexBackendClient, CodexCatalogModel, CodexRequestContex
const MAX_RESPONSE_ETAG_BYTES: usize = 256; const MAX_RESPONSE_ETAG_BYTES: usize = 256;
const ACCOUNT_CATALOG_CACHE_TTL: Duration = Duration::from_secs(5 * 60); const ACCOUNT_CATALOG_CACHE_TTL: Duration = Duration::from_secs(5 * 60);
// 调度证据覆盖默认 15 分钟刷新周期及抖动,长期未刷新时恢复上游验证 const CATALOG_REFRESH_AFTER: Duration = Duration::from_secs(30 * 60);
const SCHEDULING_CATALOG_MAX_AGE: Duration = Duration::from_secs(30 * 60);
const MAX_ACCOUNT_CATALOG_MODELS: usize = 2_048; const MAX_ACCOUNT_CATALOG_MODELS: usize = 2_048;
const MODEL_CATALOG_TIMEOUT: Duration = Duration::from_secs(15); const MODEL_CATALOG_TIMEOUT: Duration = Duration::from_secs(15);
@@ -141,7 +141,6 @@ impl CatalogContent {
struct AccountEntry { struct AccountEntry {
observed_at: SystemTime, observed_at: SystemTime,
content: Arc<CatalogContent>, content: Arc<CatalogContent>,
invalidated: Arc<std::sync::atomic::AtomicBool>,
etag: Option<String>, etag: Option<String>,
} }
@@ -197,13 +196,7 @@ impl CodexCredentialCatalogSnapshot {
pub(super) fn excludes(&self, account: &ProviderAccount, model: &str, now: SystemTime) -> bool { pub(super) fn excludes(&self, account: &ProviderAccount, model: &str, now: SystemTime) -> bool {
self.entries self.entries
.get(&CodexCatalogScope::for_account(account)) .get(&CodexCatalogScope::for_account(account))
.is_some_and(|e| { .is_some_and(|e| e.observed_at <= now && !e.content.ids.iter().any(|id| id == model))
!e.invalidated.load(std::sync::atomic::Ordering::Relaxed)
&& now
.duration_since(e.observed_at)
.is_ok_and(|age| age < SCHEDULING_CATALOG_MAX_AGE)
&& !e.content.ids.iter().any(|id| id == model)
})
} }
pub fn account_models(&self, account: &ProviderAccount) -> Option<&[String]> { pub fn account_models(&self, account: &ProviderAccount) -> Option<&[String]> {
@@ -251,6 +244,8 @@ pub enum CodexCredentialCatalogError {
InvalidEtag, InvalidEtag,
#[error("Codex model catalog changed during refresh")] #[error("Codex model catalog changed during refresh")]
ConcurrentUpdate, ConcurrentUpdate,
#[error("Codex model catalog client profile is unavailable")]
Profile(#[source] gateway_core::error::ErrorSource),
} }
#[derive(Default)] #[derive(Default)]
@@ -281,6 +276,7 @@ pub struct CodexCredentialCatalogService {
http: reqwest::Client, http: reqwest::Client,
base_url: String, base_url: String,
account_catalog_cache: Arc<dyn ProviderCatalogCachePort>, account_catalog_cache: Arc<dyn ProviderCatalogCachePort>,
runtime_policy: Arc<dyn ProviderRuntimePolicyPort>,
cache: Arc<RwLock<CatalogCacheState>>, cache: Arc<RwLock<CatalogCacheState>>,
refresh_notification: Arc<Notify>, refresh_notification: Arc<Notify>,
refresh_lock: Arc<tokio::sync::Mutex<()>>, refresh_lock: Arc<tokio::sync::Mutex<()>>,
@@ -294,6 +290,7 @@ impl CodexCredentialCatalogService {
http: reqwest::Client, http: reqwest::Client,
base_url: String, base_url: String,
account_catalog_cache: Arc<dyn ProviderCatalogCachePort>, account_catalog_cache: Arc<dyn ProviderCatalogCachePort>,
runtime_policy: Arc<dyn ProviderRuntimePolicyPort>,
) -> Self { ) -> Self {
Self { Self {
repository, repository,
@@ -301,6 +298,7 @@ impl CodexCredentialCatalogService {
http, http,
base_url, base_url,
account_catalog_cache, account_catalog_cache,
runtime_policy,
cache: Arc::new(RwLock::new(CatalogCacheState::default())), cache: Arc::new(RwLock::new(CatalogCacheState::default())),
refresh_notification: Arc::new(Notify::new()), refresh_notification: Arc::new(Notify::new()),
refresh_lock: Arc::new(tokio::sync::Mutex::new(())), refresh_lock: Arc::new(tokio::sync::Mutex::new(())),
@@ -308,6 +306,21 @@ impl CodexCredentialCatalogService {
} }
} }
/// 每轮发现冻结一次全局画像,使目录的版本参数与请求头使用同一身份
async fn discovery_client(&self) -> Result<CodexBackendClient, CodexCredentialCatalogError> {
let profile = self
.profile
.resolve_global_request_profile(self.runtime_policy.as_ref())
.await
.map_err(CodexCredentialCatalogError::Profile)?;
let client = CodexBackendClient::new(
self.http.clone(),
self.base_url.clone(),
self.profile.clone(),
);
Ok(client.with_request_profile(profile))
}
/// 原生条目取自后台发现的账号目录;用当前账号版本和冻结的 Key 政策双重过滤 /// 原生条目取自后台发现的账号目录;用当前账号版本和冻结的 Key 政策双重过滤
pub async fn client_model_catalog( pub async fn client_model_catalog(
&self, &self,
@@ -466,14 +479,7 @@ impl CodexCredentialCatalogService {
if ids.len() < MIN_CATALOG_PUSHDOWN { if ids.len() < MIN_CATALOG_PUSHDOWN {
return None; return None;
} }
let cache = self.cache.read().ok()?; let snapshot = self.cached().ok()??;
let snapshot = cache.snapshot.as_ref()?.clone();
let pending = cache
.pending
.keys()
.map(|s| s.account_id.clone())
.collect::<BTreeSet<_>>();
drop(cache);
let documents = snapshot let documents = snapshot
.sources .sources
.iter() .iter()
@@ -483,16 +489,10 @@ impl CodexCredentialCatalogService {
if documents.is_empty() { if documents.is_empty() {
return None; return None;
} }
let as_of = SystemTime::now();
Some(gateway_core::account::AccountCatalogOmissions { Some(gateway_core::account::AccountCatalogOmissions {
documents, documents,
exceptions: ids exceptions: ids.iter().filter(|id| exempt(id)).cloned().collect(),
.iter() as_of: SystemTime::now(),
.filter(|id| pending.contains(*id) || exempt(id))
.cloned()
.collect(),
observed_after: as_of.checked_sub(SCHEDULING_CATALOG_MAX_AGE)?,
as_of,
}) })
} }
@@ -521,11 +521,7 @@ impl CodexCredentialCatalogService {
.filter(|account| account.provider().as_str() == "openai") .filter(|account| account.provider().as_str() == "openai")
.ok_or(CodexCredentialCatalogError::NoEligibleCredential)?; .ok_or(CodexCredentialCatalogError::NoEligibleCredential)?;
let scope = CodexCatalogScope::for_account(&account); let scope = CodexCatalogScope::for_account(&account);
let client = CodexBackendClient::new( let client = self.discovery_client().await?;
self.http.clone(),
self.base_url.clone(),
self.profile.clone(),
);
let revision = self.state_revision()?; let revision = self.state_revision()?;
let (catalog, entry) = self let (catalog, entry) = self
.fetch_observation(&client, &account, &scope, None) .fetch_observation(&client, &account, &scope, None)
@@ -582,7 +578,6 @@ impl CodexCredentialCatalogService {
observed_at, observed_at,
content, content,
etag: observed.etag, etag: observed.etag,
invalidated: Default::default(),
}, },
)) ))
} }
@@ -595,11 +590,7 @@ impl CodexCredentialCatalogService {
&self, &self,
account: &ProviderAccount, account: &ProviderAccount,
) -> Result<(Vec<CodexCatalogModel>, SystemTime), CodexCredentialCatalogError> { ) -> Result<(Vec<CodexCatalogModel>, SystemTime), CodexCredentialCatalogError> {
let client = CodexBackendClient::new( let client = self.discovery_client().await?;
self.http.clone(),
self.base_url.clone(),
self.profile.clone(),
);
let fetched = self.fetch_account_models(&client, account).await?; let fetched = self.fetch_account_models(&client, account).await?;
if fetched.models.is_empty() { if fetched.models.is_empty() {
return Err(CodexCredentialCatalogError::NoEligibleCredential); return Err(CodexCredentialCatalogError::NoEligibleCredential);
@@ -41,7 +41,6 @@ impl CodexCredentialCatalogService {
observed_at: a.observed_at, observed_at: a.observed_at,
content: content.clone(), content: content.clone(),
etag: a.etag, etag: a.etag,
invalidated: Default::default(),
}, },
); );
} }
@@ -131,11 +130,6 @@ impl CodexCredentialCatalogService {
{ {
return Ok(false); return Ok(false);
} }
if let Some(entry) = state.snapshot.as_ref().and_then(|s| s.entries.get(&scope)) {
entry
.invalidated
.store(true, std::sync::atomic::Ordering::Relaxed);
}
state.pending.insert(scope, etag.to_owned()); state.pending.insert(scope, etag.to_owned());
drop(state); drop(state);
self.refresh_notification.notify_one(); self.refresh_notification.notify_one();
@@ -210,15 +204,12 @@ impl CodexCredentialCatalogService {
let Some(entry) = snapshot.entries.get(scope) else { let Some(entry) = snapshot.entries.get(scope) else {
return true; return true;
}; };
if entry.etag.as_deref() == Some(tag) // 发布其他账号时保留通知,只有该账号更新的观测才能确认 ETag
&& !entry.invalidated.load(std::sync::atomic::Ordering::Relaxed) entry.etag.as_deref() != Some(tag)
{ || old
return false; .as_ref()
} .and_then(|s| s.entries.get(scope))
entry .is_some_and(|previous| previous.observed_at >= entry.observed_at)
.invalidated
.store(true, std::sync::atomic::Ordering::Relaxed);
true
}); });
} }
cache.snapshot = next.clone(); cache.snapshot = next.clone();
@@ -289,7 +280,7 @@ impl CodexCredentialCatalogService {
pending.contains_key(scope) pending.contains_key(scope)
|| !entries.get(scope).is_some_and(|e| { || !entries.get(scope).is_some_and(|e| {
now.duration_since(e.observed_at) now.duration_since(e.observed_at)
.is_ok_and(|age| age < SCHEDULING_CATALOG_MAX_AGE) .is_ok_and(|age| age < CATALOG_REFRESH_AFTER)
}) })
}) })
.collect::<Vec<_>>(); .collect::<Vec<_>>();
@@ -298,11 +289,7 @@ impl CodexCredentialCatalogService {
.cached()? .cached()?
.ok_or(CodexCredentialCatalogError::NoEligibleCredential); .ok_or(CodexCredentialCatalogError::NoEligibleCredential);
} }
let client = CodexBackendClient::new( let client = self.discovery_client().await?;
self.http.clone(),
self.base_url.clone(),
self.profile.clone(),
);
let mut pace = tokio::time::interval(REFRESH_SPACING); let mut pace = tokio::time::interval(REFRESH_SPACING);
pace.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay); pace.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay);
let starts = stream::unfold(pace, |mut clock| async move { let starts = stream::unfold(pace, |mut clock| async move {
@@ -22,7 +22,8 @@ pub(crate) use affinity::{
CodexSessionAffinity, derive_cache_preference, derive_codex_cyber_policy_session_key, CodexSessionAffinity, derive_cache_preference, derive_codex_cyber_policy_session_key,
derive_codex_endpoint_session_affinity, derive_codex_session_affinity, derive_codex_endpoint_session_affinity, derive_codex_session_affinity,
derive_codex_transport_key, derive_endpoint_affinity_with_headers, derive_codex_transport_key, derive_endpoint_affinity_with_headers,
derive_live_session_affinity, derive_previous_response_id_hash, derive_turn_alias, derive_live_session_affinity, derive_messages_session_affinity,
derive_previous_response_id_hash, derive_turn_alias,
}; };
pub(crate) use eligibility::CodexSelectionModel; pub(crate) use eligibility::CodexSelectionModel;
pub(crate) use oauth::oauth_owner_ref; pub(crate) use oauth::oauth_owner_ref;
@@ -140,6 +140,8 @@ pub struct CodexWarmupSummary {
#[derive(Debug, Error)] #[derive(Debug, Error)]
pub enum CodexCredentialQuotaError { pub enum CodexCredentialQuotaError {
#[error("Codex warmup client profile is unavailable")]
Profile(#[source] gateway_core::error::ErrorSource),
#[error("Codex quota response is invalid")] #[error("Codex quota response is invalid")]
InvalidCredentialData, InvalidCredentialData,
#[error("Codex OAuth access token must be refreshed before querying quota")] #[error("Codex OAuth access token must be refreshed before querying quota")]
@@ -72,7 +72,7 @@ struct CodexQuotaRefreshAttempt {
wall_at: SystemTime, wall_at: SystemTime,
} }
#[derive(Debug, Clone, Copy)] #[derive(Debug, Clone)]
struct CodexQuotaSchedulingEntry { struct CodexQuotaSchedulingEntry {
version: u64, version: u64,
revision: CredentialRevision, revision: CredentialRevision,
@@ -211,7 +211,7 @@ impl CodexQuotaSchedulingProjection {
.filter(|entry| { .filter(|entry| {
entry.revision == account.revision() && Instant::now() < entry.expires_at entry.revision == account.revision() && Instant::now() < entry.expires_at
}) })
.and_then(|entry| entry.signals) .and_then(|entry| entry.signals.clone())
} }
pub(super) fn reserve_periodic_refreshes( pub(super) fn reserve_periodic_refreshes(
@@ -8,8 +8,8 @@ use std::time::{Duration, SystemTime};
use chrono::{DateTime, Utc}; use chrono::{DateTime, Utc};
use gateway_core::account::{ use gateway_core::account::{
AccountQuotaSignals, CredentialRevision, ProviderAccountId, QuotaEvidence, QuotaObservation, AccountQuotaResetWindow, AccountQuotaSignals, CredentialRevision, ProviderAccountId,
QuotaState, QuotaEvidence, QuotaObservation, QuotaResetPeriod, QuotaState,
}; };
use gateway_protocol::openai::events::{CreditsSnapshot, parse_credits_from_object}; use gateway_protocol::openai::events::{CreditsSnapshot, parse_credits_from_object};
use serde_json::{Map, Value}; use serde_json::{Map, Value};
@@ -371,6 +371,7 @@ pub(crate) fn quota_scheduling_signals(
let now = SystemTime::now(); let now = SystemTime::now();
let mut remaining_rank: Option<u64> = None; let mut remaining_rank: Option<u64> = None;
let mut reset_at: Option<SystemTime> = None; let mut reset_at: Option<SystemTime> = None;
let mut reset_windows = Vec::new();
for window in &snapshot.windows { for window in &snapshot.windows {
if quota_window_ttl(window, snapshot.observed_at(), now).is_none() { if quota_window_ttl(window, snapshot.observed_at(), now).is_none() {
continue; continue;
@@ -385,10 +386,22 @@ pub(crate) fn quota_scheduling_signals(
} }
if let Some(reset) = window_reset_at.filter(|reset| *reset > now) { if let Some(reset) = window_reset_at.filter(|reset| *reset > now) {
reset_at = Some(reset_at.map_or(reset, |current| current.min(reset))); reset_at = Some(reset_at.map_or(reset, |current| current.min(reset)));
if window.is_account_wide() {
let period = match window.kind() {
CodexQuotaWindowKind::ShortTerm => QuotaResetPeriod::FiveHour,
CodexQuotaWindowKind::Weekly => QuotaResetPeriod::Weekly,
CodexQuotaWindowKind::Monthly => QuotaResetPeriod::Monthly,
CodexQuotaWindowKind::Other => continue,
};
reset_windows.push(AccountQuotaResetWindow {
period,
reset_at: reset,
});
}
} }
} }
if remaining_rank.is_some() || reset_at.is_some() { if remaining_rank.is_some() || reset_at.is_some() {
Some(AccountQuotaSignals::new(reset_at, remaining_rank)) Some(AccountQuotaSignals::new(reset_at, remaining_rank).with_reset_windows(reset_windows))
} else { } else {
None None
} }
@@ -636,9 +649,7 @@ fn parse_rate_limit_windows(
.transpose()?; .transpose()?;
let limit_reached = optional_bool(window, "limit_reached")?.unwrap_or(false); let limit_reached = optional_bool(window, "limit_reached")?.unwrap_or(false);
let kind = quota_window_kind(window_seconds); let kind = quota_window_kind(window_seconds);
// primary/secondary 是协议槽位,不代表固定周期 // 槽位不代表周期,窗口键保留上游额度桶和实际时长;周期分类供展示及智能调度使用
// 窗口身份使用上游
// limit_id 和实际时长,5h/周/月分类只参与展示
let key = match window_seconds { let key = match window_seconds {
Some(seconds) => format!("{source}:{seconds}s"), Some(seconds) => format!("{source}:{seconds}s"),
None => format!("{source}:{name}"), None => format!("{source}:{name}"),
@@ -140,11 +140,17 @@ impl CodexCredentialQuotaService {
}) })
.collect::<BTreeMap<_, _>>(); .collect::<BTreeMap<_, _>>();
let profile = self
.profile
.resolve_global_request_profile(self.runtime_policy.as_ref())
.await
.map_err(CodexCredentialQuotaError::Profile)?;
let client = CodexBackendClient::new( let client = CodexBackendClient::new(
self.http.clone(), self.http.clone(),
self.base_url.clone(), self.base_url.clone(),
self.profile.clone(), self.profile.clone(),
); )
.with_request_profile(profile);
for account in accounts { for account in accounts {
if account.quota().is_exhausted() { if account.quota().is_exhausted() {
@@ -1054,6 +1054,7 @@ impl CodexCredentialSelector {
in_flight: 0, in_flight: 0,
last_started_at: None, last_started_at: None,
quota_reset_at: None, quota_reset_at: None,
quota_reset_windows: None,
quota_remaining_rank: None, quota_remaining_rank: None,
cooldown: None, cooldown: None,
failure_rate_basis_points: None, failure_rate_basis_points: None,
@@ -401,9 +401,8 @@ impl OpenAiTokenClient {
let Some(proxy) = proxy else { let Some(proxy) = proxy else {
return Ok(self.clone()); return Ok(self.clone());
}; };
let builder = Client::builder() let builder = gateway_egress::http_client(Some(proxy))
.no_proxy() .map_err(|_| TokenClientBuildError)?
.proxy(reqwest::Proxy::all(proxy.expose_url()).map_err(|_| TokenClientBuildError)?)
.redirect(Policy::none()) .redirect(Policy::none())
.connect_timeout(TOKEN_CONNECT_TIMEOUT) .connect_timeout(TOKEN_CONNECT_TIMEOUT)
.timeout(TOKEN_REQUEST_TIMEOUT); .timeout(TOKEN_REQUEST_TIMEOUT);
@@ -498,8 +497,8 @@ pub fn openai_token_client(
profile: CodexWireProfileState, profile: CodexWireProfileState,
) -> Result<OpenAiTokenClient, TokenClientBuildError> { ) -> Result<OpenAiTokenClient, TokenClientBuildError> {
ensure_rustls_provider(); ensure_rustls_provider();
let builder = Client::builder() let builder = gateway_egress::http_client(None)
.no_proxy() .map_err(|_| TokenClientBuildError)?
.redirect(Policy::none()) .redirect(Policy::none())
.connect_timeout(TOKEN_CONNECT_TIMEOUT) .connect_timeout(TOKEN_CONNECT_TIMEOUT)
.timeout(TOKEN_REQUEST_TIMEOUT); .timeout(TOKEN_REQUEST_TIMEOUT);
@@ -152,6 +152,7 @@ pub async fn initialize(
http.clone(), http.clone(),
config.base_url().to_owned(), config.base_url().to_owned(),
ports.catalog_cache(), ports.catalog_cache(),
ports.runtime_policy(),
)); ));
if let Err(error) = catalog.restore().await { if let Err(error) = catalog.restore().await {
tracing::warn!(%error,"OpenAI model catalog could not be restored"); tracing::warn!(%error,"OpenAI model catalog could not be restored");
@@ -4,6 +4,7 @@ use gateway_core::metering::{CalculatedCost, Usage};
use super::*; use super::*;
use crate::credential::CodexSelectionModel; use crate::credential::CodexSelectionModel;
use crate::transport::CodexTransportDecision;
use gateway_core::operation::RawJsonPayload; use gateway_core::operation::RawJsonPayload;
impl CodexProvider { impl CodexProvider {
@@ -925,6 +926,8 @@ pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
return; return;
} }
if let Some(capture) = session_capture.as_mut() { if let Some(capture) = session_capture.as_mut() {
capture.oversized_http = response.transport_metrics.decision
== Some(CodexTransportDecision::Http2WebSocketMessageTooLarge);
capture.continuation_scope = Some(if capture.response_store { capture.continuation_scope = Some(if capture.response_store {
OpenAiContinuationScope::Persisted OpenAiContinuationScope::Persisted
} else if response.transport == CodexBackendTransport::WebSocket } else if response.transport == CodexBackendTransport::WebSocket
@@ -918,6 +918,13 @@ pub(super) fn map_client_error(
websocket_error_kind(error), websocket_error_kind(error),
websocket_send_state(error), websocket_send_state(error),
)); ));
if matches!(
error.classified(),
CodexWebSocketExchangeError::MessageTooLarge
) {
// 可回退请求已在 transport 内转为 HTTP,剩余预热请求不能重复发送
failure.error = failure.error.with_retry_prohibited();
}
if let CodexWebSocketExchangeError::ConnectionLimitReached(upstream) = if let CodexWebSocketExchangeError::ConnectionLimitReached(upstream) =
error.classified() error.classified()
{ {
@@ -1180,6 +1187,7 @@ fn websocket_diagnostic(error: &CodexWebSocketExchangeError) -> ProviderDiagnost
CodexWebSocketExchangeError::InvalidRequest(_) => ("prepare", "invalid_request"), CodexWebSocketExchangeError::InvalidRequest(_) => ("prepare", "invalid_request"),
CodexWebSocketExchangeError::SendTimeout { .. } => ("send", "send_timeout"), CodexWebSocketExchangeError::SendTimeout { .. } => ("send", "send_timeout"),
CodexWebSocketExchangeError::SendNotStarted => ("send", "send_not_started"), CodexWebSocketExchangeError::SendNotStarted => ("send", "send_not_started"),
CodexWebSocketExchangeError::MessageTooLarge => ("prepare", "websocket_message_too_large"),
CodexWebSocketExchangeError::ReceiveIdleTimeout { .. } => { CodexWebSocketExchangeError::ReceiveIdleTimeout { .. } => {
("receive", "receive_idle_timeout") ("receive", "receive_idle_timeout")
} }
@@ -1276,6 +1284,10 @@ fn websocket_diagnostic_message(error: &CodexWebSocketExchangeError) -> Provider
CodexWebSocketExchangeError::SendNotStarted => { CodexWebSocketExchangeError::SendNotStarted => {
"OpenAI WebSocket connection closed before payload send started".to_owned() "OpenAI WebSocket connection closed before payload send started".to_owned()
} }
CodexWebSocketExchangeError::MessageTooLarge => {
"OpenAI WebSocket request exceeds the advertised message size limit before send"
.to_owned()
}
CodexWebSocketExchangeError::SendTimeout { timeout } => { CodexWebSocketExchangeError::SendTimeout { timeout } => {
format!("OpenAI WebSocket payload send timed out after {timeout:?}") format!("OpenAI WebSocket payload send timed out after {timeout:?}")
} }
@@ -1345,6 +1357,16 @@ fn websocket_raw_error(error: &CodexWebSocketExchangeError) -> Option<RawUpstrea
pub(super) fn websocket_client_visible_error( pub(super) fn websocket_client_visible_error(
error: &CodexWebSocketExchangeError, error: &CodexWebSocketExchangeError,
) -> Option<ClientVisibleUpstreamError> { ) -> Option<ClientVisibleUpstreamError> {
if matches!(
error.classified(),
CodexWebSocketExchangeError::MessageTooLarge
) {
return Some(ClientVisibleUpstreamError::new(
"The WebSocket request exceeds the server's message size limit.".to_owned(),
Some("websocket_message_too_large".to_owned()),
Some("invalid_request_error".to_owned()),
));
}
if let CodexWebSocketExchangeError::ConnectionLimitReached(failure) = error.classified() { if let CodexWebSocketExchangeError::ConnectionLimitReached(failure) = error.classified() {
return Some(ClientVisibleUpstreamError::new( return Some(ClientVisibleUpstreamError::new(
failure.message.clone(), failure.message.clone(),
@@ -1648,6 +1670,7 @@ pub(super) fn websocket_send_state(error: &CodexWebSocketExchangeError) -> Upstr
| CodexWebSocketExchangeError::OriginHalfOpenBusy | CodexWebSocketExchangeError::OriginHalfOpenBusy
| CodexWebSocketExchangeError::SharedConnectFailed | CodexWebSocketExchangeError::SharedConnectFailed
| CodexWebSocketExchangeError::SendNotStarted | CodexWebSocketExchangeError::SendNotStarted
| CodexWebSocketExchangeError::MessageTooLarge
| CodexWebSocketExchangeError::ContinuationUnavailable { .. } => UpstreamSendState::NotSent, | CodexWebSocketExchangeError::ContinuationUnavailable { .. } => UpstreamSendState::NotSent,
CodexWebSocketExchangeError::Upstream(_) CodexWebSocketExchangeError::Upstream(_)
| CodexWebSocketExchangeError::ConnectionLimitReached(_) | CodexWebSocketExchangeError::ConnectionLimitReached(_)
@@ -1670,6 +1693,7 @@ pub(super) fn websocket_send_state(error: &CodexWebSocketExchangeError) -> Upstr
pub(super) fn websocket_error_kind(error: &CodexWebSocketExchangeError) -> ProviderErrorKind { pub(super) fn websocket_error_kind(error: &CodexWebSocketExchangeError) -> ProviderErrorKind {
match error.classified() { match error.classified() {
CodexWebSocketExchangeError::MessageTooLarge => ProviderErrorKind::MessageTooBig,
CodexWebSocketExchangeError::InvalidRequest(_) CodexWebSocketExchangeError::InvalidRequest(_)
| CodexWebSocketExchangeError::InvalidSse(_) | CodexWebSocketExchangeError::InvalidSse(_)
| CodexWebSocketExchangeError::UnexpectedBinaryEvent => ProviderErrorKind::Protocol, | CodexWebSocketExchangeError::UnexpectedBinaryEvent => ProviderErrorKind::Protocol,
@@ -30,6 +30,26 @@ impl CodexProvider {
if !matches!(source, OPENAI_PROTOCOL | ANTHROPIC_PROTOCOL) { if !matches!(source, OPENAI_PROTOCOL | ANTHROPIC_PROTOCOL) {
return Err(invalid_request()); return Err(invalid_request());
} }
if !context.is_diagnostic_required_account()
&& (source == ANTHROPIC_PROTOCOL
|| selected.lease.authentication().api_protocol() == ApiKeyProtocol::Messages)
{
// attempt 改写后的身份必须在跨协议转换前复验,避免绕过已有账号绑定
let affinity = crate::credential::derive_messages_session_affinity(
generate.protocol_payload(),
&headers,
context.client_api_key_ref(),
);
self.selector
.validate_translated_selection(
&mut selected.lease,
affinity.as_ref(),
None,
context.account_selection_policy(),
)
.await
.map_err(map_selection_error)?;
}
if selected.lease.authentication().api_protocol() == ApiKeyProtocol::Messages { if selected.lease.authentication().api_protocol() == ApiKeyProtocol::Messages {
if context.continuation_attempt() == ContinuationAttempt::Native if context.continuation_attempt() == ContinuationAttempt::Native
|| generate.provider_session_state(PROVIDER_NAME).is_some() || generate.provider_session_state(PROVIDER_NAME).is_some()
@@ -124,21 +144,10 @@ impl CodexProvider {
selected: SelectedGenerate, selected: SelectedGenerate,
) -> Result<ProviderStream, ProviderError> { ) -> Result<ProviderStream, ProviderError> {
let SelectedGenerate { let SelectedGenerate {
mut lease, lease,
account_selection_wait_ms, account_selection_wait_ms,
.. ..
} = selected; } = selected;
if !context.is_diagnostic_required_account() {
self.selector
.validate_translated_selection(
&mut lease,
None,
None,
context.account_selection_policy(),
)
.await
.map_err(map_selection_error)?;
}
self.messages_stream( self.messages_stream(
body, body,
source_context, source_context,
@@ -67,7 +67,7 @@ use crate::credential::{
derive_live_session_affinity, derive_previous_response_id_hash, derive_turn_alias, derive_live_session_affinity, derive_previous_response_id_hash, derive_turn_alias,
}; };
use crate::request_identity::{account_session_with_headers, turn_id_with_headers}; use crate::request_identity::{account_session_with_headers, turn_id_with_headers};
use crate::session_transport::{CodexSessionTransportRecovery, WS_PAYLOAD_LIMIT}; use crate::session_transport::CodexSessionTransportRecovery;
use crate::transport::canonical::{ use crate::transport::canonical::{
CodexCanonicalDecoder, CodexCanonicalError, CodexCanonicalOutcome, CodexCanonicalDecoder, CodexCanonicalError, CodexCanonicalOutcome,
}; };
@@ -84,7 +84,6 @@ use crate::transport::protocol::responses::{
CodexResponsesRequest, PREVIOUS_RESPONSE_NOT_FOUND_CODE, PREVIOUS_RESPONSE_NOT_FOUND_MESSAGE, CodexResponsesRequest, PREVIOUS_RESPONSE_NOT_FOUND_CODE, PREVIOUS_RESPONSE_NOT_FOUND_MESSAGE,
PreviousResponseScope, ResponseEventSignals, TransportRequirement, transport_requirement, PreviousResponseScope, ResponseEventSignals, TransportRequirement, transport_requirement,
}; };
use crate::transport::protocol::websocket::websocket_response_create_payload_len;
use crate::transport::request::{ use crate::transport::request::{
CodexRequestEncodeError, RequestAccountScope, align_structured_location_fields, CodexRequestEncodeError, RequestAccountScope, align_structured_location_fields,
clear_request_turn_state, encode_generate_request, scope_request_to_account, clear_request_turn_state, encode_generate_request, scope_request_to_account,
@@ -674,8 +673,7 @@ impl Provider for CodexProvider {
{ {
return Err(continuation_replay_required_error("scope_unavailable")); return Err(continuation_replay_required_error("scope_unavailable"));
} }
// 其他协议必须先取得真实账号,再按固定 attempt 阶段调用转换器;选号前不能 // 跨协议转换必须在选号后执行;选号前只按各自协议读取明确会话身份
// 把未知正文当成 OpenAI wire 解释会话、亲和或传输字段
let upstream = (generate.protocol_payload().protocol() == OPENAI_PROTOCOL) let upstream = (generate.protocol_payload().protocol() == OPENAI_PROTOCOL)
.then(|| encode_generate_request(generate, upstream_model.as_str(), None)) .then(|| encode_generate_request(generate, upstream_model.as_str(), None))
.transpose() .transpose()
@@ -727,10 +725,16 @@ impl Provider for CodexProvider {
requires_websocket, requires_websocket,
) )
}); });
// 内容亲和只交给选号器;中间件复验、轮次关联和原生续接仍只认真实会话 let messages_affinity = crate::credential::derive_messages_session_affinity(
generate.protocol_payload(),
&[],
context.client_api_key_ref(),
);
// Messages 身份与内容亲和仅交给选号器,不建立 OpenAI 轮次或原生续接关联
let account_affinity = selection_session_affinity let account_affinity = selection_session_affinity
.as_ref() .as_ref()
.or(cache_preference.as_ref()); .or(cache_preference.as_ref())
.or(messages_affinity.as_ref());
let selection_started_at = Instant::now(); let selection_started_at = Instant::now();
let selection = async { let selection = async {
self.selector self.selector
@@ -1069,54 +1073,13 @@ impl CodexProvider {
&& session_transport_key && session_transport_key
.as_ref() .as_ref()
.is_some_and(|key| self.session_transport_recovery.uses_http(key)); .is_some_and(|key| self.session_transport_recovery.uses_http(key));
let mut transport = if requirement.requires_websocket() { let transport = if requirement.requires_websocket() {
CodexProviderTransport::PreferWebSocket CodexProviderTransport::PreferWebSocket
} else if context.transport() == AttemptTransport::Fallback || session_http_fallback { } else if context.transport() == AttemptTransport::Fallback || session_http_fallback {
CodexProviderTransport::HttpOnly CodexProviderTransport::HttpOnly
} else { } else {
requested_transport requested_transport
}; };
let mut oversized_http = false;
if codex_oauth
&& transport == CodexProviderTransport::PreferWebSocket
&& (matches!(
requirement,
TransportRequirement::NewChain | TransportRequirement::WebSocketNewChain
) || (requirement == TransportRequirement::ExactWebSocketContinuation
&& upstream_request
.downstream_websocket_connection_id
.is_some()))
{
let payload_bytes =
websocket_response_create_payload_len(&upstream_request).map_err(|_| {
provider_error(ProviderErrorKind::Protocol, UpstreamSendState::NotSent)
})?;
if requirement == TransportRequirement::ExactWebSocketContinuation {
if payload_bytes >= WS_PAYLOAD_LIMIT {
tracing::info!(
request_id = context.request_id().as_str(),
payload_bytes,
threshold_bytes = WS_PAYLOAD_LIMIT,
continuation_recovery_action = "client_replay_required",
"OpenAI oversized WebSocket continuation requires full client input before send"
);
return Err(continuation_replay_required_error(
"transport_payload_too_large",
));
}
} else if self
.session_transport_recovery
.http_for_full_payload(session_transport_key.as_ref(), payload_bytes)
{
transport = CodexProviderTransport::HttpOnly;
oversized_http = true;
tracing::info!(
request_id = context.request_id().as_str(),
payload_bytes,
"OpenAI full request selected HTTP by WebSocket payload size"
);
}
}
if transport == CodexProviderTransport::PreferWebSocket if transport == CodexProviderTransport::PreferWebSocket
&& middleware_headers.iter().any(|header| { && middleware_headers.iter().any(|header| {
match HeaderValue::from_bytes(header.value()) { match HeaderValue::from_bytes(header.value()) {
@@ -1173,7 +1136,7 @@ impl CodexProvider {
) )
.reasoning_effort_override, .reasoning_effort_override,
response_store, response_store,
oversized_http, oversized_http: false,
continuation_scope: None, continuation_scope: None,
}); });
let allows_account_state_mutation = lease.allows_account_state_mutation(); let allows_account_state_mutation = lease.allows_account_state_mutation();

Some files were not shown because too many files have changed in this diff Show More