Compare commits

...
12 Commits
Author SHA1 Message Date
ZyyandCodex bd69329a37 chore: bump version to 3.45.2
Co-authored-by: Codex <noreply@openai.com>
2026-11-11 17:04:39 +08:00
ZyyandCodex 151a3c6bfd fix(openai): negotiate Codex catalogs for Responses upstreams
Co-authored-by: Codex <noreply@openai.com>
2026-11-11 16:49:10 +08:00
ZyyandCodex e2f194875f refactor: clarify protocol ownership and request observations
Separate protocol identifiers from Provider identities and centralize shared
request facts and observations in gateway-protocol. Keep endpoint models
and private continuation state owned by Providers.

Reorganize the architecture documentation and clarify the shutdown guarantees
for buffered charges and observations.

Validated with workspace formatting, strict Clippy, 3013 affected-module
tests and 16 architecture checks. One reference-binary TLS test is ignored.

Co-authored-by: Codex <noreply@openai.com>
2026-11-11 16:04:16 +08:00
ZyyandCodex 07b61bdc83 fix(ui): keep help buttons aligned with form labels and clarify hints
Co-authored-by: Codex <noreply@openai.com>
2026-11-11 15:13:00 +08:00
c0fdc1bbff fix(openai): record explicit Anthropic reasoning effort (#680)
Observe top-level output_config.effort before the OpenAI encoder rejects
Messages payloads. Reuse the protocol parser without inferring an effort
from thinking budgets or changing upstream conversion behavior.

Cover absent and explicit effort values, native Messages forwarding, and
Messages-to-Responses conversion. Document the observation contract.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-11 15:12:02 +08:00
343c579017 fix(openai): rotate new round-robin sessions without affinity hits (#678)
* fix(openai): rotate new round-robin sessions without affinity hits

Strict affinity keeps a session on one account, but the shared request
cursor advanced on every scheduling load, including affinity hits. A new
session then took whatever slot that request volume happened to leave.

Advance a separate session cursor only when round-robin assigns an
account, and keep the request cursor for other strategies.

* fix(openai): advance round robin when an account assignment succeeds

Commit the session binding and round-robin cursor together after acquiring the account lease. Release a conflicting lease before retrying so concurrent first requests share a binding without consuming extra turns. Count delegated selections and preserve explicit plugin picks.

Verified with OpenAI provider tests, store tests against Redis and PostgreSQL, architecture checks, and workspace Clippy.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: koeria <riennie@qq.com>
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-11 14:40:59 +08:00
koeriaandCursor Agent 914c00785c fix(openai): skip same-account retry on NO_MORE_RETRY (#675)
Treat x-retry-metadata: NO_MORE_RETRY as a hard signal to skip
same-account backoff and transport recovery, even when Retry-After
is within maxRetryAfterSeconds. Capacity failures still rotate under
the existing policy; server_error still returns the upstream error.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-11-11 14:08:24 +08:00
ZyyandCodex b3de0d637d fix(peer): send routing headers only to discovered nodes
Reuse connection-scoped peer discovery for native HTTP and WebSocket forwarding. Strip the route header for ordinary upstreams and leave explicit plugin forwarding changes to plugins.

Co-authored-by: Codex <noreply@openai.com>
2026-11-11 13:14:25 +08:00
ZyyandCodex 1271fa0186 test(openai): verify platform TLS against official Codex
Compare native HTTP, custom CA HTTP, and WebSocket ClientHello captures with an official Codex executable on the same platform. Document the explicit reference test workflow.

Co-authored-by: Codex <noreply@openai.com>
2026-11-11 12:41:04 +08:00
ZyyandCodex 055da40476 feat(keys): enable async compaction in Codex configs
Co-authored-by: Codex <noreply@openai.com>
2026-11-11 12:37:12 +08:00
ZyyandCodex 36ba79ca5e fix(openai): preserve native Responses service tiers
Limit default tier omission to Messages-to-OAuth conversion so native Responses retain their requested tier and routing hint.

Co-authored-by: Codex <noreply@openai.com>
2026-11-11 11:33:01 +08:00
github-actions[bot] 1b24029a97 chore(release): update release channels to 3.45.1 2026-11-11 00:03:25 +00:00
112 changed files with 2935 additions and 1079 deletions
@@ -52,7 +52,7 @@ impl PeerInfoService {
});
let limits = peer.and_then(|peer| peer.key_limits);
let info = ProviderUpstreamInfo {
discovered: peer.is_some() || stored_failure == Some(UpstreamInfoFailure::Unauthorized),
discovered: snapshot::is_discovered(peer, stored_failure),
observed_at,
stale,
connection_revision: connection.revision.clone(),
@@ -282,6 +282,21 @@ impl PeerInfoService {
}
impl UpstreamInfoSource for PeerInfoService {
fn is_peer(&self, account: &ProviderAccount) -> bool {
if !self.supported(account) {
return false;
}
let connection = UpstreamConnection::for_account(account);
self.observations
.read()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.get(account.id())
.is_some_and(|cached| {
cached.connection == connection
&& snapshot::is_discovered(cached.summary.as_deref(), cached.stored_failure)
})
}
fn signal(
&self,
account: &ProviderAccount,
@@ -16,6 +16,10 @@ use serde::Deserialize as _;
pub(super) const MAX_AGE: Duration = Duration::from_secs(30 * 60);
pub(super) const FRESH_FOR: Duration = Duration::from_secs(5 * 60);
pub(super) fn is_discovered(peer: Option<&PeerLink>, failure: Option<UpstreamInfoFailure>) -> bool {
peer.is_some() || failure == Some(UpstreamInfoFailure::Unauthorized)
}
// 不支持协议是连接的识别结果,复用观测存储与版本保护,不伪造上游额度快照
pub(super) fn unsupported(connection: &UpstreamConnection) -> OpaqueProviderData {
OpaqueProviderData::new(serde_json::Map::from_iter([
@@ -140,6 +140,10 @@ impl PendingPeer {
}
impl UpstreamInfoSource for PendingPeer {
fn is_peer(&self, _: &ProviderAccount) -> bool {
false
}
fn signal(
&self,
_: &ProviderAccount,
@@ -239,6 +239,7 @@ async fn discovery_uses_provider_capabilities_and_publishes_neutral_signals() {
initialize_peer_upstream(accounts.clone(), &registry, transport.clone()).unwrap();
let initial_accounts = accounts.accounts.lock().unwrap().clone();
let reader = UpstreamInfoReader::default();
assert!(!reader.is_peer(&initial_accounts[0]));
assert!(
reader
.signal(&initial_accounts[0], Some("model"), SystemTime::now())
@@ -246,6 +247,7 @@ async fn discovery_uses_provider_capabilities_and_publishes_neutral_signals() {
);
let source = bundle.scheduling_source();
reader.bind(&source).unwrap();
assert!(!reader.is_peer(&initial_accounts[0]));
assert!(reader.bind(&source).is_err());
drop(source);
let WorkerContribution::Registration(worker) = bundle.take_worker_contributions().remove(0)
@@ -272,6 +274,10 @@ async fn discovery_uses_provider_capabilities_and_publishes_neutral_signals() {
}
);
for account in &initial_accounts[..2] {
assert_eq!(
reader.is_peer(account),
Some(account.provider().as_str()) != failed_provider
);
if Some(account.provider().as_str()) == failed_provider {
assert!(
reader
@@ -304,6 +310,7 @@ async fn discovery_uses_provider_capabilities_and_publishes_neutral_signals() {
);
drop(worker);
drop(bundle);
assert!(!reader.is_peer(&initial_accounts[0]));
assert!(
reader
.signal(&initial_accounts[0], Some("model"), SystemTime::now())
@@ -340,6 +347,7 @@ async fn account_edits_preserve_observation_and_refresh_budget_until_connection_
.is_some()
);
let after = service.refresh_quota(edited.id()).await.unwrap();
assert!(bundle.scheduling_source().is_peer(&edited));
assert_eq!(after.upstream, before.upstream);
assert_eq!(transport.0.lock().unwrap().len(), 1);
@@ -347,6 +355,7 @@ async fn account_edits_preserve_observation_and_refresh_budget_until_connection_
OutboundProxy::parse("http://proxy.invalid:8080").unwrap(),
));
*accounts.accounts.lock().unwrap() = vec![reconnected.clone()];
assert!(!bundle.scheduling_source().is_peer(&reconnected));
service.account_changed(reconnected.id()).await.unwrap();
assert!(
bundle
@@ -215,6 +215,10 @@ async fn known_peer_identity_changes_only_on_definitive_detection() {
quota.upstream
);
let account = accounts.accounts.lock().unwrap()[0].clone();
assert_eq!(
bundle.scheduling_source().is_peer(&account),
!clears_identity
);
assert_eq!(
bundle
.scheduling_source()
@@ -237,6 +241,7 @@ async fn known_peer_identity_changes_only_on_definitive_detection() {
drop(bundle);
let restarted = initialize(accounts, transport);
let restored = restarted.service().read_quota(&id).await.unwrap().unwrap();
assert!(!restarted.scheduling_source().is_peer(&account));
assert_eq!(
restored.upstream.as_ref().unwrap().discovered,
!clears_identity
@@ -258,6 +263,10 @@ async fn known_peer_identity_changes_only_on_definitive_detection() {
.upstream,
restored.upstream
);
assert_eq!(
restarted.scheduling_source().is_peer(&account),
!clears_identity
);
}
}
@@ -22,6 +22,7 @@ use gateway_core::{
error::GatewayError,
operation::Operation,
};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use crate::client_error::{ProtocolError, ProtocolErrorBody};
use crate::middleware::headers::encode_headers;
@@ -65,7 +66,7 @@ where
Box::new(move |prepared, request| {
Box::pin(async move {
let (protocol, headers, body) = request_parts(request.clone())?;
if protocol != "openai" {
if protocol != OPENAI_PROTOCOL {
return Err(MiddlewareError::Rejected);
}
let operation = request.apply_capabilities(decode(body, headers).await?)?;
@@ -80,7 +81,8 @@ where
{
Ok(started) => started,
Err(error) => {
return buffered_response("openai", gateway_error_response(&error)).await;
return buffered_response(OPENAI_PROTOCOL, gateway_error_response(&error))
.await;
}
};
collect_raw_json_response(started).await
@@ -89,7 +91,9 @@ where
)
.await;
let response = match result {
Ok(response) => into_http_response(response, ExpectedBody::SingleJson, "openai").await,
Ok(response) => {
into_http_response(response, ExpectedBody::SingleJson, OPENAI_PROTOCOL).await
}
Err(error) => error_response(error),
};
if !execution_entered.load(Ordering::Relaxed) {
@@ -143,7 +147,7 @@ async fn collect_raw_json_response(
.await
.map_err(|_| MiddlewareError::Fault)?;
Ok(pending_execution_response(
"openai".to_owned(),
OPENAI_PROTOCOL.to_owned(),
parts.status.as_u16(),
encode_headers(&parts.headers),
MiddlewareFrame::new(bytes, framing, true).with_transformed(transformed),
@@ -155,7 +159,7 @@ fn raw_json_body(events: Vec<ProviderEvent>) -> Option<Bytes> {
let mut body = None;
for event in events {
let (_, wire) = event.into_parts();
let Some(wire) = wire.filter(|wire| wire.protocol() == "openai") else {
let Some(wire) = wire.filter(|wire| wire.protocol() == OPENAI_PROTOCOL) else {
continue;
};
let Some(raw) = wire.into_raw_json_body() else {
@@ -32,6 +32,7 @@ use gateway_core::engine::{
use gateway_core::error::{GatewayError, GatewayErrorKind};
use gateway_core::event::{ProviderEvent, ProviderResponseHeader};
use gateway_core::lifecycle::{CancellationToken, ConnectionGuard};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use gateway_protocol::openai::sse::response_failed_sse_event_with_id;
use serde_json::Value;
use tokio::time::Instant;
@@ -491,7 +492,9 @@ pub async fn collect_execution_response(session: Box<dyn ExecutionSession>) -> R
)
.await
{
Ok(response) => into_http_response(response, ExpectedBody::SingleJson, "openai").await,
Ok(response) => {
into_http_response(response, ExpectedBody::SingleJson, OPENAI_PROTOCOL).await
}
Err(error) => error_response(error),
}
}
@@ -513,7 +516,9 @@ pub async fn stream_execution_response(
)
.await
{
Ok(response) => into_http_response(response, ExpectedBody::Sse, "openai").await,
Ok(response) => {
into_http_response(response, ExpectedBody::Sse, OPENAI_PROTOCOL).await
}
Err(error) => error_response(error),
}
},
@@ -15,6 +15,8 @@ use gateway_core::{
error::{GatewayError, GatewayErrorKind},
operation::OperationKind,
};
use gateway_protocol::anthropic::ANTHROPIC_PROTOCOL;
use gateway_protocol::openai::OPENAI_PROTOCOL;
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub(crate) enum GenerationProtocol {
@@ -27,7 +29,7 @@ impl GenerationProtocol {
pub(crate) fn from_request(request: &DecodedGenerationRequest) -> Self {
if request.operation().kind() == OperationKind::CountTokens {
Self::MessageTokenCount
} else if request.operation().protocol() == "anthropic" {
} else if request.operation().protocol() == ANTHROPIC_PROTOCOL {
Self::Messages
} else {
Self::Responses
@@ -36,8 +38,8 @@ impl GenerationProtocol {
pub(crate) const fn wire(self) -> &'static str {
match self {
Self::Responses => "openai",
Self::Messages | Self::MessageTokenCount => "anthropic",
Self::Responses => OPENAI_PROTOCOL,
Self::Messages | Self::MessageTokenCount => ANTHROPIC_PROTOCOL,
}
}
@@ -2,6 +2,7 @@
use axum::body::Bytes;
use gateway_core::{error::GatewayError, event::ProviderEvent};
use gateway_protocol::anthropic::ANTHROPIC_PROTOCOL;
use gateway_protocol::{
anthropic::MessagesAccumulator,
openai::sse::{
@@ -50,7 +51,7 @@ impl GenerationEncoder {
} => {
let Some(wire) = event
.wire_event()
.filter(|wire| wire.protocol() == "anthropic")
.filter(|wire| wire.protocol() == ANTHROPIC_PROTOCOL)
else {
return Vec::new();
};
@@ -1,6 +1,7 @@
//! 出口校验按协议维护边界事实,不在流式路径保存生成正文
use gateway_core::event::ProviderEvent;
use gateway_protocol::anthropic::ANTHROPIC_PROTOCOL;
use gateway_protocol::openai::sse::parse_sse_events;
use serde_json::Value;
use std::sync::{Arc, Mutex};
@@ -32,7 +33,7 @@ impl GenerationValidationFacts {
Self::Messages(id) => {
if let Some(value) = event
.wire_event()
.filter(|wire| wire.protocol() == "anthropic")
.filter(|wire| wire.protocol() == ANTHROPIC_PROTOCOL)
.map(|wire| wire.data())
&& let Some(value) = value
.pointer("/message/id")
@@ -13,6 +13,7 @@ use gateway_core::error::{GatewayError, GatewayErrorKind};
use gateway_core::operation::{
ImageRequest, ImageRequestKind, Operation, OperationKind, RawJsonPayload,
};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use serde_json::Value;
use crate::ApiState;
@@ -25,7 +26,6 @@ use crate::openai::{
router::{IMAGE_EDITS_PATH, IMAGE_GENERATIONS_PATH},
};
const OPENAI_PROTOCOL: &str = "openai";
const IMAGE_TURN_ID_CONTEXT_KEY: &str = "image_turn_id";
/// `POST /v1/images/generations`
@@ -13,6 +13,7 @@ use gateway_core::engine::execution::StartedExecution;
use gateway_core::event::ProviderEvent;
use gateway_core::live::LiveHangupRequest;
use gateway_core::operation::Operation;
use gateway_protocol::openai::OPENAI_PROTOCOL;
use serde_json::Value;
use super::{
@@ -31,9 +32,6 @@ use crate::{
},
};
/// 引导 wire 事件使用的协议名;与 Provider 侧 `PROVIDER_NAME` 一致。
const LIVE_PROTOCOL: &str = "openai";
pub(crate) async fn live_call(
State(state): State<ApiState>,
connect_info: Option<Extension<ConnectInfo<SocketAddr>>>,
@@ -344,7 +342,7 @@ fn live_call_body(events: Vec<ProviderEvent>) -> Option<Bytes> {
let mut raw_events = 0usize;
for event in events {
let (_, wire) = event.into_parts();
let Some(wire) = wire.filter(|wire| wire.protocol() == LIVE_PROTOCOL) else {
let Some(wire) = wire.filter(|wire| wire.protocol() == OPENAI_PROTOCOL) else {
continue;
};
let raw = wire.into_raw_http_body()?;
@@ -20,6 +20,7 @@ use gateway_core::event::ProviderResponseHeader;
use gateway_core::operation::{
ProviderHttpHeader, ProviderHttpMethod, ProviderHttpRequest, RawHttpPayload,
};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use serde_json::{Value, json};
use crate::openai::error::openai_error_response;
@@ -172,7 +173,7 @@ pub(crate) fn live_call_operation(
"invalid_request",
))
};
let payload = RawHttpPayload::new("openai", body).map_err(|_| invalid())?;
let payload = RawHttpPayload::new(OPENAI_PROTOCOL, body).map_err(|_| invalid())?;
ProviderHttpRequest::new(
LIVE_CALLS_ENDPOINT,
ProviderHttpMethod::Post,
@@ -20,6 +20,7 @@ use gateway_core::{
lifecycle::CancellationToken,
operation::OperationKind,
};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use std::{future::Future, sync::Arc};
use super::http::{ExpectedBody, buffered_response, error_response, into_http_response};
@@ -38,7 +39,7 @@ impl RequestInput {
pub(crate) fn query(endpoint: String, model_hint: Option<String>, headers: &HeaderMap) -> Self {
Self {
endpoint,
protocol: "openai".to_owned(),
protocol: OPENAI_PROTOCOL.to_owned(),
operation: None,
transport: ClientTransport::HttpJson,
model_hint,
@@ -10,6 +10,7 @@ use gateway_core::{
engine::execution::AuthenticatedClient,
routing::{PublicModelDescriptor, PublicModelId, PublicModelProfile},
};
use gateway_protocol::anthropic::ANTHROPIC_PROTOCOL;
use serde::Deserialize;
use serde_json::{Value, json};
@@ -52,7 +53,7 @@ pub(crate) async fn models(
let mut input = RequestInput::query(super::router::MODELS_PATH.to_owned(), None, &headers);
if anthropic {
input.protocol = "anthropic".to_owned();
input.protocol = ANTHROPIC_PROTOCOL.to_owned();
}
let response = middleware::query_response(service.execution(), client, input, move |client| {
models_response(state, client, query, anthropic)
@@ -173,7 +174,7 @@ pub(crate) async fn model_detail(
&headers,
);
if anthropic {
input.protocol = "anthropic".to_owned();
input.protocol = ANTHROPIC_PROTOCOL.to_owned();
}
let response = middleware::query_response(
service.execution(),
@@ -6,6 +6,7 @@ use axum::http::{HeaderMap, HeaderName, HeaderValue};
use base64::{Engine as _, engine::general_purpose::STANDARD};
use bytes::Bytes;
use gateway_core::operation::{ClientTurn, GenerateRequest, Operation, ProtocolPayload};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use gateway_protocol::openai::{
X_OPENAI_INTERNAL_CODEX_RESPONSES_LITE_HEADER, X_OPENAI_MEMGEN_REQUEST_HEADER,
is_transport_managed_request_header,
@@ -17,7 +18,6 @@ use crate::openai::generation::{
ContinuationIntent, DecodedGenerationRequest, GenerationRequestMetadata,
};
const OPENAI_PROTOCOL: &str = "openai";
const OPENAI_SUBAGENT_KEY: &str = "x-openai-subagent";
const CODEX_TURN_METADATA_KEY: &str = "x-codex-turn-metadata";
const PASSTHROUGH_HEADERS_CONTEXT_KEY: &str = "opaque_request_headers";
@@ -4,6 +4,7 @@ use std::collections::BTreeMap;
use bytes::Bytes;
use gateway_core::event::{GatewayEvent, ProtocolWireEvent, ProviderEvent};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use gateway_protocol::openai::sse::{
encode_sse_event_with_metadata, response_failed_sse_data_from_error_event,
};
@@ -12,8 +13,6 @@ use serde_json::Value;
use super::error::ResponseEncodeError;
use crate::client_error::capacity_error_for_client;
const OPENAI_PROTOCOL: &str = "openai";
/// OpenAI Responses wire 的转发器
///
/// OpenAI Provider 与 xAI adapter 都必须交付 OpenAI wire
@@ -6,6 +6,7 @@ use std::{
};
use gateway_core::event::{GatewayEvent, ProtocolWireEvent, ProviderEvent};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use gateway_protocol::openai::sse::{parse_sse_events, sse_frame_is_done};
use serde_json::{Map, Value};
@@ -44,7 +45,7 @@ impl ResponseValidationFacts {
if let Some(wire) = event
.middleware_origin_wire()
.or_else(|| event.wire_event())
.filter(|wire| wire.protocol() == "openai")
.filter(|wire| wire.protocol() == OPENAI_PROTOCOL)
{
facts.observe_wire(wire);
}
@@ -16,6 +16,7 @@ use gateway_core::engine::response_control::{ResponseControl, ResponseControlUna
use gateway_core::error::{GatewayError, GatewayErrorKind};
use gateway_core::event::ProviderResponseHeader;
use gateway_core::operation::ProviderSessionState;
use gateway_protocol::openai::OPENAI_PROTOCOL;
use crate::openai::error::{gateway_error_contract, gateway_error_from_engine};
use crate::openai::middleware::{PendingExecution, pending_execution_response};
@@ -118,7 +119,7 @@ pub(super) async fn execution_response(
.collect();
// 失败也要保留会话到洋葱链返回,否则释放会话会取消仍在等待 next 的插件
return Ok(pending_execution_response(
"openai".to_owned(),
OPENAI_PROTOCOL.to_owned(),
status,
response_headers,
MiddlewareFrame::new(
@@ -142,7 +143,7 @@ pub(super) async fn execution_response(
.map(|header| MiddlewareHeader::new(header.name(), header.value().clone()))
.collect();
Ok(MiddlewareResponse::new(
"openai".to_owned(),
OPENAI_PROTOCOL.to_owned(),
200,
headers,
Box::new(body),
@@ -159,7 +160,7 @@ pub(super) async fn forward_response(
response_control: &ResponseControl,
) -> ForwardOutcome {
let (protocol, status, headers, mut body, _) = response.into_parts();
if protocol != "openai" || StatusCode::from_u16(status).is_err() {
if protocol != OPENAI_PROTOCOL || StatusCode::from_u16(status).is_err() {
return fail_body(connection, body, MiddlewareError::InvalidState, &request_id).await;
}
let response_headers = headers
@@ -26,6 +26,7 @@ use gateway_core::{
lifecycle::{ConnectionGuard, ConnectionLifecycle},
operation::OperationKind,
};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use crate::middleware::headers::encode_headers;
use crate::{
@@ -328,7 +329,7 @@ async fn serve_responses_websocket(socket: Socket, session: ResponsesWebSocketSe
let validation = ResponseValidationFacts::default();
let input = RequestInput {
endpoint: crate::openai::router::RESPONSES_PATH.to_owned(),
protocol: "openai".to_owned(),
protocol: OPENAI_PROTOCOL.to_owned(),
operation: Some(OperationKind::Generate),
transport: ClientTransport::WebSocket,
model_hint,
@@ -349,7 +350,7 @@ async fn serve_responses_websocket(socket: Socket, session: ResponsesWebSocketSe
Box::new(move |prepared, request| {
Box::pin(async move {
let (protocol, headers, body) = request_parts(request.clone())?;
if protocol != "openai" {
if protocol != OPENAI_PROTOCOL {
return Err(MiddlewareError::Rejected);
}
let payload =
@@ -11,6 +11,7 @@ use axum::{
use gateway_core::engine::execution::ClientTransport;
use gateway_core::error::{GatewayError, GatewayErrorKind};
use gateway_core::operation::{Operation, OperationKind, RawJsonPayload, StandaloneSearchRequest};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use crate::ApiState;
use crate::middleware::headers::encode_headers;
@@ -22,8 +23,6 @@ use crate::openai::{
router::SEARCH_PATH,
};
const OPENAI_PROTOCOL: &str = "openai";
/// `POST /v1/alpha/search`
pub(crate) async fn standalone_search(
State(state): State<ApiState>,
@@ -15,6 +15,7 @@ use gateway_core::lifecycle::{ConnectionDraining, ConnectionGuard, ConnectionLif
use gateway_core::routing::{
ProviderCatalogUnavailable, ProviderKind, PublicModelDescriptor, PublicModelId, UpstreamModelId,
};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use uuid::Uuid;
use super::auth::ClientApiKeyAuthError;
@@ -159,7 +160,7 @@ impl OpenAiService {
upstream_model,
operation,
ExecutionRequestMetadata {
protocol: "openai".to_owned(),
protocol: OPENAI_PROTOCOL.to_owned(),
endpoint,
transport: ClientTransport::HttpJson,
stream: false,
@@ -10,6 +10,14 @@ use std::time::SystemTime;
pub(super) struct TestPeerInfo(pub Arc<Mutex<Option<ProviderQuota>>>);
impl UpstreamInfoSource for TestPeerInfo {
fn is_peer(&self, _: &ProviderAccount) -> bool {
self.0
.lock()
.unwrap()
.as_ref()
.is_some_and(|quota| quota.upstream.as_ref().is_some_and(|info| info.discovered))
}
fn signal(
&self,
_: &ProviderAccount,
@@ -84,6 +84,9 @@ pub struct UpstreamSchedulingSignal {
}
pub trait UpstreamInfoSource: Send + Sync {
/// 当前连接是否已识别为互联节点,不以额度快照的新鲜度代替身份
fn is_peer(&self, account: &ProviderAccount) -> bool;
fn signal(
&self,
account: &ProviderAccount,
@@ -110,6 +113,13 @@ impl UpstreamInfoReader {
}
impl UpstreamInfoSource for UpstreamInfoReader {
fn is_peer(&self, account: &ProviderAccount) -> bool {
self.source
.get()
.and_then(Weak::upgrade)
.is_some_and(|source| source.is_peer(account))
}
fn signal(
&self,
account: &ProviderAccount,
@@ -158,6 +158,7 @@ impl TraceContext {
"recovery": {
"replaySafe": error.replay_is_safe(),
"retryProhibited": error.retry_is_prohibited(),
"sameAccountRetryBlocked": error.same_account_retry_blocked(),
"sameAccountCredentialRetry": error.retries_same_account(),
"intent": error.pre_delivery_retry().map(|intent| format!("{intent:?}")),
"continuationFailure": error.continuation_failure().map(|failure| format!("{failure:?}")),
@@ -346,6 +347,7 @@ impl TraceContext {
for key in [
"replaySafe",
"retryProhibited",
"sameAccountRetryBlocked",
"sameAccountCredentialRetry",
] {
kept[key] = json!(recovery.get(key).and_then(Value::as_bool));
@@ -1672,15 +1672,19 @@ where
error.pre_delivery_retry(),
Some(crate::error::PreDeliveryRetry::SameAccountConnectionRetry { .. })
);
// 服务器建议不可截短;超出请求冻结的上限后沿用安全换号或原错误出口
// 服务器建议不可截短
// 超出上限,或上游拒绝继续重试当前账号时,跳过同账号等待
// 较短的 Retry-After 也直接沿用安全换号或原错误出口
let retry_after_exceeds_limit = error
.retry_after()
.is_some_and(|delay| delay > self.plan.max_retry_after());
let same_account_retry_blocked = error.same_account_retry_blocked();
// Provider 的有界同账号传输恢复是尽力重试策略,不要求把 Sent/Ambiguous
// 改写成安全证明;外部副作用、取消、预算与下游提交仍由 Core 拦截
let transport_recovery = match error.pre_delivery_retry() {
Some(crate::error::PreDeliveryRetry::SameAccountConnectionRetry { transport })
if !execution_effect_observed
if !same_account_retry_blocked
&& !execution_effect_observed
&& !self.connection_budget.exhausted()
&& !self.replay_barrier_reached
&& !self.delivery_pending
@@ -1698,7 +1702,8 @@ where
Some(crate::error::PreDeliveryRetry::SameAccountTransportRetry {
retry_index,
delay,
}) if !retry_after_exceeds_limit
}) if !same_account_retry_blocked
&& !retry_after_exceeds_limit
&& !execution_effect_observed
&& !self.connection_budget.exhausted()
&& !self.replay_barrier_reached
@@ -1707,7 +1712,8 @@ where
Some((AttemptTransport::Retry(retry_index), delay))
}
Some(crate::error::PreDeliveryRetry::SameAccountTransportFallback)
if !retry_after_exceeds_limit
if !same_account_retry_blocked
&& !retry_after_exceeds_limit
&& !execution_effect_observed
&& !self.connection_budget.exhausted()
&& !self.replay_barrier_reached
@@ -1754,7 +1760,11 @@ where
max_delay,
jitter,
..
}) if ordinary_retry && !suppress_transient_retry && !retry_after_exceeds_limit => {
}) if !same_account_retry_blocked
&& ordinary_retry
&& !suppress_transient_retry
&& !retry_after_exceeds_limit =>
{
let retries = self
.transient_retry_counts
.entry(current.metadata.provider_account_id().clone())
@@ -1776,7 +1786,8 @@ where
}
_ => None,
};
let same_account_retry = !execution_effect_observed
let same_account_retry = !same_account_retry_blocked
&& !execution_effect_observed
&& error.retries_same_account()
&& provider_proved_replay_safe
&& !self.replay_barrier_reached
@@ -1834,6 +1845,7 @@ where
"reserveConnectionRotation": reserve_connection_rotation,
"transientRetry": transient_retry.is_some(),
"retryAfterExceeded": retry_after_exceeds_limit,
"sameAccountRetryBlocked": same_account_retry_blocked,
"maxRetryAfterSeconds": self.plan.max_retry_after().as_secs(),
"clientRetrySuppressed": suppress_transient_retry,
"rotationRetry": rotation_retry, "accountRotations": self.account_rotations,
@@ -801,16 +801,38 @@ impl DefaultExecutionService {
active_request,
admission_decision_ms,
} = start_guard;
let observation = plan
.candidates()
.first()
.map_or_else(Default::default, |candidate| {
providers.request_observation(
candidate.provider(),
&operation,
client.policy.key_id(),
)
let observation_provider = match &target {
ExecutionTarget::ProviderEndpoint { provider, .. } => Some(provider),
ExecutionTarget::Model(_) => plan
.candidates()
.iter()
.find(|candidate| {
operation
.provider_session_state(candidate.provider().as_str())
.is_some_and(|state| state.extension_owner().is_none())
})
.or_else(|| plan.candidates().first())
.map(|candidate| candidate.provider()),
};
let provider_observation =
observation_provider.map_or_else(Default::default, |candidate| {
providers.request_observation(candidate, &operation, client.policy.key_id())
});
// 纯协议事实不依赖候选顺序,Provider 仅补充所属会话的旧配置和恢复关联
let observation = match &operation {
Operation::Generate(request) => {
let payload = request.protocol_payload();
gateway_protocol::request::request_observation(
payload.protocol(),
payload.body(),
payload.context(),
provider_observation
.previous_reasoning_effort_override
.as_deref(),
)
}
_ => Default::default(),
};
let (request_kind, subagent_kind) = if let Some(nested) = authorization.nested.as_ref() {
(
Some("plugin_child_model".to_owned()),
@@ -843,7 +865,9 @@ impl DefaultExecutionService {
operation: operation.kind(),
endpoint: metadata.endpoint,
client_transport: metadata.transport.as_str().to_owned(),
requested_model: target.into_public_model().or(observation.requested_model),
requested_model: target
.into_public_model()
.or(provider_observation.requested_model),
client_ip: metadata.client_ip,
user_agent: metadata.user_agent,
reasoning_effort: observation.reasoning_effort,
@@ -851,7 +875,7 @@ impl DefaultExecutionService {
request_kind,
subagent_kind,
compact: observation.compact,
continuation: observation.continuation,
continuation: provider_observation.continuation,
image_generation_requested: operation.image_generation_requested(),
admission_decision_ms,
started_at,
@@ -1088,17 +1088,13 @@ pub struct ProviderRequest {
candidate: ProviderCandidate,
}
/// Provider 对公共观测表可解释的请求语义;未知字段保持空值
/// Provider 私有状态与独立端点的观测补充,通用请求语义由协议层提取
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct ProviderRequestObservation {
/// 独立 Provider 端点声明的请求模型,仅用于观测,不参与文本模型目录路由
pub requested_model: Option<PublicModelId>,
/// Provider 从客户端请求与续接配置中解析的推理强度
pub reasoning_effort: Option<String>,
pub reasoning_preset: Option<String>,
pub request_kind: Option<String>,
pub subagent_kind: Option<String>,
pub compact: bool,
/// 已核实归属的上一响应配置;当前请求是否继承由协议层判断
pub previous_reasoning_effort_override: Option<String>,
pub continuation: ContinuationRequestObservation,
}
@@ -516,6 +516,7 @@ pub struct ProviderError {
failure_observation: Option<Box<ProviderErrorFailureObservation>>,
replay_safe: bool,
retry_prohibited: bool,
same_account_retry_blocked: bool,
pre_delivery_retry: Option<Box<PreDeliveryRetry>>,
credential_recovery_required: bool,
retry_same_account: bool,
@@ -559,6 +560,7 @@ impl ProviderError {
failure_observation: None,
replay_safe: false,
retry_prohibited: false,
same_account_retry_blocked: false,
pre_delivery_retry: None,
credential_recovery_required: false,
retry_same_account: false,
@@ -685,6 +687,22 @@ impl ProviderError {
self.retry_prohibited
}
/// 保留已选择的耗尽策略,但禁止执行同账号退避和传输恢复
///
/// Provider 在上游明确拒绝继续使用当前账号时设置
/// Core 不根据错误码自行推断
#[must_use]
pub const fn with_same_account_retry_blocked(mut self) -> Self {
self.same_account_retry_blocked = true;
self
}
/// 返回上游是否拒绝继续重试当前账号
#[must_use]
pub const fn same_account_retry_blocked(&self) -> bool {
self.same_account_retry_blocked
}
/// 允许 Core 仅在客户端尚未收到任何事件时执行一次受预算约束的换号恢复
///
/// 该标记不证明上游未执行,也不等价于 [`Self::with_replay_safe`];它只表达
@@ -1008,6 +1026,7 @@ impl ProviderError {
failure_observation: self.failure_observation.clone(),
replay_safe: self.replay_safe,
retry_prohibited: self.retry_prohibited,
same_account_retry_blocked: self.same_account_retry_blocked,
pre_delivery_retry: self.pre_delivery_retry.clone(),
credential_recovery_required: self.credential_recovery_required,
retry_same_account: self.retry_same_account,
@@ -1036,6 +1055,10 @@ impl fmt::Debug for ProviderError {
)
.field("retry_after", &self.retry_after)
.field("retry_prohibited", &self.retry_prohibited)
.field(
"same_account_retry_blocked",
&self.same_account_retry_blocked,
)
.field("continuation_failure", &self.continuation_failure)
.field(
"continuation_recovery_disposition",
+4 -35
View File
@@ -3,7 +3,7 @@
//! 这里只保留网关需要解释、路由或结算的稳定语义
//! 协议 adapter 无法写入
//! wire body 与连接级事实随 [`ProtocolPayload`] 或 [`RawJsonPayload`] 不透明传递,
//! 由对应 Provider 自己解释
//! 纯协议事实由协议层解析,私有字段与会话由对应 Provider 解释
use std::collections::BTreeSet;
use std::fmt;
@@ -458,8 +458,8 @@ impl ClientTurn {
impl GenerateRequest {
/// 创建携带协议不透明正文的生成请求
///
/// 完整请求只以原始 wire object 保存;Core 仅局部只读已知字段以推导路由能力,
/// Provider 负责解释与转换正文
/// 完整请求以原始 wire object 保存,协议层提取路由与权限所需的纯事实
/// Provider 在选定账号后编排协议转换与私有字段处理
#[must_use]
pub fn from_protocol_payload(protocol_payload: ProtocolPayload) -> Self {
Self {
@@ -521,9 +521,7 @@ impl GenerateRequest {
/// 返回客户端是否请求了图片生成工具
#[must_use]
pub fn image_generation_requested(&self) -> bool {
self.payload.source_image_generation
|| (self.protocol_payload().protocol() == "openai"
&& requests_image_generation(self.body()))
self.payload.source_image_generation || self.payload.wire_facts.image_generation
}
/// 返回指定 Provider 的连接内会话状态
@@ -596,35 +594,6 @@ impl fmt::Debug for GenerateRequest {
}
}
/// 原生工具声明授权上游生成图片,Codex 的被动 namespace 与图片输入不属于该权限
fn requests_image_generation(body: &Map<String, Value>) -> bool {
contains_image_tool(body.get("tools"))
|| body
.get("input")
.and_then(Value::as_array)
.is_some_and(|items| {
items.iter().any(|item| {
item.get("type").and_then(Value::as_str) == Some("additional_tools")
&& contains_image_tool(item.get("tools"))
})
})
|| body.get("tool_choice").is_some_and(|choice| {
is_image_tool(choice)
|| (choice.get("type").and_then(Value::as_str) == Some("allowed_tools")
&& contains_image_tool(choice.get("tools")))
})
}
fn contains_image_tool(value: Option<&Value>) -> bool {
value
.and_then(Value::as_array)
.is_some_and(|tools| tools.iter().any(is_image_tool))
}
fn is_image_tool(value: &Value) -> bool {
value.get("type").and_then(Value::as_str) == Some("image_generation")
}
/// 图像 API 的稳定端点语义
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum ImageRequestKind {
@@ -1,7 +1,7 @@
//! Provider 会话亲和、绑定和排除事实
use super::{ProviderStoreError, ProviderStoreErrorKind};
use crate::{account::ProviderAccountId, identity::ProviderKind};
use crate::{account::ProviderAccountId, identity::ProviderKind, policy::ClientApiKeyId};
use futures::future::BoxFuture;
use std::{collections::BTreeSet, fmt, time::Duration};
@@ -81,6 +81,36 @@ pub struct ProviderSessionAlias {
pub follow_only: bool,
}
/// 发送前需要核对版本的会话绑定;首次认领的 expected 为空
#[derive(Clone, Copy)]
pub struct ProviderSessionClaim<'a> {
pub key: &'a ProviderSessionAffinityKey,
pub expected: Option<&'a ProviderSessionBinding>,
pub account: &'a ProviderAccountId,
pub ttl: Duration,
}
/// 仅成功的内置轮询推进游标;有会话的分配必须同时提交绑定
#[derive(Clone, Copy)]
pub enum ProviderAccountAdmission<'a> {
Session(ProviderSessionClaim<'a>),
RoundRobin {
client_key: &'a ClientApiKeyId,
cursor: u64,
session: Option<ProviderSessionClaim<'a>>,
},
}
impl<'a> ProviderAccountAdmission<'a> {
#[must_use]
pub const fn session(self) -> Option<ProviderSessionClaim<'a>> {
match self {
Self::Session(session) => Some(session),
Self::RoundRobin { session, .. } => session,
}
}
}
/// 客户端作用域内的会话账号绑定,原始会话身份由 Provider 哈希后传入
pub trait ProviderSessionAffinityPort: Send + Sync {
fn load<'a>(
@@ -89,16 +119,19 @@ pub trait ProviderSessionAffinityPort: Send + Sync {
key: &'a ProviderSessionAffinityKey,
) -> BoxFuture<'a, Result<Option<ProviderSessionBinding>, ProviderStoreError>>;
/// 在发送前原子认领、续期或迁移;None 表示快照已变化,调用方必须释放租约重新选择
/// expected 为 None 只允许首次认领,已有快照必须连同版本匹配
fn compare_and_bind<'a>(
/// 读取本 Client Key 和 Provider 的下一轮位置,不推进游标
fn round_robin_cursor<'a>(
&'a self,
provider_kind: &'a ProviderKind,
key: &'a ProviderSessionAffinityKey,
expected: Option<&'a ProviderSessionBinding>,
account_id: &'a ProviderAccountId,
ttl: Duration,
) -> BoxFuture<'a, Result<Option<ProviderSessionBinding>, ProviderStoreError>>;
provider: &'a ProviderKind,
client_key: &'a ClientApiKeyId,
) -> BoxFuture<'a, Result<u64, ProviderStoreError>>;
/// 原子核对并提交会话版本和轮询位置;冲突时均不写入,调用方释放租约重选
fn admit_account<'a>(
&'a self,
provider: &'a ProviderKind,
admission: ProviderAccountAdmission<'a>,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>>;
/// 显式观测到的请求关联只指向会话键,不缓存账号,迁移后仍读取当前绑定
fn load_alias<'a>(
@@ -154,10 +154,16 @@ impl ProviderSchedulingLeaseRequest {
}
}
/// Lease 生命周期由具体 Store guard 管理,Provider 只能持有
pub trait ProviderLeaseGuard: Send + Sync + 'static {}
/// Drop 回收租约;同一请求需要重选时显式等待释放,避免被自己的租约阻塞
pub trait ProviderLeaseGuard: Send + Sync + 'static {
fn release(self: Box<Self>) -> BoxFuture<'static, Result<(), ProviderStoreError>>;
}
impl<T> ProviderLeaseGuard for T where T: Send + Sync + 'static {}
impl ProviderLeaseGuard for () {
fn release(self: Box<Self>) -> BoxFuture<'static, Result<(), ProviderStoreError>> {
Box::pin(async { Ok(()) })
}
}
pub enum ProviderLeaseAcquisition {
Acquired(Box<dyn ProviderLeaseGuard>),
@@ -10,8 +10,9 @@ pub use lease::{
};
mod affinity;
pub use affinity::{
ProviderSessionAffinityKey, ProviderSessionAffinityPort, ProviderSessionAlias,
ProviderSessionBinding, ProviderSessionExclusionPort, ProviderSessionExclusions,
ProviderAccountAdmission, ProviderSessionAffinityKey, ProviderSessionAffinityPort,
ProviderSessionAlias, ProviderSessionBinding, ProviderSessionClaim,
ProviderSessionExclusionPort, ProviderSessionExclusions,
};
mod catalog;
pub use catalog::{
@@ -4325,6 +4325,87 @@ fn retry_after_above_the_configured_limit_rotates_without_waiting() {
}
}
#[test]
fn no_more_retry_skips_same_account_wait_even_when_retry_after_is_short() {
let advised = Duration::from_secs(30);
for (rejection, rotates) in [
(
transient_rejection()
.with_same_account_retry_blocked()
.with_retry_after(advised),
true,
),
(
transient_rejection()
.with_transient_retry(
NonZeroU32::MIN,
Duration::ZERO,
Duration::ZERO,
Duration::ZERO,
TransientRetryPolicy::RetryThenReturnError,
)
.with_same_account_retry_blocked()
.with_retry_after(advised),
false,
),
] {
let operation = generate_operation();
let route_plan = retry_after_plan(&operation, 240);
let mut scripts = vec![Script::Stream {
account_id: "acct_first",
items: vec![Err(rejection)],
}];
if rotates {
scripts.push(Script::Stream {
account_id: "acct_second",
items: complete_stream(None),
});
}
let (coordinator, _, provider) = coordinator(scripts);
let mut session = block_on(coordinator.start(
model_request(&operation, SystemTime::now() + Duration::from_secs(30)),
operation,
route_plan,
None,
None,
CancellationToken::new(),
))
.unwrap();
let trace = session.trace().clone();
let mut collection = Box::pin(session.collect_uncommitted());
let polled = block_on(async { futures::poll!(&mut collection) });
let trace = trace.snapshot().unwrap();
let retry = trace["events"]
.as_array()
.unwrap()
.iter()
.find(|event| event["stage"] == "retry.decided")
.unwrap();
assert_eq!(retry["data"]["sameAccountRetryBlocked"], true);
assert_eq!(retry["data"]["transientRetry"], false);
assert_eq!(retry["data"]["retryAfterExceeded"], false);
if rotates {
assert!(matches!(polled, std::task::Poll::Ready(Ok(_))));
drop(collection);
let contexts = provider.contexts.lock().unwrap();
assert_eq!(contexts.len(), 2);
assert_eq!(contexts[1].required_account(), None);
assert!(
contexts[1]
.excluded_accounts()
.contains(&ProviderAccountId::new("acct_first").unwrap())
);
} else {
assert!(matches!(
polled,
std::task::Poll::Ready(Err(EngineError::Provider(error)))
if error.retry_after() == Some(advised)
));
assert_eq!(provider.contexts.lock().unwrap().len(), 1);
}
}
}
#[test]
fn excessive_retry_after_preserves_the_error_when_rotation_is_unavailable_or_disallowed() {
for (rejection, rotation_allowed) in [
@@ -148,6 +148,209 @@ fn request(service: &DefaultExecutionService, transport: ClientTransport) -> Sta
}
}
#[test]
fn request_observation_is_stable_across_candidate_order_and_respects_session_ownership() {
for order in [["alpha", "beta"], ["beta", "alpha"]] {
for (protocol, state_owner, extension_owned, body, expected) in [
(
"openai",
None,
false,
json!({"reasoning":{"effort":"medium"}}),
"medium",
),
(
"anthropic",
None,
false,
json!({"output_config":{"effort":" Future-Level "}}),
"Future-Level",
),
(
"openai",
Some("beta"),
false,
json!({"previous_response_id":"resp_previous","reasoning":{"effort":"medium"}}),
"high",
),
(
"openai",
Some("unavailable"),
false,
json!({"previous_response_id":"resp_previous","reasoning":{"effort":"medium"}}),
"medium",
),
(
"openai",
Some("beta"),
true,
json!({"previous_response_id":"resp_previous","reasoning":{"effort":"medium"}}),
"medium",
),
(
"openai",
Some("beta"),
false,
json!({"previous_response_id":"resp_previous","reasoning":{"effort":"medium"},"input":[{"type":"configuration_update","reasoning":{}}]}),
"medium",
),
] {
let kinds: Vec<_> = order
.iter()
.map(|name| ProviderKind::new(*name).unwrap())
.collect();
let directory = RuntimeAccountDirectory::new(
kinds
.iter()
.map(|kind| {
(
ProviderAccountId::new(format!("acct_{}", kind.as_str())).unwrap(),
RuntimeAccount::new(kind.clone(), BTreeSet::new()),
)
})
.collect(),
);
let snapshot = RuntimeSnapshot::new(
ConfigRevision::new(1).unwrap(),
gateway_core::settings::SettingsValues::new(
1,
"smart",
Default::default(),
None,
None,
),
kinds.clone(),
kinds
.into_iter()
.map(|kind| {
ProviderModel::new(
kind,
UpstreamModelId::new("gpt-start").unwrap(),
ModelCapabilities::new(BTreeSet::from([OperationKind::Generate]), None)
.with_upstream_feature_validation(),
)
})
.collect(),
vec![ClientPolicy::new(
ClientApiKeyId::new("key_start_test").unwrap(),
PlaintextClientApiKey::new("sk_start_test").unwrap(),
Arc::new(FrozenAccountScope::new(
Arc::new(directory),
ClientRoutingScope::all_accounts(),
)),
true,
RateLimits::unlimited(),
)],
)
.unwrap();
let store = Arc::new(TrackingExecutionStore::default());
let service =
DefaultExecutionService::new(
RuntimeSnapshotHandle::new(snapshot),
store.clone(),
ProviderRegistry::new(order.map(|name| {
Arc::new(SessionObservationProvider(name)) as Arc<dyn Provider>
}))
.unwrap(),
Arc::new(Admissions::default()),
Arc::new(UnusedContinuation),
Arc::new(RecordingClientApiKeyUsage::default()),
Arc::new(crate::RecordingDiagnostics::default()),
);
let mut generate = GenerateRequest::from_protocol_payload(
ProtocolPayload::json_object(protocol, body.as_object().unwrap().clone())
.unwrap()
.with_context(serde_json::Map::from_iter([(
"turn_metadata".to_owned(),
json!("{\"request_kind\":\"review\"}"),
)])),
);
if let Some(owner) = state_owner {
let mut state = gateway_core::operation::ProviderSessionState::new(
owner,
serde_json::Map::from_iter([("effort".to_owned(), json!("high"))]),
)
.unwrap();
if extension_owned {
state = state.with_extension_owner(
gateway_core::operation::ExtensionSessionOwner {
instance_id: "plugin".into(),
contribution_id: "adapter".into(),
adapter_id: "adapter".into(),
generation: 1,
incarnation: "incarnation".into(),
connection_local: true,
},
);
}
generate = generate.with_provider_session_state(state);
}
let mut start = request(&service, ClientTransport::HttpJson);
start.operation = Operation::Generate(generate);
start.metadata.protocol = protocol.to_owned();
let started = block_on(service.start(start)).expect("request observation is recorded");
block_on(started.session.detach_finalize());
let requests = store.requests.lock().unwrap();
let recorded = requests.first().expect("logical request");
assert_eq!(
recorded.reasoning_effort.as_deref(),
Some(expected),
"order={order:?}, state={state_owner:?}, extension={extension_owned}"
);
assert_eq!(
recorded.request_kind.as_deref(),
(protocol == "openai").then_some("review")
);
}
}
}
struct SessionObservationProvider(&'static str);
#[async_trait]
impl Provider for SessionObservationProvider {
fn name(&self) -> &'static str {
self.0
}
fn catalog_generation(&self) -> ProviderCatalogGeneration {
ProviderCatalogGeneration::default()
}
fn request_observation(
&self,
operation: &Operation,
_: &ClientApiKeyId,
) -> ProviderRequestObservation {
ProviderRequestObservation {
previous_reasoning_effort_override: operation
.provider_session_state(self.0)
.filter(|state| state.extension_owner().is_none())
.and_then(|state| state.payload().get("effort"))
.and_then(Value::as_str)
.map(str::to_owned),
..Default::default()
}
}
async fn query_model_capabilities(
&self,
) -> Result<Vec<ProviderModelCapabilities>, ProviderError> {
Ok(Vec::new())
}
async fn execute(
self: Arc<Self>,
_: ProviderRequest,
_: AttemptContext,
) -> Result<ProviderStream, ProviderError> {
Err(ProviderError::new(
ProviderErrorKind::Unavailable,
UpstreamSendState::NotSent,
))
}
}
#[test]
fn default_request_has_no_total_deadline_and_explicit_timeout_can_be_cleared() {
block_on(async {
+25 -33
View File
@@ -157,40 +157,28 @@ fn prompt_cache_and_image_generation_should_be_read_from_raw_body() {
}
#[test]
fn image_generation_detection_distinguishes_tools_from_image_inputs_and_passive_namespaces() {
for body in [
json!({"tools":[{"type":"image_generation"}]}),
json!({"tool_choice":{"type":"image_generation"}}),
json!({"tool_choice":{"type":"allowed_tools","tools":[{"type":"image_generation"}]}}),
json!({"input":[{"type":"additional_tools","tools":[{"type":"image_generation"}]}]}),
] {
assert!(
generate(body.clone()).image_generation_requested(),
"{body}"
);
}
for body in [
json!({"input":"Please generate an image"}),
json!({"input":[{"role":"user","content":[{"type":"input_image","image_url":"data:image/png;base64,AA=="}]}]}),
json!({"tools":[{"type":"namespace","name":"image_gen","tools":[{"type":"function","name":"imagegen"}]}]}),
json!({"tools":[{"type":"function","name":"image_generation"}]}),
] {
assert!(
!generate(body.clone()).image_generation_requested(),
"{body}"
);
}
let private = GenerateRequest::from_protocol_payload(
ProtocolPayload::json_object(
"private",
json!({"tools":[{"type":"image_generation"}]})
.as_object()
.unwrap()
.clone(),
fn image_generation_fact_tracks_body_replacement_and_ignores_context() {
let operation = Operation::Generate(generate(json!({"input":"hello"})));
let operation = operation
.replace_protocol_wire(
Some(Bytes::from_static(
br#"{"tools":[{"type":"image_generation"}]}"#,
)),
Map::new(),
)
.unwrap(),
);
assert!(!private.image_generation_requested());
.unwrap();
assert!(operation.image_generation_requested());
let operation = operation
.replace_protocol_wire(None, Map::from_iter([("tools".to_owned(), json!([]))]))
.unwrap();
assert!(operation.image_generation_requested());
let operation = operation
.replace_protocol_wire(
Some(Bytes::from_static(br#"{"input":"hello"}"#)),
Map::new(),
)
.unwrap();
assert!(!operation.image_generation_requested());
}
#[test]
@@ -203,6 +191,10 @@ fn image_generation_fact_survives_cross_protocol_translation() {
)
.unwrap();
assert!(translated.image_generation_requested());
let replaced = translated
.replace_protocol_wire(Some(Bytes::from_static(br#"{"messages":[]}"#)), Map::new())
.unwrap();
assert!(replaced.image_generation_requested());
}
#[test]
@@ -449,6 +449,9 @@ Ping 的自动 Pong 与关闭握手属于底层 WebSocket 协议事实;插件
或 `WebSocket { uri }`,不能在准备过程中切换传输类型。HTTP 正文是已经序列化、完成内容编码的实际字节,
插件自行处理压缩与 `Content-Encoding` 等头部的一致性;宿主不解码正文、补回字段或修复请求
内置转发在出站准备前,仅为已识别的互联上游组装 `X-CPR-Route`,向普通模型上游剥离。
插件改写目标或自行实现转发时,负责在互联节点间保留并维护路径,在最终模型上游前移除该头;宿主不在插件返回后重新判断或修复互联链路
`OutboundCall.metadata` 提供真实请求、Client Key、分组、Provider、模型、账号与 attempt 序号。
这些归属由宿主冻结,返回值只有请求;修改 URL、认证头或正文不改变账号租约、凭据版本与计费归属。
宿主不从最终字节重建模型、选号或续接事实;需要改变这些业务含义时,插件应在相应的 request / attempt 阶段保持一致。
@@ -24,6 +24,7 @@ pub use stream::{
MessagesAccumulator, MessagesToResponsesStream, ResponsesToMessagesStream,
message_to_stream_events,
};
pub use thinking::messages_request_effort;
pub use usage::MessageUsage;
/// Messages 载荷的协议标识
@@ -1,16 +1,22 @@
//! 按目标模型能力映射推理等级与预算;未知兼容上游沿用 CPA 手动预算
//! 提取 Messages 显式 effort,并按目标模型能力映射推理等级与预算
use serde_json::{Map, Value, json};
use super::MessagesModelCapabilities;
pub(super) fn to_responses(request: &Map<String, Value>) -> Value {
let configured = request
/// 返回客户端显式设置的 effort,预算与默认档位由转换器单独处理
#[must_use]
pub fn messages_request_effort(request: &Map<String, Value>) -> Option<&str> {
request
.get("output_config")
.and_then(|value| value.get("effort"))
.and_then(Value::as_str)
.map(|value| value.trim().to_ascii_lowercase())
.filter(|value| !value.is_empty());
.map(str::trim)
.filter(|value| !value.is_empty())
}
pub(super) fn to_responses(request: &Map<String, Value>) -> Value {
let configured = messages_request_effort(request).map(str::to_ascii_lowercase);
let thinking = request.get("thinking");
let effort = match thinking
.and_then(|value| value.get("type"))
@@ -127,7 +127,7 @@ pub fn codex_responses_request_semantics(
) -> CodexResponsesRequestSemantics {
codex_responses_request_semantics_with_turn_metadata(
body,
non_empty_string(context.get("turn_metadata")),
context.get("turn_metadata").and_then(Value::as_str),
None,
)
}
@@ -153,10 +153,12 @@ pub fn codex_responses_request_semantics_with_turn_metadata(
let reasoning_effort = reasoning_effort_override
.clone()
.or_else(|| reasoning_effort(body.get("reasoning")));
// 权威上下文与显式 turnMetadata 即使为空也不退回其他别名
let parsed_turn_metadata = turn_metadata
.or_else(|| body.get("turnMetadata").and_then(Value::as_str))
.or_else(|| request_turn_metadata(body))
.and_then(|value| serde_json::from_str::<Value>(value).ok());
let request_kind = parsed_turn_metadata
let mut request_kind = parsed_turn_metadata
.as_ref()
.and_then(|metadata| non_empty_string(metadata.get("request_kind")))
.map(ToOwned::to_owned);
@@ -174,6 +176,12 @@ pub fn codex_responses_request_semantics_with_turn_metadata(
&& proactive_multi_agent)
.then_some("ultra");
let compact = request_kind.as_deref() == Some("compaction") || compact_trigger;
// generate=false 才是预热,客户端分类提示不能把真实推理排除在用量统计外
if body.get("generate").and_then(Value::as_bool) == Some(false) {
request_kind = Some("prewarm".to_owned());
} else if request_kind.as_deref() == Some("prewarm") {
request_kind = None;
}
CodexResponsesRequestSemantics {
reasoning_effort,
@@ -217,7 +225,10 @@ fn latest_multi_agent_mode<'a>(
input: impl DoubleEndedIterator<Item = &'a Value>,
) -> Option<&'a str> {
input.rev().find_map(|item| {
if non_empty_string(item.get("role")) != Some("developer") {
// 显式 system 消息会由 Codex 编码器转为 developer,观测在编码前保持等价
let system_message = item.get("type").and_then(Value::as_str) == Some("message")
&& item.get("role").and_then(Value::as_str) == Some("system");
if non_empty_string(item.get("role")) != Some("developer") && !system_message {
return None;
}
item.get("content")?
@@ -1,4 +1,4 @@
//! 从 OpenAI 事件与响应头提取用量、额度和重试等待事实
//! 从 OpenAI 事件与响应头提取用量、额度和重试事实
use std::{collections::BTreeMap, time::Duration};
@@ -249,6 +249,37 @@ pub(super) fn retry_after_header(value: &Value) -> Option<(&Value, u64)> {
Some((value, seconds))
}
/// 判断上游错误是否携带 `x-retry-metadata: NO_MORE_RETRY`
///
/// 读取位置与 Retry-After 相同:优先错误对象的 headers,再看事件自身的 headers
/// 只承认这一明确信号,其他取值、数组和非字符串都不构成该事实
pub fn upstream_no_more_retry(value: &Value) -> bool {
let error = value
.pointer("/response/error")
.or_else(|| value.get("error"))
.unwrap_or(value);
no_more_retry_headers(error) || no_more_retry_headers(value)
}
/// 从上游错误正文判断是否声明不再重试当前账号
pub fn upstream_no_more_retry_from_body(body: &str) -> bool {
serde_json::from_str::<Value>(body).is_ok_and(|value| upstream_no_more_retry(&value))
}
fn no_more_retry_headers(value: &Value) -> bool {
value
.get("headers")
.and_then(Value::as_object)
.is_some_and(|headers| {
headers.iter().any(|(name, value)| {
name.eq_ignore_ascii_case("x-retry-metadata")
&& value
.as_str()
.is_some_and(|value| value.trim().eq_ignore_ascii_case("NO_MORE_RETRY"))
})
})
}
/// 标准化的单个限流窗口
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct RateLimitWindow {
@@ -1,5 +1,8 @@
//! 共享的 OpenAI 线协议类型、解析与编码入口
/// OpenAI 线协议载荷与事件的协议标识
pub const OPENAI_PROTOCOL: &str = "openai";
/// Codex Responses Lite 的 HTTP 请求头
pub const X_OPENAI_INTERNAL_CODEX_RESPONSES_LITE_HEADER: &str =
"x-openai-internal-codex-responses-lite";
+84 -3
View File
@@ -1,7 +1,56 @@
//! 从协议正文提取调度需要的纯请求事实
//! 从协议正文提取能力要求与请求观测事实
use serde_json::{Map, Value};
use crate::anthropic::{ANTHROPIC_PROTOCOL, messages_request_effort};
use crate::openai::{OPENAI_PROTOCOL, codex_responses_request_semantics_with_turn_metadata};
/// 与执行 Provider 无关的逻辑请求观测
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct RequestObservation {
/// 当前请求与已核实续接配置共同确定的推理强度
pub reasoning_effort: Option<String>,
/// 协议声明派生的推理预设
pub reasoning_preset: Option<String>,
/// 协议声明的请求类型,可信内部分类由调用方覆盖
pub request_kind: Option<String>,
/// 协议声明的子代理类型
pub subagent_kind: Option<String>,
/// 当前请求是否声明压缩
pub compact: bool,
}
/// 旧配置只由状态所属 Provider 提供,是否继承仍由当前协议的续接语义决定
#[must_use]
pub fn request_observation(
protocol: &str,
body: &Map<String, Value>,
context: &Map<String, Value>,
previous_reasoning_effort_override: Option<&str>,
) -> RequestObservation {
match protocol {
OPENAI_PROTOCOL => {
let semantics = codex_responses_request_semantics_with_turn_metadata(
body,
context.get("turn_metadata").and_then(Value::as_str),
previous_reasoning_effort_override,
);
RequestObservation {
reasoning_effort: semantics.reasoning_effort,
reasoning_preset: semantics.reasoning_preset.map(str::to_owned),
request_kind: semantics.request_kind,
subagent_kind: semantics.subagent_kind,
compact: semantics.compact,
}
}
ANTHROPIC_PROTOCOL => RequestObservation {
reasoning_effort: messages_request_effort(body).map(str::to_owned),
..RequestObservation::default()
},
_ => RequestObservation::default(),
}
}
/// 请求正文表达的能力要求
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct RequestFacts {
@@ -11,6 +60,8 @@ pub struct RequestFacts {
pub tools: bool,
/// 图片输入
pub vision: bool,
/// 图片生成工具声明
pub image_generation: bool,
/// 推理配置
pub reasoning: bool,
/// JSON Schema 输出
@@ -30,12 +81,13 @@ pub fn request_facts(protocol: &str, body: &Map<String, Value>) -> RequestFacts
..RequestFacts::default()
};
match protocol {
"openai" => {
OPENAI_PROTOCOL => {
facts.max_output_tokens = body
.get("max_output_tokens")
.and_then(Value::as_u64)
.filter(|n| *n > 0);
facts.vision = contains_type(body.get("input"), "input_image");
facts.image_generation = requests_image_generation(body);
facts.reasoning = body.get("reasoning").is_some_and(Value::is_object);
facts.json_schema = body
.get("text")
@@ -48,7 +100,7 @@ pub fn request_facts(protocol: &str, body: &Map<String, Value>) -> RequestFacts
.and_then(Value::as_str)
.is_some();
}
"anthropic" => {
ANTHROPIC_PROTOCOL => {
facts.max_output_tokens = body.get("max_tokens").and_then(Value::as_u64);
facts.vision = contains_type(body.get("messages"), "image");
facts.reasoning = body
@@ -68,6 +120,35 @@ pub fn request_facts(protocol: &str, body: &Map<String, Value>) -> RequestFacts
facts
}
/// 原生工具声明授权上游生成图片,Codex 的被动 namespace 与图片输入不属于该权限
fn requests_image_generation(body: &Map<String, Value>) -> bool {
contains_image_tool(body.get("tools"))
|| body
.get("input")
.and_then(Value::as_array)
.is_some_and(|items| {
items.iter().any(|item| {
item.get("type").and_then(Value::as_str) == Some("additional_tools")
&& contains_image_tool(item.get("tools"))
})
})
|| body.get("tool_choice").is_some_and(|choice| {
is_image_tool(choice)
|| (choice.get("type").and_then(Value::as_str) == Some("allowed_tools")
&& contains_image_tool(choice.get("tools")))
})
}
fn contains_image_tool(value: Option<&Value>) -> bool {
value
.and_then(Value::as_array)
.is_some_and(|tools| tools.iter().any(is_image_tool))
}
fn is_image_tool(value: &Value) -> bool {
value.get("type").and_then(Value::as_str) == Some("image_generation")
}
fn contains_type(value: Option<&Value>, expected: &str) -> bool {
match value {
Some(Value::Array(values)) => values.iter().any(|v| contains_type(Some(v), expected)),
@@ -219,6 +219,27 @@ fn codex_semantics_should_detect_ultra_preset_from_proactive_multi_agent() {
assert_eq!(semantics.reasoning_preset, Some("ultra"));
assert!(!semantics.compact);
for (role, message_type, expected) in [
("system", Some("message"), Some("ultra")),
("system", None, None),
("system", Some("function_call"), None),
("user", Some("message"), None),
(" system ", Some("message"), None),
] {
let mut shaped = body.clone();
let item = shaped["input"][0].as_object_mut().unwrap();
item.insert("role".to_owned(), json!(role));
if let Some(message_type) = message_type {
item.insert("type".to_owned(), json!(message_type));
} else {
item.remove("type");
}
assert_eq!(
codex_responses_request_semantics(&shaped, &Map::new()).reasoning_preset,
expected
);
}
for (effort, expected_preset) in [("high", None), ("max", Some("ultra"))] {
body["input"]
.as_array_mut()
@@ -230,6 +251,27 @@ fn codex_semantics_should_detect_ultra_preset_from_proactive_multi_agent() {
}
}
#[test]
fn empty_authoritative_turn_metadata_does_not_fall_back_to_body_aliases() {
for (context, body) in [
(
json!({"turn_metadata":""}),
json!({"turnMetadata":"{\"request_kind\":\"review\"}"}),
),
(
json!({"turn_metadata":" "}),
json!({"turnMetadata":"{\"request_kind\":\"review\"}"}),
),
(
json!({}),
json!({"turnMetadata":"","turn_metadata":"{\"request_kind\":\"review\"}"}),
),
] {
let semantics = codex_responses_request_semantics(&object(body), &object(context));
assert_eq!(semantics.request_kind, None);
}
}
fn object(value: Value) -> Map<String, Value> {
let Value::Object(object) = value else {
panic!("test value must be an object");
@@ -267,3 +309,42 @@ fn account_identity_does_not_infer_a_session_from_turn_or_cache_keys() {
assert_eq!(codex_account_session_id(&object(body), &Map::new()), None);
}
}
#[test]
fn request_observation_classifies_prewarm_from_generate() {
for (generate, request_kind, expected_kind) in [
(Some(json!(false)), None, Some("prewarm")),
(Some(json!(false)), Some("prewarm"), Some("prewarm")),
(Some(json!(false)), Some("review"), Some("prewarm")),
(Some(json!(true)), Some("prewarm"), None),
(None, Some("prewarm"), None),
(Some(json!(null)), Some("prewarm"), None),
(Some(json!("false")), Some("prewarm"), None),
(None, Some("review"), Some("review")),
(None, None, None),
] {
let mut body = Map::from_iter([
("model".to_owned(), json!("gpt-test")),
("input".to_owned(), json!("hello")),
("store".to_owned(), json!(false)),
]);
if let Some(generate) = generate {
body.insert("generate".to_owned(), generate);
}
let context = Map::from_iter([(
"turn_metadata".to_owned(),
Value::String(
json!({"request_kind": request_kind, "subagent_kind": "worker"}).to_string(),
),
)]);
let observation = codex_responses_request_semantics(&body, &context);
assert_eq!(
observation.request_kind.as_deref(),
expected_kind,
"generate={:?}, request_kind={request_kind:?}",
body.get("generate"),
);
assert_eq!(observation.subagent_kind.as_deref(), Some("worker"));
}
}
@@ -4,7 +4,8 @@ use gateway_protocol::openai::events::{
RateLimitKeySource, RateLimitWindow, TokenUsage, billable_usage_is_complete, extract_sse_usage,
extract_usage, is_codex_quota_header_name, is_rate_limit_header_name,
parse_image_rate_limit_headers, parse_rate_limit_headers, parse_rate_limits_event,
rate_limits_to_header_pairs, retry_after_seconds_from_body,
rate_limits_to_header_pairs, retry_after_seconds_from_body, upstream_no_more_retry,
upstream_no_more_retry_from_body,
};
use serde_json::json;
@@ -429,6 +430,51 @@ fn retry_after_should_not_infer_retry_from_unrelated_error_message() {
assert_eq!(retry_after_seconds_from_body(&body), None);
}
#[test]
fn no_more_retry_should_read_the_error_header_without_replacing_retry_after() {
for (body, expected) in [
(
json!({"type":"error","error":{"type":"service_unavailable_error","code":"server_is_overloaded","headers":{"x-retry-metadata":"NO_MORE_RETRY","retry-after":"30","retry-after-ms":"30000"}}}),
true,
),
(
json!({"type":"response.failed","response":{"error":{"headers":{"X-Retry-Metadata":" no_more_retry "}}}}),
true,
),
(
json!({"type":"error","headers":{"x-retry-metadata":"NO_MORE_RETRY"},"error":{"code":"server_is_overloaded"}}),
true,
),
(
json!({"error":{"headers":{"x-retry-metadata":"RETRY","retry-after":"30"}}}),
false,
),
(
json!({"error":{"headers":{"x-retry-metadata":["NO_MORE_RETRY"]}}}),
false,
),
(
json!({"error":{"code":"server_is_overloaded","headers":{"retry-after":"30"}}}),
false,
),
] {
assert_eq!(upstream_no_more_retry(&body), expected, "{body}");
assert_eq!(
upstream_no_more_retry_from_body(&body.to_string()),
expected,
"{body}"
);
}
assert!(!upstream_no_more_retry_from_body("not-json"));
assert_eq!(
retry_after_seconds_from_body(
&json!({"error":{"headers":{"x-retry-metadata":"NO_MORE_RETRY","retry-after":"30"}}})
.to_string()
),
Some(30)
);
}
#[test]
fn retry_after_should_reject_malformed_json() {
assert_eq!(retry_after_seconds_from_body("not-json"), None);
@@ -1,7 +1,7 @@
//! 验证不同协议的共享请求事实
use gateway_protocol::request::request_facts;
use serde_json::json;
use gateway_protocol::request::{request_facts, request_observation};
use serde_json::{Map, json};
#[test]
fn messages_facts_include_images_inside_tool_results() {
@@ -20,3 +20,77 @@ fn responses_facts_keep_native_continuation_separate_from_history() {
assert!(facts.native_continuation);
assert!(!facts.tools && !facts.vision && !facts.reasoning && !facts.json_schema);
}
#[test]
fn image_generation_detection_distinguishes_tools_from_image_inputs_and_passive_namespaces() {
for body in [
json!({"tools":[{"type":"image_generation"}]}),
json!({"tool_choice":{"type":"image_generation"}}),
json!({"tool_choice":{"type":"allowed_tools","tools":[{"type":"image_generation"}]}}),
json!({"input":[{"type":"additional_tools","tools":[{"type":"image_generation"}]}]}),
] {
assert!(
request_facts("openai", body.as_object().expect("请求正文")).image_generation,
"{body}"
);
}
for body in [
json!({"input":"Please generate an image"}),
json!({"input":[{"role":"user","content":[{"type":"input_image","image_url":"data:image/png;base64,AA=="}]}]}),
json!({"tools":[{"type":"namespace","name":"image_gen","tools":[{"type":"function","name":"imagegen"}]}]}),
json!({"tools":[{"type":"function","name":"image_generation"}]}),
] {
assert!(
!request_facts("openai", body.as_object().expect("请求正文")).image_generation,
"{body}"
);
}
let body = json!({"tools":[{"type":"image_generation"}]});
for protocol in ["anthropic", "private"] {
assert!(!request_facts(protocol, body.as_object().expect("请求正文")).image_generation);
}
}
#[test]
fn messages_request_observation_preserves_explicit_effort_without_inventing_defaults() {
for (options, expected) in [
(json!({"output_config":{"effort":"xhigh"}}), Some("xhigh")),
(
json!({"thinking":{"type":"adaptive"},"output_config":{"effort":"low"}}),
Some("low"),
),
(
json!({"thinking":{"type":"disabled"},"output_config":{"effort":"high"}}),
Some("high"),
),
(
json!({"output_config":{"effort":" Future-Level "}}),
Some("Future-Level"),
),
(json!({"output_config":{"effort":" "}}), None),
(json!({"output_config":{"effort":null}}), None),
(json!({"output_config":{"effort":42}}), None),
(json!({"thinking":{"type":"adaptive"}}), None),
(
json!({"thinking":{"type":"enabled","budget_tokens":32768}}),
None,
),
(json!({}), None),
] {
let mut body = json!({"model":"gpt-5.4","max_tokens":128,"messages":[{"role":"user","content":"hello"}]});
body.as_object_mut()
.unwrap()
.extend(options.as_object().unwrap().clone());
let observation = request_observation(
"anthropic",
body.as_object().unwrap(),
&Map::new(),
Some("inapplicable-previous-effort"),
);
assert_eq!(
observation.reasoning_effort.as_deref(),
expected,
"{options}"
);
}
}
@@ -5,6 +5,9 @@ use std::time::Duration;
use futures::future::BoxFuture;
use gateway_core::lifecycle::CancellationToken;
use gateway_core::provider_ports::{
ProviderLeaseGuard, ProviderStoreError, ProviderStoreErrorKind,
};
use gateway_core::task::{DaemonTask, WorkerDaemonContext, WorkerTaskError};
use tokio::sync::{RwLock, RwLockReadGuard, oneshot};
use tokio::task::JoinSet;
@@ -22,11 +25,32 @@ struct State {
}
pub(crate) struct SchedulingLeaseGuard {
_release: oneshot::Sender<()>,
release: oneshot::Sender<oneshot::Sender<Result<(), ProviderStoreError>>>,
// 活跃请求可能比组合根活得久,不能因端口析构提前中止其续期与租约
_cleanup: SchedulingLeaseCleanup,
}
impl ProviderLeaseGuard for SchedulingLeaseGuard {
fn release(self: Box<Self>) -> BoxFuture<'static, Result<(), ProviderStoreError>> {
Box::pin(async move {
let (ack, released) = oneshot::channel();
self.release.send(ack).map_err(|_| {
ProviderStoreError::new(
ProviderStoreErrorKind::Unavailable,
"request scheduling lease release",
)
})?;
released.await.map_err(|source| {
ProviderStoreError::caused_by(
ProviderStoreErrorKind::Unavailable,
"await scheduling lease release",
source,
)
})?
})
}
}
impl SchedulingLeaseCleanup {
// 关闭必须等已发出的申请完成登记;不能漏掉 Redis 已创建、尚未返回的租约
pub(crate) async fn acquire(&self) -> Option<RwLockReadGuard<'_, ()>> {
@@ -39,7 +63,8 @@ impl SchedulingLeaseCleanup {
cancellation: CancellationToken,
release: impl Future<Output = crate::StoreResult<bool>> + Send + 'static,
) -> SchedulingLeaseGuard {
let (guard, dropped) = oneshot::channel::<()>();
let (guard, dropped) =
oneshot::channel::<oneshot::Sender<Result<(), ProviderStoreError>>>();
let stopping = self.0.stopping.clone();
let mut tasks = self
.0
@@ -50,17 +75,27 @@ impl SchedulingLeaseCleanup {
report_join(result);
}
tasks.spawn(async move {
tokio::select! {
let acknowledgement = tokio::select! {
biased;
() = stopping.cancelled() => cancellation.cancel(),
_ = dropped => {}
}
if let Err(error) = release.await {
() = stopping.cancelled() => { cancellation.cancel(); None },
ack = dropped => ack.ok(),
};
let result = release.await;
if let Err(error) = &result {
tracing::warn!(%error, "账号租约释放失败,依赖 TTL 回收");
}
if let Some(ack) = acknowledgement {
let _ = ack.send(result.map(|_| ()).map_err(|source| {
ProviderStoreError::caused_by(
ProviderStoreErrorKind::Unavailable,
"release scheduling lease",
source,
)
}));
}
});
SchedulingLeaseGuard {
_release: guard,
release: guard,
_cleanup: self.clone(),
}
}
@@ -249,6 +249,20 @@ pub struct CredentialLeaseGuard {
grant: Option<CredentialLeaseGrant>,
}
impl gateway_core::provider_ports::ProviderLeaseGuard for CredentialLeaseGuard {
fn release(
self: Box<Self>,
) -> futures::future::BoxFuture<'static, Result<(), ProviderStoreError>> {
Box::pin(async move {
(*self)
.release()
.await
.map(|_| ())
.map_err(|source| crate::provider_unavailable("release credential lease", source))
})
}
}
impl CredentialLeaseGuard {
/// 自动续期仅属于返回 guard;丢失租约会取消依赖该槽位的请求
pub fn maintain(
@@ -319,6 +333,16 @@ struct RenewingSchedulingLease {
_guard: CredentialLeaseGuard,
}
impl gateway_core::provider_ports::ProviderLeaseGuard for RenewingSchedulingLease {
fn release(
self: Box<Self>,
) -> futures::future::BoxFuture<'static, Result<(), ProviderStoreError>> {
let Self { _renewal, _guard } = *self;
drop(_renewal);
Box::new(_guard).release()
}
}
impl fmt::Debug for CredentialLeaseGuard {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
@@ -430,7 +454,12 @@ impl RedisCredentialLeaseRepository {
client_api_key_id: &ClientApiKeyId,
provider_kind: &ProviderKind,
) -> StoreResult<u64> {
let key = self.scheduling_cursor_key(client_api_key_id, provider_kind)?;
let key = super::scheduling_cursor_key(
&self.namespace,
client_api_key_id,
provider_kind,
"cursor",
)?;
let mut connection = self.connection.clone();
let cursor = Script::new(ADVANCE_SCHEDULING_CURSOR_SCRIPT)
.key(key)
@@ -473,21 +502,6 @@ impl RedisCredentialLeaseRepository {
))
}
fn scheduling_cursor_key(
&self,
client_api_key_id: &ClientApiKeyId,
provider_kind: &ProviderKind,
) -> StoreResult<String> {
let fingerprint = resource_fingerprint(
"provider scheduling cursor",
&format!("{}\0{}", client_api_key_id.as_str(), provider_kind.as_str()),
)?;
Ok(format!(
"{}:scheduler:client-provider:{{{fingerprint}}}:cursor",
self.namespace
))
}
async fn acquire_with_limits(
&self,
request: &CredentialLeaseRequest,
@@ -43,6 +43,21 @@ pub(crate) fn resource_fingerprint(entity: &'static str, value: &str) -> StoreRe
Ok(hex::encode(Sha256::digest(value.as_bytes())))
}
fn scheduling_cursor_key(
namespace: &str,
client_key: &gateway_core::policy::ClientApiKeyId,
provider: &gateway_core::routing::ProviderKind,
suffix: &str,
) -> StoreResult<String> {
let fingerprint = resource_fingerprint(
"provider scheduling cursor",
&format!("{}\0{}", client_key.as_str(), provider.as_str()),
)?;
Ok(format!(
"{namespace}:scheduler:client-provider:{{{fingerprint}}}:{suffix}"
))
}
pub(crate) fn namespace(value: &str) -> StoreResult<String> {
require_nonempty("Redis namespace", "namespace", value)?;
if value.len() > 128
@@ -3,9 +3,10 @@
use std::time::Duration;
use gateway_core::account::ProviderAccountId;
use gateway_core::policy::ClientApiKeyId;
use gateway_core::provider_ports::{
ProviderSessionAffinityKey, ProviderSessionAffinityPort, ProviderSessionBinding,
ProviderStoreError, ProviderStoreErrorKind,
ProviderAccountAdmission, ProviderSessionAffinityKey, ProviderSessionAffinityPort,
ProviderSessionBinding, ProviderStoreError, ProviderStoreErrorKind,
};
use gateway_core::routing::ProviderKind;
use redis::aio::ConnectionManager;
@@ -17,14 +18,15 @@ use super::{namespace, resource_fingerprint};
const MAX_SESSION_AFFINITY_TTL: Duration =
Duration::from_secs(gateway_core::account::MAX_SESSION_AFFINITY_TTL_HOURS as u64 * 3600);
// 比较完整记录而非账号 ID;冲突时既不改绑定,也不续期
const COMPARE_AND_BIND_SCRIPT: &str = r#"
local current = redis.call('GET', KEYS[1])
if (not current and ARGV[1] == '') or current == ARGV[1] then
redis.call('PSETEX', KEYS[1], tonumber(ARGV[3]), ARGV[2])
return 1
end
return 0
const CURSOR_TTL_MILLIS: u64 = 24 * 60 * 60 * 1000;
// 先比较全部快照再写入;失去首绑或轮询位置的请求不能续期或消耗名额
const ADMIT_ACCOUNT_SCRIPT: &str = r#"
if ARGV[2] ~= '' and (redis.call('GET', KEYS[1]) or '') ~= ARGV[1] then return 0 end
if ARGV[4] ~= '' and (redis.call('GET', KEYS[#KEYS]) or '0') ~= ARGV[4] then return 0 end
if ARGV[2] ~= '' then redis.call('PSETEX', KEYS[1], ARGV[3], ARGV[2]) end
if ARGV[4] ~= '' then redis.call('PSETEX', KEYS[#KEYS], ARGV[6], ARGV[5]) end
return 1
"#;
#[derive(serde::Serialize, serde::Deserialize)]
@@ -75,6 +77,15 @@ impl RedisProviderSessionAffinityRepository {
})
}
fn cursor_key(
&self,
provider: &ProviderKind,
client_key: &ClientApiKeyId,
) -> Result<String, ProviderStoreError> {
super::scheduling_cursor_key(&self.namespace, client_key, provider, "session-cursor")
.map_err(|_| provider_invalid("encode provider round robin cursor key"))
}
fn key(
&self,
provider_kind: &ProviderKind,
@@ -113,42 +124,71 @@ impl ProviderSessionAffinityPort for RedisProviderSessionAffinityRepository {
})
}
fn compare_and_bind<'a>(
fn round_robin_cursor<'a>(
&'a self,
provider_kind: &'a ProviderKind,
key: &'a ProviderSessionAffinityKey,
expected: Option<&'a ProviderSessionBinding>,
account_id: &'a ProviderAccountId,
ttl: Duration,
) -> futures::future::BoxFuture<'a, Result<Option<ProviderSessionBinding>, ProviderStoreError>>
{
provider: &'a ProviderKind,
client_key: &'a ClientApiKeyId,
) -> futures::future::BoxFuture<'a, Result<u64, ProviderStoreError>> {
Box::pin(async move {
let ttl_millis = session_affinity_ttl_millis(ttl)?;
let expected_raw = expected
.map(BindingRecord::encode)
.transpose()?
.unwrap_or_default();
// 同账号命中仅续期;初次认领及换号生成新版本
let binding = match expected.filter(|binding| binding.account_id() == account_id) {
Some(binding) => binding.clone(),
None => ProviderSessionBinding::new(
account_id.clone(),
uuid::Uuid::new_v4().simple().to_string(),
)?,
};
let applied = redis::Script::new(COMPARE_AND_BIND_SCRIPT)
.key(self.key(provider_kind, key)?)
.arg(expected_raw)
.arg(BindingRecord::encode(&binding)?)
.arg(ttl_millis)
.invoke_async::<bool>(&mut self.connection.clone())
let key = self.cursor_key(provider, client_key)?;
redis::cmd("GET")
.arg(key)
.query_async::<Option<u64>>(&mut self.connection.clone())
.await
.map_err(|source| {
crate::provider_unavailable("admit provider session binding", source)
})?;
Ok(applied.then_some(binding))
.map(Option::unwrap_or_default)
.map_err(|source| crate::provider_unavailable("read round robin cursor", source))
})
}
fn admit_account<'a>(
&'a self,
provider: &'a ProviderKind,
admission: ProviderAccountAdmission<'a>,
) -> futures::future::BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async move {
let script = redis::Script::new(ADMIT_ACCOUNT_SCRIPT);
let mut invocation = script.prepare_invoke();
if let Some(claim) = admission.session() {
let binding = match claim.expected.filter(|b| b.account_id() == claim.account) {
Some(existing) => existing.clone(),
None => ProviderSessionBinding::new(
claim.account.clone(),
uuid::Uuid::new_v4().simple().to_string(),
)?,
};
invocation
.key(self.key(provider, claim.key)?)
.arg(
claim
.expected
.map(BindingRecord::encode)
.transpose()?
.unwrap_or_default(),
)
.arg(BindingRecord::encode(&binding)?)
.arg(session_affinity_ttl_millis(claim.ttl)?);
} else {
invocation.arg("").arg("").arg(0);
}
if let ProviderAccountAdmission::RoundRobin {
client_key, cursor, ..
} = admission
{
invocation
.key(self.cursor_key(provider, client_key)?)
.arg(cursor)
.arg(cursor.wrapping_add(1))
.arg(CURSOR_TTL_MILLIS);
} else {
invocation.arg("");
}
invocation
.invoke_async(&mut self.connection.clone())
.await
.map_err(|source| crate::provider_unavailable("admit provider account", source))
})
}
fn load_alias<'a>(
&'a self,
provider: &'a ProviderKind,
@@ -83,6 +83,14 @@ async fn shutdown_releases_owned_account_pools_and_preserves_other_instances() {
.unwrap(),
ProviderLeaseAcquisition::Busy { .. }
));
second_guard.release().await.unwrap();
let ProviderLeaseAcquisition::Acquired(second_guard) = second_port
.try_acquire(ProviderLeaseRequest::Scheduling(other.clone()))
.await
.unwrap()
else {
panic!("released slot must be available before reselection")
};
first_requests.push(request);
second_requests.push(other);
first_guards.push(first_guard);
@@ -3,12 +3,39 @@
use std::time::Duration;
use gateway_core::account::ProviderAccountId;
use gateway_core::provider_ports::{ProviderSessionAffinityKey, ProviderSessionAffinityPort};
use gateway_core::provider_ports::{
ProviderAccountAdmission, ProviderSessionAffinityKey, ProviderSessionAffinityPort,
ProviderSessionBinding, ProviderSessionClaim, ProviderStoreError,
};
use gateway_core::routing::ProviderKind;
use gateway_store::redis::RedisProviderSessionAffinityRepository;
use redis::aio::ConnectionManager;
use uuid::Uuid;
async fn bind(
repo: &RedisProviderSessionAffinityRepository,
provider: &ProviderKind,
key: &ProviderSessionAffinityKey,
expected: Option<&ProviderSessionBinding>,
account: &ProviderAccountId,
ttl: Duration,
) -> Result<Option<ProviderSessionBinding>, ProviderStoreError> {
let claim = ProviderSessionClaim {
key,
expected,
account,
ttl,
};
if repo
.admit_account(provider, ProviderAccountAdmission::Session(claim))
.await?
{
repo.load(provider, key).await
} else {
Ok(None)
}
}
#[tokio::test]
async fn concurrent_claims_admit_exactly_one_account() {
let Some((repo, mut connection, namespace)) = affinity_repository().await else {
@@ -19,8 +46,22 @@ async fn concurrent_claims_admit_exactly_one_account() {
let first = ProviderAccountId::new("acct_first").unwrap();
let second = ProviderAccountId::new("acct_second").unwrap();
let (a, b) = tokio::join!(
repo.compare_and_bind(&provider, &key, None, &first, Duration::from_secs(60)),
repo.compare_and_bind(&provider, &key, None, &second, Duration::from_secs(60))
bind(
&repo,
&provider,
&key,
None,
&first,
Duration::from_secs(60)
),
bind(
&repo,
&provider,
&key,
None,
&second,
Duration::from_secs(60)
)
);
let winners = [a.unwrap(), b.unwrap()]
.into_iter()
@@ -56,24 +97,21 @@ async fn stale_revision_cannot_overwrite_an_account_that_returned_to_the_session
let a = ProviderAccountId::new("acct_a").unwrap();
let b = ProviderAccountId::new("acct_b").unwrap();
let ttl = Duration::from_secs(60);
let first = repo
.compare_and_bind(&provider, &key, None, &a, ttl)
let first = bind(&repo, &provider, &key, None, &a, ttl)
.await
.unwrap()
.unwrap();
let second = repo
.compare_and_bind(&provider, &key, Some(&first), &b, ttl)
let second = bind(&repo, &provider, &key, Some(&first), &b, ttl)
.await
.unwrap()
.unwrap();
let third = repo
.compare_and_bind(&provider, &key, Some(&second), &a, ttl)
let third = bind(&repo, &provider, &key, Some(&second), &a, ttl)
.await
.unwrap()
.unwrap();
assert_ne!(first.revision(), third.revision());
assert!(
repo.compare_and_bind(&provider, &key, Some(&first), &b, ttl)
bind(&repo, &provider, &key, Some(&first), &b, ttl)
.await
.unwrap()
.is_none()
@@ -90,13 +128,11 @@ async fn renewal_preserves_revision_but_expiration_requires_fresh_claim() {
let key = ProviderSessionAffinityKey::try_new("expired-session").unwrap();
let a = ProviderAccountId::new("acct_a").unwrap();
let ttl = Duration::from_secs(60);
let first = repo
.compare_and_bind(&provider, &key, None, &a, ttl)
let first = bind(&repo, &provider, &key, None, &a, ttl)
.await
.unwrap()
.unwrap();
let renewed = repo
.compare_and_bind(&provider, &key, Some(&first), &a, ttl)
let renewed = bind(&repo, &provider, &key, Some(&first), &a, ttl)
.await
.unwrap()
.unwrap();
@@ -113,14 +149,13 @@ async fn renewal_preserves_revision_but_expiration_requires_fresh_claim() {
.await
.unwrap();
assert!(
repo.compare_and_bind(&provider, &key, Some(&first), &a, ttl)
bind(&repo, &provider, &key, Some(&first), &a, ttl)
.await
.unwrap()
.is_none()
);
assert!(repo.load(&provider, &key).await.unwrap().is_none());
let fresh = repo
.compare_and_bind(&provider, &key, None, &a, ttl)
let fresh = bind(&repo, &provider, &key, None, &a, ttl)
.await
.unwrap()
.unwrap();
@@ -136,7 +171,7 @@ async fn damaged_binding_is_an_error_and_cannot_be_claimed_as_absent() {
let key = ProviderSessionAffinityKey::try_new("damaged-session").unwrap();
let a = ProviderAccountId::new("acct_a").unwrap();
let ttl = Duration::from_secs(60);
repo.compare_and_bind(&provider, &key, None, &a, ttl)
bind(&repo, &provider, &key, None, &a, ttl)
.await
.unwrap()
.unwrap();
@@ -153,13 +188,140 @@ async fn damaged_binding_is_an_error_and_cannot_be_claimed_as_absent() {
.unwrap();
assert!(repo.load(&provider, &key).await.is_err());
assert!(
repo.compare_and_bind(&provider, &key, None, &a, ttl)
bind(&repo, &provider, &key, None, &a, ttl)
.await
.unwrap()
.is_none()
);
}
#[tokio::test]
async fn round_robin_commits_only_with_the_current_cursor_and_session_version() {
use gateway_core::policy::ClientApiKeyId;
let Some((repo, mut connection, namespace)) = affinity_repository().await else {
return;
};
let peer = RedisProviderSessionAffinityRepository::new(connection.clone(), &namespace).unwrap();
let provider = ProviderKind::new("openai").unwrap();
let client = ClientApiKeyId::new("client").unwrap();
let account = ProviderAccountId::new("acct_a").unwrap();
let key = ProviderSessionAffinityKey::try_new("rr-session").unwrap();
let other = ProviderSessionAffinityKey::try_new("rr-other").unwrap();
let claim = ProviderSessionClaim {
key: &key,
expected: None,
account: &account,
ttl: Duration::from_secs(60),
};
let step = |cursor, session| ProviderAccountAdmission::RoundRobin {
client_key: &client,
cursor,
session,
};
let first = step(0, Some(claim));
let (a, b) = tokio::join!(
repo.admit_account(&provider, first),
peer.admit_account(&provider, first)
);
assert_ne!(a.unwrap(), b.unwrap());
let winner = repo.load(&provider, &key).await.unwrap().unwrap();
assert_eq!(winner.account_id(), &account);
assert!(
!repo
.admit_account(&provider, step(1, Some(claim)))
.await
.unwrap()
);
assert_eq!(
peer.round_robin_cursor(&provider, &client).await.unwrap(),
1
);
let other_claim = ProviderSessionClaim {
key: &other,
..claim
};
assert!(
!peer
.admit_account(&provider, step(0, Some(other_claim)))
.await
.unwrap()
);
assert!(repo.load(&provider, &other).await.unwrap().is_none());
assert!(peer.admit_account(&provider, step(1, None)).await.unwrap());
assert_eq!(
repo.round_robin_cursor(&provider, &client).await.unwrap(),
2
);
for (p, c) in [
(&provider, ClientApiKeyId::new("other").unwrap()),
(&ProviderKind::new("xai").unwrap(), client.clone()),
] {
assert_eq!(repo.round_robin_cursor(p, &c).await.unwrap(), 0);
}
let leases =
gateway_store::redis::RedisCredentialLeaseRepository::new(connection.clone(), &namespace)
.unwrap();
assert_eq!(
leases
.advance_scheduling_cursor(&client, &provider)
.await
.unwrap(),
0
);
assert_eq!(
repo.round_robin_cursor(&provider, &client).await.unwrap(),
2
);
let cursor_keys: Vec<String> = redis::cmd("KEYS")
.arg(format!("{namespace}:*:session-cursor"))
.query_async(&mut connection)
.await
.unwrap();
let ttl: i64 = redis::cmd("PTTL")
.arg(&cursor_keys[0])
.query_async(&mut connection)
.await
.unwrap();
assert!((1..=86_400_000).contains(&ttl));
redis::cmd("DEL")
.arg(&cursor_keys)
.query_async::<i64>(&mut connection)
.await
.unwrap();
assert_eq!(
peer.round_robin_cursor(&provider, &client).await.unwrap(),
0
);
assert_eq!(repo.load(&provider, &key).await.unwrap(), Some(winner));
let fresh = ProviderSessionAffinityKey::try_new("rr-fresh").unwrap();
let claims = [
other_claim,
ProviderSessionClaim {
key: &fresh,
..claim
},
];
let (a, b) = tokio::join!(
repo.admit_account(&provider, step(0, Some(claims[0]))),
peer.admit_account(&provider, step(0, Some(claims[1])))
);
let won_first = a.unwrap();
assert_ne!(won_first, b.unwrap());
let loser = claims[usize::from(won_first)];
assert!(repo.load(&provider, loser.key).await.unwrap().is_none());
assert!(
repo.admit_account(&provider, step(1, Some(loser)))
.await
.unwrap()
);
assert_eq!(
peer.round_robin_cursor(&provider, &client).await.unwrap(),
2
);
}
async fn affinity_repository() -> Option<(
RedisProviderSessionAffinityRepository,
ConnectionManager,
@@ -221,12 +383,11 @@ async fn turn_alias_cannot_be_reassigned_and_follows_session_migration() {
.await
.unwrap()
);
let first = repo
.compare_and_bind(&provider, &session, None, &a, ttl)
let first = bind(&repo, &provider, &session, None, &a, ttl)
.await
.unwrap()
.unwrap();
repo.compare_and_bind(&provider, &session, Some(&first), &b, ttl)
bind(&repo, &provider, &session, Some(&first), &b, ttl)
.await
.unwrap()
.unwrap();
@@ -293,8 +454,7 @@ async fn binding_and_turn_alias_can_renew_beyond_one_day_and_shorten_without_sca
let mut binding = None;
for hours in [24, 168, 720, 1] {
let ttl = Duration::from_secs(hours * 3600);
binding = repo
.compare_and_bind(&provider, &session, binding.as_ref(), &account, ttl)
binding = bind(&repo, &provider, &session, binding.as_ref(), &account, ttl)
.await
.unwrap();
assert!(binding.is_some());
@@ -324,7 +484,7 @@ async fn binding_and_turn_alias_can_renew_beyond_one_day_and_shorten_without_sca
}
for ttl in [Duration::ZERO, Duration::from_secs(721 * 3600)] {
assert!(
repo.compare_and_bind(&provider, &session, binding.as_ref(), &account, ttl)
bind(&repo, &provider, &session, binding.as_ref(), &account, ttl)
.await
.is_err()
);
+1 -1
View File
@@ -79,7 +79,7 @@ openssl-sys = { version = "=0.9.116", features = ["vendored"] }
gateway-host = { path = "../../gateway-host" }
allocation-counter = "0.8.1"
tempfile = "3.27.0"
tokio = { version = "1.53.3", default-features = false, features = ["rt-multi-thread", "test-util"] }
tokio = { version = "1.53.3", default-features = false, features = ["process", "rt-multi-thread", "test-util"] }
tracing-subscriber = { version = "0.3.23", features = ["json"] }
wiremock = "0.6.5"
@@ -48,6 +48,7 @@ use gateway_core::provider_ports::{
ProviderStoreErrorKind,
};
use gateway_core::routing::{ProviderKind, UpstreamModelId};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use secrecy::{ExposeSecret as _, SecretString};
use serde::Deserialize;
use serde_json::{Map, Number, Value};
@@ -1728,7 +1729,7 @@ fn build_connection_test_operation(
);
body.insert("stream".to_owned(), Value::Bool(true));
body.insert("store".to_owned(), Value::Bool(false));
let payload = ProtocolPayload::json_object("openai", body)
let payload = ProtocolPayload::json_object(OPENAI_PROTOCOL, body)
.map_err(|_| provider_admin_error(ProviderAdminErrorKind::Invalid))?;
Ok(Operation::Generate(GenerateRequest::from_protocol_payload(
payload,
@@ -1,8 +1,19 @@
//! 跨协议生成的 Messages fast 请求补充协议头,不改写原生 Messages 请求
//! Messages 跨协议转换的目标档位与 fast 协议头适配
use bytes::Bytes;
use gateway_core::engine::middleware::MiddlewareHeader;
use reqwest::header::HeaderMap;
use serde_json::{Map, Value};
pub(crate) fn normalize_translated_codex_service_tier(body: &mut Map<String, Value>) {
// 转换生成的默认档位沿用 Codex 客户端的省略表达,原生 Responses 保留调用方声明
if matches!(
body.get("service_tier").and_then(Value::as_str),
Some("auto" | "default")
) {
body.shift_remove("service_tier");
}
}
pub(crate) fn add_translated_fast_beta(headers: &mut Vec<MiddlewareHeader>, forwarded: &HeaderMap) {
const BETA: &[u8] = b"fast-mode-2026-02-01";
@@ -11,7 +11,7 @@ use gateway_core::engine::middleware::MiddlewareHeader;
use serde_json::{Map, Value};
pub(crate) use headers::is_non_codex_request_header;
pub(crate) use messages::add_translated_fast_beta;
pub(crate) use messages::{add_translated_fast_beta, normalize_translated_codex_service_tier};
pub(crate) use request::normalize_codex_request_body;
pub(crate) use search_tools::declares_hosted_search;
@@ -35,13 +35,6 @@ pub(super) fn normalize_non_codex_request_body(body: &mut Map<String, Value>) {
}
fn normalize_codex_oauth_options(body: &mut Map<String, Value>) {
// Codex Core 省略默认档位,OAuth 不接受公开 API 的 auto/default 字面值
if matches!(
body.get("service_tier").and_then(Value::as_str),
Some("auto" | "default")
) {
body.shift_remove("service_tier");
}
// Codex OAuth 拒绝这些客户端选项,即使采样参数取默认值
for field in [
"top_p",
@@ -21,11 +21,11 @@ use gateway_core::concurrency::{
};
use gateway_core::engine::{AttemptContext, ContinuationAttempt, policy::AccountPolicyError};
use gateway_core::provider_ports::{
ProviderConcurrencyPool, ProviderCooldownPort, ProviderCooldownScope, ProviderLeaseAcquisition,
ProviderLeaseGuard, ProviderLeasePort, ProviderLeaseRequest, ProviderSchedulingLeaseRequest,
ProviderScopedCooldown, ProviderSessionAffinityKey, ProviderSessionAffinityPort,
ProviderSessionBinding, ProviderSessionExclusionPort, ProviderSessionExclusions,
ProviderStoreError,
ProviderAccountAdmission, ProviderConcurrencyPool, ProviderCooldownPort, ProviderCooldownScope,
ProviderLeaseAcquisition, ProviderLeaseGuard, ProviderLeasePort, ProviderLeaseRequest,
ProviderSchedulingLeaseRequest, ProviderScopedCooldown, ProviderSessionAffinityKey,
ProviderSessionAffinityPort, ProviderSessionBinding, ProviderSessionClaim,
ProviderSessionExclusionPort, ProviderSessionExclusions, ProviderStoreError,
};
use gateway_core::routing::ProviderKind;
use secrecy::ExposeSecret;
@@ -145,7 +145,7 @@ pub struct CodexCredentialSelector {
quota: Arc<CodexCredentialQuotaService>,
catalog: Arc<CodexCredentialCatalogService>,
cooldowns: Arc<dyn ProviderCooldownPort>,
upstream_info: Option<Arc<dyn gateway_core::account::UpstreamInfoSource>>,
upstream_info: Arc<dyn gateway_core::account::UpstreamInfoSource>,
cookie_policy: CodexCookiePolicy,
risk_recovery: Mutex<HashMap<String, RiskRecoveryState>>,
account_feedback: Arc<AccountFeedbackStats>,
@@ -296,6 +296,7 @@ impl CodexCredentialSelector {
cooldowns: Arc<dyn ProviderCooldownPort>,
account_feedback: Arc<AccountFeedbackStats>,
cookie_policy: CodexCookiePolicy,
upstream_info: Arc<dyn gateway_core::account::UpstreamInfoSource>,
) -> Self {
Self {
provider_kind,
@@ -306,7 +307,7 @@ impl CodexCredentialSelector {
quota,
catalog,
cooldowns,
upstream_info: None,
upstream_info,
cookie_policy,
risk_recovery: Mutex::new(HashMap::new()),
waiting: ConcurrencyWaitQueue::default(),
@@ -340,13 +341,8 @@ impl CodexCredentialSelector {
.await
}
#[must_use]
pub(crate) fn with_upstream_info(
mut self,
upstream_info: Arc<dyn gateway_core::account::UpstreamInfoSource>,
) -> Self {
self.upstream_info = Some(upstream_info);
self
pub(crate) fn is_peer(&self, account: &ProviderAccount) -> bool {
self.upstream_info.is_peer(account)
}
fn scheduling_signals(
@@ -363,8 +359,7 @@ impl CodexCredentialSelector {
}
let Some(signal) = self
.upstream_info
.as_ref()
.and_then(|source| source.signal(account, model.model(), SystemTime::now()))
.signal(account, model.model(), SystemTime::now())
else {
return (None, None);
};
@@ -426,12 +421,12 @@ impl CodexCredentialSelector {
.as_ref()
.is_some_and(|binding| binding.account_id() != &selected_account)
|| !self
.admit_session(
affinity.key(),
current.as_ref(),
&selected_account,
policy.openai_session_affinity_ttl(),
)
.admit_account(ProviderAccountAdmission::Session(ProviderSessionClaim {
key: affinity.key(),
expected: current.as_ref(),
account: &selected_account,
ttl: policy.openai_session_affinity_ttl(),
}))
.await?
{
return Err(CredentialSelectionError::SessionBound(Box::new(
@@ -1000,15 +995,31 @@ impl CodexCredentialSelector {
}
}
}
let scheduling = self
.leases
.load_state(
request.attempt.client_api_key_ref(),
let policy = request.attempt.account_selection_policy();
let client_key = request.attempt.client_api_key_ref();
let use_assignment_cursor = policy.strategy() == RotationStrategy::RoundRobin
&& pinned_account.is_none()
&& !(binding.is_some()
&& (follow_only || affinity_mode == AccountAffinity::Preferred))
&& queued_account.is_none();
let (scheduling, assignment_cursor) = futures::try_join!(
self.leases.load_state(
client_key,
&self.provider_kind,
&account_ids,
concurrency_pool,
)
.await?;
concurrency_pool
),
async {
if use_assignment_cursor {
self.session_affinity
.round_robin_cursor(&self.provider_kind, client_key)
.await
.map(Some)
} else {
Ok(None)
}
},
)?;
let round_robin_cursor =
if affinity_mode == AccountAffinity::Preferred && binding.is_some() {
use sha2::{Digest, Sha256};
@@ -1022,7 +1033,7 @@ impl CodexCredentialSelector {
},
)
} else {
scheduling.round_robin_cursor()
assignment_cursor.unwrap_or_else(|| scheduling.round_robin_cursor())
};
let mut candidates = accounts
.into_iter()
@@ -1135,8 +1146,6 @@ impl CodexCredentialSelector {
}
let mut shortest_retry = None;
let mut busy_accounts = std::collections::BTreeSet::new();
let policy = request.attempt.account_selection_policy();
loop {
let preferred = pinned_account
.clone()
@@ -1424,25 +1433,53 @@ impl CodexCredentialSelector {
{
binding_account = primary.account_id();
}
if let Some(key) = binding_key {
match self
.admit_session(
key,
binding.as_ref(),
binding_account,
policy.openai_session_affinity_ttl(),
)
.await
let claim = binding_key.map(|key| ProviderSessionClaim {
key,
expected: binding.as_ref(),
account: binding_account,
ttl: policy.openai_session_affinity_ttl(),
});
// 插件委托与普通内置选号共享轮询,显式选择、亲和命中及等待不消耗位置
let admission = match assignment_cursor {
Some(cursor)
if !selection.is_policy_choice()
&& selection.preferred() != PreferredAccountSelection::Hit =>
{
Ok(false) => {
drop(guard);
continue 'capacity;
}
Some(ProviderAccountAdmission::RoundRobin {
client_key,
cursor,
session: claim,
})
}
_ => claim.map(ProviderAccountAdmission::Session),
};
if let Some(admission) = admission {
let admitted = match self.admit_account(admission).await {
Ok(false) => false,
Err(error)
if cache_preference
&& affinity_mode == AccountAffinity::Preferred =>
&& affinity_mode == AccountAffinity::Preferred
&& admission.session().is_some() =>
{
tracing::warn!(%error, "OpenAI cache preference update unavailable");
// 缓存偏好允许丢失,实际的轮询分配仍须占用该位置
match admission {
ProviderAccountAdmission::RoundRobin {
client_key,
cursor,
..
} => {
self.admit_account(
ProviderAccountAdmission::RoundRobin {
client_key,
cursor,
session: None,
},
)
.await?
}
ProviderAccountAdmission::Session(_) => true,
}
}
Err(error) => return Err(error),
Ok(true) => {
@@ -1468,7 +1505,15 @@ impl CodexCredentialSelector {
}),
);
}
true
}
};
if !admitted {
guard.release().await?;
if request.attempt.deadline().is_elapsed() {
return Err(CredentialSelectionError::AccountSnapshotChanged);
}
continue 'capacity;
}
}
let affinity_telemetry = affinity.telemetry(account.id(), cache_preference);
@@ -1615,27 +1660,28 @@ impl CodexCredentialSelector {
.map_err(binding_store_error)
}
async fn admit_session(
async fn admit_account(
&self,
key: &ProviderSessionAffinityKey,
expected: Option<&ProviderSessionBinding>,
account: &ProviderAccountId,
session_ttl: Duration,
admission: ProviderAccountAdmission<'_>,
) -> Result<bool, CredentialSelectionError> {
tokio::time::timeout(
let result = tokio::time::timeout(
SESSION_AFFINITY_TIMEOUT,
self.session_affinity.compare_and_bind(
&self.provider_kind,
key,
expected,
account,
session_ttl,
),
self.session_affinity
.admit_account(&self.provider_kind, admission),
)
.await
.map_err(binding_store_error)?
.map(|binding| binding.is_some())
.map_err(binding_store_error)
.unwrap_or_else(|source| {
Err(ProviderStoreError::caused_by(
gateway_core::provider_ports::ProviderStoreErrorKind::Unavailable,
"admit provider account",
source,
))
});
if admission.session().is_some() {
result.map_err(binding_store_error)
} else {
result.map_err(Into::into)
}
}
/// 只按账号整体的当前事实迁移主绑定,临时容量和单模型限制仍允许恢复热缓存
+13 -15
View File
@@ -170,21 +170,19 @@ pub async fn initialize(
http.clone(),
config.base_url().to_owned(),
));
let selector = Arc::new(
CodexCredentialSelector::new(
provider_kind.clone(),
repository.clone(),
Arc::clone(&leases),
session_affinity,
session_exclusions,
Arc::clone(&quota),
Arc::clone(&catalog),
ports.cooldowns(),
Arc::clone(&account_feedback),
CodexCookiePolicy::official().map_err(|_| OpenAiInitializeError::CookiePolicy)?,
)
.with_upstream_info(upstream_info),
);
let selector = Arc::new(CodexCredentialSelector::new(
provider_kind.clone(),
repository.clone(),
Arc::clone(&leases),
session_affinity,
session_exclusions,
Arc::clone(&quota),
Arc::clone(&catalog),
ports.cooldowns(),
Arc::clone(&account_feedback),
CodexCookiePolicy::official().map_err(|_| OpenAiInitializeError::CookiePolicy)?,
upstream_info,
));
let core_provider: Arc<dyn Provider> = Arc::new(
CodexProvider::new(
selector,
@@ -13,7 +13,7 @@ impl CodexProvider {
candidate: &ProviderCandidate,
context: AttemptContext,
) -> Result<ProviderStream, ProviderError> {
if image.payload().protocol() != PROVIDER_NAME || candidate.upstream_model().is_some() {
if image.payload().protocol() != OPENAI_PROTOCOL || candidate.upstream_model().is_some() {
return Err(provider_error(
ProviderErrorKind::InvalidRequest,
UpstreamSendState::NotSent,
@@ -112,7 +112,7 @@ impl CodexProvider {
candidate: &ProviderCandidate,
context: AttemptContext,
) -> Result<ProviderStream, ProviderError> {
if search.payload().protocol() != PROVIDER_NAME || candidate.upstream_model().is_some() {
if search.payload().protocol() != OPENAI_PROTOCOL || candidate.upstream_model().is_some() {
return Err(provider_error(
ProviderErrorKind::InvalidRequest,
UpstreamSendState::NotSent,
@@ -255,7 +255,8 @@ impl CodexProvider {
UpstreamSendState::NotSent,
));
};
if image.kind() != original.kind() || image.payload().protocol() != PROVIDER_NAME {
if image.kind() != original.kind() || image.payload().protocol() != OPENAI_PROTOCOL
{
return Err(provider_error(
ProviderErrorKind::InvalidRequest,
UpstreamSendState::NotSent,
@@ -271,7 +272,7 @@ impl CodexProvider {
}
Operation::Search(search) => {
if !matches!(&request.operation, Operation::Search(_))
|| search.payload().protocol() != PROVIDER_NAME
|| search.payload().protocol() != OPENAI_PROTOCOL
{
return Err(provider_error(
ProviderErrorKind::InvalidRequest,
@@ -330,7 +331,7 @@ impl CodexProvider {
));
// Standalone Provider 端点没有可证明的账号 owner;Images/Search metadata 必须按
// 跨账号输入收敛到当前 lease,不能沿用下游声明的账号或 installation identity
let client = self.client_for_request(&context)?;
let client = self.client_for_request(&context, lease.account())?;
let installation_id = client.profile_state().effective_installation_id(
context.client_api_key_ref(),
lease.account_id(),
@@ -538,7 +539,7 @@ pub(super) fn attach_openai_session_update(
pub(super) fn terminal_response_output(event: &ProviderEvent) -> Option<&[Value]> {
let wire = event.wire_event()?;
if wire.protocol() != PROVIDER_NAME {
if wire.protocol() != OPENAI_PROTOCOL {
return None;
}
let event_type = wire
@@ -750,7 +751,7 @@ pub(super) fn cold_json_response_stream(request: ColdJsonResponse) -> EventStrea
yield ProviderEvent::canonical(GatewayEvent::CalculatedCost(cost));
}
}
let wire = ProtocolWireEvent::raw_json(PROVIDER_NAME, response.body).map_err(|_| {
let wire = ProtocolWireEvent::raw_json(OPENAI_PROTOCOL, response.body).map_err(|_| {
provider_error(ProviderErrorKind::Protocol, UpstreamSendState::Sent)
})?;
yield ProviderEvent::wire(wire);
@@ -488,8 +488,8 @@ pub(super) fn apply_websocket_recovery_policy(
failure: &mut MappedProviderFailure,
context: WebSocketRecoveryContext<'_>,
) {
// 未匹配网关恢复规则的上游拒绝直达客户端,不能转换成传输重试或污染会话回退状态
if failure.error.retry_is_prohibited() {
// 明确禁止重试,或上游拒绝继续使用当前账号时,不能改写成同账号传输恢复
if failure.error.retry_is_prohibited() || failure.error.same_account_retry_blocked() {
return;
}
// 明确账号拒绝走已有换号路径,业务瞬态失败走请求内退避,不消耗 WS 传输预算
@@ -1504,6 +1504,10 @@ pub(super) fn map_upstream_failure(
policy,
);
}
if failure.no_more_retry {
// 保留耗尽策略供 Core 决定换号或返回错误,但不再对当前账号退避
error = error.with_same_account_retry_blocked();
}
if let Some(code) = failure.upstream_code() {
error = error.with_upstream_code(OpaqueUpstreamValue::new(code.to_owned()));
}
@@ -365,7 +365,7 @@ impl CodexProvider {
.collect::<Vec<_>>();
let events = cold_live_call_stream(ColdLiveCall {
client: self
.client_for_request(&context)?
.client_for_request(&context, lease.account())?
.for_account(lease.account())
.map_err(|error| map_client_error(error, UpstreamSendState::NotSent, None).error)?
.with_responses_api_base_url(lease.authentication().responses_api_base_url())
@@ -526,7 +526,7 @@ fn cold_live_call_stream(request: ColdLiveCall) -> EventStream {
let response_meta = ResponseMeta::for_provider_endpoint(&request_id);
yield ProviderEvent::canonical(GatewayEvent::Started(response_meta.clone()));
let wire = ProtocolWireEvent::raw_http_body(PROVIDER_NAME, response.body).map_err(|_| {
let wire = ProtocolWireEvent::raw_http_body(OPENAI_PROTOCOL, response.body).map_err(|_| {
provider_error(ProviderErrorKind::Protocol, UpstreamSendState::Sent)
})?;
yield ProviderEvent::wire(wire);
@@ -27,13 +27,13 @@ impl CodexProvider {
return Err(invalid_request());
};
let source = generate.protocol_payload().protocol();
if !matches!(source, PROVIDER_NAME | ANTHROPIC_PROTOCOL) {
if !matches!(source, OPENAI_PROTOCOL | ANTHROPIC_PROTOCOL) {
return Err(invalid_request());
}
if selected.lease.authentication().api_protocol() == ApiKeyProtocol::Messages {
if context.continuation_attempt() == ContinuationAttempt::Native
|| generate.provider_session_state(PROVIDER_NAME).is_some()
|| (source == PROVIDER_NAME
|| (source == OPENAI_PROTOCOL
&& generate
.protocol_payload()
.body()
@@ -61,7 +61,7 @@ impl CodexProvider {
};
body.insert("model".to_owned(), Value::String(model.as_str().to_owned()));
let mut source_context = generate.protocol_payload().context().clone();
if source == PROVIDER_NAME {
if source == OPENAI_PROTOCOL {
retain_target_headers(source, &mut source_context, &mut headers);
if body.get("speed").and_then(Value::as_str) == Some("fast") {
crate::compat::add_translated_fast_beta(
@@ -70,7 +70,7 @@ impl CodexProvider {
);
}
}
let translator = (source == PROVIDER_NAME)
let translator = (source == OPENAI_PROTOCOL)
.then(|| translator::MessagesResponseTranslator::new(original));
drop(operation);
let stream = self
@@ -87,13 +87,19 @@ impl CodexProvider {
let mut source_context = generate.protocol_payload().context().clone();
retain_target_headers(source, &mut source_context, &mut headers);
let translator = translator::ResponsesResponseTranslator::new(original);
let translated =
let mut translated =
messages_to_responses_request(original).map_err(|_| invalid_request())?;
if matches!(
selected.lease.authentication(),
CodexRuntimeAuthentication::OAuth(_)
) {
crate::compat::normalize_translated_codex_service_tier(&mut translated);
}
let body = serde_json::to_vec(&translated).map_err(|_| invalid_request())?;
let operation = operation
.replace_protocol_wire(None, source_context)
.map_err(|_| invalid_request())?
.translate_protocol_wire(PROVIDER_NAME, body.into())
.translate_protocol_wire(OPENAI_PROTOCOL, body.into())
.map_err(|_| invalid_request())?;
// 原始能力已在上面核对;Responses 终端只校验它实际消费的 wire 事实
if let Operation::Generate(generate) = &operation {
@@ -181,7 +187,7 @@ impl CodexProvider {
lease.capacity_snapshot(),
));
let client = self
.client_for_request(&context)?
.client_for_request(&context, lease.account())?
.for_account(lease.account())
.map_err(|error| map_client_error(error, UpstreamSendState::NotSent, None).error)?
.with_messages_api_base_url(&authentication.configuration.base_url)
@@ -21,7 +21,7 @@ impl NativeResponseTranslator for MessagesResponseTranslator {
ANTHROPIC_PROTOCOL
}
fn target_protocol(&self) -> &str {
PROVIDER_NAME
OPENAI_PROTOCOL
}
fn translate(
&mut self,
@@ -33,7 +33,7 @@ impl NativeResponseTranslator for MessagesResponseTranslator {
.stream
.complete_json_event(&message)
.map_err(|_| protocol_error())?;
return wire_events(PROVIDER_NAME, vec![event]);
return wire_events(OPENAI_PROTOCOL, vec![event]);
}
if !event.has_json_data() {
return Ok(Vec::new());
@@ -45,7 +45,7 @@ impl NativeResponseTranslator for MessagesResponseTranslator {
if event.data().get("type").and_then(Value::as_str) == Some("message_stop") {
self.stream.finish().map_err(|_| protocol_error())?;
}
wire_events(PROVIDER_NAME, events)
wire_events(OPENAI_PROTOCOL, events)
}
}
@@ -63,7 +63,7 @@ impl ResponsesResponseTranslator {
impl NativeResponseTranslator for ResponsesResponseTranslator {
fn source_protocol(&self) -> &str {
PROVIDER_NAME
OPENAI_PROTOCOL
}
fn target_protocol(&self) -> &str {
ANTHROPIC_PROTOCOL
@@ -44,6 +44,7 @@ use gateway_core::task::{
WorkerRunnable, WorkerSchedule, WorkerTaskError,
};
use gateway_core::upstream::{UpstreamSendState, UpstreamTransport};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use gateway_protocol::openai::events::{
ParsedRateLimits, parse_rate_limit_headers, rate_limits_to_header_pairs,
};
@@ -301,9 +302,14 @@ impl CodexProvider {
fn client_for_request(
&self,
context: &AttemptContext,
account: &ProviderAccount,
) -> Result<CodexBackendClient, ProviderError> {
let mut client = self.client.clone();
client.peer_route = context.extension_scope().peer_route().cloned();
client.peer_route = context
.extension_scope()
.peer_route()
.filter(|_| self.selector.is_peer(account))
.cloned();
let Some(profile) = context.request_profile() else {
return Ok(client);
};
@@ -502,13 +508,11 @@ impl Provider for CodexProvider {
let Ok(encoded) = encode_generate_request(request, "observability", None) else {
return ProviderRequestObservation::default();
};
let previous_session = decode_openai_session_state(request);
let semantics = encoded.semantics(
previous_session
.as_ref()
.and_then(|state| state.reasoning_effort_override.as_deref()),
);
let reasoning_effort = semantics.reasoning_effort.clone();
let previous_reasoning_effort_override = request
.provider_session_state(PROVIDER_NAME)
.filter(|state| state.extension_owner().is_none())
.and_then(|_| decode_openai_session_state(request))
.and_then(|state| state.reasoning_effort_override);
let previous_response_id = encoded.previous_response_id();
let continuation = ContinuationRequestObservation {
affinity_hash: derive_codex_session_affinity(&encoded, client_api_key_id)
@@ -520,11 +524,7 @@ impl Provider for CodexProvider {
};
ProviderRequestObservation {
requested_model: None,
reasoning_effort,
reasoning_preset: semantics.reasoning_preset.map(str::to_owned),
request_kind: semantics.request_kind,
subagent_kind: semantics.subagent_kind,
compact: semantics.compact,
previous_reasoning_effort_override,
continuation,
}
}
@@ -635,7 +635,7 @@ impl Provider for CodexProvider {
let mut operation = Operation::Generate(generate.clone());
if context.fast_mode() != gateway_core::account::FastMode::Default
&& let Operation::Generate(generate) = &operation
&& generate.protocol_payload().protocol() == PROVIDER_NAME
&& generate.protocol_payload().protocol() == OPENAI_PROTOCOL
{
let mut request =
CodexResponsesRequest::from_body(generate.protocol_payload().body().clone());
@@ -653,7 +653,7 @@ impl Provider for CodexProvider {
)
})?;
operation = operation
.replace_middleware_wire(PROVIDER_NAME, body.into())
.replace_middleware_wire(OPENAI_PROTOCOL, body.into())
.map_err(|_| {
provider_error(
ProviderErrorKind::InvalidRequest,
@@ -676,7 +676,7 @@ impl Provider for CodexProvider {
}
// 其他协议必须先取得真实账号,再按固定 attempt 阶段调用转换器;选号前不能
// 把未知正文当成 OpenAI wire 解释会话、亲和或传输字段
let upstream = (generate.protocol_payload().protocol() == PROVIDER_NAME)
let upstream = (generate.protocol_payload().protocol() == OPENAI_PROTOCOL)
.then(|| encode_generate_request(generate, upstream_model.as_str(), None))
.transpose()
.map_err(map_request_error)?;
@@ -843,7 +843,7 @@ impl CodexProvider {
UpstreamSendState::NotSent,
));
};
if generate.protocol_payload().protocol() != PROVIDER_NAME
if generate.protocol_payload().protocol() != OPENAI_PROTOCOL
|| generate.wire_capability_requirements() != frozen_requirements
{
return Err(provider_error(
@@ -1000,7 +1000,7 @@ impl CodexProvider {
.get(context.client_api_key_ref(), lease.account(), response_id)
.map(|affinity| (response_id.to_owned(), affinity))
});
let client = self.client_for_request(&context)?;
let client = self.client_for_request(&context, lease.account())?;
let installation_id = client.profile_state().effective_installation_id(
context.client_api_key_ref(),
lease.account_id(),
@@ -5,7 +5,7 @@ use super::*;
pub(super) fn endpoint_requested_model(
payload: &gateway_core::operation::RawJsonPayload,
) -> Option<gateway_core::routing::PublicModelId> {
if payload.protocol() != PROVIDER_NAME {
if payload.protocol() != OPENAI_PROTOCOL {
return None;
}
// 选号与观测共用模型提取,跳过编辑请求中的图片且不改写原始正文
@@ -43,7 +43,7 @@ impl CodexProvider {
}
let requested_service_tier = match &operation {
Operation::Generate(generate)
if generate.protocol_payload().protocol() == PROVIDER_NAME =>
if generate.protocol_payload().protocol() == OPENAI_PROTOCOL =>
{
let request =
CodexResponsesRequest::from_body(generate.protocol_payload().body().clone());
@@ -14,6 +14,7 @@ use gateway_core::event::{
};
use gateway_core::metering::Usage;
use gateway_core::upstream::UpstreamSendState;
use gateway_protocol::openai::OPENAI_PROTOCOL;
use gateway_protocol::openai::events::{TokenUsage, billable_usage_is_complete, extract_usage};
use gateway_protocol::openai::sse::{SseEvent, SseEventDecoder, SseFrame, sse_frame_is_done};
use serde_json::Value;
@@ -301,7 +302,7 @@ impl CodexCanonicalDecoder {
}
let raw = Bytes::from(raw);
if events.is_empty() {
if let Ok(wire) = ProtocolWireEvent::raw_sse("openai", raw) {
if let Ok(wire) = ProtocolWireEvent::raw_sse(OPENAI_PROTOCOL, raw) {
output.push(ProviderEvent::wire(wire));
}
continue;
@@ -346,10 +347,10 @@ impl CodexCanonicalDecoder {
);
let wire = match raw_sse_frame {
Some(RawResponseFrame::Sse(raw)) => {
ProtocolWireEvent::raw_sse("openai", raw).ok()
ProtocolWireEvent::raw_sse(OPENAI_PROTOCOL, raw).ok()
}
Some(RawResponseFrame::WebSocket(raw)) => {
ProtocolWireEvent::raw_websocket("openai", raw).ok()
ProtocolWireEvent::raw_websocket(OPENAI_PROTOCOL, raw).ok()
}
None => None,
};
@@ -474,12 +475,12 @@ impl CodexCanonicalDecoder {
let sse_id = event.id;
match raw_sse_frame {
Some(RawResponseFrame::WebSocket(raw)) => {
ProtocolWireEvent::json("openai", event_type, value)
ProtocolWireEvent::json(OPENAI_PROTOCOL, event_type, value)
.map(|wire| wire.with_raw_websocket_message(raw))
}
Some(RawResponseFrame::Sse(raw_sse_frame)) => {
ProtocolWireEvent::json_with_raw_sse_metadata(
"openai",
OPENAI_PROTOCOL,
event_type,
value,
raw_sse_frame,
@@ -488,7 +489,7 @@ impl CodexCanonicalDecoder {
)
}
None => ProtocolWireEvent::json_with_sse_metadata(
"openai",
OPENAI_PROTOCOL,
event_type,
value,
sse_id,
@@ -7,6 +7,7 @@ use bytes::Bytes;
use gateway_core::operation::RawJsonPayload;
use gateway_core::routing::{ModelServiceTier, UpstreamModelId};
use gateway_protocol::anthropic::MessagesModelCapabilities;
use gateway_protocol::openai::OPENAI_PROTOCOL;
use reqwest::header::{ETAG, HeaderMap};
use serde::Deserialize;
@@ -697,7 +698,7 @@ pub(super) fn parse_api_model_catalog(
.unwrap_or_else(|| request_model.as_str().to_owned());
validate_public_text(&display_name, MAX_DISPLAY_NAME_BYTES, false)?;
let document = RawJsonPayload::new(
"openai",
OPENAI_PROTOCOL,
Bytes::from(
serde_json::to_vec(&serde_json::json!({"id": request_model.as_str()}))
.map_err(|_| CodexModelCatalogError::InvalidWire)?,
@@ -10,6 +10,7 @@ use super::{
},
diagnostics::CodexUpstreamSendPhase,
endpoints::endpoint_url,
profile::CodexWireProfile,
response_meta,
};
@@ -20,9 +21,24 @@ impl CodexBackendClient {
context: CodexRequestContext<'_>,
client_version: Option<&str>,
) -> CodexClientResult<CodexModelCatalogSnapshot> {
let (body, etag) = self
.fetch_models_page(context, client_version, None)
.await?;
let profile = self.profile.snapshot();
let query = match self.protocol {
OpenAiUpstreamProtocol::Codex | OpenAiUpstreamProtocol::ResponsesApi => (
"client_version",
client_version.unwrap_or(&profile.codex_version),
),
OpenAiUpstreamProtocol::MessagesApi => ("limit", "1000"),
};
let (body, etag) = match self.fetch_models_page(context, &profile, &[query]).await {
Err(CodexClientError::Upstream {
status, ref body, ..
}) if self.protocol == OpenAiUpstreamProtocol::ResponsesApi
&& rejects_catalog_version(status, body) =>
{
self.fetch_models_page(context, &profile, &[]).await?
}
result => result?,
};
match self.protocol {
OpenAiUpstreamProtocol::Codex => Ok(parse_codex_model_catalog(&body, etag.as_deref())?),
OpenAiUpstreamProtocol::ResponsesApi => Ok(super::catalog::parse_api_model_catalog(
@@ -56,7 +72,9 @@ impl CodexBackendClient {
if !cursors.insert(cursor.to_owned()) {
return Err(super::CodexModelCatalogError::InvalidWire.into());
}
(page, _) = self.fetch_models_page(context, None, Some(cursor)).await?;
(page, _) = self
.fetch_models_page(context, &profile, &[query, ("after_id", cursor)])
.await?;
}
let body = serde_json::to_vec(&serde_json::json!({"data":models}))
.map_err(|_| super::CodexModelCatalogError::InvalidWire)?;
@@ -72,32 +90,20 @@ impl CodexBackendClient {
async fn fetch_models_page(
&self,
context: CodexRequestContext<'_>,
client_version: Option<&str>,
after_id: Option<&str>,
profile: &CodexWireProfile,
query: &[(&str, &str)],
) -> CodexClientResult<(bytes::Bytes, Option<String>)> {
let path = match self.protocol {
OpenAiUpstreamProtocol::Codex => "codex/models",
OpenAiUpstreamProtocol::ResponsesApi | OpenAiUpstreamProtocol::MessagesApi => "models",
};
let profile = self.profile.snapshot();
let headers = self.model_request_headers(&profile, context)?;
let headers = self.model_request_headers(profile, context)?;
let mut request = self
.client
.get(endpoint_url(&self.base_url, path))
.headers(headers);
let client_version = client_version.or_else(|| {
(self.protocol == OpenAiUpstreamProtocol::Codex)
.then_some(profile.codex_version.as_str())
});
if self.protocol == OpenAiUpstreamProtocol::MessagesApi {
request = request.query(&[("limit", "1000")]);
if let Some(after_id) = after_id {
request = request.query(&[("after_id", after_id)]);
}
} else if let Some(client_version) = client_version {
request = request.query(&[("client_version", client_version)]);
}
let mut request = request.build()?;
.headers(headers)
.query(query)
.build()?;
if let Some(route) = self.route_to(request.url()) {
request
.headers_mut()
@@ -134,3 +140,19 @@ impl CodexBackendClient {
Ok((body, etag))
}
}
/// 仅在上游明确不接受版本参数时读取普通目录,其他失败保留原错误
fn rejects_catalog_version(status: reqwest::StatusCode, body: &str) -> bool {
if !matches!(status.as_u16(), 400 | 422) {
return false;
}
let Ok(value) = serde_json::from_str::<serde_json::Value>(body) else {
return false;
};
let error = &value["error"];
error["param"].as_str() == Some("client_version")
&& matches!(
error["code"].as_str(),
Some("unknown_parameter" | "unsupported_parameter")
)
}
@@ -79,6 +79,8 @@ pub struct CodexUpstreamFailure {
/// 官方 `usage_limit_reached` 错误体携带的窗口重置时间(Unix 秒)
pub(crate) usage_limit_resets_at: Option<i64>,
pub(crate) retry_after_seconds: Option<u64>,
/// 上游错误头声明不再重试当前账号
pub(crate) no_more_retry: bool,
pub(crate) request_id: Option<String>,
pub(crate) response_origin: Option<url::Url>,
pub(crate) set_cookie_headers: Vec<String>,
@@ -127,6 +129,7 @@ impl CodexUpstreamFailure {
.then_some(fields.resets_at)
.flatten(),
retry_after_seconds,
no_more_retry: gateway_protocol::openai::events::upstream_no_more_retry_from_body(body),
request_id: diagnostics.request_id.clone(),
response_origin: diagnostics.response_origin.clone(),
set_cookie_headers: set_cookie_headers.to_vec(),
@@ -179,6 +182,7 @@ impl CodexUpstreamFailure {
identity_error_code: diagnostics.identity_error_code.clone(),
usage_limit_resets_at: None,
retry_after_seconds: failure.retry_after_seconds,
no_more_retry: failure.no_more_retry,
request_id: failure
.request_id
.clone()
@@ -26,7 +26,7 @@ impl CodexBackendClient {
self
}
// 官方端点不参与节点转发,按插件准备后的实际目标决定是否发送
// 已识别的互联账号才携带路径,官方端点不参与节点转发
pub(super) fn route_to(&self, target: &Url) -> Option<&http::HeaderValue> {
self.peer_route.as_ref().filter(|_| {
self.protocol != super::client::OpenAiUpstreamProtocol::Codex
@@ -40,20 +40,15 @@ impl CodexBackendClient {
pub(super) async fn prepare_http_request(
&self,
endpoint: String,
headers: HeaderMap,
mut headers: HeaderMap,
body: Bytes,
) -> CodexClientResult<reqwest::Request> {
let mut request = self.prepare_http_outbound(endpoint, headers, body).await?;
let path = self.route_to(request.url());
request
.headers_mut()
.remove(gateway_protocol::peer::ROUTE_HEADER);
if let Some(path) = path {
request
.headers_mut()
.insert(gateway_protocol::peer::ROUTE_HEADER, path.clone());
let target = Url::parse(&endpoint).map_err(|_| invalid_request())?;
headers.remove(gateway_protocol::peer::ROUTE_HEADER);
if let Some(path) = self.route_to(&target) {
headers.insert(gateway_protocol::peer::ROUTE_HEADER, path.clone());
}
Ok(request)
self.prepare_http_outbound(endpoint, headers, body).await
}
async fn prepare_http_outbound(
@@ -109,9 +104,8 @@ impl CodexBackendClient {
pub(super) async fn prepare_websocket_request(
&self,
request: CodexWebSocketRequest,
mut request: CodexWebSocketRequest,
) -> CodexClientResult<(CodexWebSocketRequest, Option<String>)> {
let (mut request, profile) = self.prepare_websocket_outbound(request).await?;
let path = if self.peer_route.is_some()
&& self.protocol != super::client::OpenAiUpstreamProtocol::Codex
{
@@ -125,7 +119,7 @@ impl CodexBackendClient {
None
};
request.connection.set_peer_route(path);
Ok((request, profile))
self.prepare_websocket_outbound(request).await
}
async fn prepare_websocket_outbound(
@@ -496,6 +496,8 @@ pub struct ResponsesSseFailure {
pub explicit_status_code: Option<u16>,
/// 上游显式重试间隔,或从官方限流消息中解析出的重试间隔
pub retry_after_seconds: Option<u64>,
/// 上游 `x-retry-metadata: NO_MORE_RETRY`,与重试间隔独立
pub no_more_retry: bool,
/// 当前错误事件自身携带的请求 ID;不是连接 opening ID
pub(crate) request_id: Option<String>,
/// 上游错误事件的原始 JSON data;只应在明确的失败审计边界读取
@@ -512,6 +514,7 @@ impl fmt::Debug for ResponsesSseFailure {
.field("has_upstream_type", &self.upstream_type.is_some())
.field("explicit_status_code", &self.explicit_status_code)
.field("retry_after_seconds", &self.retry_after_seconds)
.field("no_more_retry", &self.no_more_retry)
.finish()
}
}
@@ -531,6 +534,7 @@ impl ResponsesSseFailure {
upstream_type: failure_type(value),
explicit_status_code: failure_explicit_status_code(value),
retry_after_seconds: events::retry_after_seconds_from_value(value),
no_more_retry: events::upstream_no_more_retry(value),
request_id:
crate::transport::diagnostics::CodexUpstreamDiagnostics::error_event_request_id(
value,
@@ -768,19 +772,11 @@ impl CodexResponsesRequest {
&self,
previous_reasoning_effort_override: Option<&str>,
) -> CodexRequestSemantics {
let mut semantics = codex_responses_request_semantics_with_turn_metadata(
codex_responses_request_semantics_with_turn_metadata(
self.body(),
self.turn_metadata.as_deref(),
previous_reasoning_effort_override,
);
// 官方 generate=false 只准备连接与上下文,不是模型推理
// 预热分类参与用量筛选,不能仅凭客户端的 request_kind 提示排除真实推理
if !self.generate() {
semantics.request_kind = Some("prewarm".to_owned());
} else if semantics.request_kind.as_deref() == Some("prewarm") {
semantics.request_kind = None;
}
semantics
)
}
/// 返回请求语义与 child transport 隔离所用的子代理区分值
@@ -5,7 +5,8 @@ use std::io;
use base64::{Engine as _, engine::general_purpose::STANDARD};
use gateway_core::operation::GenerateRequest;
use gateway_protocol::openai::{
WS_REQUEST_HEADER_RESPONSES_LITE_CLIENT_METADATA_KEY, is_transport_managed_request_header,
OPENAI_PROTOCOL, WS_REQUEST_HEADER_RESPONSES_LITE_CLIENT_METADATA_KEY,
is_transport_managed_request_header,
};
use reqwest::header::{HeaderMap, HeaderName, HeaderValue};
use roxmltree::Document;
@@ -108,7 +109,7 @@ pub fn encode_generate_request(
location: Option<&CodexRequestLocation>,
) -> Result<CodexResponsesRequest, CodexRequestEncodeError> {
let payload = request.protocol_payload();
if payload.protocol() != "openai" {
if payload.protocol() != OPENAI_PROTOCOL {
return Err(CodexRequestEncodeError::InvalidProtocolPayload);
}
let mut body = payload.body().clone();
@@ -709,7 +709,7 @@ async fn reset_credit_explicit_http_rejection_should_preserve_the_raw_body() {
}
#[tokio::test]
async fn openai_core_provider_projects_codex_request_observation_without_routing_side_effects() {
async fn openai_core_provider_projects_private_request_observation_without_execution() {
let config = valid_config();
let bundle = provider_openai::initialize(
config.config.clone(),
@@ -723,14 +723,11 @@ async fn openai_core_provider_projects_codex_request_observation_without_routing
Map::from_iter([
("model".to_owned(), json!("gpt-5.4")),
("input".to_owned(), json!("summarize")),
("reasoning".to_owned(), json!({"effort": "high"})),
("session_id".to_owned(), json!("observation-session")),
("previous_response_id".to_owned(), json!("resp_previous")),
]),
)
.expect("OpenAI payload")
.with_context(Map::from_iter([(
"turn_metadata".to_owned(),
Value::String(r#"{"request_kind":"compaction","subagent_kind":"review"}"#.to_owned()),
)]));
.expect("OpenAI payload");
let operation = Operation::Generate(GenerateRequest::from_protocol_payload(payload));
let client_key_id = ClientApiKeyId::new("key_openai_admin_observation").expect("client key");
@@ -738,11 +735,9 @@ async fn openai_core_provider_projects_codex_request_observation_without_routing
.core_provider()
.request_observation(&operation, &client_key_id);
assert_eq!(observation.request_kind.as_deref(), Some("compaction"));
assert_eq!(observation.subagent_kind.as_deref(), Some("review"));
// Codex 当前只在特定多代理预设组合下给出 reasoning_preset;普通 high 保持空值
assert_eq!(observation.reasoning_preset, None);
assert!(observation.compact);
assert!(observation.continuation.requested);
assert!(observation.continuation.previous_response_id_hash.is_some());
assert!(observation.continuation.affinity_hash.is_some());
}
#[tokio::test]
@@ -15,7 +15,7 @@ use provider_openai::credential::{
CodexCredentialCatalogError, CodexCredentialCatalogService, ImportCodexOAuthCredential,
};
use provider_openai::transport::profile::{CodexWireProfile, CodexWireProfileState};
use wiremock::matchers::{header, method, path};
use wiremock::matchers::{header, method, path, query_param};
use wiremock::{Mock, MockServer, Request, Respond, ResponseTemplate};
use crate::support::{MemoryAccountStore, catalog_cache, profile, secret};
@@ -778,7 +778,7 @@ async fn api_key_catalogs_are_isolated_and_join_oauth_without_claiming_native_me
}
#[tokio::test]
async fn api_key_native_catalog_preserves_unknown_reasoning_and_background_entitlements() {
async fn api_key_background_catalog_preserves_negotiated_reasoning_levels_and_entitlements() {
use gateway_core::routing::ProviderModelContent;
use provider_openai::credential::ResponsesTransport;
use provider_openai::transport::CodexCatalogCapabilityEvidence;
@@ -790,12 +790,25 @@ async fn api_key_native_catalog_preserves_unknown_reasoning_and_background_entit
.seed_api_key("acct_api", upstream.uri(), ResponsesTransport::Http)
.await;
let original = json!({"slug":"z-unknown-reasoning", "display_name":"Unknown reasoning"});
let second = json!({"slug":"vendor/model", "display_name":"Second upstream model"});
let second = json!({
"slug":"vendor/model", "display_name":"Second upstream model",
"default_reasoning_level":"high",
"supported_reasoning_levels":[{"effort":"high","description":"Upstream description"}]
});
Mock::given(method("GET"))
.and(path("/models"))
.respond_with(
ResponseTemplate::new(200).set_body_json(json!({"models":[original, second]})),
)
.respond_with({
let native = json!({"models":[original, second]});
move |request: &Request| {
if request.url.query() == Some("client_version=0.144.0") {
ResponseTemplate::new(200).set_body_json(&native)
} else {
ResponseTemplate::new(200).set_body_json(json!({"data":[
{"id":"z-unknown-reasoning"}, {"id":"vendor/model"}
]}))
}
}
})
.expect(1)
.mount(&upstream)
.await;
@@ -824,16 +837,69 @@ async fn api_key_native_catalog_preserves_unknown_reasoning_and_background_entit
.collect::<Vec<_>>(),
["z-unknown-reasoning", "vendor/model"]
);
let ProviderModelContent::Native(payload) = &models[0].content else {
panic!("native model expected")
};
assert_eq!(
serde_json::from_slice::<serde_json::Value>(payload.body()).unwrap(),
original
);
for (model, expected) in models.iter().zip([original, second]) {
let ProviderModelContent::Native(payload) = &model.content else {
panic!("native model expected")
};
assert_eq!(
serde_json::from_slice::<serde_json::Value>(payload.body()).unwrap(),
expected
);
}
upstream.verify().await;
}
#[tokio::test]
async fn api_key_catalog_retries_only_explicit_version_parameter_rejections() {
use provider_openai::credential::ResponsesTransport;
use serde_json::json;
for (status, code, param, retry) in [
(400, "unknown_parameter", "client_version", true),
(422, "unsupported_parameter", "client_version", true),
(400, "unknown_parameter", "other", false),
(400, "invalid_value", "client_version", false),
(429, "unsupported_parameter", "client_version", false),
] {
let upstream = MockServer::start().await;
let store = Arc::new(MemoryAccountStore::default());
store
.seed_api_key("acct_api", upstream.uri(), ResponsesTransport::Http)
.await;
Mock::given(method("GET"))
.and(path("/models"))
.and(query_param("client_version", "0.144.0"))
.respond_with(
ResponseTemplate::new(status)
.set_body_json(json!({"error":{"code":code,"param":param}})),
)
.expect(1)
.mount(&upstream)
.await;
Mock::given(method("GET"))
.and(path("/models"))
.and(|request: &Request| request.url.query().is_none())
.respond_with(
ResponseTemplate::new(200).set_body_json(json!({"data":[{"id":"api-model"}]})),
)
.expect(u64::from(retry))
.mount(&upstream)
.await;
let service = service_with_catalog_cache(&store, upstream.uri(), catalog_cache());
let account = store.account("acct_api").unwrap();
let result = service.refresh_account_catalog(account.id()).await;
if retry {
assert_eq!(result.unwrap().models(), ["api-model"]);
} else {
assert!(matches!(
result,
Err(CodexCredentialCatalogError::Upstream { .. })
));
}
upstream.verify().await;
}
}
#[tokio::test]
async fn api_key_client_catalog_distinguishes_filtered_empty_from_fetch_failure() {
use gateway_core::account::AccountModelAccess;
@@ -617,3 +617,133 @@ async fn all_filtered_is_terminal_before_queue_or_lease_even_for_native_pins() {
assert!(leases.requests.lock().unwrap().is_empty());
}
}
#[tokio::test]
async fn round_robin_counts_concurrent_first_requests_as_one_session() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_first", "test-first");
create_account(&store, "acct_second", "test-second");
let leases = Arc::new(TestLeaseCoordinator::default());
let affinity = Arc::new(MemorySessionAffinity::with_initial_claim_barrier(2));
let selector = selector_with_affinity(&store, leases, affinity.clone());
let url = Url::parse(OFFICIAL_CODEX_BASE_URL).unwrap();
let mut winners = Vec::new();
for key in ["session-1", "session-2"] {
let (a, b) = tokio::join!(biased;
select_strict_round_robin_session(&selector, &url, key),
select_strict_round_robin_session(&selector, &url, key),
);
assert_eq!(a, b);
winners.push(a);
}
assert_eq!(affinity.binding_count(), 2);
assert_ne!(winners[0], winners[1]);
}
#[tokio::test]
async fn busy_leases_do_not_consume_the_next_round_robin_assignment() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_first", "test-first");
create_account(&store, "acct_second", "test-second");
let leases = Arc::new(TestLeaseCoordinator::default());
let selector = selector(&store, leases.clone());
let url = Url::parse(OFFICIAL_CODEX_BASE_URL).unwrap();
let attempt = round_robin_attempt();
let key = ProviderSessionAffinityKey::try_new("busy-session").unwrap();
*leases.busy.lock().unwrap() = true;
let result = selector
.select(&SelectCodexCredential {
upstream_model: "gpt-5.4",
request_url: &url,
attempt: &attempt,
session_affinity_key: Some(&key),
})
.await;
assert!(
matches!(
result,
Err(CredentialSelectionError::CapacityUnavailable { .. })
),
"{result:?}"
);
*leases.busy.lock().unwrap() = false;
assert_eq!(
select_strict_round_robin_session(&selector, &url, "available").await,
"acct_first"
);
}
#[tokio::test]
async fn round_robin_counts_delegated_assignments_and_skips_explicit_picks() {
for decision in [
AccountScheduleDecision::Delegate,
AccountScheduleDecision::DelegateFiltered(BTreeSet::from([ProviderAccountId::new(
"acct_second",
)
.unwrap()])),
] {
let store = Arc::new(MemoryAccountStore::default());
for id in ["acct_first", "acct_second", "acct_fallback"] {
create_account(&store, id, id);
}
let selector = selector(&store, Arc::new(TestLeaseCoordinator::default()));
let attempt_for = |decision| {
capacity_attempt(
AccountSelectionPolicy::new(
RotationStrategy::RoundRobin,
NonZeroU32::new(2).unwrap(),
),
Some(RequestPolicyContext::new(
Arc::new(Scheduler { decision }),
ExtensionSetReference::new(
ExtensionSetId::new("rr-policy".into()).unwrap(),
Arc::new(ExtensionLease),
),
ModelRequestId::new("req_capacity_contract").unwrap(),
ClientApiKeyId::new("key_codex_contract").unwrap(),
vec![],
)),
)
};
let url = Url::parse(OFFICIAL_CODEX_BASE_URL).unwrap();
let picked = attempt_for(AccountScheduleDecision::Pick(
ProviderAccountId::new("acct_second").unwrap(),
));
let delegated = attempt_for(decision.clone());
let mut actual = Vec::new();
for (session, attempt) in [
("pick", &picked),
("one", &delegated),
("one", &delegated),
("two", &delegated),
("three", &delegated),
] {
let key = ProviderSessionAffinityKey::try_new(session).unwrap();
let selected = selector
.select(&SelectCodexCredential {
upstream_model: "gpt-5.4",
request_url: &url,
attempt,
session_affinity_key: Some(&key),
})
.await
.unwrap();
actual.push(selected.account_id().as_str().to_owned());
}
let third = if matches!(decision, AccountScheduleDecision::Delegate) {
"acct_second"
} else {
"acct_fallback"
};
assert_eq!(
actual,
[
"acct_second",
"acct_fallback",
"acct_fallback",
"acct_first",
third
]
);
}
}
@@ -11,11 +11,11 @@ use std::time::{Duration, SystemTime};
use futures::executor::block_on;
use gateway_core::account::{
AccountAttemptFeedback, AccountConcurrencyLimit, AccountErrorReason, AccountFeedbackStats,
AccountSelectionPolicy, AccountStateChange, AccountWeight, CredentialState, OpaqueProviderData,
ProviderAccount, ProviderAccountId, ProviderAccountStore as _, QuotaAccessChange,
QuotaAccessState, QuotaEvidence, QuotaObservation, QuotaState, QuotaWriteOutcome,
RotationStrategy,
AccountAffinity, AccountAttemptFeedback, AccountConcurrencyLimit, AccountErrorReason,
AccountFeedbackStats, AccountSelectionPolicy, AccountStateChange, AccountWeight,
CredentialState, OpaqueProviderData, ProviderAccount, ProviderAccountId,
ProviderAccountStore as _, QuotaAccessChange, QuotaAccessState, QuotaEvidence,
QuotaObservation, QuotaState, QuotaWriteOutcome, RotationStrategy,
};
use gateway_core::engine::continuation::{
ContinuationBinding, NativeContinuationPin, PreviousResponseId,
@@ -263,6 +263,7 @@ fn selector_with_runtime(
cooldowns,
account_feedback,
CodexCookiePolicy::official().expect("official cookie policy"),
store.clone(),
)
}
@@ -795,8 +796,8 @@ fn selector_does_not_retry_invalid_credential_data() {
assert!(leases.requests.lock().expect("lease requests").is_empty());
}
#[test]
fn selector_round_robin_cursor_advances_across_requests() {
#[tokio::test]
async fn selector_round_robin_cursor_advances_across_requests() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_first", "at-first");
create_account(&store, "acct_second", "at-second");
@@ -807,13 +808,15 @@ fn selector_round_robin_cursor_advances_across_requests() {
for _ in 0..4 {
let attempt = round_robin_attempt();
let lease = block_on(selector.select(&SelectCodexCredential {
upstream_model: "gpt-5.4",
request_url: &request_url,
attempt: &attempt,
session_affinity_key: None,
}))
.expect("select round robin account");
let lease = selector
.select(&SelectCodexCredential {
upstream_model: "gpt-5.4",
request_url: &request_url,
attempt: &attempt,
session_affinity_key: None,
})
.await
.expect("select round robin account");
selected.push(lease.account_id().as_str().to_owned());
}
@@ -823,6 +826,86 @@ fn selector_round_robin_cursor_advances_across_requests() {
);
}
#[tokio::test]
async fn strict_round_robin_rotates_each_new_session_across_eligible_accounts() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_fallback", "at-fallback");
create_account(&store, "acct_first", "at-first");
create_account(&store, "acct_second", "at-second");
let leases = Arc::new(TestLeaseCoordinator::default());
let selector = selector_with_affinity(
&store,
Arc::clone(&leases),
Arc::new(MemorySessionAffinity::default()),
);
let request_url =
Url::parse("https://chatgpt.com/backend-api/codex/responses").expect("request URL");
assert_eq!(
select_strict_round_robin_session(&selector, &request_url, "session-1").await,
"acct_fallback"
);
for _ in 0..5 {
assert_eq!(
select_strict_round_robin_session(&selector, &request_url, "session-1").await,
"acct_fallback"
);
}
assert_eq!(
select_strict_round_robin_session(&selector, &request_url, "session-2").await,
"acct_first"
);
assert_eq!(
select_strict_round_robin_session(&selector, &request_url, "session-1").await,
"acct_fallback"
);
assert_eq!(
select_strict_round_robin_session(&selector, &request_url, "session-3").await,
"acct_second"
);
assert_eq!(
select_strict_round_robin_session(&selector, &request_url, "session-4").await,
"acct_fallback"
);
}
async fn select_strict_round_robin_session(
selector: &CodexCredentialSelector,
request_url: &Url,
key: &str,
) -> String {
let attempt = AttemptContext::new(
RequestAttemptContext::new(
ModelRequestId::new("req_codex_round_robin_session").expect("request id"),
ClientApiKeyId::new("key_codex_contract").expect("client key id"),
),
NonZeroU32::new(1).expect("attempt"),
SystemTime::now() + Duration::from_secs(30),
AccountSelectionPolicy::new(
RotationStrategy::RoundRobin,
NonZeroU32::new(2).expect("concurrency"),
)
.with_openai_account_affinity(AccountAffinity::Strict),
AccountAttemptContext::new(BTreeSet::new(), None, None)
.with_account_scope(contract_account_scope()),
None,
CancellationToken::new(),
);
let affinity_key = ProviderSessionAffinityKey::try_new(key).expect("affinity key");
selector
.select(&SelectCodexCredential {
upstream_model: "gpt-5.4",
request_url,
attempt: &attempt,
session_affinity_key: Some(&affinity_key),
})
.await
.expect("select round robin session")
.account_id()
.as_str()
.to_owned()
}
#[tokio::test]
async fn selector_should_claim_the_initial_session_account_before_upstream_send() {
let store = Arc::new(MemoryAccountStore::default());
@@ -149,6 +149,41 @@ async fn cache_preference_storage_timeouts_follow_the_affinity_mode() {
}
}
#[tokio::test]
async fn round_robin_retains_the_assignment_when_a_cache_preference_cannot_be_saved() {
use gateway_core::account::{AccountAffinity, AccountSelectionPolicy, RotationStrategy};
use gateway_core::provider_ports::ProviderSessionAffinityPort;
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_affinity").await;
let affinity = Arc::new(MemorySessionAffinity::with_initial_claim_barrier(2));
let server = MockServer::start().await;
let provider = provider_with_affinity_and_base_url(&store, affinity.clone(), server.uri());
let attempt = session_binding::context_with_affinity_policy(
"req_cache_rr",
contract_account_scope(),
AccountSelectionPolicy::new(RotationStrategy::RoundRobin, NonZeroU32::MIN)
.with_openai_account_affinity(AccountAffinity::Preferred),
);
let key = attempt.client_api_key_ref().clone();
let result = provider
.execute(planned_request("openai", generate_operation()), attempt)
.await
.unwrap();
assert_eq!(
result.metadata().provider_account_id().as_str(),
"acct_affinity"
);
assert_eq!(affinity.binding_count(), 0);
assert_eq!(
affinity
.round_robin_cursor(&ProviderKind::new("openai").unwrap(), &key)
.await
.unwrap(),
1
);
}
#[tokio::test]
async fn strict_cache_preference_waits_and_preserves_binding_on_rejected_waits() {
use gateway_core::concurrency::ConcurrencyQueuePolicy;
@@ -35,6 +35,7 @@ fn cookie_provider(store: &Arc<MemoryAccountStore>, base_url: String) -> Arc<Cod
Arc::clone(&feedback),
// 测试仅允许本机模拟上游,生产 Cookie 白名单不放宽
CodexCookiePolicy::new(["cf_clearance"], ["127.0.0.1"]).unwrap(),
store.clone(),
));
Arc::new(
CodexProvider::new(
@@ -29,6 +29,7 @@ async fn seed(store: &Arc<MemoryAccountStore>, server: &MockServer, protocol: Ap
fn messages_operation(stream: bool) -> Operation {
let body = json!({
"model":"gpt-5.4", "max_tokens":128, "stream":stream,
"thinking":{"type":"adaptive"}, "output_config":{"effort":"xhigh"},
"messages":[
{"role":"assistant","content":[{"type":"thinking","thinking":"prior thought","signature":"real-native-signature"},{"type":"text","text":"before"}]},
{"role":"user","content":"hello"},
@@ -174,7 +175,8 @@ async fn native_messages_preserve_json_sse_headers_and_real_signatures() {
.await;
let operation = messages_operation(streaming);
let original = operation.middleware_body().unwrap();
let mut stream = provider(&store)
let provider = provider(&store);
let mut stream = provider
.execute(
planned_request("openai", operation),
context("req_messages_native", CancellationToken::new()),
@@ -290,6 +292,8 @@ async fn messages_and_responses_translate_with_target_header_boundaries() {
assert!(!headers.contains_key(name));
}
} else {
let body: Value = serde_json::from_slice(&requests[0].body).unwrap();
assert_eq!(body["reasoning"]["effort"], "xhigh");
for name in [
"anthropic-version",
"anthropic-beta",
@@ -302,6 +306,71 @@ async fn messages_and_responses_translate_with_target_header_boundaries() {
}
}
#[tokio::test]
async fn translated_messages_service_tiers_follow_the_selected_responses_upstream() {
for api_key in [false, true] {
for (options, oauth_tier, api_tier) in [
(json!({"service_tier":"auto"}), None, Some("auto")),
(
json!({"service_tier":"standard_only"}),
None,
Some("default"),
),
(json!({"speed":"fast"}), Some("priority"), Some("priority")),
] {
let server = MockServer::start().await;
let store = Arc::new(MemoryAccountStore::default());
if api_key {
seed(&store, &server, ApiKeyProtocol::Responses).await;
} else {
create_account(&store, "acct_provider_contract").await;
}
Mock::given(method("POST"))
.and(path(if api_key {
"/responses"
} else {
"/codex/responses"
}))
.respond_with(
ResponseTemplate::new(200)
.set_body_raw(CAPTURE_COMPLETED_SSE, "text/event-stream"),
)
.expect(1)
.mount(&server)
.await;
let mut body = json!({"model":"gpt-5.4","max_tokens":128,"stream":true,"messages":[{"role":"user","content":"hello"}]});
body.as_object_mut()
.unwrap()
.extend(options.as_object().unwrap().clone());
let operation = Operation::Generate(GenerateRequest::from_protocol_payload(
ProtocolPayload::json_object("anthropic", body.as_object().unwrap().clone())
.unwrap()
.with_context(Map::from_iter([("use_websocket".to_owned(), json!(false))])),
));
let mut stream = provider_with_base_url(&store, server.uri())
.execute(
planned_request("openai", operation),
context("req_translated_tier", CancellationToken::new()),
)
.await
.unwrap();
while let Some(event) = stream.next().await {
event.unwrap();
}
let requests = server.received_requests().await.unwrap();
let sent = captured_request_body(&requests[0]);
let expected = if api_key { api_tier } else { oauth_tier };
assert_eq!(sent.get("service_tier").and_then(Value::as_str), expected);
let hint = expected.map_or_else(
|| "model=gpt-5.4".to_owned(),
|tier| format!("model=gpt-5.4;tier={tier}"),
);
assert_eq!(requests[0].headers["x-codex-routing-hint"], hint);
server.verify().await;
}
}
}
#[tokio::test]
async fn custom_messages_and_responses_count_locally_without_billable_usage() {
for (target, source) in [
@@ -29,7 +29,7 @@ mod upstream_adapter;
use std::collections::{BTreeMap, BTreeSet};
use std::io::{self, Write};
use std::num::NonZeroU32;
use std::sync::{Arc, Mutex};
use std::sync::{Arc, Mutex, atomic::Ordering};
use std::time::{Duration, SystemTime};
use base64::{Engine as _, engine::general_purpose::STANDARD};
@@ -1305,6 +1305,7 @@ fn provider_and_quota_with_runtime_ports(
cooldowns,
Arc::clone(&account_feedback),
CodexCookiePolicy::official().expect("cookie policy"),
store.clone(),
));
let provider = CodexProvider::new(
@@ -1749,13 +1750,21 @@ fn diagnostic_context(request_id: &str, account_id: &str) -> AttemptContext {
)
}
fn request_with_peer_route(request_id: &str) -> RequestAttemptContext {
RequestAttemptContext::new(
ModelRequestId::new(request_id).unwrap(),
ClientApiKeyId::new("key_openai_contract").unwrap(),
)
.with_request_location(Some(global_request_location()))
.with_extension_scope(
gateway_core::engine::extensions::ExtensionCallScope::default()
.with_peer_route("0123456789abcdef0123456789abcdef".parse().unwrap()),
)
}
fn probe_context(request_id: &str, target: AccountProbeTarget) -> AttemptContext {
AttemptContext::new(
RequestAttemptContext::new(
ModelRequestId::new(request_id).expect("request id"),
ClientApiKeyId::new("key_openai_contract").expect("client key id"),
)
.with_request_location(Some(global_request_location())),
request_with_peer_route(request_id),
NonZeroU32::new(1).expect("attempt"),
SystemTime::now() + Duration::from_secs(30),
account_policy(),
@@ -8326,7 +8335,7 @@ async fn responses_should_observe_and_bill_the_outbound_service_tier_on_both_tra
false,
Some(json!("default")),
Some("priority"),
None,
Some("default"),
Some(3_437_500),
),
(
@@ -8342,7 +8351,7 @@ async fn responses_should_observe_and_bill_the_outbound_service_tier_on_both_tra
false,
Some(json!("priority")),
Some("priority"),
None,
Some("default"),
Some(3_437_500),
),
(
@@ -8350,7 +8359,7 @@ async fn responses_should_observe_and_bill_the_outbound_service_tier_on_both_tra
false,
Some(json!("fast")),
None,
None,
Some("default"),
Some(3_437_500),
),
(
@@ -8358,7 +8367,7 @@ async fn responses_should_observe_and_bill_the_outbound_service_tier_on_both_tra
false,
Some(json!("default")),
None,
None,
Some("default"),
Some(3_437_500),
),
(FastMode::Disabled, false, None, None, None, Some(3_437_500)),
@@ -8431,8 +8440,8 @@ async fn responses_should_observe_and_bill_the_outbound_service_tier_on_both_tra
true,
Some(json!("auto")),
None,
Some("auto"),
None,
Some(3_437_500),
),
(FastMode::Enabled, false, None, None, None, Some(3_437_500)),
(
@@ -8440,7 +8449,7 @@ async fn responses_should_observe_and_bill_the_outbound_service_tier_on_both_tra
false,
Some(json!("default")),
None,
None,
Some("default"),
Some(3_437_500),
),
] {
@@ -10498,80 +10507,6 @@ async fn successful_http_sse_rate_limit_event_persists_structured_exhaustion() {
assert!(store.has_quota(account_id));
}
#[test]
fn request_observation_reads_openai_metadata() {
let store = Arc::new(MemoryAccountStore::default());
let payload = ProtocolPayload::json_object(
"openai",
Map::from_iter([
("model".to_owned(), json!("gpt-5.4")),
("input".to_owned(), json!("hello")),
]),
)
.expect("OpenAI payload")
.with_context(Map::from_iter([(
"turn_metadata".to_owned(),
serde_json::Value::String(
r#"{"request_kind":"review","subagent_kind":"worker"}"#.to_owned(),
),
)]));
let generation = GenerateRequest::from_protocol_payload(payload);
let operation = Operation::Generate(generation);
let client_key_id = ClientApiKeyId::new("key_openai_observation").expect("client key");
let observation = provider(&store).request_observation(&operation, &client_key_id);
assert_eq!(observation.request_kind.as_deref(), Some("review"));
assert_eq!(observation.subagent_kind.as_deref(), Some("worker"));
assert!(!observation.compact);
}
#[test]
fn request_observation_classifies_prewarm_from_generate() {
let store = Arc::new(MemoryAccountStore::default());
let provider = provider(&store);
let client_key_id = ClientApiKeyId::new("key_prewarm_observation").expect("client key");
for (generate, request_kind, expected_kind) in [
(Some(json!(false)), None, Some("prewarm")),
(Some(json!(false)), Some("prewarm"), Some("prewarm")),
(Some(json!(false)), Some("review"), Some("prewarm")),
(Some(json!(true)), Some("prewarm"), None),
(None, Some("prewarm"), None),
(Some(json!(null)), Some("prewarm"), None),
(Some(json!("false")), Some("prewarm"), None),
(None, Some("review"), Some("review")),
(None, None, None),
] {
let mut body = Map::from_iter([
("model".to_owned(), json!("gpt-test")),
("input".to_owned(), json!("hello")),
("store".to_owned(), json!(false)),
]);
if let Some(generate) = generate {
body.insert("generate".to_owned(), generate);
}
let payload = ProtocolPayload::json_object("openai", body.clone())
.expect("OpenAI payload")
.with_context(Map::from_iter([(
"turn_metadata".to_owned(),
Value::String(
json!({"request_kind": request_kind, "subagent_kind": "worker"}).to_string(),
),
)]));
let operation = Operation::Generate(GenerateRequest::from_protocol_payload(payload));
let observation = provider.request_observation(&operation, &client_key_id);
assert_eq!(
observation.request_kind.as_deref(),
expected_kind,
"generate={:?}, request_kind={request_kind:?}",
body.get("generate"),
);
assert_eq!(observation.subagent_kind.as_deref(), Some("worker"));
}
}
#[test]
fn endpoint_observation_should_read_models_without_requiring_a_catalog() {
let store = Arc::new(MemoryAccountStore::default());
@@ -10611,26 +10546,6 @@ fn endpoint_observation_should_read_models_without_requiring_a_catalog() {
}
}
#[test]
fn request_observation_preserves_the_raw_reasoning_effort() {
let store = Arc::new(MemoryAccountStore::default());
for (effort, expected) in [
(json!("future-value"), "future-value"),
(json!(64), "64"),
(json!(0), "0"),
] {
let payload = ProtocolPayload::json_object(
"openai",
Map::from_iter([("reasoning".to_owned(), json!({"effort":effort}))]),
)
.unwrap();
let operation = Operation::Generate(GenerateRequest::from_protocol_payload(payload));
let client_key_id = ClientApiKeyId::new("key_openai_observation").unwrap();
let observation = provider(&store).request_observation(&operation, &client_key_id);
assert_eq!(observation.reasoning_effort.as_deref(), Some(expected));
}
}
#[tokio::test]
async fn reasoning_effort_observation_follows_configuration_updates_across_continuations() {
fn completed(index: usize) -> Value {
@@ -10736,7 +10651,19 @@ async fn reasoning_effort_observation_follows_configuration_updates_across_conti
generate = generate.with_provider_session_state(state);
}
let operation = Operation::Generate(generate);
let observation = provider.request_observation(&operation, &client_key);
let provider_observation = provider.request_observation(&operation, &client_key);
let Operation::Generate(generate) = &operation else {
panic!("generate operation");
};
let payload = generate.protocol_payload();
let observation = gateway_protocol::request::request_observation(
payload.protocol(),
payload.body(),
payload.context(),
provider_observation
.previous_reasoning_effort_override
.as_deref(),
);
assert_eq!(observation.reasoning_effort.as_deref(), Some(expected));
let binding = previous.as_ref().map(|previous| {
ContinuationBinding::Pinned(NativeContinuationPin::new(
@@ -10808,6 +10735,60 @@ async fn reasoning_effort_observation_follows_configuration_updates_across_conti
}
}
#[test]
fn request_observation_only_inherits_native_openai_session_configuration() {
let store = Arc::new(MemoryAccountStore::default());
let provider = provider(&store);
let client = ClientApiKeyId::new("key_session_observation").unwrap();
for (owner, extension_owned, expected) in [
("openai", false, Some("high")),
("xai", false, None),
("openai", true, None),
] {
let mut state = ProviderSessionState::new(
owner,
json!({
"account_id":"acct_observation", "conversation_id":"session",
"reasoning_effort_override":"high", "continuation_scope":"persisted"
})
.as_object()
.unwrap()
.clone(),
)
.unwrap();
if extension_owned {
state = state.with_extension_owner(gateway_core::operation::ExtensionSessionOwner {
instance_id: "plugin".into(),
contribution_id: "adapter".into(),
adapter_id: "adapter".into(),
generation: 1,
incarnation: "incarnation".into(),
connection_local: true,
});
}
let operation = Operation::Generate(
GenerateRequest::from_protocol_payload(
ProtocolPayload::json_object(
"openai",
json!({"previous_response_id":"resp_previous"})
.as_object()
.unwrap()
.clone(),
)
.unwrap(),
)
.with_provider_session_state(state),
);
assert_eq!(
provider
.request_observation(&operation, &client)
.previous_reasoning_effort_override
.as_deref(),
expected
);
}
}
#[tokio::test]
async fn provider_compiles_catalog_presentation_for_codex_models() {
let store = Arc::new(MemoryAccountStore::default());
@@ -11515,10 +11496,14 @@ async fn connection_limit_payload_survives_exhausted_retry_budget() {
#[tokio::test]
async fn api_key_native_endpoints_preserve_bodies_headers_and_own_base_url() {
for prefix in ["", "/v1", "/custom/v2"] {
for (prefix, discovered) in ["", "/v1", "/custom/v2"]
.into_iter()
.flat_map(|prefix| [false, true].map(|discovered| (prefix, discovered)))
{
let upstream = MockServer::start().await;
let oauth = MockServer::start().await;
let store = Arc::new(MemoryAccountStore::default());
store.peer_discovered.store(discovered, Ordering::SeqCst);
store
.seed_api_key(
"acct_provider_contract",
@@ -11536,10 +11521,6 @@ async fn api_key_native_endpoints_preserve_bodies_headers_and_own_base_url() {
Mock::given(method("POST"))
.and(path(format!("{prefix}{endpoint}")))
.and(header("authorization", "Bearer sk-api-test-only"))
.and(header(
gateway_protocol::peer::ROUTE_HEADER,
"0123456789abcdef0123456789abcdef",
))
.and(body_bytes(body.to_vec()))
.respond_with(
ResponseTemplate::new(200)
@@ -11608,6 +11589,13 @@ async fn api_key_native_endpoints_preserve_bodies_headers_and_own_base_url() {
}
assert!(oauth.received_requests().await.unwrap().is_empty());
for request in upstream.received_requests().await.unwrap() {
assert_eq!(
request
.headers
.get(gateway_protocol::peer::ROUTE_HEADER)
.map(|value| value.to_str().unwrap()),
discovered.then_some("0123456789abcdef0123456789abcdef")
);
for name in ["cookie", "chatgpt-account-id"] {
assert!(!request.headers.contains_key(name));
}
@@ -11735,10 +11723,14 @@ fn generate_with_downstream_headers() -> Operation {
#[tokio::test]
async fn api_key_default_http_uses_own_prefix_plain_json_and_only_own_authentication() {
for prefix in ["", "/v1", "/custom/v2"] {
for (prefix, discovered) in ["", "/v1", "/custom/v2"]
.into_iter()
.flat_map(|prefix| [false, true].map(|discovered| (prefix, discovered)))
{
let upstream = MockServer::start().await;
let oauth = MockServer::start().await;
let store = Arc::new(MemoryAccountStore::default());
store.peer_discovered.store(discovered, Ordering::SeqCst);
store
.seed_api_key(
"acct_provider_contract",
@@ -11748,7 +11740,6 @@ async fn api_key_default_http_uses_own_prefix_plain_json_and_only_own_authentica
.await;
Mock::given(method("GET"))
.and(path(format!("{prefix}/models")))
.and(move |request: &wiremock::Request| request.url.query().is_none())
.and(header("authorization", "Bearer sk-api-test-only"))
.respond_with(
ResponseTemplate::new(200).set_body_json(json!({"data":[{"id":"gpt-5.4"}]})),
@@ -11796,7 +11787,10 @@ async fn api_key_default_http_uses_own_prefix_plain_json_and_only_own_authentica
let mut stream = provider
.execute(
planned_request("openai", generate_with_downstream_headers()),
context("req_api_http", CancellationToken::new()),
context_for_request(
request_with_peer_route("req_api_http"),
CancellationToken::new(),
),
)
.await
.expect("scheduled API account");
@@ -11811,6 +11805,13 @@ async fn api_key_default_http_uses_own_prefix_plain_json_and_only_own_authentica
.find(|request| request.method == "POST")
.unwrap();
assert!(!request.headers.contains_key("content-encoding"));
assert_eq!(
request
.headers
.get(gateway_protocol::peer::ROUTE_HEADER)
.map(|value| value.to_str().unwrap()),
discovered.then_some("0123456789abcdef0123456789abcdef")
);
for name in API_KEY_DOWNSTREAM_HEADERS
.iter()
.filter(|name| **name != "authorization")
@@ -11841,7 +11842,7 @@ async fn api_key_default_http_uses_own_prefix_plain_json_and_only_own_authentica
assert_eq!(model_requests.len(), 1);
{
let model_request = model_requests[0];
assert!(model_request.url.query().is_none());
assert_eq!(model_request.url.query(), Some("client_version=0.144.0"));
assert_eq!(
model_request.headers["authorization"],
"Bearer sk-api-test-only"
@@ -12015,72 +12016,82 @@ async fn api_key_http_account_is_rejected_before_websocket_warmup_or_old_revisio
#[tokio::test]
async fn api_key_websocket_uses_api_path_and_bearer_without_oauth_identity() {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let base = format!("http://{}/custom/v2", listener.local_addr().unwrap());
let store = Arc::new(MemoryAccountStore::default());
store
.seed_api_key(
"acct_provider_contract",
base,
provider_openai::credential::ResponsesTransport::PreferWebsocket,
)
.await;
let server = tokio::spawn(async move {
let (stream, _) = listener.accept().await.unwrap();
let mut websocket =
crate::transport::accept_codex_test_websocket_with(stream, |request, _| {
assert_eq!(request.uri().path(), "/custom/v2/responses");
assert_eq!(
request.headers()["authorization"],
"Bearer sk-api-test-only"
);
assert!(!request.headers().contains_key("chatgpt-account-id"));
assert!(!request.headers().contains_key("cookie"));
assert_eq!(request.headers()["originator"], "codex_cli_rs");
assert_eq!(request.headers()["version"], "0.144.0");
assert_eq!(
request.headers()["user-agent"],
wire_profile().snapshot().user_agent()
);
for name in API_KEY_DOWNSTREAM_HEADERS
.iter()
.filter(|name| **name != "authorization")
{
assert!(!request.headers().contains_key(*name), "leaked {name}");
}
assert_eq!(request.headers()["x-business-extension"], "keep");
for name in API_KEY_BUSINESS_HEADERS {
if matches!(*name, "session-id" | "thread-id") {
assert_scoped_id(
request.headers()[*name].to_str().unwrap(),
"downstream-value",
);
} else {
assert_eq!(request.headers()[*name], "downstream-value", "lost {name}");
}
}
})
for discovered in [false, true] {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let base = format!("http://{}/custom/v2", listener.local_addr().unwrap());
let store = Arc::new(MemoryAccountStore::default());
store.peer_discovered.store(discovered, Ordering::SeqCst);
store
.seed_api_key(
"acct_provider_contract",
base,
provider_openai::credential::ResponsesTransport::PreferWebsocket,
)
.await;
let frame = websocket.next().await.unwrap().unwrap();
let payload: Value = serde_json::from_str(frame.to_text().unwrap()).unwrap();
assert_eq!(payload["type"], "response.create");
websocket.send(Message::Text(json!({"type":"response.completed","response":{"id":"resp_api_ws","model":"gpt-5.4","status":"completed","output":[],"usage":{"input_tokens":1,"output_tokens":1,"total_tokens":2}}}).to_string().into())).await.unwrap();
});
let provider = provider(&store);
let mut stream = provider
.execute(
planned_request("openai", generate_with_downstream_headers()),
diagnostic_context("req_api_ws", "acct_provider_contract"),
)
.await
.unwrap();
while let Some(event) = stream.next().await {
event.expect("API WebSocket completion");
let server = tokio::spawn(async move {
let (stream, _) = listener.accept().await.unwrap();
let mut websocket =
crate::transport::accept_codex_test_websocket_with(stream, |request, _| {
assert_eq!(request.uri().path(), "/custom/v2/responses");
assert_eq!(
request.headers()["authorization"],
"Bearer sk-api-test-only"
);
assert!(!request.headers().contains_key("chatgpt-account-id"));
assert!(!request.headers().contains_key("cookie"));
assert_eq!(
request
.headers()
.get(gateway_protocol::peer::ROUTE_HEADER)
.map(|value| value.to_str().unwrap()),
discovered.then_some("0123456789abcdef0123456789abcdef")
);
assert_eq!(request.headers()["originator"], "codex_cli_rs");
assert_eq!(request.headers()["version"], "0.144.0");
assert_eq!(
request.headers()["user-agent"],
wire_profile().snapshot().user_agent()
);
for name in API_KEY_DOWNSTREAM_HEADERS
.iter()
.filter(|name| **name != "authorization")
{
assert!(!request.headers().contains_key(*name), "leaked {name}");
}
assert_eq!(request.headers()["x-business-extension"], "keep");
for name in API_KEY_BUSINESS_HEADERS {
if matches!(*name, "session-id" | "thread-id") {
assert_scoped_id(
request.headers()[*name].to_str().unwrap(),
"downstream-value",
);
} else {
assert_eq!(request.headers()[*name], "downstream-value", "lost {name}");
}
}
})
.await;
let frame = websocket.next().await.unwrap().unwrap();
let payload: Value = serde_json::from_str(frame.to_text().unwrap()).unwrap();
assert_eq!(payload["type"], "response.create");
websocket.send(Message::Text(json!({"type":"response.completed","response":{"id":"resp_api_ws","model":"gpt-5.4","status":"completed","output":[],"usage":{"input_tokens":1,"output_tokens":1,"total_tokens":2}}}).to_string().into())).await.unwrap();
});
let provider = provider(&store);
let mut stream = provider
.execute(
planned_request("openai", generate_with_downstream_headers()),
diagnostic_context("req_api_ws", "acct_provider_contract"),
)
.await
.unwrap();
while let Some(event) = stream.next().await {
event.expect("API WebSocket completion");
}
timeout(Duration::from_secs(5), server)
.await
.unwrap()
.unwrap();
}
timeout(Duration::from_secs(5), server)
.await
.unwrap()
.unwrap();
}
#[tokio::test]
@@ -12622,18 +12633,27 @@ async fn fast_policy_changes_preserve_the_websocket_continuation_and_meter_each_
let mut websocket =
crate::transport::accept_codex_test_websocket_with(socket, |request, response| {
// 提示头只在首次握手发送;后续档位由各自 response.create 正文指定
assert_eq!(request.headers()["x-codex-routing-hint"], "model=gpt-5.4");
assert_eq!(
request.headers()["x-codex-routing-hint"],
"model=gpt-5.4;tier=default"
);
response.headers_mut().insert(
"sec-websocket-extensions",
"permessage-deflate".parse().unwrap(),
);
})
.await;
for (index, tier) in [None, Some("priority"), None, None].into_iter().enumerate() {
for (index, tier) in ["default", "priority", "default", "default"]
.into_iter()
.enumerate()
{
let message = websocket.next().await.unwrap().unwrap();
let frame: Value = serde_json::from_str(message.to_text().unwrap()).unwrap();
assert_eq!(frame["type"], "response.create");
assert_eq!(frame.get("service_tier").and_then(Value::as_str), tier);
assert_eq!(
frame.get("service_tier").and_then(Value::as_str),
Some(tier)
);
assert_eq!(
frame.get("previous_response_id").cloned(),
index
@@ -12790,7 +12810,11 @@ async fn fast_policy_changes_preserve_the_websocket_continuation_and_meter_each_
);
assert_eq!(
observed_tier.as_deref(),
(fast_mode == FastMode::Enabled).then_some("priority")
Some(if fast_mode == FastMode::Enabled {
"priority"
} else {
"default"
})
);
assert_eq!(
costs,
@@ -83,6 +83,7 @@ async fn outbound_http_sends_final_encoded_bytes_and_target_without_restoring_de
request.target = Target::Http { method: reqwest::Method::PATCH, uri: uri.parse().unwrap() };
request.headers.retain(|header| !matches!(header.name(), "user-agent" | "content-encoding" | "content-type" | "authorization" | "accept"));
request.headers.extend([
MiddlewareHeader::new(gateway_protocol::peer::ROUTE_HEADER, Bytes::from_static(b"plugin-owned-route")),
MiddlewareHeader::new("x-plugin-values", Bytes::from_static(b"first")),
MiddlewareHeader::new("x-plugin-values", Bytes::from_static(b"second")),
MiddlewareHeader::new("x-plugin-bytes", Bytes::from_static(b"\x80")),
@@ -122,6 +123,10 @@ async fn outbound_http_sends_final_encoded_bytes_and_target_without_restoring_de
);
assert_eq!(request.url.query(), Some("source=outbound"));
assert_eq!(request.headers["x-plugin-bytes"].as_bytes(), b"\x80");
assert_eq!(
request.headers[gateway_protocol::peer::ROUTE_HEADER],
"plugin-owned-route"
);
}
#[tokio::test]
@@ -302,13 +302,28 @@ async fn websocket_messages_preserve_text_and_metadata_despite_unrecognized_json
}
#[tokio::test]
async fn default_service_tiers_are_omitted_only_for_codex_oauth() {
for tier in ["auto", "default", "priority", "flex"] {
async fn responses_service_tiers_reach_each_upstream_on_both_transports() {
for tier in [
None,
Some(Value::Null),
Some(json!("auto")),
Some(json!("default")),
Some(json!("priority")),
Some(json!("flex")),
] {
for api_key in [false, true] {
let body = json!({"model":"gpt-5.4","input":"hello","service_tier":tier});
let sent = capture_downstream_request(body, api_key, false, false).await;
let expected = (api_key || !matches!(tier, "auto" | "default")).then_some(tier);
assert_eq!(sent.get("service_tier").and_then(Value::as_str), expected);
for websocket in [false, true] {
let mut body = json!({"model":"gpt-5.4","input":"hello"});
if let Some(tier) = &tier {
body["service_tier"] = tier.clone();
}
let sent = capture_downstream_request(body, api_key, websocket, false).await;
assert_eq!(
sent.get("service_tier"),
tier.as_ref(),
"api_key={api_key}, websocket={websocket}"
);
}
}
}
}
@@ -483,3 +483,108 @@ mod native_tls {
server.await.unwrap();
}
}
fn overload_event(no_more_retry: bool) -> Value {
let mut headers = json!({"retry-after": "30", "retry-after-ms": "30000"});
if no_more_retry {
headers["x-retry-metadata"] = json!("NO_MORE_RETRY");
}
json!({
"type": "error",
"error": {
"type": "service_unavailable_error",
"code": "server_is_overloaded",
"message": "Our servers are currently overloaded. Please try again later.",
"headers": headers
}
})
}
#[tokio::test]
async fn upstream_no_more_retry_blocks_same_account_recovery_for_a_short_retry_after() {
for (name, websocket, status, body, content_type, blocked, policy, kind) in [
(
"sse-capacity",
false,
200,
format!("event: error\ndata: {}\n\n", overload_event(true)),
"text/event-stream",
true,
TransientRetryPolicy::RetryThenRotateAccount,
ProviderErrorKind::UpstreamCapacityUnavailable,
),
(
"sse-capacity-without-signal",
false,
200,
format!("event: error\ndata: {}\n\n", overload_event(false)),
"text/event-stream",
false,
TransientRetryPolicy::RetryThenRotateAccount,
ProviderErrorKind::UpstreamCapacityUnavailable,
),
(
"http-server-error",
false,
503,
json!({"error":{"code":"server_error","type":"server_error","message":"retry later","headers":{"x-retry-metadata":"NO_MORE_RETRY","retry-after":"30"}}}).to_string(),
"application/json",
true,
TransientRetryPolicy::RetryThenReturnError,
ProviderErrorKind::Unavailable,
),
(
"websocket-capacity",
true,
503,
json!({"error":{"code":"server_is_overloaded","type":"server_error","message":"Selected model is at capacity. Please try a different model.","headers":{"x-retry-metadata":"NO_MORE_RETRY","retry-after":"30"}}}).to_string(),
"application/json",
true,
TransientRetryPolicy::RetryThenRotateAccount,
ProviderErrorKind::UpstreamCapacityUnavailable,
),
] {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_provider_contract").await;
let server = MockServer::start().await;
let mut response = ResponseTemplate::new(status).insert_header("content-type", content_type);
if status != 200 {
response = response.insert_header("retry-after", "30");
}
Mock::given(method(if websocket { "GET" } else { "POST" }))
.and(path("/codex/responses"))
.respond_with(response.set_body_string(body))
.expect(1)
.mount(&server)
.await;
let operation = if websocket {
capacity_websocket_operation()
} else {
http_generate_operation()
};
let mut stream = provider_with_base_url(&store, server.uri())
.execute(
planned_request("openai", operation),
context("req_no_more_retry", CancellationToken::new()),
)
.await
.unwrap();
let (_, error) = stream_failure(&mut stream).await;
assert_eq!(error.kind(), kind, "{name}");
assert_eq!(error.same_account_retry_blocked(), blocked, "{name}");
assert!(!error.retry_is_prohibited(), "{name}");
assert_eq!(error.retry_after(), Some(Duration::from_secs(30)), "{name}");
assert!(
matches!(
error.pre_delivery_retry(),
Some(PreDeliveryRetry::SameAccountTransientRetry {
policy: actual,
..
}) if actual == policy
),
"{name} preserved the exhaustion policy: {:?}",
error.pre_delivery_retry()
);
server.verify().await;
}
}
@@ -19,12 +19,12 @@ use gateway_core::account::{
use gateway_core::error::{StoreError, StoreErrorKind};
use gateway_core::policy::ClientApiKeyId;
use gateway_core::provider_ports::{
ProviderCatalogCacheKey, ProviderCatalogCachePort, ProviderCooldown, ProviderCooldownPort,
ProviderCooldownScope, ProviderLeaseAcquisition, ProviderLeasePort, ProviderLeaseRequest,
ProviderRefreshPolicy, ProviderRuntimePolicyPort, ProviderSchedulingLeaseRequest,
ProviderSchedulingState, ProviderScopedCooldown, ProviderSessionAffinityKey,
ProviderSessionAffinityPort, ProviderSessionExclusionPort, ProviderSessionExclusions,
ProviderStoreError,
ProviderAccountAdmission, ProviderCatalogCacheKey, ProviderCatalogCachePort, ProviderCooldown,
ProviderCooldownPort, ProviderCooldownScope, ProviderLeaseAcquisition, ProviderLeasePort,
ProviderLeaseRequest, ProviderRefreshPolicy, ProviderRuntimePolicyPort,
ProviderSchedulingLeaseRequest, ProviderSchedulingState, ProviderScopedCooldown,
ProviderSessionAffinityKey, ProviderSessionAffinityPort, ProviderSessionBinding,
ProviderSessionExclusionPort, ProviderSessionExclusions, ProviderStoreError,
};
use gateway_core::routing::ProviderKind;
use provider_openai::credential::{
@@ -52,6 +52,7 @@ type CredentialLoadHook = dyn for<'a> Fn(
#[derive(Default)]
pub(crate) struct MemoryAccountStore {
pub(crate) peer_discovered: AtomicBool,
accounts: Mutex<BTreeMap<ProviderAccountId, StoredAccount>>,
catalogs: Mutex<
BTreeMap<
@@ -73,6 +74,21 @@ pub(crate) struct MemoryAccountStore {
credential_loads: AtomicUsize,
}
impl gateway_core::account::UpstreamInfoSource for MemoryAccountStore {
fn is_peer(&self, _: &ProviderAccount) -> bool {
self.peer_discovered.load(Ordering::SeqCst)
}
fn signal(
&self,
_: &ProviderAccount,
_: Option<&str>,
_: SystemTime,
) -> Option<gateway_core::account::UpstreamSchedulingSignal> {
None
}
}
impl MemoryAccountStore {
pub(crate) fn with_pending_account_reads() -> Self {
Self {
@@ -989,6 +1005,7 @@ impl ProviderLeasePort for TestLeaseCoordinator {
#[derive(Default)]
pub(crate) struct MemorySessionAffinity {
cursors: Mutex<BTreeMap<(ClientApiKeyId, ProviderKind), u64>>,
operation_delay: Duration,
initial_claim_barrier: Option<Arc<tokio::sync::Barrier>>,
aliases: Mutex<BTreeMap<(String, String), gateway_core::provider_ports::ProviderSessionAlias>>,
@@ -1100,48 +1117,77 @@ impl ProviderSessionAffinityPort for MemorySessionAffinity {
})
}
fn compare_and_bind<'a>(
fn round_robin_cursor<'a>(
&'a self,
provider: &'a ProviderKind,
key: &'a ProviderSessionAffinityKey,
expected: Option<&'a gateway_core::provider_ports::ProviderSessionBinding>,
account: &'a ProviderAccountId,
ttl: Duration,
) -> BoxFuture<
'a,
Result<Option<gateway_core::provider_ports::ProviderSessionBinding>, ProviderStoreError>,
> {
client: &'a ClientApiKeyId,
) -> BoxFuture<'a, Result<u64, ProviderStoreError>> {
Box::pin(async move {
Ok(self
.cursors
.lock()
.unwrap()
.get(&(client.clone(), provider.clone()))
.copied()
.unwrap_or_default())
})
}
fn admit_account<'a>(
&'a self,
provider: &'a ProviderKind,
admission: ProviderAccountAdmission<'a>,
) -> BoxFuture<'a, Result<bool, ProviderStoreError>> {
Box::pin(async move {
self.delay_operation().await;
// 让并发首请求都读到空绑定,再验证发送前的原子认领冲突
if expected.is_none()
if admission
.session()
.is_some_and(|claim| claim.expected.is_none())
&& let Some(barrier) = &self.initial_claim_barrier
{
barrier.wait().await;
}
let key = (
provider.as_str().to_owned(),
key.expose_to_store().to_owned(),
);
let mut bindings = self.bindings.lock().expect("bindings");
if bindings.get(&key) != expected {
return Ok(None);
}
let binding = expected
.filter(|binding| binding.account_id() == account)
.cloned()
.unwrap_or_else(|| {
gateway_core::provider_ports::ProviderSessionBinding::new(
account.clone(),
let mut cursors = self.cursors.lock().unwrap();
let step = if let ProviderAccountAdmission::RoundRobin {
client_key, cursor, ..
} = admission
{
let current = cursors
.entry((client_key.clone(), provider.clone()))
.or_default();
if *current != cursor {
return Ok(false);
}
Some(current)
} else {
None
};
if let Some(claim) = admission.session() {
let key = (
provider.as_str().to_owned(),
claim.key.expose_to_store().to_owned(),
);
let mut bindings = self.bindings.lock().unwrap();
if bindings.get(&key) != claim.expected {
return Ok(false);
}
let binding = match claim.expected.filter(|b| b.account_id() == claim.account) {
Some(binding) => binding.clone(),
None => ProviderSessionBinding::new(
claim.account.clone(),
uuid::Uuid::new_v4().simple().to_string(),
)
.expect("binding")
});
bindings.insert(key, binding.clone());
self.renewal_ttls.lock().expect("TTLs").push(ttl);
Ok(Some(binding))
)?,
};
bindings.insert(key, binding);
self.renewal_ttls.lock().unwrap().push(claim.ttl);
}
if let Some(cursor) = step {
*cursor = cursor.wrapping_add(1);
}
Ok(true)
})
}
fn load_alias<'a>(
&'a self,
provider: &'a ProviderKind,
@@ -1,11 +1,17 @@
//! 验证 HTTP 与 WebSocket 的 TLS 握手特征
use std::collections::BTreeMap;
use std::path::Path;
use std::process::Stdio;
use bytes::{Buf as _, Bytes};
use super::*;
const CASE_ENV: &str = "CODEX_PROXY_TEST_HTTP_CLIENT_HELLO";
const CASE_COMPLETED: &str = "http-tls-client-hello-case-completed";
const REFERENCE_ENV: &str = "CODEX_PROXY_TEST_REFERENCE_BINARY";
// 基线来自官方 Codex 823ea830c0 的客户端源码,各路径重复采样五次
// HTTP 另与本机 Core 0.160.0 的采样核对,源码客户端与本项目还通过真实 models 请求验证
// HTTP/native-tls 的扩展顺序固定;WebSocket/rustls 会随机化扩展顺序和临时密钥
@@ -25,9 +31,6 @@ struct ClientHello {
#[test]
fn http_client_hellos_should_match_official_tls_transports() {
const CASE_ENV: &str = "CODEX_PROXY_TEST_HTTP_CLIENT_HELLO";
const CASE_COMPLETED: &str = "http-tls-client-hello-case-completed";
if let Ok(case) = std::env::var(CASE_ENV) {
let runtime = tokio::runtime::Builder::new_current_thread()
.enable_all()
@@ -54,22 +57,81 @@ fn http_client_hellos_should_match_official_tls_transports() {
return;
}
run_isolated_cases(
"transport::tls::http_client_hellos_should_match_official_tls_transports",
&["native", "custom-ca"],
);
}
#[test]
#[ignore = "requires CODEX_PROXY_TEST_REFERENCE_BINARY pointing to the official Codex executable"]
fn client_hellos_should_match_the_official_binary_on_this_platform() {
let reference = std::env::var_os(REFERENCE_ENV)
.map(std::path::PathBuf::from)
.expect("set CODEX_PROXY_TEST_REFERENCE_BINARY to the official Codex executable")
.canonicalize()
.expect("official Codex executable path");
if let Ok(case) = std::env::var(CASE_ENV) {
let runtime = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.unwrap();
runtime.block_on(async {
for sample in 0..3 {
let mut expected = capture_official_hello(&reference, case == "websocket").await;
let mut actual = if case == "websocket" {
capture_websocket_hello().await
} else {
capture_http_hello(provider_openai::transport::build_reqwest_client().unwrap())
.await
};
// rustls 随机排列扩展,其他字段及各自列表的顺序仍逐项核对
if case != "native" {
expected.extensions.sort_unstable();
actual.extensions.sort_unstable();
}
assert_eq!(actual, expected, "{case} ClientHello sample {sample}");
}
});
println!("\n{CASE_COMPLETED}");
return;
}
let version = Command::new(&reference)
.arg("--version")
.output()
.expect("read official Codex version");
assert!(version.status.success());
println!(
"TLS reference: {} on {}-{}",
String::from_utf8_lossy(&version.stdout).trim(),
std::env::consts::OS,
std::env::consts::ARCH,
);
run_isolated_cases(
"transport::tls::client_hellos_should_match_the_official_binary_on_this_platform",
&["native", "custom-ca", "websocket"],
);
}
fn run_isolated_cases(test_name: &str, cases: &[&str]) {
// Cargo 运行测试时可能注入 SSL_CERT_FILE,生产代码会把它视作自定义 CA 并切到 rustls;
// 因此在清理相关环境变量的子进程里验证默认路径
let current_exe = std::env::current_exe().expect("current test binary path");
let directory = tempfile::tempdir().unwrap();
let ca_path = directory.path().join("ca.pem");
std::fs::write(&ca_path, TEST_CA_PEM).unwrap();
for case in ["native", "custom-ca"] {
for case in cases {
let mut command = Command::new(&current_exe);
command
.arg("--exact")
.arg("transport::tls::http_client_hellos_should_match_official_tls_transports")
.arg(test_name)
.arg("--include-ignored")
.arg("--nocapture")
.env(CASE_ENV, case)
.env_remove(provider_openai::transport::tls::CODEX_CA_CERT_ENV)
.env_remove(provider_openai::transport::tls::SSL_CERT_FILE_ENV);
if case == "custom-ca" {
if *case == "custom-ca" {
command.env(provider_openai::transport::tls::CODEX_CA_CERT_ENV, &ca_path);
}
let output = command.output().expect("run isolated TLS ClientHello case");
@@ -91,6 +153,70 @@ fn http_client_hellos_should_match_official_tls_transports() {
}
}
async fn capture_official_hello(reference: &Path, websocket: bool) -> ClientHello {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let endpoint = format!(
"https://localhost:{}/v1",
listener.local_addr().unwrap().port()
);
let directory = tempfile::tempdir().unwrap();
// 官方进程只访问本机采集端点,握手后即终止,认证使用无效测试值
let mut command = tokio::process::Command::new(reference);
command
.args([
"exec",
"--ignore-user-config",
"--ignore-rules",
"--ephemeral",
"--skip-git-repo-check",
"--sandbox",
"read-only",
"--json",
"--model",
"tls-probe",
])
.current_dir(directory.path())
.env("CPR_TLS_PROBE_API_KEY", "test-local-handshake-only")
.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null())
.kill_on_drop(true);
for name in [
"HTTP_PROXY",
"HTTPS_PROXY",
"ALL_PROXY",
"http_proxy",
"https_proxy",
"all_proxy",
] {
command.env_remove(name);
}
for setting in [
"model_provider=\"cpr_tls_probe\"".to_owned(),
"model_providers.cpr_tls_probe.name=\"CPR TLS Probe\"".to_owned(),
format!("model_providers.cpr_tls_probe.base_url=\"{endpoint}\""),
"model_providers.cpr_tls_probe.env_key=\"CPR_TLS_PROBE_API_KEY\"".to_owned(),
"model_providers.cpr_tls_probe.wire_api=\"responses\"".to_owned(),
format!("model_providers.cpr_tls_probe.supports_websockets={websocket}"),
"model_providers.cpr_tls_probe.request_max_retries=0".to_owned(),
"model_providers.cpr_tls_probe.stream_max_retries=0".to_owned(),
"project_doc_max_bytes=0".to_owned(),
] {
command.args(["--config", &setting]);
}
let mut child = command
.arg("TLS handshake capture")
.spawn()
.expect("start official Codex TLS reference");
let hello = timeout(Duration::from_secs(30), read_client_hello(listener)).await;
let _ = child.kill().await;
child
.wait()
.await
.expect("reap official Codex TLS reference");
hello.expect("official Codex ClientHello within timeout")
}
async fn capture_http_hello(client: reqwest::Client) -> ClientHello {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let url = format!(
@@ -111,6 +237,12 @@ async fn capture_http_hello(client: reqwest::Client) -> ClientHello {
#[tokio::test]
async fn websocket_client_hello_should_match_official_rustls_transport() {
let mut hello = capture_websocket_hello().await;
hello.extensions.sort_unstable();
assert_eq!(hello, official_websocket_hello());
}
async fn capture_websocket_hello() -> ClientHello {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let url = format!("wss://localhost:{}/", listener.local_addr().unwrap().port());
let connector =
@@ -129,9 +261,7 @@ async fn websocket_client_hello_should_match_official_rustls_transport() {
response.is_err(),
"capture endpoint rejects TLS after ClientHello"
);
let mut normalized = hello;
normalized.extensions.sort_unstable();
assert_eq!(normalized, official_websocket_hello());
hello
}
#[cfg(target_os = "linux")]
+2 -1
View File
@@ -39,6 +39,7 @@ use gateway_core::provider_ports::{
ProviderStoreError, ProviderStoreErrorKind,
};
use gateway_core::routing::{ProviderKind, UpstreamModelId};
use gateway_protocol::openai::OPENAI_PROTOCOL;
use serde::Deserialize;
use serde_json::{Map, Number, Value};
use sha2::{Digest as _, Sha256};
@@ -1459,7 +1460,7 @@ fn build_connection_test_operation(
);
body.insert("stream".to_owned(), Value::Bool(true));
body.insert("store".to_owned(), Value::Bool(false));
let payload = ProtocolPayload::json_object("openai", body)
let payload = ProtocolPayload::json_object(OPENAI_PROTOCOL, body)
.map_err(|_| provider_error(ProviderAdminErrorKind::Invalid))?;
Ok(Operation::Generate(GenerateRequest::from_protocol_payload(
payload,
@@ -12,8 +12,8 @@ use gateway_core::account::{
use gateway_core::engine::continuation::ContinuationBinding;
use gateway_core::engine::middleware::MiddlewareHeader;
use gateway_core::engine::provider::{
EventStream, Provider, ProviderCallMetadata, ProviderRequest, ProviderRequestObservation,
ProviderSelectionObservation, ProviderStream,
EventStream, Provider, ProviderCallMetadata, ProviderRequest, ProviderSelectionObservation,
ProviderStream,
};
use gateway_core::engine::upstream_adapter::UpstreamAdapterSelection;
use gateway_core::engine::{AttemptContext, ContinuationAttempt};
@@ -39,7 +39,7 @@ use gateway_core::task::{
WorkerTaskError,
};
use gateway_core::upstream::{UpstreamSendState, UpstreamTransport};
use gateway_protocol::openai::codex_responses_request_semantics;
use gateway_protocol::openai::OPENAI_PROTOCOL;
use serde::{Deserialize, Serialize};
use serde_json::{Map, Value};
use url::Url;
@@ -182,30 +182,6 @@ impl Provider for GrokBuildProvider {
self.catalog.catalog_generation()
}
fn request_observation(
&self,
operation: &Operation,
_client_api_key_id: &gateway_core::policy::ClientApiKeyId,
) -> ProviderRequestObservation {
let Operation::Generate(request) = operation else {
return ProviderRequestObservation::default();
};
let payload = request.protocol_payload();
if payload.protocol() != "openai" {
return ProviderRequestObservation::default();
}
let semantics = codex_responses_request_semantics(payload.body(), payload.context());
ProviderRequestObservation {
requested_model: None,
reasoning_effort: semantics.reasoning_effort,
reasoning_preset: semantics.reasoning_preset.map(str::to_owned),
request_kind: semantics.request_kind,
subagent_kind: semantics.subagent_kind,
compact: semantics.compact,
continuation: Default::default(),
}
}
async fn query_model_capabilities(
&self,
) -> Result<Vec<ProviderModelCapabilities>, ProviderError> {
@@ -387,7 +363,7 @@ impl GrokBuildProvider {
let Operation::Generate(generate) = operation else {
return Err(protocol_not_sent());
};
if generate.protocol_payload().protocol() != "openai"
if generate.protocol_payload().protocol() != OPENAI_PROTOCOL
|| native_request_requirements(&generate) != frozen_requirements
{
return Err(provider_error(
@@ -643,7 +619,7 @@ fn grok_selection_facts(
} else {
decode_xai_session_state(generate)?
};
let native_source = !adapted && generate.protocol_payload().protocol() == "openai";
let native_source = !adapted && generate.protocol_payload().protocol() == OPENAI_PROTOCOL;
let native_compaction =
native_source && crate::transport::compaction::has_terminal_compaction_trigger(generate);
let (selection_model, operation_account, affinity) = if native_compaction {
@@ -15,6 +15,7 @@ use gateway_core::metering::{
CurrencyCode, Decimal, Money, ProviderReportedCost, Usage,
};
use gateway_core::upstream::UpstreamSendState;
use gateway_protocol::openai::OPENAI_PROTOCOL;
use gateway_protocol::openai::events::{TokenUsage, billable_usage_is_complete, extract_usage};
use gateway_protocol::openai::sse::{SseEvent, SseEventDecoder};
use serde_json::Value;
@@ -26,7 +27,6 @@ const CONTENTS_PER_OUTPUT: u32 = 1_024;
const LONG_CONTEXT_THRESHOLD: u64 = 200_000;
const GROK_PING_SSE_COMMENT: &[u8] = b": ping\n\n";
const XAI_RESPONSE_PROTOCOL: &str = "xai";
const OPENAI_RESPONSE_PROTOCOL: &str = "openai";
pub(crate) struct GrokDecodedResponseBatch {
pub(crate) source_events: Vec<ProviderEvent>,
@@ -311,7 +311,7 @@ impl NativeResponseTranslator for GrokNativeResponseTranslator {
}
fn target_protocol(&self) -> &str {
OPENAI_RESPONSE_PROTOCOL
OPENAI_PROTOCOL
}
fn translate(
@@ -344,7 +344,7 @@ impl NativeResponseTranslator for GrokNativeResponseTranslator {
.is_some_and(|frame| frame.as_ref() == GROK_PING_SSE_COMMENT)
{
return ProtocolWireEvent::raw_sse(
OPENAI_RESPONSE_PROTOCOL,
OPENAI_PROTOCOL,
bytes::Bytes::from_static(GROK_PING_SSE_COMMENT),
)
.map(|wire| vec![wire])
@@ -485,7 +485,7 @@ impl GrokCanonicalDecoder {
)
.map_err(|_| protocol_error_marker())?;
let projected_wire = ProtocolWireEvent::raw_sse(
OPENAI_RESPONSE_PROTOCOL,
OPENAI_PROTOCOL,
bytes::Bytes::from_static(GROK_PING_SSE_COMMENT),
)
.map_err(|_| protocol_error_marker())?;
@@ -937,7 +937,7 @@ fn project_response_event(
Some(transformed_type.clone())
};
let wire = ProtocolWireEvent::json_with_sse_metadata(
OPENAI_RESPONSE_PROTOCOL,
OPENAI_PROTOCOL,
wire_event,
value,
(index == 0).then(|| sse_id.map(str::to_owned)).flatten(),
@@ -7,6 +7,7 @@ use gateway_core::error::IdentifierError;
use gateway_core::event::{GatewayEvent, ProtocolWireEvent, ProviderEvent, ResponseMeta};
use gateway_core::operation::GenerateRequest;
use gateway_core::policy::ClientApiKeyId;
use gateway_protocol::openai::OPENAI_PROTOCOL;
use serde_json::{Map, Value, json};
use uuid::Uuid;
@@ -303,7 +304,7 @@ pub(crate) fn compaction_wire_events(
);
Ok(GrokCompactionWireEvents {
created: ProtocolWireEvent::json_with_sse_metadata(
"openai",
OPENAI_PROTOCOL,
Some("response.created".to_owned()),
json!({
"type": "response.created",
@@ -313,7 +314,7 @@ pub(crate) fn compaction_wire_events(
None,
)?,
output_done: ProtocolWireEvent::json_with_sse_metadata(
"openai",
OPENAI_PROTOCOL,
Some("response.output_item.done".to_owned()),
json!({
"type": "response.output_item.done",
@@ -324,7 +325,7 @@ pub(crate) fn compaction_wire_events(
None,
)?,
terminal: ProtocolWireEvent::json_with_sse_metadata(
"openai",
OPENAI_PROTOCOL,
Some("response.completed".to_owned()),
json!({
"type": "response.completed",
@@ -5,6 +5,7 @@ use std::fmt;
use gateway_core::operation::GenerateRequest;
use gateway_core::policy::ClientApiKeyId;
use gateway_protocol::openai::OPENAI_PROTOCOL;
use serde_json::{Map, Value, json};
use sha2::{Digest as _, Sha256};
@@ -217,7 +218,7 @@ impl GrokResponsesRequest {
should_consume_terminal_compaction_trigger: bool,
) -> Result<Self, GrokRequestEncodeError> {
let payload = request.protocol_payload();
if payload.protocol() != "openai" {
if payload.protocol() != OPENAI_PROTOCOL {
return Err(GrokRequestEncodeError::InvalidProtocolPayload);
}
// Grok Build 会把 Codex 的预热请求当作真实生成;必须在发送前拒绝,
@@ -2014,54 +2014,6 @@ async fn account_selection_uses_the_exact_routed_model() {
);
}
#[tokio::test]
async fn request_observation_preserves_raw_xai_reasoning_effort() {
let provider = provider(StubSelector::success(), StubInferenceTransport::success()).await;
let client_key_id =
gateway_core::policy::ClientApiKeyId::new("key_xai_observation").expect("client key");
let observation = provider.request_observation(
&operation_with_reasoning_effort("future-value"),
&client_key_id,
);
assert_eq!(
observation.reasoning_effort.as_deref(),
Some("future-value")
);
}
#[tokio::test]
async fn request_observation_preserves_codex_semantics_for_xai() {
let provider = provider(StubSelector::success(), StubInferenceTransport::success()).await;
let payload = ProtocolPayload::json_object(
"openai",
object(json!({
"model": "client-model",
"reasoning": {"effort": "xhigh"},
"input": [
{"type": "message", "role": "user", "content": "history"},
{"type": "compaction_trigger"}
]
})),
)
.expect("OpenAI payload")
.with_context(Map::from_iter([(
"turn_metadata".to_owned(),
json!("{\"request_kind\":\"compaction\",\"subagent_kind\":\"compact\"}"),
)]));
let operation = Operation::Generate(GenerateRequest::from_protocol_payload(payload));
let client_key_id =
gateway_core::policy::ClientApiKeyId::new("key_xai_observation").expect("client key");
let observation = provider.request_observation(&operation, &client_key_id);
assert_eq!(observation.reasoning_effort.as_deref(), Some("xhigh"));
assert_eq!(observation.request_kind.as_deref(), Some("compaction"));
assert_eq!(observation.subagent_kind.as_deref(), Some("compact"));
assert!(observation.compact);
}
fn operation_with_state(body: serde_json::Value, state: ProviderSessionState) -> Operation {
let serde_json::Value::Object(body) = body else {
panic!("request body must be an object");
+11 -16
View File
@@ -836,24 +836,19 @@ impl ProviderSessionAffinityPort for TestSessionAffinity {
> {
Box::pin(async { Ok(None) })
}
fn compare_and_bind<'a>(
fn round_robin_cursor<'a>(
&'a self,
_: &'a ProviderKind,
_: &'a ProviderSessionAffinityKey,
_: Option<&'a gateway_core::provider_ports::ProviderSessionBinding>,
account: &'a ProviderAccountId,
_: Duration,
) -> futures::future::BoxFuture<
'a,
Result<Option<gateway_core::provider_ports::ProviderSessionBinding>, ProviderStoreError>,
> {
Box::pin(async move {
gateway_core::provider_ports::ProviderSessionBinding::new(
account.clone(),
"00000000000000000000000000000000".to_owned(),
)
.map(Some)
})
_: &'a gateway_core::policy::ClientApiKeyId,
) -> futures::future::BoxFuture<'a, Result<u64, ProviderStoreError>> {
panic!("xAI does not read session round robin cursors")
}
fn admit_account<'a>(
&'a self,
_: &'a ProviderKind,
_: gateway_core::provider_ports::ProviderAccountAdmission<'a>,
) -> futures::future::BoxFuture<'a, Result<bool, ProviderStoreError>> {
panic!("xAI does not claim OpenAI sessions")
}
fn load_alias<'a>(
&'a self,
+6 -2
View File
@@ -532,7 +532,7 @@ SSE 注释保活用于防止传输链路空闲断开,不会重置 Codex 等待
管理端生成的 CC Switch 一键导入包含连接信息和当前 Key 的日/周额度查询,默认每 30 分钟刷新。
查询地址和凭据随导入生成,在 CC Switch 中修改 Provider 的地址或 Key 后,需重新导入以同步用量查询。
导入不包含下方模板的远程压缩、原生生图、WebSocket 和语音设置。需要这些设置时,使用下方配置直连 CPR,
导入不包含下方模板的远程压缩、异步压缩、原生生图、WebSocket 和语音设置。需要这些设置时,使用下方配置直连 CPR,
避免 CC Switch 切换或接管后重写 Provider 配置
Linux/macOS 默认目录为 `~/.codex/`,Windows 为 `%USERPROFILE%\.codex\`;
@@ -572,6 +572,7 @@ X-OpenAI-Actor-Authorization = "proxy-managed"
[features]
api_key_model_discovery = true
async_compaction = true
image_generation = true
goals = true
```
@@ -668,9 +669,12 @@ metrics_exporter = "none"
## 停止与重启
管理端重启、SIGTERM 和 SIGINT 使用同一关闭策略:停止接收新连接,不等待现有 HTTP、SSE、
WebSocket 会话完成,进程退出时强制断开剩余连接。已入队的观测写入与后台清理保留
WebSocket 会话完成,进程退出时强制断开剩余连接。待写费用、已入队的观测与后台清理保留
`host.worker_shutdown_timeout_seconds`(默认 30 秒)的收尾预算;客户端在新进程就绪后重新连接
费用被写入队列接管不等于已落库。数据库持续失败、收尾超时或进程强制退出时,未持久化费用与
未写入的观测可能丢失,重启不会恢复这些内存队列。延长收尾预算不能保证费用不丢失
正常关闭会在后台收尾预算内释放本实例的账号并发占位,租约清理最多等待 2 秒。
强制终止、清理超时或 Redis 清理失败时,占位由最后一次续期后的 10 分钟 TTL 回收,不影响其他实例的租约
+19 -5
View File
@@ -395,7 +395,7 @@ adaptive 路径将普通强制选择降为 `auto`,保留串行约束。
服务档位按目标协议映射:Responses `default` 对应 Messages `standard_only`,`auto` 保留,无法对应的档位省略。
按 CPA 将 Responses `priority` 与 Messages `speed:fast` 双向映射;这两个字段不代表相同的服务承诺。
跨协议生成 fast 请求时补充 `fast-mode-2026-02-01` beta,并保留已有 beta;目标是否支持 fast 由上游校验。
Codex OAuth 出站省略 `auto/default`,Responses API Key 保留这些公开 API 参数。
Messages 转向 Codex OAuth 时省略转换生成的 `auto/default`,转向 Responses API Key 时保留。
会话内配置、工具增删和指令生效期限等扩展需要完整保留时使用同协议上游
Messages 错误使用 `{"type":"error","error":{"type":"...","message":"..."}}`;SSE 开始后使用 `event: error`。
@@ -813,6 +813,11 @@ OpenAI 仅在允许安全重放且尚未交付不可重放输出时执行下表
保留原始 `Retry-After`。等于上限仍可等待,0 表示不接受正数等待;未提供有效建议时继续使用本地退避。
该设置不限制请求总时长,不改变已交付不可重放输出、续接绑定和换号次数等限制;保存后对新请求生效。
上游错误对象 headers 中的 `x-retry-metadata: NO_MORE_RETRY` 表示拒绝继续重试当前账号。
此时无论 `Retry-After` 长短,都跳过同账号退避和同账号传输恢复,并立即执行上表的耗尽动作:
允许换号的容量拒绝按调度换号,`server_error` 直接返回该次上游错误。
没有其他可重试账号、换号预算或安全重放条件时,同样返回该次上游错误,并保留原始 `Retry-After`。
已发送的容量失败计入[智能调度健康分](architecture.md#智能调度),可用账号的会话亲和优先。
容量不足不触发 Provider 全局熔断,也不作为账号额度耗尽;账号冷却由独立的[错误保护规则](#账号错误保护)决定。
客户端错误兼容由 API 编码出口统一处理:最终交付的 `server_is_overloaded`、`slow_down` 错误码
@@ -1027,9 +1032,12 @@ CPR 默认路径为 `/v1/cpr/link`,使用同一数据面入口认证,不接
某个模型还有未纳入观测的候选账号时,其覆盖为 `partial`,不能据此断定该模型已耗尽。
这些开关和账号组织方式属于 CPR 内置实现;第三方通过自身授权规则生成协议数据
推理转发使用 `X-CPR-Route`,HTTP 和 WebSocket 握手规则相同。每个部署在转发给自定义 API Key 上游前追加自己的标记
推理转发使用 `X-CPR-Route`,HTTP 和 WebSocket 握手规则相同。每个部署追加自己的标记,仅向当前连接已识别的互联节点发送
(32 位小写十六进制值,多个标记以逗号分隔)。收到已含本部署标记的请求时返回 `508 peer_routing_loop`,不设固定跳数上限。
中间代理须保留该头;若有一段丢弃路径,则无法识别跨过该段的环路
普通或尚未识别的 API Key 上游、OAuth 及官方模型端点不携带该头;最后一个互联节点向模型上游发送时剥离。
重启后由后台恢复已有识别结果,加载完成前按未识别处理。
防环检测要求链路中的节点已完成互联识别,中间代理保留该头;若有一段丢弃路径,则无法识别跨过该段的环路。
插件自行实现转发或改写出站目标时,自行负责互联路径的保留、追加与最终剥离,见 [SDK 出站准备](../backend/crates/gateway-plugin/sdk/docs/capabilities.md#上游出站准备)
缺失、非法、禁用或删除的 Key 返回 `401`;未开启共享或没有显式分组返回 `403 peer_sharing_disabled`;
未知参数返回 `400 invalid_peer_query`;数量或响应大小超限、本地读取失败返回 `503 peer_snapshot_unavailable`。
@@ -1597,7 +1605,10 @@ Messages 上游追加 `messages`、`messages/count_tokens` 或 `models`。官方
HTTP 会明文传输 API Key 和请求内容,仅用于可信网络。
`transport` 缺省为 `http`,Responses 可设为 `prefer_websocket`,Messages 只接受 `http`。
Responses 使用 Bearer 认证,Messages 使用 `x-api-key` 与 `anthropic-version`,都不携带 OAuth Cookie 或 ChatGPT 身份。
上游模型列表从 API 前缀下的 `/models` 读取,按账号和凭据版本隔离;
上游模型列表从 API 前缀下的 `/models` 读取,按账号和凭据版本隔离。
Responses 账号使用服务端 Codex 画像的 `client_version` 请求目录,级联 CPR 因此能保留上游声明的思考档位。
普通 `data` 列表同样可用;上游返回 400 或 422 且明确以 `unknown_parameter` 或 `unsupported_parameter`
拒绝 `client_version` 时,不带该参数重试一次。
目录用于模型发现,不作为能力白名单,未列出的模型仍交由上游判断。
API Key 每次导入创建独立账号;更新已有账号使用 `/api/admin/accounts/update` 的 `connection` 字段。
导出保留协议和传输设置,并显式包含密钥,沿用敏感导出的确认合同
@@ -1946,7 +1957,7 @@ GET 和 POST 均返回 `enabled`、`leadMinutes`、`weeklyTriggerRemainingPercen
- 关闭:将 `priority`(含 `fast` 别名)改为显式 `default`,继续处理请求
策略只作用于 OpenAI Responses 的顶层 `service_tier`,不改变 `flex`、`ultrafast` 等其他显式档位、嵌套字段或其他 Provider。
选定 Codex OAuth 后,出站兼容层按其合同省略 `auto/default`,默认档位仍按标准价格计费;Responses API Key 保留这两个值。
OAuth 和 Responses API Key 共用上述档位规则,出站兼容层不额外删除原生 Responses 的 `auto/default`。
宿主策略在 request 中间件之后应用,形成 attempt 输入基线;attempt 中间件继续改写档位时,以改写后的实际发送值为准。
HTTP 和每个 WebSocket `response.create` 均使用请求开始时的分组策略,同一请求重试保持该策略;
HTTP 请求头及新建 WS 的握手提示按当时的最终出站档位构造;复用 WS 时不重发握手头,
@@ -2833,6 +2844,9 @@ OpenAI 推理强度优先取 `input` 中最后一条 `type: "configuration_updat
原生续接未再次更新时沿用已关联响应的配置,否则取请求顶层 `reasoning.effort`。
该字段是请求观测,不改写上游请求;网关未持有的续接历史无法用于推断档位
Messages 推理强度取请求顶层 `output_config.effort` 的非空字符串,保留客户端设置的档位。
未显式设置时保持未知,不根据 `thinking.budget_tokens` 或模型默认值推断;该记录不代表上游最终执行的档位
首页趋势、请求健康与热点诊断统一口径:错误为服务失败与未完成之和,成功率为 `成功 /(成功 + 错误)`
已恢复记录、客户端取消与调用方错误不计入,进行中不参与比例;调用方错误仅指失败终态的 4xx,不含取消请求的 499
无有效样本时,成功率与错误率返回 `null`,趋势线保留断点
+273 -195
View File
@@ -6,10 +6,13 @@
| 阅读目标 | 章节 |
| --- | --- |
| 了解模块与插件边界 | [运行拓扑](#2-运行拓扑)、[Workspace](#3-workspace-边界)、[插件扩展](#31-插件扩展)、[前端职责](#34-前端模块职责) |
| 跟踪一次请求 | [请求生命周期](#4-数据面请求生命周期)、[协议边界](#5-provider-与协议边界)、[路由与结算](#6-路由账号范围与-continuation) |
| 修改配置与持久化 | [控制面](#7-控制面与-revision)、[状态所有权](#8-状态所有权) |
| 维护运行行为 | [凭据与额度](#9-credential额度与主动重置)、[观测与任务](#10-观测与后台任务)、[生命周期](#11-生命周期安全与恢复) |
| 确定概念和修改位置 | [系统定位](#1-系统定位)、[Workspace](#3-workspace-边界)、[Core 内部职责](#33-gateway-core-内部-owner)、[修改与验收](#12-修改与验收) |
| 跟踪一次请求或增加协议字段 | [请求生命周期](#4-数据面请求生命周期)、[请求策略](#请求策略与事实)、[协议边界](#5-provider-与协议边界)、[路由与结算](#6-路由账号范围与-continuation) |
| 修改管理配置、导入或登录 | [控制面与发布](#7-控制面与-revision)、[后台导入](#后台账号导入)、[登录身份](#登录与查询身份) |
| 调整账号与上游行为 | [模型目录](#账号模型目录)、[出站代理](#账号出站代理)、[凭据与额度](#9-credential额度与主动重置) |
| 接入扩展或修改界面 | [插件扩展](#31-插件扩展)、[前端职责](#34-前端模块职责),作者合同与布局分别见 SDK 和主题文档 |
| 判断取消、重启和恢复的影响 | [状态与资源所有权](#8-状态所有权)、[启动与关闭](#启动与关闭)、[数据库恢复](#数据库恢复) |
| 解释记录、错误和后台任务 | [观测与任务](#10-观测与后台任务)、[错误边界](#错误与诊断三层边界)、[请求记录](#请求记录与交付) |
## 1. 系统定位
@@ -27,6 +30,16 @@ Responses、Chat Completions 和 Messages 共用生成执行链,API Key 账号
Client Key 限定账号分组,而不是绑定某个 Provider;一次请求的
Provider 候选由账号范围、模型能力和运行时健康共同决定
| 概念 | 所有者与含义 |
| --- | --- |
| Provider 身份 | Core 的 `ProviderKind` 标识已注册执行适配器,决定账号与执行能力归属 |
| 入站协议 | `gateway-protocol` 定义线协议标识和纯解析,API 标记输入格式;当前正文经过转换后由转换入口更新协议标识 |
| 认证方式 | Provider 解释账号 credential schema,并生成对应上游认证材料 |
| 上游协议 | Provider 根据所选账号的配置与能力决定实际发送格式,调用协议层转换器 |
Provider 注册名与协议标识分别维护,即使字符串相同也不互相代替。
`providers/openai` 执行兼容 API Key 账号的 Messages 请求,表示该适配器支持此账号与协议组合,不代表入站协议决定 Provider 身份
## 2. 运行拓扑
```mermaid
@@ -329,42 +342,15 @@ Provider 模型能力、目录代次与 `ProviderCatalogPort` 由 `routing::cata
Provider 可为模型提供来源账号集合;Core 在公开目录查询时结合冻结账号范围与政策过滤来源,
配置发布复用已缓存的来源事实。来源集合只约束目录展示,不直接作为推理账号白名单;Provider 独占上游资格判断
OpenAI 在 PostgreSQL 保存账号最近一次成功的模型目录、凭据版本、观测时间及 ETag。启动时批量恢复,按不同正文解析一次,保留每个账号的独立归属。没有历史目录也能发布路由快照;能力读取与推理不等待上游查询。
后台只刷新目录缺失、已满 30 分钟或收到新 ETag 的账号;凭据轮换后按新版本重新发现。每轮最多并发 32 个,新请求至少间隔 10 ms。失败保留最后成功的结果,过期资料只用于展示,不能据此排除候选账号
客户端原生模型目录同样由 Provider 拥有,通过 `ExecutionService` 和现有 Registry 的只读调用传递。
`routing::catalog` 的原生条目只包含模型 ID 与不透明协议正文,Core 负责冻结账号范围和整对象模型映射,
不解析上游字段;API 负责协议输出,不直接依赖具体 Provider。原生目录失败不会退回通用画像,只有不提供
该协议原生目录的 Provider(如 xAI 的 Codex 适配)才走画像转换。目录读取不创建计量请求或推理 attempt。
OpenAI 从账号目录生成原生列表;先按冻结政策选择来源,再读取本地账号状态核对凭据版本。同一份正文可以复用,但不会把一个账号的模型授予同套餐的其他账号。客户端版本和 Key 身份不触发额外发现
全局请求位置及开关由运行设置持久化;快照仅在开关开启时生成全局覆盖,并随
`RuntimeSnapshot → RoutingPlan → AttemptContext` 冻结传递,关闭时保留客户端原有字段;
Provider 在选定账号后应用代理位置覆盖。请求期间不额外查询全局设置,配置发布不改变已开始请求的全局值
隐私规则合同及编译端口由 Core 定义,OpenAI Provider 拥有路径、正则和承载结构的解释,Admin 的保存校验与样本预览复用同一实现。
Store 将策略与设置 revision 原子持久化;编译结果随快照、计划及 attempt 冻结,未改变策略的请求设置覆盖复用已编译对象。
规则在宿主请求头和 WS metadata 投影完成后作用于本次出站副本,重试及传输回退从原请求重新构造,避免重复替换。
每条规则跨承载副本原子提交;发送前拒绝携带 `NotSent` 和禁止重试标记。被规则改写的握手头参与 WS 池身份,防止复用旧值,
关闭策略不进入改写管线。字段与执行限制见[隐私策略 API](api.md#隐私策略)
Fast 策略由 Core 合并 Client Key 的全部绑定分组并随同一快照链路冻结,禁用分组也参与合并;
不按所选账号的分组重新解释。三态覆盖顺序与字段合同见[账号分组 API](api.md#6-账号分组)。
OpenAI Provider 在独立编码请求上、生成上游头与观测前统一应用顶层档位覆盖,HTTP、WS 与重试共用。
该策略不改变选号、会话亲和或共享原始请求;每个新 WS 请求重新取得当前策略。
WS 路由提示属于握手,连接复用时不重发;档位变化不重建连接或打断 `previous_response_id` 续写,
各帧正文及计费仍使用本次请求的最终档位
生图权限由 Core 随 Client Key 的账号范围冻结,绑定分组中禁止优先,禁用分组仍贡献限制。
执行入口在路由与准入前拒绝生图 operation,attempt 中间件提交最终 operation 时再次校验;
拒绝使用统一 `PolicyDenied`,发送前失败标记为 `NotSent` 且禁止重试,不改写原始工具声明。
识别复用 operation 的生图语义,跨协议翻译保留原始生图事实;普通图片输入不视为生图。
字段、默认值与协议边界见[账号分组 API](api.md#6-账号分组)
请求位置、隐私、Fast 和生图的跨层策略见[请求策略与事实](#请求策略与事实),Provider 目录的刷新与凭据隔离见[账号模型目录](#账号模型目录)
`engine::observation` 统一维护单次响应的用量、费用、时间和响应 ID,并负责重试前清理;协调器继续
独占发送、提交、重试和终结顺序。Provider 上报费用优先于本地估算,丢弃的 attempt 不得污染最终计量。
Provider 本地估算按当前 attempt 实际发送的上游模型查价,响应声明的模型只作观测;费用明细复用同一口径。
Client Key 费用账本独立累计各次 attempt 的实际费用,不能因请求重试而清空已产生的费用或未知计费状态
独占发送、提交、重试和终结顺序。响应观测与累计费用分别遵守[请求记录](#请求记录与交付)和[Client Key 结算](#client-key-限额与结算)的合同
### 3.4 前端模块职责
@@ -398,6 +384,16 @@ Key 创建与编辑共用 `useApiKeyEditor` 和 `ApiKeyFormModal`,`useApiKeyUs
可由草稿与保存快照比较得到的修改状态使用派生值,不另维护同步标志。
展示字段和业务资格沿用后端事实,数字与持续时长可以在前端格式化,日期与时区合同见 [API 页面时间](api.md#页面时间合同)
Vue 普通管理请求的错误提示由 `api/request/` 请求封装统一负责:优先展示安全信封的 `message`,
缺失或空白时才使用请求层的 HTTP、网络或超时兜底;成功业务码为 `200`,HTTP 成功但业务码失败也会拒绝。
规范化异常保留 `status`、`code`、`requestId` 与 `kind`。页面和 `useAsyncAction` 不重复弹出接口错误;
查询可以保留失败状态与重试入口,本地校验、文件操作、SSE 诊断和成功响应中的业务结果仍归各自 owner
取消或已被新查询取代的请求不弹提示;后台轮询、重启探测等显式使用 `silent`,它只关闭提示,
不吞异常,也不跳过会话失效处理。
批量操作的部分成功汇总、不可逆操作的结果未知等必要业务处理先将对应请求静默,再由业务 owner 提供
一次有上下文的反馈;不得为普通失败重新维护一套消息或业务码映射
## 4. 数据面请求生命周期
```mermaid
@@ -450,6 +446,7 @@ Client Key 鉴权完成后,API adapter 从有界请求头识别客户端,Cor
- API 在最终错误编码出口投影客户端恢复信号;该投影不修改 Provider 上游事实,也不改变 Core 的
重试与提交边界。具体错误合同见 [数据面接口](api.md#3-模型数据面与模型目录)
- Provider 为有界同账号退避显式选择耗尽动作:允许换号,或返回客户端错误;Core 在既有安全重放边界内执行,按账号维护请求内预算
上游 `x-retry-metadata: NO_MORE_RETRY` 时不执行这次同账号退避,直接采用已选择的耗尽动作
该退避消耗总路由预算,与 WS 传输恢复、OAuth 刷新及账号额度冷却分开;具体次数、退避与 `Retry-After` 规则见[容量拒绝与重试](api.md#容量拒绝与重试)
- `RetryThenReturnError` 的终态失败按 Client Key、逻辑线程与轮次写入 Redis,后续同轮次请求命中时跳过该策略的退避重试,直接交回错误;其他恢复策略不受影响
API 从明确的客户端推理请求提取轮次,Core 不解析协议正文;预热、压缩、内部调用和探测不参与该状态
@@ -466,7 +463,7 @@ Client Key 鉴权完成后,API adapter 从有界请求头识别客户端,Cor
以最后一个原始上游错误终态;配置与总尝试上限见[账号亲和与换号](api.md#账号亲和与换号)
- Provider 显式标记的终止拒绝禁止自动重试与传输回退,优先于未发送状态和安全重放证明
- 跨 Provider 只在账号范围和能力都允许,且请求尚未到达上游或已被证明可安全重放时发生
- 可恢复观测写入失败不能替换已经确定的客户端协议结果
- 观测写入失败不能替换已经确定的客户端协议结果
生成请求按模型目录编译候选;全局模型映射是精确映射,未命中时模型名原样交给候选 Provider。
Responses 与 Messages 保留各自的协议正文进入共用 Generate 执行链;Chat 在 API 层规范化为 Responses,
@@ -474,7 +471,7 @@ Responses 与 Messages 保留各自的协议正文进入共用 Generate 执行
实际 endpoint 保留客户端路由。Provider 选定账号后按账号协议转换出站,`outbound` 处理实际发送的上游请求
三个入口共用提交、取消与终结生命周期;Core 独占路由、重试、交付事实与结算,
Provider 独占上游身份、字段适配、隐私策略及账号隔离。`gateway-protocol` 只拥有结构转换和单次响应的流状态,
Provider 独占上游身份、账号相关适配与转换编排、隐私策略及账号隔离。`gateway-protocol` 拥有纯字段解析、结构转换和单次响应的流状态,
响应转换器只保留客户端模型名及还原工具调用所需的名称映射,不持有输入历史、账号、连接或调度状态。
Messages 与 Responses 的映射、忽略和降级规则参考 CPA,
同语义字段交给目标上游校验;结构解码或必要流状态失败按协议错误终结,不补造成功结果。
@@ -491,11 +488,42 @@ API 在响应期间发送控制帧,终态后还可读取原连接事件;空
控制句柄使用弱引用,不延长连接生命周期,也不经过全局响应查询、新选号或独立 attempt。
API 暂存的下一轮请求和连接泵接收队列共用容量名额,控制帧处理不取消中间件正在读取响应体的 future
### 请求策略与事实
`gateway-protocol` 从当前协议正文提取 `RequestFacts`,包括工具、模态、推理、结构化输出、原生续接和生图声明。
`GenerateRequest` 在构造或替换正文时重算事实,仅改上下文不重复扫描正文;跨协议转换保留来源请求的能力约束和生图事实。
Core 将这些事实转为路由能力要求与生图权限判断,协议解析不决定是否允许请求
全局请求位置及开关由运行设置持久化;快照仅在开关开启时生成全局覆盖,并随
`RuntimeSnapshot → RoutingPlan → AttemptContext` 冻结传递,关闭时保留客户端原有字段;
Provider 在选定账号后应用代理位置覆盖。请求期间不额外查询全局设置,配置发布不改变已开始请求的全局值
隐私规则合同及编译端口由 Core 定义,OpenAI Provider 拥有路径、正则和承载结构的解释,Admin 的保存校验与样本预览复用同一实现。
Store 将策略与设置 revision 原子持久化;编译结果随快照、计划及 attempt 冻结,未改变策略的请求设置覆盖复用已编译对象。
规则在宿主请求头和 WS metadata 投影完成后作用于本次出站副本,重试及传输回退从原请求重新构造,避免重复替换。
每条规则跨承载副本原子提交;发送前拒绝携带 `NotSent` 和禁止重试标记。被规则改写的握手头参与 WS 池身份,防止复用旧值,
关闭策略不进入改写管线。字段与执行限制见[隐私策略 API](api.md#隐私策略)
Fast 策略由 Core 合并 Client Key 的全部绑定分组并随同一快照链路冻结,禁用分组也参与合并;
不按所选账号的分组重新解释。三态覆盖顺序与字段合同见[账号分组 API](api.md#6-账号分组)。
OpenAI Provider 在独立编码请求上、生成上游头与观测前统一应用顶层档位覆盖,HTTP、WS 与重试共用。
该策略不改变选号、会话亲和或共享原始请求;每个新 WS 请求重新取得当前策略。
WS 路由提示属于握手,连接复用时不重发;档位变化不重建连接或打断 `previous_response_id` 续写,
各帧正文及计费仍使用本次请求的最终档位
生图权限由 Core 随 Client Key 的账号范围冻结,绑定分组中禁止优先,禁用分组仍贡献限制。
执行入口在路由与准入前拒绝生图 operation,attempt 中间件提交最终 operation 时再次校验;
拒绝使用统一 `PolicyDenied`,发送前失败标记为 `NotSent` 且禁止重试,不改写原始工具声明。
识别复用 operation 的生图语义,跨协议翻译保留原始生图事实;普通图片输入不视为生图。
字段、默认值与协议边界见[账号分组 API](api.md#6-账号分组)
## 5. Provider 与协议边界
Core 只理解 `Operation`、能力要求、Provider 候选、稳定错误和 canonical event,不读取 Provider SDK
类型。Provider 独占 credential schema、OAuth、账号选择、模型目录、额度投影和上游 transport
### 连接与上游身份
Responses 下游连接的心跳和退出由 API 拥有,每条 WS 使用独立取消信号,沿 Core 的 `ResponseControl` 传到 Provider 请求上下文。
单轮执行结束不取消连接生命周期,连接退出则取消当前执行及仍归它所有的上游资源;API 不直接调用具体 Provider 清池。
OpenAI 的连接池监督该取消信号,按当前归属保护绑定连接;精确续接接手 socket 时撤销旧控制收发并转移归属,旧客户端不能消费新响应或清理新 owner 的连接。
@@ -517,6 +545,8 @@ OpenAI Provider 统一处理 Responses 的账号归属与 turn state 隔离,HT
全量请求缺少本地归属记录时保留客户端回传的路由 token,明确换号或可信续接状态不属于同一轮时清除;
turn state 不参与选号,也不授权其他账号状态或 response ID 的复用
### 账号协议与字段适配
OpenAI 的 OAuth 与 API Key 共用现有账号和事务。API Key 的 Base URL、密钥、上游协议和传输策略属于 Provider 凭据 JSON,
随 credential revision 更新;普通详情只投影非敏感连接设置。API Key 目录按账号和凭据版本隔离,
标准 `data` 模型列表通过通用画像输出,完整 Codex `models` 目录保留原生对象。通用账号层按 Provider 提交的
@@ -582,50 +612,6 @@ xAI Provider 负责 Codex custom 工具与 Grok function 工具的双向转换
`call_id` 配对;超限或转换失败终止流。默认 `store: false` 的续接由现有会话 owner 重放完整历史;
原生续接按上游约束处理 `instructions` 与 `previous_response_id`,不把协议差异交给 Core
### 后台账号导入
Admin 拥有进程内导入任务、管理员归属、条目状态与有界队列,以 `account_import` Daemon 贡献给 Host。
API 只校验任务信封、生成完整输入摘要并投影安全结果,不解释 Provider 文档;每个执行条目复用已有
OpenAI / xAI 导入用例。并发槽位由单个 Worker 统一管理,条目状态是进度统计的唯一来源。
停止只跳过待执行输入;Host 关闭时停止接收新任务并在关闭预算内等待已开始条目完成。
完成或跳过即释放原始输入,终态记录有时限和数量上限,不写 PostgreSQL 或 Redis。
前端通过当前管理员的任务列表恢复视图,轮询与任务执行互不拥有生命周期。接口、限额与重启语义见
[后台导入任务](api.md#后台导入任务)
### 账号出站代理
出站代理属于账号配置,由各 Provider 在推理、OAuth 服务端交换/刷新、额度、目录和资料等请求中统一使用。
启用的指定代理不可用时请求失败,不退回直连;未配置或代理被禁用时直连,不继承进程的全局代理环境变量,TLS 仍验证证书。
用户浏览器访问第三方授权页时的出口不受网关设置控制
连接池按出口隔离。修改或启停代理推进配置 revision,使后续请求使用新出口;已执行请求可沿原连接完成。
代理变更不推进 `credential_revision`,不会使进行中的令牌刷新因凭据版本冲突而丢失结果。
xAI 的推理连接池继续按账号绑定隔离;OAuth、目录/计费辅助请求在各自 transport 内复用有界出口
client,OIDC 的 JWKS 缓存与单飞归属对应出口状态。凭据变更后由目录任务按账号当前出口重新发现;JWKS 过期或获取失败仍不使用 stale fallback
代理独立保存和测试,通过 `outboundProxyId` 绑定账号;账号保留解析后的 URL 供 Provider 使用。
出口测试结果属于诊断信息,不作为选择、授权或导入绑定的准入条件。
双栈出口并发检测,延迟对应首选可用出口的 HTTP 探测耗时(IPv4 优先),不包含另一地址族失败的等待;
检测仍等待两个地址族完成或超时,以保留双栈可用性结果。两者均失败时,延迟为整体检测耗时。
代理启停由代理记录独立持有;关闭时保留绑定、连接配置和位置规则,账号执行投影改为直连,
不使用该代理的位置覆盖,重新启用后恢复原出口。
连接配置改变时在同一事务内同步关联账号并清除旧测试结果。
删除代理在同一事务中校验版本、解除全部账号绑定并写入审计与配置 revision,账号其余事实及凭据版本保持不变。
关联账号支持单独移除:在原子更新中校验当前代理 ID,只清除账号的代理 ID 和 URL,
保留账号其他设置;提交配置版本与审计后复用快照发布流程,使后续请求使用直连。
代理目录只携带关联数量;账号明细通过独立分页读端口按需查询,
在同一个只读快照中统计并读取有界页面,不在目录中聚合完整账号数组。
文件及 AT/RT 导入在上游凭据交换前取得 PostgreSQL 会话级共享咨询锁,直到凭据提交后释放。
代理修改、删除和测试结果写入通过同一资源的事务级独占锁检查,导入期间直接返回冲突。
导入保护连接数量有界,不占用提交所需的连接池槽位,也不持有空闲事务。
请求取消或进程退出时关闭会话并释放锁。探测器由组合根注入 OpenAI 的 HTTP 客户端构建函数,
复用其自定义 CA 加载与证书校验规则,Host 不依赖具体 Provider 包
导入器在认证交换前解析并校验账号出口。sub2api 的代理引用按完整 URL 登记为共享代理并绑定账号;
其 SOCKS5 配置按远端 DNS 语义转换为 SOCKS5H。账号导入只迁移账号及出口,不迁移下游 Key、余额或历史用量。
代理认证信息仅通过敏感账号导出返回,列表、详情、Debug 和普通审计不得暴露;数据库及备份按凭据保护。
输入字段、协议支持与导入校验见 [账号 API](api.md#5-账号)
### Codex 原生生图与认证
管理端导出的 Codex 配置使用代理 Bearer 密钥,并声明服务端托管账号认证。
@@ -641,97 +627,6 @@ client,OIDC 的 JWKS 缓存与单飞归属对应出口状态。凭据变更后
以终态会话更新提示下游重连;API 负责确认成功写出和入站队列空闲。响应终态前的 Close 1000 仍视为失败,
后续恢复请求成功也不改写原失败请求的结果
### 错误与诊断三层边界
错误信息按用途分成三层,不能用同一个 `message` 同时承担协议、界面和诊断职责:
1. **数据面协议错误**:`/v1/*` 遵守调用入口的 wire 合同,Messages 使用 Anthropic 错误封套与 SSE error 事件。
可交付原始上游响应时保留其状态、headers、content type 和 body;原生续写额度恢复由 Provider 单独投影客户端响应,
真实上游事实仍用于诊断和记账。
本地 fallback 使用数据面稳定机器码与安全英文,不受管理端中文化影响
2. **控制面展示错误**:`/api/admin/*` 由 API 统一 HTTP 状态与数值业务码,由 Admin/API owner 提供安全中文
文案。extractor rejection、namespace 404 和 method 405 也使用同一 JSON 信封;任意 Store、Serde、
Provider `Display` 不得直接跨越 HTTP 边界
3. **受控错误详情**:运维错误详情分别保存本地原因链与上游响应,原始字段不翻译、不补造,
不进入普通 Admin 错误信封、Debug、普通日志或持久化错误摘要。连接测试只展示该接口允许的诊断投影
错误分类负责公共响应、重试和账号处置,原始上游 code 独立保存,不受已知分类白名单限制。
本地异常通过标准 `Error::source` 跨层传播;Core 的不透明来源载体保持底层类型,不依赖数据库或 HTTP 库,
普通格式化不展开来源。Store 统一基础设施不可用的转换,Provider 拥有业务分类,
插件 Runtime 集中投影 RPC、fault 与上游适配事件,调用点只附加原始来源与执行上下文
Core 的 `ErrorDetails` 统一投影受控详情,Provider、管理请求和后台操作复用该格式。
原始上游正文与本地原因链分开承载,主原因与附属清理失败合计最多保留 32 个节点、64 KiB 文本,
截断明确标记;显式回滚仍等待资源释放,回滚失败附加到主失败,不替换其分类与原因。
本地续写拒绝和 WS Close 派生分类不写入原始上游 code;关闭码与原因保存在关闭事实中。
请求 URL 等请求方向的敏感上下文在附加来源前移除并标记。错误快照保留来源与错误详情,
协议透传响应和未交付事件仍由当前请求独占。详情计入观测队列的内存预算;
队列满、关闭超时或写入失败仍可能丢失记录,沿用丢弃与写入失败指标,不自动重放不保证幂等的写入
管理请求在 API 出口按 request ID 记录一次内部失败,后台任务由 Host 监督器记录;
OAuth 刷新和准入释放等内部处理后继续运行的失败,由消费失败结果的 owner 记录。
这些操作通过 `OperationalDiagnostics` 复用现有有界观测队列与 `ops_events`,
不创建模型请求,也不把基础设施失败冒充成 Provider 上游响应。
OAuth 响应中的令牌字段和回显的刷新令牌在进入受控详情前脱敏,普通日志不展开正文。
日志由作出处置的边界记录恢复、放弃或终态;仅传播错误的层不重复打印。
传输收包使用 debug 摘要,普通日志使用稳定阶段、分类与 request/attempt 关联,不展开原始正文、
任意上游 code 或关闭原因。受控详情无法写入时不能回退到普通日志打印原文
Core 从执行错误生成一份失败诊断,展示与导出共用其中的分类;事件摘要截断时保留有界的失败诊断,
并显式标记截断。完整上游错误归运维错误记录;Admin 按故障目标与关联请求收集已存证据,
API 在认证后的下载边界统一移除凭据并生成诊断文件,Vue 只发起下载。
采集使用共享截止时间和记录上限,部分读取失败保留已有证据并报告缺口;当前环境与事发配置分别标记
Provider 管理适配在仍持有结构化失败事实时选择静态 `public_message`;Admin 将其转为可安全展示的
`AdminError.message`,API 保留其中的 502/503 具体原因,无公开消息时才使用通用回退。认证与未知
内部异常仍保持固定文案,不能把任意 Provider/Store 的 message 直接放行。
管理提示与 Worker 的处置分类是不同职责:例如 OpenAI 刷新已收到 401 时,可以提示已解析的令牌
拒绝原因,但不因此改变现有有界恢复退避和账号终态判定。刷新繁忙、已知上游失败与结果未知分别映射为
409、50201、50202;结果未知不能触发自动重放一次性凭据,Vue 也不重新解析上游错误码或正文
连接测试的 `gateway` / `provider` / `upstream` 来源以及 `not_sent` / `sent` / `ambiguous` 发送状态由 Core 在
仍持有完整执行错误时一次判定;Vue 只能根据稳定字段生成摘要,不能匹配英文错误句子反推来源
Vue 普通管理请求的错误提示由 `api/request/` 请求封装统一负责:优先展示安全信封的 `message`,
缺失或空白时才使用请求层的 HTTP、网络或超时兜底;成功业务码为 `200`,HTTP 成功但业务码失败也会拒绝。
规范化异常保留 `status`、`code`、`requestId` 与 `kind`。页面和 `useAsyncAction` 不重复弹出接口错误;
查询可以保留失败状态与重试入口,本地校验、文件操作、SSE 诊断和成功响应中的业务结果仍归各自 owner
取消或已被新查询取代的请求不弹提示;后台轮询、重启探测等显式使用 `silent`,它只关闭提示,
不吞异常,也不跳过会话失效处理。
批量操作的部分成功汇总、不可逆操作的结果未知等必要业务处理先将对应请求静默,再由业务 owner 提供
一次有上下文的反馈;不得为普通失败重新维护一套消息或业务码映射
### 登录与查询身份
管理员和密钥登录共用 `/api/auth/*`、AuthService、Redis 会话结构和 `cpr_session` Cookie。
登录类型只选择凭据校验方式,权限来自服务端保存的身份。管理入口只接受管理员身份或部署级管理 API Key;
AuthService 每次恢复 Key 会话时重新检查 Key 是否存在且启用;Key 会话不能访问管理员页面和管理接口。
前端只维护一份 Auth Store,不在每个 API 请求上标记身份;401 会话失效、403 权限不足和 503 依赖故障分别处理。
成功登录替换当前浏览器的旧会话,登出必须确认服务端撤销。
管理员访问凭据和续期凭据绑定同一逻辑会话,AuthService 负责签发、身份校验、轮换、不活动期限和最长有效期。
Store 保存凭据摘要与上一代重试所需的派生盐,原子比较后替换记录或撤销;Redis TTL 对齐续期期限,
访问到期不删除逻辑会话,刷新不能重建已退出或过期的键。短窗口重试返回同一代凭据,避免响应丢失与并发刷新互相撤销。
API 把管理员签发结果投影为访问与续期两种 HttpOnly Cookie;Key 会话仅使用访问 Cookie,保持固定期限。
前端首次进入受保护页面先读取状态,未认证时尝试续期;点击、滚动、聚焦等页面活动不单独触发续期;
认证 Cookie 写入在支持 Web Locks 的环境中跨标签页串行,其他环境在当前页面串行。
请求层仅在业务接口返回 `40101` 时共用一次会话续期,失败请求及恢复期间的新请求等待同一结果;
成功后原请求最多重放一次,取消或身份已切换的请求不重放,确认失效才退出。恢复接口跳过该协调以避免循环等待。
读取请求和明确幂等的续期请求遇到临时网络或服务故障时有限重试,写操作不默认执行此类重试。
首次恢复由路由守卫等待校验后放行;临时故障阻止导航,恢复联网或重新聚焦时再次校验,不增加恢复页面。
已有页面遇到临时故障不清除身份
管理员会话保存由已加盐密码哈希派生的指纹,每次恢复时与 PostgreSQL 当前密码核对;普通设置变更不影响该绑定。
改密在 AuthService 验证当前密码和新密码策略,Store 以旧哈希条件更新密码并在同一 PostgreSQL 事务记录审计。
事务提交后旧管理员会话的指纹失配,不依赖 Redis 批量删除完成撤销;原始密码及密码哈希不进入 Redis。
KeyUsageService 从 AuthService 的服务端身份或 Core 的入口认证结果确定唯一查询范围,复用 ClientKeyStore 的额度账本投影和
ObservabilityStore 的范围查询;数据面额度查询不执行推理准入或开启窗口,入口认证和适用的请求中间件仍会执行。
API 只输出各入口所需的字段白名单,不复用管理员的宽响应。
客户端配置通过 ClientKeyStore 显式读取当前会话绑定 Key 的明文,不进入用量响应。
前端 `/key-usage` 独立于管理布局,不挂载管理员菜单或请求管理接口;配置弹窗、Codex 配置及 CC Switch
导入链接生成逻辑与管理端共用,明文仅在打开弹窗时获取,关闭后清除,不持久化到浏览器
## 6. 路由、账号范围与 continuation
Client Key 与账号分组形成授权范围:
@@ -822,7 +717,8 @@ OpenAI Provider 解释官方逻辑 `session_id` 与 `thread_id`,按 Client Key
已有绑定的后代只跟随当前账号,账号不可用时排队,根迁移后重读共享绑定
可关联的 Search、Images、Live 创建请求共用会话绑定,宽松和优先模式不依赖线程身份分流;严格模式下后代请求保留跟随约束。
需要选号时统一使用配置的调度策略;插件显式选号由 Core 复核账号资格并取得租约,native continuation 受独立的状态 owner 与安全重放边界约束
内置 `round_robin` 在最高可调度权重层按账号 ID 轮换。游标按 Client Key 和 Provider 隔离,在取得租约后推进;有会话绑定时,两者原子提交。并发首绑只占一个位置,亲和命中和插件显式选号不推进游标。插件委托仍走内置轮询,优先模式已有绑定时沿用会话派生的后备顺序
插件显式选号由 Core 复核账号资格并取得租约,native continuation 受独立的状态 owner 与安全重放边界约束
OpenAI 固定归属请求直接读取目标账号,其余请求按冻结的账号 ID 范围查询 Store
原生会话的完整候选投影由 Provider 短时持有,按 Client Key、冻结范围、模型、调度策略、传输和容量池隔离
@@ -922,7 +818,8 @@ Admin 的手动重置复用同一账本与 Key 行锁,在一个事务中清零
普通准入通过同一数据库快照读取预算窗口与已落库请求,合并待结算费用并避免重复计算,不执行补账或等待结算行锁;
仅首次使用或窗口到期时开启窗口,仍与重置和结算使用相同锁序。预算准入总等待上限为一秒。
待结算费用达到 4096 笔后拒绝新准入,已准入请求的费用继续接收;缓存上限因此还包含在途请求的收尾费用。
进程退出后无法恢复的费用不会形成欠账。PostgreSQL 不可用或费用队列不可用时拒绝新的计费请求,Redis 继续管理并发/RPM 租约。
已接管但未落库的费用仅保存在进程内存;关闭排空超时或进程异常退出时可能丢失,重启不会恢复这部分费用或建立待核账记录。
PostgreSQL 不可用或费用队列不可用时拒绝新的计费请求,Redis 继续管理并发/RPM 租约。
预算窗口与费用事件按请求准入时的 Key ID 保存,独立于凭据和请求观测日志。删除 Key 后,在途结算锁定已开启的预算窗口,沿用请求 ID 去重。
日志清理不重置金额,已有日志不会回填为账本费用。
字段与错误合同见 [Client Key API](api.md#7-client-key)
@@ -949,7 +846,8 @@ Provider 是内置价目、服务档位、模态和工具费规则的唯一 owne
编译 RuntimeSnapshot 时合并同步与人工配置,Provider 缺省项仍由其内置表解释。不可变价格集合经
Arc 随 RoutingPlan 冻结并进入所有 attempt,不在推理请求中读取数据库、Redis 或外部价目,也不复制
整份价目。Provider 按实际发送模型选择配置;自定义倍率只作用于本地计算费用,上游报告金额不变
整份价目。Provider 按当前 attempt 实际发送的上游模型查价,响应声明的模型只作观测,费用明细复用同一口径;
自定义倍率只作用于本地计算费用,上游报告金额不变
本地计算费用携带当次拆分、有效单价和倍率,终态观测将其写入版本化费用快照。查询优先还原快照,
不使用新配置解释历史;无快照的旧记录保留总额核对后补充拆分的路径。Client Key 账本使用同一费用
@@ -993,21 +891,66 @@ PostgreSQL 周期对账才是正确性基础
新账号授权仍按当前配置发布,后续目录 generation 变化或刷新成功后更新模型事实。
成功返回空目录表示模型撤回,移除 Provider 或切换扩展集合时不复用旧归属
### 后台账号导入
Admin 拥有进程内导入任务、管理员归属、条目状态与有界队列,以 `account_import` Daemon 贡献给 Host。
API 只校验任务信封、生成完整输入摘要并投影安全结果,不解释 Provider 文档;每个执行条目复用已有
OpenAI / xAI 导入用例。并发槽位由单个 Worker 统一管理,条目状态是进度统计的唯一来源。
停止只跳过待执行输入;Host 关闭时停止接收新任务并在关闭预算内等待已开始条目完成。
完成或跳过即释放原始输入,终态记录有时限和数量上限,不写 PostgreSQL 或 Redis。
前端通过当前管理员的任务列表恢复视图,轮询与任务执行互不拥有生命周期。接口、限额与重启语义见
[后台导入任务](api.md#后台导入任务)
### 登录与查询身份
管理员和密钥登录共用 `/api/auth/*`、AuthService、Redis 会话结构和 `cpr_session` Cookie。
登录类型只选择凭据校验方式,权限来自服务端保存的身份。管理入口只接受管理员身份或部署级管理 API Key;
AuthService 每次恢复 Key 会话时重新检查 Key 是否存在且启用;Key 会话不能访问管理员页面和管理接口。
前端只维护一份 Auth Store,不在每个 API 请求上标记身份;401 会话失效、403 权限不足和 503 依赖故障分别处理。
成功登录替换当前浏览器的旧会话,登出必须确认服务端撤销。
管理员访问凭据和续期凭据绑定同一逻辑会话,AuthService 负责签发、身份校验、轮换、不活动期限和最长有效期。
Store 保存凭据摘要与上一代重试所需的派生盐,原子比较后替换记录或撤销;Redis TTL 对齐续期期限,
访问到期不删除逻辑会话,刷新不能重建已退出或过期的键。短窗口重试返回同一代凭据,避免响应丢失与并发刷新互相撤销。
API 把管理员签发结果投影为访问与续期两种 HttpOnly Cookie;Key 会话仅使用访问 Cookie,保持固定期限。
前端首次进入受保护页面先读取状态,未认证时尝试续期;点击、滚动、聚焦等页面活动不单独触发续期;
认证 Cookie 写入在支持 Web Locks 的环境中跨标签页串行,其他环境在当前页面串行。
请求层仅在业务接口返回 `40101` 时共用一次会话续期,失败请求及恢复期间的新请求等待同一结果;
成功后原请求最多重放一次,取消或身份已切换的请求不重放,确认失效才退出。恢复接口跳过该协调以避免循环等待。
读取请求和明确幂等的续期请求遇到临时网络或服务故障时有限重试,写操作不默认执行此类重试。
首次恢复由路由守卫等待校验后放行;临时故障阻止导航,恢复联网或重新聚焦时再次校验,不增加恢复页面。
已有页面遇到临时故障不清除身份
管理员会话保存由已加盐密码哈希派生的指纹,每次恢复时与 PostgreSQL 当前密码核对;普通设置变更不影响该绑定。
改密在 AuthService 验证当前密码和新密码策略,Store 以旧哈希条件更新密码并在同一 PostgreSQL 事务记录审计。
事务提交后旧管理员会话的指纹失配,不依赖 Redis 批量删除完成撤销;原始密码及密码哈希不进入 Redis。
KeyUsageService 从 AuthService 的服务端身份或 Core 的入口认证结果确定唯一查询范围,复用 ClientKeyStore 的额度账本投影和
ObservabilityStore 的范围查询;数据面额度查询不执行推理准入或开启窗口,入口认证和适用的请求中间件仍会执行。
API 只输出各入口所需的字段白名单,不复用管理员的宽响应。
客户端配置通过 ClientKeyStore 显式读取当前会话绑定 Key 的明文,不进入用量响应。
前端 `/key-usage` 独立于管理布局,不挂载管理员菜单或请求管理接口;配置弹窗、Codex 配置及 CC Switch
导入链接生成逻辑与管理端共用,明文仅在打开弹窗时获取,关闭后清除,不持久化到浏览器
## 8. 状态所有权
| 状态 | 唯一权威 | 说明 |
| --- | --- | --- |
| 账号、credential、分组、Client Key、设置、审计、请求与备份记录 | PostgreSQL | 业务持久化事实 |
| Client Key 金额窗口与费用事件 | PostgreSQL | 日历/显式窗口共用准入与结算账本,窗口版本及最近操作参数随窗口原子提交,独立于请求观测与日志保留策略 |
| CPR 互联共享权限与上游观测 | PostgreSQL | 权限随 Key 保存,当前快照复用账号额度字段,刷新排程属于运行期;原始观测时间与本地查询时间分别解释 |
| 插件包体、安装来源、制品接受事实、实例配置与私有状态 | PostgreSQL | 完整信任决定绑定已接受摘要;进程内发布集合由持久化事实构建 |
| admission、lease、cooldown、circuit、会话亲和、continuation、失败轮次、OAuth pending、目录 cache | Redis | 可重建、可过期的协调状态 |
| 控制面统一登录会话与登录限流桶 | Redis | AuthService 唯一拥有;保存 Admin / Key 身份、绑定 ID、绝对有效期和计数,不保存原始凭据 |
| 日志、OAuth 恢复记录、在线更新状态、备份暂存 | `.runtime/` | 部署节点本地运行文件 |
| 重置卡库存与消费结果 | 对应 Provider 的上游 | 本地到期计划不能证明卡片仍可用;手动查询直连上游,自动消费前重新核对 |
| 重置卡自动使用配置、发现时间与未决幂等键 | PostgreSQL 账号记录 | Admin 拥有计划与重试编排;账号删除时一并清理,重启沿用已保存时间及幂等键 |
| Provider 公开模型与官方发布资料 | Provider 的账号目录表或运行时缓存 | 由官方目录或发布源刷新,与 PostgreSQL 中的用户身份选择分别管理 |
| Windows 安装包临时直链 | Host 进程内短缓存 | 按需解析、严格校验、到期前丢弃;不写 PostgreSQL/Redis,也不代理包字节 |
状态的语义所有者与存储位置分别说明。已接管、已持久化和可重新查询具有不同保证,存储产品本身不决定恢复行为
| 状态 | 语义与执行所有者 | 存放位置 | 失效或退出后的处理 |
| --- | --- | --- | --- |
| 账号、credential、分组、Client Key、设置、审计与备份任务 | Admin 编排,Provider 解释凭据,Store 原子提交 | PostgreSQL | 已提交事实用于恢复配置;发布快照由这些事实构建 |
| 已接管但未落库的 Client Key 费用 | Core 提交结算,Store writer 持有待写队列 | Store 进程内存 | 运行期保留并重试,关闭时有界排空;未落库费用可能随进程退出丢失 |
| Client Key 金额窗口与已落库费用事件 | Core 定义预算与时序,Store 执行事务 | PostgreSQL | 按请求 ID 幂等登记,日历/显式窗口共用账本;窗口版本及最近操作参数原子提交,独立于请求观测与日志保留策略 |
| 请求、进度与运维观测 | Core 或相应操作所有者产生事实,Store 异步写入 | 有界内存队列与 PostgreSQL | 排队、写入或关闭失败可能丢失;已确定的客户端结果不随记录失败改变 |
| CPR 互联共享权限与上游观测 | Admin | PostgreSQL 的 Key 与账号额度记录 | 刷新排程属于运行期;原始观测时间与本地查询时间分别解释 |
| 插件包体、安装来源、制品接受事实、实例配置与私有状态 | Admin / Store | PostgreSQL | 信任绑定已接受摘要;Runtime 按持久化事实准备发布集合,不自动重放未提交迁移 |
| admission、lease、cooldown、circuit、亲和、continuation、失败轮次、OAuth pending、目录 cache | Core 或对应 Provider / Admin 定义各自合同,Store 执行原子操作 | Redis | 按所属合同续期、过期、重查或拒绝;状态丢失不统一解释为可无条件重建 |
| 控制面登录会话与登录限流桶 | Admin AuthService | Redis | 保存身份、绑定 ID、有效期和计数,不保存原始凭据;无法恢复有效会话时不据客户端 Cookie 重建身份 |
| 日志、OAuth 恢复记录、在线更新状态、备份暂存 | Host 或所属任务所有者 | `.runtime/` | 本地文件各自按写入与清理合同维护,不替代业务数据库 |
| 重置卡库存与消费结果 | 对应 Provider 的上游 | 上游服务 | 本地计划不能证明卡片仍可用,消费前重新核对 |
| 重置卡自动使用配置、发现时间与未决幂等键 | Admin 编排,Store 持久化 | PostgreSQL 账号记录 | 账号删除时清理,重启沿用已保存时间及幂等键 |
| Provider 公开模型与官方发布资料 | Provider | 账号目录表或运行时缓存 | 由官方来源刷新,与用户身份选择分开;过期目录的资格边界见[模型目录](#账号模型目录) |
| Windows 安装包临时直链 | Host | 进程内短缓存 | 按需解析、严格校验并在到期前丢弃,不持久化或代理包字节 |
资源寿命由实际持有者管理:Core 持有逻辑请求与 attempt,API 持有下游连接,Provider/transport 持有上游连接,Runtime 持有 RPC 调用资源。
Host 监督进程和后台任务,各所有者负责传播取消及收尾;连接重建不等于恢复原生续接状态。进程关闭策略见[启动与关闭](#启动与关闭)
账号对外状态不是独立列,而是 PostgreSQL credential/quota 事实与 Redis cooldown 的统一投影:
`normal`、`quota_paused`、`quota_exhausted`、`rate_limited`、`disabled`、`error`。只有明确上游证据才能恢复或终态化账号,
@@ -1017,7 +960,19 @@ PostgreSQL 周期对账才是正确性基础
PostgreSQL schema 由迁移目录按编号管理;冻结、升级与校验规则统一维护在[迁移文档](../backend/migrations/README.md)
## 9. Credential、额度与主动重置
<a id="9-credential额度与主动重置"></a>
## 9. 账号与上游状态
### 账号模型目录
OpenAI 在 PostgreSQL 保存账号最近一次成功的模型目录、凭据版本、观测时间及 ETag。启动时批量恢复,按不同正文解析一次,保留每个账号的独立归属。没有历史目录也能发布路由快照;能力读取与推理不等待上游查询。
后台只刷新目录缺失、已过刷新窗口或收到新 ETag 的账号;凭据轮换后按新版本重新发现。失败保留最后成功的结果,过期资料只用于展示,不能据此排除候选账号。
新鲜度窗口、刷新并发与节奏由 [Provider 目录实现](../backend/crates/providers/openai/src/credential/catalog/mod.rs)统一维护
OpenAI 从账号目录生成原生列表;先按冻结政策选择来源,再读取本地账号状态核对凭据版本。同一份正文可以复用,但不会把一个账号的模型授予同套餐的其他账号。客户端版本和 Key 身份不触发额外发现
### 凭据、额度与主动重置
credential 与 quota 是两组独立事实:credential refresh 不等于 quota refresh,额度接口的 401/403
也不能单独证明 refresh token 永久失效
@@ -1077,10 +1032,45 @@ Admin 仅为开启自动使用的账号保存卡片 ID、到期时间、下一
上游明确返回 `nothing_to_reset` 时清除未决键并保留卡片,下一次消费使用新键;未确认操作始终沿原键完成确认。
计划写入通过版本比较拒绝过期覆盖,全局使用时机、提前时间或周额度上限变更唤醒本地重算,关闭自动使用保留未决键供重新开启后恢复
### 账号出站代理
出站代理属于账号配置,由各 Provider 在推理、OAuth 服务端交换/刷新、额度、目录和资料等请求中统一使用。
启用的指定代理不可用时请求失败,不退回直连;未配置或代理被禁用时直连,不继承进程的全局代理环境变量,TLS 仍验证证书。
用户浏览器访问第三方授权页时的出口不受网关设置控制
连接池按出口隔离。修改或启停代理推进配置 revision,使后续请求使用新出口;已执行请求可沿原连接完成。
代理变更不推进 `credential_revision`,不会使进行中的令牌刷新因凭据版本冲突而丢失结果。
xAI 的推理连接池继续按账号绑定隔离;OAuth、目录/计费辅助请求在各自 transport 内复用有界出口
client,OIDC 的 JWKS 缓存与单飞归属对应出口状态。凭据变更后由目录任务按账号当前出口重新发现;JWKS 过期或获取失败仍不使用 stale fallback
代理独立保存和测试,通过 `outboundProxyId` 绑定账号;账号保留解析后的 URL 供 Provider 使用。
出口测试结果属于诊断信息,不作为选择、授权或导入绑定的准入条件。
双栈出口并发检测,延迟对应首选可用出口的 HTTP 探测耗时(IPv4 优先),不包含另一地址族失败的等待;
检测仍等待两个地址族完成或超时,以保留双栈可用性结果。两者均失败时,延迟为整体检测耗时。
代理启停由代理记录独立持有;关闭时保留绑定、连接配置和位置规则,账号执行投影改为直连,
不使用该代理的位置覆盖,重新启用后恢复原出口。
连接配置改变时在同一事务内同步关联账号并清除旧测试结果。
删除代理在同一事务中校验版本、解除全部账号绑定并写入审计与配置 revision,账号其余事实及凭据版本保持不变。
关联账号支持单独移除:在原子更新中校验当前代理 ID,只清除账号的代理 ID 和 URL,
保留账号其他设置;提交配置版本与审计后复用快照发布流程,使后续请求使用直连。
代理目录只携带关联数量;账号明细通过独立分页读端口按需查询,
在同一个只读快照中统计并读取有界页面,不在目录中聚合完整账号数组。
文件及 AT/RT 导入在上游凭据交换前取得 PostgreSQL 会话级共享咨询锁,直到凭据提交后释放。
代理修改、删除和测试结果写入通过同一资源的事务级独占锁检查,导入期间直接返回冲突。
导入保护连接数量有界,不占用提交所需的连接池槽位,也不持有空闲事务。
请求取消或进程退出时关闭会话并释放锁。探测器由组合根注入 OpenAI 的 HTTP 客户端构建函数,
复用其自定义 CA 加载与证书校验规则,Host 不依赖具体 Provider 包
导入器在认证交换前解析并校验账号出口。sub2api 的代理引用按完整 URL 登记为共享代理并绑定账号;
其 SOCKS5 配置按远端 DNS 语义转换为 SOCKS5H。账号导入只迁移账号及出口,不迁移下游 Key、余额或历史用量。
代理认证信息仅通过敏感账号导出返回,列表、详情、Debug 和普通审计不得暴露;数据库及备份按凭据保护。
输入字段、协议支持与导入校验见 [账号 API](api.md#5-账号)
### CPR 互联观测
推理转发路径随请求作用域继承。部署标记持久化于 PostgreSQL,同库副本共用,启动后从内存读取。
OpenAI Provider 向自定义 API Key 上游发送路径,WebSocket 连接池按路径隔离;入口发现本部署已出现时拒绝本次转发。
Admin 从当前连接的观测发布互联身份,Provider 经已有只读端口获取,只向已识别节点发送路径,向普通模型上游剥离。
身份判断不依赖额度新鲜度,连接变更立即使旧身份失效;WebSocket 连接池按路径隔离,入口发现本部署已出现时拒绝本次转发。
共享端由 Admin 汇聚已授权分组中的本地账号事实、Provider 只读额度和当前 Key 费用账本,API 负责版本化线协议输出。
导出同时核对请求冻结权限和当前共享设置、分组及认证方式,不回源查询,不输出账号凭据或本地账号标识。
@@ -1109,7 +1099,7 @@ OAuth 额度解析属于 Provider,CPR 发现、解析与投影属于 Admin;
互联解析只提交有效且明确的预算访问事实,额度同步不改变凭据或独立冷却
Admin 按连接身份及额度观察时间缓存管理摘要与调度投影,解析和投影在锁外完成。
Core 定义中立调度信号与只读端口,组合根绑定观测来源;Provider 选号不持有控制面服务,弱引用避免互相保活。
Core 定义互联身份、调度信号的只读端口,组合根绑定观测来源;Provider 不持有控制面服务,弱引用避免互相保活。
互联密钥的并发上限随有效观测参与普通账号选号和租约准入,账号手动配置优先。分组与概览按额度观测版本核对容量,账号配置与观测各自保存。
账号和凭据提交后的共用通知更新原生 Provider 与互联派生状态,插件账号写入复用同一流程;
互联按当前连接身份失效观测,普通信息编辑保留已有摘要与刷新退避,停用或删除时释放运行态。
@@ -1117,7 +1107,9 @@ Core 定义中立调度信号与只读端口,组合根绑定观测来源;Pro
界面复用现有额度窗口,保留上游原始观测时间;Key 金额、原生账号百分比和本机 Token 用量分别解释。
请求与字段合同见 [CPR 互联 API](api.md#cpr-互联只读协议)
## 10. 观测与后台任务
<a id="10-观测与后台任务"></a>
## 10. 观测、错误与后台任务
管理观测查询与统计 Worker 共用 PostgreSQL 连接预算,为请求执行保留连接容量。每项操作使用一个受预算约束的连接,排队计入总操作预算;
Store 声明配置后的读取时限,缓存沿用该值,统计事务只在事务内设置 SQL 时限,归还连接时不污染其他查询;
@@ -1168,6 +1160,21 @@ Provider 文档按账号和完成时间索引读取,每个采样槽只保留
### 请求记录与交付
逻辑请求观测由 Core 在初始化 `NewModelRequest` 时从当前 operation 组装,协议层统一解析通用字段。
这一阶段可能已包含 request 中间件改写,不自动等同于未经处理的客户端原文;实际出站与响应事实由所属 attempt 另行记录
| 观测来源 | 归属与合并方式 |
| --- | --- |
| 当前正文与协议上下文 | 协议层解析推理强度、预热与压缩等通用事实,不依赖首个候选 Provider |
| 端点模型与续接私有状态 | 端点模型由目标 Provider 补充;模型请求优先查询候选中持有原生会话状态的 Provider,否则按候选补充恢复关联;插件私有会话不作为原生历史配置 |
| 内部调用分类 | Core 从可信执行上下文确定,客户端同名字段不能覆盖 |
| 上游发送与响应 | Provider/transport 提供 attempt 事实,不反向覆盖成客户端原始声明 |
Provider 的补充不覆盖协议层通用解析。历史推理配置只作为协议解析的显式输入,配置更新、历史配置与顶层值的优先级,以及 Messages 的显式值口径,统一遵守 [API 记录口径](api.md#记录范围与统计口径)
重试前清理所属响应的用量、费用、计时与响应 ID,丢弃的 attempt 不得污染最终响应计量。
Client Key 账本独立累计各次 attempt 已取得的费用,不能因重试清空既有费用或未知计费状态
Provider 独占首事件、首字、首推理和首正文的观测,以及上游计时的协议解析,通过类型化响应观测交给 Core。
Core 只接收这些事实并记录请求总耗时,不从 canonical 事件补记输出计时;
未观测到的时间保持未知,各 Provider 的首字边界见 [记录范围与统计口径](api.md#记录范围与统计口径)
@@ -1211,7 +1218,7 @@ Core 负责路由、权限、取消、期限与准入拒绝。是否进入 Core
或响应头推断,也不在两个 owner 重复记录同一拒绝。入口 ID 关联 HTTP 运维事件,不冒充模型请求 ID。
异步投影仍可能延迟或丢弃,不能将“无记录”解释成没有发生错误
只有完整交付客户端的成功推理响应进入 Token、延迟和成本聚合。OpenAI Provider 根据请求的
只有完整交付客户端的成功推理响应进入 Token、延迟和成本聚合。Responses 协议解析根据请求的
`generate: false` 将连接与上下文准备归类为 `prewarm`,不信任客户端单独声明的同名 metadata。
Store 的共享用量口径排除这些预热记录,账号用量与额度预测复用同一规则;原始请求审计、响应额度
观测和费用事实仍保留,不将未知费用改写为零,也不影响 Client Key 结算账本。
@@ -1219,6 +1226,61 @@ OpenAI Responses 的统计档位与本地费用估算统一使用 Provider 最
不由响应回显覆盖;未发送档位时保留缺失值,展示与估算按标准档处理。上游响应档位独立保留在
Provider metadata 的 `upstreamServiceTier`,不改写客户端收到的响应,也不据此断言实际加速效果
### 错误与诊断三层边界
错误信息按用途分成三层,不能用同一个 `message` 同时承担协议、界面和诊断职责:
1. **数据面协议错误**:`/v1/*` 遵守调用入口的 wire 合同,Messages 使用 Anthropic 错误封套与 SSE error 事件。
可交付原始上游响应时保留其状态、headers、content type 和 body;原生续写额度恢复由 Provider 单独投影客户端响应,
真实上游事实仍用于诊断和记账。
本地 fallback 使用数据面稳定机器码与安全英文,不受管理端中文化影响
2. **控制面展示错误**:`/api/admin/*` 由 API 统一 HTTP 状态与数值业务码,由 Admin/API owner 提供安全中文
文案。extractor rejection、namespace 404 和 method 405 也使用同一 JSON 信封;任意 Store、Serde、
Provider `Display` 不得直接跨越 HTTP 边界
3. **受控错误详情**:运维错误详情分别保存本地原因链与上游响应,原始字段不翻译、不补造,
不进入普通 Admin 错误信封、Debug、普通日志或持久化错误摘要。连接测试只展示该接口允许的诊断投影
错误分类负责公共响应、重试和账号处置,原始上游 code 独立保存,不受已知分类白名单限制。
本地异常通过标准 `Error::source` 跨层传播;Core 的不透明来源载体保持底层类型,不依赖数据库或 HTTP 库,
普通格式化不展开来源。Store 统一基础设施不可用的转换,Provider 拥有业务分类,
插件 Runtime 集中投影 RPC、fault 与上游适配事件,调用点只附加原始来源与执行上下文
Core 的 `ErrorDetails` 统一投影受控详情,Provider、管理请求和后台操作复用该格式。
原始上游正文与本地原因链分开承载,主原因与附属清理失败合计最多保留 32 个节点、64 KiB 文本,
截断明确标记;显式回滚仍等待资源释放,回滚失败附加到主失败,不替换其分类与原因。
本地续写拒绝和 WS Close 派生分类不写入原始上游 code;关闭码与原因保存在关闭事实中。
请求 URL 等请求方向的敏感上下文在附加来源前移除并标记。错误快照保留来源与错误详情,
协议透传响应和未交付事件仍由当前请求独占。详情计入观测队列的内存预算;
队列满、关闭超时或写入失败仍可能丢失记录,沿用丢弃与写入失败指标,不自动重放不保证幂等的写入
管理请求在 API 出口按 request ID 记录一次内部失败,后台任务由 Host 监督器记录;
OAuth 刷新和准入释放等内部处理后继续运行的失败,由消费失败结果的 owner 记录。
这些操作通过 `OperationalDiagnostics` 复用现有有界观测队列与 `ops_events`,
不创建模型请求,也不把基础设施失败冒充成 Provider 上游响应。
OAuth 响应中的令牌字段和回显的刷新令牌在进入受控详情前脱敏,普通日志不展开正文。
日志由作出处置的边界记录恢复、放弃或终态;仅传播错误的层不重复打印。
传输收包使用 debug 摘要,普通日志使用稳定阶段、分类与 request/attempt 关联,不展开原始正文、
任意上游 code 或关闭原因。受控详情无法写入时不能回退到普通日志打印原文
Core 从执行错误生成一份失败诊断,展示与导出共用其中的分类;事件摘要截断时保留有界的失败诊断,
并显式标记截断。完整上游错误归运维错误记录;Admin 按故障目标与关联请求收集已存证据,
API 在认证后的下载边界统一移除凭据并生成诊断文件,Vue 只发起下载。
采集使用共享截止时间和记录上限,部分读取失败保留已有证据并报告缺口;当前环境与事发配置分别标记
Provider 管理适配在仍持有结构化失败事实时选择静态 `public_message`;Admin 将其转为可安全展示的
`AdminError.message`,API 保留其中的 502/503 具体原因,无公开消息时才使用通用回退。认证与未知
内部异常仍保持固定文案,不能把任意 Provider/Store 的 message 直接放行。
管理提示与 Worker 的处置分类是不同职责:例如 OpenAI 刷新已收到 401 时,可以提示已解析的令牌
拒绝原因,但不因此改变现有有界恢复退避和账号终态判定。刷新繁忙、已知上游失败与结果未知分别映射为
409、50201、50202;结果未知不能触发自动重放一次性凭据,Vue 也不重新解析上游错误码或正文
连接测试的 `gateway` / `provider` / `upstream` 来源以及 `not_sent` / `sent` / `ambiguous` 发送状态由 Core 在
仍持有完整执行错误时一次判定;Vue 只能根据稳定字段生成摘要,不能匹配英文错误句子反推来源
管理端的请求反馈、静默查询和批量操作提示由[前端请求层](#34-前端模块职责)持有,不在服务端重复定义界面交互
### Worker 与账号冻结
Worker 由各 Bundle 贡献、由 Host 统一监督:
@@ -1273,8 +1335,10 @@ Core 与 Store 在各自的探针上声明接收请求所需的依赖。API 的
不阻塞启动。失败时使用官方稳定地址,临时直链不持久化或代理下载,字段与来源规则见 [客户端下载 API](api.md#windows-客户端下载)
Host 对管理端重启、OS 关闭信号和内部取消使用统一策略:停止监听并拒绝新连接注册,
不等待存量会话,随后在独立预算内取消并等待 Worker 完成已入队观测写入和后台清理。
剩余连接随进程退出终止,超时后只丢弃仍未落盘的可恢复观测,不执行隐式业务重放。
不等待存量会话,随后在独立预算内取消并等待 Worker 排空队列和完成后台清理。
剩余连接随进程退出终止。观测队列和费用 writer 分别收尾,费用被 Store 接管不代表已写入 PostgreSQL;
排空超时、持续写入失败或进程异常退出时,未落库的观测和费用都可能丢失。
重启不恢复内存费用队列,也不执行隐式业务重放;费用接管与幂等写入边界见[Client Key 结算](#client-key-限额与结算)。
Compose 的 `stop_grace_period` 必须覆盖后台关闭预算并留出进程清理余量
### 凭据、日志与诊断
@@ -1345,8 +1409,22 @@ PostgreSQL 备份恢复属于人工维护操作。当前没有部署级维护模
## 12. 修改与验收
变更应落在拥有该事实的边界:协议适配进 API,执行策略进 Core,Provider 差异进对应 Provider,持久化
实现进 Store,生命周期进 Host,管理规则进 Admin。不要用兼容 shim、第二套状态机或跨层旁路绕开 owner
变更按事实与资源的所有者分配,不能仅按“协议”或“生命周期”等宽泛标签决定目录:
| 改动内容 | 所属边界 |
| --- | --- |
| 入口解码与规范化、客户端连接和协议交付 | API |
| 跨层共享的纯协议解析、能力事实与结构转换 | `gateway-protocol` |
| 路由、准入、重试、交付及结算时序 | Core |
| 所选账号的认证、上游身份、转换编排与实际传输 | Provider |
| 管理用例、校验与提交后发布 | Admin |
| 持久化、原子约束与存储适配 | Store |
| 进程启动关闭、受管资源与后台任务监督 | Host |
| 插件 RPC、调用关联与调用资源回收 | Runtime |
Chat 兼容入口在 API 中规范化为 Responses;Provider 在选号后按账号协议调用 `gateway-protocol` 的 Responses/Messages 转换器。
请求、连接和 RPC 的寿命由各自所有者管理。
状态归属与失败后的保证见[状态所有权](#8-状态所有权)。不以兼容 shim、第二套状态机或跨层旁路绕开已有合同
### 后端自审
+26
View File
@@ -75,6 +75,32 @@ HTTP 入口位于 `backend/crates/gateway-api/src/openai/messages.rs`,
再核对工具往返、取消、错误和输入计数;请求格式及六种组合见 [模型数据面](api.md#3-模型数据面与模型目录)。
Messages 账号只配置 HTTP;本地计数估算不能替代上游计费核对
TLS 指纹在发布目标对应的操作系统和架构上核验,参考程序使用同机运行的官方 Codex CLI 可执行文件。
`CODEX_PROXY_TEST_REFERENCE_BINARY` 指向实际二进制,Windows 使用 `.exe`,npm 安装时使用平台包中的二进制。
macOS / Linux 可设置:
```bash
export CODEX_PROXY_TEST_REFERENCE_BINARY=/absolute/path/to/codex
export RUST_MIN_STACK=16777216
```
Windows PowerShell 可设置:
```powershell
$env:CODEX_PROXY_TEST_REFERENCE_BINARY = 'C:\path\to\codex.exe'
$env:RUST_MIN_STACK = '16777216'
```
随后在仓库根目录执行:
```text
cargo +1.99.1 test --manifest-path backend/Cargo.toml --locked -p provider-openai --test main transport::tls:: -- --include-ignored --nocapture --test-threads=1
```
对照用例记录官方版本及运行平台,分别采样默认 HTTP、自定义 CA 的 HTTP 和 WebSocket,各重复三次。
默认 HTTP 保留扩展顺序逐项比较;rustls 路径对随机扩展顺序排序后比较,算法列表、密钥交换组和 ALPN 仍核对原有顺序。
采集在本机 TLS 握手阶段结束,使用测试认证值,无需真实账号。普通测试默认忽略官方二进制对照用例,须以上述命令显式执行
## 源码联调
宿主、组件库和官方插件保留独立仓库,通过 Git 子模块在同一目录开发:
+1 -1
View File
@@ -6,7 +6,7 @@ defineProps<{ label: string, content: string }>()
</script>
<template>
<ZPopover trigger="hover-click" placement="top-start" :hover-delay="240">
<ZPopover class="h-4 items-center" trigger="hover-click" placement="top-start" :hover-delay="240">
<template #reference="{ open }">
<ZIconButton variant="link" size="small" :aria-label="label" :aria-expanded="open" aria-haspopup="dialog">
<CircleAlert class="size-3.5" aria-hidden="true" />

Some files were not shown because too many files have changed in this diff Show More