Compare commits

...
10 Commits
Author SHA1 Message Date
ZyyandCodex 705f0fad06 chore(release): v3.22.2
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 18:11:24 +08:00
ZyyandCodex f0c60f501b fix(client): enable remote compaction in Codex config
Declare the V2 remote compaction capability in generated Codex configuration and deployment examples. Document that CC Switch imports do not preserve the full capability configuration.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 18:05:07 +08:00
ZyyandCodex d1febdd249 fix(openai): remove precommit byte limit
Keep large structural events buffered until semantic output, terminal completion, EOF, or the replay grace deadline. Retain byte counts for diagnostics and preserve historical byte_limit exports.

Refs #369

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 18:05:01 +08:00
ZyyandCodex 53d4086cc0 fix(openai): preserve opaque request fields and websocket messages
Forward numeric and extended reasoning effort values and preserve standalone endpoint business headers.

Retain business metadata and original WebSocket text through provider, middleware, and client delivery. Validate rewritten plugin output without rejecting unchanged heartbeats or opaque messages.

Add HTTP, WebSocket, and plugin process regressions for the passthrough boundaries.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 17:23:40 +08:00
ZyyandCodex ce03e8119d fix(diagnostics): retain health and xAI worker failure causes
Record health component failures through operational diagnostics with transition deduplication. Preserve quota and catalog failure context without interrupting the account batch.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 17:10:29 +08:00
ZyyandCodex 3f05b6450c fix(usage): preserve precommit facts in diagnostics export
Export allowlisted release reasons, prefetch bytes and wait times, and document diagnostics schema v4.

Refs #369

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 16:55:12 +08:00
ZyyandCodex a4b73099b5 fix(openai): pass through websocket control frames
Forward non-create text frames unchanged over the original upstream WebSocket while preserving response.create admission and serialization. Keep the control transport available after completion and relay idle upstream events without creating another inference attempt.

Cover unknown frames, late interrupts, terminal settlement, binding revocation, and connection reuse.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 16:26:31 +08:00
ZyyandCodex d875a62652 chore(release): v3.22.1
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 14:10:51 +08:00
ZyyandCodex 98704dfde8 fix(usage): parse response critical path timings
Read response-scoped engine and API overhead metrics while excluding logical-turn totals and incomplete observations. Validate nested response IDs and cover the observed payload over SSE and WebSocket.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 13:51:40 +08:00
ZyyandCodex b07bca312a fix(upgrade): handle long migrations and unify forced shutdown
Extend the startup healthcheck grace period to five minutes and report elapsed migration time every fifteen seconds without changing published migrations.

Use one shutdown path for OS signals, internal cancellation, and managed restarts. Stop listening without waiting for active sessions, preserve bounded worker cleanup, and remove the obsolete connection drain timeout.

Cover shutdown entry points and worker cleanup with real HTTP lifecycle tests and document the deployment behavior.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 13:51:14 +08:00
76 changed files with 2495 additions and 920 deletions
+1 -1
View File
@@ -1,4 +1,4 @@
//! 服务模式完成全部组装后交给 Host 监听,并在请求和 Worker 排空后关闭插件
//! 服务模式完成全部组装后交给 Host 监听,并在停止监听与 Worker 收尾后关闭插件
use gateway_host::ConfigError;
+89 -15
View File
@@ -1,10 +1,12 @@
//! `/healthz` 对 Core、Store 与 Host 关键 worker 健康事实的聚合
use std::sync::Arc;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use axum::{extract::State, http::StatusCode};
use futures::future::join_all;
use gateway_core::diagnostics::{OperationalDiagnostics, OperationalFailure};
use gateway_core::error::{ErrorDetails, ErrorSource};
use gateway_core::health::{
HealthProbe, HealthState, WorkerHealthKey, WorkerHealthSnapshot, WorkerHealthSource,
WorkerRuntimeState,
@@ -19,6 +21,16 @@ const HEALTH_CHECK_TIMEOUT: Duration = Duration::from_secs(2);
pub(crate) struct HealthStatus {
probes: Arc<[Arc<dyn HealthProbe>]>,
workers: Arc<dyn WorkerHealthSource>,
diagnostics: Arc<dyn OperationalDiagnostics>,
failures: Arc<Mutex<Vec<HealthFailure>>>,
}
#[derive(Clone, PartialEq, Eq, Debug, thiserror::Error)]
#[error("{component} is {state}: {reason}")]
struct HealthFailure {
component: String,
state: &'static str,
reason: String,
}
impl HealthStatus {
@@ -26,31 +38,93 @@ impl HealthStatus {
pub(crate) fn new(
probes: Vec<Arc<dyn HealthProbe>>,
workers: Arc<dyn WorkerHealthSource>,
diagnostics: Arc<dyn OperationalDiagnostics>,
) -> Self {
Self {
probes: probes.into(),
workers,
diagnostics,
failures: Arc::default(),
}
}
pub(crate) async fn healthy(&self) -> bool {
if !self
let mut failures: Vec<_> = self
.workers
.snapshot()
.iter()
.filter(|worker| worker_affects_healthz(worker))
.all(worker_is_healthy)
{
return false;
}
let checks = self.probes.iter().map(|probe| probe.check());
tokio::time::timeout(HEALTH_CHECK_TIMEOUT, join_all(checks))
.await
.is_ok_and(|states| {
states
.into_iter()
.all(|state| matches!(state, HealthState::Healthy))
.into_iter()
.filter(|worker| worker_affects_healthz(worker) && !worker_is_healthy(worker))
.map(|worker| HealthFailure {
component: match worker.key {
WorkerHealthKey::Task(id) => format!("worker:{id}"),
WorkerHealthKey::Disabled(kind) => format!("worker:{kind:?}"),
},
state: "unhealthy",
reason: format!(
"state={:?}; last_error={}",
worker.state,
worker.last_error.as_deref().unwrap_or("none")
),
})
.collect();
// 逐个探针共享截止时间,保留已完成结果并指出具体超时组件
let deadline = tokio::time::Instant::now() + HEALTH_CHECK_TIMEOUT;
let checks = self.probes.iter().map(|probe| async move {
let (state, reason) = match tokio::time::timeout_at(deadline, probe.check()).await {
Ok(HealthState::Healthy) => return None,
Ok(HealthState::Degraded(reason)) => ("degraded", reason),
Ok(HealthState::Unhealthy(reason)) => ("unhealthy", reason),
Err(_) => ("timed_out", "health probe exceeded 2s deadline".to_owned()),
};
Some(HealthFailure {
component: format!("probe:{}", probe.name()),
state,
reason,
})
});
failures.extend(join_all(checks).await.into_iter().flatten());
let healthy = failures.is_empty();
let changed = {
let mut previous = self
.failures
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let changed = failures
.iter()
.filter(|failure| !previous.contains(failure))
.cloned()
.collect::<Vec<_>>();
for recovered in previous
.iter()
.filter(|old| !failures.iter().any(|new| new.component == old.component))
{
tracing::info!(component = %recovered.component, "health component recovered");
}
*previous = failures;
changed
};
// 只记录故障变化;使用既有有界观测队列,文件日志失效时仍有独立的运维证据
for failure in changed {
tracing::warn!(component = %failure.component, state = failure.state, "health check failed");
let mut event = OperationalFailure::new(
"health",
"check",
"health_check_failed",
"Health check failed",
);
event.correlation_id = Some(failure.component.clone());
event.details =
ErrorDetails::capture(Some(&ErrorSource::new(failure.clone())), None, false);
if self.diagnostics.record_failure(event).await.is_err() {
tracing::warn!("health diagnostic could not be recorded");
// 入队失败不能抑制下次探测的记录机会,也不改变本次健康判定
self.failures
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.retain(|previous| previous != &failure);
}
}
healthy
}
}
+1 -1
View File
@@ -190,7 +190,7 @@ pub fn initialize(
let state = ApiState {
admin,
openai: OpenAiService::new(execution, lifecycle),
health: HealthStatus::new(probes, worker_health),
health: HealthStatus::new(probes, worker_health, diagnostics.clone()),
};
let index = config.asset_directory.join("index.html");
let mut router = Router::new()
@@ -95,15 +95,25 @@ impl OpenAiRequestHeaders {
}
}
/// 独立端点只消费会话身份与 turn metadata,不携带 Responses 的连接状态
/// 独立端点保留业务扩展头与会话身份,不携带 Responses 的连接状态
pub(crate) fn session_context(&self) -> Map<String, Value> {
let mut context = Map::new();
insert_protocol_context(&mut context, "session_id", self.session_id.as_ref());
insert_protocol_context(&mut context, "thread_id", self.thread_id.as_ref());
insert_protocol_context(&mut context, "turn_metadata", self.turn_metadata.as_ref());
self.insert_passthrough_headers(&mut context);
context
}
fn insert_passthrough_headers(&self, context: &mut Map<String, Value>) {
if !self.passthrough_headers.is_empty() {
context.insert(
PASSTHROUGH_HEADERS_CONTEXT_KEY.to_owned(),
Value::Array(self.passthrough_headers.clone()),
);
}
}
fn protocol_context(
&self,
use_websocket: Option<bool>,
@@ -153,12 +163,7 @@ impl OpenAiRequestHeaders {
insert_protocol_context(&mut context, "turn_id", self.turn_id.as_ref());
insert_protocol_context(&mut context, "responses_lite", self.responses_lite.as_ref());
insert_protocol_context(&mut context, "memgen_request", self.memgen_request.as_ref());
if !self.passthrough_headers.is_empty() {
context.insert(
PASSTHROUGH_HEADERS_CONTEXT_KEY.to_owned(),
Value::Array(self.passthrough_headers.clone()),
);
}
self.insert_passthrough_headers(&mut context);
if let Some(use_websocket) = use_websocket {
context.insert("use_websocket".to_owned(), Value::Bool(use_websocket));
}
@@ -47,7 +47,7 @@ impl OpenAiResponsesEncoder {
return Vec::new();
};
self.observe_wire(wire);
let projected = client_failure_payload(wire);
let projected = client_payload(wire);
let data = projected.as_ref().unwrap_or_else(|| wire.data());
// 当前 Codex 不消费 Responses `error` event,会在 EOF 时丢失失败原因
// 在客户端 SSE 边界统一投影成它能识别的 `response.failed`
@@ -70,22 +70,29 @@ impl OpenAiResponsesEncoder {
{
return vec![raw_sse_frame.clone()];
}
let data = if projected.is_none() {
wire.raw_websocket_message()
.map(str::to_owned)
.unwrap_or_else(|| data.to_string())
} else {
data.to_string()
};
vec![Bytes::from(encode_sse_event_with_metadata(
effective_event_type(wire).unwrap_or_default(),
&data.to_string(),
&data,
wire.sse_id(),
wire.sse_retry(),
))]
}
/// 消费一个 Provider event,并返回 WebSocket JSON messages
/// 消费一个 Provider event,并返回完整 WebSocket 文本消息
pub fn push_websocket(&mut self, event: &ProviderEvent) -> Vec<String> {
self.observe_canonical_identity(event);
let Some(wire) = openai_wire(event).filter(|wire| wire.has_json_data()) else {
let Some(wire) = openai_wire(event) else {
return Vec::new();
};
self.observe_wire(wire);
let projected = client_failure_payload(wire);
let projected = client_payload(wire);
let data = projected.as_ref().unwrap_or_else(|| wire.data());
// WS 客户端只对它无法消费的裸 `error` 帧做投影:codex 的 WS 端点
// 会静默忽略缺少 status 且不含内置可重试错误码的 `error` 帧,客户
@@ -104,6 +111,14 @@ impl OpenAiResponsesEncoder {
{
return vec![data.to_string()];
}
if projected.is_none()
&& let Some(raw) = wire.raw_websocket_message()
{
return vec![raw.to_owned()];
}
if !wire.has_json_data() {
return Vec::new();
}
vec![data.to_string()]
}
@@ -248,15 +263,43 @@ fn effective_event_type(wire: &ProtocolWireEvent) -> Option<&str> {
.or_else(|| wire.data().get("type").and_then(Value::as_str))
}
fn client_failure_payload(wire: &ProtocolWireEvent) -> Option<Value> {
fn client_payload(wire: &ProtocolWireEvent) -> Option<Value> {
if matches!(
effective_event_type(wire),
Some("error" | "response.failed")
) {
capacity_error_for_client(wire.data())
} else {
None
return capacity_error_for_client(wire.data());
}
if !matches!(
effective_event_type(wire),
Some("response.metadata" | "codex.response.metadata")
) {
return None;
}
let headers = wire.data().get("headers")?.as_object()?;
let connection_options: Vec<String> = headers
.iter()
.filter(|(name, _)| name.eq_ignore_ascii_case("connection"))
.flat_map(|(_, value)| match value {
Value::Array(values) => values.iter().filter_map(Value::as_str).collect::<Vec<_>>(),
_ => value.as_str().into_iter().collect(),
})
.flat_map(|value| value.split(','))
.map(|name| name.trim().to_ascii_lowercase())
.collect();
if headers
.keys()
.all(|name| super::response_header_is_forwardable(name, &connection_options))
{
return None;
}
// 只剥离既有响应头隔离策略禁止的字段,metadata 业务扩展保持不透明
let mut projected = wire.data().clone();
projected
.get_mut("headers")?
.as_object_mut()?
.retain(|name, _| super::response_header_is_forwardable(name, &connection_options));
Some(projected)
}
/// 判断 WS 上游的 `error` 帧是否已经是客户端可直接消费的形状
@@ -209,6 +209,11 @@ impl ResponsesDeliveryValidator {
transformed: bool,
facts: &ResponseValidationFacts,
) -> Result<(), ResponseValidationError> {
// 未改写的未知载荷不参与本地关联判断,不能因旁路解析失败截断原流
// 插件实际改写的载荷仍须满足交付合同
if parsed.is_err() && !transformed {
return Ok(());
}
if !self.active && !transformed {
if parsed.and_then(|frame| self.state.apply(frame)).is_err() {
self.invalid_prefix = true;
@@ -253,7 +258,7 @@ impl ResponsesDeliveryValidator {
return Err(ResponseValidationError);
}
}
ParsedFrame::Event { .. } | ParsedFrame::Done => {}
ParsedFrame::Event { .. } | ParsedFrame::Done | ParsedFrame::NonEvent => {}
}
self.state.apply(parsed)
}
@@ -292,6 +297,7 @@ impl ResponsesDeliveryValidator {
enum ParsedFrame {
Event { event_type: String, value: Value },
Done,
NonEvent,
}
impl ParsedFrame {
@@ -301,6 +307,10 @@ impl ParsedFrame {
return Ok(Self::Done);
}
let mut events = parse_sse_events(frame).map_err(|_| ResponseValidationError)?;
// 注释和仅含 SSE 元数据的帧不承载响应业务事实
if events.is_empty() {
return Ok(Self::NonEvent);
}
if events.len() != 1 {
return Err(ResponseValidationError);
}
@@ -363,6 +373,9 @@ struct DeliveryState {
impl DeliveryState {
fn apply(&mut self, frame: ParsedFrame) -> Result<(), ResponseValidationError> {
if matches!(frame, ParsedFrame::NonEvent) {
return Ok(());
}
if self.done {
return Err(ResponseValidationError);
}
@@ -12,7 +12,7 @@ use gateway_core::engine::middleware::{
MiddlewareBody, MiddlewareError, MiddlewareFrame, MiddlewareFraming, MiddlewareHeader,
MiddlewareResponse,
};
use gateway_core::engine::response_control::ResponseControl;
use gateway_core::engine::response_control::{ResponseControl, ResponseControlUnavailable};
use gateway_core::error::{GatewayError, GatewayErrorKind};
use gateway_core::event::ProviderResponseHeader;
use gateway_core::operation::ProviderSessionState;
@@ -25,7 +25,7 @@ use super::{
super::{DecodedResponsesRequest, OpenAiResponsesEncoder, ProtocolErrorBody},
connection::{ConnectionEvent, FramePhase, ResponsesWebSocketConnection, WriteContext},
protocol::{
decode_response_interrupt, error_event, initial_engine_error_event, response_metadata_event,
error_event, initial_engine_error_event, is_response_create, response_metadata_event,
},
};
@@ -137,7 +137,7 @@ pub(super) async fn forward_response(
replay: &mut ConnectionReplaySnapshot,
capture: ReplayCaptureHandle,
validation: ResponseValidationFacts,
response_control: ResponseControl,
response_control: &ResponseControl,
) -> ForwardOutcome {
let (protocol, status, headers, mut body, _) = response.into_parts();
if protocol != "openai" || StatusCode::from_u16(status).is_err() {
@@ -150,8 +150,14 @@ pub(super) async fn forward_response(
ProviderResponseHeader::new(name, value)
})
.collect::<Vec<_>>();
let first = match next_body_input(connection, body.as_mut(), &response_control, &request_id)
.await
let first = match next_body_input(
connection,
body.as_mut(),
response_control,
&request_id,
false,
)
.await
{
BodyInput::Frame(Ok(Some(frame))) => frame,
BodyInput::Frame(Ok(None)) => {
@@ -197,8 +203,14 @@ pub(super) async fn forward_response(
let frame = match current.take() {
Some(frame) => frame,
None => {
match next_body_input(connection, body.as_mut(), &response_control, &request_id)
.await
match next_body_input(
connection,
body.as_mut(),
response_control,
&request_id,
terminal_seen,
)
.await
{
BodyInput::Frame(Ok(Some(frame))) => frame,
BodyInput::Frame(Ok(None)) if terminal_seen && body.is_finalized() => {
@@ -284,8 +296,10 @@ fn detach_body(body: Box<dyn MiddlewareBody>) {
}
fn validate_frame(frame: &MiddlewareFrame) -> Result<(), MiddlewareError> {
if frame.framing() != MiddlewareFraming::JsonDocument
|| serde_json::from_slice::<serde::de::IgnoredAny>(frame.bytes()).is_err()
if !matches!(
frame.framing(),
MiddlewareFraming::JsonDocument | MiddlewareFraming::RawBytes
) || std::str::from_utf8(frame.bytes()).is_err()
{
return Err(MiddlewareError::InvalidState);
}
@@ -302,6 +316,7 @@ async fn next_body_input(
body: &mut dyn MiddlewareBody,
response_control: &ResponseControl,
request_id: &Arc<str>,
terminal_seen: bool,
) -> BodyInput {
// 处理控制帧时继续持有同一个读取 future,不能取消正在加工正文的中间件
let frame = body.next_frame();
@@ -312,19 +327,11 @@ async fn next_body_input(
let Some(event) = event else { return BodyInput::Disconnect; };
match &event.event {
ConnectionEvent::Text(payload) => {
let error = match decode_response_interrupt(payload) {
Ok(Some(response_id)) => response_control.interrupt(&response_id).err().map(|_| {
super::super::RequestDecodeError::InvalidValue { field: "response_id".to_owned() }.protocol_body()
}),
Ok(None) => {
connection.defer(event);
continue;
}
Err(error) => Some(error.protocol_body()),
};
if let Some(error) = error
&& send_protocol_error(connection, StatusCode::BAD_REQUEST, error, request_id).await == ForwardOutcome::Disconnect
{
if is_response_create(payload) {
connection.defer(event);
continue;
}
if send_control(connection, response_control, payload, request_id).await == ForwardOutcome::Disconnect {
return BodyInput::Disconnect;
}
}
@@ -337,6 +344,11 @@ async fn next_body_input(
ConnectionEvent::Exited(_) => return BodyInput::Disconnect,
}
}
event = response_control.receive(), if terminal_seen => {
if forward_control_event(connection, response_control, event).await == ForwardOutcome::Disconnect {
return BodyInput::Disconnect;
}
}
frame = &mut frame => return BodyInput::Frame(frame),
}
}
@@ -537,19 +549,26 @@ impl WebSocketExecutionBody {
self.transformed_pending = transformed;
continue;
}
messages.extend(encoded.into_iter().map(|message| (message, transformed)));
let framing = if event.wire_event().is_some_and(|wire| {
wire.raw_websocket_message().is_some() && !wire.has_json_data()
}) {
MiddlewareFraming::RawBytes
} else {
MiddlewareFraming::JsonDocument
};
messages.extend(
encoded
.into_iter()
.map(|message| (message, transformed, framing)),
);
self.transformed_pending = false;
}
let terminal = self.encoder.is_completed() || self.encoder.has_wire_failure();
let last = messages.len().saturating_sub(1);
self.pending.extend(messages.into_iter().enumerate().map(
|(index, (message, transformed))| {
MiddlewareFrame::new(
Bytes::from(message),
MiddlewareFraming::JsonDocument,
terminal && index == last,
)
.with_transformed(transformed)
|(index, (message, transformed, framing))| {
MiddlewareFrame::new(Bytes::from(message), framing, terminal && index == last)
.with_transformed(transformed)
},
));
self.awaiting_terminal_eof = terminal;
@@ -720,3 +739,46 @@ async fn send_metadata(
.await
.is_ok()
}
pub(super) async fn send_control(
connection: &mut ResponsesWebSocketConnection,
control: &ResponseControl,
payload: &str,
request_id: &Arc<str>,
) -> ForwardOutcome {
let result = tokio::select! {
_ = connection.wait_for_exit() => return ForwardOutcome::Disconnect,
result = control.send(payload) => result,
};
if result.is_err() {
return send_gateway_error(
connection,
&GatewayError::new(
GatewayErrorKind::InvalidRequest,
"No upstream WebSocket connection is available for control messages",
),
request_id,
)
.await;
}
ForwardOutcome::Continue
}
pub(super) async fn forward_control_event(
connection: &mut ResponsesWebSocketConnection,
control: &ResponseControl,
event: Result<String, ResponseControlUnavailable>,
) -> ForwardOutcome {
let Ok(payload) = event else {
control.clear();
return ForwardOutcome::Continue;
};
if connection
.send_text(payload, WriteContext::connection(FramePhase::Data))
.await
.is_err()
{
return ForwardOutcome::Disconnect;
}
ForwardOutcome::Continue
}
@@ -24,6 +24,7 @@ use gateway_core::{
engine::{
execution::{AuthenticatedClient, ClientTransport},
middleware::{FrozenMiddlewarePlan, MiddlewareError},
response_control::ResponseControl,
},
lifecycle::{ConnectionGuard, ConnectionLifecycle},
operation::OperationKind,
@@ -46,11 +47,12 @@ use super::{
};
use connection::{ConnectionEvent, FramePhase, ResponsesWebSocketConnection, WriteContext};
use forward::{
ConnectionReplaySnapshot, ForwardOutcome, execution_response, forward_response,
new_replay_capture, send_gateway_error, send_middleware_error, send_protocol_error,
ConnectionReplaySnapshot, ForwardOutcome, execution_response, forward_control_event,
forward_response, new_replay_capture, send_control, send_gateway_error, send_middleware_error,
send_protocol_error,
};
use protocol::connection_limit_event;
pub use protocol::{ResponseCreateFrameError, decode_response_create_with_context};
use protocol::{connection_limit_event, is_response_create};
const TEXT_FRAMES_ONLY: &str = "Responses WebSocket accepts text frames only";
const CONNECTION_LIMIT_CLOSE_REASON: &str = "Responses websocket connection limit reached";
@@ -191,10 +193,20 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
);
let mut request_count = 0_u64;
let mut replay = ConnectionReplaySnapshot::default();
let mut response_control = ResponseControl::default();
loop {
let Some(event) = connection.next_event().await else {
break;
let event = tokio::select! {
event = connection.next_event() => {
let Some(event) = event else { break };
event
}
event = response_control.receive() => {
if forward_control_event(&mut connection, &response_control, event).await == ForwardOutcome::Disconnect {
break;
}
continue;
}
};
let payload = match event {
ConnectionEvent::Text(payload) => payload,
@@ -212,8 +224,22 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
expire_connection(&mut connection).await;
break;
}
request_count = request_count.saturating_add(1);
let correlation_id = Arc::<str>::from(service.next_request_id());
if !is_response_create(&payload) {
if send_control(
&mut connection,
&response_control,
&payload,
&correlation_id,
)
.await
== ForwardOutcome::Disconnect
{
break;
}
continue;
}
request_count = request_count.saturating_add(1);
// 初步解码只保留路由事实,避免整份正文跨越准入等待和响应交付
let model_hint = match decode_response_create_with_context(&payload, &request_headers) {
Ok(decoded) => Some(decoded.metadata().requested_model().to_owned()),
@@ -251,6 +277,7 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
expire_connection(&mut connection).await;
break;
}
response_control.clear();
let execution = service.execution();
let preparation = async {
// 握手的显式改写可继承,Key 策略与宿主设置仍在每轮执行前刷新
@@ -291,7 +318,7 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
}
};
let request_id = Arc::<str>::from(prepared.request_id().to_string());
let response_control = prepared.response_control();
response_control = prepared.response_control();
let capture = new_replay_capture();
let validation = ResponseValidationFacts::default();
let input = RequestInput {
@@ -381,7 +408,7 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
&mut replay,
capture,
validation,
response_control,
&response_control,
)
.await
== ForwardOutcome::Disconnect
@@ -31,42 +31,22 @@ pub fn decode_response_create_with_context(
decode_response_create_inner(payload, request_headers)
}
/// 中断只引用活动响应;普通创建帧仍交给原有解码与串行准入路径
pub(super) fn decode_response_interrupt(
payload: &str,
) -> Result<Option<String>, ResponseCreateFrameError> {
/// 只识别需要本地准入的创建帧,其余文本不在网关解释
pub(super) fn is_response_create(payload: &str) -> bool {
#[derive(serde::Deserialize)]
struct FrameType {
#[serde(rename = "type")]
message_type: Option<String>,
message_type: String,
}
// 非控制帧仍在原来的串行请求边界校验,不能抢先拒绝排队的普通请求
// 只投影类型,避免为排队的大型 response.create 再构造完整 JSON 树
let Ok(frame) = serde_json::from_str::<FrameType>(payload) else {
return Ok(None);
};
if frame.message_type.as_deref() != Some("response.interrupt") {
return Ok(None);
match serde_json::from_str::<FrameType>(payload) {
Ok(frame) => frame.message_type == "response.create",
// struct 投影拒绝重复 type;沿用创建解码器的 JSON object 语义,
// 避免这类创建请求绕过准入进入透传路径
Err(_) => serde_json::from_str::<Value>(payload).is_ok_and(|frame| {
frame.get("type").and_then(Value::as_str) == Some("response.create")
}),
}
let value: Value =
serde_json::from_str(payload).map_err(|_| ResponseCreateFrameError::InvalidJson)?;
if value.get("mode").and_then(Value::as_str) != Some("discard_partial_items") {
return Err(ResponseCreateFrameError::Request(
RequestDecodeError::InvalidValue {
field: "mode".to_owned(),
},
));
}
let id = value
.get("response_id")
.and_then(Value::as_str)
.filter(|id| !id.is_empty())
.ok_or_else(|| {
ResponseCreateFrameError::Request(RequestDecodeError::InvalidValue {
field: "response_id".to_owned(),
})
})?;
Ok(Some(id.to_owned()))
}
fn decode_response_create_inner(
+215 -2
View File
@@ -1,6 +1,6 @@
//! 验证健康接口按基础设施与关键 Worker 状态判定可用性
use std::sync::Arc;
use std::sync::{Arc, Mutex};
use axum::{
body::Body,
@@ -13,12 +13,17 @@ use gateway_core::{
StartProviderExecution, StartedExecution,
},
error::GatewayError,
health::{WorkerHealthKey, WorkerHealthSnapshot, WorkerHealthSource, WorkerRuntimeState},
health::{
HealthProbe, HealthState, WorkerHealthKey, WorkerHealthSnapshot, WorkerHealthSource,
WorkerRuntimeState,
},
routing::PublicModelId,
task::{WorkerId, WorkerKind},
};
use tower::ServiceExt;
use crate::support::RecordingDiagnostics;
#[tokio::test]
async fn healthz_should_return_no_content_when_all_inputs_are_healthy() {
let response = crate::openai::api_router(Arc::new(UnusedExecution))
@@ -81,6 +86,214 @@ async fn healthz_should_reject_critical_worker_failure() {
assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE);
}
#[tokio::test]
async fn healthz_records_probe_changes_once_and_records_recurrence_after_recovery() {
let probe = Arc::new(MutableProbe(Mutex::new(HealthState::Unhealthy(
"file logging rejected records".to_owned(),
))));
let diagnostics = Arc::new(RecordingDiagnostics::default());
let router = health_router(vec![probe.clone()], Vec::new(), diagnostics.clone()).await;
let responses =
futures::future::join_all((0..8).map(|_| health_response(router.clone()))).await;
for response in responses {
assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE);
assert!(
axum::body::to_bytes(response.into_body(), 1024)
.await
.unwrap()
.is_empty()
);
}
{
let events = diagnostics.0.lock().unwrap();
assert_eq!(events.len(), 1);
assert_eq!(events[0].component, "health");
assert_eq!(
events[0].correlation_id.as_deref(),
Some("probe:file_logging")
);
assert_eq!(events[0].message, "Health check failed");
assert!(
events[0]
.details
.as_ref()
.unwrap()
.as_str()
.contains("file logging rejected records")
);
}
*probe.0.lock().unwrap() = HealthState::Degraded("file logging maintenance failed".to_owned());
assert_eq!(
health_response(router.clone()).await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert_eq!(diagnostics.0.lock().unwrap().len(), 2);
*probe.0.lock().unwrap() = HealthState::Healthy;
assert_eq!(
health_response(router.clone()).await.status(),
StatusCode::NO_CONTENT
);
*probe.0.lock().unwrap() = HealthState::Degraded("file logging maintenance failed".to_owned());
assert_eq!(
health_response(router).await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert_eq!(diagnostics.0.lock().unwrap().len(), 3);
}
#[tokio::test(start_paused = true)]
async fn healthz_retains_worker_and_completed_probe_failures_when_another_probe_times_out() {
let diagnostics = Arc::new(RecordingDiagnostics::default());
let router = health_router(
vec![
Arc::new(MutableProbe(Mutex::new(HealthState::Unhealthy(
"file output failed".to_owned(),
)))),
Arc::new(PendingProbe),
],
vec![
worker_snapshot(
WorkerKind::RuntimeSnapshotReconciliation,
WorkerRuntimeState::BackingOff,
),
worker_snapshot(
WorkerKind::QuotaCatalogHealth,
WorkerRuntimeState::BackingOff,
),
],
diagnostics.clone(),
)
.await;
assert_eq!(
health_response(router).await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
let events = diagnostics.0.lock().unwrap();
assert_eq!(events.len(), 3);
for (component, reason) in [
(
"worker:runtime_snapshot_reconciliation:health-test",
"test worker failure",
),
("probe:file_logging", "file output failed"),
("probe:pending", "timed_out"),
] {
let event = events
.iter()
.find(|event| event.correlation_id.as_deref() == Some(component))
.expect(component);
assert!(event.details.as_ref().unwrap().as_str().contains(reason));
}
}
#[tokio::test]
async fn healthz_retries_failed_diagnostic_admission_without_changing_health_result() {
let diagnostics = Arc::new(FailingOnceDiagnostics::default());
let router = health_router(
vec![Arc::new(MutableProbe(Mutex::new(HealthState::Unhealthy(
"file output failed".to_owned(),
))))],
Vec::new(),
diagnostics.clone(),
)
.await;
assert_eq!(
health_response(router.clone()).await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert!(diagnostics.recorded.0.lock().unwrap().is_empty());
assert_eq!(
health_response(router).await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert_eq!(diagnostics.recorded.0.lock().unwrap().len(), 1);
}
#[derive(Default)]
struct FailingOnceDiagnostics {
failed: std::sync::atomic::AtomicBool,
recorded: RecordingDiagnostics,
}
#[async_trait::async_trait]
impl gateway_core::diagnostics::OperationalDiagnostics for FailingOnceDiagnostics {
async fn record_failure(
&self,
failure: gateway_core::diagnostics::OperationalFailure,
) -> Result<(), gateway_core::error::StoreError> {
if !self.failed.swap(true, std::sync::atomic::Ordering::SeqCst) {
return Err(gateway_core::error::StoreError::new(
gateway_core::error::StoreErrorKind::Unavailable,
));
}
self.recorded.record_failure(failure).await
}
}
async fn health_router(
probes: Vec<Arc<dyn HealthProbe>>,
workers: Vec<WorkerHealthSnapshot>,
diagnostics: Arc<dyn gateway_core::diagnostics::OperationalDiagnostics>,
) -> axum::Router {
let admin = crate::admin::AdminTestFixture::new().await;
gateway_api::initialize(
gateway_api::ApiConfig {
asset_directory: std::env::temp_dir(),
cors_allowed_origins: Vec::new(),
request_timeout_seconds: None,
request_id_header: "x-request-id".to_owned(),
},
Arc::new(UnusedExecution),
admin.services,
probes,
Arc::new(StaticWorkerHealth(workers)),
Arc::new(crate::openai::TestLifecycle::default()),
diagnostics,
)
.unwrap()
.router()
}
async fn health_response(router: axum::Router) -> axum::response::Response {
router
.oneshot(
Request::builder()
.uri("/healthz")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap()
}
struct MutableProbe(Mutex<HealthState>);
impl HealthProbe for MutableProbe {
fn name(&self) -> &'static str {
"file_logging"
}
fn check(&self) -> BoxFuture<'_, HealthState> {
Box::pin(std::future::ready(self.0.lock().unwrap().clone()))
}
}
struct PendingProbe;
impl HealthProbe for PendingProbe {
fn name(&self) -> &'static str {
"pending"
}
fn check(&self) -> BoxFuture<'_, HealthState> {
Box::pin(std::future::pending())
}
}
#[derive(Clone)]
struct StaticWorkerHealth(Vec<WorkerHealthSnapshot>);
@@ -10,6 +10,7 @@ use axum::{
body::{Body, to_bytes},
http::{Request, StatusCode, header::AUTHORIZATION},
};
use base64::{Engine as _, engine::general_purpose::STANDARD};
use bytes::Bytes;
use futures::future::BoxFuture;
use gateway_core::engine::execution::{
@@ -300,6 +301,11 @@ async fn image_routes_should_not_decode_bodies_and_should_preserve_both_directio
.header("x-codex-image-turn-id", "turn_image_route")
.header("session-id", "root-image-session")
.header("thread-id", "child-image-thread")
.header("x-future-business", "first")
.header("x-future-business", "second")
.header("cookie", "client-secret")
.header("connection", "x-private-hop")
.header("x-private-hop", "client-hop")
.body(Body::from(body.to_vec()))
.expect("image request"),
)
@@ -338,8 +344,37 @@ async fn image_routes_should_not_decode_bodies_and_should_preserve_both_directio
assert_eq!(captured.transport, ClientTransport::HttpJson);
assert_eq!(captured.kind, *kind);
assert_eq!(captured.body.as_ref(), *body);
let mut context = captured.context.clone();
let headers = context
.as_object_mut()
.unwrap()
.remove("opaque_request_headers")
.unwrap();
let headers: Vec<_> = headers
.as_array()
.unwrap()
.iter()
.map(|entry| {
(
entry[0].as_str().unwrap(),
STANDARD.decode(entry[1].as_str().unwrap()).unwrap(),
)
})
.collect();
assert_eq!(
captured.context,
headers
.iter()
.filter(|(name, _)| *name == "x-future-business")
.map(|(_, value)| value.as_slice())
.collect::<Vec<_>>(),
[b"first".as_slice(), b"second".as_slice()]
);
assert!(!headers.iter().any(|(name, _)| matches!(
*name,
"authorization" | "cookie" | "connection" | "x-private-hop"
)));
assert_eq!(
context,
json!({"image_turn_id": "turn_image_route", "session_id": "root-image-session", "thread_id": "child-image-thread"})
);
}
@@ -323,7 +323,7 @@ fn snapshot_with_client_key(
}
#[derive(Default)]
struct TestLifecycle {
pub(crate) struct TestLifecycle {
cancellation: gateway_core::lifecycle::CancellationToken,
}
@@ -3022,3 +3022,67 @@ async fn compressed_http_request_above_default_limit_should_reach_execution_afte
}
}
}
#[tokio::test]
async fn streaming_middleware_preserves_heartbeats_before_and_after_rewriting() {
for opaque in [
b": ping\n\n".as_slice(),
b"id: next\nretry: 1000\n\n".as_slice(),
b"event: vendor.future\ndata: future non-JSON payload\n\n".as_slice(),
] {
let admin = crate::admin::AdminTestFixture::new().await.services;
let trace = Arc::new(Trace::default());
let heartbeat = Bytes::copy_from_slice(opaque);
let future = Bytes::from_static(
b"event: response.future\ndata: {\"type\":\"response.future\",\"extension\":true}\n\n",
);
let session = FakeSession::streaming(
Arc::clone(&trace),
vec![
NextStep::Event(delivery(started(), CommitRequirement::CommitBeforeDelivery)),
NextStep::Event(delivery_provider(
ProviderEvent::wire(
ProtocolWireEvent::raw_sse("openai", heartbeat.clone()).unwrap(),
),
CommitRequirement::AlreadyCommitted,
)),
NextStep::Event(delivery_provider(
ProviderEvent::wire(
ProtocolWireEvent::raw_sse("openai", future.clone()).unwrap(),
),
CommitRequirement::AlreadyCommitted,
)),
NextStep::Event(delivery_provider(
ProviderEvent::wire(
ProtocolWireEvent::raw_sse("openai", heartbeat.clone()).unwrap(),
),
CommitRequirement::AlreadyCommitted,
)),
NextStep::Event(delivery(completed(), CommitRequirement::AlreadyCommitted)),
NextStep::FinalizeSuccess,
],
);
let replacement = Bytes::from_static(b"event: response.future\ndata: {\"type\":\"response.future\",\"extension\":\"changed\"}\n\n");
let middleware = Arc::new(responses_middleware().with_response_actions(vec![
ResponseFrameAction::Continue,
ResponseFrameAction::Continue,
ResponseFrameAction::Replace(replacement.clone()),
ResponseFrameAction::Continue,
ResponseFrameAction::Continue,
ResponseFrameAction::Continue,
]));
let response =
response_with_middleware(&admin, "openai", session, true, Some(middleware)).await;
let body = to_bytes(response.into_body(), usize::MAX).await.unwrap();
let text = std::str::from_utf8(&body).unwrap();
assert_eq!(
text.matches(std::str::from_utf8(opaque).unwrap()).count(),
2
);
assert!(text.contains(std::str::from_utf8(&replacement).unwrap()));
assert!(text.contains("response.completed"));
assert!(!text.contains("middleware_failed"));
assert!(text.ends_with("data: [DONE]\n\n"));
assert!(!trace.is_cancelled());
}
}
@@ -1071,3 +1071,35 @@ fn transparent_encoder_should_only_require_wire_terminal_for_buffered_conversion
ResponseEncodeError::MissingWireTerminal
);
}
#[test]
fn metadata_keeps_business_extensions_and_filters_only_protected_headers() {
let data = json!({
"type":"response.metadata",
"headers":{"x-future-header":["first","second"],"authorization":"private","set-cookie":"private","chatgpt-account-id":"private","connection":"x-private-hop","x-private-hop":"private"},
"metadata":{"type":"safety_buffering","use_cases":["cyber"],"reasons":["user_risk"],"openai_verification_recommendation":{"future":true},"openai_chatgpt_moderation_metadata":{"future":true}},
"future_extension":{"keep":true}
});
let raw = data.to_string();
let event = ProviderEvent::wire(
ProtocolWireEvent::json("openai", Some("response.metadata".to_owned()), data.clone())
.unwrap()
.with_raw_websocket_message(raw.as_str()),
);
let mut encoder = OpenAiResponsesEncoder::new();
let ws = encoder.push_websocket(&event);
let delivered: Value = serde_json::from_str(&ws[0]).unwrap();
assert_eq!(delivered["metadata"], data["metadata"]);
assert_eq!(delivered["future_extension"], data["future_extension"]);
assert_eq!(
delivered["headers"],
json!({"x-future-header":["first","second"]})
);
let sse = encoder.push_sse(&event);
assert!(!std::str::from_utf8(&sse[0]).unwrap().contains("private"));
assert!(
std::str::from_utf8(&sse[0])
.unwrap()
.contains("safety_buffering")
);
}
@@ -106,10 +106,18 @@ async fn next_error(socket: &mut TestSocket) -> Value {
}
async fn assert_no_duplicate_failure(socket: &mut TestSocket) {
// 后续入站帧只会在本轮收敛后处理,比固定等待窗口更可靠地检查没有重复失败终态
socket.send(Message::Text("{".into())).await.unwrap();
// 无效创建帧仍走串行队列,以本地业务校验作为本轮收敛后的屏障
socket
.send(Message::Text(
json!({"type":"response.create","model":"model-a","stream":false})
.to_string()
.into(),
))
.await
.unwrap();
let error = next_event(socket).await;
assert_eq!(error["error"]["code"], "invalid_json");
assert_eq!(error["error"]["code"], "invalid_value");
assert_eq!(error["error"]["param"], "stream");
assert_eq!(error["status"], 400);
assert_eq!(error["headers"]["x-request-id"], error["request_id"]);
assert_eq!(
@@ -1,12 +1,70 @@
//! 验证 Responses WebSocket 中断向当前执行传递并释放相关资源
//! 验证未知控制消息、迟到中断、结算期间收发和创建请求的串行准入
use super::*;
use gateway_core::engine::response_control::{ResponseControl, ResponseInterruptError};
use gateway_core::engine::response_control::{
ResponseControlTransport, ResponseControlUnavailable,
};
struct ControlTransport {
id: String,
active: std::sync::atomic::AtomicBool,
requests: Mutex<Vec<String>>,
sender: tokio::sync::mpsc::Sender<Value>,
receiver: tokio::sync::Mutex<tokio::sync::mpsc::Receiver<Value>>,
}
impl ControlTransport {
fn new(id: String) -> Self {
let (sender, receiver) = tokio::sync::mpsc::channel(32);
Self {
id,
active: std::sync::atomic::AtomicBool::new(true),
requests: Mutex::default(),
sender,
receiver: tokio::sync::Mutex::new(receiver),
}
}
}
#[async_trait]
impl ResponseControlTransport for ControlTransport {
async fn send(&self, payload: &str) -> Result<(), ResponseControlUnavailable> {
self.requests.lock().unwrap().push(payload.to_owned());
let original: Value =
serde_json::from_str(payload).unwrap_or_else(|_| Value::String(payload.to_owned()));
let event = if original["type"] == "future.reject" {
upstream_control_error()
} else if original["type"] == "response.interrupt"
&& original["response_id"] == self.id
&& original["mode"] == "discard_partial_items"
&& self.active.load(Ordering::SeqCst)
{
json!({"type":"response.incomplete", "response":{"id":self.id,"model":"model-a","status":"incomplete","output":[],"incomplete_details":{"reason":"interrupted"}}})
} else {
json!({"type":"future.control.ack", "original":original})
};
self.sender
.send(event)
.await
.map_err(|_| ResponseControlUnavailable)
}
async fn receive(&self) -> Result<String, ResponseControlUnavailable> {
self.receiver
.lock()
.await
.recv()
.await
.map(|event| event.to_string())
.ok_or(ResponseControlUnavailable)
}
}
#[derive(Default)]
struct InterruptProvider {
calls: AtomicUsize,
controls: Mutex<Vec<ResponseControl>>,
transport: Mutex<Option<Arc<ControlTransport>>>,
complete: tokio::sync::Notify,
initial_messages: Mutex<std::collections::VecDeque<ProtocolWireEvent>>,
}
#[async_trait]
@@ -30,12 +88,10 @@ impl Provider for InterruptProvider {
) -> Result<ProviderStream, ProviderError> {
let number = self.calls.fetch_add(1, Ordering::SeqCst);
let id = format!("resp_interrupt_{number}");
let control = context
.response_control()
.expect("root control propagated through Core")
.clone();
let active = control.activate(id.clone()).unwrap();
self.controls.lock().unwrap().push(control);
let transport = Arc::new(ControlTransport::new(id));
let owner: Arc<dyn ResponseControlTransport> = transport.clone();
context.response_control().unwrap().bind(&owner);
*self.transport.lock().unwrap() = Some(transport.clone());
let candidate = request.candidate();
let metadata = ProviderCallMetadata::new(
candidate.provider().clone(),
@@ -44,41 +100,55 @@ impl Provider for InterruptProvider {
UpstreamTransport::new("websocket").unwrap(),
);
let body = futures::stream::unfold(
(0, Some(active), id),
|(phase, mut active, id)| async move {
let (event_type, status, event) = match phase {
(0, transport, self),
|(phase, transport, provider)| async move {
let pending = if phase == 1 {
provider.initial_messages.lock().unwrap().pop_front()
} else {
None
};
if let Some(wire) = pending {
return Some((Ok(ProviderEvent::wire(wire)), (phase, transport, provider)));
}
let (wire, events, next_phase) = match phase {
0 => (
"response.created",
"in_progress",
GatewayEvent::Started(ResponseMeta::new(&id, "model-a")),
json!({"type":"response.created","response":{"id":transport.id,"model":"model-a","status":"in_progress","output":[]}}),
vec![GatewayEvent::Started(ResponseMeta::new(
&transport.id,
"model-a",
))],
1,
),
1 => {
active.as_ref().unwrap().requested().await;
drop(active.take());
(
"response.incomplete",
"incomplete",
GatewayEvent::Completed(ResponseMeta::new(&id, "model-a")),
)
let wire = tokio::select! {
reply = transport.receive() => serde_json::from_str::<Value>(&reply.unwrap()).unwrap(),
() = provider.complete.notified() => json!({"type":"response.completed","response":{"id":transport.id,"model":"model-a","status":"completed","output":[]}}),
};
if matches!(
wire["type"].as_str(),
Some("response.completed" | "response.incomplete")
) {
transport.active.store(false, Ordering::SeqCst);
(
wire,
vec![GatewayEvent::Completed(ResponseMeta::new(
&transport.id,
"model-a",
))],
2,
)
} else {
(wire, Vec::new(), 1)
}
}
_ => return None,
};
let mut response = json!({"id":id,"model":"model-a","status":status,"output":[]});
if phase == 1 {
response["incomplete_details"] = json!({"reason":"interrupted"});
}
let event_type = wire["type"].as_str().unwrap().to_owned();
let event = ProviderEvent::canonical_with_wire(
vec![event],
ProtocolWireEvent::json(
"openai",
Some(event_type.to_owned()),
json!({
"type":event_type, "response":response
}),
)
.unwrap(),
events,
ProtocolWireEvent::json("openai", Some(event_type), wire).unwrap(),
);
Some((Ok(event), (phase + 1, active, id)))
Some((Ok(event), (next_phase, transport, provider)))
},
);
Ok(ProviderStream::new(metadata, body, ()))
@@ -88,6 +158,8 @@ impl Provider for InterruptProvider {
#[derive(Default)]
struct InterruptAdmissions {
active: AtomicUsize,
release_started: tokio::sync::Notify,
release_gate: Mutex<Option<tokio::sync::oneshot::Receiver<()>>>,
}
impl ClientAdmissionPort for InterruptAdmissions {
@@ -109,6 +181,11 @@ impl ClientAdmissionPort for InterruptAdmissions {
_: &'a ModelRequestId,
) -> BoxFuture<'a, Result<bool, ClientAdmissionError>> {
Box::pin(async {
let gate = self.release_gate.lock().unwrap().take();
self.release_started.notify_one();
if let Some(gate) = gate {
gate.await.unwrap();
}
self.active.fetch_sub(1, Ordering::SeqCst);
Ok(true)
})
@@ -151,15 +228,15 @@ async fn next(socket: &mut TestSocket) -> Value {
}
}
#[tokio::test]
async fn interrupt_is_live_scoped_and_keeps_queued_creates_serial() {
let provider = Arc::new(InterruptProvider::default());
let admissions = Arc::new(InterruptAdmissions::default());
async fn connect(
provider: Arc<InterruptProvider>,
admissions: Arc<InterruptAdmissions>,
) -> (TestSocket, Server) {
let execution = Arc::new(DefaultExecutionService::new(
RuntimeSnapshotHandle::new(snapshot("sk_ws_interrupt", "openai")),
Arc::new(SettlementPorts::default()),
ProviderRegistry::new([provider.clone() as Arc<dyn Provider>]).unwrap(),
admissions.clone(),
ProviderRegistry::new([provider as Arc<dyn Provider>]).unwrap(),
admissions,
Arc::new(UnusedContinuation),
Arc::new(IgnoredClientApiKeyUsage),
Arc::new(crate::support::RecordingDiagnostics::default()),
@@ -167,7 +244,7 @@ async fn interrupt_is_live_scoped_and_keeps_queued_creates_serial() {
let app = api_router(execution).await;
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let address = listener.local_addr().unwrap();
let _server = Server(tokio::spawn(async move {
let server = Server(tokio::spawn(async move {
axum::serve(listener, app).await.unwrap()
}));
let mut request = format!("ws://{address}/v1/responses")
@@ -177,29 +254,136 @@ async fn interrupt_is_live_scoped_and_keeps_queued_creates_serial() {
AUTHORIZATION,
HeaderValue::from_static("Bearer sk_ws_interrupt"),
);
let (mut socket, _) = connect_async(request).await.unwrap();
let (socket, _) = connect_async(request).await.unwrap();
(socket, server)
}
fn upstream_control_error() -> Value {
json!({"type":"error", "status":400, "error":{"type":"invalid_request_error","code":"future_rejected","message":"Synthetic upstream rejection","param":"type"},"vendor_extension":true})
}
fn interrupt(id: &str, mode: &str) -> Value {
json!({"type":"response.interrupt","response_id":id,"mode":mode,"future_extension":true})
}
#[tokio::test]
async fn late_interrupt_after_terminal_is_sent_upstream_and_keeps_the_connection_reusable() {
let provider = Arc::new(InterruptProvider::default());
let admissions = Arc::new(InterruptAdmissions::default());
let (mut socket, _server) = connect(provider.clone(), admissions.clone()).await;
let create = json!({"type":"response.create","model":"model-a","input":"hello"});
for (number, terminal) in [(0, "response.completed"), (1, "response.incomplete")] {
let id = format!("resp_interrupt_{number}");
send(&mut socket, create.clone()).await;
assert_eq!(next(&mut socket).await["type"], "response.created");
if number == 0 {
provider.complete.notify_one();
} else {
send(&mut socket, interrupt(&id, "discard_partial_items")).await;
}
assert_eq!(next(&mut socket).await["type"], terminal);
tokio::time::timeout(Duration::from_secs(5), async {
while admissions.active.load(Ordering::SeqCst) != 0 {
tokio::task::yield_now().await;
}
})
.await
.unwrap();
for control in [
interrupt(&id, "discard_partial_items"),
interrupt(&id, "future_mode"),
json!({"type":"future.control","extra":[1,2]}),
] {
send(&mut socket, control.clone()).await;
assert_eq!(
next(&mut socket).await,
json!({"type":"future.control.ack","original":control})
);
}
send(&mut socket, json!({"type":"future.reject"})).await;
assert_eq!(next(&mut socket).await, upstream_control_error());
assert_eq!(provider.calls.load(Ordering::SeqCst), number + 1);
assert_eq!(admissions.active.load(Ordering::SeqCst), 0);
}
socket.close(None).await.unwrap();
}
#[tokio::test]
async fn late_interrupt_during_terminal_settlement_does_not_abort_or_reorder_requests() {
let provider = Arc::new(InterruptProvider::default());
let (release, wait_for_release) = tokio::sync::oneshot::channel();
let admissions = Arc::new(InterruptAdmissions {
release_gate: Mutex::new(Some(wait_for_release)),
..Default::default()
});
let (mut socket, _server) = connect(provider.clone(), admissions.clone()).await;
let create = json!({"type":"response.create","model":"model-a","input":"hello"});
let interrupt =
|id: &str, mode: &str| json!({"type":"response.interrupt","response_id":id,"mode":mode});
send(
&mut socket,
interrupt("resp_interrupt_0", "discard_partial_items"),
)
.await;
assert_eq!(next(&mut socket).await["type"], "error");
assert_eq!(provider.calls.load(Ordering::SeqCst), 0);
send(&mut socket, create.clone()).await;
assert_eq!(next(&mut socket).await["type"], "response.created");
provider.complete.notify_one();
assert_eq!(next(&mut socket).await["type"], "response.completed");
tokio::time::timeout(
Duration::from_secs(5),
admissions.release_started.notified(),
)
.await
.unwrap();
send(&mut socket, create).await;
for invalid in [
interrupt("resp_other", "discard_partial_items"),
interrupt("resp_interrupt_0", "unknown"),
interrupt("", "discard_partial_items"),
let late = interrupt("resp_interrupt_0", "discard_partial_items");
send(&mut socket, late.clone()).await;
assert_eq!(
next(&mut socket).await,
json!({"type":"future.control.ack", "original":late})
);
assert_eq!(provider.calls.load(Ordering::SeqCst), 1);
assert_eq!(admissions.active.load(Ordering::SeqCst), 1);
release.send(()).unwrap();
assert_eq!(
next(&mut socket).await["response"]["id"],
"resp_interrupt_1"
);
socket.close(None).await.unwrap();
}
#[tokio::test]
async fn unknown_controls_preserve_bytes_and_do_not_admit_queued_creates() {
let provider = Arc::new(InterruptProvider::default());
let admissions = Arc::new(InterruptAdmissions::default());
let (mut socket, _server) = connect(provider.clone(), admissions.clone()).await;
let create = json!({"type":"response.create","model":"model-a","input":"hello"});
send(&mut socket, json!({"type":"future.control"})).await;
let error = next(&mut socket).await;
assert_eq!(error["type"], "error");
assert_ne!(error["error"]["param"], "type");
assert_eq!(provider.calls.load(Ordering::SeqCst), 0);
send(&mut socket, create).await;
assert_eq!(next(&mut socket).await["type"], "response.created");
// 重复 type 仍是创建请求,不能借投影解码失败绕过串行准入
socket.send(ClientMessage::Text(r#"{"type":"response.create","type":"response.create","model":"model-a","input":"next"}"#.into())).await.unwrap();
for raw in [
r#"{ "type": "future.control", "mode": "unknown", "extension": [1, 2] }"#,
r#"{"extension":"no local type contract"}"#,
r#"{"type":42,"extension":true}"#,
"not-json",
] {
send(&mut socket, invalid).await;
let error = next(&mut socket).await;
assert_eq!(error["type"], "error");
assert_eq!(error["status"], 400);
socket.send(ClientMessage::Text(raw.into())).await.unwrap();
let expected =
serde_json::from_str::<Value>(raw).unwrap_or_else(|_| Value::String(raw.to_owned()));
assert_eq!(next(&mut socket).await["original"], expected);
assert_eq!(
provider
.transport
.lock()
.unwrap()
.as_ref()
.unwrap()
.requests
.lock()
.unwrap()
.last()
.unwrap(),
raw
);
assert_eq!(provider.calls.load(Ordering::SeqCst), 1);
}
send(
@@ -207,27 +391,12 @@ async fn interrupt_is_live_scoped_and_keeps_queued_creates_serial() {
interrupt("resp_interrupt_0", "discard_partial_items"),
)
.await;
let terminal = next(&mut socket).await;
assert_eq!(terminal["type"], "response.incomplete");
assert_eq!(
terminal["response"]["incomplete_details"]["reason"],
"interrupted"
);
assert_eq!(next(&mut socket).await["type"], "response.incomplete");
assert_eq!(
next(&mut socket).await["response"]["id"],
"resp_interrupt_1"
);
assert_eq!(provider.calls.load(Ordering::SeqCst), 2);
send(
&mut socket,
interrupt("resp_interrupt_0", "discard_partial_items"),
)
.await;
assert_eq!(next(&mut socket).await["type"], "error");
assert_eq!(
provider.controls.lock().unwrap()[0].interrupt("resp_interrupt_0"),
Err(ResponseInterruptError::Unavailable)
);
socket.close(None).await.unwrap();
tokio::time::timeout(Duration::from_secs(5), async {
while admissions.active.load(Ordering::SeqCst) != 0 {
@@ -236,8 +405,46 @@ async fn interrupt_is_live_scoped_and_keeps_queued_creates_serial() {
})
.await
.expect("disconnect releases active execution");
assert_eq!(
provider.controls.lock().unwrap()[1].interrupt("resp_interrupt_1"),
Err(ResponseInterruptError::Unavailable)
);
}
#[tokio::test]
async fn active_websocket_delivery_preserves_unparsed_messages_and_business_metadata() {
let messages = vec![
"{ \"type\": \"future.event\", \"number\": 1e3, \"escaped\": \"\\u0061\" }\r\n".to_owned(),
r#"{"type":"response.metadata","metadata":{"type":"safety_buffering","use_cases":["cyber"],"reasons":["user_risk"],"future":{"keep":true}}}"#.to_owned(),
format!("{{\"type\":\"future.deep\",\"extension\":{}0{}}}", "[".repeat(140), "]".repeat(140)),
"future non-JSON text".to_owned(), "[DONE]".to_owned(),
];
let provider = Arc::new(InterruptProvider::default());
*provider.initial_messages.lock().unwrap() = messages
.iter()
.map(|raw| match serde_json::from_str::<Value>(raw) {
Ok(value) => ProtocolWireEvent::json(
"openai",
value.get("type").and_then(Value::as_str).map(str::to_owned),
value,
)
.unwrap()
.with_raw_websocket_message(raw.as_str()),
Err(_) => ProtocolWireEvent::raw_websocket("openai", raw.as_str()).unwrap(),
})
.collect();
let (mut socket, _server) =
connect(provider.clone(), Arc::new(InterruptAdmissions::default())).await;
send(
&mut socket,
json!({"type":"response.create","model":"model-a","input":"synthetic"}),
)
.await;
assert_eq!(next(&mut socket).await["type"], "response.created");
for expected in &messages {
let actual = tokio::time::timeout(Duration::from_secs(5), socket.next())
.await
.unwrap()
.unwrap()
.unwrap();
assert_eq!(actual.to_text().unwrap(), expected);
}
provider.complete.notify_one();
assert_eq!(next(&mut socket).await["type"], "response.completed");
}
@@ -7,6 +7,7 @@ use axum::{
body::{Body, to_bytes},
http::{Request, StatusCode, header::AUTHORIZATION},
};
use base64::{Engine as _, engine::general_purpose::STANDARD};
use bytes::Bytes;
use futures::future::BoxFuture;
use gateway_core::engine::ModelRequestId;
@@ -181,6 +182,11 @@ async fn search_route_should_preserve_request_and_success_response_bytes() {
.header("x-codex-turn-metadata", turn_metadata)
.header("session-id", "root-session")
.header("thread-id", "child-thread")
.header("x-future-business", "first")
.header("x-future-business", "second")
.header("cookie", "client-secret")
.header("connection", "x-private-hop")
.header("x-private-hop", "client-hop")
.body(Body::from(request_body.to_vec()))
.expect("search request"),
)
@@ -213,8 +219,37 @@ async fn search_route_should_preserve_request_and_success_response_bytes() {
assert_eq!(captured[0].endpoint, "/v1/alpha/search");
assert_eq!(captured[0].transport, ClientTransport::HttpJson);
assert_eq!(captured[0].body.as_ref(), request_body);
let mut context = captured[0].context.clone();
let headers = context
.as_object_mut()
.unwrap()
.remove("opaque_request_headers")
.unwrap();
let headers: Vec<_> = headers
.as_array()
.unwrap()
.iter()
.map(|entry| {
(
entry[0].as_str().unwrap(),
STANDARD.decode(entry[1].as_str().unwrap()).unwrap(),
)
})
.collect();
assert_eq!(
captured[0].context,
headers
.iter()
.filter(|(name, _)| *name == "x-future-business")
.map(|(_, value)| value.as_slice())
.collect::<Vec<_>>(),
[b"first".as_slice(), b"second".as_slice()]
);
assert!(!headers.iter().any(|(name, _)| matches!(
*name,
"authorization" | "cookie" | "connection" | "x-private-hop"
)));
assert_eq!(
context,
json!({"turn_metadata": turn_metadata, "session_id": "root-session", "thread_id": "child-thread"})
);
assert_eq!(
@@ -605,10 +605,15 @@ fn provider_event_to_middleware_frame(
ClientTransport::WebSocket
| ClientTransport::InternalProbe
| ClientTransport::InternalPlugin => (
serde_json::to_vec(wire.data())
.map(Bytes::from)
wire.raw_websocket_message()
.map(|raw| Bytes::copy_from_slice(raw.as_bytes()))
.map_or_else(|| serde_json::to_vec(wire.data()).map(Bytes::from), Ok)
.map_err(|_| middleware_protocol_error(UpstreamSendState::Sent))?,
MiddlewareFraming::JsonDocument,
if wire.raw_websocket_message().is_some() && !wire.has_json_data() {
MiddlewareFraming::RawBytes
} else {
MiddlewareFraming::JsonDocument
},
),
};
Ok(MiddlewareFrame::from_provider_event(
@@ -631,10 +636,19 @@ fn encode_sse_wire_event(wire: &ProtocolWireEvent) -> Result<Bytes, ProviderErro
encoded.extend_from_slice(event_type.as_bytes());
encoded.push(b'\n');
}
encoded.extend_from_slice(b"data: ");
serde_json::to_writer(&mut encoded, wire.data())
.map_err(|_| middleware_protocol_error(UpstreamSendState::Sent))?;
encoded.extend_from_slice(b"\n\n");
if let Some(raw) = wire.raw_websocket_message() {
for line in raw.split('\n') {
encoded.extend_from_slice(b"data: ");
encoded.extend_from_slice(line.as_bytes());
encoded.push(b'\n');
}
encoded.push(b'\n');
} else {
encoded.extend_from_slice(b"data: ");
serde_json::to_writer(&mut encoded, wire.data())
.map_err(|_| middleware_protocol_error(UpstreamSendState::Sent))?;
encoded.extend_from_slice(b"\n\n");
}
Ok(Bytes::from(encoded))
}
@@ -701,12 +715,19 @@ fn middleware_frame_to_provider_event(
.raw_json_body()
.or_else(|| wire.raw_http_body_bytes())
.cloned()
.or_else(|| {
wire.raw_websocket_message()
.map(|raw| Bytes::copy_from_slice(raw.as_bytes()))
})
.or_else(|| serde_json::to_vec(wire.data()).ok().map(Bytes::from)),
MiddlewareFraming::SseEvent => wire
.raw_sse_frame()
.cloned()
.or_else(|| encode_sse_wire_event(wire).ok()),
MiddlewareFraming::RawBytes => wire.raw_http_body_bytes().cloned(),
MiddlewareFraming::RawBytes => wire.raw_http_body_bytes().cloned().or_else(|| {
wire.raw_websocket_message()
.map(|raw| Bytes::copy_from_slice(raw.as_bytes()))
}),
};
if original.as_ref() == Some(&bytes)
&& let Some(mut event) = envelope.take()
@@ -1,14 +1,27 @@
//! 当前执行的响应中断信号;只由持有活动响应的 Provider 注册,不查找其他执行或连接
//! 原上游连接的控制消息端口;不解释协议类型、不创建执行或延长连接生命周期
use std::fmt;
use std::sync::{Arc, Mutex, Weak};
use crate::lifecycle::CancellationToken;
use async_trait::async_trait;
/// 单次根执行的控制入口;重试不继承已结束响应的控制目标
/// Provider 实现的连接控制通道,消息内容由上游协议解释
#[async_trait]
pub trait ResponseControlTransport: Send + Sync {
async fn send(&self, payload: &str) -> Result<(), ResponseControlUnavailable>;
/// 仅在响应正文已结束后读取,调用方必须先取消此读取再开始下一轮执行
async fn receive(&self) -> Result<String, ResponseControlUnavailable>;
}
/// 控制消息没有可用的原上游连接,不能通过重新选号或重试恢复
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct ResponseControlUnavailable;
/// 单次根执行绑定的原连接;终态后仍可使用,连接销毁或重新绑定时失效
#[derive(Clone, Default)]
pub struct ResponseControl {
active: Arc<Mutex<Weak<InterruptTarget>>>,
target: Arc<Mutex<Option<Weak<dyn ResponseControlTransport>>>>,
}
impl fmt::Debug for ResponseControl {
@@ -17,64 +30,42 @@ impl fmt::Debug for ResponseControl {
}
}
struct InterruptTarget {
response_id: String,
requested: CancellationToken,
}
/// Provider 持有的活动响应控制权;释放后客户端无法再向该响应发出中断
pub struct ActiveResponseInterrupt(Arc<InterruptTarget>);
/// 控制帧不能作用于当前执行时的稳定原因,不携带其他响应身份
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ResponseInterruptError {
Unavailable,
ResponseMismatch,
}
impl ResponseControl {
/// 在上游确认响应 ID 后注册;同一执行只允许一个存活的响应 owner
pub fn activate(&self, response_id: String) -> Option<ActiveResponseInterrupt> {
let mut active = self
.active
/// Provider 在请求成功写入上游后绑定;连接 owner 持有强引用并负责撤销
pub fn bind(&self, transport: &Arc<dyn ResponseControlTransport>) {
*self
.target
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if active.upgrade().is_some() || response_id.is_empty() {
return None;
}
let target = Arc::new(InterruptTarget {
response_id,
requested: CancellationToken::new(),
});
*active = Arc::downgrade(&target);
Some(ActiveResponseInterrupt(target))
.unwrap_or_else(std::sync::PoisonError::into_inner) = Some(Arc::downgrade(transport));
}
/// 重复中断合并为一次信号;ID 必须匹配本执行当前的上游响应
pub fn interrupt(&self, response_id: &str) -> Result<(), ResponseInterruptError> {
let active = self
.active
pub fn clear(&self) {
*self
.target
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let target = active
.upgrade()
.ok_or(ResponseInterruptError::Unavailable)?;
if target.response_id != response_id {
return Err(ResponseInterruptError::ResponseMismatch);
.unwrap_or_else(std::sync::PoisonError::into_inner) = None;
}
fn transport(&self) -> Option<Arc<dyn ResponseControlTransport>> {
self.target
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.as_ref()
.and_then(Weak::upgrade)
}
pub async fn send(&self, payload: &str) -> Result<(), ResponseControlUnavailable> {
self.transport()
.ok_or(ResponseControlUnavailable)?
.send(payload)
.await
}
/// 未绑定时保持等待,供连接空闲循环与客户端输入共同 select
pub async fn receive(&self) -> Result<String, ResponseControlUnavailable> {
match self.transport() {
Some(transport) => transport.receive().await,
None => std::future::pending().await,
}
target.requested.cancel();
Ok(())
}
}
impl ActiveResponseInterrupt {
pub fn response_id(&self) -> &str {
&self.0.response_id
}
pub fn requested(&self) -> impl Future<Output = ()> + Send + 'static {
// 等待者只持有信号,不能延长活动响应 owner 的生命周期
let requested = self.0.requested.clone();
async move { requested.cancelled().await }
}
}
+37 -1
View File
@@ -203,6 +203,7 @@ pub struct ProtocolWireEvent {
data: Value,
has_json_data: bool,
raw_sse_frame: Option<Bytes>,
raw_websocket_message: Option<Arc<str>>,
raw_json_body: Option<Bytes>,
raw_http_body: Option<Bytes>,
sse_id: Option<String>,
@@ -245,6 +246,7 @@ impl ProtocolWireEvent {
data,
has_json_data: true,
raw_sse_frame: None,
raw_websocket_message: None,
raw_json_body: None,
raw_http_body: None,
sse_id,
@@ -293,6 +295,7 @@ impl ProtocolWireEvent {
data: Value::Null,
has_json_data: false,
raw_sse_frame: Some(raw_sse_frame),
raw_websocket_message: None,
raw_json_body: None,
raw_http_body: None,
sse_id: None,
@@ -300,6 +303,33 @@ impl ProtocolWireEvent {
})
}
/// 创建原始 WebSocket 文本;JSON 解析失败不影响交付
///
/// # Errors
///
/// 协议名不满足 wire 安全约束时返回错误
pub fn raw_websocket(
protocol: impl Into<String>,
message: impl Into<Arc<str>>,
) -> Result<Self, IdentifierError> {
let mut wire = Self::json(protocol, None, Value::Null)?;
wire.has_json_data = false;
Ok(wire.with_raw_websocket_message(message))
}
/// 绑定同一事件未经改写的 WebSocket 文本,解析视图只用于旁路事实
#[must_use]
pub fn with_raw_websocket_message(mut self, message: impl Into<Arc<str>>) -> Self {
self.raw_websocket_message = Some(message.into());
self
}
/// 返回未经改写的 WebSocket 文本
#[must_use]
pub fn raw_websocket_message(&self) -> Option<&str> {
self.raw_websocket_message.as_deref()
}
/// 创建未经改写的完整 JSON 响应正文
///
/// 该载荷用于非流式协议端点;Core 不解析或重编码其中的值
@@ -319,6 +349,7 @@ impl ProtocolWireEvent {
data: Value::Null,
has_json_data: false,
raw_sse_frame: None,
raw_websocket_message: None,
raw_json_body: Some(raw_json_body),
raw_http_body: None,
sse_id: None,
@@ -343,6 +374,7 @@ impl ProtocolWireEvent {
data: Value::Null,
has_json_data: false,
raw_sse_frame: None,
raw_websocket_message: None,
raw_json_body: None,
raw_http_body: Some(raw_http_body),
sse_id: None,
@@ -431,6 +463,10 @@ impl fmt::Debug for ProtocolWireEvent {
.field("event_type", &self.event_type)
.field("has_json_data", &self.has_json_data)
.field("has_raw_sse_frame", &self.raw_sse_frame.is_some())
.field(
"has_raw_websocket_message",
&self.raw_websocket_message.is_some(),
)
.field("has_raw_json_body", &self.raw_json_body.is_some())
.field("has_raw_http_body", &self.raw_http_body.is_some())
.field("has_sse_id", &self.sse_id.is_some())
@@ -500,7 +536,7 @@ pub struct ProviderResponseTimings {
pub upstream_response_ms: Option<u64>,
/// 上游排除引擎与客户端工具时间后的 API 耗时
pub upstream_api_overhead_ms: Option<f64>,
/// 上游引擎服务总耗时;以下专项指标保留毫秒小数及各自内部计量层级
/// 上游响应级引擎耗时;以下专项指标保留毫秒小数及各自内部计量层级
pub upstream_engine_ms: Option<f64>,
pub upstream_engine_iapi_ttft_ms: Option<f64>,
pub upstream_engine_service_ttft_ms: Option<f64>,
+1 -1
View File
@@ -126,7 +126,7 @@ pub trait ConnectionGuard: Send + 'static {}
pub trait ConnectionLifecycle: Send + Sync {
/// 原子地检查 drain 状态并注册一个活跃连接
///
/// 当本方法成功时,drain 必须等待返回的 guard 被释放;
/// 成功注册不延长关闭流程,活跃连接可随进程退出被中断;
/// 当 drain 已经线性化生效时,本方法必须返回 [`ConnectionDraining`]
fn try_register(&self) -> Result<Box<dyn ConnectionGuard>, ConnectionDraining>;
@@ -1,45 +1,68 @@
//! 验证响应中断的当前 owner 校验及独立执行间的控制隔离
//! 验证原连接控制端口的透明传递、owner 生命周期与执行隔离
use futures::FutureExt;
use gateway_core::engine::response_control::{ResponseControl, ResponseInterruptError};
use std::sync::{Arc, Mutex};
#[test]
fn interrupt_requires_the_current_response_owner_and_does_not_extend_its_lifetime() {
let control = ResponseControl::default();
assert_eq!(
control.interrupt("resp_a"),
Err(ResponseInterruptError::Unavailable)
);
assert!(control.activate(String::new()).is_none());
let active = control.activate("resp_a".to_owned()).unwrap();
let requested = active.requested();
assert!(control.activate("resp_b".to_owned()).is_none());
assert_eq!(
control.interrupt("resp_b"),
Err(ResponseInterruptError::ResponseMismatch)
);
assert!(active.requested().now_or_never().is_none());
control.interrupt("resp_a").unwrap();
control.interrupt("resp_a").unwrap();
assert_eq!(requested.now_or_never(), Some(()));
let retained_waiter = active.requested();
drop(active);
assert_eq!(
control.interrupt("resp_a"),
Err(ResponseInterruptError::Unavailable)
);
let next = control.activate("resp_b".to_owned()).unwrap();
assert!(next.requested().now_or_never().is_none());
drop(retained_waiter);
use async_trait::async_trait;
use futures::{FutureExt, executor::block_on};
use gateway_core::engine::response_control::{
ResponseControl, ResponseControlTransport, ResponseControlUnavailable,
};
#[derive(Default)]
struct Transport(Mutex<Vec<String>>);
#[async_trait]
impl ResponseControlTransport for Transport {
async fn send(&self, payload: &str) -> Result<(), ResponseControlUnavailable> {
self.0.lock().unwrap().push(payload.to_owned());
Ok(())
}
async fn receive(&self) -> Result<String, ResponseControlUnavailable> {
Ok("upstream reply".to_owned())
}
}
#[test]
fn matching_response_ids_do_not_share_control_between_root_executions() {
let first = ResponseControl::default();
let second = ResponseControl::default();
let active_first = first.activate("resp_same".to_owned()).unwrap();
let active_second = second.activate("resp_same".to_owned()).unwrap();
first.interrupt("resp_same").unwrap();
assert_eq!(active_first.requested().now_or_never(), Some(()));
assert!(active_second.requested().now_or_never().is_none());
fn control_requires_a_live_connection_owner_and_preserves_unknown_payloads() {
block_on(async {
let control = ResponseControl::default();
assert_eq!(
control.send("unknown").await,
Err(ResponseControlUnavailable)
);
assert!(control.receive().now_or_never().is_none());
let transport = Arc::new(Transport::default());
let owner: Arc<dyn ResponseControlTransport> = transport.clone();
control.bind(&owner);
let payload = "{ \"type\": \"future.control\", \"extension\": [1, 2] }";
control.send(payload).await.unwrap();
control.send(payload).await.unwrap();
assert_eq!(*transport.0.lock().unwrap(), [payload, payload]);
assert_eq!(control.receive().await.unwrap(), "upstream reply");
drop(owner);
drop(transport);
assert_eq!(control.send(payload).await, Err(ResponseControlUnavailable));
});
}
#[test]
fn replacing_or_clearing_a_binding_does_not_share_control_between_executions() {
block_on(async {
let first = ResponseControl::default();
let second = ResponseControl::default();
let old: Arc<dyn ResponseControlTransport> = Arc::new(Transport::default());
let current = Arc::new(Transport::default());
let owner: Arc<dyn ResponseControlTransport> = current.clone();
first.bind(&old);
second.bind(&owner);
drop(old);
assert_eq!(first.send("old").await, Err(ResponseControlUnavailable));
second.send("current").await.unwrap();
assert_eq!(*current.0.lock().unwrap(), ["current"]);
second.clear();
assert_eq!(
second.send("cleared").await,
Err(ResponseControlUnavailable)
);
});
}
-14
View File
@@ -81,8 +81,6 @@ pub struct HostConfig {
pub logging: LoggingConfig,
#[serde(default)]
pub system_update: SystemUpdateConfig,
#[serde(default = "default_drain_timeout_seconds")]
pub drain_timeout_seconds: u64,
#[serde(default = "default_worker_shutdown_timeout_seconds")]
pub worker_shutdown_timeout_seconds: u64,
}
@@ -110,9 +108,6 @@ impl HostConfig {
if self.listen.port == 0 {
return Err(ConfigError::InvalidField("host.listen.port"));
}
if self.drain_timeout_seconds == 0 {
return Err(ConfigError::InvalidField("host.drain_timeout_seconds"));
}
if self.worker_shutdown_timeout_seconds == 0 {
return Err(ConfigError::InvalidField(
"host.worker_shutdown_timeout_seconds",
@@ -136,11 +131,6 @@ impl HostConfig {
&self.runtime_data_dir
}
#[must_use]
pub const fn drain_timeout(&self) -> Duration {
Duration::from_secs(self.drain_timeout_seconds)
}
#[must_use]
pub const fn worker_shutdown_timeout(&self) -> Duration {
Duration::from_secs(self.worker_shutdown_timeout_seconds)
@@ -264,10 +254,6 @@ fn resolve_relative_path(base: &Path, path: &mut PathBuf) {
}
}
const fn default_drain_timeout_seconds() -> u64 {
30
}
const fn default_worker_shutdown_timeout_seconds() -> u64 {
30
}
+2 -3
View File
@@ -164,7 +164,7 @@ impl HostBundle {
Ok(())
}
/// 进程唯一阻塞点;返回前完成 HTTP drain 与 worker join
/// 进程唯一阻塞点;停止 HTTP 监听后完成 worker join
pub async fn serve(self, router: Router) -> Result<(), HostError> {
let result = serve_router(
router,
@@ -172,7 +172,6 @@ impl HostBundle {
self.config.listen.port,
self.cancellation.clone(),
Arc::clone(&self.connections),
self.config.drain_timeout(),
)
.await;
self.cancellation.cancel();
@@ -181,7 +180,7 @@ impl HostBundle {
.await;
match &result {
Ok(()) => {
tracing::info!(target: "gateway_shutdown", pid = std::process::id(), "HTTP 服务与后台任务已停止")
tracing::info!(target: "gateway_shutdown", pid = std::process::id(), "HTTP 监听已关闭,后台任务已结束")
}
Err(error) => {
tracing::error!(target: "gateway_shutdown", pid = std::process::id(), %error, "HTTP 服务异常停止,后台任务已清理")
+14 -74
View File
@@ -1,13 +1,12 @@
//! HTTP 监听、OS signal、原子连接注册与优雅 drain
//! HTTP 监听、关闭信号与停止时的原子连接注册
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::{Arc, OnceLock};
use std::time::Duration;
use axum::Router;
use gateway_core::lifecycle::CancellationToken;
use gateway_core::lifecycle::{ConnectionDraining, ConnectionGuard, ConnectionLifecycle};
use tokio::sync::Notify;
const DRAINING_BIT: usize = 1usize << (usize::BITS - 1);
const ACTIVE_MASK: usize = !DRAINING_BIT;
@@ -24,7 +23,6 @@ pub struct ConnectionTracker {
struct ConnectionState {
value: AtomicUsize,
idle: Notify,
}
impl ConnectionTracker {
@@ -33,36 +31,14 @@ impl ConnectionTracker {
Self {
state: Arc::new(ConnectionState {
value: AtomicUsize::new(0),
idle: Notify::new(),
}),
cancellation,
}
}
pub(crate) fn begin_draining(&self) {
let previous = self.state.value.fetch_or(DRAINING_BIT, Ordering::AcqRel);
self.state.value.fetch_or(DRAINING_BIT, Ordering::AcqRel);
self.cancellation.cancel();
if previous & ACTIVE_MASK == 0 {
self.state.idle.notify_waiters();
}
}
pub async fn wait_until_idle(&self, timeout: Duration) {
let wait = async {
loop {
let notified = self.state.idle.notified();
tokio::pin!(notified);
// notify_waiters 不保存许可:必须先 enable 注册等待者,再读
// 活跃计数;否则最后一个 guard 在读数与首次 poll 之间 drop
// 时唤醒丢失,只能等满整个超时
notified.as_mut().enable();
if self.state.value.load(Ordering::Acquire) & ACTIVE_MASK == 0 {
return;
}
notified.await;
}
};
let _ = tokio::time::timeout(timeout, wait).await;
}
}
@@ -106,10 +82,7 @@ impl ConnectionGuard for ActiveConnection {}
impl Drop for ActiveConnection {
fn drop(&mut self) {
let previous = self.state.value.fetch_sub(1, Ordering::AcqRel);
if previous & ACTIVE_MASK == 1 {
self.state.idle.notify_waiters();
}
self.state.value.fetch_sub(1, Ordering::AcqRel);
}
}
@@ -140,59 +113,26 @@ pub(crate) async fn serve_router(
port: u16,
cancellation: CancellationToken,
connections: Arc<ConnectionTracker>,
drain_timeout: Duration,
) -> Result<(), ServeError> {
let listener = bind_listener(&format!("{host}:{port}"))
.await
.map_err(ServeError::Bind)?;
tracing::info!(target: "gateway_startup", pid = std::process::id(), host, port, "网关开始监听");
let shutdown_cancellation = cancellation.clone();
let shutdown_connections = Arc::clone(&connections);
let shutdown = async move {
wait_for_shutdown(&shutdown_cancellation).await;
shutdown_connections.begin_draining();
};
// axum 的优雅关闭会无限等待存量连接结束,而 SSE 等长响应体不观察
// 取消信号:慢客户端可以把关闭拖住直到被 SIGKILL
// 整个 drain(axum
// 优雅关闭 + 游离 guard 等待)共享同一个从取消信号起算的绝对截止点,
// 逾期即放弃等待,存量连接随进程退出终止
let drain_deadline_at = Arc::new(OnceLock::new());
let drain_deadline = {
let cancellation = cancellation.clone();
let deadline_at = Arc::clone(&drain_deadline_at);
async move {
cancellation.cancelled().await;
let deadline = tokio::time::Instant::now() + drain_timeout;
let _ = deadline_at.set(deadline);
tokio::time::sleep_until(deadline).await;
}
};
// 所有关闭入口都停止等待长请求;丢弃 serve 关闭监听,存量连接随进程退出终止
// 后台落盘由 Host 的 Worker 关闭阶段独立收尾
let serve = axum::serve(
listener,
router.into_make_service_with_connect_info::<std::net::SocketAddr>(),
)
.with_graceful_shutdown(shutdown);
tokio::select! {
);
let result = tokio::select! {
biased;
result = serve => {
result.map_err(ServeError::Serve)?;
connections.begin_draining();
let remaining = drain_deadline_at.get().map_or(drain_timeout, |deadline| {
deadline.saturating_duration_since(tokio::time::Instant::now())
});
connections.wait_until_idle(remaining).await;
}
() = drain_deadline => {
tracing::warn!(
target: "gateway_shutdown",
timeout_secs = drain_timeout.as_secs(),
"优雅 drain 超时,放弃等待存量连接"
);
}
}
Ok(())
() = wait_for_shutdown(&cancellation) => Ok(()),
result = serve => result.map_err(ServeError::Serve),
};
connections.begin_draining();
tracing::info!(target: "gateway_shutdown", "停止等待存量连接,后台落盘完成后退出进程");
result
}
pub(crate) async fn wait_for_shutdown(cancellation: &CancellationToken) {
@@ -538,6 +538,7 @@ impl SystemOperations for ProcessSystemOperations {
let cancellation = self.cancellation.clone();
drop(tokio::spawn(async move {
tokio::time::sleep(Duration::from_millis(500)).await;
// 留出受理响应返回时间,再走统一关闭流程
cancellation.cancel();
}));
Ok(SystemOperationAccepted::Restart {
+2 -3
View File
@@ -127,9 +127,9 @@ fn host_config_derives_update_paths_from_runtime_data_dir() {
}
#[test]
fn host_config_rejects_zero_drain_window() {
fn host_config_rejects_zero_worker_shutdown_window() {
let mut config = valid_config();
config.drain_timeout_seconds = 0;
config.worker_shutdown_timeout_seconds = 0;
assert!(
config
@@ -223,7 +223,6 @@ fn valid_config() -> HostConfig {
request_dump_retention_days: 1,
},
system_update,
drain_timeout_seconds: 30,
worker_shutdown_timeout_seconds: 30,
}
}
@@ -51,7 +51,6 @@ fn logging_requires_at_least_one_sink() {
request_dump_retention_days: 1,
},
system_update: SystemUpdateConfig::default(),
drain_timeout_seconds: 30,
worker_shutdown_timeout_seconds: 30,
};
@@ -329,7 +328,6 @@ fn logging_config(directory: PathBuf, request_dump: bool) -> HostConfig {
request_dump_retention_days: 1,
},
system_update: SystemUpdateConfig::default(),
drain_timeout_seconds: 30,
worker_shutdown_timeout_seconds: 30,
}
}
+127 -118
View File
@@ -1,4 +1,4 @@
//! 验证宿主服务关闭时的连接排空、超时与并发唤醒
//! 验证统一关闭入口跳过会话等待、保留后台落盘及清理超时
use std::sync::Arc;
use std::sync::Mutex;
@@ -6,17 +6,19 @@ use std::time::Duration;
use axum::{Router, routing::get};
use futures::future::BoxFuture;
use gateway_admin::ports::system::{
SystemOperationError, SystemRestartPreflight, SystemUpdateCandidate,
};
use gateway_core::diagnostics::{OperationalDiagnostics, OperationalFailure};
use gateway_core::lifecycle::CancellationToken;
use gateway_core::lifecycle::ConnectionLifecycle as _;
use gateway_core::task::{
DaemonRestartPolicy, DaemonTask, WorkerContribution, WorkerId, WorkerKind,
WorkerLeaderLeasePort, WorkerLeaseAcquisition, WorkerLeaseError, WorkerLeaseRequest,
WorkerRegistration, WorkerRunnable, WorkerTaskError,
};
use gateway_host::HostBundle;
use gateway_host::serve::{ConnectionTracker, bind_listener};
use tokio::io::{AsyncReadExt as _, AsyncWriteExt as _};
use gateway_host::serve::bind_listener;
use tokio::io::AsyncWriteExt as _;
use tokio::sync::{Semaphore, mpsc};
use crate::support::host::{configuration, run_in_child};
@@ -28,65 +30,6 @@ fn host_bundle_serve_is_a_consuming_process_entrypoint() {
assert_eq!(std::mem::size_of_val(&_serve), 0);
}
#[tokio::test(start_paused = true)]
async fn wait_until_idle_should_return_immediately_without_active_connections() {
let tracker = ConnectionTracker::new(CancellationToken::new());
let started = tokio::time::Instant::now();
tracker.wait_until_idle(Duration::from_secs(30)).await;
assert_eq!(started.elapsed(), Duration::ZERO);
}
#[tokio::test(start_paused = true)]
async fn wait_until_idle_should_give_up_at_timeout_while_connections_remain() {
let tracker = ConnectionTracker::new(CancellationToken::new());
let _guard = tracker.try_register().expect("register connection");
let started = tokio::time::Instant::now();
tracker.wait_until_idle(Duration::from_secs(30)).await;
assert_eq!(started.elapsed(), Duration::from_secs(30));
}
#[tokio::test]
async fn wait_until_idle_should_wake_when_last_guard_drops_after_first_poll() {
let tracker = ConnectionTracker::new(CancellationToken::new());
let guard = tracker.try_register().expect("register connection");
let mut wait = Box::pin(tracker.wait_until_idle(Duration::from_secs(30)));
assert!(futures::poll!(wait.as_mut()).is_pending());
drop(guard);
tokio::time::timeout(Duration::from_secs(2), wait)
.await
.expect("woken by last guard drop instead of waiting out the timeout");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn wait_until_idle_should_observe_guard_drop_racing_the_idle_check() {
// 回归约束:notified 必须在读取活跃计数前完成注册(enable),否则最后
// 一个 guard 在计数检查与首次 poll 之间 drop 时唤醒丢失,只能等满整个
// 超时
// 多线程反复交错,任何一次丢唤醒都会撞上 2s 超时并触发断言
for _ in 0..256 {
let tracker = Arc::new(ConnectionTracker::new(CancellationToken::new()));
let guard = tracker.try_register().expect("register connection");
let waiter = tokio::spawn({
let tracker = Arc::clone(&tracker);
async move { tracker.wait_until_idle(Duration::from_secs(2)).await }
});
let dropper = std::thread::spawn(move || drop(guard));
let started = std::time::Instant::now();
waiter.await.expect("waiter completes");
dropper.join().expect("dropper completes");
assert!(
started.elapsed() < Duration::from_millis(1_900),
"idle wakeup was lost and the waiter slept until the drain timeout"
);
}
}
#[tokio::test]
async fn bind_listener_should_retry_until_previous_listener_releases_port() {
let holder = tokio::net::TcpListener::bind("127.0.0.1:0")
@@ -110,76 +53,148 @@ async fn bind_listener_should_retry_until_previous_listener_releases_port() {
}
#[test]
fn http_drain_keeps_writer_running_until_the_request_finishes() {
fn cancellation_skips_connections_but_flushes_and_joins_workers() {
run_in_child(
"serve::http_drain_keeps_writer_running_until_the_request_finishes",
async {
let directory = tempfile::tempdir().unwrap();
let (host, address) = listening_host(directory.path()).await;
let cancellation = host.cancellation();
let connections = host.connection_lifecycle();
let probe = start_workers(&host, false);
probe.started.cancelled().await;
let (router, entered, finish_request) = request_router(&host, &probe);
let server = tokio::spawn(host.serve(router));
let mut client = connect_request(address).await;
entered.cancelled().await;
cancellation.cancel();
wait_for_draining(connections.as_ref()).await;
assert!(connections.try_register().is_err());
assert!(
tokio::time::timeout(Duration::from_millis(50), probe.cancelled.cancelled())
.await
.is_err(),
"writer cancelled before the in-flight request finished"
);
finish_request.add_permits(1);
let mut response = String::new();
client.read_to_string(&mut response).await.unwrap();
server.await.unwrap().unwrap();
assert!(response.starts_with("HTTP/1.1 200 OK"));
assert_eq!(*probe.writes.lock().unwrap(), [1]);
assert!(probe.finished.is_cancelled());
},
"serve::cancellation_skips_connections_but_flushes_and_joins_workers",
shutdown_with_pending_request(ShutdownTrigger::Cancellation),
);
}
#[test]
fn drain_timeout_starts_a_separate_worker_shutdown_phase() {
fn self_restart_skips_connections_but_flushes_and_joins_workers() {
run_in_child(
"serve::drain_timeout_starts_a_separate_worker_shutdown_phase",
"serve::self_restart_skips_connections_but_flushes_and_joins_workers",
shutdown_with_pending_request(ShutdownTrigger::SelfRestart),
);
}
#[cfg(unix)]
#[test]
fn sigterm_skips_connections_but_flushes_and_joins_workers() {
run_in_child(
"serve::sigterm_skips_connections_but_flushes_and_joins_workers",
shutdown_with_pending_request(ShutdownTrigger::Signal("-TERM")),
);
}
#[cfg(unix)]
#[test]
fn sigint_skips_connections_but_flushes_and_joins_workers() {
run_in_child(
"serve::sigint_skips_connections_but_flushes_and_joins_workers",
shutdown_with_pending_request(ShutdownTrigger::Signal("-INT")),
);
}
enum ShutdownTrigger {
Cancellation,
SelfRestart,
#[cfg(unix)]
Signal(&'static str),
}
async fn shutdown_with_pending_request(trigger: ShutdownTrigger) {
let directory = tempfile::tempdir().unwrap();
let reservation = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let address = reservation.local_addr().unwrap();
let mut config = configuration(directory.path());
config.listen.port = address.port();
config.system_update.deployment_mode = "docker".to_owned();
config.system_update.build_type = "source".to_owned();
config.system_update.self_restart_enabled = true;
config.system_update.update_lock_file = directory.path().join("update.lock");
let host = gateway_host::initialize(config).await.unwrap();
drop(reservation);
let system = host.system_operations();
let cancellation = host.cancellation();
let connections = host.connection_lifecycle();
let guard = connections.try_register().unwrap();
let probe = start_workers(&host, true);
probe.started.cancelled().await;
let (router, entered, _finish_request) = request_router(&host, &probe);
let server = tokio::spawn(host.serve(router));
let client = connect_request(address).await;
entered.cancelled().await;
probe.sender.send(7).unwrap();
match trigger {
ShutdownTrigger::Cancellation => cancellation.cancel(),
ShutdownTrigger::SelfRestart => {
system.restart(Arc::new(AllowRestart)).await.unwrap();
}
#[cfg(unix)]
ShutdownTrigger::Signal(signal) => {
// 真实请求进入后信号监听已被 poll,只向隔离的当前测试子进程发送信号
assert!(
std::process::Command::new("kill")
.args([signal, &std::process::id().to_string()])
.status()
.unwrap()
.success()
);
}
}
tokio::time::timeout(Duration::from_secs(2), probe.cancelled.cancelled())
.await
.expect("shutdown must not wait for the blocked HTTP request or live guard");
assert!(connections.is_draining());
assert!(connections.try_register().is_err());
drop(guard);
assert!(connections.try_register().is_err());
assert!(
tokio::net::TcpStream::connect(address).await.is_err(),
"listener must close before worker cleanup finishes"
);
assert!(
!server.is_finished(),
"worker cleanup still needs to finish"
);
probe.finish_shutdown.add_permits(1);
server.await.unwrap().unwrap();
assert_eq!(*probe.writes.lock().unwrap(), [7]);
assert!(probe.finished.is_cancelled());
// 请求仍未结束,独立进程退出时终止连接和挂起 handler
drop(client);
}
#[test]
fn shutdown_aborts_workers_that_exceed_the_cleanup_budget() {
run_in_child(
"serve::shutdown_aborts_workers_that_exceed_the_cleanup_budget",
async {
let directory = tempfile::tempdir().unwrap();
let (host, address) = listening_host(directory.path()).await;
let cancellation = host.cancellation();
let connections = host.connection_lifecycle();
let probe = start_workers(&host, true);
probe.started.cancelled().await;
let (router, entered, finish_request) = request_router(&host, &probe);
let (router, entered, _finish_request) = request_router(&host, &probe);
let server = tokio::spawn(host.serve(router));
let client = connect_request(address).await;
entered.cancelled().await;
let started = tokio::time::Instant::now();
cancellation.cancel();
wait_for_draining(connections.as_ref()).await;
probe.cancelled.cancelled().await;
assert!(started.elapsed() >= Duration::from_secs(1));
assert!(!server.is_finished(), "worker cleanup was not joined");
assert!(!probe.finished.is_cancelled());
probe.finish_shutdown.add_permits(1);
server.await.unwrap().unwrap();
assert!(probe.finished.is_cancelled());
// 超时后真实连接可以仍在 axum 任务中;释放测试端,避免遗留挂起 handler
tokio::time::timeout(Duration::from_secs(3), server)
.await
.expect("worker cleanup budget must bound process shutdown")
.unwrap()
.unwrap();
probe.dropped.cancelled().await;
assert!(probe.cancelled.is_cancelled());
assert!(!probe.finished.is_cancelled());
drop(client);
finish_request.add_permits(1);
},
);
}
struct AllowRestart;
#[async_trait::async_trait]
impl SystemRestartPreflight for AllowRestart {
async fn prepare(&self, _: Option<SystemUpdateCandidate>) -> Result<(), SystemOperationError> {
Ok(())
}
}
#[test]
fn bind_failure_cancels_and_joins_workers_before_returning() {
run_in_child(
@@ -291,12 +306,6 @@ async fn connect_request(address: std::net::SocketAddr) -> tokio::net::TcpStream
client
}
async fn wait_for_draining(connections: &dyn gateway_core::lifecycle::ConnectionLifecycle) {
while !connections.is_draining() {
tokio::task::yield_now().await;
}
}
fn request_router(
host: &HostBundle,
probe: &WriterProbe,
@@ -319,7 +328,7 @@ fn request_router(
let _connection = connections.try_register().unwrap();
entered.cancel();
finish.acquire().await.unwrap().forget();
// 对应请求结束后写入终态的时机;drain 超时后的写入允许被拒绝
// 对应请求结束后写入终态的时机;关闭开始后的写入允许被拒绝
if sender.send(1).is_ok() {
"recorded"
} else {
@@ -59,7 +59,6 @@ pub fn configuration(directory: &Path) -> HostConfig {
request_dump_retention_days: 1,
},
system_update: Default::default(),
drain_timeout_seconds: 1,
worker_shutdown_timeout_seconds: 1,
}
}
@@ -4,8 +4,8 @@
"version": 3,
"replacement_version": 4,
"introduced_in": "3.20.0",
"last_counted_in": "3.22.0",
"remaining_releases": 2,
"last_counted_in": "3.22.2",
"remaining_releases": 0,
"migration": "将执行设置和来源中的 disable_fast 改为 fast_mode,使用 default、enabled、disabled 三态,并将 middleware 声明升级到 v4"
},
{
@@ -13,8 +13,8 @@
"version": 1,
"replacement_version": 2,
"introduced_in": "3.20.0",
"last_counted_in": "3.22.0",
"remaining_releases": 2,
"last_counted_in": "3.22.2",
"remaining_releases": 0,
"migration": "使用新版 SDK 读取 fast_mode 三态,并将 upstream_adapter 声明升级到 v2"
}
]
@@ -467,7 +467,11 @@ impl MiddlewareBody for PluginMiddlewareBody {
&& frame
.event()
.is_some_and(|event| event.wire_event().is_none());
if !facts_only && !valid_plugin_frame(self.framing, frame.bytes()) {
// 来源句柄已核对正文与交付归属,未改写的原始内容不依赖 JSON 解析
if frame.transformed()
&& !facts_only
&& !valid_plugin_frame(self.framing, frame.bytes())
{
return Err(invalid());
}
if self.pending_transformed {
@@ -447,6 +447,9 @@ async fn dispatch_websocket_queue(
}
fn websocket_payload(observation: &WebSocketResponseObservation) -> Result<Vec<u8>, ()> {
if let Some(raw) = observation.wire().raw_websocket_message() {
return Ok(raw.as_bytes().to_vec());
}
if let Some(raw) = observation.wire().raw_json_body() {
return Ok(raw.to_vec());
}
@@ -287,6 +287,15 @@ fn project_wire(wire: &ProtocolWireEvent) -> Result<WireEvent, PluginFault> {
WirePayload::RawSse {
frame: frame.to_vec(),
}
} else if let Some(message) = wire.raw_websocket_message() {
// 子模型回调采用 SDK 的事件流表达,不把 WebSocket 文本冒充 HTTP body
WirePayload::RawSse {
frame: gateway_protocol::openai::sse::encode_sse_event(
wire.event_type().unwrap_or_default(),
message,
)
.into_bytes(),
}
} else {
return Err(PluginFault::new(
ErrorCode::Fault,
@@ -212,10 +212,11 @@ impl BodyResource {
};
let transformed = frame.transformed();
let (bytes, framing, terminal, envelope) = frame.into_parts();
// canonical-only 事件没有客户端正文,仍须保留并允许读取其完整事实
let facts_only =
framing == MiddlewareFraming::RawBytes && bytes.is_empty() && envelope.is_some();
if (!facts_only && framing != self.expected_framing) || bytes.len() > maximum {
// 宿主原始字节可能没有 JSON 解析视图,包含未知 WS 文本和 canonical-only 事实
// 预期 framing 约束插件改写后的输出,不能阻止插件读取并原样归还来源
if (framing != MiddlewareFraming::RawBytes && framing != self.expected_framing)
|| bytes.len() > maximum
{
return Err(invalid());
}
let source_id = state.next_source_id.checked_add(1).ok_or_else(invalid)?;
@@ -21,6 +21,92 @@ use gateway_core::{
upstream::{OpaqueUpstreamValue, UpstreamTransport},
};
#[tokio::test]
async fn reading_websocket_facts_preserves_original_text_and_opaque_messages() {
let records = tempfile::tempdir().unwrap();
let marker = records.path().join("facts.jsonl");
let worker = std::fs::read(env!("CARGO_BIN_EXE_gateway-plugin-test-middleware")).unwrap();
let package = crate::support::package_with_contributions(
&worker,
Contributions::from([crate::support::contribution(
Capability::Middleware,
vec![Stage::Attempt],
vec!["openai".into()],
vec!["openai".into()],
)]),
);
let (_cache, runtime) = setup_package(vec![InstanceFixture {
id: "facts-reader",
configuration: serde_json::json!({"mode":"facts", "attempt":true, "facts_marker":marker}),
bindings: vec![binding(MIDDLEWARE_CONTRIBUTION, "attempt", 0, PluginFailurePolicy::Reject)],
}], package).await;
let generation = prepare(&runtime).await;
let plan = runtime
.execution_registry()
.middleware(&generation)
.unwrap();
let messages = [
"{ \"type\" : \"future.event\", \"number\" : 1e3, \"text\" : \"\\u0061\" }".to_owned(),
format!(
"{{\"type\":\"future.deep\",\"value\":{}0{}}}",
"[".repeat(140),
"]".repeat(140)
),
"future non-JSON text".to_owned(),
];
let events: Vec<_> = messages
.iter()
.map(|raw| {
let wire = match serde_json::from_str::<serde_json::Value>(raw) {
Ok(data) => ProtocolWireEvent::json("openai", None, data)
.unwrap()
.with_raw_websocket_message(raw.as_str()),
Err(_) => ProtocolWireEvent::raw_websocket("openai", raw.as_str()).unwrap(),
};
Ok(ProviderEvent::wire(wire))
})
.collect();
let mut stream = execute_attempt_middleware(
Some(&plan),
attempt_middleware_context_for_transport(Arc::default(), ClientTransport::WebSocket),
operation(),
ClientTransport::WebSocket,
Box::new(move |_, _| {
Box::pin(async move {
Ok(ProviderStream::new(
ProviderCallMetadata::new(
ProviderKind::new("openai").unwrap(),
UpstreamModelId::new("native-model").unwrap(),
ProviderAccountId::new("acct_original").unwrap(),
UpstreamTransport::new("websocket").unwrap(),
),
futures::stream::iter(events),
(),
))
})
}),
)
.await
.unwrap();
let mut delivered = Vec::new();
while let Some(event) = stream.next().await {
let event = event.unwrap();
assert!(!event.middleware_transformed());
delivered.push(
event
.wire_event()
.unwrap()
.raw_websocket_message()
.unwrap()
.to_owned(),
);
}
assert_eq!(delivered, messages);
assert_eq!(marker_lines(&marker, 4).await.len(), 4);
drop(stream);
runtime.shutdown().await;
}
#[tokio::test]
async fn facts_are_readable_without_replacing_or_duplicating_the_original_execution() {
let records = tempfile::tempdir().unwrap();
@@ -705,6 +705,13 @@ fn middleware_context(transport: ClientTransport) -> MiddlewareContext {
}
fn attempt_middleware_context(effects: Arc<ExecutionEffects>) -> MiddlewareContext {
attempt_middleware_context_for_transport(effects, ClientTransport::HttpJson)
}
fn attempt_middleware_context_for_transport(
effects: Arc<ExecutionEffects>,
transport: ClientTransport,
) -> MiddlewareContext {
MiddlewareContext::new(
MiddlewareTarget {
request_id: ModelRequestId::new("req_attempt_http").unwrap(),
@@ -712,7 +719,7 @@ fn attempt_middleware_context(effects: Arc<ExecutionEffects>) -> MiddlewareConte
attempt_index: Some(NonZeroU32::MIN),
operation: Some(OperationKind::Generate),
endpoint: "/v1/responses".into(),
transport: ClientTransport::HttpJson,
transport,
provider: Some(ProviderKind::new("openai").unwrap()),
model: Some("public-a".into()),
account_id: Some(ProviderAccountId::new("acct_openai").unwrap()),
@@ -1,5 +1,7 @@
//! PostgreSQL 迁移屏障、连接池与会话级资源预算
use std::time::Duration;
use sqlx::{
PgPool,
postgres::{PgConnectOptions, PgPoolOptions},
@@ -37,11 +39,34 @@ pub async fn connect_and_migrate(
)
.await
.map_err(|source| postgres_unavailable("connect PostgreSQL for migrations", source))?;
if let Err(error) = MIGRATOR.run(&migration_pool).await {
migration_pool.close().await;
let started = tokio::time::Instant::now();
tracing::info!(target: "gateway_startup", "开始检查并执行数据库迁移");
let result = {
let migration = MIGRATOR.run(&migration_pool);
tokio::pin!(migration);
let progress_interval = Duration::from_secs(15);
let mut progress = tokio::time::interval_at(started + progress_interval, progress_interval);
progress.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Skip);
loop {
tokio::select! {
biased;
result = &mut migration => break result,
_ = progress.tick() => {
// 迁移或迁移锁等待都不能被进度提示中断,不将已等待时间冒充完成比例
tracing::info!(target: "gateway_startup", elapsed_seconds = started.elapsed().as_secs(),
"数据库迁移仍在执行或等待锁,请等待完成,避免重复重启");
}
}
}
};
migration_pool.close().await;
if let Err(error) = result {
tracing::error!(target: "gateway_startup", elapsed_seconds = started.elapsed().as_secs_f64(),
"数据库迁移失败");
return Err(postgres_unavailable("apply PostgreSQL migrations", error));
}
migration_pool.close().await;
tracing::info!(target: "gateway_startup", elapsed_seconds = started.elapsed().as_secs_f64(),
"数据库迁移检查完成");
connect_pool(connect_options, pool_config, false).await
}
@@ -229,7 +229,7 @@ impl CodexProvider {
.map_err(map_selection_error)?;
}
let lease = Arc::new(lease);
match operation {
let passthrough_headers = match operation {
Operation::GenerateImage(image) => {
let Operation::GenerateImage(original) = &request.operation else {
return Err(provider_error(
@@ -250,6 +250,7 @@ impl CodexProvider {
.get("image_turn_id")
.and_then(Value::as_str)
.map(str::to_owned);
crate::transport::request::decode_passthrough_headers(image.payload().context())
}
Operation::Search(search) => {
if !matches!(&request.operation, Operation::Search(_))
@@ -267,6 +268,7 @@ impl CodexProvider {
.get("turn_metadata")
.and_then(Value::as_str)
.map(str::to_owned);
crate::transport::request::decode_passthrough_headers(search.payload().context())
}
_ => {
return Err(provider_error(
@@ -274,7 +276,7 @@ impl CodexProvider {
UpstreamSendState::NotSent,
));
}
}
};
if !context.is_diagnostic_required_account()
&& let Some(affinity) = affinity.as_ref()
&& let Some(turn) = affinity.turn_alias()
@@ -319,6 +321,7 @@ impl CodexProvider {
response_origin: request.response_origin,
endpoint_path: request.endpoint_path,
body: request.body,
passthrough_headers,
image_turn_id: request.image_turn_id,
turn_metadata,
context,
@@ -372,6 +375,7 @@ pub(super) struct ColdJsonResponse {
pub(super) response_origin: Url,
pub(super) endpoint_path: &'static str,
pub(super) body: Bytes,
pub(super) passthrough_headers: reqwest::header::HeaderMap,
pub(super) image_turn_id: Option<String>,
pub(super) turn_metadata: Option<String>,
pub(super) context: AttemptContext,
@@ -572,6 +576,7 @@ pub(super) async fn create_json_attempt(
response = request.client.post_raw_json(
request.endpoint_path,
request.body.clone(),
&request.passthrough_headers,
request.image_turn_id.as_deref(),
request_context,
) => response.map_err(CodexHandshakeAttemptError::Client),
@@ -910,10 +915,8 @@ pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
OpenAiPassiveQuotaObservation::new(response.rate_limit_headers);
let rate_limit_updates = response.rate_limit_updates;
let response_metadata_updates = response.response_metadata_updates;
// OpenAI 线路为透明代理:HTTP SSE 与 WebSocket 两条上游均启用 raw 透传,
// 下游按字节转发上游原文,避免 serde 往返改写数值/精度(大整数→f64、logprobs 等)
// WS 帧由 reducer 以 encode_sse_event(&event, raw) 逐字节内嵌上游原始 JSON
// (transport/protocol/websocket.rs),push_frames 抽出的 data 即上游原文
// 同协议保留原始帧;canonical 解析只提取计费、续接和状态事实
let websocket_transport = response.transport == CodexBackendTransport::WebSocket;
let mut decoder = CodexCanonicalDecoder::new(upstream_model.as_str())
.with_pricing(context.pricing().get("openai").and_then(|models| models.get(upstream_model.as_str())).cloned())
.with_reported_model(response.response_metadata.effective_model.as_deref())
@@ -1046,7 +1049,13 @@ pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
let first_event_changed =
observation_state.observe_stream_chunk(&chunk, context.timing_started_at());
let chunk_len = chunk.len();
let (mut events, canonical_failure) = match decoder.push(&chunk) {
let decoded = if websocket_transport {
let text = std::str::from_utf8(&chunk).map_err(|_| provider_error(ProviderErrorKind::Protocol, UpstreamSendState::Sent))?;
decoder.push_websocket(text)
} else {
decoder.push(&chunk)
};
let (mut events, canonical_failure) = match decoded {
CodexCanonicalOutcome::Events(events) => (events, None),
CodexCanonicalOutcome::Failed(failure) => {
let (events, error, semantic_output_seen) = failure.into_parts();
@@ -84,10 +84,8 @@ pub(super) async fn wait_for_replay_grace(deadline: Option<Instant>) {
/// 提交边界前的上游事件预取
///
/// 原始 chunk 计数而不是重编码后的 event 大小
/// 时间与字节阈值共同限定无感换号
/// 窗口;任一边界到达都会提交已缓存 wire,不能因网关私有资源规则伪造上游协议
/// 失败
/// 结构事件保留到宽限期结束、语义输出、终态或 EOF,再提交已缓存 wire
/// 大段配置回显不应提前结束无感换号窗口,原始 chunk 字节数只用于诊断
/// 一旦提交,后续事件不再具备无痕重放资格
pub(super) struct PreCommitClientEvents {
pending: Vec<ProviderEvent>,
@@ -102,7 +100,6 @@ pub(super) struct PreCommitClientEvents {
pub(super) enum PreCommitReleaseReason {
SemanticOutput,
Terminal,
ByteLimit,
GraceTimeout,
Eof,
}
@@ -141,9 +138,6 @@ impl PreCommitClientEvents {
if completed {
return self.commit_pending(PreCommitReleaseReason::Terminal);
}
if self.prefetched_bytes > MAX_STREAM_PREFETCH_BYTES {
return self.commit_pending(PreCommitReleaseReason::ByteLimit);
}
if starts_replay_grace && self.replay_grace_started_at.is_none() {
self.replay_grace_started_at = Some(Instant::now());
}
@@ -125,10 +125,6 @@ const WEBSOCKET_TRANSPORT: &str = "websocket";
// 在已观测到的 Codex OAuth 上游 16 MiB 附近消息边界前留出传输 metadata 余量
const WEBSOCKET_HTTP_FALLBACK_THRESHOLD_BYTES: usize = 15 * 1024 * 1024;
const MAX_COOKIE_HEADER_BYTES: usize = 16 * 1024;
/// 提交边界前预取 128 KiB 原始上游 chunk;容纳携带配置回显的前导事件,
/// 超过阈值后结束无感换号窗口(最后一个 chunk 可越过阈值),
/// 但不会把上游数据改写成协议失败
const MAX_STREAM_PREFETCH_BYTES: usize = 128 * 1024;
/// 短暂保留 response.created 等结构事件,让随后到达的明确拒绝可以无感换号;
/// 到期即放行,避免模型长时间思考时让客户端一直收不到首事件
const STREAM_REPLAY_GRACE: Duration = Duration::from_millis(2_500);
@@ -752,7 +748,6 @@ impl CodexProvider {
UpstreamSendState::NotSent,
));
}
validate_openai_reasoning(generate.protocol_payload().body())?;
let mut upstream = encode_generate_request(&generate, upstream_model.as_str(), None)
.map_err(map_request_error)?;
upstream.client_account_follow_only = crate::request_identity::follows_session_with_headers(
@@ -1057,26 +1052,3 @@ fn native_request_requirements(request: &GenerateRequest) -> CapabilityRequireme
))
.capability_requirements()
}
fn validate_openai_reasoning(body: &Map<String, Value>) -> Result<(), ProviderError> {
let Some(effort) = body
.get("reasoning")
.and_then(Value::as_object)
.and_then(|reasoning| reasoning.get("effort"))
else {
return Ok(());
};
let Some(effort) = effort.as_str() else {
return Err(provider_error(
ProviderErrorKind::InvalidRequest,
UpstreamSendState::NotSent,
));
};
if effort.is_empty() || effort.len() > 64 || effort.chars().any(char::is_control) {
return Err(provider_error(
ProviderErrorKind::InvalidRequest,
UpstreamSendState::NotSent,
));
}
Ok(())
}
@@ -1,8 +1,9 @@
//! Codex Responses SSE 到核心 canonical event 的单一解码边界
//! Codex Responses 原始 SSE 与 WebSocket 文本的旁路事实解码
use std::{
collections::{BTreeMap, BTreeSet},
fmt,
sync::Arc,
};
use bytes::Bytes;
@@ -29,6 +30,11 @@ use super::usage::{
OpenAiBillingUsage, WebSearchPricing, normalize_service_tier, web_search_pricing,
};
enum RawResponseFrame {
Sse(Bytes),
WebSocket(Arc<str>),
}
const CONTENTS_PER_OUTPUT: u32 = 1_024;
/// 单 attempt 的增量 Responses decoder
@@ -181,6 +187,23 @@ impl CodexCanonicalDecoder {
self
}
/// 消费完整 WebSocket 文本,保留原文并旁路提取 canonical facts
pub fn push_websocket(&mut self, message: &str) -> CodexCanonicalOutcome {
self.timing_signals = ResponseEventSignals::default();
let mut output = Vec::new();
let event = SseEvent {
event: None,
data: message.to_owned(),
id: None,
retry: None,
};
let raw = Some(RawResponseFrame::WebSocket(Arc::from(message)));
match self.decode_one(event, raw, &mut output) {
Ok(()) => CodexCanonicalOutcome::Events(output),
Err(error) => self.failure(output, error),
}
}
/// 使用最终发送给上游的请求档位估算费用,不由响应回显覆盖
#[must_use]
pub fn with_requested_service_tier(mut self, service_tier: Option<&str>) -> Self {
@@ -304,7 +327,7 @@ impl CodexCanonicalDecoder {
continue;
}
for (index, event) in events.into_iter().enumerate() {
let raw_sse_frame = (index == 0).then(|| raw.clone());
let raw_sse_frame = (index == 0).then(|| RawResponseFrame::Sse(raw.clone()));
if let Err(error) = self.decode_one(event, raw_sse_frame, &mut output) {
return self.failure(output, error);
}
@@ -316,18 +339,27 @@ impl CodexCanonicalDecoder {
fn decode_one(
&mut self,
event: SseEvent,
raw_sse_frame: Option<Bytes>,
raw_sse_frame: Option<RawResponseFrame>,
output: &mut Vec<ProviderEvent>,
) -> Result<(), CodexCanonicalError> {
if event.data.trim() == "[DONE]" {
if event.data.trim() == "[DONE]"
&& !matches!(raw_sse_frame, Some(RawResponseFrame::WebSocket(_)))
{
return Ok(());
}
let value = match serde_json::from_str::<Value>(&event.data) {
Ok(value) => value,
Err(_) => {
if let Some(raw_sse_frame) = raw_sse_frame
&& let Ok(wire) = ProtocolWireEvent::raw_sse("openai", raw_sse_frame)
{
let wire = match raw_sse_frame {
Some(RawResponseFrame::Sse(raw)) => {
ProtocolWireEvent::raw_sse("openai", raw).ok()
}
Some(RawResponseFrame::WebSocket(raw)) => {
ProtocolWireEvent::raw_websocket("openai", raw).ok()
}
None => None,
};
if let Some(wire) = wire {
output.push(ProviderEvent::wire(wire));
}
return Ok(());
@@ -344,10 +376,15 @@ impl CodexCanonicalDecoder {
}
if event_type == Some("responsesapi.websocket_timing") && self.started && !self.completed {
// 无 ID 的官方计时帧仅属于当前已开始的响应,复用连接的尾帧不能跨边界归属
let response_id = value
.get("response_id")
.or_else(|| value.pointer("/response/id"));
if response_id.is_none_or(|id| id.as_str() == self.response_id.as_deref()) {
let response_ids = [
value.get("response_id"),
value.pointer("/response/id"),
value.pointer("/timing_metrics/response_id"),
];
if response_ids.into_iter().flatten().all(|id| {
id.as_str()
.is_some_and(|id| Some(id) == self.response_id.as_deref())
}) {
self.upstream_timing_metrics
.merge(ResponseTimingMetrics::from_event(&value));
}
@@ -425,19 +462,27 @@ impl CodexCanonicalDecoder {
fn wire_for_event(
event: SseEvent,
value: Value,
raw_sse_frame: Option<Bytes>,
raw_sse_frame: Option<RawResponseFrame>,
) -> Option<ProtocolWireEvent> {
let event_type = event.event;
let event_type = event
.event
.or_else(|| value.get("type").and_then(Value::as_str).map(str::to_owned));
let sse_id = event.id;
match raw_sse_frame {
Some(raw_sse_frame) => ProtocolWireEvent::json_with_raw_sse_metadata(
"openai",
event_type,
value,
raw_sse_frame,
sse_id,
event.retry,
),
Some(RawResponseFrame::WebSocket(raw)) => {
ProtocolWireEvent::json("openai", event_type, value)
.map(|wire| wire.with_raw_websocket_message(raw))
}
Some(RawResponseFrame::Sse(raw_sse_frame)) => {
ProtocolWireEvent::json_with_raw_sse_metadata(
"openai",
event_type,
value,
raw_sse_frame,
sse_id,
event.retry,
)
}
None => ProtocolWireEvent::json_with_sse_metadata(
"openai",
event_type,
@@ -609,9 +609,9 @@ pub type CodexRateLimitUpdates = CodexWebSocketRateLimitUpdates;
/// 响应头之后在 live 流中采集的请求级 metadata 更新
pub type CodexResponseMetadataUpdates = CodexWebSocketResponseMetadataUpdates;
/// Codex Responses 上游 live SSE 响应
/// Codex Responses 上游流式响应
pub struct CodexBackendStreamingResponse {
/// 上游 SSE 字节流
/// HTTP 为 SSE 字节块,WebSocket 为完整 UTF-8 文本消息
pub body: CodexBackendSseStream,
/// 实际使用的上游传输
pub transport: CodexBackendTransport,
@@ -21,6 +21,7 @@ impl CodexBackendClient {
&self,
endpoint_path: &'static str,
body: Bytes,
passthrough_headers: &reqwest::header::HeaderMap,
image_turn_id: Option<&str>,
context: CodexRequestContext<'_>,
) -> CodexClientResult<CodexBackendJsonResponse> {
@@ -35,6 +36,11 @@ impl CodexBackendClient {
};
let profile = self.profile.snapshot();
let mut headers = self.model_request_headers(&profile, context)?;
super::headers::append_passthrough_headers(&mut headers, passthrough_headers);
// 独立端点没有 Responses 原连接归属,续接状态不能跨账号继承
// turn metadata 随后仅使用当前 lease 已处理的值
headers.remove("x-codex-turn-state");
headers.remove("x-codex-turn-metadata");
headers.insert(CONTENT_TYPE, HeaderValue::from_static("application/json"));
insert_optional_protocol_header(&mut headers, "x-codex-image-turn-id", image_turn_id);
insert_optional_protocol_header(
@@ -629,8 +629,14 @@ async fn await_websocket_delivery_boundary(
}
fn is_websocket_lifecycle_prelude(frame: &[u8]) -> bool {
frame.starts_with(b"event: response.created\n")
|| frame.starts_with(b"event: response.in_progress\n")
serde_json::from_slice::<serde_json::Value>(frame)
.ok()
.is_some_and(|value| {
matches!(
value.get("type").and_then(serde_json::Value::as_str),
Some("response.created" | "response.in_progress")
)
})
}
async fn read_model_catalog_body(response: ReqwestResponse) -> CodexClientResult<Vec<u8>> {
@@ -222,7 +222,7 @@ impl CodexBackendClient {
None => format!("model={}", request.model()),
};
insert_optional_protocol_header(&mut headers, "x-codex-routing-hint", Some(&routing_hint));
append_passthrough_headers(&mut headers, request);
append_passthrough_headers(&mut headers, &request.passthrough_headers);
self.append_middleware_headers(&mut headers)?;
Ok(headers)
}
@@ -244,8 +244,8 @@ impl CodexBackendClient {
}
}
fn append_passthrough_headers(headers: &mut HeaderMap, request: &CodexResponsesRequest) {
for name in request.passthrough_headers.keys() {
pub(super) fn append_passthrough_headers(headers: &mut HeaderMap, passthrough: &HeaderMap) {
for name in passthrough.keys() {
// 身份与传输字段只由画像/正文生成;其余协议头保留原始多值字节
if is_managed_identity_header(name.as_str())
|| matches!(
@@ -262,7 +262,7 @@ fn append_passthrough_headers(headers: &mut HeaderMap, request: &CodexResponsesR
continue;
}
headers.remove(name);
for value in request.passthrough_headers.get_all(name) {
for value in passthrough.get_all(name) {
headers.append(name.clone(), value.clone());
}
}
@@ -251,22 +251,54 @@ pub(crate) struct ResponseTimingMetrics {
impl ResponseTimingMetrics {
pub(crate) fn from_event(value: &Value) -> Self {
let metrics = value.get("timing_metrics");
let milliseconds = |key: &str| {
metrics?
.get(key)?
.as_f64()
.filter(|value| value.is_finite() && *value >= 0.0 && *value < i64::MAX as f64)
let Some(metrics) = value.get("timing_metrics") else {
return Self::default();
};
let valid_milliseconds =
|value: &f64| value.is_finite() && *value >= 0.0 && *value < i64::MAX as f64;
let milliseconds =
|source: &Value, key: &str| source.get(key)?.as_f64().filter(valid_milliseconds);
if let Some(path) = metrics.get("critical_path") {
// 外层 logical_turn 可累计多次响应与工具等待,只取完整的当前响应阶段
if path.get("scope").and_then(Value::as_str) != Some("response")
|| path.get("coverage").and_then(Value::as_str) != Some("complete")
|| path.get("boundary_type").and_then(Value::as_str)
!= Some("actionable_output_item_done")
{
return Self::default();
}
// API 开销由推理前与其他处理构成,缺少任一分项时不能按零补齐
let overhead = milliseconds(path, "responses_pre_inference_ms")
.zip(milliseconds(path, "responses_other_ms"))
.map(|(before, other)| before + other)
.filter(valid_milliseconds);
return Self {
upstream_api_overhead_ms: overhead,
upstream_engine_ms: milliseconds(path, "engine_wall_ms"),
..Self::default()
};
}
// 直接返回的专项指标保留原口径,明确标为整轮累计的值不能归入单次请求
if metrics
.get("timing_scope")
.is_some_and(|scope| scope.as_str() != Some("response"))
{
return Self::default();
}
Self {
upstream_api_overhead_ms: milliseconds(
metrics,
"responses_duration_excl_engine_and_client_tool_time_ms",
),
upstream_engine_ms: milliseconds("engine_service_total_ms"),
upstream_engine_iapi_ttft_ms: milliseconds("engine_iapi_ttft_total_ms"),
upstream_engine_service_ttft_ms: milliseconds("engine_service_ttft_total_ms"),
upstream_engine_iapi_tbt_ms: milliseconds("engine_iapi_tbt_across_engine_calls_ms"),
upstream_engine_ms: milliseconds(metrics, "engine_service_total_ms"),
upstream_engine_iapi_ttft_ms: milliseconds(metrics, "engine_iapi_ttft_total_ms"),
upstream_engine_service_ttft_ms: milliseconds(metrics, "engine_service_ttft_total_ms"),
upstream_engine_iapi_tbt_ms: milliseconds(
metrics,
"engine_iapi_tbt_across_engine_calls_ms",
),
upstream_engine_service_tbt_ms: milliseconds(
metrics,
"engine_service_tbt_across_engine_calls_ms",
),
}
@@ -54,14 +54,14 @@ pub struct WebSocketAuditArtifact {
/// 将一条公开 WebSocket JSON 事件编码为 SSE 帧
pub fn websocket_event_to_sse_frame(raw: &str) -> Option<String> {
let value = serde_json::from_str::<Value>(raw).ok()?;
let value = serde_json::from_str::<Value>(raw).unwrap_or(Value::Null);
websocket_event_frame(&value, raw)
}
/// 复用已解析的 JSON,剥离内部帧并原样转发公开事件
pub(crate) fn websocket_event_frame(value: &Value, raw: &str) -> Option<String> {
let event = websocket_event_type(value)?;
if is_internal_websocket_event(event) || event == "response.metadata" {
let event = websocket_event_type(value).unwrap_or_default();
if is_internal_websocket_event(event) {
return None;
}
Some(encode_sse_event(event, raw))
@@ -107,20 +107,16 @@ fn is_websocket_metadata_event(event: Option<&str>) -> bool {
pub(crate) const WEBSOCKET_CONNECTION_LIMIT_REACHED_CODE: &str =
"websocket_connection_limit_reached";
/// 若首个可投递 SSE 帧是连接寿命限制错误,保留原始失败事实而不改变恢复分类
/// 若首个可投递 WebSocket 帧是连接寿命限制错误,保留原始失败事实而不改变恢复分类
pub(crate) fn websocket_connection_limit_failure(frame: &[u8]) -> Option<ResponsesSseFailure> {
let text = std::str::from_utf8(frame).ok()?;
let event = gateway_protocol::openai::sse::parse_sse_events(text)
.ok()?
.into_iter()
.next()?;
if event.event.as_deref() != Some("error") {
let value = serde_json::from_str::<Value>(text).ok()?;
if websocket_event_type(&value) != Some("error") {
return None;
}
let value = serde_json::from_str::<Value>(&event.data).ok()?;
let code = value.pointer("/error/code").and_then(Value::as_str)?;
(code == WEBSOCKET_CONNECTION_LIMIT_REACHED_CODE).then(|| {
let mut failure = ResponsesSseFailure::from_raw_event("error", &event.data, &value);
let mut failure = ResponsesSseFailure::from_raw_event("error", text, &value);
if value
.pointer("/error/message")
.and_then(Value::as_str)
@@ -754,7 +754,7 @@ fn apply_protocol_context(request: &mut CodexResponsesRequest, context: &Map<Str
}
}
fn decode_passthrough_headers(context: &Map<String, Value>) -> HeaderMap {
pub(crate) fn decode_passthrough_headers(context: &Map<String, Value>) -> HeaderMap {
let mut headers = HeaderMap::new();
let Some(entries) = context
.get(PASSTHROUGH_HEADERS_CONTEXT_KEY)
@@ -0,0 +1,60 @@
//! WebSocket 有界消息通道与原连接文本透传;空闲读取与响应正文消费互斥
use std::{sync::Arc, time::Duration};
use async_trait::async_trait;
use gateway_core::engine::response_control::{
ResponseControlTransport, ResponseControlUnavailable,
};
use tokio::sync::{Mutex, mpsc, oneshot};
use tokio_tungstenite::tungstenite::{self, Message};
pub(super) const WEBSOCKET_SEND_TIMEOUT: Duration = Duration::from_secs(5 * 60);
pub(super) enum PumpCommand {
Send {
message: Message,
ack: oneshot::Sender<Result<(), tungstenite::Error>>,
},
}
pub(super) type SharedMessages = Arc<Mutex<mpsc::Receiver<Result<Message, tungstenite::Error>>>>;
pub(super) struct PumpControl {
pub(super) tx_command: mpsc::Sender<PumpCommand>,
pub(super) rx_message: SharedMessages,
}
pub(super) async fn send_message(
commands: &mpsc::Sender<PumpCommand>,
message: Message,
) -> Result<(), tungstenite::Error> {
let (ack, rx_ack) = oneshot::channel();
commands
.send(PumpCommand::Send { message, ack })
.await
.map_err(|_| tungstenite::Error::ConnectionClosed)?;
rx_ack
.await
.unwrap_or(Err(tungstenite::Error::ConnectionClosed))
}
#[async_trait]
impl ResponseControlTransport for PumpControl {
async fn send(&self, payload: &str) -> Result<(), ResponseControlUnavailable> {
tokio::time::timeout(
WEBSOCKET_SEND_TIMEOUT,
send_message(&self.tx_command, Message::Text(payload.to_owned().into())),
)
.await
.map_err(|_| ResponseControlUnavailable)?
.map_err(|_| ResponseControlUnavailable)
}
async fn receive(&self) -> Result<String, ResponseControlUnavailable> {
match self.rx_message.lock().await.recv().await {
Some(Ok(Message::Text(text))) => Ok(text.to_string()),
_ => Err(ResponseControlUnavailable),
}
}
}
@@ -23,12 +23,12 @@ const WEBSOCKET_STREAM_BUFFER: usize = 16;
pub(super) use self::stream::{WebSocketStreamPoolReturn, stream_websocket_response};
/// Responses WebSocket live SSE 交互结果
/// Responses WebSocket 文本流交互结果
pub struct CodexWebSocketStreamingExchange {
/// 关联请求日志与底层 pump 生命周期日志的连接标识
pub(crate) websocket_connection_id: Uuid,
/// 由 WebSocket 事件转换出的 live SSE 字节流
pub body: CodexWebSocketSseStream,
/// 上游完整 WebSocket 文本消息流
pub body: CodexWebSocketTextStream,
/// 上游为本次响应返回的首个 turn state
pub turn_state: Option<String>,
/// 上游握手响应里的 `set-cookie` 列表
@@ -49,8 +49,8 @@ pub struct CodexWebSocketStreamingExchange {
pub response_metadata: CodexResponseMetadata,
}
/// Responses WebSocket live SSE 字节流
pub type CodexWebSocketSseStream =
/// Responses WebSocket 完整文本消息流
pub type CodexWebSocketTextStream =
Pin<Box<dyn Stream<Item = Result<Bytes, CodexWebSocketExchangeError>> + Send + 'static>>;
/// live 流中的结构化限流动态更新
pub type CodexWebSocketRateLimitUpdates = Arc<Mutex<Vec<ParsedRateLimits>>>;
@@ -4,9 +4,8 @@ use gateway_protocol::openai::events;
use serde_json::Value;
use crate::transport::protocol::websocket::{
websocket_event_frame, websocket_event_type, websocket_metadata_headers,
websocket_metadata_turn_state, websocket_response_completed_id,
websocket_response_is_interrupted,
websocket_event_type, websocket_metadata_headers, websocket_metadata_turn_state,
websocket_response_completed_id, websocket_response_is_interrupted,
};
use crate::transport::response_meta;
@@ -24,14 +23,12 @@ pub(in crate::transport::websocket) enum WebSocketTerminalKind {
pub(super) enum ExchangeAction {
RateLimits(events::ParsedRateLimits),
Forward {
frame: String,
terminal: Option<WebSocketTerminalKind>,
},
Ignore,
}
pub(super) struct ReducedWebSocketEvent {
pub(super) created_response_id: Option<String>,
pub(super) action: ExchangeAction,
pub(super) diagnostic_event_type: Option<String>,
pub(super) turn_state_update: Option<String>,
@@ -42,12 +39,10 @@ pub(super) fn reduce_websocket_event(
metadata: &mut CodexWebSocketConnectionMetadata,
continuation: &mut WebSocketContinuationState,
) -> Result<ReducedWebSocketEvent, CodexWebSocketExchangeError> {
// 每帧只解析一次 JSON,后续提取全部复用同一 Value;
// 不可解析的帧不承载可路由的事件类型,忽略
// 解析只服务连接状态和观测,未知或不可解析文本仍交付下游
let Ok(value) = serde_json::from_str::<Value>(raw) else {
return Ok(ReducedWebSocketEvent {
created_response_id: None,
action: ExchangeAction::Ignore,
action: ExchangeAction::Forward { terminal: None },
diagnostic_event_type: None,
turn_state_update: None,
});
@@ -58,7 +53,6 @@ pub(super) fn reduce_websocket_event(
metadata.rate_limit_headers.extend(headers);
return Ok(ReducedWebSocketEvent {
action: ExchangeAction::RateLimits(parsed),
created_response_id: None,
diagnostic_event_type,
turn_state_update: None,
});
@@ -93,15 +87,12 @@ pub(super) fn reduce_websocket_event(
Some("response.failed" | "error") => Some(WebSocketTerminalKind::Failed),
_ => None,
};
let action = match websocket_event_frame(&value, raw) {
Some(frame) => ExchangeAction::Forward { frame, terminal },
None => ExchangeAction::Ignore,
let action = if event == Some("codex.rate_limits") {
ExchangeAction::Ignore
} else {
ExchangeAction::Forward { terminal }
};
Ok(ReducedWebSocketEvent {
created_response_id: (event == Some("response.created"))
.then(|| value.pointer("/response/id").and_then(Value::as_str))
.flatten()
.map(ToOwned::to_owned),
action,
diagnostic_event_type,
turn_state_update,
@@ -4,7 +4,7 @@ use std::{sync::Arc, time::Duration};
use bytes::Bytes;
use gateway_core::diagnostics::{TraceContext, diagnostic_json};
use gateway_core::engine::response_control::{ActiveResponseInterrupt, ResponseControl};
use gateway_core::engine::response_control::ResponseControl;
use serde_json::json;
use tokio::sync::{Mutex, mpsc};
use tokio_util::sync::CancellationToken;
@@ -60,7 +60,7 @@ impl StreamWebSocketDiscardReason {
}
pub(in crate::transport::websocket) fn stream_websocket_response(
websocket: PumpedWebSocket,
mut websocket: PumpedWebSocket,
metadata: CodexWebSocketConnectionMetadata,
pool_return: Option<WebSocketStreamPoolReturn>,
reused_connection: bool,
@@ -68,6 +68,7 @@ pub(in crate::transport::websocket) fn stream_websocket_response(
trace: TraceContext,
response_control: Option<ResponseControl>,
) -> CodexWebSocketStreamingExchange {
websocket.bind_control(response_control.as_ref());
let websocket_connection_id = websocket.connection_id();
let response_metadata = metadata.clone();
let rate_limit_updates = Arc::new(Mutex::new(Vec::new()));
@@ -87,7 +88,6 @@ pub(in crate::transport::websocket) fn stream_websocket_response(
);
let forward = async move {
forward_websocket_response_stream(WebSocketStreamForwardState {
response_control,
websocket,
metadata,
pool_return,
@@ -127,7 +127,6 @@ pub(in crate::transport::websocket) fn stream_websocket_response(
}
struct WebSocketStreamForwardState {
response_control: Option<ResponseControl>,
trace: TraceContext,
websocket: PumpedWebSocket,
metadata: CodexWebSocketConnectionMetadata,
@@ -142,7 +141,6 @@ struct WebSocketStreamForwardState {
async fn forward_websocket_response_stream(state: WebSocketStreamForwardState) {
let WebSocketStreamForwardState {
response_control,
trace,
mut websocket,
mut metadata,
@@ -160,10 +158,6 @@ async fn forward_websocket_response_stream(state: WebSocketStreamForwardState) {
.map(|pool_return| std::mem::take(&mut pool_return.continuation))
.unwrap_or_default();
let mut last_event_type = None;
let mut active_interrupt: Option<ActiveResponseInterrupt> = None;
let mut interrupt_requested: futures::future::BoxFuture<'static, ()> =
Box::pin(std::future::pending());
let mut interrupt_sent = false;
// 官方 run_websocket_response_stream 按响应消费 metadata 事件;它们不属于握手头
// 复用连接时恢复握手快照,避免上一轮模型信息及普通响应头随每次请求累积
let opening_response_metadata = metadata.response_metadata.clone();
@@ -173,7 +167,6 @@ async fn forward_websocket_response_stream(state: WebSocketStreamForwardState) {
// 客户端断开下游 SSE 流:立即丢弃连接并释放池 slot,
// 不再傻等上游 idle 超时(否则同会话后续请求会一直 bypass/busy)
() = tx.closed() => {
drop(active_interrupt.take());
trace.record("upstream.cancelled", json!({"reason": "receiver_dropped"}));
discard_stream_websocket(
websocket,
@@ -183,7 +176,6 @@ async fn forward_websocket_response_stream(state: WebSocketStreamForwardState) {
return;
}
() = shutdown.cancelled() => {
drop(active_interrupt.take());
trace.record("upstream.cancelled", json!({"reason": "pool_shutdown"}));
discard_stream_websocket(
websocket,
@@ -192,24 +184,6 @@ async fn forward_websocket_response_stream(state: WebSocketStreamForwardState) {
).await;
return;
}
() = &mut interrupt_requested, if !interrupt_sent => {
let Some(active) = active_interrupt.as_ref() else { continue };
let payload = json!({
"type": "response.interrupt",
"response_id": active.response_id(),
"mode": "discard_partial_items",
}).to_string();
// 复用正在消费此响应的原 socket;信号不经过选号或新请求恢复
if let Err(error) = super::super::handshake::send_websocket_request(&websocket, &payload).await {
drop(active_interrupt.take());
discard_stream_websocket(websocket, pool_return, StreamWebSocketDiscardReason::UpstreamReceiveFailed).await;
let _ = tx.send(Err(error)).await;
return;
}
interrupt_sent = true;
trace.record("upstream.interrupt.sent", json!({"mode": "discard_partial_items"}));
continue;
}
message = next_websocket_message(
&mut websocket,
stream_idle_timeout.filter(|timeout| !timeout.is_zero()).unwrap_or(super::super::pool::DEFAULT_STREAM_IDLE_TIMEOUT),
@@ -218,8 +192,6 @@ async fn forward_websocket_response_stream(state: WebSocketStreamForwardState) {
let message = match message {
Ok(message) => message,
Err(error) => {
// 先撤销 owner 再发布失败;下一次 attempt 不等待旧 socket 异步清理
drop(active_interrupt.take());
let observation = connection_observation(&websocket, &error);
trace.record(
"upstream.read.failed",
@@ -255,7 +227,6 @@ async fn forward_websocket_response_stream(state: WebSocketStreamForwardState) {
let raw = match message {
tungstenite::Message::Text(text) => text.to_string(),
tungstenite::Message::Binary(bytes) => {
drop(active_interrupt.take());
trace.capture("upstream.binary", &bytes);
let error = CodexWebSocketExchangeError::UnexpectedBinaryEvent;
let observation = connection_observation(&websocket, &error);
@@ -271,7 +242,6 @@ async fn forward_websocket_response_stream(state: WebSocketStreamForwardState) {
return;
}
tungstenite::Message::Close(frame) => {
drop(active_interrupt.take());
if let Some(frame) = &frame {
trace.dump("upstream.close.reason", frame.reason.as_bytes());
}
@@ -310,7 +280,6 @@ async fn forward_websocket_response_stream(state: WebSocketStreamForwardState) {
let reduced = match reduce_websocket_event(&raw, &mut metadata, &mut continuation) {
Ok(reduced) => reduced,
Err(error) => {
drop(active_interrupt.take());
let observation =
(!matches!(error.classified(), CodexWebSocketExchangeError::Upstream(_)))
.then(|| connection_observation(&websocket, &error));
@@ -331,31 +300,6 @@ async fn forward_websocket_response_stream(state: WebSocketStreamForwardState) {
if let Some(event_type) = reduced.diagnostic_event_type {
last_event_type = Some(event_type);
}
if active_interrupt.is_none()
&& let (Some(control), Some(response_id)) =
(&response_control, reduced.created_response_id)
{
active_interrupt = control.activate(response_id);
if let Some(active) = &active_interrupt {
// 跨消息复用同一个等待 future,避免每帧留下新的取消等待者
interrupt_requested = Box::pin(active.requested());
}
if active_interrupt.is_none() {
discard_stream_websocket(
websocket,
pool_return,
StreamWebSocketDiscardReason::FailedResponse,
)
.await;
let _ = tx
.send(Err(CodexWebSocketExchangeError::PostSendAmbiguous {
message: "response control already has an active owner".to_owned(),
source: None,
}))
.await;
return;
}
}
if let Some(turn_state) = reduced.turn_state_update {
let mut pending = response_metadata_updates.lock().await;
if pending.turn_state.is_none() {
@@ -365,20 +309,15 @@ async fn forward_websocket_response_stream(state: WebSocketStreamForwardState) {
if let Some(model) = metadata.response_metadata.effective_model.as_ref() {
response_metadata_updates.lock().await.reported_model = Some(model.clone());
}
let (frame, terminal) = match reduced.action {
let terminal = match reduced.action {
ExchangeAction::RateLimits(rate_limits) => {
rate_limit_updates.lock().await.push(rate_limits);
continue;
}
ExchangeAction::Forward { frame, terminal } => (frame, terminal),
ExchangeAction::Forward { terminal } => terminal,
ExchangeAction::Ignore => continue,
};
if terminal.is_some() {
// 先撤销控制权,再交付终态和归还连接,避免迟到的中断影响下一轮
drop(active_interrupt.take());
}
if tx.send(Ok(Bytes::from(frame))).await.is_err() {
drop(active_interrupt.take());
if tx.send(Ok(Bytes::from(raw))).await.is_err() {
trace.record(
"upstream.forward.failed",
json!({"reason": "receiver_dropped"}),
@@ -456,7 +395,6 @@ async fn forward_websocket_response_stream(state: WebSocketStreamForwardState) {
last_event_type,
},
};
drop(active_interrupt.take());
let observation = connection_observation(&websocket, &error);
discard_stream_websocket(
websocket,
@@ -33,9 +33,10 @@ use super::{
pump::{PumpKeepalive, PumpLogContext, PumpedWebSocket, RawWsStream},
};
use super::control::WEBSOCKET_SEND_TIMEOUT;
const WEBSOCKET_EXTENSIONS: &str = "permessage-deflate; client_max_window_bits";
const WEBSOCKET_CONNECT_TIMEOUT: Duration = Duration::from_secs(15);
const WEBSOCKET_SEND_TIMEOUT: Duration = Duration::from_secs(5 * 60);
impl CodexWebSocketConnection {
/// 构造 Responses WebSocket 连接描述
@@ -2,6 +2,7 @@
mod audit;
mod breaker;
mod control;
mod coordinator;
mod error;
mod exchange;
@@ -23,7 +24,7 @@ pub use self::{
error::{CodexWebSocketCloseError, CodexWebSocketExchangeError, CodexWebSocketUpstreamError},
exchange::{
CodexWebSocketRateLimitUpdates, CodexWebSocketResponseMetadataUpdates,
CodexWebSocketSseStream, CodexWebSocketStreamingExchange,
CodexWebSocketStreamingExchange, CodexWebSocketTextStream,
},
handshake::responses_websocket_endpoint,
model::{
@@ -13,6 +13,8 @@
//! - 接收:`next` 从 message channel 取出 pump 转发的入站帧(`Ping`/`Pong` 已被 pump 吞掉)
//! - 入站缓冲满时暂停读取 socket 和主动探活,仅继续处理发送/关闭命令;消费恢复后按原顺序继续转发
use gateway_core::engine::response_control::{ResponseControl, ResponseControlTransport};
use std::sync::{
Arc, Mutex,
atomic::{AtomicBool, Ordering},
@@ -21,14 +23,17 @@ use std::time::Duration;
use futures::{Sink, SinkExt, Stream, StreamExt};
use tokio::{
sync::{mpsc, oneshot},
sync::mpsc,
task::JoinHandle,
time::{Instant, MissedTickBehavior},
};
use tokio_tungstenite::tungstenite::Message;
use uuid::Uuid;
use super::CodexWebSocketCloseError;
use super::{
CodexWebSocketCloseError,
control::{PumpCommand, PumpControl, SharedMessages, send_message},
};
/// 底层 tungstenite WebSocket 流
pub(crate) trait WebSocketIo:
@@ -89,13 +94,6 @@ impl PumpKeepalive {
}
}
enum PumpCommand {
Send {
message: Message,
ack: oneshot::Sender<Result<(), tungstenite::Error>>,
},
}
const PUMP_MESSAGE_BUFFER: usize = 64;
#[derive(Debug, Clone)]
@@ -284,7 +282,8 @@ impl PumpLifecycleState {
pub(crate) struct PumpedWebSocket {
connection_id: Uuid,
tx_command: mpsc::Sender<PumpCommand>,
rx_message: mpsc::Receiver<Result<Message, tungstenite::Error>>,
rx_message: SharedMessages,
control: Option<Arc<dyn ResponseControlTransport>>,
closed: Arc<AtomicBool>,
lifecycle: Arc<Mutex<PumpLifecycleState>>,
pump: Option<JoinHandle<()>>,
@@ -335,7 +334,8 @@ impl PumpedWebSocket {
Self {
connection_id,
tx_command,
rx_message,
rx_message: Arc::new(tokio::sync::Mutex::new(rx_message)),
control: None,
closed,
lifecycle,
pump: Some(pump),
@@ -344,25 +344,26 @@ impl PumpedWebSocket {
/// 通过 pump 发送一帧,返回底层 `send` 的结果
pub(crate) async fn send(&self, message: Message) -> Result<(), tungstenite::Error> {
let (ack, rx_ack) = oneshot::channel();
if self
.tx_command
.send(PumpCommand::Send { message, ack })
.await
.is_err()
{
return Err(tungstenite::Error::ConnectionClosed);
}
rx_ack
.await
.unwrap_or(Err(tungstenite::Error::ConnectionClosed))
send_message(&self.tx_command, message).await
}
/// 每轮发送后替换控制 owner,旧执行的弱引用不能控制下一轮连接
pub(crate) fn bind_control(&mut self, control: Option<&ResponseControl>) {
self.control = control.map(|control| {
let transport: Arc<dyn ResponseControlTransport> = Arc::new(PumpControl {
tx_command: self.tx_command.clone(),
rx_message: Arc::clone(&self.rx_message),
});
control.bind(&transport);
transport
});
}
/// 取出下一帧入站消息(`Ping`/`Pong` 已被 pump 处理,不会到达这里)
///
/// 返回 `None` 表示连接已结束(pump 已退出且缓冲已排空)
pub(crate) async fn next(&mut self) -> Option<Result<Message, tungstenite::Error>> {
self.rx_message.recv().await
self.rx_message.lock().await.recv().await
}
/// 连接是否已被后台 pump 判定关闭/失活
@@ -3,6 +3,7 @@
mod account_isolation;
mod capacity;
mod error_details;
mod passthrough;
mod precommit;
mod response_interrupt;
mod session_binding;
@@ -7972,6 +7973,7 @@ async fn responses_should_observe_and_bill_the_outbound_service_tier_on_both_tra
let response_frames = format!(
"event: response.created\ndata: {created}\n\nevent: response.completed\ndata: {completed}\n\n"
);
let expected_messages = vec![created.to_string(), completed.to_string()];
let (base_url, http_server, websocket_server) = if use_websocket {
let listener = TcpListener::bind("127.0.0.1:0").await.expect("listener");
let base_url = format!("http://{}", listener.local_addr().expect("address"));
@@ -8062,6 +8064,7 @@ async fn responses_should_observe_and_bill_the_outbound_service_tier_on_both_tra
let mut observations = Vec::new();
let mut costs = Vec::new();
let mut raw_response = Vec::new();
let mut raw_messages = Vec::new();
while let Some(event) = stream.next().await {
let event = event.expect("provider event");
if let Some(observation) = event.response_observation() {
@@ -8075,6 +8078,12 @@ async fn responses_should_observe_and_bill_the_outbound_service_tier_on_both_tra
if let Some(frame) = event.wire_event().and_then(|wire| wire.raw_sse_frame()) {
raw_response.extend_from_slice(frame);
}
if let Some(message) = event
.wire_event()
.and_then(|wire| wire.raw_websocket_message())
{
raw_messages.push(message.to_owned());
}
}
let outbound = if let Some(server) = http_server {
server.verify().await;
@@ -8127,7 +8136,11 @@ async fn responses_should_observe_and_bill_the_outbound_service_tier_on_both_tra
expected_cost.into_iter().collect::<Vec<_>>(),
"WebSocket={use_websocket}, requested={requested:?}, reported={reported:?}"
);
assert_eq!(raw_response, response_frames.as_bytes());
if use_websocket {
assert_eq!(raw_messages, expected_messages);
} else {
assert_eq!(raw_response, response_frames.as_bytes());
}
}
}
}
@@ -9565,7 +9578,7 @@ async fn exact_websocket_busy_then_replay_scope_relaxation_is_rejected_before_se
}
#[tokio::test]
async fn continuation_prefetch_over_128_kib_should_commit_wire_without_protocol_failure() {
async fn continuation_prefetch_over_128_kib_should_wait_for_grace_without_protocol_failure() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_prefetch_limit").await;
let padding = "x".repeat(128 * 1024);
@@ -9583,7 +9596,7 @@ async fn continuation_prefetch_over_128_kib_should_commit_wire_without_protocol_
);
assert!(body.len() > 128 * 1024);
let (base_url, release, _first_chunk_sent, server) =
paused_chunked_sse_server(body, String::new()).await;
paused_chunked_sse_server(body.clone(), String::new()).await;
let mut stream = provider_with_base_url(&store, base_url)
.execute(
planned_request("openai", http_generate_operation()),
@@ -9592,25 +9605,52 @@ async fn continuation_prefetch_over_128_kib_should_commit_wire_without_protocol_
)
.await
.expect("prepare provider stream");
let visible = loop {
let event = timeout(Duration::from_secs(1), stream.next())
let visible = {
let next_visible = async {
loop {
let event = stream
.next()
.await
.expect("provider stream remains open")
.expect("large prefetch cannot create a protocol failure");
if event.has_client_event() {
break event;
}
}
};
tokio::pin!(next_visible);
assert!(
timeout(Duration::from_secs(1), next_visible.as_mut())
.await
.is_err(),
"large structural events must remain buffered during the grace period"
);
timeout(Duration::from_secs(3), next_visible)
.await
.expect("prefetch threshold must release buffered wire")
.expect("provider stream remains open")
.expect("threshold cannot create a protocol failure");
if event.has_client_event() {
break event;
}
.expect("grace expiry must release buffered wire")
};
assert_eq!(
visible.wire_event().and_then(|wire| wire.event_type()),
Some("response.created")
);
let mut wire = visible
.wire_event()
.unwrap()
.raw_sse_frame()
.unwrap()
.to_vec();
release.send(()).expect("finish upstream response");
while let Some(event) = stream.next().await {
event.expect("clean upstream EOF cannot become a protocol failure");
if let Some(frame) = event
.expect("clean upstream EOF cannot become a protocol failure")
.wire_event()
.and_then(|wire| wire.raw_sse_frame())
{
wire.extend_from_slice(frame);
}
}
assert_eq!(wire, body.as_bytes());
server.await.expect("chunked SSE server");
}
@@ -10710,17 +10750,16 @@ async fn assert_websocket_failure_headers(headers: Value, request_id: Option<&st
.iter()
.filter_map(|event| event.wire_event()?.event_type())
.collect::<Vec<_>>(),
vec![event_type],
"original failure stays atomically deliverable",
vec!["response.metadata", event_type],
"original metadata and failure stay atomically deliverable",
);
let wire = events
.iter()
.find_map(|event| event.wire_event()?.raw_sse_frame())
.filter_map(|event| event.wire_event())
.find(|wire| wire.event_type() == Some(event_type))
.and_then(|wire| wire.raw_websocket_message())
.expect("raw frame");
assert_eq!(
wire.as_ref(),
format!("event: {event_type}\ndata: {raw}\n\n").as_bytes()
);
assert_eq!(wire, raw);
}
#[tokio::test]
@@ -0,0 +1,207 @@
//! 透传请求参数、独立端点业务头和原始 WebSocket 响应的网络回归
use super::*;
#[tokio::test]
async fn effort_values_reach_upstream_before_and_after_middleware() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_provider_contract").await;
let server = MockServer::start().await;
Mock::given(method("POST"))
.and(path("/codex/responses"))
.respond_with(
ResponseTemplate::new(200)
.insert_header("content-type", "text/event-stream")
.set_body_string(CAPTURE_COMPLETED_SSE),
)
.mount(&server)
.await;
let provider = provider_with_base_url(&store, server.uri());
let efforts = [
json!(64),
json!(0),
json!("future-effort"),
json!({"future": [1, true]}),
json!(null),
json!("x".repeat(65)),
];
for rewritten in [false, true] {
for effort in &efforts {
let reasoning = json!({"effort":effort,"future_extension":true});
let initial_reasoning = if rewritten {
json!({"effort":"high"})
} else {
reasoning.clone()
};
let body = json!({"model":"gpt-5.4","input":"synthetic","reasoning":initial_reasoning});
let operation = Operation::Generate(GenerateRequest::from_protocol_payload(
ProtocolPayload::json_object("openai", body.as_object().unwrap().clone())
.unwrap()
.with_context(Map::from_iter([("use_websocket".to_owned(), json!(false))])),
));
let context = if rewritten {
context_with_middleware(
"req_effort",
Arc::new(RecordingMiddleware {
observed: Arc::default(),
replacement: ("reasoning".to_owned(), reasoning.clone()),
request_headers: Vec::new(),
}),
FastMode::Default,
)
} else {
context("req_effort", CancellationToken::new())
};
let mut stream = Arc::clone(&provider)
.execute(planned_request("openai", operation), context)
.await
.unwrap();
while let Some(event) = stream.next().await {
event.unwrap();
}
let requests = server.received_requests().await.unwrap();
assert_eq!(
captured_request_body(requests.last().unwrap())["reasoning"],
reasoning
);
}
}
assert_eq!(
server.received_requests().await.unwrap().len(),
efforts.len() * 2
);
}
#[tokio::test]
async fn independent_endpoints_send_opaque_headers_without_client_identity() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_provider_contract").await;
let server = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({"output":"ok"})))
.mount(&server)
.await;
let provider = provider_with_base_url(&store, server.uri());
let entries: Vec<_> = [
("x-future-business", "first"),
("x-future-business", "second"),
("authorization", "Bearer client-secret"),
("chatgpt-account-id", "client-account"),
("x-stainless-runtime", "client-runtime"),
("connection", "close"),
("x-codex-turn-state", "untrusted-state"),
("x-codex-turn-metadata", "untrusted-metadata"),
]
.into_iter()
.map(|(name, value)| json!([name, STANDARD.encode(value)]))
.collect();
for endpoint in [
"/codex/images/generations",
"/codex/images/edits",
"/codex/alpha/search",
] {
let raw = Bytes::from_static(
br#"{ "id":"test-search", "prompt":"synthetic", "future":1, "future":2 }"#,
);
let payload = RawJsonPayload::new("openai", raw.clone())
.unwrap()
.with_context(Map::from_iter([(
"opaque_request_headers".to_owned(),
json!(entries),
)]));
let operation = if endpoint.ends_with("search") {
Operation::Search(StandaloneSearchRequest::from_raw_json(payload))
} else {
Operation::GenerateImage(ImageRequest::from_raw_json(
if endpoint.ends_with("edits") {
ImageRequestKind::Edit
} else {
ImageRequestKind::Generation
},
payload,
))
};
let mut stream = Arc::clone(&provider)
.execute(
planned_provider_endpoint_request("openai", operation),
context("req_opaque_headers", CancellationToken::new()),
)
.await
.unwrap();
while let Some(event) = stream.next().await {
event.unwrap();
}
let requests = server.received_requests().await.unwrap();
let sent = requests.last().unwrap();
assert_eq!(sent.url.path(), endpoint);
assert_eq!(sent.body, raw);
assert_eq!(
captured_header_values(sent, "x-future-business"),
vec![b"first".to_vec(), b"second".to_vec()]
);
assert!(
!captured_header_values(sent, "authorization")
.contains(&b"Bearer client-secret".to_vec())
);
assert!(
!captured_header_values(sent, "chatgpt-account-id")
.contains(&b"client-account".to_vec())
);
assert!(captured_header_values(sent, "x-stainless-runtime").is_empty());
assert!(captured_header_values(sent, "x-codex-turn-state").is_empty());
assert!(captured_header_values(sent, "x-codex-turn-metadata").is_empty());
assert!(!captured_header_values(sent, "connection").contains(&b"close".to_vec()));
}
}
#[tokio::test]
async fn websocket_messages_preserve_text_and_metadata_despite_unrecognized_json() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_provider_contract").await;
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let provider =
provider_with_base_url(&store, format!("http://{}", listener.local_addr().unwrap()));
let messages = vec![
"{ \"type\" : \"future.event\", \"extension\" : { \"escaped\" : \"\\u0061\", \"number\" : 1e3 } }\r\n".to_owned(),
json!({"type":"response.metadata","headers":{"x-codex-turn-state":"synthetic-state"},"metadata":{"type":"safety_buffering","use_cases":["cyber"],"reasons":["user_risk"],"future":true}}).to_string(),
format!("{{\"type\":\"future.deep\",\"extension\":{}0{}}}", "[".repeat(140), "]".repeat(140)),
"future non-JSON text".to_owned(), "[DONE]".to_owned(),
r#"{"type":42,"future":true}"#.to_owned(),
];
let sent = messages.clone();
let server = tokio::spawn(async move {
let (socket, _) = listener.accept().await.unwrap();
let mut socket = accept_codex_test_websocket(socket).await;
socket.next().await.unwrap().unwrap();
socket.send(Message::Text(json!({"type":"response.created","response":{"id":"resp_raw","model":"gpt-5.4","status":"in_progress","output":[]}}).to_string().into())).await.unwrap();
for message in sent {
socket.send(Message::Text(message.into())).await.unwrap();
}
socket.send(Message::Text(json!({"type":"response.completed","response":{"id":"resp_raw","model":"gpt-5.4","status":"completed","output":[],"usage":{"input_tokens":1,"output_tokens":1,"total_tokens":2}}}).to_string().into())).await.unwrap();
});
let mut stream = provider
.execute(
planned_request("openai", generate_operation()),
context("req_raw_ws", CancellationToken::new()),
)
.await
.unwrap();
let mut delivered = Vec::new();
let mut completed = false;
while let Some(event) = stream.next().await {
let event = event.unwrap();
completed |= event
.canonical_facts()
.iter()
.any(|event| matches!(event, GatewayEvent::Completed(_)));
if let Some(raw) = event
.wire_event()
.and_then(|wire| wire.raw_websocket_message())
{
delivered.push(raw.to_owned());
}
}
server.await.unwrap();
assert!(completed);
assert_eq!(&delivered[1..delivered.len() - 1], messages);
}
@@ -79,14 +79,13 @@ async fn stream_failure(stream: &mut ProviderStream) -> (Vec<ProviderEvent>, Pro
}
#[tokio::test]
async fn large_structural_events_preserve_overload_replay_past_the_old_grace_on_http_and_websocket()
{
async fn large_structural_events_preserve_overload_replay_on_http_and_websocket() {
for websocket in [false, true] {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_provider_contract").await;
// #259 仅保留了事件长度;用合成配置回显复现尺寸,不依赖现场私有正文
let created = structural_event("response.created", 38_781);
let progress = structural_event("response.in_progress", 38_785);
// 大段配置回显不应提前结束重试窗口;使用合成正文覆盖单帧与累计的大体积前导事件
let created = structural_event("response.created", 300 * 1024);
let progress = structural_event("response.in_progress", 300 * 1024);
let failure = overload();
let expected = vec![created.clone(), progress.clone(), failure.clone()];
let (base_url, release, server) = if websocket {
@@ -160,8 +159,8 @@ async fn large_structural_events_preserve_overload_replay_past_the_old_grace_on_
async fn later_structural_events_do_not_extend_grace_and_late_overload_is_not_replayable() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_provider_contract").await;
let created = sse(&structural_event("response.created", 38_781));
let progress = sse(&structural_event("response.in_progress", 38_785));
let created = sse(&structural_event("response.created", 96 * 1024));
let progress = sse(&structural_event("response.in_progress", 96 * 1024));
let expected_bytes = created.len() + progress.len();
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let base_url = format!("http://{}", listener.local_addr().unwrap());
@@ -220,21 +219,20 @@ async fn immediate_release_preserves_wire_and_records_the_boundary_once() {
for (reason, body) in [
(
"semantic_output",
sse(&structural_event("response.created", 512)) + &sse(&semantic),
sse(&structural_event("response.created", 300 * 1024)) + &sse(&semantic),
),
(
"semantic_output",
sse(&structural_event("response.created", 512)) + &sse(&tool),
sse(&structural_event("response.created", 300 * 1024)) + &sse(&tool),
),
(
"terminal",
sse(&structural_event("response.created", 512)) + &sse(&terminal),
sse(&structural_event("response.created", 300 * 1024)) + &sse(&terminal),
),
(
"byte_limit",
sse(&structural_event("response.created", 129 * 1024)),
"eof",
sse(&structural_event("response.created", 300 * 1024)),
),
("eof", sse(&structural_event("response.created", 512))),
] {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_provider_contract").await;
@@ -274,11 +272,7 @@ async fn immediate_release_preserves_wire_and_records_the_boundary_once() {
let releases = releases(&trace);
assert_eq!(releases.len(), 1);
assert_eq!(releases[0]["data"]["reason"], reason);
if reason == "byte_limit" {
assert!(releases[0]["data"]["prefetchedBytes"].as_u64().unwrap() > 128 * 1024);
} else {
assert_eq!(releases[0]["data"]["prefetchedBytes"], body.len());
}
assert_eq!(releases[0]["data"]["prefetchedBytes"], body.len());
server.await.unwrap();
}
}
@@ -288,7 +282,7 @@ async fn cancelling_buffered_structural_events_does_not_release_or_offer_replay(
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_provider_contract").await;
let (base_url, release, _, server) = paused_chunked_sse_server(
sse(&structural_event("response.created", 38_781)),
sse(&structural_event("response.created", 300 * 1024)),
String::new(),
)
.await;
@@ -1,7 +1,7 @@
//! 验证响应中断使用当前连接并在复用前释放控制权
//! 验证未知控制帧和迟到中断使用原连接,并在复用时撤销旧执行控制权
use super::*;
use gateway_core::engine::response_control::{ResponseControl, ResponseInterruptError};
use gateway_core::engine::response_control::{ResponseControl, ResponseControlUnavailable};
async fn next_json(socket: &mut tokio_tungstenite::WebSocketStream<TcpStream>) -> Value {
loop {
@@ -46,11 +46,12 @@ fn interrupt_context(control: ResponseControl, previous: Option<&str>) -> Attemp
}
#[tokio::test]
async fn response_interrupt_uses_the_active_socket_and_releases_control_before_reuse() {
async fn response_controls_use_the_original_socket_through_terminal_and_revoke_on_reuse() {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_provider_contract").await;
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let base_url = format!("http://{}", listener.local_addr().unwrap());
let (finish, finished) = tokio::sync::oneshot::channel();
let server = tokio::spawn(async move {
let (socket, _) = listener.accept().await.unwrap();
let mut socket = accept_codex_test_websocket(socket).await;
@@ -72,6 +73,19 @@ async fn response_interrupt_uses_the_active_socket_and_releases_control_before_r
.await
.unwrap();
socket.send(Message::Text(json!({"type":"response.output_text.delta","output_index":0,"content_index":0,"delta":"partial"}).to_string().into())).await.unwrap();
let unknown = next_json(&mut socket).await;
assert_eq!(
unknown,
json!({"type":"future.control", "extension":{"mode":"future"}})
);
socket
.send(Message::Text(
json!({"type":"future.ack", "extension":true})
.to_string()
.into(),
))
.await
.unwrap();
let interrupt = next_json(&mut socket).await;
assert_eq!(
interrupt,
@@ -88,10 +102,25 @@ async fn response_interrupt_uses_the_active_socket_and_releases_control_before_r
))
.await
.unwrap();
let late = next_json(&mut socket).await;
assert_eq!(
late,
json!({"type":"response.interrupt", "response_id":id,"mode":"future_mode", "extra":1})
);
socket
.send(Message::Text(
json!({"type":"future.idle_ack", "original":late})
.to_string()
.into(),
))
.await
.unwrap();
}
let _ = finished.await;
});
let provider = provider_with_base_url(&store, base_url);
let mut session_state = None;
let mut old_control: Option<ResponseControl> = None;
for id in ["resp_interrupt_a", "resp_interrupt_b"] {
let control = ResponseControl::default();
let previous = (id == "resp_interrupt_b").then_some("resp_interrupt_a");
@@ -112,34 +141,62 @@ async fn response_interrupt_uses_the_active_socket_and_releases_control_before_r
)
.await
.unwrap();
let mut interrupted = false;
let mut unknown_reply = false;
let mut sent = false;
timeout(Duration::from_secs(5), async {
while let Some(event) = stream.next().await {
let event = event.unwrap();
if let Some(state) = event.session_update() {
session_state = Some(state.clone());
}
if !interrupted && control.interrupt(id).is_ok() {
assert_eq!(
control.interrupt("resp_foreign"),
Err(ResponseInterruptError::ResponseMismatch)
);
control.interrupt(id).unwrap();
interrupted = true;
if !sent {
if let Some(old) = old_control.take() {
assert_eq!(old.send("old control").await, Err(ResponseControlUnavailable));
}
control.send(&json!({"type":"future.control", "extension":{"mode":"future"}}).to_string()).await.unwrap();
control.send(&json!({"type":"response.interrupt", "response_id":id,"mode":"discard_partial_items"}).to_string()).await.unwrap();
sent = true;
}
unknown_reply |= event.wire_event().and_then(|wire| wire.event_type()) == Some("future.ack");
}
})
.await
.expect("interrupt completes the active response");
assert!(interrupted);
.expect("controls complete the active response");
assert!(unknown_reply, "unknown upstream events remain visible");
assert!(session_state.is_some());
let late =
json!({"type":"response.interrupt", "response_id":id,"mode":"future_mode", "extra":1});
control.send(&late.to_string()).await.unwrap();
let reply = timeout(Duration::from_secs(5), control.receive())
.await
.unwrap()
.unwrap();
assert_eq!(
control.interrupt(id),
Err(ResponseInterruptError::Unavailable)
serde_json::from_str::<Value>(&reply).unwrap(),
json!({"type":"future.idle_ack", "original":late})
);
// 取消空闲读取后,下一轮正文必须能继续消费同一个有界接收通道
assert!(
timeout(Duration::from_millis(10), control.receive())
.await
.is_err()
);
old_control = Some(control);
}
finish.send(()).unwrap();
timeout(Duration::from_secs(5), server)
.await
.unwrap()
.unwrap();
let control = old_control.unwrap();
assert_eq!(
timeout(Duration::from_secs(5), control.receive())
.await
.unwrap(),
Err(ResponseControlUnavailable)
);
assert_eq!(
control.send("closed connection").await,
Err(ResponseControlUnavailable)
);
}
@@ -55,11 +55,71 @@ async fn finish_timings(
.expect("complete upstream stream")
}
async fn observe_timing_events(websocket: bool, events: Vec<Value>) -> ProviderResponseTimings {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_provider_contract").await;
let (base_url, server) = if websocket {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let base_url = format!("http://{}", listener.local_addr().unwrap());
let server = tokio::spawn(async move {
let (socket, _) = listener.accept().await.unwrap();
let mut ws = crate::transport::accept_codex_test_websocket_with(
socket,
|request: &tokio_tungstenite::tungstenite::handshake::server::Request, response| {
assert_eq!(
request.headers()["x-responsesapi-include-timing-metrics"],
"true"
);
response.headers_mut().insert(
"sec-websocket-extensions",
"permessage-deflate".parse().unwrap(),
);
},
)
.await;
ws.next().await.unwrap().unwrap();
for event in events {
ws.send(Message::Text(event.to_string().into()))
.await
.unwrap();
}
});
(base_url, server)
} else {
let (base_url, release, _, server) = paused_chunked_sse_server(
events[..1]
.iter()
.map(|event| format!("data: {event}\n\n"))
.collect(),
events[1..]
.iter()
.map(|event| format!("data: {event}\n\n"))
.collect(),
)
.await;
release.send(()).unwrap();
(base_url, server)
};
let operation = if websocket {
generate_operation()
} else {
http_generate_operation()
};
let mut stream = provider_with_base_url(&store, base_url)
.execute(
planned_request("openai", operation),
timed_context(Instant::now() - Duration::from_secs(20), 2),
)
.await
.unwrap();
let timings = finish_timings(&mut stream, ProviderResponseTimings::default()).await;
server.await.unwrap();
timings
}
#[tokio::test]
async fn official_response_duration_is_observed_independently_of_the_request_clock() {
for websocket in [false, true] {
let store = Arc::new(MemoryAccountStore::default());
create_account(&store, "acct_provider_contract").await;
let created = json!({"type":"response.created","response":{"id":"resp_official_time","created_at":100}});
let completed = json!({"type":"response.completed","response":{"id":"resp_official_time","status":"completed","created_at":100,"completed_at":107,"output":[],"usage":{"input_tokens":1,"output_tokens":112,"total_tokens":113}}});
let metrics = json!({"type":"responsesapi.websocket_timing","timing_metrics":{
@@ -81,62 +141,7 @@ async fn official_response_duration_is_observed_independently_of_the_request_clo
}});
let stale = json!({"type":"responsesapi.websocket_timing","timing_metrics":{"engine_service_total_ms":999.0}});
let events = [stale, created, metrics, wrong_id, invalid, completed];
let (base_url, server) = if websocket {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let base_url = format!("http://{}", listener.local_addr().unwrap());
let server = tokio::spawn(async move {
let (socket, _) = listener.accept().await.unwrap();
let mut ws = crate::transport::accept_codex_test_websocket_with(
socket,
|request: &tokio_tungstenite::tungstenite::handshake::server::Request,
response| {
assert_eq!(
request.headers()["x-responsesapi-include-timing-metrics"],
"true"
);
response.headers_mut().insert(
"sec-websocket-extensions",
"permessage-deflate".parse().unwrap(),
);
},
)
.await;
ws.next().await.unwrap().unwrap();
for event in events {
ws.send(Message::Text(event.to_string().into()))
.await
.unwrap();
}
});
(base_url, server)
} else {
let (base_url, release, _, server) = paused_chunked_sse_server(
events[..3]
.iter()
.map(|event| format!("data: {event}\n\n"))
.collect(),
events[3..]
.iter()
.map(|event| format!("data: {event}\n\n"))
.collect(),
)
.await;
release.send(()).unwrap();
(base_url, server)
};
let operation = if websocket {
generate_operation()
} else {
http_generate_operation()
};
let mut stream = provider_with_base_url(&store, base_url)
.execute(
planned_request("openai", operation),
timed_context(Instant::now() - Duration::from_secs(20), 2),
)
.await
.unwrap();
let timings = finish_timings(&mut stream, ProviderResponseTimings::default()).await;
let timings = observe_timing_events(websocket, events.to_vec()).await;
assert_eq!(timings.upstream_response_ms, Some(7_000));
assert_eq!(timings.upstream_api_overhead_ms, Some(120.25));
assert_eq!(timings.upstream_engine_ms, Some(6400.0));
@@ -146,7 +151,133 @@ async fn official_response_duration_is_observed_independently_of_the_request_clo
assert_eq!(timings.upstream_engine_service_tbt_ms, Some(20.12));
assert_eq!(timings.first_token_ms, None);
assert!(timings.first_event_ms.is_some_and(|value| value >= 20_000));
server.await.unwrap();
}
}
// 保留实测计时帧的层级和小数,响应身份使用虚构值,不包含请求正文
fn response_critical_path_event() -> Value {
json!({"type":"responsesapi.websocket_timing","timing_metrics":{
"timing_scope":"logical_turn",
"response_id":"resp_critical_path",
"pre_inference_ms":211.691468,
"client_tool_pause_total_ms":3730.088369,
"total_turn_time_s":25.017284596,
"num_engine_calls":4,
"first_sampled_message_ttft_ms":null,
"critical_path":{
"scope":"response",
"coverage":"complete",
"boundary_type":"actionable_output_item_done",
"num_engine_calls":1,
"responses_harness_unblocked_ms":6792.094664,
"responses_pre_inference_ms":35.747883,
"engine_wall_ms":6710.68507,
"client_tool_pause_ms":0.0,
"responses_other_ms":45.66171099999974,
"engine_iapi_total_ms":null,
"taas_request_setup_total_ms":null,
"taas_provider_wait_total_ms":null,
"sampling_and_stream_total_ms":null,
"inference_residual_total_ms":null
}
}})
}
fn critical_path_response_events(metrics: Value) -> Vec<Value> {
vec![
json!({"type":"response.created","response":{"id":"resp_critical_path"}}),
metrics,
json!({"type":"response.completed","response":{"id":"resp_critical_path","status":"completed","created_at":100,"completed_at":107,"output":[],"usage":{"input_tokens":1,"output_tokens":1,"total_tokens":2}}}),
]
}
#[tokio::test]
async fn response_critical_path_is_observed_without_logical_turn_totals_on_both_transports() {
for websocket in [false, true] {
let mut metrics = response_critical_path_event();
// 同帧即使携带整轮专项值,也不能与响应级 critical path 混算
metrics["timing_metrics"]["engine_service_total_ms"] = json!(25_000);
metrics["timing_metrics"]["engine_iapi_ttft_total_ms"] = json!(500);
let mut events = critical_path_response_events(metrics);
let mut wrong_id = response_critical_path_event();
wrong_id["timing_metrics"]["response_id"] = json!("resp_other");
wrong_id["timing_metrics"]["critical_path"]["engine_wall_ms"] = json!(999);
events.insert(2, wrong_id);
let timings = observe_timing_events(websocket, events).await;
assert_eq!(timings.upstream_engine_ms, Some(6710.68507));
assert!((timings.upstream_api_overhead_ms.unwrap() - 81.409594).abs() < 0.000001);
assert_eq!(timings.upstream_response_ms, Some(7000));
assert_eq!(timings.first_token_ms, None);
assert_eq!(timings.upstream_engine_iapi_ttft_ms, None);
assert_eq!(timings.upstream_engine_service_ttft_ms, None);
assert_eq!(timings.upstream_engine_iapi_tbt_ms, None);
assert_eq!(timings.upstream_engine_service_tbt_ms, None);
}
}
#[tokio::test]
async fn critical_path_rejects_mismatched_identity_and_incomplete_or_cumulative_scope() {
for (pointer, value) in [
("/timing_metrics/response_id", json!("resp_other")),
("/timing_metrics/response_id", Value::Null),
("/timing_metrics/critical_path/scope", json!("logical_turn")),
("/timing_metrics/critical_path/scope", Value::Null),
("/timing_metrics/critical_path/coverage", json!("partial")),
("/timing_metrics/critical_path/coverage", Value::Null),
(
"/timing_metrics/critical_path/boundary_type",
json!("unknown"),
),
("/timing_metrics/critical_path", Value::Null),
] {
let mut metrics = response_critical_path_event();
// 正确的顶层 ID 不能掩盖嵌套 ID 的冲突
metrics["response_id"] = json!("resp_critical_path");
*metrics.pointer_mut(pointer).unwrap() = value;
let timings = observe_timing_events(false, critical_path_response_events(metrics)).await;
assert_eq!(timings.upstream_engine_ms, None, "{pointer}");
assert_eq!(timings.upstream_api_overhead_ms, None, "{pointer}");
}
for identity in ["response_id", "response"] {
let mut metrics = response_critical_path_event();
metrics[identity] = if identity == "response" {
json!({"id":"resp_other"})
} else {
json!("resp_other")
};
let timings = observe_timing_events(false, critical_path_response_events(metrics)).await;
assert_eq!(timings.upstream_engine_ms, None);
}
let metrics = json!({"type":"responsesapi.websocket_timing","timing_metrics":{
"timing_scope":"logical_turn",
"engine_service_total_ms":25_000,
"responses_duration_excl_engine_and_client_tool_time_ms":500,
"engine_iapi_ttft_total_ms":200
}});
let timings = observe_timing_events(false, critical_path_response_events(metrics)).await;
assert_eq!(timings.upstream_engine_ms, None);
assert_eq!(timings.upstream_api_overhead_ms, None);
assert_eq!(timings.upstream_engine_iapi_ttft_ms, None);
}
#[tokio::test]
async fn critical_path_keeps_missing_or_invalid_components_unknown_and_preserves_zero() {
for value in [Value::Null, json!(-1), json!("45"), json!(1e99)] {
let mut metrics = response_critical_path_event();
metrics["timing_metrics"]["critical_path"]["responses_other_ms"] = value;
let timings = observe_timing_events(false, critical_path_response_events(metrics)).await;
assert_eq!(timings.upstream_api_overhead_ms, None);
assert_eq!(timings.upstream_engine_ms, Some(6710.68507));
}
for (component, expected) in [(0.0, Some(0.0)), (5e18, None)] {
let mut metrics = response_critical_path_event();
let path = &mut metrics["timing_metrics"]["critical_path"];
path["responses_other_ms"] = json!(component);
path["responses_pre_inference_ms"] = json!(component);
path["engine_wall_ms"] = json!(0);
let timings = observe_timing_events(false, critical_path_response_events(metrics)).await;
assert_eq!(timings.upstream_api_overhead_ms, expected);
assert_eq!(timings.upstream_engine_ms, Some(0.0));
}
}
@@ -268,7 +268,16 @@ async fn collect_backend_response(
.unwrap_or(i64::MAX)
.max(1)
});
body_bytes.extend_from_slice(&chunk);
if transport == CodexBackendTransport::WebSocket {
let raw = std::str::from_utf8(&chunk).unwrap();
body_bytes.extend_from_slice(
provider_openai::transport::protocol::websocket::websocket_event_to_sse_frame(raw)
.unwrap()
.as_bytes(),
);
} else {
body_bytes.extend_from_slice(&chunk);
}
}
if let Some(updates) = rate_limit_updates {
for update in updates.lock().await.iter() {
@@ -156,10 +156,16 @@ fn websocket_event_to_sse_should_forward_public_events_and_strip_internal_events
format!("event: response.completed\ndata: {event}\n\n")
);
assert!(websocket_event_to_sse_frame(r#"{"type":"codex.rate_limits"}"#).is_none());
assert!(websocket_event_to_sse_frame(r#"{"type":"response.metadata"}"#).is_none());
assert!(websocket_event_to_sse_frame(r#"{"type":"response.metadata"}"#).is_some());
assert!(websocket_event_to_sse_frame(r#"{"type":"codex.response.metadata"}"#).is_some());
assert!(websocket_event_to_sse_frame(r#"{"response":{}}"#).is_none());
assert!(websocket_event_to_sse_frame("not-json").is_none());
assert_eq!(
websocket_event_to_sse_frame(r#"{"response":{}}"#).unwrap(),
"data: {\"response\":{}}\n\n"
);
assert_eq!(
websocket_event_to_sse_frame("not-json").unwrap(),
"data: not-json\n\n"
);
}
#[test]
@@ -972,7 +972,7 @@ async fn websocket_execute_response_create_request_should_capture_internal_metad
let response = execute_response_create_request(&prepared)
.await
.expect("internal events should update metadata without forwarding");
.expect("metadata should be forwarded while rate limits remain local");
server.await.unwrap();
assert_eq!(response.turn_state.as_deref(), Some("turn-from-metadata"));
@@ -981,7 +981,7 @@ async fn websocket_execute_response_create_request_should_capture_internal_metad
Some("gpt-internal-report")
);
assert!(!response.body.contains("codex.rate_limits"));
assert!(!response.body.contains("response.metadata"));
assert!(response.body.contains("response.metadata"));
assert!(response.body.contains("event: response.completed"));
assert!(
response
@@ -1043,7 +1043,7 @@ async fn websocket_codex_metadata_should_capture_first_turn_state_without_changi
assert_eq!(response.turn_state.as_deref(), Some("turn-first"));
assert!(response.body.contains("event: codex.response.metadata"));
assert!(!response.body.contains("event: response.metadata\n"));
assert!(response.body.contains("event: response.metadata\n"));
assert!(response.body.contains("event: response.completed"));
}
@@ -1929,7 +1929,8 @@ async fn codex_backend_client_stream_should_preserve_burst_during_downstream_bac
server.await.unwrap();
assert_eq!(
body.matches("event: response.output_text.delta").count(),
body.matches("\"type\":\"response.output_text.delta\"")
.count(),
BURST_FRAMES + 1
);
assert!(body.contains("initial;"));
@@ -153,7 +153,7 @@ impl ScheduledTask for XaiQuotaCatalogTask {
.map_err(|source| {
WorkerTaskError::safe("xAI Provider accounts unavailable").with_source(source)
})?;
let mut failures = 0_u64;
let mut failures = QuotaCatalogFailure::default();
let now = SystemTime::now();
let accounts = self.reserve_periodic_refreshes(accounts, now);
for account in accounts {
@@ -162,24 +162,51 @@ impl ScheduledTask for XaiQuotaCatalogTask {
}
match self.quota.refresh_account(account.id()).await {
Ok(_) | Err(GrokQuotaError::AccountUnavailable) => {}
Err(_) => failures = failures.saturating_add(1),
Err(error) => {
failures.quota_failures = failures.quota_failures.saturating_add(1);
failures.first_quota_error.get_or_insert(error);
}
}
}
match self.catalog.query_models().await {
Ok(_) | Err(GrokCredentialCatalogError::NoEligibleCredential) => {}
Err(_) => failures = failures.saturating_add(1),
Err(error) => failures.catalog_error = Some(error),
}
if failures == 0 {
if failures.quota_failures == 0 && failures.catalog_error.is_none() {
Ok(())
} else {
Err(WorkerTaskError::safe(
"xAI quota or catalog synchronization failed",
))
Err(
WorkerTaskError::safe("xAI quota or catalog synchronization failed")
.with_source(failures),
)
}
})
}
}
// 额度按账号批量执行,只保留首个失败及总数,目录失败独立保留,避免诊断随账号数增长
#[derive(Debug, Default)]
struct QuotaCatalogFailure {
quota_failures: u64,
first_quota_error: Option<GrokQuotaError>,
catalog_error: Option<GrokCredentialCatalogError>,
}
impl std::fmt::Display for QuotaCatalogFailure {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(formatter, "quota failures={}", self.quota_failures)?;
if let Some(error) = &self.first_quota_error {
write!(formatter, "; first quota error: {error}")?;
}
if let Some(error) = &self.catalog_error {
write!(formatter, "; catalog error: {error}")?;
}
Ok(())
}
}
impl std::error::Error for QuotaCatalogFailure {}
impl XaiQuotaCatalogTask {
fn reserve_periodic_refreshes(
&self,
@@ -214,6 +214,67 @@ async fn xai_quota_catalog_worker_preserves_store_failures() {
assert_eq!(error.as_safe_str(), "xAI Provider accounts unavailable");
}
#[tokio::test]
async fn xai_quota_catalog_worker_retains_both_failure_stages_and_finishes_the_batch() {
for refresh_due in [true, false] {
let store = MemoryProviderAccountStore::shared();
for suffix in ["broken_one", "broken_two"] {
let input = create_input(suffix, "synthetic-subject");
let mut account = crate::support::prepare_input(&input).unwrap();
// 模拟历史持久化凭据损坏,额度与目录读取均应保留各自的失败分类
account.credential = gateway_core::account::PlaintextCredential::new(Map::new());
let revision = account.account.revision();
store.create_account(account).await.unwrap();
if refresh_due {
store
.apply_quota_access(gateway_core::account::QuotaAccessChange {
account_id: input.account_id,
expected_revision: revision,
state: gateway_core::account::QuotaState::exhausted(
gateway_core::account::QuotaEvidence::ProviderDenied,
SystemTime::UNIX_EPOCH,
None,
),
})
.await
.unwrap();
}
}
let mut bundle = provider_xai::initialize(provider_ports_with(
store,
Arc::new(TestOAuthPending::default()),
))
.await
.unwrap();
let error = run_quota_catalog_cycle(&mut bundle)
.await
.expect_err("invalid stored credentials");
assert_eq!(
error.as_safe_str(),
"xAI quota or catalog synchronization failed"
);
let details = error.error_details().expect("synchronization causes");
let details: Value = serde_json::from_str(details.as_str()).unwrap();
let cause = details["causes"]["messages"][0].as_str().unwrap();
assert!(
cause.contains("catalog error: Grok provider account store is unavailable"),
"{cause}"
);
if refresh_due {
assert!(cause.contains("quota failures=2"), "{cause}");
assert!(
cause
.contains("first quota error: xAI quota credential or billing data is invalid"),
"{cause}"
);
} else {
assert!(cause.contains("quota failures=0"), "{cause}");
assert!(!cause.contains("first quota error"), "{cause}");
}
}
}
async fn run_quota_catalog_cycle(
bundle: &mut provider_xai::ProviderBundle,
) -> Result<(), WorkerTaskError> {
+2 -1
View File
@@ -70,7 +70,8 @@ ENV PATH="/app/bin:/usr/lib/postgresql/18/bin:${PATH}"
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
# 首次升级的历史请求迁移需要启动宽限期
HEALTHCHECK --interval=30s --timeout=5s --start-period=5m --retries=3 \
CMD curl -fsS http://127.0.0.1:8080/healthz || exit 1
CMD ["/app/bin/codex-proxy-rs"]
+27 -10
View File
@@ -9,7 +9,7 @@
| 安装与启动 | [手动安装](#手动安装) · [启动](#启动) · [公网访问](#公网访问) |
| 接入客户端 | [客户端配置](#客户端配置) · [时区与客户端环境信息](#时区与客户端环境信息) · [登录与生图排查](#登录与生图排查) |
| 日常运维 | [运行设置](#启动后的运行设置) · [小内存优化](#小内存优化) · [持久化与日志](#持久化与备份) · [请求排查](#请求错误排查) · [密码轮换](#密码语义) |
| 更新与恢复 | [镜像升级](#镜像升级与源码构建) · [在线更新](#管理端在线更新) · [备份与恢复](#备份与恢复) · [优雅关停](#优雅关停) |
| 更新与恢复 | [镜像升级](#镜像升级与源码构建) · [在线更新](#管理端在线更新) · [备份与恢复](#备份与恢复) · [停止与重启](#停止与重启) |
| 运行源码 | [开发与源码联调](../docs/development.md) |
## 配置归属
@@ -196,6 +196,16 @@ curl -i http://127.0.0.1:8080/healthz
`204 No Content` 表示应用、PostgreSQL、Redis 和后台任务的健康检查通过,不代表每个上游账号都可用
`503 Service Unavailable` 的失败组件与原因保存在运维错误记录中,组件为 `health`,关联标识以
`probe:` 或 `worker:` 开头;探针超时也会指明组件。连续相同故障去重,恢复后再次失败重新记录,
应用日志记录故障与恢复摘要。健康响应保持空正文,详情沿用有界观测队列,队列或存储故障时可能缺失
HTTP 在数据库迁移和初始化完成后才监听。迁移期间启动日志每 15 秒报告已等待时间,
包含执行迁移和等待迁移锁的时间,不代表完成比例;不要因暂时无法访问而反复重启。
镜像和 Compose 默认提供 5 分钟健康检查启动宽限期,探测成功后立即转为健康,无需等满宽限期。
历史数据较多时应提前延长 `healthcheck.start_period` 和部署工具的等待超时;只修改 Compose
文件不会改变已创建容器的检查配置,需要在维护窗口重新创建容器
不要把未脱敏的 `docker compose config` 或 `docker inspect` 输出上传到工单;它们会包含
PostgreSQL/Redis 启动密码。日常校验使用 `config --quiet`
@@ -281,7 +291,7 @@ SSE 注释保活用于防止传输链路空闲断开,不会重置 Codex 等待
管理端生成的 CC Switch 一键导入包含连接信息和当前 Key 的日/周额度查询,默认每 30 分钟刷新。
查询地址和凭据随导入生成,在 CC Switch 中修改 Provider 的地址或 Key 后,需重新导入以同步用量查询。
导入不包含下方模板的原生生图和 WebSocket 设置。需要原生生图时,使用下方配置直连 CPR,
导入不包含下方模板的远程压缩、原生生图和 WebSocket 设置。需要这些设置时,使用下方配置直连 CPR,
避免 CC Switch 切换或接管后重写 Provider 配置
Linux/macOS 默认目录为 `~/.codex/`,Windows 为 `%USERPROFILE%\.codex\`;
@@ -308,6 +318,10 @@ requires_openai_auth = false
# 填写代理密钥,真实账号由服务端管理。
experimental_bearer_token = "<client-api-key>"
[model_providers.OpenAI.capabilities]
# Codex 0.162.0+ 显式启用 V2 远程压缩,不依赖 Provider 名称识别
remote_compaction = "v2"
[model_providers.OpenAI.http_headers]
# 供客户端识别服务端托管认证,本身不是密钥。
X-OpenAI-Actor-Authorization = "proxy-managed"
@@ -321,6 +335,9 @@ goals = true
不要重复添加 `[features]` 或 Provider 表。更换模型时也要检查其支持的推理强度。
密钥以明文保存,文件仅供本人读取,不要提交到 Git。接入代理不需要扩大命令沙箱的联网或文件权限
V2 远程压缩通过 Responses 接口发送 `compaction_trigger`,网关和上游需支持该协议。
若 Provider ID 改为 `custom`,相应能力配置也要改到 `[model_providers.custom.capabilities]`
需要指定完整模型目录时,在账号的模型列表中导出所选 Codex 模型,并在 `config.toml` 顶层设置
`model_catalog_json = "/absolute/path/to/cpr-model-catalog.json"`。导出文件不含账号凭据;
它是一次目录快照,调整选择或上游模型能力变化后需重新导出
@@ -388,16 +405,14 @@ metrics_exporter = "none"
其他客户端使用 Responses API、`/v1` Base URL 和代理密钥即可,不需要 Actor 标记。
路由与请求格式见 [API 参考](../docs/api.md#3-openai-数据面与模型目录)
## 优雅关停
## 停止与重启
收到停止信号后,应用先停止接收新连接并 drain 存量连接;整个 drain 共享一个从停止信号
起算的绝对截止点(`host.drain_timeout_seconds`,默认 30 秒),逾期放弃等待,存量连接随
进程退出终止。drain 结束后才关停后台 worker,预算为
`host.worker_shutdown_timeout_seconds`(默认 30 秒),两段预算按最坏情况串联
管理端重启、SIGTERM 和 SIGINT 使用同一关闭策略:停止接收新连接,不等待现有 HTTP、SSE、
WebSocket 会话完成,进程退出时强制断开剩余连接。已入队的观测写入与后台清理保留
`host.worker_shutdown_timeout_seconds`(默认 30 秒)的收尾预算;客户端在新进程就绪后重新连接
Compose 的 `stop_grace_period` 为 75 秒,覆盖默认 30 秒 HTTP drain、30 秒 worker 收尾和额外调度
余量。若调大任一应用超时,也必须把 `stop_grace_period` 调到大于两段超时之和;否则 Docker 会在
宽限期结束时 SIGKILL
Compose 的 `stop_grace_period` 为 75 秒,覆盖默认 30 秒后台收尾及进程清理余量。
调大后台关闭预算时,也必须同步预留容器停止宽限期,否则 Docker 会在宽限期结束时 SIGKILL
## 本地开发
@@ -622,6 +637,8 @@ Compose 提供以下在线更新运行参数:
- `CPR_ENABLE_SELF_RESTART=true`:更新或回滚完成后允许管理端请求重启;Docker 进程退出后由
Compose 的 `restart: unless-stopped` 拉起新进程
管理端自重启遵循统一的[停止与重启](#停止与重启)策略,客户端需在新进程完成迁移与初始化后重新连接
Release 必须提供当前 OS/架构的 `codex-proxy-rs_<version>_<os>_<arch>.tar.gz` 与
`checksums.txt`。服务会在替换前再次查询远端最新版本,校验下载 host、声明大小、SHA-256 和
归档路径;二进制或静态资源任一替换失败时恢复旧文件。成功后的旧二进制和旧静态目录分别保留为
+3 -3
View File
@@ -80,8 +80,7 @@ services:
restart: unless-stopped
user: '10001:10001'
init: true
# 关闭是串行两段:HTTP drain(host.drain_timeout_seconds=30s)之后 worker join
# (host.worker_shutdown_timeout_seconds=30s),宽限期必须大于两者之和。
# 不等待存量会话;宽限期需覆盖后台收尾(默认 30s)及进程清理余量
stop_grace_period: 75s
depends_on:
postgres:
@@ -116,7 +115,8 @@ services:
interval: 30s
timeout: 5s
retries: 3
start_period: 10s
# 首次升级需回填历史请求,宽限期只容忍启动失败,探测成功后立即转为健康
start_period: 5m
environment:
HOME: /app
CPR_SERVER_HOST: 0.0.0.0
-1
View File
@@ -37,7 +37,6 @@ host:
# deployment_mode: 'binary'
# self_restart_enabled: true
# executable_path: '/opt/codex-proxy-rs/codex-proxy-rs' # 可选:覆盖启动时解析的程序路径。
drain_timeout_seconds: 30
worker_shutdown_timeout_seconds: 30
store:
+33 -12
View File
@@ -309,13 +309,16 @@ turn metadata 的 `workspaces` 也会保留仓库绝对路径、Git 远端地址
已升级的 Responses 连接在入站解析前和出站写入前经过 `websocket` 中间件,完整消息与主动发送接口见
[SDK 洋葱中间件](../backend/crates/gateway-plugin/sdk/docs/capabilities.md#洋葱中间件)。下述规则描述默认协议处理
Responses WebSocket 接受文本 `response.create` 和 `response.interrupt`,创建请求在同一连接串行执行。当前响应期间收到的后续业务帧
Responses WebSocket 接受文本消息,`response.create` 在同一连接串行执行。当前响应期间收到的后续创建请求
留在有界接收队列中,待当前响应完成终结和写出后再逐条校验、准入与执行,不因请求提前到达而断开。
接收队列容量为 32 个事件,超载仍关闭连接;Ping/Pong、客户端关闭和服务关闭不等待队列中的请求执行。
活动响应期间会即时处理 `{"type":"response.interrupt","response_id":"当前响应 ID","mode":"discard_partial_items"}`。
中断只能发送到该执行占用的原上游 WS,不重新选号或创建推理 attempt;重复中断合并为一次发送。
ID 不匹配、没有可中断响应、实际走 HTTP 或 Provider 不支持控制时返回 `400` 协议错误,原执行继续;
客户端需要终止这类执行时可关闭连接,后续按既有续接合同恢复。
接收队列容量为 32 个事件,超载仍关闭连接;Ping/Pong、客户端关闭和服务关闭不等待队列中的请求执行
其余文本消息原样发送到当前执行绑定的原上游 WebSocket,包括
`response.interrupt` 和未知类型;类型、响应 ID、模式及扩展字段由上游校验。
控制帧不重新选号、不创建推理 attempt,也不合并重复消息。响应结束后原连接仍可接收控制帧并返回上游事件;
下一轮执行使用新绑定的控制入口。尚未建立原连接、原连接已失效、实际走 HTTP 或 Provider 未提供控制通道时返回 `400`,
不能通过控制帧新建上游连接
中断后继续转发上游事件与终态;只有 `response.incomplete` 的 `incomplete_details.reason` 为
`interrupted` 时,才按官方中断语义保留原连接续接能力。部分输出是否被丢弃以上游终态为准。
控制帧发送成功不等于上游已确认中断;若上游仍返回 `response.completed`,按正常完成处理
@@ -398,15 +401,21 @@ Codex 专用目录中的 `context_window` 与 `max_context_window` 分别表示
#### OpenAI 透传
OpenAI 路径保留客户端 Responses wire 语义:请求 body 的未知字段和字段顺序保持不变(受控模型
映射除外),HTTP SSE 与 WebSocket 的上游业务事件除下述客户端错误兼容外按原始字节转发,
映射除外),HTTP SSE 与 WebSocket 的上游业务事件除下述客户端错误兼容及响应头隔离外按原始字节转发,
response ID 按 opaque 值处理而不假设 UUID 或固定长度;除客户端错误兼容与原生续写额度恢复外,
OpenAI 上游错误 envelope 和允许下发的 opaque header 值也不由 canonical 观测结果重写。
`reasoning.effort` 的数字和未知扩展值保留给上游判断,本地观测不限制参数形态。
`response.metadata` 的业务内容继续交付,仅其中的响应头沿用凭据、账号身份和逐跳头隔离规则。
未改写的 WebSocket 文本不依赖 JSON 旁路解析成功;插件改写后的响应仍复核必要关联,原始 SSE 心跳
和无法提取事实的未改写内容不构成协议失败。
Images 请求不读取或重建 JSON,也不要求或映射模型字段;
它固定使用 OpenAI Provider,
只在原始字节之外完成账号选择、鉴权头替换和端点路由,成功与非容量失败响应正文保持原始字节。
`/v1/alpha/search` 使用相同的 OpenAI Provider 原生端点边界:body(包括 `model`)不解析、不映射,
`x-codex-turn-metadata` 在移除客户端账号身份并按当前 lease 重写 installation ID 后转发;上游账号
Authorization、Cookie、account ID、originator 和 User-Agent 均由代理安全重建。
Images 与 Search 保留通过现有过滤规则的普通业务请求头及多值字节;Responses 原连接的 turn state
不随独立端点转发,turn metadata 仍由所属端点按当前账号处理。
模型映射是全局精确映射,未命中时模型名原样交给候选 Provider;分组只限定账号集合,不参与模型改名
#### xAI 适配
@@ -1806,10 +1815,15 @@ OpenAI 从当前响应的 `responsesapi.websocket_timing.timing_metrics` 读取
WebSocket 握手默认请求专项计时,客户端显式提供的开关保留原值。
各字段独立可选;响应开始前、终态后的计时和明确属于其他响应的事件不写入当前请求,重试时清空
`critical_path` 仅采集 `scope=response`、`coverage=complete` 且边界为
`actionable_output_item_done` 的响应级指标:引擎耗时取 `engine_wall_ms`,API 开销为
`responses_pre_inference_ms` 与 `responses_other_ms` 之和,缺失分项不按零补齐。
外层 `logical_turn` 累计时间不进入请求指标,也不用于补齐引擎 TTFT 或 Token 间隔
| 字段 | 上游口径 |
| --- | --- |
| `upstreamApiOverheadMs` | API 排除引擎与客户端工具时间后的耗时 |
| `upstreamEngineMs` | Engine Service 总耗时 |
| `upstreamEngineMs` | 响应级引擎耗时,取完整 critical path 的墙钟时间或直接返回的 Engine Service 时间 |
| `upstreamEngineIapiTtftMs` | Engine IAPI TTFT 总计 |
| `upstreamEngineServiceTtftMs` | Engine Service TTFT 总计 |
| `upstreamEngineIapiTbtMs` | 跨引擎调用的 IAPI Token 间隔 |
@@ -1837,10 +1851,17 @@ WebSocket 握手默认请求专项计时,客户端显式提供的开关保留
由 Core 错误类型与 Provider 静态诊断生成;`diagnostic.message` 保存安全摘要。展示与导出共用这些字段,
`sendState` 为 `not_sent / sent / ambiguous`,摘要被截断时带有 `truncated` 标记
管理端下载的诊断包 `schemaVersion: 3` 用于人工反馈,不是备份或导入格式。它包含关联 ID、错误分类摘要、
请求与错误事件各自的状态、attempt、时间线阶段、计时和上述失败分类;不自动导出 message/raw error、任意 metadata、
其他 trace event data、请求响应正文和头部。`availability` 与 `omitted` 明示未采集、不完整或主动省略的内容,
`null` 不代表没有发生错误。版本、环境及原始错误片段仍需操作者另行补充并审阅脱敏
管理端下载的诊断包 `schemaVersion: 4` 用于人工反馈,不是备份或导入格式。它包含关联 ID、错误分类摘要、
请求与错误事件各自的状态、attempt、时间线阶段、计时和上述失败分类。
`provider.precommit.released` 事件的 `precommitRelease` 保留 `reason / prefetchedBytes / waitMs`,
分别表示 Provider 释放缓存的原因、累计预取字节数和等待毫秒数,不等同于下游已提交。
当前释放原因为 `semantic_output / terminal / grace_timeout / eof`,导出也接受历史记录中的 `byte_limit`。
累计字节数只用于诊断,不触发缓存释放;数值只允许非负安全整数,
缺失或未通过校验的字段为 `null`;其他事件的 `precommitRelease` 为 `null`
诊断包不自动导出 message/raw error、任意 metadata、其他 trace event data、请求响应正文和头部。
`availability` 与 `omitted` 明示未采集、不完整或主动省略的内容,`null` 不代表没有发生错误。
版本、环境及原始错误片段仍需操作者另行补充并审阅脱敏
错误记录中的“已自动恢复”表示系统关联到了后续成功请求,不会把原来的失败记录改为成功。
`upstreamSendState = ambiguous` 表示无法确认该次上游执行结果,不代表后续恢复请求失败;
+11 -6
View File
@@ -404,10 +404,12 @@ OpenAI 模型目录用于发现,不因目录缺项拒绝请求;管理员配
- OpenAI 是透明边界。Responses 请求保留未知字段和字段顺序;SSE、WebSocket、Images 与 standalone
Search 的业务正文按原始字节转发,原生续写额度恢复遵循下述 continuation 例外。
canonical facts 从同一数据旁路提取,用于路由、恢复判断、观测和计费。
WebSocket transport 按完整文本消息交付,wire 同时保留原文与可选解析视图;同协议出口使用原文,
SSE 转换只发生在需要该表达的出口。插件未改写的封套继续携带原始文本,观测失败不否定交付。
非流式 Responses 由 API 聚合 wire:终态省略或清空 `output` 时,使用同一响应的 `output_item.done`
按 `output_index` 还原完整输出;已有非空终态输出不改写。完成项缺失或冲突时在下游提交前拒绝,
不凭 canonical 增量补造内容,也不让 SSE/WS 转发额外保存整份输出
- Responses 的业务扩展头保留原始多值字节。API 负责剥离鉴权、账号身份和 HTTP 传输字段,
- Responses、Images 与 standalone Search 的业务扩展头保留原始多值字节。API 负责剥离鉴权、账号身份和 HTTP 传输字段,
并提取会话语义;`gateway-protocol` 共享 HTTP 传输与网关链路字段分类。客户端兼容规则集中在
`providers/openai/src/transport/downstream/`:`headers.rs` 管理下游环境头和已提取语义的头部别名,
`body.rs` 管理已知顶层参数的过滤、缺省值补齐和已确认不兼容的 `input` 形状适配;兼容基准为 Codex Core/Desktop 请求协议,
@@ -708,9 +710,10 @@ Admin 的手动重置复用同一账本与 Key 行锁,在一个事务中清零
- SSE 持有并发名额直到终态;WebSocket 每个 `response.create` 独立准入,空闲连接不占名额。
Core 在每次请求开始时重新鉴权并冻结当前策略,既有连接也应用已发布的限额与授权范围变更。
换号和内部重试复用同一名额,Key 与账号并发上限同时生效;预算拒绝或启动失败释放名额
- 响应中断入口由 Core 的根执行持有,API 只控制当前连接正在交付的执行。Provider 在收到上游
`response.created` 后注册活动响应 owner,在终态、失败或断连时释放;控制句柄本身不延长 owner
生命周期。中断信号交给占用原账号、原 socket 的 exchange,不经过全局响应查询、新选号或独立 attempt。
- Core 的根执行携带不解释协议的连接控制端口,Provider 在请求写入上游后绑定原 socket。
API 在响应期间发送控制帧,终态后还可读取原连接事件;空闲读取必须在下一轮执行前取消,
与 Provider 正文消费共用单一有界接收通道。连接 owner 持有控制目标,复用时替换,销毁时释放;
控制句柄使用弱引用,不延长连接生命周期,也不经过全局响应查询、新选号或独立 attempt。
API 暂存的下一轮请求和连接泵接收队列共用容量名额,控制帧处理不取消中间件正在读取响应体的 future
完成、失败、取消和断连使用同一结算路径,在结束时以网关请求 ID 幂等累计已取得费用。
@@ -935,8 +938,10 @@ worker 复用连接测试探针执行真实上游调用,并按冻结代次处
客户端下载解析由 Host 的 `ClientDistributionResolver` 实现,组合根注入 Admin;只在管理员请求时访问外部来源,
不阻塞启动。失败时使用官方稳定地址,临时直链不持久化或代理下载,字段与来源规则见 [客户端下载 API](api.md#windows-客户端下载)
关闭分为两段:先停止接收新连接并 drain HTTP/WS,再取消并等待 Worker;两段各有独立预算,Compose 的
`stop_grace_period` 必须覆盖二者之和。超时后只丢弃仍未落盘的可恢复观测,不执行隐式业务重放
Host 对管理端重启、OS 关闭信号和内部取消使用统一策略:停止监听并拒绝新连接注册,
不等待存量会话,随后在独立预算内取消并等待 Worker 完成已入队观测写入和后台清理。
剩余连接随进程退出终止,超时后只丢弃仍未落盘的可恢复观测,不执行隐式业务重放。
Compose 的 `stop_grace_period` 必须覆盖后台关闭预算并留出进程清理余量
### 凭据、日志与诊断
+4
View File
@@ -27,6 +27,10 @@ requires_openai_auth = false
# 代理密钥仅用于网关鉴权,真实账号登录状态由服务端管理
experimental_bearer_token = ${JSON.stringify(input.apiKey)}
[model_providers.OpenAI.capabilities]
# Codex 0.162.0+ 显式启用 V2 远程压缩,不依赖 Provider 名称识别
remote_compaction = "v2"
[model_providers.OpenAI.http_headers]
# 声明服务端托管认证,让官方客户端启用原生生图,该标记不是密钥
X-OpenAI-Actor-Authorization = "proxy-managed"
@@ -1,6 +1,12 @@
import type { OpsError, UsageRecordDetail } from '@/api'
import { failureClassText } from './opsErrorPresentation'
const precommitReleaseReasons = new Set(['semantic_output', 'terminal', 'byte_limit', 'grace_timeout', 'eof'])
function nonNegativeInteger(value: unknown) {
return typeof value === 'number' && Number.isSafeInteger(value) && value >= 0 ? value : null
}
export function requestDiagnosticsBundle(
requestId: string,
detail: UsageRecordDetail | null,
@@ -14,7 +20,7 @@ export function requestDiagnosticsBundle(
const request = record ?? error
return {
schemaVersion: 3,
schemaVersion: 4,
source: 'usage.request_diagnostics',
exportPolicy: 'allowlisted_fields_without_payloads',
request: {
@@ -53,7 +59,7 @@ export function requestDiagnosticsBundle(
createdAt: error.createdAt,
}
: null,
// 只导出 Core 从错误类型生成的分类,正文与其他事件 data 不进入诊断包
// 只导出失败分类与缓存释放事实的白名单,正文与任意事件 data 不进入诊断包
trace: trace
? {
schemaVersion: trace.schemaVersion,
@@ -78,6 +84,15 @@ export function requestDiagnosticsBundle(
upstreamStatus: event.data.upstreamStatus,
}
: null,
precommitRelease: event.stage === 'provider.precommit.released'
? {
reason: typeof event.data.reason === 'string' && precommitReleaseReasons.has(event.data.reason)
? event.data.reason
: null,
prefetchedBytes: nonNegativeInteger(event.data.prefetchedBytes),
waitMs: nonNegativeInteger(event.data.waitMs),
}
: null,
})),
}
: null,
@@ -111,8 +126,8 @@ export function requestDiagnosticsBundle(
? 'unknown'
: !trace
? 'not_recorded'
: trace.events.some(event => event.stage === 'attempt.failed')
? 'stages_and_failure_classifications'
: trace.events.some(event => event.stage === 'attempt.failed' || event.stage === 'provider.precommit.released')
? 'stages_and_allowlisted_facts'
: 'stages_only',
traceEventsDropped: trace?.droppedEvents ?? null,
attemptsComplete: record?.attemptsComplete ?? null,
@@ -126,16 +141,16 @@ export function requestDiagnosticsBundle(
'attempts.providerErrorCode',
'request.message',
'request.metadata',
'trace.events.data_except_failure_classification',
'trace.events.data_except_failure_classification_and_precommit_release',
'request_and_response_headers_and_bodies',
'user_supplied_model_names',
'account_names_emails_and_credential_names',
'client_ip_user_agent_and_api_key',
],
notes: [
'schemaVersion 3 仅供人工排障,null 表示未知或未采集,不代表没有发生错误',
'schemaVersion 4 仅供人工排障,null 表示未知、未采集或未通过导出校验,不代表没有发生错误',
'错误摘要只使用稳定分类,不含上游错误码或原文,error 为 null 时请结合 attempts 的分类与状态',
'时间线导出阶段、顺序、计时及 Core 记录的失败分类,不含错误原文和其他事件 data',
'时间线导出阶段、顺序、计时、Core 记录的失败分类和 Provider 缓存释放的原因、预取字节数及等待毫秒数,不含错误原文和其他事件 data',
'attemptsComplete 不为 true 时,尝试列表不完整,没有时间线的旧记录无法补回未采集事件',
'未自动采集网关版本、客户端版本、部署环境和故障发生时区,请另行补充',
'关联 ID 仍可能属于内部信息,分享前请审阅,原始错误、正文和日志需另行审阅脱敏,勿直接公开',
+11 -17
View File
@@ -1,23 +1,17 @@
# v3.22.0
## 用量与性能观测
- 用量记录统一保存请求、调度、传输与恢复观测,保留历史记录的诊断信息
- 延迟明细区分上游性能、请求观测与传输观测,展示上游返回的响应耗时、引擎耗时、TTFT 和 Token 间隔等指标
- 输出速率改用同一完成响应的上游时间戳计算,避免把网关等待与传输时间计入生成耗时;缺少有效上游时间戳时显示「—」
- 首字等待按首个输出项开始事件计时,不再把其他结构事件计为首字
# v3.22.2
## 修复与改进
- 账号与 Key 表单的分组选择使用轻量查询,避免加载选择项时重复计算分组聚合统计
- Codex 配置导出统一使用 `config.toml` 中的 Provider Bearer 鉴权;CC Switch 导入对齐 4.0.4 支持的连接信息与用量查询字段
- 改进 Codex 0.162 协议兼容:Grok 转换采用最新的增量工具声明,修正同名工具及命名空间成员更新;流内错误的 `Retry-After` 解析与官方头部处理语义保持一致
- 配置发布复用未变化的插件实例,减少进程重建;完善插件状态迁移失败后的恢复与发布流程,保留进行中请求使用的实例
- 修复 PostgreSQL 事务开始被取消后,连接复用可能残留未结束事务的问题
- 移除 OpenAI Responses 提交前缓存的 128 KiB 字节阈值,避免大段前导事件提前结束透明重试窗口;HTTP/SSE 与 WebSocket 均保留 2.5 秒宽限期及语义输出、终态等放行规则
- 完善 WebSocket 控制消息透传:中断及其他非创建文本消息原样发送到当前绑定的上游连接,响应结束后仍可转发控制消息与上游事件,不额外创建推理请求
- 完善 OpenAI 扩展字段和原文透传,保留推理强度的数字及扩展值、WebSocket 原始文本、业务元数据,以及 Images/Search 的业务请求头;避免未改写的心跳或未知内容因本地解析失败而被截断
- 诊断导出升级为 schemaVersion 4,补齐提交前缓存的释放原因、累计字节数与等待时间,便于区分实际输出和宽限期到期等提交原因
- 健康探测保留具体失败组件、原因及超时信息,持续相同故障避免重复记录;xAI 后台任务保留额度与模型目录同步的失败原因
- Codex 配置模板显式启用 V2 远程压缩,并补充 CC Switch 一键导入的配置边界说明
## 升级事项
- 从 v3.21.3 升级会自动执行数据库迁移 `0025_request_observation.sql`,回填现有请求观测并调整存储结构;升级前备份数据库,停止共用该数据库的旧版本实例,为历史请求回填预留时间
- 迁移完成后不能仅切回旧二进制或镜像回滚;如需回退,应同时恢复升级前的数据库备份
- 历史记录缺少新增的上游性能数据时,相应指标保持空缺,输出速率显示「—」
- 手动配置 Codex 时使用完整的 `config.toml` 模板,无需另外生成 `auth.json`;CC Switch 4.0.4 会重建配置,导入连接后如需原生生图等模板设置,请核对并补齐对应配置
- 从 v3.22.1 升级没有新增数据库迁移;从更早版本升级仍会执行相应历史迁移
- 使用 Codex 0.162.0 及以上版本时,可重新复制并合并配置模板以启用 V2 远程压缩;CC Switch 一键导入不包含该能力配置
- 本版为插件 `middleware v3`、`upstream_adapter v1` 的最后一个兼容正式版;插件作者应迁移至 `middleware v4`、`upstream_adapter v2`,以 `fast_mode` 三态替代 `disable_fast`
- 移除字节阈值不会放宽已经交付输出后的重放限制,上游错误仍需满足安全重试条件
+1 -1
View File
@@ -1 +1 @@
version: "3.22.0"
version: "3.22.2"