Compare commits

...
22 Commits
Author SHA1 Message Date
ZyyandCodex 705f0fad06 chore(release): v3.22.2
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 18:11:24 +08:00
ZyyandCodex f0c60f501b fix(client): enable remote compaction in Codex config
Declare the V2 remote compaction capability in generated Codex configuration and deployment examples. Document that CC Switch imports do not preserve the full capability configuration.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 18:05:07 +08:00
ZyyandCodex d1febdd249 fix(openai): remove precommit byte limit
Keep large structural events buffered until semantic output, terminal completion, EOF, or the replay grace deadline. Retain byte counts for diagnostics and preserve historical byte_limit exports.

Refs #369

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 18:05:01 +08:00
ZyyandCodex 53d4086cc0 fix(openai): preserve opaque request fields and websocket messages
Forward numeric and extended reasoning effort values and preserve standalone endpoint business headers.

Retain business metadata and original WebSocket text through provider, middleware, and client delivery. Validate rewritten plugin output without rejecting unchanged heartbeats or opaque messages.

Add HTTP, WebSocket, and plugin process regressions for the passthrough boundaries.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 17:23:40 +08:00
ZyyandCodex ce03e8119d fix(diagnostics): retain health and xAI worker failure causes
Record health component failures through operational diagnostics with transition deduplication. Preserve quota and catalog failure context without interrupting the account batch.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 17:10:29 +08:00
ZyyandCodex 3f05b6450c fix(usage): preserve precommit facts in diagnostics export
Export allowlisted release reasons, prefetch bytes and wait times, and document diagnostics schema v4.

Refs #369

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 16:55:12 +08:00
ZyyandCodex a4b73099b5 fix(openai): pass through websocket control frames
Forward non-create text frames unchanged over the original upstream WebSocket while preserving response.create admission and serialization. Keep the control transport available after completion and relay idle upstream events without creating another inference attempt.

Cover unknown frames, late interrupts, terminal settlement, binding revocation, and connection reuse.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 16:26:31 +08:00
ZyyandCodex d875a62652 chore(release): v3.22.1
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 14:10:51 +08:00
ZyyandCodex 98704dfde8 fix(usage): parse response critical path timings
Read response-scoped engine and API overhead metrics while excluding logical-turn totals and incomplete observations. Validate nested response IDs and cover the observed payload over SSE and WebSocket.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 13:51:40 +08:00
ZyyandCodex b07bca312a fix(upgrade): handle long migrations and unify forced shutdown
Extend the startup healthcheck grace period to five minutes and report elapsed migration time every fifteen seconds without changing published migrations.

Use one shutdown path for OS signals, internal cancellation, and managed restarts. Stop listening without waiting for active sessions, preserve bounded worker cleanup, and remove the obsolete connection drain timeout.

Cover shutdown entry points and worker cleanup with real HTTP lifecycle tests and document the deployment behavior.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 13:51:14 +08:00
ZyyandCodex 60966d935e chore(release): v3.22.0
Co-authored-by: Codex <noreply@openai.com>
2026-10-09 13:03:32 +08:00
ZyyandCodex 52981f40f9 fix(providers): align Codex 0.162 protocol handling
Apply latest incremental tool declarations during Grok normalization and align nested Retry-After header parsing with official Codex semantics.

Cover developer instructions in input, Responses Lite tool declarations, partial_answer phases, and end_turn preservation.

Pin the upstream SQLx fix for cancelled BEGIN rollback and add a deterministic pooled-connection regression test.

Report safe Redis error categories and timeout status when the cooldown persistence regression fails.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 12:10:02 +08:00
ZyyandCodex a51c4cd640 refactor(plugins): simplify lifecycle and reuse unchanged instances
Share unchanged prepared instances across extension sets and close each
process when its last owner releases it. Keep inflight request plans intact.

Unify migration recovery and publication, centralize authentication binding
validation, avoid duplicate package inspection, and separate middleware plans.

Validated with 292 admin, 182 runtime, 37 plugin storage and 15 architecture
tests, rustfmt, and strict workspace Clippy.

Co-authored-by: Codex <noreply@openai.com>
2026-10-09 00:12:29 +08:00
ZyyandCodex bda6d973a8 docs: streamline documentation around current behavior
Remove version migration narratives, consolidate repeated explanations, and organize settings and architecture references.

Co-authored-by: Codex <noreply@openai.com>
2026-10-08 22:26:12 +08:00
ZyyandCodex 66a7e4a695 fix(client): align Codex config export with CC Switch 4.0.4
Export config.toml with provider bearer authentication and send supported connection and usage fields to CC Switch. Remove the separate auth.json output and size the configuration preview to its content.

Co-authored-by: Codex <noreply@openai.com>
2026-10-08 22:09:02 +08:00
think-backandCodex 9483f2c91b perf(groups): avoid aggregate queries when loading group selectors
Load account and API key group selectors through a lightweight admin options
endpoint while preserving pagination, filters, disabled groups and revision
metadata. Keep aggregate statistics on the group management endpoint.

Co-authored-by: Codex <noreply@openai.com>
2026-10-08 18:20:24 +08:00
ZyyandCodex faf50c28ad feat(usage): unify request observations and upstream performance
Consolidate request diagnostics into typed request_observation_json with
migration 0025, preserving historical facts and query constraints.

Use upstream response timestamps for output throughput, align TTFT event
boundaries, and surface official timing metrics in the latency popover.

Co-authored-by: Codex <noreply@openai.com>
2026-10-08 18:17:09 +08:00
ZyyandCodex cb397ae53e chore(release): v3.21.3
Co-authored-by: Codex <noreply@openai.com>
2026-10-08 12:12:34 +08:00
ZyyandCodex 63802f9b1a fix(usage): exclude cached tokens from displayed input
Reuse the fresh input calculation across usage records, token details and key usage metrics.

Co-authored-by: Codex <noreply@openai.com>
2026-10-08 11:43:26 +08:00
ZyyandCodex de599ed768 fix(openai): apply timezone overrides without content kinds
Recognize complete Codex environment contexts when custom providers omit
content classifications, while keeping explicit classifications authoritative.

Cover HTTP and WebSocket overrides and proxy location precedence, and
document timezone and client privacy boundaries.

Refs #366

Co-authored-by: Codex <noreply@openai.com>
2026-10-08 11:16:31 +08:00
ZyyandCodex 3315ddbff6 fix(openai): isolate guardian concurrency from ordinary requests
Give Guardian requests a separate per-account lease pool and wait queue while preserving ordinary concurrency and shared request intervals. Use the ordinary pool when the Guardian limit is zero, and align scheduling, diagnostics, settings copy, and documentation.

Co-authored-by: Codex <noreply@openai.com>
2026-10-08 10:38:52 +08:00
ZyyandCodex 6c583e341a fix(openai): reject invalid continuation replay before account selection
Check the recorded native continuation scope before selecting an account
or committing a session binding. Reuse the same scope validation after
middleware while preserving native connection checks and migration policy.

Cover ReplayOwner and ReplayAny with persisted provider metadata or a
connection-local pin, with and without an existing session binding.
Rejected replays must not acquire leases, write bindings or aliases, send
upstream requests, or redirect subsequent child requests.

Refs #364

Co-authored-by: Codex <noreply@openai.com>
2026-10-07 22:45:42 +08:00
218 changed files with 8286 additions and 2932 deletions
+2 -3
View File
@@ -21,7 +21,7 @@
## 快速预览
无需部署,打开 [快速预览服务](https://codex-proxy-rs.ainz.cc) 即可体验管理端的系统概览、账号分组、代理管理与用量统计
无需部署即可体验管理端的系统概览、账号分组、代理管理与用量统计:
| 登录信息 | 默认值 |
| --- | --- |
@@ -53,8 +53,7 @@ curl -fsSL https://raw.githubusercontent.com/zyycn/codex-proxy-rs/main/deploy/in
[安装脚本](deploy/install.sh) 默认安装到当前目录下的 `codex-proxy-rs/`,下载同一正式 Release 的部署文件,
自动生成密码、设置目录权限并启动服务
完成后会显示访问地址和管理员密码,请保存密码,再按下方步骤
[添加账号与客户端密钥](#添加账号与客户端密钥)
完成后保存显示的访问地址和管理员密码,再[登录管理端](#登录管理端)并[添加账号与客户端密钥](#添加账号与客户端密钥)
可在执行 `bash install.sh` 时传入环境变量:
+59 -43
View File
@@ -303,7 +303,7 @@ dependencies = [
"lru",
"percent-encoding",
"regex-lite",
"sha2 0.11.0",
"sha2",
"tracing",
"url",
]
@@ -325,7 +325,7 @@ dependencies = [
"hmac",
"http",
"percent-encoding",
"sha2 0.11.0",
"sha2",
"time",
"tracing",
]
@@ -358,7 +358,7 @@ dependencies = [
"md-5",
"pin-project-lite",
"sha1 0.11.0",
"sha2 0.11.0",
"sha2",
"tracing",
]
@@ -877,7 +877,7 @@ dependencies = [
"gateway-plugin-sdk",
"hex",
"serde_json",
"sha2 0.11.0",
"sha2",
"tar",
"tempfile",
"thiserror",
@@ -1688,7 +1688,7 @@ dependencies = [
"semver",
"serde",
"serde_json",
"sha2 0.11.0",
"sha2",
"subtle",
"syn 3.0.6",
"tempfile",
@@ -1719,7 +1719,7 @@ dependencies = [
"secrecy",
"serde",
"serde_json",
"sha2 0.11.0",
"sha2",
"thiserror",
"tokio",
"tokio-tungstenite 0.29.0",
@@ -1749,7 +1749,7 @@ dependencies = [
"semver",
"serde",
"serde_json",
"sha2 0.11.0",
"sha2",
"thiserror",
"tracing",
"url",
@@ -1789,7 +1789,7 @@ dependencies = [
"serde_ignored",
"serde_json",
"serde_path_to_error",
"sha2 0.11.0",
"sha2",
"tar",
"tempfile",
"thiserror",
@@ -1830,7 +1830,7 @@ dependencies = [
"semver",
"serde",
"serde_json",
"sha2 0.11.0",
"sha2",
"sqlx",
"tar",
"tempfile",
@@ -1858,6 +1858,7 @@ dependencies = [
name = "gateway-protocol"
version = "0.0.0"
dependencies = [
"http",
"httpdate",
"memchr",
"serde",
@@ -1886,7 +1887,7 @@ dependencies = [
"secrecy",
"serde",
"serde_json",
"sha2 0.11.0",
"sha2",
"sqlx",
"tempfile",
"thiserror",
@@ -2524,6 +2525,15 @@ version = "0.2.186"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
[[package]]
name = "libredox"
version = "0.1.25"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61ff90caf6077a803a240f62fdbe88645a890bbca49ef8174c3cb0404362171d"
dependencies = [
"libc",
]
[[package]]
name = "libsqlite3-sys"
version = "0.37.0"
@@ -2841,6 +2851,24 @@ dependencies = [
"libc",
]
[[package]]
name = "objc2-core-foundation"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2a180dd8642fa45cdb7dd721cd4c11b1cadd4929ce112ebd8b9f5803cc79d536"
dependencies = [
"bitflags",
]
[[package]]
name = "objc2-system-configuration"
version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7216bd11cbda54ccabcab84d523dc93b858ec75ecfb3a7d89513fa22464da396"
dependencies = [
"objc2-core-foundation",
]
[[package]]
name = "once_cell"
version = "1.21.4"
@@ -2929,7 +2957,7 @@ dependencies = [
"elliptic-curve",
"primefield",
"primeorder",
"sha2 0.11.0",
"sha2",
]
[[package]]
@@ -3270,7 +3298,7 @@ dependencies = [
"semver",
"serde",
"serde_json",
"sha2 0.11.0",
"sha2",
"subtle",
"tar",
"tempfile",
@@ -3312,7 +3340,7 @@ dependencies = [
"semver",
"serde",
"serde_json",
"sha2 0.11.0",
"sha2",
"thiserror",
"tokio",
"tracing",
@@ -4023,17 +4051,6 @@ version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbfa15b3dddfee50a0fff136974b3e1bde555604ba463834a7eb7deb6417705d"
[[package]]
name = "sha2"
version = "0.10.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a7507d819769d01a365ab707794a4084392c824f54a7a6a7862f8c3d0892b283"
dependencies = [
"cfg-if",
"cpufeatures 0.2.17",
"digest 0.10.7",
]
[[package]]
name = "sha2"
version = "0.11.0"
@@ -4154,8 +4171,7 @@ dependencies = [
[[package]]
name = "sqlx"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "378620ccc25c62c89d8be1c819e76a88d59bdcc3304733330788948e619bfd71"
source = "git+https://github.com/transact-rs/sqlx?rev=f1e94ec34dd5f11ad6488a3a8ac7988439a3f757#f1e94ec34dd5f11ad6488a3a8ac7988439a3f757"
dependencies = [
"sqlx-core",
"sqlx-macros",
@@ -4167,8 +4183,7 @@ dependencies = [
[[package]]
name = "sqlx-core"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05b44e85bf579a8eeb4ceaa77a3a523baf2bf0e9bac7e40f405d537b5d2d5ccb"
source = "git+https://github.com/transact-rs/sqlx?rev=f1e94ec34dd5f11ad6488a3a8ac7988439a3f757#f1e94ec34dd5f11ad6488a3a8ac7988439a3f757"
dependencies = [
"base64 0.22.1",
"bytes",
@@ -4191,7 +4206,7 @@ dependencies = [
"rustls",
"serde",
"serde_json",
"sha2 0.10.9",
"sha2",
"smallvec",
"thiserror",
"tokio",
@@ -4205,8 +4220,7 @@ dependencies = [
[[package]]
name = "sqlx-macros"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bd2b84f2bc39a5705ef27ec785a11c934a41bbd4a24941e257927cddc26b60bf"
source = "git+https://github.com/transact-rs/sqlx?rev=f1e94ec34dd5f11ad6488a3a8ac7988439a3f757#f1e94ec34dd5f11ad6488a3a8ac7988439a3f757"
dependencies = [
"proc-macro2",
"quote",
@@ -4218,8 +4232,7 @@ dependencies = [
[[package]]
name = "sqlx-macros-core"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb8d96de5fdc85a5c4ec813432b523ec637e80ba98f046555f75f7908ddac7c3"
source = "git+https://github.com/transact-rs/sqlx?rev=f1e94ec34dd5f11ad6488a3a8ac7988439a3f757#f1e94ec34dd5f11ad6488a3a8ac7988439a3f757"
dependencies = [
"cfg-if",
"dotenvy",
@@ -4230,7 +4243,7 @@ dependencies = [
"quote",
"serde",
"serde_json",
"sha2 0.10.9",
"sha2",
"sqlx-core",
"sqlx-mysql",
"sqlx-postgres",
@@ -4244,8 +4257,7 @@ dependencies = [
[[package]]
name = "sqlx-mysql"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "90b8020fe17c5f2c245bfa2505d7ef59c5604839527c740266ad2214acebea27"
source = "git+https://github.com/transact-rs/sqlx?rev=f1e94ec34dd5f11ad6488a3a8ac7988439a3f757#f1e94ec34dd5f11ad6488a3a8ac7988439a3f757"
dependencies = [
"bitflags",
"byteorder",
@@ -4262,7 +4274,7 @@ dependencies = [
"percent-encoding",
"serde",
"sha1 0.11.0",
"sha2 0.11.0",
"sha2",
"sqlx-core",
"thiserror",
"tracing",
@@ -4272,8 +4284,7 @@ dependencies = [
[[package]]
name = "sqlx-postgres"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "87a2bdd6e83f6b3ea525ca9fee568030508b58355a43d0b2c1674d5f79dcd65e"
source = "git+https://github.com/transact-rs/sqlx?rev=f1e94ec34dd5f11ad6488a3a8ac7988439a3f757#f1e94ec34dd5f11ad6488a3a8ac7988439a3f757"
dependencies = [
"atoi",
"base64 0.22.1",
@@ -4296,7 +4307,7 @@ dependencies = [
"rand 0.10.2",
"serde",
"serde_json",
"sha2 0.11.0",
"sha2",
"smallvec",
"sqlx-core",
"stringprep",
@@ -4309,8 +4320,7 @@ dependencies = [
[[package]]
name = "sqlx-sqlite"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "488e99c397a62007e4229aec669a179816339afc6d2620ca6fa420dbee2e982c"
source = "git+https://github.com/transact-rs/sqlx?rev=f1e94ec34dd5f11ad6488a3a8ac7988439a3f757#f1e94ec34dd5f11ad6488a3a8ac7988439a3f757"
dependencies = [
"atoi",
"chrono",
@@ -4493,7 +4503,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom 0.4.3",
"getrandom 0.3.4",
"once_cell",
"rustix",
"windows-sys 0.61.2",
@@ -5182,6 +5192,12 @@ name = "whoami"
version = "2.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "998767ef88740d1f5b0682a9c53c24431453923962269c2db68ee43788c5a40d"
dependencies = [
"libc",
"libredox",
"objc2-system-configuration",
"web-sys",
]
[[package]]
name = "winapi"
+2
View File
@@ -59,3 +59,5 @@ panic = "abort"
[patch.crates-io]
# Upstream SOCKS IPv6 fix; remove once a release includes hyperium/hyper-util#302.
hyper-util = { git = "https://github.com/hyperium/hyper-util", rev = "191b1acf0a6b18e807e4abf5a6d4a7c47979ecd6" }
# 固定上游 BEGIN 取消回滚修复,避免未结束事务复用;见 transact-rs/sqlx#4394
sqlx = { git = "https://github.com/transact-rs/sqlx", rev = "f1e94ec34dd5f11ad6488a3a8ac7988439a3f757" }
+1 -1
View File
@@ -1,4 +1,4 @@
//! 服务模式完成全部组装后交给 Host 监听,并在请求和 Worker 排空后关闭插件
//! 服务模式完成全部组装后交给 Host 监听,并在停止监听与 Worker 收尾后关闭插件
use gateway_host::ConfigError;
+2 -5
View File
@@ -28,11 +28,8 @@ cpr-plugin package \
资源映射相对于作者清单所在目录解析。输出包括完整插件 ID、版本和目标 triple 命名的 `.tar.gz` 及 `.sha256`;归档根目录直接包含 `plugin.json`、可执行文件和已声明资源,不包含开发依赖
作者清单可省略普通贡献项的 `id`、`version` 和固定阶段;`middleware` 必须显式声明版本 `4`,
并从 `http`、`websocket`、`service`、`request`、`attempt` 中选择挂载阶段;`upstream_adapter` 必须显式声明版本 `2`。
CLI 与 SDK 的 `Manifest::from_author_slice`、`PluginBuilder::from_json` 共用规范化入口;归档中的
`plugin.json` 写入补全后的贡献项字段,不保留另一份作者声明。
完整规则见 [SDK 清单](../../crates/gateway-plugin/sdk/docs/manifest.md)
作者清单的省略规则、能力版本和挂载阶段见 [SDK 扩展项声明](../../crates/gateway-plugin/sdk/docs/manifest.md#扩展项简写)。
CLI 与 SDK 的 `Manifest::from_author_slice`、`PluginBuilder::from_json` 共用规范化入口,归档只保留补全字段后的 `plugin.json`
支持 `x86_64-unknown-linux-gnu`、`aarch64-unknown-linux-gnu` 和 `aarch64-apple-darwin`。
完整开发示例在独立仓库 `codex-proxy-plugins` 的 `examples/workbench`,可通过[子模块](../../../docs/development.md#源码联调)检出;本工具不负责源码构建、上传或启用插件
@@ -126,6 +126,16 @@ pub struct AccountGroupPage {
pub page_size: u16,
}
/// 分组选择器使用的轻量分页结果
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct AccountGroupOptionsPage {
pub config_revision: Revision,
pub items: Vec<AccountGroupRef>,
pub total: u64,
pub page: u32,
pub page_size: u16,
}
/// 创建账号分组
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct CreateAccountGroup {
@@ -646,7 +646,7 @@ pub struct DashboardObservation {
}
/// 使用记录表格的窄读模型
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq)]
pub struct UsageListRecord {
pub client_api_key_name: Option<String>,
pub id: String,
@@ -687,6 +687,13 @@ pub struct UsageListRecord {
pub first_text_ms: Option<u64>,
pub first_token_ms: Option<u64>,
pub provider_processing_ms: Option<u64>,
pub upstream_response_ms: Option<u64>,
pub upstream_api_overhead_ms: Option<f64>,
pub upstream_engine_ms: Option<f64>,
pub upstream_engine_iapi_ttft_ms: Option<f64>,
pub upstream_engine_service_ttft_ms: Option<f64>,
pub upstream_engine_iapi_tbt_ms: Option<f64>,
pub upstream_engine_service_tbt_ms: Option<f64>,
pub latency_ms: Option<u64>,
pub admission_decision_ms: Option<u64>,
pub account_selection_wait_ms: Option<u64>,
@@ -702,7 +709,7 @@ pub struct UsageListRecord {
}
/// 一次完整模型请求的公共观测记录
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq)]
pub struct UsageRecord {
pub id: String,
pub client_api_key_ref: String,
@@ -761,6 +768,13 @@ pub struct UsageRecord {
pub first_text_ms: Option<u64>,
pub first_token_ms: Option<u64>,
pub provider_processing_ms: Option<u64>,
pub upstream_response_ms: Option<u64>,
pub upstream_api_overhead_ms: Option<f64>,
pub upstream_engine_ms: Option<f64>,
pub upstream_engine_iapi_ttft_ms: Option<f64>,
pub upstream_engine_service_ttft_ms: Option<f64>,
pub upstream_engine_iapi_tbt_ms: Option<f64>,
pub upstream_engine_service_tbt_ms: Option<f64>,
pub latency_ms: Option<u64>,
pub admission_decision_ms: Option<u64>,
pub account_selection_wait_ms: Option<u64>,
@@ -781,7 +795,7 @@ pub struct UsageRecord {
}
/// 用量分页结果
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq)]
pub struct UsagePage {
pub items: Vec<UsageListRecord>,
pub current_page: u32,
@@ -827,7 +841,7 @@ pub struct UsageAttempt {
}
/// 一条请求及其全部尝试
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq)]
pub struct UsageDetail {
pub trace: Option<serde_json::Value>,
pub related_requests: Vec<serde_json::Value>,
@@ -175,12 +175,13 @@ pub trait PluginDistribution: Send + Sync {
/// Runtime 只解释插件包格式;安装事务与来源选择归 Admin
#[async_trait]
pub trait PluginPackageInspector: Send + Sync {
/// 可解析但版本范围未经宿主承诺的诊断,不决定启动资格
/// 完整校验包并返回版本范围诊断;校验失败与可加载但未经宿主承诺的提醒分开
async fn compatibility_warning(
&self,
_archive: Arc<[u8]>,
_expected_sha256: String,
archive: Arc<[u8]>,
expected_sha256: String,
) -> Result<Option<String>, AdminError> {
self.inspect(archive, Some(expected_sha256)).await?;
Ok(None)
}
@@ -12,8 +12,9 @@ use super::backup::BackupStorePorts;
use crate::model::{
MutationContext, Revision,
account_groups::{
AccountGroupListQuery, AccountGroupMemberFact, AccountGroupMutation, AccountGroupPage,
DeleteAccountGroup, NewAccountGroup, SetAccountGroupEnabled, UpdateAccountGroup,
AccountGroupListQuery, AccountGroupMemberFact, AccountGroupMutation,
AccountGroupOptionsPage, AccountGroupPage, DeleteAccountGroup, NewAccountGroup,
SetAccountGroupEnabled, UpdateAccountGroup,
},
accounts::{
AccountListQuery, AccountPage, AccountPageItem, AccountRuntimeSnapshot,
@@ -385,6 +386,11 @@ pub trait AccountGroupStore: Send + Sync {
query: AccountGroupListQuery,
) -> AdminStoreResult<AccountGroupPage>;
async fn list_account_group_options(
&self,
query: AccountGroupListQuery,
) -> AdminStoreResult<AccountGroupOptionsPage>;
async fn load_account_group_members(
&self,
group_ids: &[gateway_core::routing::AccountGroupId],
@@ -19,9 +19,9 @@ use crate::{
AdminError, MutationContext,
account_groups::{
AccountGroupAccountSummary, AccountGroupCapacity, AccountGroupListQuery,
AccountGroupMemberFact, AccountGroupMutation, AccountGroupPage, AccountGroupRecord,
CreateAccountGroup, DeleteAccountGroup, NewAccountGroup, SetAccountGroupEnabled,
UpdateAccountGroup,
AccountGroupMemberFact, AccountGroupMutation, AccountGroupOptionsPage,
AccountGroupPage, AccountGroupRecord, CreateAccountGroup, DeleteAccountGroup,
NewAccountGroup, SetAccountGroupEnabled, UpdateAccountGroup,
},
},
ports::store::{AccountGroupStore, AccountRuntimeStore},
@@ -33,6 +33,10 @@ use super::{map_store_error, publish_committed};
#[async_trait]
pub trait AccountGroupService: Send + Sync {
async fn list(&self, query: AccountGroupListQuery) -> Result<AccountGroupPage, AdminError>;
async fn list_options(
&self,
query: AccountGroupListQuery,
) -> Result<AccountGroupOptionsPage, AdminError>;
async fn create(
&self,
context: &MutationContext,
@@ -130,6 +134,16 @@ impl AccountGroupService for DefaultAccountGroupService {
Ok(page)
}
async fn list_options(
&self,
query: AccountGroupListQuery,
) -> Result<AccountGroupOptionsPage, AdminError> {
self.store
.list_account_group_options(query)
.await
.map_err(|error| map_store_error(error, "account group options"))
}
async fn create(
&self,
context: &MutationContext,
@@ -2,7 +2,10 @@
use std::collections::{BTreeMap, BTreeSet};
use gateway_core::{policy::ClientApiKeyId, routing::AccountGroupId};
use gateway_core::{
policy::ClientApiKeyId,
routing::{AccountGroupId, extensions::ExtensionSetReference},
};
use secrecy::ExposeSecret;
use super::PluginsService;
@@ -41,14 +44,11 @@ impl PluginsService {
.map_err(|error| map_store_error(error, "plugin"))?;
let warning = match self
.inspector
.inspect(artifact.archive.clone(), Some(digest.to_owned()))
.compatibility_warning(artifact.archive, digest.to_owned())
.await
{
Ok(_) => PackageStatus {
warning: self
.inspector
.compatibility_warning(artifact.archive, digest.to_owned())
.await?,
Ok(warning) => PackageStatus {
warning,
load_error: None,
},
Err(error) if error.kind() == AdminErrorKind::Invalid => PackageStatus {
@@ -368,25 +368,8 @@ impl PluginsService {
)
.await
.map_err(|e| map_store_error(e, "plugin"))?;
let prepared = if result.config_revision == candidate_revision
&& result.instance.revision == candidate_revision
{
prepared
} else {
snapshot.config_revision = result.config_revision;
snapshot
.instances
.retain(|item| item.id != result.instance.id);
snapshot.instances.push(result.instance.clone());
apply_replacements(&mut snapshot, &replacements);
self.preparation.prepare(snapshot).await?
};
self.preparation
.activate_state(&prepared, &result.instance)
self.publish_instance(snapshot, prepared, &result, &replacements)
.await?;
// prepared 的强引用跨过提交与唯一发布入口,防止候选在被读取之前回收
publish_committed(self.snapshots.as_ref(), result.config_revision).await?;
drop(prepared);
Ok(result)
}
@@ -463,48 +446,45 @@ impl PluginsService {
.await);
}
};
if let Err(error) = self
.preparation
.migrate_state(&migration_prepared, transition.clone())
.await
{
return Err(self
.recover_failed_state_transition(
Some(&transition.id),
&previous,
&previous_state,
disabled_revision,
// 迁移开始后的所有提交前失败都走同一恢复出口;提交成功后不再回滚旧配置
let committed = async {
self.preparation
.migrate_state(&migration_prepared, transition.clone())
.await?;
let mut current = self
.store
.load_instances()
.await
.map_err(|error| map_store_error(error, "plugin"))?;
if current.config_revision != expected_revision {
return Err(AdminError::conflict("迁移期间插件配置已发生变化"));
}
let candidate_revision = next_revision(expected_revision)?;
target.revision = candidate_revision;
current.instances.retain(|item| item.id != target.id);
current.instances.push(target.clone());
current.config_revision = candidate_revision;
apply_replacements(&mut current, replacements);
let prepared = self.preparation.prepare(current.clone()).await?;
let result = self
.store
.save_instance_replacing(
target,
expected_revision,
PluginStateCommit {
configuration: target_state,
transition_id: Some(transition.id.clone()),
},
replacements,
context,
error,
)
.await);
.await
.map_err(|error| map_store_error(error, "plugin"))?;
Ok((current, prepared, result))
}
let mut current = self
.store
.load_instances()
.await
.map_err(|error| map_store_error(error, "plugin"))?;
if current.config_revision != expected_revision {
return Err(self
.recover_failed_state_transition(
Some(&transition.id),
&previous,
&previous_state,
disabled_revision,
context,
AdminError::conflict("迁移期间插件配置已发生变化"),
)
.await);
}
let candidate_revision = next_revision(expected_revision)?;
target.revision = candidate_revision;
current.instances.retain(|item| item.id != target.id);
current.instances.push(target.clone());
current.config_revision = candidate_revision;
apply_replacements(&mut current, replacements);
let prepared = match self.preparation.prepare(current.clone()).await {
Ok(prepared) => prepared,
.await;
let (current, prepared, result) = match committed {
Ok(committed) => committed,
Err(error) => {
return Err(self
.recover_failed_state_transition(
@@ -518,53 +498,9 @@ impl PluginsService {
.await);
}
};
let result = match self
.store
.save_instance_replacing(
target,
expected_revision,
PluginStateCommit {
configuration: target_state,
transition_id: Some(transition.id.clone()),
},
replacements,
context,
)
.await
{
Ok(result) => result,
Err(error) => {
return Err(self
.recover_failed_state_transition(
Some(&transition.id),
&previous,
&previous_state,
disabled_revision,
context,
map_store_error(error, "plugin"),
)
.await);
}
};
let prepared = if result.config_revision == candidate_revision
&& result.instance.revision == candidate_revision
{
prepared
} else {
current.config_revision = result.config_revision;
current
.instances
.retain(|item| item.id != result.instance.id);
current.instances.push(result.instance.clone());
apply_replacements(&mut current, replacements);
self.preparation.prepare(current).await?
};
self.preparation
.activate_state(&prepared, &result.instance)
self.publish_instance(current, prepared, &result, replacements)
.await?;
publish_committed(self.snapshots.as_ref(), result.config_revision).await?;
drop(migration_prepared);
drop(prepared);
Ok(result)
}
@@ -615,22 +551,8 @@ impl PluginsService {
)
.await
.map_err(|error| map_store_error(error, "plugin"))?;
let prepared = if mutation.config_revision == candidate_revision
&& mutation.instance.revision == candidate_revision
{
prepared
} else {
snapshot.config_revision = mutation.config_revision;
snapshot
.instances
.retain(|item| item.id != mutation.instance.id);
snapshot.instances.push(mutation.instance.clone());
self.preparation.prepare(snapshot).await?
};
self.preparation
.activate_state(&prepared, &mutation.instance)
self.publish_instance(snapshot, prepared, &mutation, &[])
.await?;
publish_committed(self.snapshots.as_ref(), mutation.config_revision).await?;
Ok::<(), AdminError>(())
}
.await;
@@ -646,6 +568,35 @@ impl PluginsService {
}
}
/// 以事务返回的版本激活并发布,候选强引用必须覆盖整个提交后阶段
async fn publish_instance(
&self,
mut snapshot: PluginInstanceSnapshot,
prepared: ExtensionSetReference,
mutation: &PluginInstanceMutation,
replacements: &[PluginInstanceReplacement],
) -> Result<(), AdminError> {
let prepared = if mutation.config_revision == snapshot.config_revision
&& mutation.instance.revision == snapshot.config_revision
{
prepared
} else {
snapshot.config_revision = mutation.config_revision;
snapshot
.instances
.retain(|item| item.id != mutation.instance.id);
snapshot.instances.push(mutation.instance.clone());
apply_replacements(&mut snapshot, replacements);
self.preparation.prepare(snapshot).await?
};
self.preparation
.activate_state(&prepared, &mutation.instance)
.await?;
publish_committed(self.snapshots.as_ref(), mutation.config_revision).await?;
drop(prepared);
Ok(())
}
pub async fn delete_instance(
&self,
id: &str,
@@ -10,12 +10,13 @@ use async_trait::async_trait;
use chrono::{DateTime, Duration, Utc};
use gateway_admin::{
model::{
MutationContext, PageSize, Revision,
AdminErrorKind, MutationContext, PageSize, Revision,
account_groups::{
AccountGroupAccountSummary, AccountGroupCapacity, AccountGroupColor,
AccountGroupListQuery, AccountGroupMemberFact, AccountGroupMutation, AccountGroupPage,
AccountGroupRecord, AccountGroupUsage, DeleteAccountGroup, NewAccountGroup,
SetAccountGroupEnabled, UpdateAccountGroup,
AccountGroupListQuery, AccountGroupMemberFact, AccountGroupMutation,
AccountGroupOptionsPage, AccountGroupPage, AccountGroupRecord, AccountGroupRef,
AccountGroupUsage, DeleteAccountGroup, NewAccountGroup, SetAccountGroupEnabled,
UpdateAccountGroup,
},
accounts::AccountRuntimeSnapshot,
observability::DecimalAmount,
@@ -90,6 +91,62 @@ async fn group_query_service_enriches_only_current_page_members_with_runtime_fac
);
}
#[tokio::test]
async fn group_options_query_should_skip_member_and_runtime_enrichment() {
let groups = Arc::new(FakeGroupStore::default());
let runtime = Arc::new(FakeRuntimeStore::default());
let service = AdminHarness::new()
.account_groups(groups.clone())
.account_runtime(runtime.clone())
.build()
.await;
let page = service
.account_groups()
.list_options(AccountGroupListQuery {
page: 1,
page_size: PageSize::new(20).expect("page size"),
search: None,
enabled: None,
})
.await
.expect("list account group options");
assert_eq!(page.items[0].id.as_str(), GROUP_ID);
assert!(
groups
.requested_groups
.lock()
.expect("requested groups")
.is_empty()
);
assert!(
runtime
.requested_accounts
.lock()
.expect("requested accounts")
.is_empty()
);
}
#[tokio::test]
async fn group_options_query_should_preserve_store_failures() {
let service = AdminHarness::new().build().await;
let error = service
.account_groups()
.list_options(AccountGroupListQuery {
page: 1,
page_size: PageSize::new(20).expect("page size"),
search: None,
enabled: None,
})
.await
.expect_err("unavailable options store must fail");
assert_eq!(error.kind(), AdminErrorKind::Unavailable);
}
#[derive(Default)]
struct FakeGroupStore {
requested_groups: Mutex<Vec<String>>,
@@ -144,6 +201,24 @@ impl AccountGroupStore for FakeGroupStore {
})
}
async fn list_account_group_options(
&self,
query: AccountGroupListQuery,
) -> AdminStoreResult<AccountGroupOptionsPage> {
Ok(AccountGroupOptionsPage {
config_revision: Revision::new(1).expect("revision"),
items: vec![AccountGroupRef {
id: group_id(),
name: "Primary".to_owned(),
color: AccountGroupColor::parse("#2563EBFF").expect("color"),
enabled: true,
}],
total: 1,
page: query.page,
page_size: query.page_size.get(),
})
}
async fn load_account_group_members(
&self,
group_ids: &[AccountGroupId],
@@ -300,7 +375,7 @@ fn unused() -> AdminStoreError {
#[tokio::test]
async fn invalid_group_fields_are_rejected_before_store_mutations() {
use gateway_admin::model::{AdminErrorKind, MutationActor, account_groups::CreateAccountGroup};
use gateway_admin::model::{MutationActor, account_groups::CreateAccountGroup};
let service = AdminHarness::new()
.account_groups(Arc::new(FakeGroupStore::default()))
.build()
@@ -29,8 +29,9 @@ use gateway_admin::{
model::{
MutationContext, Revision,
account_groups::{
AccountGroupListQuery, AccountGroupMemberFact, AccountGroupMutation, AccountGroupPage,
DeleteAccountGroup, NewAccountGroup, SetAccountGroupEnabled, UpdateAccountGroup,
AccountGroupListQuery, AccountGroupMemberFact, AccountGroupMutation,
AccountGroupOptionsPage, AccountGroupPage, DeleteAccountGroup, NewAccountGroup,
SetAccountGroupEnabled, UpdateAccountGroup,
},
accounts::{
AccountListQuery, AccountPage, AccountRuntimeSnapshot, AccountUpdateResult,
@@ -424,6 +425,13 @@ impl AccountGroupStore for UnavailableAccountGroupStore {
Err(unavailable("account groups"))
}
async fn list_account_group_options(
&self,
_: AccountGroupListQuery,
) -> AdminStoreResult<AccountGroupOptionsPage> {
Err(unavailable("account group options"))
}
async fn load_account_group_members(
&self,
_: &[gateway_core::routing::AccountGroupId],
@@ -1138,6 +1138,13 @@ fn total_record(
first_text_ms: None,
first_token_ms: None,
provider_processing_ms: None,
upstream_response_ms: None,
upstream_api_overhead_ms: None,
upstream_engine_ms: None,
upstream_engine_iapi_ttft_ms: None,
upstream_engine_service_ttft_ms: None,
upstream_engine_iapi_tbt_ms: None,
upstream_engine_service_tbt_ms: None,
latency_ms: Some(100),
admission_decision_ms: None,
account_selection_wait_ms: None,
@@ -85,9 +85,10 @@ impl gateway_admin::ports::plugins::PluginPackageInspector for LifecycleFixture
}
async fn compatibility_warning(
&self,
_: Arc<[u8]>,
_: String,
archive: Arc<[u8]>,
digest: String,
) -> Result<Option<String>, AdminError> {
self.inspect(archive, Some(digest)).await?;
Ok(self.data.lock().unwrap().compatibility_warning.clone())
}
async fn inspect(
@@ -130,6 +131,9 @@ struct FixtureData {
saves: Vec<SavedInstance>,
history: BTreeMap<String, PluginVersionConfiguration>,
aborted: usize,
transition_active: bool,
fail_reload_after_migration: bool,
fail_next_load: bool,
activated: usize,
quiesced: usize,
diagnostics: Option<BTreeMap<String, PluginInstanceRuntime>>,
@@ -183,6 +187,9 @@ impl LifecycleFixture {
},
)]),
aborted: 0,
transition_active: false,
fail_reload_after_migration: false,
fail_next_load: false,
activated: 0,
quiesced: 0,
diagnostics: None,
@@ -235,6 +242,9 @@ impl LifecycleFixture {
.instances
.retain(|current| current.id != instance.id);
data.snapshot.instances.push(instance.clone());
if transition_id.is_some() {
data.transition_active = false;
}
data.saves.push(SavedInstance {
artifact_sha256: instance.artifact_sha256.clone(),
enabled: instance.enabled,
@@ -334,7 +344,11 @@ impl gateway_admin::ports::plugins::PluginStateLifecycle for LifecycleFixture {
_: PluginStateTransition,
) -> Result<(), AdminError> {
match self.behavior {
MigrationBehavior::Succeed => Ok(()),
MigrationBehavior::Succeed => {
let mut data = self.data.lock().unwrap();
data.fail_next_load = std::mem::take(&mut data.fail_reload_after_migration);
Ok(())
}
MigrationBehavior::Fail => Err(AdminError::invalid("migration fixture failed")),
MigrationBehavior::ConcurrentChangeThenFail => {
let mut data = self.data.lock().unwrap();
@@ -398,9 +412,10 @@ impl PluginStateStore for LifecycleFixture {
artifact_sha256: &str,
_: PluginStateConfiguration,
) -> PluginStateStoreResult<PluginStateTransition> {
let data = self.data.lock().unwrap();
let mut data = self.data.lock().unwrap();
let current = &data.snapshot.instances[0];
if current.enabled
if data.transition_active
|| current.enabled
|| current.id != instance_id
|| current.revision != expected_instance_revision
|| artifact_sha256 != NEW_ARTIFACT
@@ -409,6 +424,7 @@ impl PluginStateStore for LifecycleFixture {
PluginStateStoreErrorKind::Conflict,
));
}
data.transition_active = true;
Ok(PluginStateTransition {
id: "00000000-0000-7000-8000-000000000002".into(),
instance_id: instance_id.into(),
@@ -438,7 +454,9 @@ impl PluginStateStore for LifecycleFixture {
}
async fn abort_transition(&self, _: &str) -> PluginStateStoreResult<()> {
self.data.lock().unwrap().aborted += 1;
let mut data = self.data.lock().unwrap();
data.aborted += 1;
data.transition_active = false;
Ok(())
}
}
@@ -463,7 +481,11 @@ impl PluginStore for LifecycleFixture {
}))
}
async fn load_instances(&self) -> AdminStoreResult<PluginInstanceSnapshot> {
Ok(self.snapshot())
let mut data = self.data.lock().unwrap();
if std::mem::take(&mut data.fail_next_load) {
return Err(admin_error(AdminStoreErrorKind::Unavailable));
}
Ok(data.snapshot.clone())
}
async fn load_version_configuration(
@@ -1046,6 +1068,36 @@ async fn failed_state_migration_restores_the_previous_enabled_version_with_cas()
assert_eq!(published.revisions.lock().unwrap().as_slice(), [11, 12]);
}
#[tokio::test]
async fn reload_failure_after_migration_aborts_restores_and_allows_retry() {
let fixture = LifecycleFixture::new(MigrationBehavior::Succeed);
fixture.data.lock().unwrap().fail_reload_after_migration = true;
let published = Arc::new(Published::default());
let service = service(fixture.clone(), published.clone());
let id = "00000000-0000-7000-8000-000000000001";
let error = service
.configure_instance(Some(id), input(), &context())
.await
.err()
.unwrap();
assert_eq!(error.kind(), AdminErrorKind::Unavailable);
{
let data = fixture.data.lock().unwrap();
assert_eq!(data.aborted, 1);
assert!(!data.transition_active);
assert!(data.snapshot.instances[0].enabled);
assert_eq!(data.snapshot.instances[0].artifact_sha256, OLD_ARTIFACT);
assert_eq!(*published.revisions.lock().unwrap(), [11, 12]);
}
let result = service
.configure_instance(Some(id), input(), &context())
.await
.unwrap();
assert_eq!(result.instance.artifact_sha256, NEW_ARTIFACT);
assert!(result.instance.enabled);
assert!(!fixture.data.lock().unwrap().transition_active);
}
#[tokio::test]
async fn failed_state_migration_never_overwrites_a_concurrent_configuration_change() {
let fixture = LifecycleFixture::new(MigrationBehavior::ConcurrentChangeThenFail);
@@ -16,8 +16,9 @@ use gateway_admin::model::{
PageSize,
account_groups::{
AccountGroupAccountSummary, AccountGroupCapacity, AccountGroupColor, AccountGroupListQuery,
AccountGroupMutation, AccountGroupPage, AccountGroupRecord, AccountGroupUsage,
CreateAccountGroup, DeleteAccountGroup, SetAccountGroupEnabled, UpdateAccountGroup,
AccountGroupMutation, AccountGroupOptionsPage, AccountGroupPage, AccountGroupRecord,
AccountGroupUsage, CreateAccountGroup, DeleteAccountGroup, SetAccountGroupEnabled,
UpdateAccountGroup,
},
};
use gateway_core::{account::FastMode, routing::AccountGroupId};
@@ -212,6 +213,38 @@ struct AccountGroupPageData {
config_revision: u64,
}
#[derive(Debug, Clone, Serialize)]
#[serde(rename_all = "camelCase")]
struct AccountGroupOptionsPageData {
items: Vec<AccountGroupRefView>,
page: PageMeta,
config_revision: u64,
}
impl From<AccountGroupOptionsPage> for AccountGroupOptionsPageData {
fn from(page: AccountGroupOptionsPage) -> Self {
let total_pages = if page.total == 0 {
0
} else {
page.total.div_ceil(u64::from(page.page_size))
};
Self {
items: page
.items
.into_iter()
.map(AccountGroupRefView::from)
.collect(),
page: PageMeta::new(
page.page,
u32::from(page.page_size),
page.total,
u32::try_from(total_pages).unwrap_or(u32::MAX),
),
config_revision: page.config_revision.get(),
}
}
}
impl From<(AccountGroupPage, crate::time::TimePresenter)> for AccountGroupPageData {
fn from((page, time): (AccountGroupPage, crate::time::TimePresenter)) -> Self {
let total_pages = if page.total == 0 {
@@ -263,6 +296,7 @@ where
{
Router::new()
.route("/api/admin/account-groups", get(list::<S>))
.route("/api/admin/account-groups/options", get(list_options::<S>))
.route("/api/admin/account-groups/create", post(create::<S>))
.route("/api/admin/account-groups/update", post(update::<S>))
.route("/api/admin/account-groups/enable", post(enable::<S>))
@@ -270,6 +304,26 @@ where
.route("/api/admin/account-groups/delete", post(delete::<S>))
}
async fn list_options<S>(
_auth: AdminAuth,
State(state): State<S>,
AdminQuery(query): AdminQuery<ListAccountGroupsQuery>,
) -> Result<impl IntoResponse, AdminError>
where
S: SessionState + Send + Sync,
{
let result = state
.admin_services()
.account_groups()
.list_options(query.into_command().map_err(map_wire_error)?)
.await
.map_err(map_service_error)?;
Ok(AdminResponse::new(
StatusCode::OK,
AdminEnvelope::ok(AccountGroupOptionsPageData::from(result)),
))
}
async fn list<S>(
_auth: AdminAuth,
State(state): State<S>,
@@ -307,8 +307,14 @@ pub(crate) fn usage_list_record_view(
first_event_ms: record.first_event_ms,
first_reasoning_ms: record.first_reasoning_ms,
first_text_ms: record.first_text_ms,
first_token_ms: record.first_token_ms,
openai_processing_ms: record.provider_processing_ms,
upstream_response_ms: record.upstream_response_ms,
upstream_api_overhead_ms: record.upstream_api_overhead_ms,
upstream_engine_ms: record.upstream_engine_ms,
upstream_engine_iapi_ttft_ms: record.upstream_engine_iapi_ttft_ms,
upstream_engine_service_ttft_ms: record.upstream_engine_service_ttft_ms,
upstream_engine_iapi_tbt_ms: record.upstream_engine_iapi_tbt_ms,
upstream_engine_service_tbt_ms: record.upstream_engine_service_tbt_ms,
},
first_token_latency_ms: record.first_token_ms,
latency_ms: record.latency_ms,
@@ -345,7 +351,6 @@ pub(crate) fn usage_record_view(
.error_message
.clone()
.unwrap_or_else(|| outcome.clone());
let first_token_display = display_duration(record.first_token_ms);
let latency_display = display_duration(record.latency_ms);
let cost_coverage = match record.cost_source.as_str() {
"provider_reported" | "calculated" => CostCoverageView {
@@ -409,15 +414,20 @@ pub(crate) fn usage_record_view(
first_event_ms: record.first_event_ms,
first_reasoning_ms: record.first_reasoning_ms,
first_text_ms: record.first_text_ms,
first_token_ms: record.first_token_ms,
openai_processing_ms: record.provider_processing_ms,
upstream_response_ms: record.upstream_response_ms,
upstream_api_overhead_ms: record.upstream_api_overhead_ms,
upstream_engine_ms: record.upstream_engine_ms,
upstream_engine_iapi_ttft_ms: record.upstream_engine_iapi_ttft_ms,
upstream_engine_service_ttft_ms: record.upstream_engine_service_ttft_ms,
upstream_engine_iapi_tbt_ms: record.upstream_engine_iapi_tbt_ms,
upstream_engine_service_tbt_ms: record.upstream_engine_service_tbt_ms,
},
attempt_index: None,
attempt_count: u64::from(record.attempt_count),
response_id: record.client_response_id,
upstream_request_id: record.upstream_request_id,
latency_ms: record.latency_ms,
first_token_ms: record.first_token_ms,
input_tokens: record.input_tokens,
output_tokens: record.output_tokens,
cached_tokens: record.cached_tokens,
@@ -441,7 +451,6 @@ pub(crate) fn usage_record_view(
costs,
cost_coverage,
first_token_latency_ms: record.first_token_ms,
first_token_latency_ms_display: first_token_display,
latency_ms_display: latency_display,
logical_outcome: outcome,
}
@@ -756,7 +765,7 @@ pub(crate) fn trend_summary_view(
ratio: None,
},
TrendSummaryView {
label: "吞吐 P50".to_owned(),
label: "速率 P50".to_owned(),
value: summary
.minimum_output_throughput_p50
.map_or_else(|| "—".to_owned(), |value| format!("{value} tok/s")),
@@ -160,7 +160,6 @@ pub struct UsageRecordView {
pub response_id: Option<String>,
pub upstream_request_id: Option<String>,
pub latency_ms: Option<u64>,
pub first_token_ms: Option<u64>,
pub input_tokens: Option<u64>,
pub output_tokens: Option<u64>,
pub cached_tokens: Option<u64>,
@@ -185,7 +184,6 @@ pub struct UsageRecordView {
pub costs: Vec<CostView>,
pub cost_coverage: CostCoverageView,
pub first_token_latency_ms: Option<u64>,
pub first_token_latency_ms_display: String,
pub latency_ms_display: String,
pub logical_outcome: String,
}
@@ -222,9 +220,21 @@ pub struct UsageLatencyDetailsView {
#[serde(skip_serializing_if = "Option::is_none")]
pub first_text_ms: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub first_token_ms: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub openai_processing_ms: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub upstream_response_ms: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub upstream_api_overhead_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub upstream_engine_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub upstream_engine_iapi_ttft_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub upstream_engine_service_ttft_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub upstream_engine_iapi_tbt_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
pub upstream_engine_service_tbt_ms: Option<f64>,
}
/// 单次上游尝试展示
+89 -15
View File
@@ -1,10 +1,12 @@
//! `/healthz` 对 Core、Store 与 Host 关键 worker 健康事实的聚合
use std::sync::Arc;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use axum::{extract::State, http::StatusCode};
use futures::future::join_all;
use gateway_core::diagnostics::{OperationalDiagnostics, OperationalFailure};
use gateway_core::error::{ErrorDetails, ErrorSource};
use gateway_core::health::{
HealthProbe, HealthState, WorkerHealthKey, WorkerHealthSnapshot, WorkerHealthSource,
WorkerRuntimeState,
@@ -19,6 +21,16 @@ const HEALTH_CHECK_TIMEOUT: Duration = Duration::from_secs(2);
pub(crate) struct HealthStatus {
probes: Arc<[Arc<dyn HealthProbe>]>,
workers: Arc<dyn WorkerHealthSource>,
diagnostics: Arc<dyn OperationalDiagnostics>,
failures: Arc<Mutex<Vec<HealthFailure>>>,
}
#[derive(Clone, PartialEq, Eq, Debug, thiserror::Error)]
#[error("{component} is {state}: {reason}")]
struct HealthFailure {
component: String,
state: &'static str,
reason: String,
}
impl HealthStatus {
@@ -26,31 +38,93 @@ impl HealthStatus {
pub(crate) fn new(
probes: Vec<Arc<dyn HealthProbe>>,
workers: Arc<dyn WorkerHealthSource>,
diagnostics: Arc<dyn OperationalDiagnostics>,
) -> Self {
Self {
probes: probes.into(),
workers,
diagnostics,
failures: Arc::default(),
}
}
pub(crate) async fn healthy(&self) -> bool {
if !self
let mut failures: Vec<_> = self
.workers
.snapshot()
.iter()
.filter(|worker| worker_affects_healthz(worker))
.all(worker_is_healthy)
{
return false;
}
let checks = self.probes.iter().map(|probe| probe.check());
tokio::time::timeout(HEALTH_CHECK_TIMEOUT, join_all(checks))
.await
.is_ok_and(|states| {
states
.into_iter()
.all(|state| matches!(state, HealthState::Healthy))
.into_iter()
.filter(|worker| worker_affects_healthz(worker) && !worker_is_healthy(worker))
.map(|worker| HealthFailure {
component: match worker.key {
WorkerHealthKey::Task(id) => format!("worker:{id}"),
WorkerHealthKey::Disabled(kind) => format!("worker:{kind:?}"),
},
state: "unhealthy",
reason: format!(
"state={:?}; last_error={}",
worker.state,
worker.last_error.as_deref().unwrap_or("none")
),
})
.collect();
// 逐个探针共享截止时间,保留已完成结果并指出具体超时组件
let deadline = tokio::time::Instant::now() + HEALTH_CHECK_TIMEOUT;
let checks = self.probes.iter().map(|probe| async move {
let (state, reason) = match tokio::time::timeout_at(deadline, probe.check()).await {
Ok(HealthState::Healthy) => return None,
Ok(HealthState::Degraded(reason)) => ("degraded", reason),
Ok(HealthState::Unhealthy(reason)) => ("unhealthy", reason),
Err(_) => ("timed_out", "health probe exceeded 2s deadline".to_owned()),
};
Some(HealthFailure {
component: format!("probe:{}", probe.name()),
state,
reason,
})
});
failures.extend(join_all(checks).await.into_iter().flatten());
let healthy = failures.is_empty();
let changed = {
let mut previous = self
.failures
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let changed = failures
.iter()
.filter(|failure| !previous.contains(failure))
.cloned()
.collect::<Vec<_>>();
for recovered in previous
.iter()
.filter(|old| !failures.iter().any(|new| new.component == old.component))
{
tracing::info!(component = %recovered.component, "health component recovered");
}
*previous = failures;
changed
};
// 只记录故障变化;使用既有有界观测队列,文件日志失效时仍有独立的运维证据
for failure in changed {
tracing::warn!(component = %failure.component, state = failure.state, "health check failed");
let mut event = OperationalFailure::new(
"health",
"check",
"health_check_failed",
"Health check failed",
);
event.correlation_id = Some(failure.component.clone());
event.details =
ErrorDetails::capture(Some(&ErrorSource::new(failure.clone())), None, false);
if self.diagnostics.record_failure(event).await.is_err() {
tracing::warn!("health diagnostic could not be recorded");
// 入队失败不能抑制下次探测的记录机会,也不改变本次健康判定
self.failures
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.retain(|previous| previous != &failure);
}
}
healthy
}
}
@@ -201,6 +201,20 @@ pub(super) struct RecordView {
#[derive(Default, Serialize)]
#[serde(rename_all = "camelCase")]
struct OutputTimingView {
#[serde(skip_serializing_if = "Option::is_none")]
upstream_response_ms: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
upstream_api_overhead_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
upstream_engine_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
upstream_engine_iapi_ttft_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
upstream_engine_service_ttft_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
upstream_engine_iapi_tbt_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
upstream_engine_service_tbt_ms: Option<f64>,
#[serde(skip_serializing_if = "Option::is_none")]
first_event_ms: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
@@ -226,6 +240,13 @@ fn success_record(value: UsageListRecord, time: crate::time::TimePresenter) -> R
latency_ms: value.latency_ms,
first_token_latency_ms: value.first_token_ms,
latency_details: OutputTimingView {
upstream_response_ms: value.upstream_response_ms,
upstream_api_overhead_ms: value.upstream_api_overhead_ms,
upstream_engine_ms: value.upstream_engine_ms,
upstream_engine_iapi_ttft_ms: value.upstream_engine_iapi_ttft_ms,
upstream_engine_service_ttft_ms: value.upstream_engine_service_ttft_ms,
upstream_engine_iapi_tbt_ms: value.upstream_engine_iapi_tbt_ms,
upstream_engine_service_tbt_ms: value.upstream_engine_service_tbt_ms,
first_event_ms: value.first_event_ms,
first_reasoning_ms: value.first_reasoning_ms,
first_text_ms: value.first_text_ms,
+1 -1
View File
@@ -190,7 +190,7 @@ pub fn initialize(
let state = ApiState {
admin,
openai: OpenAiService::new(execution, lifecycle),
health: HealthStatus::new(probes, worker_health),
health: HealthStatus::new(probes, worker_health, diagnostics.clone()),
};
let index = config.asset_directory.join("index.html");
let mut router = Router::new()
@@ -95,15 +95,25 @@ impl OpenAiRequestHeaders {
}
}
/// 独立端点只消费会话身份与 turn metadata,不携带 Responses 的连接状态
/// 独立端点保留业务扩展头与会话身份,不携带 Responses 的连接状态
pub(crate) fn session_context(&self) -> Map<String, Value> {
let mut context = Map::new();
insert_protocol_context(&mut context, "session_id", self.session_id.as_ref());
insert_protocol_context(&mut context, "thread_id", self.thread_id.as_ref());
insert_protocol_context(&mut context, "turn_metadata", self.turn_metadata.as_ref());
self.insert_passthrough_headers(&mut context);
context
}
fn insert_passthrough_headers(&self, context: &mut Map<String, Value>) {
if !self.passthrough_headers.is_empty() {
context.insert(
PASSTHROUGH_HEADERS_CONTEXT_KEY.to_owned(),
Value::Array(self.passthrough_headers.clone()),
);
}
}
fn protocol_context(
&self,
use_websocket: Option<bool>,
@@ -153,12 +163,7 @@ impl OpenAiRequestHeaders {
insert_protocol_context(&mut context, "turn_id", self.turn_id.as_ref());
insert_protocol_context(&mut context, "responses_lite", self.responses_lite.as_ref());
insert_protocol_context(&mut context, "memgen_request", self.memgen_request.as_ref());
if !self.passthrough_headers.is_empty() {
context.insert(
PASSTHROUGH_HEADERS_CONTEXT_KEY.to_owned(),
Value::Array(self.passthrough_headers.clone()),
);
}
self.insert_passthrough_headers(&mut context);
if let Some(use_websocket) = use_websocket {
context.insert("use_websocket".to_owned(), Value::Bool(use_websocket));
}
@@ -47,7 +47,7 @@ impl OpenAiResponsesEncoder {
return Vec::new();
};
self.observe_wire(wire);
let projected = client_failure_payload(wire);
let projected = client_payload(wire);
let data = projected.as_ref().unwrap_or_else(|| wire.data());
// 当前 Codex 不消费 Responses `error` event,会在 EOF 时丢失失败原因
// 在客户端 SSE 边界统一投影成它能识别的 `response.failed`
@@ -70,22 +70,29 @@ impl OpenAiResponsesEncoder {
{
return vec![raw_sse_frame.clone()];
}
let data = if projected.is_none() {
wire.raw_websocket_message()
.map(str::to_owned)
.unwrap_or_else(|| data.to_string())
} else {
data.to_string()
};
vec![Bytes::from(encode_sse_event_with_metadata(
effective_event_type(wire).unwrap_or_default(),
&data.to_string(),
&data,
wire.sse_id(),
wire.sse_retry(),
))]
}
/// 消费一个 Provider event,并返回 WebSocket JSON messages
/// 消费一个 Provider event,并返回完整 WebSocket 文本消息
pub fn push_websocket(&mut self, event: &ProviderEvent) -> Vec<String> {
self.observe_canonical_identity(event);
let Some(wire) = openai_wire(event).filter(|wire| wire.has_json_data()) else {
let Some(wire) = openai_wire(event) else {
return Vec::new();
};
self.observe_wire(wire);
let projected = client_failure_payload(wire);
let projected = client_payload(wire);
let data = projected.as_ref().unwrap_or_else(|| wire.data());
// WS 客户端只对它无法消费的裸 `error` 帧做投影:codex 的 WS 端点
// 会静默忽略缺少 status 且不含内置可重试错误码的 `error` 帧,客户
@@ -104,6 +111,14 @@ impl OpenAiResponsesEncoder {
{
return vec![data.to_string()];
}
if projected.is_none()
&& let Some(raw) = wire.raw_websocket_message()
{
return vec![raw.to_owned()];
}
if !wire.has_json_data() {
return Vec::new();
}
vec![data.to_string()]
}
@@ -248,15 +263,43 @@ fn effective_event_type(wire: &ProtocolWireEvent) -> Option<&str> {
.or_else(|| wire.data().get("type").and_then(Value::as_str))
}
fn client_failure_payload(wire: &ProtocolWireEvent) -> Option<Value> {
fn client_payload(wire: &ProtocolWireEvent) -> Option<Value> {
if matches!(
effective_event_type(wire),
Some("error" | "response.failed")
) {
capacity_error_for_client(wire.data())
} else {
None
return capacity_error_for_client(wire.data());
}
if !matches!(
effective_event_type(wire),
Some("response.metadata" | "codex.response.metadata")
) {
return None;
}
let headers = wire.data().get("headers")?.as_object()?;
let connection_options: Vec<String> = headers
.iter()
.filter(|(name, _)| name.eq_ignore_ascii_case("connection"))
.flat_map(|(_, value)| match value {
Value::Array(values) => values.iter().filter_map(Value::as_str).collect::<Vec<_>>(),
_ => value.as_str().into_iter().collect(),
})
.flat_map(|value| value.split(','))
.map(|name| name.trim().to_ascii_lowercase())
.collect();
if headers
.keys()
.all(|name| super::response_header_is_forwardable(name, &connection_options))
{
return None;
}
// 只剥离既有响应头隔离策略禁止的字段,metadata 业务扩展保持不透明
let mut projected = wire.data().clone();
projected
.get_mut("headers")?
.as_object_mut()?
.retain(|name, _| super::response_header_is_forwardable(name, &connection_options));
Some(projected)
}
/// 判断 WS 上游的 `error` 帧是否已经是客户端可直接消费的形状
@@ -209,6 +209,11 @@ impl ResponsesDeliveryValidator {
transformed: bool,
facts: &ResponseValidationFacts,
) -> Result<(), ResponseValidationError> {
// 未改写的未知载荷不参与本地关联判断,不能因旁路解析失败截断原流
// 插件实际改写的载荷仍须满足交付合同
if parsed.is_err() && !transformed {
return Ok(());
}
if !self.active && !transformed {
if parsed.and_then(|frame| self.state.apply(frame)).is_err() {
self.invalid_prefix = true;
@@ -253,7 +258,7 @@ impl ResponsesDeliveryValidator {
return Err(ResponseValidationError);
}
}
ParsedFrame::Event { .. } | ParsedFrame::Done => {}
ParsedFrame::Event { .. } | ParsedFrame::Done | ParsedFrame::NonEvent => {}
}
self.state.apply(parsed)
}
@@ -292,6 +297,7 @@ impl ResponsesDeliveryValidator {
enum ParsedFrame {
Event { event_type: String, value: Value },
Done,
NonEvent,
}
impl ParsedFrame {
@@ -301,6 +307,10 @@ impl ParsedFrame {
return Ok(Self::Done);
}
let mut events = parse_sse_events(frame).map_err(|_| ResponseValidationError)?;
// 注释和仅含 SSE 元数据的帧不承载响应业务事实
if events.is_empty() {
return Ok(Self::NonEvent);
}
if events.len() != 1 {
return Err(ResponseValidationError);
}
@@ -363,6 +373,9 @@ struct DeliveryState {
impl DeliveryState {
fn apply(&mut self, frame: ParsedFrame) -> Result<(), ResponseValidationError> {
if matches!(frame, ParsedFrame::NonEvent) {
return Ok(());
}
if self.done {
return Err(ResponseValidationError);
}
@@ -12,7 +12,7 @@ use gateway_core::engine::middleware::{
MiddlewareBody, MiddlewareError, MiddlewareFrame, MiddlewareFraming, MiddlewareHeader,
MiddlewareResponse,
};
use gateway_core::engine::response_control::ResponseControl;
use gateway_core::engine::response_control::{ResponseControl, ResponseControlUnavailable};
use gateway_core::error::{GatewayError, GatewayErrorKind};
use gateway_core::event::ProviderResponseHeader;
use gateway_core::operation::ProviderSessionState;
@@ -25,7 +25,7 @@ use super::{
super::{DecodedResponsesRequest, OpenAiResponsesEncoder, ProtocolErrorBody},
connection::{ConnectionEvent, FramePhase, ResponsesWebSocketConnection, WriteContext},
protocol::{
decode_response_interrupt, error_event, initial_engine_error_event, response_metadata_event,
error_event, initial_engine_error_event, is_response_create, response_metadata_event,
},
};
@@ -137,7 +137,7 @@ pub(super) async fn forward_response(
replay: &mut ConnectionReplaySnapshot,
capture: ReplayCaptureHandle,
validation: ResponseValidationFacts,
response_control: ResponseControl,
response_control: &ResponseControl,
) -> ForwardOutcome {
let (protocol, status, headers, mut body, _) = response.into_parts();
if protocol != "openai" || StatusCode::from_u16(status).is_err() {
@@ -150,8 +150,14 @@ pub(super) async fn forward_response(
ProviderResponseHeader::new(name, value)
})
.collect::<Vec<_>>();
let first = match next_body_input(connection, body.as_mut(), &response_control, &request_id)
.await
let first = match next_body_input(
connection,
body.as_mut(),
response_control,
&request_id,
false,
)
.await
{
BodyInput::Frame(Ok(Some(frame))) => frame,
BodyInput::Frame(Ok(None)) => {
@@ -197,8 +203,14 @@ pub(super) async fn forward_response(
let frame = match current.take() {
Some(frame) => frame,
None => {
match next_body_input(connection, body.as_mut(), &response_control, &request_id)
.await
match next_body_input(
connection,
body.as_mut(),
response_control,
&request_id,
terminal_seen,
)
.await
{
BodyInput::Frame(Ok(Some(frame))) => frame,
BodyInput::Frame(Ok(None)) if terminal_seen && body.is_finalized() => {
@@ -284,8 +296,10 @@ fn detach_body(body: Box<dyn MiddlewareBody>) {
}
fn validate_frame(frame: &MiddlewareFrame) -> Result<(), MiddlewareError> {
if frame.framing() != MiddlewareFraming::JsonDocument
|| serde_json::from_slice::<serde::de::IgnoredAny>(frame.bytes()).is_err()
if !matches!(
frame.framing(),
MiddlewareFraming::JsonDocument | MiddlewareFraming::RawBytes
) || std::str::from_utf8(frame.bytes()).is_err()
{
return Err(MiddlewareError::InvalidState);
}
@@ -302,6 +316,7 @@ async fn next_body_input(
body: &mut dyn MiddlewareBody,
response_control: &ResponseControl,
request_id: &Arc<str>,
terminal_seen: bool,
) -> BodyInput {
// 处理控制帧时继续持有同一个读取 future,不能取消正在加工正文的中间件
let frame = body.next_frame();
@@ -312,19 +327,11 @@ async fn next_body_input(
let Some(event) = event else { return BodyInput::Disconnect; };
match &event.event {
ConnectionEvent::Text(payload) => {
let error = match decode_response_interrupt(payload) {
Ok(Some(response_id)) => response_control.interrupt(&response_id).err().map(|_| {
super::super::RequestDecodeError::InvalidValue { field: "response_id".to_owned() }.protocol_body()
}),
Ok(None) => {
connection.defer(event);
continue;
}
Err(error) => Some(error.protocol_body()),
};
if let Some(error) = error
&& send_protocol_error(connection, StatusCode::BAD_REQUEST, error, request_id).await == ForwardOutcome::Disconnect
{
if is_response_create(payload) {
connection.defer(event);
continue;
}
if send_control(connection, response_control, payload, request_id).await == ForwardOutcome::Disconnect {
return BodyInput::Disconnect;
}
}
@@ -337,6 +344,11 @@ async fn next_body_input(
ConnectionEvent::Exited(_) => return BodyInput::Disconnect,
}
}
event = response_control.receive(), if terminal_seen => {
if forward_control_event(connection, response_control, event).await == ForwardOutcome::Disconnect {
return BodyInput::Disconnect;
}
}
frame = &mut frame => return BodyInput::Frame(frame),
}
}
@@ -537,19 +549,26 @@ impl WebSocketExecutionBody {
self.transformed_pending = transformed;
continue;
}
messages.extend(encoded.into_iter().map(|message| (message, transformed)));
let framing = if event.wire_event().is_some_and(|wire| {
wire.raw_websocket_message().is_some() && !wire.has_json_data()
}) {
MiddlewareFraming::RawBytes
} else {
MiddlewareFraming::JsonDocument
};
messages.extend(
encoded
.into_iter()
.map(|message| (message, transformed, framing)),
);
self.transformed_pending = false;
}
let terminal = self.encoder.is_completed() || self.encoder.has_wire_failure();
let last = messages.len().saturating_sub(1);
self.pending.extend(messages.into_iter().enumerate().map(
|(index, (message, transformed))| {
MiddlewareFrame::new(
Bytes::from(message),
MiddlewareFraming::JsonDocument,
terminal && index == last,
)
.with_transformed(transformed)
|(index, (message, transformed, framing))| {
MiddlewareFrame::new(Bytes::from(message), framing, terminal && index == last)
.with_transformed(transformed)
},
));
self.awaiting_terminal_eof = terminal;
@@ -720,3 +739,46 @@ async fn send_metadata(
.await
.is_ok()
}
pub(super) async fn send_control(
connection: &mut ResponsesWebSocketConnection,
control: &ResponseControl,
payload: &str,
request_id: &Arc<str>,
) -> ForwardOutcome {
let result = tokio::select! {
_ = connection.wait_for_exit() => return ForwardOutcome::Disconnect,
result = control.send(payload) => result,
};
if result.is_err() {
return send_gateway_error(
connection,
&GatewayError::new(
GatewayErrorKind::InvalidRequest,
"No upstream WebSocket connection is available for control messages",
),
request_id,
)
.await;
}
ForwardOutcome::Continue
}
pub(super) async fn forward_control_event(
connection: &mut ResponsesWebSocketConnection,
control: &ResponseControl,
event: Result<String, ResponseControlUnavailable>,
) -> ForwardOutcome {
let Ok(payload) = event else {
control.clear();
return ForwardOutcome::Continue;
};
if connection
.send_text(payload, WriteContext::connection(FramePhase::Data))
.await
.is_err()
{
return ForwardOutcome::Disconnect;
}
ForwardOutcome::Continue
}
@@ -24,6 +24,7 @@ use gateway_core::{
engine::{
execution::{AuthenticatedClient, ClientTransport},
middleware::{FrozenMiddlewarePlan, MiddlewareError},
response_control::ResponseControl,
},
lifecycle::{ConnectionGuard, ConnectionLifecycle},
operation::OperationKind,
@@ -46,11 +47,12 @@ use super::{
};
use connection::{ConnectionEvent, FramePhase, ResponsesWebSocketConnection, WriteContext};
use forward::{
ConnectionReplaySnapshot, ForwardOutcome, execution_response, forward_response,
new_replay_capture, send_gateway_error, send_middleware_error, send_protocol_error,
ConnectionReplaySnapshot, ForwardOutcome, execution_response, forward_control_event,
forward_response, new_replay_capture, send_control, send_gateway_error, send_middleware_error,
send_protocol_error,
};
use protocol::connection_limit_event;
pub use protocol::{ResponseCreateFrameError, decode_response_create_with_context};
use protocol::{connection_limit_event, is_response_create};
const TEXT_FRAMES_ONLY: &str = "Responses WebSocket accepts text frames only";
const CONNECTION_LIMIT_CLOSE_REASON: &str = "Responses websocket connection limit reached";
@@ -191,10 +193,20 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
);
let mut request_count = 0_u64;
let mut replay = ConnectionReplaySnapshot::default();
let mut response_control = ResponseControl::default();
loop {
let Some(event) = connection.next_event().await else {
break;
let event = tokio::select! {
event = connection.next_event() => {
let Some(event) = event else { break };
event
}
event = response_control.receive() => {
if forward_control_event(&mut connection, &response_control, event).await == ForwardOutcome::Disconnect {
break;
}
continue;
}
};
let payload = match event {
ConnectionEvent::Text(payload) => payload,
@@ -212,8 +224,22 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
expire_connection(&mut connection).await;
break;
}
request_count = request_count.saturating_add(1);
let correlation_id = Arc::<str>::from(service.next_request_id());
if !is_response_create(&payload) {
if send_control(
&mut connection,
&response_control,
&payload,
&correlation_id,
)
.await
== ForwardOutcome::Disconnect
{
break;
}
continue;
}
request_count = request_count.saturating_add(1);
// 初步解码只保留路由事实,避免整份正文跨越准入等待和响应交付
let model_hint = match decode_response_create_with_context(&payload, &request_headers) {
Ok(decoded) => Some(decoded.metadata().requested_model().to_owned()),
@@ -251,6 +277,7 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
expire_connection(&mut connection).await;
break;
}
response_control.clear();
let execution = service.execution();
let preparation = async {
// 握手的显式改写可继承,Key 策略与宿主设置仍在每轮执行前刷新
@@ -291,7 +318,7 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
}
};
let request_id = Arc::<str>::from(prepared.request_id().to_string());
let response_control = prepared.response_control();
response_control = prepared.response_control();
let capture = new_replay_capture();
let validation = ResponseValidationFacts::default();
let input = RequestInput {
@@ -381,7 +408,7 @@ async fn serve_responses_websocket(socket: WebSocket, session: ResponsesWebSocke
&mut replay,
capture,
validation,
response_control,
&response_control,
)
.await
== ForwardOutcome::Disconnect
@@ -31,42 +31,22 @@ pub fn decode_response_create_with_context(
decode_response_create_inner(payload, request_headers)
}
/// 中断只引用活动响应;普通创建帧仍交给原有解码与串行准入路径
pub(super) fn decode_response_interrupt(
payload: &str,
) -> Result<Option<String>, ResponseCreateFrameError> {
/// 只识别需要本地准入的创建帧,其余文本不在网关解释
pub(super) fn is_response_create(payload: &str) -> bool {
#[derive(serde::Deserialize)]
struct FrameType {
#[serde(rename = "type")]
message_type: Option<String>,
message_type: String,
}
// 非控制帧仍在原来的串行请求边界校验,不能抢先拒绝排队的普通请求
// 只投影类型,避免为排队的大型 response.create 再构造完整 JSON 树
let Ok(frame) = serde_json::from_str::<FrameType>(payload) else {
return Ok(None);
};
if frame.message_type.as_deref() != Some("response.interrupt") {
return Ok(None);
match serde_json::from_str::<FrameType>(payload) {
Ok(frame) => frame.message_type == "response.create",
// struct 投影拒绝重复 type;沿用创建解码器的 JSON object 语义,
// 避免这类创建请求绕过准入进入透传路径
Err(_) => serde_json::from_str::<Value>(payload).is_ok_and(|frame| {
frame.get("type").and_then(Value::as_str) == Some("response.create")
}),
}
let value: Value =
serde_json::from_str(payload).map_err(|_| ResponseCreateFrameError::InvalidJson)?;
if value.get("mode").and_then(Value::as_str) != Some("discard_partial_items") {
return Err(ResponseCreateFrameError::Request(
RequestDecodeError::InvalidValue {
field: "mode".to_owned(),
},
));
}
let id = value
.get("response_id")
.and_then(Value::as_str)
.filter(|id| !id.is_empty())
.ok_or_else(|| {
ResponseCreateFrameError::Request(RequestDecodeError::InvalidValue {
field: "response_id".to_owned(),
})
})?;
Ok(Some(id.to_owned()))
}
fn decode_response_create_inner(
@@ -49,6 +49,36 @@ async fn list_route_should_keep_camel_case_group_and_page_wire() {
assert!(value["data"]["items"][0].get("member_count").is_none());
}
#[tokio::test]
async fn options_route_should_return_only_lightweight_group_fields() {
let fixture = authenticated_fixture().await;
let response = request(
router(&fixture),
Method::GET,
"/api/admin/account-groups/options?page=1&pageSize=1&search=alpha&enabled=true",
None,
true,
)
.await;
assert_eq!(response.status(), StatusCode::OK);
let value = response_json(response).await;
assert_eq!(
value["data"]["items"][0],
json!({
"id": PRIMARY_GROUP_ID,
"name": "Alpha routing",
"color": "#2563EBFF",
"enabled": true
})
);
assert_eq!(value["data"]["page"]["page"], 1);
assert_eq!(value["data"]["page"]["pageSize"], 1);
assert_eq!(value["data"]["page"]["total"], 1);
assert_eq!(value["data"]["page"]["totalPages"], 1);
assert_eq!(value["data"]["configRevision"], 7);
}
#[tokio::test]
async fn create_route_should_create_an_empty_group() {
let fixture = authenticated_fixture().await;
@@ -163,6 +193,11 @@ async fn list_route_should_reject_unknown_and_invalid_pagination_fields() {
"/api/admin/account-groups?page=0",
"/api/admin/account-groups?pageSize=0",
"/api/admin/account-groups?pageSize=201",
"/api/admin/account-groups/options?other=true",
"/api/admin/account-groups/options?page=0",
"/api/admin/account-groups/options?pageSize=0",
"/api/admin/account-groups/options?pageSize=201",
"/api/admin/account-groups/options?search=%00",
] {
let response = request(router(&fixture), Method::GET, uri, None, true).await;
assert_eq!(response.status(), StatusCode::BAD_REQUEST, "{uri}");
@@ -219,6 +254,14 @@ async fn account_group_routes_should_require_admin_authentication() {
false,
)
.await;
let options_response = request(
router(&fixture),
Method::GET,
"/api/admin/account-groups/options",
None,
false,
)
.await;
let post_response = request(
router(&fixture),
Method::POST,
@@ -229,9 +272,29 @@ async fn account_group_routes_should_require_admin_authentication() {
.await;
assert_eq!(get_response.status(), StatusCode::UNAUTHORIZED);
assert_eq!(options_response.status(), StatusCode::UNAUTHORIZED);
assert_eq!(post_response.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn options_route_should_reject_client_key_identity() {
let fixture = crate::support::key_fixture().await;
let response = router(&fixture)
.oneshot(
Request::builder()
.method(Method::GET)
.uri("/api/admin/account-groups/options")
.header("x-api-key", crate::support::RAW_KEY)
.header("x-request-id", "req_account_group_options_client_key")
.body(Body::empty())
.expect("client key options request"),
)
.await
.expect("client key options response");
assert_eq!(response.status(), StatusCode::UNAUTHORIZED);
}
async fn authenticated_fixture() -> AdminTestFixture {
let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session");
+46 -3
View File
@@ -18,9 +18,10 @@ use gateway_admin::{
MutationContext, Revision,
account_groups::{
AccountGroupAccountSummary, AccountGroupCapacity, AccountGroupColor,
AccountGroupListQuery, AccountGroupMemberFact, AccountGroupMutation, AccountGroupPage,
AccountGroupRecord, AccountGroupUsage, DeleteAccountGroup, NewAccountGroup,
SetAccountGroupEnabled, UpdateAccountGroup,
AccountGroupListQuery, AccountGroupMemberFact, AccountGroupMutation,
AccountGroupOptionsPage, AccountGroupPage, AccountGroupRecord, AccountGroupRef,
AccountGroupUsage, DeleteAccountGroup, NewAccountGroup, SetAccountGroupEnabled,
UpdateAccountGroup,
},
accounts::{
AccountListQuery, AccountPage, AccountPageItem, AccountRuntimeSnapshot,
@@ -789,6 +790,48 @@ impl AccountGroupStore for MemoryAccountGroupStore {
})
}
async fn list_account_group_options(
&self,
query: AccountGroupListQuery,
) -> AdminStoreResult<AccountGroupOptionsPage> {
let state = self.state.lock().expect("account groups");
let search = query.search.as_ref().map(|value| value.to_lowercase());
let mut matching = state
.groups
.values()
.filter(|record| {
query
.enabled
.is_none_or(|enabled| record.enabled == enabled)
})
.filter(|record| {
search
.as_ref()
.is_none_or(|search| record.name.to_lowercase().contains(search))
})
.map(|record| AccountGroupRef {
id: record.id.clone(),
name: record.name.clone(),
color: record.color.clone(),
enabled: record.enabled,
})
.collect::<Vec<_>>();
matching.sort_by(|left, right| left.name.cmp(&right.name));
let total = matching.len() as u64;
let page_size = usize::from(query.page_size.get());
let offset = usize::try_from(query.page.saturating_sub(1))
.unwrap_or(usize::MAX)
.saturating_mul(page_size);
let items = matching.into_iter().skip(offset).take(page_size).collect();
Ok(AccountGroupOptionsPage {
config_revision: state.revision,
items,
total,
page: query.page,
page_size: query.page_size.get(),
})
}
async fn load_account_group_members(
&self,
group_ids: &[gateway_core::routing::AccountGroupId],
@@ -233,6 +233,22 @@ async fn usage_detail_should_keep_attempt_snapshot_contract() {
let fixture = AdminTestFixture::new().await;
fixture.auth.insert_session("valid-session");
let now = Utc::now();
let mut record = usage_record_with_account(
"req_detail",
"acct_snap_a",
"Snapshot Alpha",
"alpha@example.invalid",
"oauth",
now,
);
record.first_token_ms = Some(20);
record.upstream_response_ms = Some(1_000);
record.upstream_api_overhead_ms = Some(120.25);
record.upstream_engine_ms = Some(6400.0);
record.upstream_engine_iapi_ttft_ms = Some(650.5);
record.upstream_engine_service_ttft_ms = Some(720.25);
record.upstream_engine_iapi_tbt_ms = Some(18.45);
record.upstream_engine_service_tbt_ms = Some(20.12);
fixture
.usage_detail
.lock()
@@ -240,14 +256,7 @@ async fn usage_detail_should_keep_attempt_snapshot_contract() {
.replace(UsageDetail {
trace: None,
related_requests: Vec::new(),
request: usage_record_with_account(
"req_detail",
"acct_snap_a",
"Snapshot Alpha",
"alpha@example.invalid",
"oauth",
now,
),
request: record,
attempts: vec![UsageAttempt {
source: "ops_event".to_owned(),
id: "ops_detail".to_owned(),
@@ -300,6 +309,39 @@ async fn usage_detail_should_keep_attempt_snapshot_contract() {
.await
.expect("usage detail body");
let value: serde_json::Value = serde_json::from_slice(&body).expect("usage detail JSON");
assert_eq!(value["data"]["firstTokenLatencyMs"], 20);
assert_eq!(value["data"]["latencyDetails"]["upstreamResponseMs"], 1_000);
assert_eq!(
value["data"]["latencyDetails"]["upstreamApiOverheadMs"],
json!(120.25)
);
assert_eq!(
value["data"]["latencyDetails"]["upstreamEngineMs"],
json!(6400.0)
);
assert_eq!(
value["data"]["latencyDetails"]["upstreamEngineIapiTtftMs"],
json!(650.5)
);
assert_eq!(
value["data"]["latencyDetails"]["upstreamEngineServiceTtftMs"],
json!(720.25)
);
assert_eq!(
value["data"]["latencyDetails"]["upstreamEngineIapiTbtMs"],
json!(18.45)
);
assert_eq!(
value["data"]["latencyDetails"]["upstreamEngineServiceTbtMs"],
json!(20.12)
);
assert!(value["data"].get("firstTokenMs").is_none());
assert!(value["data"].get("firstTokenLatencyMsDisplay").is_none());
assert!(
value["data"]["latencyDetails"]
.get("firstTokenMs")
.is_none()
);
assert_eq!(
serde_json::json!({
"accountId": value["data"]["accountId"],
@@ -702,6 +744,13 @@ fn usage_record_with_account(
first_text_ms: None,
first_token_ms: None,
provider_processing_ms: None,
upstream_response_ms: None,
upstream_api_overhead_ms: None,
upstream_engine_ms: None,
upstream_engine_iapi_ttft_ms: None,
upstream_engine_service_ttft_ms: None,
upstream_engine_iapi_tbt_ms: None,
upstream_engine_service_tbt_ms: None,
latency_ms: None,
admission_decision_ms: None,
account_selection_wait_ms: None,
@@ -834,6 +883,13 @@ async fn usage_route_should_expose_table_facts_without_detail_payload() {
first_text_ms: Some(19),
first_token_ms: Some(20),
provider_processing_ms: Some(21),
upstream_response_ms: Some(2_000),
upstream_api_overhead_ms: None,
upstream_engine_ms: None,
upstream_engine_iapi_ttft_ms: None,
upstream_engine_service_ttft_ms: None,
upstream_engine_iapi_tbt_ms: None,
upstream_engine_service_tbt_ms: None,
latency_ms: Some(31),
admission_decision_ms: Some(1),
account_selection_wait_ms: Some(2),
@@ -970,8 +1026,8 @@ async fn usage_route_should_expose_table_facts_without_detail_payload() {
"firstEventMs": 17,
"firstReasoningMs": 18,
"firstTextMs": 19,
"firstTokenMs": 20,
"openaiProcessingMs": 21,
"upstreamResponseMs": 2000,
},
"firstTokenLatencyMs": 20,
"latencyMs": 31,
+215 -2
View File
@@ -1,6 +1,6 @@
//! 验证健康接口按基础设施与关键 Worker 状态判定可用性
use std::sync::Arc;
use std::sync::{Arc, Mutex};
use axum::{
body::Body,
@@ -13,12 +13,17 @@ use gateway_core::{
StartProviderExecution, StartedExecution,
},
error::GatewayError,
health::{WorkerHealthKey, WorkerHealthSnapshot, WorkerHealthSource, WorkerRuntimeState},
health::{
HealthProbe, HealthState, WorkerHealthKey, WorkerHealthSnapshot, WorkerHealthSource,
WorkerRuntimeState,
},
routing::PublicModelId,
task::{WorkerId, WorkerKind},
};
use tower::ServiceExt;
use crate::support::RecordingDiagnostics;
#[tokio::test]
async fn healthz_should_return_no_content_when_all_inputs_are_healthy() {
let response = crate::openai::api_router(Arc::new(UnusedExecution))
@@ -81,6 +86,214 @@ async fn healthz_should_reject_critical_worker_failure() {
assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE);
}
#[tokio::test]
async fn healthz_records_probe_changes_once_and_records_recurrence_after_recovery() {
let probe = Arc::new(MutableProbe(Mutex::new(HealthState::Unhealthy(
"file logging rejected records".to_owned(),
))));
let diagnostics = Arc::new(RecordingDiagnostics::default());
let router = health_router(vec![probe.clone()], Vec::new(), diagnostics.clone()).await;
let responses =
futures::future::join_all((0..8).map(|_| health_response(router.clone()))).await;
for response in responses {
assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE);
assert!(
axum::body::to_bytes(response.into_body(), 1024)
.await
.unwrap()
.is_empty()
);
}
{
let events = diagnostics.0.lock().unwrap();
assert_eq!(events.len(), 1);
assert_eq!(events[0].component, "health");
assert_eq!(
events[0].correlation_id.as_deref(),
Some("probe:file_logging")
);
assert_eq!(events[0].message, "Health check failed");
assert!(
events[0]
.details
.as_ref()
.unwrap()
.as_str()
.contains("file logging rejected records")
);
}
*probe.0.lock().unwrap() = HealthState::Degraded("file logging maintenance failed".to_owned());
assert_eq!(
health_response(router.clone()).await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert_eq!(diagnostics.0.lock().unwrap().len(), 2);
*probe.0.lock().unwrap() = HealthState::Healthy;
assert_eq!(
health_response(router.clone()).await.status(),
StatusCode::NO_CONTENT
);
*probe.0.lock().unwrap() = HealthState::Degraded("file logging maintenance failed".to_owned());
assert_eq!(
health_response(router).await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert_eq!(diagnostics.0.lock().unwrap().len(), 3);
}
#[tokio::test(start_paused = true)]
async fn healthz_retains_worker_and_completed_probe_failures_when_another_probe_times_out() {
let diagnostics = Arc::new(RecordingDiagnostics::default());
let router = health_router(
vec![
Arc::new(MutableProbe(Mutex::new(HealthState::Unhealthy(
"file output failed".to_owned(),
)))),
Arc::new(PendingProbe),
],
vec![
worker_snapshot(
WorkerKind::RuntimeSnapshotReconciliation,
WorkerRuntimeState::BackingOff,
),
worker_snapshot(
WorkerKind::QuotaCatalogHealth,
WorkerRuntimeState::BackingOff,
),
],
diagnostics.clone(),
)
.await;
assert_eq!(
health_response(router).await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
let events = diagnostics.0.lock().unwrap();
assert_eq!(events.len(), 3);
for (component, reason) in [
(
"worker:runtime_snapshot_reconciliation:health-test",
"test worker failure",
),
("probe:file_logging", "file output failed"),
("probe:pending", "timed_out"),
] {
let event = events
.iter()
.find(|event| event.correlation_id.as_deref() == Some(component))
.expect(component);
assert!(event.details.as_ref().unwrap().as_str().contains(reason));
}
}
#[tokio::test]
async fn healthz_retries_failed_diagnostic_admission_without_changing_health_result() {
let diagnostics = Arc::new(FailingOnceDiagnostics::default());
let router = health_router(
vec![Arc::new(MutableProbe(Mutex::new(HealthState::Unhealthy(
"file output failed".to_owned(),
))))],
Vec::new(),
diagnostics.clone(),
)
.await;
assert_eq!(
health_response(router.clone()).await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert!(diagnostics.recorded.0.lock().unwrap().is_empty());
assert_eq!(
health_response(router).await.status(),
StatusCode::SERVICE_UNAVAILABLE
);
assert_eq!(diagnostics.recorded.0.lock().unwrap().len(), 1);
}
#[derive(Default)]
struct FailingOnceDiagnostics {
failed: std::sync::atomic::AtomicBool,
recorded: RecordingDiagnostics,
}
#[async_trait::async_trait]
impl gateway_core::diagnostics::OperationalDiagnostics for FailingOnceDiagnostics {
async fn record_failure(
&self,
failure: gateway_core::diagnostics::OperationalFailure,
) -> Result<(), gateway_core::error::StoreError> {
if !self.failed.swap(true, std::sync::atomic::Ordering::SeqCst) {
return Err(gateway_core::error::StoreError::new(
gateway_core::error::StoreErrorKind::Unavailable,
));
}
self.recorded.record_failure(failure).await
}
}
async fn health_router(
probes: Vec<Arc<dyn HealthProbe>>,
workers: Vec<WorkerHealthSnapshot>,
diagnostics: Arc<dyn gateway_core::diagnostics::OperationalDiagnostics>,
) -> axum::Router {
let admin = crate::admin::AdminTestFixture::new().await;
gateway_api::initialize(
gateway_api::ApiConfig {
asset_directory: std::env::temp_dir(),
cors_allowed_origins: Vec::new(),
request_timeout_seconds: None,
request_id_header: "x-request-id".to_owned(),
},
Arc::new(UnusedExecution),
admin.services,
probes,
Arc::new(StaticWorkerHealth(workers)),
Arc::new(crate::openai::TestLifecycle::default()),
diagnostics,
)
.unwrap()
.router()
}
async fn health_response(router: axum::Router) -> axum::response::Response {
router
.oneshot(
Request::builder()
.uri("/healthz")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap()
}
struct MutableProbe(Mutex<HealthState>);
impl HealthProbe for MutableProbe {
fn name(&self) -> &'static str {
"file_logging"
}
fn check(&self) -> BoxFuture<'_, HealthState> {
Box::pin(std::future::ready(self.0.lock().unwrap().clone()))
}
}
struct PendingProbe;
impl HealthProbe for PendingProbe {
fn name(&self) -> &'static str {
"pending"
}
fn check(&self) -> BoxFuture<'_, HealthState> {
Box::pin(std::future::pending())
}
}
#[derive(Clone)]
struct StaticWorkerHealth(Vec<WorkerHealthSnapshot>);
@@ -135,6 +135,13 @@ fn usage_record() -> UsageListRecord {
first_text_ms: Some(600),
first_token_ms: Some(210),
provider_processing_ms: None,
upstream_response_ms: Some(1_000),
upstream_api_overhead_ms: None,
upstream_engine_ms: None,
upstream_engine_iapi_ttft_ms: None,
upstream_engine_service_ttft_ms: None,
upstream_engine_iapi_tbt_ms: Some(2.450638),
upstream_engine_service_tbt_ms: None,
latency_ms: Some(1250),
admission_decision_ms: None,
account_selection_wait_ms: None,
@@ -305,9 +305,20 @@ async fn records_keep_pagination_and_hide_admin_and_upstream_data() {
assert_eq!(record["firstTokenLatencyMs"], 210);
assert_fields(
&record["latencyDetails"],
&["firstEventMs", "firstReasoningMs", "firstTextMs"],
&[
"upstreamResponseMs",
"upstreamEngineIapiTbtMs",
"firstEventMs",
"firstReasoningMs",
"firstTextMs",
],
);
assert_eq!(record["latencyDetails"]["firstEventMs"], 100);
assert_eq!(record["latencyDetails"]["upstreamResponseMs"], 1000);
assert_eq!(
record["latencyDetails"]["upstreamEngineIapiTbtMs"],
2.450638
);
assert!(record["statusCode"].is_null());
} else {
assert_eq!(record["statusCode"], 502);
@@ -10,6 +10,7 @@ use axum::{
body::{Body, to_bytes},
http::{Request, StatusCode, header::AUTHORIZATION},
};
use base64::{Engine as _, engine::general_purpose::STANDARD};
use bytes::Bytes;
use futures::future::BoxFuture;
use gateway_core::engine::execution::{
@@ -300,6 +301,11 @@ async fn image_routes_should_not_decode_bodies_and_should_preserve_both_directio
.header("x-codex-image-turn-id", "turn_image_route")
.header("session-id", "root-image-session")
.header("thread-id", "child-image-thread")
.header("x-future-business", "first")
.header("x-future-business", "second")
.header("cookie", "client-secret")
.header("connection", "x-private-hop")
.header("x-private-hop", "client-hop")
.body(Body::from(body.to_vec()))
.expect("image request"),
)
@@ -338,8 +344,37 @@ async fn image_routes_should_not_decode_bodies_and_should_preserve_both_directio
assert_eq!(captured.transport, ClientTransport::HttpJson);
assert_eq!(captured.kind, *kind);
assert_eq!(captured.body.as_ref(), *body);
let mut context = captured.context.clone();
let headers = context
.as_object_mut()
.unwrap()
.remove("opaque_request_headers")
.unwrap();
let headers: Vec<_> = headers
.as_array()
.unwrap()
.iter()
.map(|entry| {
(
entry[0].as_str().unwrap(),
STANDARD.decode(entry[1].as_str().unwrap()).unwrap(),
)
})
.collect();
assert_eq!(
captured.context,
headers
.iter()
.filter(|(name, _)| *name == "x-future-business")
.map(|(_, value)| value.as_slice())
.collect::<Vec<_>>(),
[b"first".as_slice(), b"second".as_slice()]
);
assert!(!headers.iter().any(|(name, _)| matches!(
*name,
"authorization" | "cookie" | "connection" | "x-private-hop"
)));
assert_eq!(
context,
json!({"image_turn_id": "turn_image_route", "session_id": "root-image-session", "thread_id": "child-image-thread"})
);
}
@@ -323,7 +323,7 @@ fn snapshot_with_client_key(
}
#[derive(Default)]
struct TestLifecycle {
pub(crate) struct TestLifecycle {
cancellation: gateway_core::lifecycle::CancellationToken,
}
@@ -2279,6 +2279,7 @@ async fn buffered_response_commits_only_after_complete_json_is_encoded() {
async fn buffered_response_collects_completed_output_items_without_rewriting_streams() {
let message = json!({
"id": "msg_test", "type": "message", "role": "assistant", "status": "completed",
"phase": "partial_answer",
"content": [{"type": "output_text", "text": "完整文本", "annotations": []}],
"future_item_field": {"keep": true}
});
@@ -2288,6 +2289,7 @@ async fn buffered_response_collects_completed_output_items_without_rewriting_str
});
let terminal = json!({
"id": "resp_test", "status": "completed", "output": [],
"end_turn": false,
"usage": {"input_tokens": 20, "output_tokens": 8, "total_tokens": 28},
"future_terminal_field": {"keep": true}
});
@@ -3020,3 +3022,67 @@ async fn compressed_http_request_above_default_limit_should_reach_execution_afte
}
}
}
#[tokio::test]
async fn streaming_middleware_preserves_heartbeats_before_and_after_rewriting() {
for opaque in [
b": ping\n\n".as_slice(),
b"id: next\nretry: 1000\n\n".as_slice(),
b"event: vendor.future\ndata: future non-JSON payload\n\n".as_slice(),
] {
let admin = crate::admin::AdminTestFixture::new().await.services;
let trace = Arc::new(Trace::default());
let heartbeat = Bytes::copy_from_slice(opaque);
let future = Bytes::from_static(
b"event: response.future\ndata: {\"type\":\"response.future\",\"extension\":true}\n\n",
);
let session = FakeSession::streaming(
Arc::clone(&trace),
vec![
NextStep::Event(delivery(started(), CommitRequirement::CommitBeforeDelivery)),
NextStep::Event(delivery_provider(
ProviderEvent::wire(
ProtocolWireEvent::raw_sse("openai", heartbeat.clone()).unwrap(),
),
CommitRequirement::AlreadyCommitted,
)),
NextStep::Event(delivery_provider(
ProviderEvent::wire(
ProtocolWireEvent::raw_sse("openai", future.clone()).unwrap(),
),
CommitRequirement::AlreadyCommitted,
)),
NextStep::Event(delivery_provider(
ProviderEvent::wire(
ProtocolWireEvent::raw_sse("openai", heartbeat.clone()).unwrap(),
),
CommitRequirement::AlreadyCommitted,
)),
NextStep::Event(delivery(completed(), CommitRequirement::AlreadyCommitted)),
NextStep::FinalizeSuccess,
],
);
let replacement = Bytes::from_static(b"event: response.future\ndata: {\"type\":\"response.future\",\"extension\":\"changed\"}\n\n");
let middleware = Arc::new(responses_middleware().with_response_actions(vec![
ResponseFrameAction::Continue,
ResponseFrameAction::Continue,
ResponseFrameAction::Replace(replacement.clone()),
ResponseFrameAction::Continue,
ResponseFrameAction::Continue,
ResponseFrameAction::Continue,
]));
let response =
response_with_middleware(&admin, "openai", session, true, Some(middleware)).await;
let body = to_bytes(response.into_body(), usize::MAX).await.unwrap();
let text = std::str::from_utf8(&body).unwrap();
assert_eq!(
text.matches(std::str::from_utf8(opaque).unwrap()).count(),
2
);
assert!(text.contains(std::str::from_utf8(&replacement).unwrap()));
assert!(text.contains("response.completed"));
assert!(!text.contains("middleware_failed"));
assert!(text.ends_with("data: [DONE]\n\n"));
assert!(!trace.is_cancelled());
}
}
@@ -1071,3 +1071,35 @@ fn transparent_encoder_should_only_require_wire_terminal_for_buffered_conversion
ResponseEncodeError::MissingWireTerminal
);
}
#[test]
fn metadata_keeps_business_extensions_and_filters_only_protected_headers() {
let data = json!({
"type":"response.metadata",
"headers":{"x-future-header":["first","second"],"authorization":"private","set-cookie":"private","chatgpt-account-id":"private","connection":"x-private-hop","x-private-hop":"private"},
"metadata":{"type":"safety_buffering","use_cases":["cyber"],"reasons":["user_risk"],"openai_verification_recommendation":{"future":true},"openai_chatgpt_moderation_metadata":{"future":true}},
"future_extension":{"keep":true}
});
let raw = data.to_string();
let event = ProviderEvent::wire(
ProtocolWireEvent::json("openai", Some("response.metadata".to_owned()), data.clone())
.unwrap()
.with_raw_websocket_message(raw.as_str()),
);
let mut encoder = OpenAiResponsesEncoder::new();
let ws = encoder.push_websocket(&event);
let delivered: Value = serde_json::from_str(&ws[0]).unwrap();
assert_eq!(delivered["metadata"], data["metadata"]);
assert_eq!(delivered["future_extension"], data["future_extension"]);
assert_eq!(
delivered["headers"],
json!({"x-future-header":["first","second"]})
);
let sse = encoder.push_sse(&event);
assert!(!std::str::from_utf8(&sse[0]).unwrap().contains("private"));
assert!(
std::str::from_utf8(&sse[0])
.unwrap()
.contains("safety_buffering")
);
}
@@ -106,10 +106,18 @@ async fn next_error(socket: &mut TestSocket) -> Value {
}
async fn assert_no_duplicate_failure(socket: &mut TestSocket) {
// 后续入站帧只会在本轮收敛后处理,比固定等待窗口更可靠地检查没有重复失败终态
socket.send(Message::Text("{".into())).await.unwrap();
// 无效创建帧仍走串行队列,以本地业务校验作为本轮收敛后的屏障
socket
.send(Message::Text(
json!({"type":"response.create","model":"model-a","stream":false})
.to_string()
.into(),
))
.await
.unwrap();
let error = next_event(socket).await;
assert_eq!(error["error"]["code"], "invalid_json");
assert_eq!(error["error"]["code"], "invalid_value");
assert_eq!(error["error"]["param"], "stream");
assert_eq!(error["status"], 400);
assert_eq!(error["headers"]["x-request-id"], error["request_id"]);
assert_eq!(
@@ -1,12 +1,70 @@
//! 验证 Responses WebSocket 中断向当前执行传递并释放相关资源
//! 验证未知控制消息、迟到中断、结算期间收发和创建请求的串行准入
use super::*;
use gateway_core::engine::response_control::{ResponseControl, ResponseInterruptError};
use gateway_core::engine::response_control::{
ResponseControlTransport, ResponseControlUnavailable,
};
struct ControlTransport {
id: String,
active: std::sync::atomic::AtomicBool,
requests: Mutex<Vec<String>>,
sender: tokio::sync::mpsc::Sender<Value>,
receiver: tokio::sync::Mutex<tokio::sync::mpsc::Receiver<Value>>,
}
impl ControlTransport {
fn new(id: String) -> Self {
let (sender, receiver) = tokio::sync::mpsc::channel(32);
Self {
id,
active: std::sync::atomic::AtomicBool::new(true),
requests: Mutex::default(),
sender,
receiver: tokio::sync::Mutex::new(receiver),
}
}
}
#[async_trait]
impl ResponseControlTransport for ControlTransport {
async fn send(&self, payload: &str) -> Result<(), ResponseControlUnavailable> {
self.requests.lock().unwrap().push(payload.to_owned());
let original: Value =
serde_json::from_str(payload).unwrap_or_else(|_| Value::String(payload.to_owned()));
let event = if original["type"] == "future.reject" {
upstream_control_error()
} else if original["type"] == "response.interrupt"
&& original["response_id"] == self.id
&& original["mode"] == "discard_partial_items"
&& self.active.load(Ordering::SeqCst)
{
json!({"type":"response.incomplete", "response":{"id":self.id,"model":"model-a","status":"incomplete","output":[],"incomplete_details":{"reason":"interrupted"}}})
} else {
json!({"type":"future.control.ack", "original":original})
};
self.sender
.send(event)
.await
.map_err(|_| ResponseControlUnavailable)
}
async fn receive(&self) -> Result<String, ResponseControlUnavailable> {
self.receiver
.lock()
.await
.recv()
.await
.map(|event| event.to_string())
.ok_or(ResponseControlUnavailable)
}
}
#[derive(Default)]
struct InterruptProvider {
calls: AtomicUsize,
controls: Mutex<Vec<ResponseControl>>,
transport: Mutex<Option<Arc<ControlTransport>>>,
complete: tokio::sync::Notify,
initial_messages: Mutex<std::collections::VecDeque<ProtocolWireEvent>>,
}
#[async_trait]
@@ -30,12 +88,10 @@ impl Provider for InterruptProvider {
) -> Result<ProviderStream, ProviderError> {
let number = self.calls.fetch_add(1, Ordering::SeqCst);
let id = format!("resp_interrupt_{number}");
let control = context
.response_control()
.expect("root control propagated through Core")
.clone();
let active = control.activate(id.clone()).unwrap();
self.controls.lock().unwrap().push(control);
let transport = Arc::new(ControlTransport::new(id));
let owner: Arc<dyn ResponseControlTransport> = transport.clone();
context.response_control().unwrap().bind(&owner);
*self.transport.lock().unwrap() = Some(transport.clone());
let candidate = request.candidate();
let metadata = ProviderCallMetadata::new(
candidate.provider().clone(),
@@ -44,41 +100,55 @@ impl Provider for InterruptProvider {
UpstreamTransport::new("websocket").unwrap(),
);
let body = futures::stream::unfold(
(0, Some(active), id),
|(phase, mut active, id)| async move {
let (event_type, status, event) = match phase {
(0, transport, self),
|(phase, transport, provider)| async move {
let pending = if phase == 1 {
provider.initial_messages.lock().unwrap().pop_front()
} else {
None
};
if let Some(wire) = pending {
return Some((Ok(ProviderEvent::wire(wire)), (phase, transport, provider)));
}
let (wire, events, next_phase) = match phase {
0 => (
"response.created",
"in_progress",
GatewayEvent::Started(ResponseMeta::new(&id, "model-a")),
json!({"type":"response.created","response":{"id":transport.id,"model":"model-a","status":"in_progress","output":[]}}),
vec![GatewayEvent::Started(ResponseMeta::new(
&transport.id,
"model-a",
))],
1,
),
1 => {
active.as_ref().unwrap().requested().await;
drop(active.take());
(
"response.incomplete",
"incomplete",
GatewayEvent::Completed(ResponseMeta::new(&id, "model-a")),
)
let wire = tokio::select! {
reply = transport.receive() => serde_json::from_str::<Value>(&reply.unwrap()).unwrap(),
() = provider.complete.notified() => json!({"type":"response.completed","response":{"id":transport.id,"model":"model-a","status":"completed","output":[]}}),
};
if matches!(
wire["type"].as_str(),
Some("response.completed" | "response.incomplete")
) {
transport.active.store(false, Ordering::SeqCst);
(
wire,
vec![GatewayEvent::Completed(ResponseMeta::new(
&transport.id,
"model-a",
))],
2,
)
} else {
(wire, Vec::new(), 1)
}
}
_ => return None,
};
let mut response = json!({"id":id,"model":"model-a","status":status,"output":[]});
if phase == 1 {
response["incomplete_details"] = json!({"reason":"interrupted"});
}
let event_type = wire["type"].as_str().unwrap().to_owned();
let event = ProviderEvent::canonical_with_wire(
vec![event],
ProtocolWireEvent::json(
"openai",
Some(event_type.to_owned()),
json!({
"type":event_type, "response":response
}),
)
.unwrap(),
events,
ProtocolWireEvent::json("openai", Some(event_type), wire).unwrap(),
);
Some((Ok(event), (phase + 1, active, id)))
Some((Ok(event), (next_phase, transport, provider)))
},
);
Ok(ProviderStream::new(metadata, body, ()))
@@ -88,6 +158,8 @@ impl Provider for InterruptProvider {
#[derive(Default)]
struct InterruptAdmissions {
active: AtomicUsize,
release_started: tokio::sync::Notify,
release_gate: Mutex<Option<tokio::sync::oneshot::Receiver<()>>>,
}
impl ClientAdmissionPort for InterruptAdmissions {
@@ -109,6 +181,11 @@ impl ClientAdmissionPort for InterruptAdmissions {
_: &'a ModelRequestId,
) -> BoxFuture<'a, Result<bool, ClientAdmissionError>> {
Box::pin(async {
let gate = self.release_gate.lock().unwrap().take();
self.release_started.notify_one();
if let Some(gate) = gate {
gate.await.unwrap();
}
self.active.fetch_sub(1, Ordering::SeqCst);
Ok(true)
})
@@ -151,15 +228,15 @@ async fn next(socket: &mut TestSocket) -> Value {
}
}
#[tokio::test]
async fn interrupt_is_live_scoped_and_keeps_queued_creates_serial() {
let provider = Arc::new(InterruptProvider::default());
let admissions = Arc::new(InterruptAdmissions::default());
async fn connect(
provider: Arc<InterruptProvider>,
admissions: Arc<InterruptAdmissions>,
) -> (TestSocket, Server) {
let execution = Arc::new(DefaultExecutionService::new(
RuntimeSnapshotHandle::new(snapshot("sk_ws_interrupt", "openai")),
Arc::new(SettlementPorts::default()),
ProviderRegistry::new([provider.clone() as Arc<dyn Provider>]).unwrap(),
admissions.clone(),
ProviderRegistry::new([provider as Arc<dyn Provider>]).unwrap(),
admissions,
Arc::new(UnusedContinuation),
Arc::new(IgnoredClientApiKeyUsage),
Arc::new(crate::support::RecordingDiagnostics::default()),
@@ -167,7 +244,7 @@ async fn interrupt_is_live_scoped_and_keeps_queued_creates_serial() {
let app = api_router(execution).await;
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let address = listener.local_addr().unwrap();
let _server = Server(tokio::spawn(async move {
let server = Server(tokio::spawn(async move {
axum::serve(listener, app).await.unwrap()
}));
let mut request = format!("ws://{address}/v1/responses")
@@ -177,29 +254,136 @@ async fn interrupt_is_live_scoped_and_keeps_queued_creates_serial() {
AUTHORIZATION,
HeaderValue::from_static("Bearer sk_ws_interrupt"),
);
let (mut socket, _) = connect_async(request).await.unwrap();
let (socket, _) = connect_async(request).await.unwrap();
(socket, server)
}
fn upstream_control_error() -> Value {
json!({"type":"error", "status":400, "error":{"type":"invalid_request_error","code":"future_rejected","message":"Synthetic upstream rejection","param":"type"},"vendor_extension":true})
}
fn interrupt(id: &str, mode: &str) -> Value {
json!({"type":"response.interrupt","response_id":id,"mode":mode,"future_extension":true})
}
#[tokio::test]
async fn late_interrupt_after_terminal_is_sent_upstream_and_keeps_the_connection_reusable() {
let provider = Arc::new(InterruptProvider::default());
let admissions = Arc::new(InterruptAdmissions::default());
let (mut socket, _server) = connect(provider.clone(), admissions.clone()).await;
let create = json!({"type":"response.create","model":"model-a","input":"hello"});
for (number, terminal) in [(0, "response.completed"), (1, "response.incomplete")] {
let id = format!("resp_interrupt_{number}");
send(&mut socket, create.clone()).await;
assert_eq!(next(&mut socket).await["type"], "response.created");
if number == 0 {
provider.complete.notify_one();
} else {
send(&mut socket, interrupt(&id, "discard_partial_items")).await;
}
assert_eq!(next(&mut socket).await["type"], terminal);
tokio::time::timeout(Duration::from_secs(5), async {
while admissions.active.load(Ordering::SeqCst) != 0 {
tokio::task::yield_now().await;
}
})
.await
.unwrap();
for control in [
interrupt(&id, "discard_partial_items"),
interrupt(&id, "future_mode"),
json!({"type":"future.control","extra":[1,2]}),
] {
send(&mut socket, control.clone()).await;
assert_eq!(
next(&mut socket).await,
json!({"type":"future.control.ack","original":control})
);
}
send(&mut socket, json!({"type":"future.reject"})).await;
assert_eq!(next(&mut socket).await, upstream_control_error());
assert_eq!(provider.calls.load(Ordering::SeqCst), number + 1);
assert_eq!(admissions.active.load(Ordering::SeqCst), 0);
}
socket.close(None).await.unwrap();
}
#[tokio::test]
async fn late_interrupt_during_terminal_settlement_does_not_abort_or_reorder_requests() {
let provider = Arc::new(InterruptProvider::default());
let (release, wait_for_release) = tokio::sync::oneshot::channel();
let admissions = Arc::new(InterruptAdmissions {
release_gate: Mutex::new(Some(wait_for_release)),
..Default::default()
});
let (mut socket, _server) = connect(provider.clone(), admissions.clone()).await;
let create = json!({"type":"response.create","model":"model-a","input":"hello"});
let interrupt =
|id: &str, mode: &str| json!({"type":"response.interrupt","response_id":id,"mode":mode});
send(
&mut socket,
interrupt("resp_interrupt_0", "discard_partial_items"),
)
.await;
assert_eq!(next(&mut socket).await["type"], "error");
assert_eq!(provider.calls.load(Ordering::SeqCst), 0);
send(&mut socket, create.clone()).await;
assert_eq!(next(&mut socket).await["type"], "response.created");
provider.complete.notify_one();
assert_eq!(next(&mut socket).await["type"], "response.completed");
tokio::time::timeout(
Duration::from_secs(5),
admissions.release_started.notified(),
)
.await
.unwrap();
send(&mut socket, create).await;
for invalid in [
interrupt("resp_other", "discard_partial_items"),
interrupt("resp_interrupt_0", "unknown"),
interrupt("", "discard_partial_items"),
let late = interrupt("resp_interrupt_0", "discard_partial_items");
send(&mut socket, late.clone()).await;
assert_eq!(
next(&mut socket).await,
json!({"type":"future.control.ack", "original":late})
);
assert_eq!(provider.calls.load(Ordering::SeqCst), 1);
assert_eq!(admissions.active.load(Ordering::SeqCst), 1);
release.send(()).unwrap();
assert_eq!(
next(&mut socket).await["response"]["id"],
"resp_interrupt_1"
);
socket.close(None).await.unwrap();
}
#[tokio::test]
async fn unknown_controls_preserve_bytes_and_do_not_admit_queued_creates() {
let provider = Arc::new(InterruptProvider::default());
let admissions = Arc::new(InterruptAdmissions::default());
let (mut socket, _server) = connect(provider.clone(), admissions.clone()).await;
let create = json!({"type":"response.create","model":"model-a","input":"hello"});
send(&mut socket, json!({"type":"future.control"})).await;
let error = next(&mut socket).await;
assert_eq!(error["type"], "error");
assert_ne!(error["error"]["param"], "type");
assert_eq!(provider.calls.load(Ordering::SeqCst), 0);
send(&mut socket, create).await;
assert_eq!(next(&mut socket).await["type"], "response.created");
// 重复 type 仍是创建请求,不能借投影解码失败绕过串行准入
socket.send(ClientMessage::Text(r#"{"type":"response.create","type":"response.create","model":"model-a","input":"next"}"#.into())).await.unwrap();
for raw in [
r#"{ "type": "future.control", "mode": "unknown", "extension": [1, 2] }"#,
r#"{"extension":"no local type contract"}"#,
r#"{"type":42,"extension":true}"#,
"not-json",
] {
send(&mut socket, invalid).await;
let error = next(&mut socket).await;
assert_eq!(error["type"], "error");
assert_eq!(error["status"], 400);
socket.send(ClientMessage::Text(raw.into())).await.unwrap();
let expected =
serde_json::from_str::<Value>(raw).unwrap_or_else(|_| Value::String(raw.to_owned()));
assert_eq!(next(&mut socket).await["original"], expected);
assert_eq!(
provider
.transport
.lock()
.unwrap()
.as_ref()
.unwrap()
.requests
.lock()
.unwrap()
.last()
.unwrap(),
raw
);
assert_eq!(provider.calls.load(Ordering::SeqCst), 1);
}
send(
@@ -207,27 +391,12 @@ async fn interrupt_is_live_scoped_and_keeps_queued_creates_serial() {
interrupt("resp_interrupt_0", "discard_partial_items"),
)
.await;
let terminal = next(&mut socket).await;
assert_eq!(terminal["type"], "response.incomplete");
assert_eq!(
terminal["response"]["incomplete_details"]["reason"],
"interrupted"
);
assert_eq!(next(&mut socket).await["type"], "response.incomplete");
assert_eq!(
next(&mut socket).await["response"]["id"],
"resp_interrupt_1"
);
assert_eq!(provider.calls.load(Ordering::SeqCst), 2);
send(
&mut socket,
interrupt("resp_interrupt_0", "discard_partial_items"),
)
.await;
assert_eq!(next(&mut socket).await["type"], "error");
assert_eq!(
provider.controls.lock().unwrap()[0].interrupt("resp_interrupt_0"),
Err(ResponseInterruptError::Unavailable)
);
socket.close(None).await.unwrap();
tokio::time::timeout(Duration::from_secs(5), async {
while admissions.active.load(Ordering::SeqCst) != 0 {
@@ -236,8 +405,46 @@ async fn interrupt_is_live_scoped_and_keeps_queued_creates_serial() {
})
.await
.expect("disconnect releases active execution");
assert_eq!(
provider.controls.lock().unwrap()[1].interrupt("resp_interrupt_1"),
Err(ResponseInterruptError::Unavailable)
);
}
#[tokio::test]
async fn active_websocket_delivery_preserves_unparsed_messages_and_business_metadata() {
let messages = vec![
"{ \"type\": \"future.event\", \"number\": 1e3, \"escaped\": \"\\u0061\" }\r\n".to_owned(),
r#"{"type":"response.metadata","metadata":{"type":"safety_buffering","use_cases":["cyber"],"reasons":["user_risk"],"future":{"keep":true}}}"#.to_owned(),
format!("{{\"type\":\"future.deep\",\"extension\":{}0{}}}", "[".repeat(140), "]".repeat(140)),
"future non-JSON text".to_owned(), "[DONE]".to_owned(),
];
let provider = Arc::new(InterruptProvider::default());
*provider.initial_messages.lock().unwrap() = messages
.iter()
.map(|raw| match serde_json::from_str::<Value>(raw) {
Ok(value) => ProtocolWireEvent::json(
"openai",
value.get("type").and_then(Value::as_str).map(str::to_owned),
value,
)
.unwrap()
.with_raw_websocket_message(raw.as_str()),
Err(_) => ProtocolWireEvent::raw_websocket("openai", raw.as_str()).unwrap(),
})
.collect();
let (mut socket, _server) =
connect(provider.clone(), Arc::new(InterruptAdmissions::default())).await;
send(
&mut socket,
json!({"type":"response.create","model":"model-a","input":"synthetic"}),
)
.await;
assert_eq!(next(&mut socket).await["type"], "response.created");
for expected in &messages {
let actual = tokio::time::timeout(Duration::from_secs(5), socket.next())
.await
.unwrap()
.unwrap()
.unwrap();
assert_eq!(actual.to_text().unwrap(), expected);
}
provider.complete.notify_one();
assert_eq!(next(&mut socket).await["type"], "response.completed");
}
@@ -7,6 +7,7 @@ use axum::{
body::{Body, to_bytes},
http::{Request, StatusCode, header::AUTHORIZATION},
};
use base64::{Engine as _, engine::general_purpose::STANDARD};
use bytes::Bytes;
use futures::future::BoxFuture;
use gateway_core::engine::ModelRequestId;
@@ -181,6 +182,11 @@ async fn search_route_should_preserve_request_and_success_response_bytes() {
.header("x-codex-turn-metadata", turn_metadata)
.header("session-id", "root-session")
.header("thread-id", "child-thread")
.header("x-future-business", "first")
.header("x-future-business", "second")
.header("cookie", "client-secret")
.header("connection", "x-private-hop")
.header("x-private-hop", "client-hop")
.body(Body::from(request_body.to_vec()))
.expect("search request"),
)
@@ -213,8 +219,37 @@ async fn search_route_should_preserve_request_and_success_response_bytes() {
assert_eq!(captured[0].endpoint, "/v1/alpha/search");
assert_eq!(captured[0].transport, ClientTransport::HttpJson);
assert_eq!(captured[0].body.as_ref(), request_body);
let mut context = captured[0].context.clone();
let headers = context
.as_object_mut()
.unwrap()
.remove("opaque_request_headers")
.unwrap();
let headers: Vec<_> = headers
.as_array()
.unwrap()
.iter()
.map(|entry| {
(
entry[0].as_str().unwrap(),
STANDARD.decode(entry[1].as_str().unwrap()).unwrap(),
)
})
.collect();
assert_eq!(
captured[0].context,
headers
.iter()
.filter(|(name, _)| *name == "x-future-business")
.map(|(_, value)| value.as_slice())
.collect::<Vec<_>>(),
[b"first".as_slice(), b"second".as_slice()]
);
assert!(!headers.iter().any(|(name, _)| matches!(
*name,
"authorization" | "cookie" | "connection" | "x-private-hop"
)));
assert_eq!(
context,
json!({"turn_metadata": turn_metadata, "session_id": "root-session", "thread_id": "child-thread"})
);
assert_eq!(
@@ -90,18 +90,6 @@ impl AccountConcurrency {
Self::Limited(limit) => Some(limit),
}
}
/// 扣除当前请求不可占用的预留名额;不限并发不受影响,有限上限至少保留 1 个名额
#[must_use]
pub const fn excluding_reserved(self, reserved: u32) -> Self {
match self {
Self::Unlimited => Self::Unlimited,
Self::Limited(limit) => match NonZeroU32::new(limit.get().saturating_sub(reserved)) {
Some(remaining) => Self::Limited(remaining),
None => Self::Limited(NonZeroU32::MIN),
},
}
}
}
impl From<NonZeroU32> for AccountConcurrency {
@@ -206,6 +206,7 @@ impl AccountSelectionPolicy {
/// Store 提供并发事实,Provider 叠加自己解释的额度事实;全部信号均可重建
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct AccountRuntimeSignals {
/// 本次调度容量池的在途数,不合并其他独立池
pub in_flight: u32,
pub last_started_at: Option<SystemTime>,
pub quota_reset_at: Option<SystemTime>,
@@ -523,7 +524,7 @@ pub struct AccountSelectionContext {
pub round_robin_cursor: u64,
pub eligibility: AccountEligibilityPolicy,
pub account_scope: Option<std::sync::Arc<crate::account::scope::FrozenAccountScope>>,
/// 本请求不可占用的每账号预留并发名额,由 Provider 按请求类别决定;0 表示不预留
/// 本请求使用的独立预留池上限,由 Provider 按请求类别决定;0 表示使用普通账号池
pub reserved_concurrency: u32,
}
@@ -531,9 +532,11 @@ impl AccountSelectionContext {
/// 本请求在该账号上可使用的并发上限;资格判断、租约与策略投影必须共用这一口径
#[must_use]
pub fn concurrency_limit(&self, account: &ProviderAccount) -> AccountConcurrency {
account
.effective_concurrency(self.policy.max_concurrent_per_account())
.excluding_reserved(self.reserved_concurrency)
if self.reserved_concurrency > 0 {
AccountConcurrency::new(self.reserved_concurrency)
} else {
account.effective_concurrency(self.policy.max_concurrent_per_account())
}
}
}
@@ -723,7 +726,6 @@ impl AccountSelector {
let candidate = match context.policy.strategy() {
RotationStrategy::QuotaResetPriority => {
let default_concurrency = context.policy.max_concurrent_per_account();
eligible.sort_by(|left, right| {
(
left.signals.quota_reset_at.is_none(),
@@ -734,8 +736,8 @@ impl AccountSelector {
right.signals.quota_reset_at,
))
.then_with(|| {
capacity_utilization(left, default_concurrency)
.total_cmp(&capacity_utilization(right, default_concurrency))
capacity_utilization(left, context)
.total_cmp(&capacity_utilization(right, context))
})
.then_with(|| {
left.signals
@@ -751,13 +753,7 @@ impl AccountSelector {
let index = context.round_robin_cursor as usize % eligible.len();
eligible.get(index).copied()?
}
RotationStrategy::Smart => select_smart_candidate(
&eligible,
context.policy.max_concurrent_per_account(),
context.round_robin_cursor,
context.policy.smart_scheduling(),
context.now,
)?,
RotationStrategy::Smart => select_smart_candidate(&eligible, context)?,
RotationStrategy::Sticky => {
eligible.sort_by_key(|candidate| {
(
@@ -843,17 +839,7 @@ impl AccountSelector {
// 账号信号在队列锁外评分,锁内只读取实时队长并查表,避免扫描候选阻塞其他等待者
let scores = candidates
.iter()
.map(|candidate| {
(
candidate.account.id(),
smart_score(
candidate,
context.policy.max_concurrent_per_account(),
config,
context.now,
),
)
})
.map(|candidate| (candidate.account.id(), smart_score(candidate, context)))
.collect::<HashMap<_, _>>();
waiting
.wait_with_priority(keys, |key, count| {
@@ -914,13 +900,9 @@ const SMART_LATENCY_HALF_SCORE_MS: f64 = 10_000.0;
// 距重置一小时时得分减半;未知和已过期时间不提供重置奖励
const SMART_RESET_HALF_SCORE_SECONDS: f64 = 3_600.0;
fn capacity_utilization(
candidate: &AccountCandidate,
default_concurrency: AccountConcurrency,
) -> f64 {
candidate
.account
.effective_concurrency(default_concurrency)
fn capacity_utilization(candidate: &AccountCandidate, context: &AccountSelectionContext) -> f64 {
context
.concurrency_limit(&candidate.account)
.limit()
.map_or(0.0, |limit| {
f64::from(candidate.signals.in_flight) / f64::from(limit.get())
@@ -929,38 +911,29 @@ fn capacity_utilization(
fn select_smart_candidate<'a>(
candidates: &[&'a AccountCandidate],
default_concurrency: AccountConcurrency,
cursor: u64,
config: SmartSchedulingConfig,
now: SystemTime,
context: &AccountSelectionContext,
) -> Option<&'a AccountCandidate> {
let mut ranked = candidates
.iter()
.map(|candidate| {
(
*candidate,
smart_score(candidate, default_concurrency, config, now),
)
})
.map(|candidate| (*candidate, smart_score(candidate, context)))
.collect::<Vec<_>>();
let best_score = ranked
.iter()
.map(|(_, score)| *score)
.max_by(f64::total_cmp)?;
ranked.retain(|(_, score)| best_score - score <= config.score_tolerance());
ranked.retain(|(_, score)| {
best_score - score <= context.policy.smart_scheduling().score_tolerance()
});
// 轮换顺序保持稳定,避免分数轻微交错与 cursor 同步后仍反复命中同一账号
ranked.sort_unstable_by(|(left, _), (right, _)| left.account.id().cmp(right.account.id()));
let index = (cursor % ranked.len() as u64) as usize;
let index = (context.round_robin_cursor % ranked.len() as u64) as usize;
Some(ranked[index].0)
}
pub(crate) fn smart_score(
candidate: &AccountCandidate,
default_concurrency: AccountConcurrency,
config: SmartSchedulingConfig,
now: SystemTime,
) -> f64 {
let load = 1.0 - capacity_utilization(candidate, default_concurrency).clamp(0.0, 1.0);
pub(crate) fn smart_score(candidate: &AccountCandidate, context: &AccountSelectionContext) -> f64 {
let config = context.policy.smart_scheduling();
let now = context.now;
let load = 1.0 - capacity_utilization(candidate, context).clamp(0.0, 1.0);
let quota = candidate
.signals
.quota_remaining_rank
@@ -41,17 +41,14 @@ impl TraceContext {
"blocker": AccountSelector.scheduling_blocker(candidate, context)
.map(|blocker| format!("{blocker:?}")),
"inFlight": signals.in_flight,
"concurrencyLimit": candidate.account
.effective_concurrency(context.policy.max_concurrent_per_account()).get(),
"concurrencyLimit": context.concurrency_limit(&candidate.account).get(),
"quotaRemainingBasisPoints": signals.quota_remaining_rank,
"quotaResetAtUnixMs": signals.quota_reset_at.and_then(|reset|
reset.duration_since(std::time::UNIX_EPOCH).ok()
).map(|duration| duration.as_millis()),
"failureRateBasisPoints": signals.failure_rate_basis_points,
"firstOutputLatencyMs": signals.first_output_latency_ms,
"smartScore": smart.then(|| smart_score(
candidate, context.policy.max_concurrent_per_account(), context.policy.smart_scheduling(), context.now
)),
"smartScore": smart.then(|| smart_score(candidate, context)),
})
})
.collect::<Vec<_>>();
@@ -878,8 +878,8 @@ pub struct ProbeFailure {
pub latency: Duration,
}
/// `model_requests` 可用的毫秒级阶段耗时
#[derive(Debug, Clone, Default, PartialEq, Eq)]
/// 请求与上游响应的耗时事实,统一以毫秒保存但不提高来源精度
#[derive(Debug, Clone, Default, PartialEq)]
pub struct ModelRequestTimings {
pub transport_decision_wait_ms: Option<u64>,
pub connect_ms: Option<u64>,
@@ -889,6 +889,13 @@ pub struct ModelRequestTimings {
pub first_text_ms: Option<u64>,
pub first_token_ms: Option<u64>,
pub provider_processing_ms: Option<u64>,
pub upstream_response_ms: Option<u64>,
pub upstream_api_overhead_ms: Option<f64>,
pub upstream_engine_ms: Option<f64>,
pub upstream_engine_iapi_ttft_ms: Option<f64>,
pub upstream_engine_service_ttft_ms: Option<f64>,
pub upstream_engine_iapi_tbt_ms: Option<f64>,
pub upstream_engine_service_tbt_ms: Option<f64>,
pub latency_ms: Option<u64>,
}
@@ -237,7 +237,7 @@ pub enum RequestObservationOutcome {
}
/// Core 在业务结果确定后产生的一次最终观察,不包含原始正文或凭据
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq)]
pub struct RequestObservation {
event_id: String,
request_id: ModelRequestId,
@@ -768,8 +768,6 @@ impl ResponseObservation {
}
pub(super) fn observe_event(&mut self, event: &GatewayEvent) {
let elapsed = self.elapsed_ms();
observe_event_timing(&mut self.timings, event, elapsed);
if let GatewayEvent::Usage(observed) = event {
self.usage.merge(observed);
}
@@ -806,6 +804,13 @@ impl ResponseObservation {
self.timings.first_text_ms = None;
self.timings.first_token_ms = None;
self.timings.provider_processing_ms = None;
self.timings.upstream_response_ms = None;
self.timings.upstream_api_overhead_ms = None;
self.timings.upstream_engine_ms = None;
self.timings.upstream_engine_iapi_ttft_ms = None;
self.timings.upstream_engine_service_ttft_ms = None;
self.timings.upstream_engine_iapi_tbt_ms = None;
self.timings.upstream_engine_service_tbt_ms = None;
}
pub(super) fn observe_response(&mut self, observation: &ProviderResponseObservation) {
@@ -834,26 +839,26 @@ impl ResponseObservation {
if let Some(value) = observed.provider_processing_ms {
self.timings.provider_processing_ms = Some(value);
}
}
}
fn observe_event_timing(timings: &mut ModelRequestTimings, event: &GatewayEvent, elapsed_ms: u64) {
timings.first_event_ms.get_or_insert(elapsed_ms);
match event {
GatewayEvent::ReasoningDelta(delta) if !delta.text.is_empty() => {
timings.first_reasoning_ms.get_or_insert(elapsed_ms);
timings.first_token_ms.get_or_insert(elapsed_ms);
if let Some(value) = observed.upstream_response_ms {
self.timings.upstream_response_ms = Some(value);
}
GatewayEvent::TextDelta(delta) if !delta.text.is_empty() => {
timings.first_text_ms.get_or_insert(elapsed_ms);
timings.first_token_ms.get_or_insert(elapsed_ms);
if let Some(value) = observed.upstream_api_overhead_ms {
self.timings.upstream_api_overhead_ms = Some(value);
}
// `response.output_item.added` 会先投影一个空参数的 tool delta;它只是结构帧,
// 不能抢在真实工具参数之前成为首个可消费 token
GatewayEvent::ToolCallDelta(delta) if !delta.arguments_delta.is_empty() => {
timings.first_token_ms.get_or_insert(elapsed_ms);
if let Some(value) = observed.upstream_engine_ms {
self.timings.upstream_engine_ms = Some(value);
}
if let Some(value) = observed.upstream_engine_iapi_ttft_ms {
self.timings.upstream_engine_iapi_ttft_ms = Some(value);
}
if let Some(value) = observed.upstream_engine_service_ttft_ms {
self.timings.upstream_engine_service_ttft_ms = Some(value);
}
if let Some(value) = observed.upstream_engine_iapi_tbt_ms {
self.timings.upstream_engine_iapi_tbt_ms = Some(value);
}
if let Some(value) = observed.upstream_engine_service_tbt_ms {
self.timings.upstream_engine_service_tbt_ms = Some(value);
}
GatewayEvent::CalculatedCost(_) | GatewayEvent::ProviderCost(_) => {}
_ => {}
}
}
@@ -605,10 +605,15 @@ fn provider_event_to_middleware_frame(
ClientTransport::WebSocket
| ClientTransport::InternalProbe
| ClientTransport::InternalPlugin => (
serde_json::to_vec(wire.data())
.map(Bytes::from)
wire.raw_websocket_message()
.map(|raw| Bytes::copy_from_slice(raw.as_bytes()))
.map_or_else(|| serde_json::to_vec(wire.data()).map(Bytes::from), Ok)
.map_err(|_| middleware_protocol_error(UpstreamSendState::Sent))?,
MiddlewareFraming::JsonDocument,
if wire.raw_websocket_message().is_some() && !wire.has_json_data() {
MiddlewareFraming::RawBytes
} else {
MiddlewareFraming::JsonDocument
},
),
};
Ok(MiddlewareFrame::from_provider_event(
@@ -631,10 +636,19 @@ fn encode_sse_wire_event(wire: &ProtocolWireEvent) -> Result<Bytes, ProviderErro
encoded.extend_from_slice(event_type.as_bytes());
encoded.push(b'\n');
}
encoded.extend_from_slice(b"data: ");
serde_json::to_writer(&mut encoded, wire.data())
.map_err(|_| middleware_protocol_error(UpstreamSendState::Sent))?;
encoded.extend_from_slice(b"\n\n");
if let Some(raw) = wire.raw_websocket_message() {
for line in raw.split('\n') {
encoded.extend_from_slice(b"data: ");
encoded.extend_from_slice(line.as_bytes());
encoded.push(b'\n');
}
encoded.push(b'\n');
} else {
encoded.extend_from_slice(b"data: ");
serde_json::to_writer(&mut encoded, wire.data())
.map_err(|_| middleware_protocol_error(UpstreamSendState::Sent))?;
encoded.extend_from_slice(b"\n\n");
}
Ok(Bytes::from(encoded))
}
@@ -701,12 +715,19 @@ fn middleware_frame_to_provider_event(
.raw_json_body()
.or_else(|| wire.raw_http_body_bytes())
.cloned()
.or_else(|| {
wire.raw_websocket_message()
.map(|raw| Bytes::copy_from_slice(raw.as_bytes()))
})
.or_else(|| serde_json::to_vec(wire.data()).ok().map(Bytes::from)),
MiddlewareFraming::SseEvent => wire
.raw_sse_frame()
.cloned()
.or_else(|| encode_sse_wire_event(wire).ok()),
MiddlewareFraming::RawBytes => wire.raw_http_body_bytes().cloned(),
MiddlewareFraming::RawBytes => wire.raw_http_body_bytes().cloned().or_else(|| {
wire.raw_websocket_message()
.map(|raw| Bytes::copy_from_slice(raw.as_bytes()))
}),
};
if original.as_ref() == Some(&bytes)
&& let Some(mut event) = envelope.take()
@@ -1,14 +1,27 @@
//! 当前执行的响应中断信号;只由持有活动响应的 Provider 注册,不查找其他执行或连接
//! 原上游连接的控制消息端口;不解释协议类型、不创建执行或延长连接生命周期
use std::fmt;
use std::sync::{Arc, Mutex, Weak};
use crate::lifecycle::CancellationToken;
use async_trait::async_trait;
/// 单次根执行的控制入口;重试不继承已结束响应的控制目标
/// Provider 实现的连接控制通道,消息内容由上游协议解释
#[async_trait]
pub trait ResponseControlTransport: Send + Sync {
async fn send(&self, payload: &str) -> Result<(), ResponseControlUnavailable>;
/// 仅在响应正文已结束后读取,调用方必须先取消此读取再开始下一轮执行
async fn receive(&self) -> Result<String, ResponseControlUnavailable>;
}
/// 控制消息没有可用的原上游连接,不能通过重新选号或重试恢复
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct ResponseControlUnavailable;
/// 单次根执行绑定的原连接;终态后仍可使用,连接销毁或重新绑定时失效
#[derive(Clone, Default)]
pub struct ResponseControl {
active: Arc<Mutex<Weak<InterruptTarget>>>,
target: Arc<Mutex<Option<Weak<dyn ResponseControlTransport>>>>,
}
impl fmt::Debug for ResponseControl {
@@ -17,64 +30,42 @@ impl fmt::Debug for ResponseControl {
}
}
struct InterruptTarget {
response_id: String,
requested: CancellationToken,
}
/// Provider 持有的活动响应控制权;释放后客户端无法再向该响应发出中断
pub struct ActiveResponseInterrupt(Arc<InterruptTarget>);
/// 控制帧不能作用于当前执行时的稳定原因,不携带其他响应身份
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ResponseInterruptError {
Unavailable,
ResponseMismatch,
}
impl ResponseControl {
/// 在上游确认响应 ID 后注册;同一执行只允许一个存活的响应 owner
pub fn activate(&self, response_id: String) -> Option<ActiveResponseInterrupt> {
let mut active = self
.active
/// Provider 在请求成功写入上游后绑定;连接 owner 持有强引用并负责撤销
pub fn bind(&self, transport: &Arc<dyn ResponseControlTransport>) {
*self
.target
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if active.upgrade().is_some() || response_id.is_empty() {
return None;
}
let target = Arc::new(InterruptTarget {
response_id,
requested: CancellationToken::new(),
});
*active = Arc::downgrade(&target);
Some(ActiveResponseInterrupt(target))
.unwrap_or_else(std::sync::PoisonError::into_inner) = Some(Arc::downgrade(transport));
}
/// 重复中断合并为一次信号;ID 必须匹配本执行当前的上游响应
pub fn interrupt(&self, response_id: &str) -> Result<(), ResponseInterruptError> {
let active = self
.active
pub fn clear(&self) {
*self
.target
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let target = active
.upgrade()
.ok_or(ResponseInterruptError::Unavailable)?;
if target.response_id != response_id {
return Err(ResponseInterruptError::ResponseMismatch);
.unwrap_or_else(std::sync::PoisonError::into_inner) = None;
}
fn transport(&self) -> Option<Arc<dyn ResponseControlTransport>> {
self.target
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.as_ref()
.and_then(Weak::upgrade)
}
pub async fn send(&self, payload: &str) -> Result<(), ResponseControlUnavailable> {
self.transport()
.ok_or(ResponseControlUnavailable)?
.send(payload)
.await
}
/// 未绑定时保持等待,供连接空闲循环与客户端输入共同 select
pub async fn receive(&self) -> Result<String, ResponseControlUnavailable> {
match self.transport() {
Some(transport) => transport.receive().await,
None => std::future::pending().await,
}
target.requested.cancel();
Ok(())
}
}
impl ActiveResponseInterrupt {
pub fn response_id(&self) -> &str {
&self.0.response_id
}
pub fn requested(&self) -> impl Future<Output = ()> + Send + 'static {
// 等待者只持有信号,不能延长活动响应 owner 的生命周期
let requested = self.0.requested.clone();
async move { requested.cancelled().await }
}
}
+52 -4
View File
@@ -203,6 +203,7 @@ pub struct ProtocolWireEvent {
data: Value,
has_json_data: bool,
raw_sse_frame: Option<Bytes>,
raw_websocket_message: Option<Arc<str>>,
raw_json_body: Option<Bytes>,
raw_http_body: Option<Bytes>,
sse_id: Option<String>,
@@ -245,6 +246,7 @@ impl ProtocolWireEvent {
data,
has_json_data: true,
raw_sse_frame: None,
raw_websocket_message: None,
raw_json_body: None,
raw_http_body: None,
sse_id,
@@ -293,6 +295,7 @@ impl ProtocolWireEvent {
data: Value::Null,
has_json_data: false,
raw_sse_frame: Some(raw_sse_frame),
raw_websocket_message: None,
raw_json_body: None,
raw_http_body: None,
sse_id: None,
@@ -300,6 +303,33 @@ impl ProtocolWireEvent {
})
}
/// 创建原始 WebSocket 文本;JSON 解析失败不影响交付
///
/// # Errors
///
/// 协议名不满足 wire 安全约束时返回错误
pub fn raw_websocket(
protocol: impl Into<String>,
message: impl Into<Arc<str>>,
) -> Result<Self, IdentifierError> {
let mut wire = Self::json(protocol, None, Value::Null)?;
wire.has_json_data = false;
Ok(wire.with_raw_websocket_message(message))
}
/// 绑定同一事件未经改写的 WebSocket 文本,解析视图只用于旁路事实
#[must_use]
pub fn with_raw_websocket_message(mut self, message: impl Into<Arc<str>>) -> Self {
self.raw_websocket_message = Some(message.into());
self
}
/// 返回未经改写的 WebSocket 文本
#[must_use]
pub fn raw_websocket_message(&self) -> Option<&str> {
self.raw_websocket_message.as_deref()
}
/// 创建未经改写的完整 JSON 响应正文
///
/// 该载荷用于非流式协议端点;Core 不解析或重编码其中的值
@@ -319,6 +349,7 @@ impl ProtocolWireEvent {
data: Value::Null,
has_json_data: false,
raw_sse_frame: None,
raw_websocket_message: None,
raw_json_body: Some(raw_json_body),
raw_http_body: None,
sse_id: None,
@@ -343,6 +374,7 @@ impl ProtocolWireEvent {
data: Value::Null,
has_json_data: false,
raw_sse_frame: None,
raw_websocket_message: None,
raw_json_body: None,
raw_http_body: Some(raw_http_body),
sse_id: None,
@@ -431,6 +463,10 @@ impl fmt::Debug for ProtocolWireEvent {
.field("event_type", &self.event_type)
.field("has_json_data", &self.has_json_data)
.field("has_raw_sse_frame", &self.raw_sse_frame.is_some())
.field(
"has_raw_websocket_message",
&self.raw_websocket_message.is_some(),
)
.field("has_raw_json_body", &self.raw_json_body.is_some())
.field("has_raw_http_body", &self.raw_http_body.is_some())
.field("has_sse_id", &self.sse_id.is_some())
@@ -479,12 +515,14 @@ impl UpstreamHttpVersion {
}
}
/// Provider transport 边界测得的时间
/// Provider transport 观测与上游返回的响应耗时
///
/// `first_*_ms` 是相对 `AttemptContext::timing_started_at()` 的请求级偏移,
/// 与 Core 总耗时使用同一起点并包含选号与重试等待;其余字段是各阶段独立耗时
/// 与 Core 总耗时使用同一起点并包含选号与重试等待
/// `upstream_response_ms` 使用上游的响应计时边界,其余字段是各阶段独立耗时
/// 首事件与输出阶段时间由 Provider 独占采集,Core 不从 canonical 事件补记
/// 首字边界由 Provider 协议定义,缺失时保留 `None`,不使用首包代替
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
#[derive(Debug, Clone, Copy, Default, PartialEq)]
pub struct ProviderResponseTimings {
pub transport_decision_wait_ms: Option<u64>,
pub connect_ms: Option<u64>,
@@ -494,6 +532,16 @@ pub struct ProviderResponseTimings {
pub first_text_ms: Option<u64>,
pub first_token_ms: Option<u64>,
pub provider_processing_ms: Option<u64>,
/// 上游返回的单次响应创建到完成耗时,不使用本地时钟补齐
pub upstream_response_ms: Option<u64>,
/// 上游排除引擎与客户端工具时间后的 API 耗时
pub upstream_api_overhead_ms: Option<f64>,
/// 上游响应级引擎耗时;以下专项指标保留毫秒小数及各自内部计量层级
pub upstream_engine_ms: Option<f64>,
pub upstream_engine_iapi_ttft_ms: Option<f64>,
pub upstream_engine_service_ttft_ms: Option<f64>,
pub upstream_engine_iapi_tbt_ms: Option<f64>,
pub upstream_engine_service_tbt_ms: Option<f64>,
}
/// Provider 已筛选的响应观测 JSON
@@ -572,7 +620,7 @@ impl fmt::Debug for ProviderResponseHeader {
}
/// Core 消费的 Provider 执行观测,不会原样进入客户端响应
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq)]
pub struct ProviderResponseObservation {
transport: UpstreamTransport,
http_version: Option<UpstreamHttpVersion>,
+1 -1
View File
@@ -126,7 +126,7 @@ pub trait ConnectionGuard: Send + 'static {}
pub trait ConnectionLifecycle: Send + Sync {
/// 原子地检查 drain 状态并注册一个活跃连接
///
/// 当本方法成功时,drain 必须等待返回的 guard 被释放;
/// 成功注册不延长关闭流程,活跃连接可随进程退出被中断;
/// 当 drain 已经线性化生效时,本方法必须返回 [`ConnectionDraining`]
fn try_register(&self) -> Result<Box<dyn ConnectionGuard>, ConnectionDraining>;
@@ -9,6 +9,14 @@ use crate::{
use futures::future::BoxFuture;
use std::{collections::BTreeMap, fmt, num::NonZeroU32, time::Duration};
/// 账号内独立计数的调度容量池;Provider 决定请求归属,Store 只隔离租约
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub enum ProviderConcurrencyPool {
#[default]
Shared,
Reserved,
}
/// 一个 Provider 的完整可重建调度状态
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ProviderSchedulingState {
@@ -46,6 +54,7 @@ pub struct ProviderSchedulingLeaseRequest {
account_id: ProviderAccountId,
credential_revision: CredentialRevision,
max_concurrent: AccountConcurrency,
concurrency_pool: ProviderConcurrencyPool,
request_interval: Duration,
deadline: crate::lifecycle::Deadline,
cancellation: crate::lifecycle::CancellationToken,
@@ -66,6 +75,7 @@ impl ProviderSchedulingLeaseRequest {
account_id,
credential_revision,
max_concurrent: max_concurrent.into(),
concurrency_pool: ProviderConcurrencyPool::Shared,
request_interval,
deadline: deadline.into(),
cancellation: crate::lifecycle::CancellationToken::new(),
@@ -78,6 +88,17 @@ impl ProviderSchedulingLeaseRequest {
self
}
#[must_use]
pub const fn with_concurrency_pool(mut self, pool: ProviderConcurrencyPool) -> Self {
self.concurrency_pool = pool;
self
}
#[must_use]
pub const fn concurrency_pool(&self) -> ProviderConcurrencyPool {
self.concurrency_pool
}
#[must_use]
pub fn cancellation(&self) -> crate::lifecycle::CancellationToken {
self.cancellation.clone()
@@ -145,11 +166,13 @@ pub enum ProviderLeaseRequest {
}
pub trait ProviderLeasePort: Send + Sync {
/// 按目标容量池读取在途数,账号最小请求间隔仍跨池共享
fn load_state<'a>(
&'a self,
client_api_key_id: &'a ClientApiKeyId,
provider_kind: &'a ProviderKind,
accounts: &'a [ProviderAccountId],
pool: ProviderConcurrencyPool,
) -> BoxFuture<'a, Result<ProviderSchedulingState, ProviderStoreError>>;
fn try_acquire(
@@ -157,7 +180,7 @@ pub trait ProviderLeasePort: Send + Sync {
request: ProviderLeaseRequest,
) -> BoxFuture<'_, Result<ProviderLeaseAcquisition, ProviderStoreError>>;
/// 读取指定账号当前的在途请求数;只用于容量熔断的峰值证据,
/// 读取指定账号普通池的在途请求数;只用于容量熔断的峰值证据,
/// 支持租约信号的存储实现覆盖,否则视为不可观测(空映射)
fn account_in_flight<'a>(
&'a self,
@@ -4,9 +4,9 @@ mod error;
pub use error::{ProviderStoreError, ProviderStoreErrorKind};
mod lease;
pub use lease::{
ProviderLeaseAcquisition, ProviderLeaseGuard, ProviderLeasePort, ProviderLeaseRequest,
ProviderRefreshCapacityRequest, ProviderRefreshLeaseRequest, ProviderSchedulingLeaseRequest,
ProviderSchedulingState,
ProviderConcurrencyPool, ProviderLeaseAcquisition, ProviderLeaseGuard, ProviderLeasePort,
ProviderLeaseRequest, ProviderRefreshCapacityRequest, ProviderRefreshLeaseRequest,
ProviderSchedulingLeaseRequest, ProviderSchedulingState,
};
mod affinity;
pub use affinity::{
@@ -15,13 +15,12 @@ use std::time::{Duration, SystemTime};
use serde_json::{Map, Value};
use gateway_core::account::{
AccountAttemptFeedback, AccountCandidate, AccountConcurrency, AccountConcurrencyLimit,
AccountEligibilityPolicy, AccountFeedbackStats, AccountQuotaSignals, AccountRuntimeSignals,
AccountSchedulingBlocker, AccountSelectionContext, AccountSelectionPolicy, AccountSelector,
AccountStatus, AccountWeight, CredentialCasUpdate, CredentialRevision, CredentialState,
OpaqueProviderData, PlaintextCredential, PreferredAccountSelection, ProviderAccount,
ProviderAccountId, ProviderAccountIdentity, ProviderAccountUpdate, QuotaEvidence, QuotaState,
RotationStrategy,
AccountAttemptFeedback, AccountCandidate, AccountConcurrencyLimit, AccountEligibilityPolicy,
AccountFeedbackStats, AccountQuotaSignals, AccountRuntimeSignals, AccountSchedulingBlocker,
AccountSelectionContext, AccountSelectionPolicy, AccountSelector, AccountStatus, AccountWeight,
CredentialCasUpdate, CredentialRevision, CredentialState, OpaqueProviderData,
PlaintextCredential, PreferredAccountSelection, ProviderAccount, ProviderAccountId,
ProviderAccountIdentity, ProviderAccountUpdate, QuotaEvidence, QuotaState, RotationStrategy,
};
use gateway_core::routing::{
ClientRoutingScope, FrozenAccountScope, ProviderKind, RuntimeAccount, RuntimeAccountDirectory,
@@ -1099,40 +1098,57 @@ fn due_probe_freeze_blocks_scheduling_and_remains_visible_as_rate_limited() {
}
#[test]
fn reserved_concurrency_only_blocks_requests_that_cannot_use_the_reserved_slots() {
fn reserved_concurrency_uses_its_own_limit_without_reducing_normal_capacity() {
let normal = context(RotationStrategy::Smart);
let candidates = [candidate_with_concurrency("acct_reserved", 4, 5)];
let mut normal = context(RotationStrategy::Smart);
normal.reserved_concurrency = 1;
assert_eq!(normal.concurrency_limit(&candidates[0].account).get(), 4);
assert!(AccountSelector.select(&candidates, &normal).is_none());
assert!(AccountSelector.select(&candidates, &normal).is_some());
let capacity = AccountSelector
.capacity_snapshot(&candidates, &normal)
.unwrap();
assert_eq!((capacity.used_slots(), capacity.total_slots()), (4, 4));
// 预留造成的暂满仍是可等待的容量约束,不能被当成不可用账号
assert_eq!(
AccountSelector.wait_candidates(&candidates, &normal).len(),
1
);
let prioritized = context(RotationStrategy::Smart);
let capacity = AccountSelector
.capacity_snapshot(&candidates, &prioritized)
.unwrap();
assert_eq!((capacity.used_slots(), capacity.total_slots()), (4, 5));
let selected = AccountSelector
.select(&candidates, &prioritized)
.expect("reserved slot available");
assert_eq!(selected.candidate().account.id().as_str(), "acct_reserved");
let full = [candidate_with_concurrency("acct_reserved", 5, 5)];
assert!(AccountSelector.select(&full, &prioritized).is_none());
// 预留不能让普通请求失去全部名额,也不改变不限并发
let mut oversized = context(RotationStrategy::Smart);
oversized.reserved_concurrency = 10;
assert_eq!(oversized.concurrency_limit(&candidates[0].account).get(), 1);
let mut reserved = normal.clone();
reserved.reserved_concurrency = 3;
let candidates = [candidate_with_concurrency("acct_reserved", 2, 5)];
assert!(AccountSelector.select(&candidates, &reserved).is_some());
let capacity = AccountSelector
.capacity_snapshot(&candidates, &reserved)
.unwrap();
assert_eq!((capacity.used_slots(), capacity.total_slots()), (2, 3));
let full = [candidate_with_concurrency("acct_reserved", 3, 5)];
assert!(AccountSelector.select(&full, &reserved).is_none());
assert_eq!(AccountSelector.wait_candidates(&full, &reserved).len(), 1);
assert!(AccountSelector.select(&full, &normal).is_some());
// 独立额度可以大于账号普通上限,极值不能回绕成无限并发
reserved.reserved_concurrency = u32::MAX;
assert_eq!(
AccountConcurrency::Unlimited.excluding_reserved(2),
AccountConcurrency::Unlimited
reserved.concurrency_limit(&candidates[0].account).get(),
u32::MAX
);
assert_eq!(normal.concurrency_limit(&candidates[0].account).get(), 5);
}
#[test]
fn reserved_capacity_stays_bounded_when_normal_concurrency_is_unlimited() {
let candidates = [candidate("acct_unlimited", 3, None)];
let mut normal = context(RotationStrategy::Smart);
normal.policy = AccountSelectionPolicy::new(
RotationStrategy::Smart,
gateway_core::account::AccountConcurrency::Unlimited,
Duration::ZERO,
);
assert!(AccountSelector.select(&candidates, &normal).is_some());
assert!(
AccountSelector
.capacity_snapshot(&candidates, &normal)
.is_none()
);
let mut reserved = normal;
reserved.reserved_concurrency = 3;
assert!(AccountSelector.select(&candidates, &reserved).is_none());
let capacity = AccountSelector
.capacity_snapshot(&candidates, &reserved)
.unwrap();
assert_eq!((capacity.used_slots(), capacity.total_slots()), (3, 3));
}
@@ -13,6 +13,31 @@ use gateway_core::concurrency::{
use super::{candidate_with_concurrency, context, weighted_candidate};
#[test]
fn reserved_pool_scoring_and_trace_use_its_capacity_instead_of_the_normal_limit() {
let mut context = context(RotationStrategy::Smart);
context.reserved_concurrency = 5;
context.policy = context.policy.with_smart_scheduling(
SmartSchedulingConfig::new([1.0, 0.0, 0.0, 0.0, 0.0, 0.0], false).unwrap(),
);
let candidates = [
candidate_with_concurrency("acct_large_normal", 2, 100),
candidate_with_concurrency("acct_small_normal", 1, 1),
];
let selection = AccountSelector.select(&candidates, &context).unwrap();
assert_eq!(
selection.candidate().account.id().as_str(),
"acct_small_normal"
);
let trace = gateway_core::diagnostics::TraceContext::new("req_reserved_scores");
trace.account_selection(&candidates, &context, Some(&selection));
let snapshot = trace.snapshot().unwrap();
let selected = &snapshot["events"][0]["data"]["candidates"][0];
assert_eq!(selected["concurrencyLimit"], 5);
assert_eq!(selected["inFlight"], 1);
assert_eq!(selected["smartScore"], 0.8);
}
#[test]
fn smart_config_rejects_invalid_weights_and_incomplete_wire_values() {
for value in [-0.1, 10.1, 0.01, f64::NAN, f64::INFINITY] {
@@ -51,7 +51,7 @@ use gateway_core::routing::{
use gateway_core::upstream::{UpstreamSendState, UpstreamTransport};
use serde_json::{Map, Value, json};
#[derive(Debug, Clone, PartialEq, Eq)]
#[derive(Debug, Clone, PartialEq)]
struct FinalState {
error_kind: Option<GatewayErrorKind>,
diagnostic_trace_json: Option<String>,
@@ -85,6 +85,13 @@ struct FinalState {
first_text_ms: Option<u64>,
first_token_ms: Option<u64>,
provider_processing_ms: Option<u64>,
upstream_response_ms: Option<u64>,
upstream_api_overhead_ms: Option<f64>,
upstream_engine_ms: Option<f64>,
upstream_engine_iapi_ttft_ms: Option<f64>,
upstream_engine_service_ttft_ms: Option<f64>,
upstream_engine_iapi_tbt_ms: Option<f64>,
upstream_engine_service_tbt_ms: Option<f64>,
provider_metadata_json: Option<String>,
cost_source: CostSource,
cost_ticks: Option<u128>,
@@ -253,6 +260,15 @@ impl ExecutionStore for FakeStore {
first_text_ms: finalization.timings.first_text_ms,
first_token_ms: finalization.timings.first_token_ms,
provider_processing_ms: finalization.timings.provider_processing_ms,
upstream_response_ms: finalization.timings.upstream_response_ms,
upstream_api_overhead_ms: finalization.timings.upstream_api_overhead_ms,
upstream_engine_ms: finalization.timings.upstream_engine_ms,
upstream_engine_iapi_ttft_ms: finalization.timings.upstream_engine_iapi_ttft_ms,
upstream_engine_service_ttft_ms: finalization
.timings
.upstream_engine_service_ttft_ms,
upstream_engine_iapi_tbt_ms: finalization.timings.upstream_engine_iapi_tbt_ms,
upstream_engine_service_tbt_ms: finalization.timings.upstream_engine_service_tbt_ms,
provider_metadata_json: finalization.provider_metadata_json,
cost_source: finalization.cost.source(),
cost_ticks: finalization
@@ -1510,7 +1526,7 @@ fn empty_tool_call_delta_should_not_preempt_provider_first_token_timing() {
}
#[test]
fn empty_content_deltas_do_not_create_first_token_timings() {
fn canonical_output_does_not_invent_missing_provider_timings() {
let operation = generate_operation();
let route_plan = plan(&operation);
let mut items = complete_stream(None);
@@ -1523,7 +1539,7 @@ fn empty_content_deltas_do_not_create_first_token_timings() {
))),
Ok(GatewayEvent::TextDelta(gateway_core::event::TextDelta {
content_index: 0,
text: String::new(),
text: "real content".to_owned(),
})),
Ok(GatewayEvent::ContentAdded(ContentItem::new(
1,
@@ -1532,7 +1548,7 @@ fn empty_content_deltas_do_not_create_first_token_timings() {
Ok(GatewayEvent::ReasoningDelta(
gateway_core::event::ReasoningDelta {
content_index: 1,
text: String::new(),
text: "real content".to_owned(),
},
)),
],
@@ -1555,7 +1571,7 @@ fn empty_content_deltas_do_not_create_first_token_timings() {
let state = store.state.lock().unwrap();
let timings = &state.finalizations[0];
assert!(timings.first_event_ms.is_some());
assert_eq!(timings.first_event_ms, None);
assert_eq!(
(
timings.first_token_ms,
@@ -1629,6 +1645,13 @@ fn discarded_attempt_observation_does_not_leak_into_retry_result() {
headers_ms: Some(13),
first_event_ms: Some(987_654),
provider_processing_ms: Some(41),
upstream_response_ms: Some(7_000),
upstream_api_overhead_ms: Some(120.25),
upstream_engine_ms: Some(6400.0),
upstream_engine_iapi_ttft_ms: Some(650.5),
upstream_engine_service_ttft_ms: Some(720.25),
upstream_engine_iapi_tbt_ms: Some(18.45),
upstream_engine_service_tbt_ms: Some(20.12),
..ProviderResponseTimings::default()
});
let second_observation = ProviderResponseObservation::new(
@@ -1719,17 +1742,20 @@ fn discarded_attempt_observation_does_not_leak_into_retry_result() {
assert_eq!(finalization.connect_ms, None);
assert_eq!(finalization.headers_ms, None);
assert_eq!(finalization.provider_processing_ms, None);
assert_eq!(finalization.upstream_response_ms, None);
assert_eq!(finalization.upstream_api_overhead_ms, None);
assert_eq!(finalization.upstream_engine_ms, None);
assert_eq!(finalization.upstream_engine_iapi_ttft_ms, None);
assert_eq!(finalization.upstream_engine_service_ttft_ms, None);
assert_eq!(finalization.upstream_engine_iapi_tbt_ms, None);
assert_eq!(finalization.upstream_engine_service_tbt_ms, None);
let contexts = provider.contexts.lock().unwrap();
assert_eq!(contexts.len(), 2);
assert_eq!(
contexts[0].timing_started_at(),
contexts[1].timing_started_at()
);
assert!(
finalization
.first_event_ms
.is_some_and(|elapsed| elapsed < 987_654)
);
assert_eq!(finalization.first_event_ms, None);
assert_eq!(state.intermediate_status_codes, vec![Some(504)]);
assert_eq!(
state.intermediate_request_ids,
@@ -1,45 +1,68 @@
//! 验证响应中断的当前 owner 校验及独立执行间的控制隔离
//! 验证原连接控制端口的透明传递、owner 生命周期与执行隔离
use futures::FutureExt;
use gateway_core::engine::response_control::{ResponseControl, ResponseInterruptError};
use std::sync::{Arc, Mutex};
#[test]
fn interrupt_requires_the_current_response_owner_and_does_not_extend_its_lifetime() {
let control = ResponseControl::default();
assert_eq!(
control.interrupt("resp_a"),
Err(ResponseInterruptError::Unavailable)
);
assert!(control.activate(String::new()).is_none());
let active = control.activate("resp_a".to_owned()).unwrap();
let requested = active.requested();
assert!(control.activate("resp_b".to_owned()).is_none());
assert_eq!(
control.interrupt("resp_b"),
Err(ResponseInterruptError::ResponseMismatch)
);
assert!(active.requested().now_or_never().is_none());
control.interrupt("resp_a").unwrap();
control.interrupt("resp_a").unwrap();
assert_eq!(requested.now_or_never(), Some(()));
let retained_waiter = active.requested();
drop(active);
assert_eq!(
control.interrupt("resp_a"),
Err(ResponseInterruptError::Unavailable)
);
let next = control.activate("resp_b".to_owned()).unwrap();
assert!(next.requested().now_or_never().is_none());
drop(retained_waiter);
use async_trait::async_trait;
use futures::{FutureExt, executor::block_on};
use gateway_core::engine::response_control::{
ResponseControl, ResponseControlTransport, ResponseControlUnavailable,
};
#[derive(Default)]
struct Transport(Mutex<Vec<String>>);
#[async_trait]
impl ResponseControlTransport for Transport {
async fn send(&self, payload: &str) -> Result<(), ResponseControlUnavailable> {
self.0.lock().unwrap().push(payload.to_owned());
Ok(())
}
async fn receive(&self) -> Result<String, ResponseControlUnavailable> {
Ok("upstream reply".to_owned())
}
}
#[test]
fn matching_response_ids_do_not_share_control_between_root_executions() {
let first = ResponseControl::default();
let second = ResponseControl::default();
let active_first = first.activate("resp_same".to_owned()).unwrap();
let active_second = second.activate("resp_same".to_owned()).unwrap();
first.interrupt("resp_same").unwrap();
assert_eq!(active_first.requested().now_or_never(), Some(()));
assert!(active_second.requested().now_or_never().is_none());
fn control_requires_a_live_connection_owner_and_preserves_unknown_payloads() {
block_on(async {
let control = ResponseControl::default();
assert_eq!(
control.send("unknown").await,
Err(ResponseControlUnavailable)
);
assert!(control.receive().now_or_never().is_none());
let transport = Arc::new(Transport::default());
let owner: Arc<dyn ResponseControlTransport> = transport.clone();
control.bind(&owner);
let payload = "{ \"type\": \"future.control\", \"extension\": [1, 2] }";
control.send(payload).await.unwrap();
control.send(payload).await.unwrap();
assert_eq!(*transport.0.lock().unwrap(), [payload, payload]);
assert_eq!(control.receive().await.unwrap(), "upstream reply");
drop(owner);
drop(transport);
assert_eq!(control.send(payload).await, Err(ResponseControlUnavailable));
});
}
#[test]
fn replacing_or_clearing_a_binding_does_not_share_control_between_executions() {
block_on(async {
let first = ResponseControl::default();
let second = ResponseControl::default();
let old: Arc<dyn ResponseControlTransport> = Arc::new(Transport::default());
let current = Arc::new(Transport::default());
let owner: Arc<dyn ResponseControlTransport> = current.clone();
first.bind(&old);
second.bind(&owner);
drop(old);
assert_eq!(first.send("old").await, Err(ResponseControlUnavailable));
second.send("current").await.unwrap();
assert_eq!(*current.0.lock().unwrap(), ["current"]);
second.clear();
assert_eq!(
second.send("cleared").await,
Err(ResponseControlUnavailable)
);
});
}
-14
View File
@@ -81,8 +81,6 @@ pub struct HostConfig {
pub logging: LoggingConfig,
#[serde(default)]
pub system_update: SystemUpdateConfig,
#[serde(default = "default_drain_timeout_seconds")]
pub drain_timeout_seconds: u64,
#[serde(default = "default_worker_shutdown_timeout_seconds")]
pub worker_shutdown_timeout_seconds: u64,
}
@@ -110,9 +108,6 @@ impl HostConfig {
if self.listen.port == 0 {
return Err(ConfigError::InvalidField("host.listen.port"));
}
if self.drain_timeout_seconds == 0 {
return Err(ConfigError::InvalidField("host.drain_timeout_seconds"));
}
if self.worker_shutdown_timeout_seconds == 0 {
return Err(ConfigError::InvalidField(
"host.worker_shutdown_timeout_seconds",
@@ -136,11 +131,6 @@ impl HostConfig {
&self.runtime_data_dir
}
#[must_use]
pub const fn drain_timeout(&self) -> Duration {
Duration::from_secs(self.drain_timeout_seconds)
}
#[must_use]
pub const fn worker_shutdown_timeout(&self) -> Duration {
Duration::from_secs(self.worker_shutdown_timeout_seconds)
@@ -264,10 +254,6 @@ fn resolve_relative_path(base: &Path, path: &mut PathBuf) {
}
}
const fn default_drain_timeout_seconds() -> u64 {
30
}
const fn default_worker_shutdown_timeout_seconds() -> u64 {
30
}
+2 -3
View File
@@ -164,7 +164,7 @@ impl HostBundle {
Ok(())
}
/// 进程唯一阻塞点;返回前完成 HTTP drain 与 worker join
/// 进程唯一阻塞点;停止 HTTP 监听后完成 worker join
pub async fn serve(self, router: Router) -> Result<(), HostError> {
let result = serve_router(
router,
@@ -172,7 +172,6 @@ impl HostBundle {
self.config.listen.port,
self.cancellation.clone(),
Arc::clone(&self.connections),
self.config.drain_timeout(),
)
.await;
self.cancellation.cancel();
@@ -181,7 +180,7 @@ impl HostBundle {
.await;
match &result {
Ok(()) => {
tracing::info!(target: "gateway_shutdown", pid = std::process::id(), "HTTP 服务与后台任务已停止")
tracing::info!(target: "gateway_shutdown", pid = std::process::id(), "HTTP 监听已关闭,后台任务已结束")
}
Err(error) => {
tracing::error!(target: "gateway_shutdown", pid = std::process::id(), %error, "HTTP 服务异常停止,后台任务已清理")
+14 -74
View File
@@ -1,13 +1,12 @@
//! HTTP 监听、OS signal、原子连接注册与优雅 drain
//! HTTP 监听、关闭信号与停止时的原子连接注册
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::{Arc, OnceLock};
use std::time::Duration;
use axum::Router;
use gateway_core::lifecycle::CancellationToken;
use gateway_core::lifecycle::{ConnectionDraining, ConnectionGuard, ConnectionLifecycle};
use tokio::sync::Notify;
const DRAINING_BIT: usize = 1usize << (usize::BITS - 1);
const ACTIVE_MASK: usize = !DRAINING_BIT;
@@ -24,7 +23,6 @@ pub struct ConnectionTracker {
struct ConnectionState {
value: AtomicUsize,
idle: Notify,
}
impl ConnectionTracker {
@@ -33,36 +31,14 @@ impl ConnectionTracker {
Self {
state: Arc::new(ConnectionState {
value: AtomicUsize::new(0),
idle: Notify::new(),
}),
cancellation,
}
}
pub(crate) fn begin_draining(&self) {
let previous = self.state.value.fetch_or(DRAINING_BIT, Ordering::AcqRel);
self.state.value.fetch_or(DRAINING_BIT, Ordering::AcqRel);
self.cancellation.cancel();
if previous & ACTIVE_MASK == 0 {
self.state.idle.notify_waiters();
}
}
pub async fn wait_until_idle(&self, timeout: Duration) {
let wait = async {
loop {
let notified = self.state.idle.notified();
tokio::pin!(notified);
// notify_waiters 不保存许可:必须先 enable 注册等待者,再读
// 活跃计数;否则最后一个 guard 在读数与首次 poll 之间 drop
// 时唤醒丢失,只能等满整个超时
notified.as_mut().enable();
if self.state.value.load(Ordering::Acquire) & ACTIVE_MASK == 0 {
return;
}
notified.await;
}
};
let _ = tokio::time::timeout(timeout, wait).await;
}
}
@@ -106,10 +82,7 @@ impl ConnectionGuard for ActiveConnection {}
impl Drop for ActiveConnection {
fn drop(&mut self) {
let previous = self.state.value.fetch_sub(1, Ordering::AcqRel);
if previous & ACTIVE_MASK == 1 {
self.state.idle.notify_waiters();
}
self.state.value.fetch_sub(1, Ordering::AcqRel);
}
}
@@ -140,59 +113,26 @@ pub(crate) async fn serve_router(
port: u16,
cancellation: CancellationToken,
connections: Arc<ConnectionTracker>,
drain_timeout: Duration,
) -> Result<(), ServeError> {
let listener = bind_listener(&format!("{host}:{port}"))
.await
.map_err(ServeError::Bind)?;
tracing::info!(target: "gateway_startup", pid = std::process::id(), host, port, "网关开始监听");
let shutdown_cancellation = cancellation.clone();
let shutdown_connections = Arc::clone(&connections);
let shutdown = async move {
wait_for_shutdown(&shutdown_cancellation).await;
shutdown_connections.begin_draining();
};
// axum 的优雅关闭会无限等待存量连接结束,而 SSE 等长响应体不观察
// 取消信号:慢客户端可以把关闭拖住直到被 SIGKILL
// 整个 drain(axum
// 优雅关闭 + 游离 guard 等待)共享同一个从取消信号起算的绝对截止点,
// 逾期即放弃等待,存量连接随进程退出终止
let drain_deadline_at = Arc::new(OnceLock::new());
let drain_deadline = {
let cancellation = cancellation.clone();
let deadline_at = Arc::clone(&drain_deadline_at);
async move {
cancellation.cancelled().await;
let deadline = tokio::time::Instant::now() + drain_timeout;
let _ = deadline_at.set(deadline);
tokio::time::sleep_until(deadline).await;
}
};
// 所有关闭入口都停止等待长请求;丢弃 serve 关闭监听,存量连接随进程退出终止
// 后台落盘由 Host 的 Worker 关闭阶段独立收尾
let serve = axum::serve(
listener,
router.into_make_service_with_connect_info::<std::net::SocketAddr>(),
)
.with_graceful_shutdown(shutdown);
tokio::select! {
);
let result = tokio::select! {
biased;
result = serve => {
result.map_err(ServeError::Serve)?;
connections.begin_draining();
let remaining = drain_deadline_at.get().map_or(drain_timeout, |deadline| {
deadline.saturating_duration_since(tokio::time::Instant::now())
});
connections.wait_until_idle(remaining).await;
}
() = drain_deadline => {
tracing::warn!(
target: "gateway_shutdown",
timeout_secs = drain_timeout.as_secs(),
"优雅 drain 超时,放弃等待存量连接"
);
}
}
Ok(())
() = wait_for_shutdown(&cancellation) => Ok(()),
result = serve => result.map_err(ServeError::Serve),
};
connections.begin_draining();
tracing::info!(target: "gateway_shutdown", "停止等待存量连接,后台落盘完成后退出进程");
result
}
pub(crate) async fn wait_for_shutdown(cancellation: &CancellationToken) {
@@ -538,6 +538,7 @@ impl SystemOperations for ProcessSystemOperations {
let cancellation = self.cancellation.clone();
drop(tokio::spawn(async move {
tokio::time::sleep(Duration::from_millis(500)).await;
// 留出受理响应返回时间,再走统一关闭流程
cancellation.cancel();
}));
Ok(SystemOperationAccepted::Restart {
+2 -3
View File
@@ -127,9 +127,9 @@ fn host_config_derives_update_paths_from_runtime_data_dir() {
}
#[test]
fn host_config_rejects_zero_drain_window() {
fn host_config_rejects_zero_worker_shutdown_window() {
let mut config = valid_config();
config.drain_timeout_seconds = 0;
config.worker_shutdown_timeout_seconds = 0;
assert!(
config
@@ -223,7 +223,6 @@ fn valid_config() -> HostConfig {
request_dump_retention_days: 1,
},
system_update,
drain_timeout_seconds: 30,
worker_shutdown_timeout_seconds: 30,
}
}
@@ -51,7 +51,6 @@ fn logging_requires_at_least_one_sink() {
request_dump_retention_days: 1,
},
system_update: SystemUpdateConfig::default(),
drain_timeout_seconds: 30,
worker_shutdown_timeout_seconds: 30,
};
@@ -329,7 +328,6 @@ fn logging_config(directory: PathBuf, request_dump: bool) -> HostConfig {
request_dump_retention_days: 1,
},
system_update: SystemUpdateConfig::default(),
drain_timeout_seconds: 30,
worker_shutdown_timeout_seconds: 30,
}
}
+127 -118
View File
@@ -1,4 +1,4 @@
//! 验证宿主服务关闭时的连接排空、超时与并发唤醒
//! 验证统一关闭入口跳过会话等待、保留后台落盘及清理超时
use std::sync::Arc;
use std::sync::Mutex;
@@ -6,17 +6,19 @@ use std::time::Duration;
use axum::{Router, routing::get};
use futures::future::BoxFuture;
use gateway_admin::ports::system::{
SystemOperationError, SystemRestartPreflight, SystemUpdateCandidate,
};
use gateway_core::diagnostics::{OperationalDiagnostics, OperationalFailure};
use gateway_core::lifecycle::CancellationToken;
use gateway_core::lifecycle::ConnectionLifecycle as _;
use gateway_core::task::{
DaemonRestartPolicy, DaemonTask, WorkerContribution, WorkerId, WorkerKind,
WorkerLeaderLeasePort, WorkerLeaseAcquisition, WorkerLeaseError, WorkerLeaseRequest,
WorkerRegistration, WorkerRunnable, WorkerTaskError,
};
use gateway_host::HostBundle;
use gateway_host::serve::{ConnectionTracker, bind_listener};
use tokio::io::{AsyncReadExt as _, AsyncWriteExt as _};
use gateway_host::serve::bind_listener;
use tokio::io::AsyncWriteExt as _;
use tokio::sync::{Semaphore, mpsc};
use crate::support::host::{configuration, run_in_child};
@@ -28,65 +30,6 @@ fn host_bundle_serve_is_a_consuming_process_entrypoint() {
assert_eq!(std::mem::size_of_val(&_serve), 0);
}
#[tokio::test(start_paused = true)]
async fn wait_until_idle_should_return_immediately_without_active_connections() {
let tracker = ConnectionTracker::new(CancellationToken::new());
let started = tokio::time::Instant::now();
tracker.wait_until_idle(Duration::from_secs(30)).await;
assert_eq!(started.elapsed(), Duration::ZERO);
}
#[tokio::test(start_paused = true)]
async fn wait_until_idle_should_give_up_at_timeout_while_connections_remain() {
let tracker = ConnectionTracker::new(CancellationToken::new());
let _guard = tracker.try_register().expect("register connection");
let started = tokio::time::Instant::now();
tracker.wait_until_idle(Duration::from_secs(30)).await;
assert_eq!(started.elapsed(), Duration::from_secs(30));
}
#[tokio::test]
async fn wait_until_idle_should_wake_when_last_guard_drops_after_first_poll() {
let tracker = ConnectionTracker::new(CancellationToken::new());
let guard = tracker.try_register().expect("register connection");
let mut wait = Box::pin(tracker.wait_until_idle(Duration::from_secs(30)));
assert!(futures::poll!(wait.as_mut()).is_pending());
drop(guard);
tokio::time::timeout(Duration::from_secs(2), wait)
.await
.expect("woken by last guard drop instead of waiting out the timeout");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn wait_until_idle_should_observe_guard_drop_racing_the_idle_check() {
// 回归约束:notified 必须在读取活跃计数前完成注册(enable),否则最后
// 一个 guard 在计数检查与首次 poll 之间 drop 时唤醒丢失,只能等满整个
// 超时
// 多线程反复交错,任何一次丢唤醒都会撞上 2s 超时并触发断言
for _ in 0..256 {
let tracker = Arc::new(ConnectionTracker::new(CancellationToken::new()));
let guard = tracker.try_register().expect("register connection");
let waiter = tokio::spawn({
let tracker = Arc::clone(&tracker);
async move { tracker.wait_until_idle(Duration::from_secs(2)).await }
});
let dropper = std::thread::spawn(move || drop(guard));
let started = std::time::Instant::now();
waiter.await.expect("waiter completes");
dropper.join().expect("dropper completes");
assert!(
started.elapsed() < Duration::from_millis(1_900),
"idle wakeup was lost and the waiter slept until the drain timeout"
);
}
}
#[tokio::test]
async fn bind_listener_should_retry_until_previous_listener_releases_port() {
let holder = tokio::net::TcpListener::bind("127.0.0.1:0")
@@ -110,76 +53,148 @@ async fn bind_listener_should_retry_until_previous_listener_releases_port() {
}
#[test]
fn http_drain_keeps_writer_running_until_the_request_finishes() {
fn cancellation_skips_connections_but_flushes_and_joins_workers() {
run_in_child(
"serve::http_drain_keeps_writer_running_until_the_request_finishes",
async {
let directory = tempfile::tempdir().unwrap();
let (host, address) = listening_host(directory.path()).await;
let cancellation = host.cancellation();
let connections = host.connection_lifecycle();
let probe = start_workers(&host, false);
probe.started.cancelled().await;
let (router, entered, finish_request) = request_router(&host, &probe);
let server = tokio::spawn(host.serve(router));
let mut client = connect_request(address).await;
entered.cancelled().await;
cancellation.cancel();
wait_for_draining(connections.as_ref()).await;
assert!(connections.try_register().is_err());
assert!(
tokio::time::timeout(Duration::from_millis(50), probe.cancelled.cancelled())
.await
.is_err(),
"writer cancelled before the in-flight request finished"
);
finish_request.add_permits(1);
let mut response = String::new();
client.read_to_string(&mut response).await.unwrap();
server.await.unwrap().unwrap();
assert!(response.starts_with("HTTP/1.1 200 OK"));
assert_eq!(*probe.writes.lock().unwrap(), [1]);
assert!(probe.finished.is_cancelled());
},
"serve::cancellation_skips_connections_but_flushes_and_joins_workers",
shutdown_with_pending_request(ShutdownTrigger::Cancellation),
);
}
#[test]
fn drain_timeout_starts_a_separate_worker_shutdown_phase() {
fn self_restart_skips_connections_but_flushes_and_joins_workers() {
run_in_child(
"serve::drain_timeout_starts_a_separate_worker_shutdown_phase",
"serve::self_restart_skips_connections_but_flushes_and_joins_workers",
shutdown_with_pending_request(ShutdownTrigger::SelfRestart),
);
}
#[cfg(unix)]
#[test]
fn sigterm_skips_connections_but_flushes_and_joins_workers() {
run_in_child(
"serve::sigterm_skips_connections_but_flushes_and_joins_workers",
shutdown_with_pending_request(ShutdownTrigger::Signal("-TERM")),
);
}
#[cfg(unix)]
#[test]
fn sigint_skips_connections_but_flushes_and_joins_workers() {
run_in_child(
"serve::sigint_skips_connections_but_flushes_and_joins_workers",
shutdown_with_pending_request(ShutdownTrigger::Signal("-INT")),
);
}
enum ShutdownTrigger {
Cancellation,
SelfRestart,
#[cfg(unix)]
Signal(&'static str),
}
async fn shutdown_with_pending_request(trigger: ShutdownTrigger) {
let directory = tempfile::tempdir().unwrap();
let reservation = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
let address = reservation.local_addr().unwrap();
let mut config = configuration(directory.path());
config.listen.port = address.port();
config.system_update.deployment_mode = "docker".to_owned();
config.system_update.build_type = "source".to_owned();
config.system_update.self_restart_enabled = true;
config.system_update.update_lock_file = directory.path().join("update.lock");
let host = gateway_host::initialize(config).await.unwrap();
drop(reservation);
let system = host.system_operations();
let cancellation = host.cancellation();
let connections = host.connection_lifecycle();
let guard = connections.try_register().unwrap();
let probe = start_workers(&host, true);
probe.started.cancelled().await;
let (router, entered, _finish_request) = request_router(&host, &probe);
let server = tokio::spawn(host.serve(router));
let client = connect_request(address).await;
entered.cancelled().await;
probe.sender.send(7).unwrap();
match trigger {
ShutdownTrigger::Cancellation => cancellation.cancel(),
ShutdownTrigger::SelfRestart => {
system.restart(Arc::new(AllowRestart)).await.unwrap();
}
#[cfg(unix)]
ShutdownTrigger::Signal(signal) => {
// 真实请求进入后信号监听已被 poll,只向隔离的当前测试子进程发送信号
assert!(
std::process::Command::new("kill")
.args([signal, &std::process::id().to_string()])
.status()
.unwrap()
.success()
);
}
}
tokio::time::timeout(Duration::from_secs(2), probe.cancelled.cancelled())
.await
.expect("shutdown must not wait for the blocked HTTP request or live guard");
assert!(connections.is_draining());
assert!(connections.try_register().is_err());
drop(guard);
assert!(connections.try_register().is_err());
assert!(
tokio::net::TcpStream::connect(address).await.is_err(),
"listener must close before worker cleanup finishes"
);
assert!(
!server.is_finished(),
"worker cleanup still needs to finish"
);
probe.finish_shutdown.add_permits(1);
server.await.unwrap().unwrap();
assert_eq!(*probe.writes.lock().unwrap(), [7]);
assert!(probe.finished.is_cancelled());
// 请求仍未结束,独立进程退出时终止连接和挂起 handler
drop(client);
}
#[test]
fn shutdown_aborts_workers_that_exceed_the_cleanup_budget() {
run_in_child(
"serve::shutdown_aborts_workers_that_exceed_the_cleanup_budget",
async {
let directory = tempfile::tempdir().unwrap();
let (host, address) = listening_host(directory.path()).await;
let cancellation = host.cancellation();
let connections = host.connection_lifecycle();
let probe = start_workers(&host, true);
probe.started.cancelled().await;
let (router, entered, finish_request) = request_router(&host, &probe);
let (router, entered, _finish_request) = request_router(&host, &probe);
let server = tokio::spawn(host.serve(router));
let client = connect_request(address).await;
entered.cancelled().await;
let started = tokio::time::Instant::now();
cancellation.cancel();
wait_for_draining(connections.as_ref()).await;
probe.cancelled.cancelled().await;
assert!(started.elapsed() >= Duration::from_secs(1));
assert!(!server.is_finished(), "worker cleanup was not joined");
assert!(!probe.finished.is_cancelled());
probe.finish_shutdown.add_permits(1);
server.await.unwrap().unwrap();
assert!(probe.finished.is_cancelled());
// 超时后真实连接可以仍在 axum 任务中;释放测试端,避免遗留挂起 handler
tokio::time::timeout(Duration::from_secs(3), server)
.await
.expect("worker cleanup budget must bound process shutdown")
.unwrap()
.unwrap();
probe.dropped.cancelled().await;
assert!(probe.cancelled.is_cancelled());
assert!(!probe.finished.is_cancelled());
drop(client);
finish_request.add_permits(1);
},
);
}
struct AllowRestart;
#[async_trait::async_trait]
impl SystemRestartPreflight for AllowRestart {
async fn prepare(&self, _: Option<SystemUpdateCandidate>) -> Result<(), SystemOperationError> {
Ok(())
}
}
#[test]
fn bind_failure_cancels_and_joins_workers_before_returning() {
run_in_child(
@@ -291,12 +306,6 @@ async fn connect_request(address: std::net::SocketAddr) -> tokio::net::TcpStream
client
}
async fn wait_for_draining(connections: &dyn gateway_core::lifecycle::ConnectionLifecycle) {
while !connections.is_draining() {
tokio::task::yield_now().await;
}
}
fn request_router(
host: &HostBundle,
probe: &WriterProbe,
@@ -319,7 +328,7 @@ fn request_router(
let _connection = connections.try_register().unwrap();
entered.cancel();
finish.acquire().await.unwrap().forget();
// 对应请求结束后写入终态的时机;drain 超时后的写入允许被拒绝
// 对应请求结束后写入终态的时机;关闭开始后的写入允许被拒绝
if sender.send(1).is_ok() {
"recorded"
} else {
@@ -59,7 +59,6 @@ pub fn configuration(directory: &Path) -> HostConfig {
request_dump_retention_days: 1,
},
system_update: Default::default(),
drain_timeout_seconds: 1,
worker_shutdown_timeout_seconds: 1,
}
}
@@ -4,8 +4,8 @@
"version": 3,
"replacement_version": 4,
"introduced_in": "3.20.0",
"last_counted_in": "3.21.2",
"remaining_releases": 4,
"last_counted_in": "3.22.2",
"remaining_releases": 0,
"migration": "将执行设置和来源中的 disable_fast 改为 fast_mode,使用 default、enabled、disabled 三态,并将 middleware 声明升级到 v4"
},
{
@@ -13,8 +13,8 @@
"version": 1,
"replacement_version": 2,
"introduced_in": "3.20.0",
"last_counted_in": "3.21.2",
"remaining_releases": 4,
"last_counted_in": "3.22.2",
"remaining_releases": 0,
"migration": "使用新版 SDK 读取 fast_mode 三态,并将 upstream_adapter 声明升级到 v2"
}
]
@@ -28,6 +28,7 @@ use crate::{RpcSession, callback::PluginCallbacks};
const IDENTIFIER_TIMEOUT: Duration = Duration::from_secs(5);
#[derive(Clone)]
pub(crate) struct FrontendAuthenticationEntry {
identities: BTreeMap<String, ClientApiKeyId>,
exclusive: bool,
@@ -23,6 +23,7 @@ pub(crate) use registration::prepare;
pub(crate) const MAXIMUM_BODY_BYTES: usize = 1024 * 1024;
#[derive(Clone)]
pub(crate) struct ManagementEntry {
pub(crate) view: PluginManagementView,
routes: Vec<ManagementRoute>,
@@ -0,0 +1,156 @@
//! 选择挂载边界并用 Core 续体组合中间件链
use super::{MiddlewareEntry, PluginMiddlewarePlan};
use futures::future::BoxFuture;
use gateway_core::engine::middleware::{
MiddlewareContext, MiddlewareError, MiddlewareMount, MiddlewareNext, MiddlewarePlan,
MiddlewareRequest, MiddlewareResponse,
};
use gateway_core::middleware::{Middleware, Next};
use std::{sync::Arc, time::Duration};
// 协议适配只提供选择条件与单层调用;顺序、上下文捕获和续体拼接共用这一个入口
type Invoke<Context, Input, Output, Error> = fn(
Arc<MiddlewareEntry>,
Context,
Input,
Next<Input, Output, Error>,
Duration,
) -> BoxFuture<'static, Result<Output, Error>>;
impl PluginMiddlewarePlan {
fn chain<Context, Input, Output, Error>(
&self,
mount: MiddlewareMount,
context: Context,
select: impl Fn(&MiddlewareEntry, &Context) -> bool,
invoke: Invoke<Context, Input, Output, Error>,
next: Next<Input, Output, Error>,
) -> Next<Input, Output, Error>
where
Context: Clone + Send + 'static,
Input: Send + 'static,
Output: Send + 'static,
Error: Send + 'static,
{
let Some(entries) = self.middleware.get(&mount) else {
return next;
};
let layers = entries
.iter()
.filter(|entry| select(entry, &context))
.map(|entry| -> Middleware<Input, Output, Error> {
let entry = Arc::clone(entry);
let context = context.clone();
let timeout = self.middleware_timeout;
Box::new(move |input, next| invoke(entry, context, input, next, timeout))
})
.collect();
gateway_core::middleware::compose(layers, move |input| next.run(input))
}
}
impl MiddlewarePlan for PluginMiddlewarePlan {
fn has_service(&self) -> bool {
self.middleware.contains_key(&MiddlewareMount::Service)
}
fn handle_service(
&self,
context: gateway_core::engine::middleware::service::Context,
input: gateway_core::engine::middleware::service::Value,
next: gateway_core::engine::middleware::service::Next,
) -> BoxFuture<
'static,
Result<
gateway_core::engine::middleware::service::Value,
gateway_core::engine::middleware::service::Error,
>,
> {
self.chain(
MiddlewareMount::Service,
context,
|entry, context| !context.extensions.contains(&entry.instance_id),
|entry, context, input, next, timeout| {
Box::pin(super::service::invoke(entry, context, input, next, timeout))
},
next,
)
.run(input)
}
fn has_websocket(&self) -> bool {
self.middleware.contains_key(&MiddlewareMount::WebSocket)
}
fn handle_websocket(
&self,
context: gateway_core::engine::middleware::websocket::Context,
message: gateway_core::engine::middleware::websocket::Message,
next: gateway_core::engine::middleware::websocket::Next,
) -> BoxFuture<
'static,
Result<Option<gateway_core::engine::middleware::websocket::Message>, MiddlewareError>,
> {
self.chain(
MiddlewareMount::WebSocket,
context,
|_entry, _context| true,
|entry, context, message, next, timeout| {
Box::pin(super::websocket::invoke(
entry, context, message, next, timeout,
))
},
next,
)
.run(message)
}
fn has_http(&self) -> bool {
self.middleware.contains_key(&MiddlewareMount::Http)
}
fn handle_http(
&self,
context: gateway_core::engine::middleware::http::Context,
request: gateway_core::engine::middleware::http::Request,
next: gateway_core::engine::middleware::http::Next,
) -> BoxFuture<'static, Result<gateway_core::engine::middleware::http::Response, MiddlewareError>>
{
self.chain(
MiddlewareMount::Http,
context,
|entry, context| !context.extensions.contains(&entry.instance_id),
|entry, context, request, next, timeout| {
Box::pin(super::http::invoke(entry, context, request, next, timeout))
},
next,
)
.run(request)
}
fn handle(
&self,
context: MiddlewareContext,
request: MiddlewareRequest,
next: MiddlewareNext,
) -> BoxFuture<'static, Result<MiddlewareResponse, MiddlewareError>> {
self.chain(
context.mount(),
context,
|entry, context| {
!context.extension_scope().contains(&entry.instance_id)
&& entry.scope.matches_processing(
context.client_key_id(),
context.account_group_ids(),
context.provider(),
context.model(),
)
},
|entry, context, request, next, timeout| {
Box::pin(super::request::invoke(
entry, context, request, next, timeout,
))
},
next,
)
.run(request)
}
}
@@ -12,7 +12,7 @@ use gateway_plugin_sdk::{
use http_body::Frame;
use http_body_util::{BodyExt as _, StreamBody};
use super::super::MiddlewareEntry;
use super::MiddlewareEntry;
use crate::callback::http_middleware::{Invocation, headers};
pub(super) async fn invoke(
@@ -0,0 +1,207 @@
//! 编译各挂载边界的中间件绑定,组合为独立执行计划
mod execution;
mod http;
mod request;
mod service;
mod websocket;
use crate::{RpcSession, adapter::scope::BindingScope, callback::PluginCallbacks};
use gateway_admin::model::{
AdminError,
plugins::instances::{PluginCapabilityBinding, PluginFailurePolicy, PluginInstance},
};
use gateway_core::engine::middleware::MiddlewareMount;
use gateway_plugin_sdk::{Capability, Manifest, Stage};
use std::{
collections::{BTreeMap, BTreeSet},
fmt,
sync::Arc,
time::Duration,
};
#[derive(Clone)]
struct MiddlewareInvocationPorts {
session: Arc<RpcSession>,
callbacks: Arc<PluginCallbacks>,
}
#[derive(Clone)]
pub(crate) struct MiddlewareEntry {
order: i32,
plugin_id: String,
instance_id: String,
mount: MiddlewareMount,
invocation: Option<MiddlewareInvocationPorts>,
scope: BindingScope,
failure_policy: PluginFailurePolicy,
}
pub(crate) fn validate_bindings(
manifest: &Manifest,
bindings: &[PluginCapabilityBinding],
) -> Result<(), AdminError> {
let mut stages = BTreeSet::new();
for binding in bindings {
if crate::contribution::resolve(manifest, binding)?.capability != Capability::Middleware {
continue;
}
let stage: Stage = serde_json::from_value(serde_json::Value::String(binding.stage.clone()))
.map_err(|_| AdminError::invalid("插件中间件阶段无效"))?;
if !matches!(
binding.failure_policy,
PluginFailurePolicy::Reject | PluginFailurePolicy::Delegate
) {
return Err(AdminError::invalid("数据面绑定的故障策略无效"));
}
if !matches!(
stage,
Stage::Http | Stage::WebSocket | Stage::Service | Stage::Request | Stage::Attempt
) || !stages.insert(stage)
{
return Err(AdminError::invalid(
"同一插件实例的中间件只能在 http/websocket/service/request/attempt 各绑定一次",
));
}
if matches!(stage, Stage::Http | Stage::WebSocket | Stage::Service)
&& (!binding.client_key_ids.is_empty()
|| !binding.account_group_ids.is_empty()
|| !binding.provider_ids.is_empty()
|| !binding.models.is_empty())
{
return Err(AdminError::invalid(
"入口和公开服务不保证具有模型执行身份,不能绑定 Key、分组、Provider 或模型条件",
));
}
if BindingScope::compile(binding)?.has_provider_condition() && stage == Stage::Request {
return Err(AdminError::invalid(
"Provider 尚未冻结的阶段不能绑定 Provider 条件",
));
}
}
Ok(())
}
pub(crate) fn compile_entries(
manifest: &Manifest,
instance_id: &str,
bindings: &[PluginCapabilityBinding],
session: Arc<RpcSession>,
callbacks: Arc<PluginCallbacks>,
) -> Result<Vec<MiddlewareEntry>, AdminError> {
validate_bindings(manifest, bindings)?;
let plugin_id = manifest
.plugin_id()
.map_err(|_| AdminError::invalid("插件身份无效"))?;
let invocation = MiddlewareInvocationPorts { session, callbacks };
bindings
.iter()
.filter_map(|binding| {
let capability = match crate::contribution::resolve(manifest, binding) {
Ok(contribution) => contribution.capability,
Err(error) => return Some(Err(error)),
};
(capability == Capability::Middleware)
.then(|| compile_entry(&plugin_id, instance_id, binding, Some(invocation.clone())))
})
.collect()
}
/// 进程不可用时保留原绑定的范围与故障策略
pub(crate) fn unavailable_entries(
instance: &PluginInstance,
) -> Result<Vec<MiddlewareEntry>, AdminError> {
instance
.bindings
.iter()
.filter(|binding| {
matches!(
binding.stage.as_str(),
"http" | "websocket" | "service" | "request" | "attempt"
)
})
.map(|binding| {
compile_entry(
binding
.contribution
.rsplit_once('.')
.map_or(binding.contribution.as_str(), |(plugin, _)| plugin),
&instance.id,
binding,
None,
)
})
.collect()
}
fn compile_entry(
plugin_id: &str,
instance_id: &str,
binding: &PluginCapabilityBinding,
invocation: Option<MiddlewareInvocationPorts>,
) -> Result<MiddlewareEntry, AdminError> {
let scope = BindingScope::compile(binding)?;
Ok(MiddlewareEntry {
order: binding.order,
plugin_id: plugin_id.to_owned(),
instance_id: instance_id.to_owned(),
mount: match binding.stage.as_str() {
"http" => MiddlewareMount::Http,
"service" => MiddlewareMount::Service,
"websocket" => MiddlewareMount::WebSocket,
"request" => MiddlewareMount::Request,
"attempt" => MiddlewareMount::Attempt,
_ => return Err(AdminError::invalid("插件中间件阶段无效")),
},
invocation,
scope,
failure_policy: binding.failure_policy.clone(),
})
}
pub(crate) struct PluginMiddlewarePlan {
middleware: BTreeMap<MiddlewareMount, Box<[Arc<MiddlewareEntry>]>>,
middleware_timeout: Duration,
}
impl fmt::Debug for PluginMiddlewarePlan {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("PluginMiddlewarePlan")
.field("mounts", &self.middleware.keys())
.finish_non_exhaustive()
}
}
impl PluginMiddlewarePlan {
pub(crate) fn compile(
mut entries: Vec<MiddlewareEntry>,
maximum_call_timeout: Duration,
) -> Option<Arc<Self>> {
if entries.is_empty() {
return None;
}
entries.sort_by(|left, right| {
(left.order, &left.plugin_id, &left.instance_id).cmp(&(
right.order,
&right.plugin_id,
&right.instance_id,
))
});
// 发布时固定各挂载位置的有序候选,消息处理不再扫描其他边界的绑定
let mut by_mount = BTreeMap::<_, Vec<_>>::new();
for entry in entries {
by_mount
.entry(entry.mount)
.or_default()
.push(Arc::new(entry));
}
Some(Arc::new(Self {
middleware: by_mount
.into_iter()
.map(|(mount, entries)| (mount, entries.into_boxed_slice()))
.collect(),
middleware_timeout: maximum_call_timeout,
}))
}
}
@@ -1,4 +1,4 @@
//! 插件中间件链的执行、失败回退与响应正文生命周期管理
//! 请求与尝试中间件的 RPC 调用、失败回退和响应正文生命周期
use std::{
sync::{Arc, Mutex},
@@ -10,9 +10,8 @@ use futures::future::BoxFuture;
use gateway_admin::model::plugins::instances::PluginFailurePolicy;
use gateway_core::engine::middleware::{
MiddlewareBody, MiddlewareContext, MiddlewareError, MiddlewareFrame, MiddlewareFraming,
MiddlewareMount, MiddlewareNext, MiddlewarePlan, MiddlewareRequest, MiddlewareResponse,
MiddlewareMount, MiddlewareNext, MiddlewareRequest, MiddlewareResponse,
};
use gateway_core::middleware::{Middleware, Next};
use gateway_plugin_sdk::{
ErrorCode, SendState, Stage,
call::middleware::{
@@ -28,156 +27,12 @@ use crate::{
callback::{MiddlewareBodyAuthority, MiddlewareCompletionBody, MiddlewareInvocation},
};
use super::super::{MiddlewareEntry, PluginRequestPolicyPlan};
use super::MiddlewareEntry;
const MAXIMUM_MAPPED_FRAMES_PER_SOURCE: usize = 64;
const MAXIMUM_MAPPED_BYTES_PER_SOURCE: usize = 8 * 1024 * 1024;
// 协议适配只提供选择条件与单层调用;顺序、上下文捕获和续体拼接共用这一个入口
type Invoke<Context, Input, Output, Error> = fn(
Arc<MiddlewareEntry>,
Context,
Input,
Next<Input, Output, Error>,
Duration,
) -> BoxFuture<'static, Result<Output, Error>>;
impl PluginRequestPolicyPlan {
fn chain<Context, Input, Output, Error>(
&self,
mount: MiddlewareMount,
context: Context,
select: impl Fn(&MiddlewareEntry, &Context) -> bool,
invoke: Invoke<Context, Input, Output, Error>,
next: Next<Input, Output, Error>,
) -> Next<Input, Output, Error>
where
Context: Clone + Send + 'static,
Input: Send + 'static,
Output: Send + 'static,
Error: Send + 'static,
{
let Some(entries) = self.middleware.get(&mount) else {
return next;
};
let layers = entries
.iter()
.filter(|entry| select(entry, &context))
.map(|entry| -> Middleware<Input, Output, Error> {
let entry = Arc::clone(entry);
let context = context.clone();
let timeout = self.middleware_timeout;
Box::new(move |input, next| invoke(entry, context, input, next, timeout))
})
.collect();
gateway_core::middleware::compose(layers, move |input| next.run(input))
}
}
impl MiddlewarePlan for PluginRequestPolicyPlan {
fn has_service(&self) -> bool {
self.middleware.contains_key(&MiddlewareMount::Service)
}
fn handle_service(
&self,
context: gateway_core::engine::middleware::service::Context,
input: gateway_core::engine::middleware::service::Value,
next: gateway_core::engine::middleware::service::Next,
) -> BoxFuture<
'static,
Result<
gateway_core::engine::middleware::service::Value,
gateway_core::engine::middleware::service::Error,
>,
> {
self.chain(
MiddlewareMount::Service,
context,
|entry, context| !context.extensions.contains(&entry.instance_id),
|entry, context, input, next, timeout| {
Box::pin(super::service::invoke(entry, context, input, next, timeout))
},
next,
)
.run(input)
}
fn has_websocket(&self) -> bool {
self.middleware.contains_key(&MiddlewareMount::WebSocket)
}
fn handle_websocket(
&self,
context: gateway_core::engine::middleware::websocket::Context,
message: gateway_core::engine::middleware::websocket::Message,
next: gateway_core::engine::middleware::websocket::Next,
) -> BoxFuture<
'static,
Result<Option<gateway_core::engine::middleware::websocket::Message>, MiddlewareError>,
> {
self.chain(
MiddlewareMount::WebSocket,
context,
|_entry, _context| true,
|entry, context, message, next, timeout| {
Box::pin(super::websocket::invoke(
entry, context, message, next, timeout,
))
},
next,
)
.run(message)
}
fn has_http(&self) -> bool {
self.middleware.contains_key(&MiddlewareMount::Http)
}
fn handle_http(
&self,
context: gateway_core::engine::middleware::http::Context,
request: gateway_core::engine::middleware::http::Request,
next: gateway_core::engine::middleware::http::Next,
) -> BoxFuture<'static, Result<gateway_core::engine::middleware::http::Response, MiddlewareError>>
{
self.chain(
MiddlewareMount::Http,
context,
|entry, context| !context.extensions.contains(&entry.instance_id),
|entry, context, request, next, timeout| {
Box::pin(super::http::invoke(entry, context, request, next, timeout))
},
next,
)
.run(request)
}
fn handle(
&self,
context: MiddlewareContext,
request: MiddlewareRequest,
next: MiddlewareNext,
) -> BoxFuture<'static, Result<MiddlewareResponse, MiddlewareError>> {
self.chain(
context.mount(),
context,
|entry, context| {
!context.extension_scope().contains(&entry.instance_id)
&& entry.scope.matches_processing(
context.client_key_id(),
context.account_group_ids(),
context.provider(),
context.model(),
)
},
|entry, context, request, next, timeout| {
Box::pin(invoke_middleware(entry, context, request, next, timeout))
},
next,
)
.run(request)
}
}
async fn invoke_middleware(
pub(super) async fn invoke(
entry: Arc<MiddlewareEntry>,
context: MiddlewareContext,
request: MiddlewareRequest,
@@ -612,7 +467,11 @@ impl MiddlewareBody for PluginMiddlewareBody {
&& frame
.event()
.is_some_and(|event| event.wire_event().is_none());
if !facts_only && !valid_plugin_frame(self.framing, frame.bytes()) {
// 来源句柄已核对正文与交付归属,未改写的原始内容不依赖 JSON 解析
if frame.transformed()
&& !facts_only
&& !valid_plugin_frame(self.framing, frame.bytes())
{
return Err(invalid());
}
if self.pending_transformed {
@@ -2,7 +2,7 @@
use std::sync::Arc;
use super::super::MiddlewareEntry;
use super::MiddlewareEntry;
use crate::callback::services::Invocation;
use gateway_admin::model::plugins::instances::PluginFailurePolicy;
use gateway_core::engine::middleware::service as core;
@@ -2,7 +2,7 @@
use std::sync::Arc;
use super::super::MiddlewareEntry;
use super::MiddlewareEntry;
use crate::callback::websocket_middleware::Invocation;
use gateway_admin::model::plugins::instances::PluginFailurePolicy;
use gateway_core::{engine::middleware::MiddlewareError, engine::middleware::websocket as core};
@@ -4,6 +4,7 @@ pub(super) mod catalog;
pub(super) mod command_line;
pub(super) mod frontend_authentication;
pub(super) mod management;
pub(super) mod middleware;
pub(super) mod observer;
pub(super) mod policy;
pub(super) mod scope;
@@ -34,6 +34,7 @@ use crate::{RpcSession, adapter::scope::BindingScope};
const OBSERVATION_TIMEOUT: Duration = Duration::from_secs(2);
const OBSERVATION_ENVELOPE_RESERVE: usize = 16 * 1024;
#[derive(Clone)]
pub(crate) struct ObserverEntry {
order: i32,
plugin_id: String,
@@ -446,6 +447,9 @@ async fn dispatch_websocket_queue(
}
fn websocket_payload(observation: &WebSocketResponseObservation) -> Result<Vec<u8>, ()> {
if let Some(raw) = observation.wire().raw_websocket_message() {
return Ok(raw.as_bytes().to_vec());
}
if let Some(raw) = observation.wire().raw_json_body() {
return Ok(raw.to_vec());
}
@@ -553,6 +557,13 @@ fn wire_timings(observation: &RequestObservation) -> RequestTimings {
first_text_ms: timings.first_text_ms,
first_token_ms: timings.first_token_ms,
provider_processing_ms: timings.provider_processing_ms,
upstream_response_ms: timings.upstream_response_ms,
upstream_api_overhead_ms: timings.upstream_api_overhead_ms,
upstream_engine_ms: timings.upstream_engine_ms,
upstream_engine_iapi_ttft_ms: timings.upstream_engine_iapi_ttft_ms,
upstream_engine_service_ttft_ms: timings.upstream_engine_service_ttft_ms,
upstream_engine_iapi_tbt_ms: timings.upstream_engine_iapi_tbt_ms,
upstream_engine_service_tbt_ms: timings.upstream_engine_service_tbt_ms,
latency_ms: timings.latency_ms,
}
}
@@ -1,6 +0,0 @@
//! HTTP、WebSocket、逻辑请求与服务中间件的协议适配入口
mod execution;
mod http;
mod service;
mod websocket;
@@ -1,27 +1,25 @@
//! 编译插件请求策略与中间件绑定,组合为请求级执行计划
//! 编译模型路由、账号调度与重试策略的请求级计划
mod middleware;
mod retry;
mod route_schedule;
use std::{collections::BTreeMap, fmt, sync::Arc, time::Duration};
use std::{fmt, sync::Arc, time::Duration};
use gateway_admin::model::{
AdminError,
plugins::instances::{PluginCapabilityBinding, PluginFailurePolicy, PluginInstance},
};
use gateway_core::engine::middleware::MiddlewareMount;
use gateway_plugin_sdk::{Capability, Manifest, Stage};
use crate::{RpcSession, adapter::scope::BindingScope, callback::PluginCallbacks};
const POLICY_TIMEOUT: Duration = Duration::from_secs(2);
#[derive(Clone)]
pub(crate) enum PolicyEntry {
Router(ModelRouterEntry),
Scheduler(AccountSchedulerEntry),
Retry(RetryEntry),
Middleware(MiddlewareEntry),
}
#[derive(Clone)]
@@ -30,6 +28,7 @@ struct PolicyInvocation {
callbacks: Arc<PluginCallbacks>,
}
#[derive(Clone)]
pub(crate) struct RetryEntry {
order: i32,
plugin_id: String,
@@ -38,6 +37,7 @@ pub(crate) struct RetryEntry {
scope: BindingScope,
}
#[derive(Clone)]
pub(crate) struct ModelRouterEntry {
order: i32,
plugin_id: String,
@@ -47,6 +47,7 @@ pub(crate) struct ModelRouterEntry {
failure_policy: PluginFailurePolicy,
}
#[derive(Clone)]
pub(crate) struct AccountSchedulerEntry {
plugin_id: String,
instance_id: String,
@@ -55,16 +56,6 @@ pub(crate) struct AccountSchedulerEntry {
failure_policy: PluginFailurePolicy,
}
pub(crate) struct MiddlewareEntry {
order: i32,
plugin_id: String,
instance_id: String,
mount: MiddlewareMount,
invocation: Option<PolicyInvocation>,
scope: BindingScope,
failure_policy: PluginFailurePolicy,
}
pub(crate) fn validate_bindings(
manifest: &Manifest,
bindings: &[PluginCapabilityBinding],
@@ -72,15 +63,11 @@ pub(crate) fn validate_bindings(
let mut has_router = false;
let mut has_scheduler = false;
let mut has_retry = false;
let mut middleware_stages = std::collections::BTreeSet::new();
for binding in bindings {
let capability = crate::contribution::resolve(manifest, binding)?.capability;
if !matches!(
capability,
Capability::ModelRouter
| Capability::Scheduler
| Capability::Middleware
| Capability::RetryPolicy
Capability::ModelRouter | Capability::Scheduler | Capability::RetryPolicy
) {
continue;
}
@@ -117,38 +104,10 @@ pub(crate) fn validate_bindings(
));
}
}
Capability::Middleware => {
if !matches!(
stage,
Stage::Http
| Stage::WebSocket
| Stage::Service
| Stage::Request
| Stage::Attempt
) || !middleware_stages.insert(stage)
{
return Err(AdminError::invalid(
"同一插件实例的中间件只能在 http/websocket/service/request/attempt 各绑定一次",
));
}
}
_ => unreachable!("capability was filtered above"),
}
if matches!(stage, Stage::Http | Stage::WebSocket | Stage::Service)
&& (!binding.client_key_ids.is_empty()
|| !binding.account_group_ids.is_empty()
|| !binding.provider_ids.is_empty()
|| !binding.models.is_empty())
{
return Err(AdminError::invalid(
"入口和公开服务不保证具有模型执行身份,不能绑定 Key、分组、Provider 或模型条件",
));
}
let scope = BindingScope::compile(binding)?;
if (capability == Capability::ModelRouter
|| (capability == Capability::Middleware && stage == Stage::Request))
&& scope.has_provider_condition()
{
if capability == Capability::ModelRouter && scope.has_provider_condition() {
return Err(AdminError::invalid(
"Provider 尚未冻结的阶段不能绑定 Provider 条件",
));
@@ -178,10 +137,7 @@ pub(crate) fn compile_entries(
};
matches!(
capability,
Capability::ModelRouter
| Capability::Scheduler
| Capability::Middleware
| Capability::RetryPolicy
Capability::ModelRouter | Capability::Scheduler | Capability::RetryPolicy
)
.then(|| {
compile_entry(
@@ -208,7 +164,6 @@ pub(crate) fn unavailable_entries(
"routing" => Capability::ModelRouter,
"scheduling" => Capability::Scheduler,
"retry" => Capability::RetryPolicy,
"http" | "websocket" | "service" | "request" | "attempt" => Capability::Middleware,
_ => return None,
};
Some(compile_entry(
@@ -256,22 +211,6 @@ fn compile_entry(
scope,
failure_policy: binding.failure_policy.clone(),
}),
Capability::Middleware => PolicyEntry::Middleware(MiddlewareEntry {
order: binding.order,
plugin_id: plugin_id.to_owned(),
instance_id: instance_id.to_owned(),
mount: match binding.stage.as_str() {
"http" => MiddlewareMount::Http,
"service" => MiddlewareMount::Service,
"websocket" => MiddlewareMount::WebSocket,
"request" => MiddlewareMount::Request,
"attempt" => MiddlewareMount::Attempt,
_ => return Err(AdminError::invalid("插件中间件阶段无效")),
},
invocation,
scope,
failure_policy: binding.failure_policy.clone(),
}),
_ => return Err(AdminError::invalid("插件请求策略能力无效")),
})
}
@@ -280,9 +219,7 @@ pub(crate) struct PluginRequestPolicyPlan {
routers: Arc<[ModelRouterEntry]>,
schedulers: Arc<[AccountSchedulerEntry]>,
retries: Arc<[RetryEntry]>,
middleware: BTreeMap<MiddlewareMount, Box<[Arc<MiddlewareEntry>]>>,
policy_timeout: Duration,
middleware_timeout: Duration,
}
impl fmt::Debug for PluginRequestPolicyPlan {
@@ -292,14 +229,6 @@ impl fmt::Debug for PluginRequestPolicyPlan {
.field("router_count", &self.routers.len())
.field("scheduler_count", &self.schedulers.len())
.field("retry_count", &self.retries.len())
.field(
"middleware_count",
&self
.middleware
.values()
.map(|entries| entries.len())
.sum::<usize>(),
)
.finish_non_exhaustive()
}
}
@@ -312,20 +241,14 @@ impl PluginRequestPolicyPlan {
let mut routers = Vec::new();
let mut schedulers = Vec::new();
let mut retries = Vec::new();
let mut middleware = Vec::new();
for entry in entries {
match entry {
PolicyEntry::Router(entry) => routers.push(entry),
PolicyEntry::Scheduler(entry) => schedulers.push(entry),
PolicyEntry::Retry(entry) => retries.push(entry),
PolicyEntry::Middleware(entry) => middleware.push(entry),
}
}
if routers.is_empty()
&& schedulers.is_empty()
&& middleware.is_empty()
&& retries.is_empty()
{
if routers.is_empty() && schedulers.is_empty() && retries.is_empty() {
return Ok(None);
}
routers.sort_by(|left, right| {
@@ -350,41 +273,11 @@ impl PluginRequestPolicyPlan {
return Err(AdminError::invalid("账号调度绑定作用范围重叠"));
}
}
middleware.sort_by(|left, right| {
(left.order, &left.plugin_id, &left.instance_id).cmp(&(
right.order,
&right.plugin_id,
&right.instance_id,
))
});
// 发布时固定各挂载位置的有序候选,消息处理不再扫描其他边界的绑定
let mut by_mount = BTreeMap::<_, Vec<_>>::new();
for entry in middleware {
by_mount
.entry(entry.mount)
.or_default()
.push(Arc::new(entry));
}
Ok(Some(Arc::new(Self {
routers: routers.into(),
schedulers: schedulers.into(),
retries: retries.into(),
middleware: by_mount
.into_iter()
.map(|(mount, entries)| (mount, entries.into_boxed_slice()))
.collect(),
policy_timeout: maximum_call_timeout.min(POLICY_TIMEOUT),
middleware_timeout: maximum_call_timeout,
})))
}
#[must_use]
pub(crate) fn has_request_policy(&self) -> bool {
!self.routers.is_empty() || !self.schedulers.is_empty() || !self.retries.is_empty()
}
#[must_use]
pub(crate) fn has_middleware(&self) -> bool {
!self.middleware.is_empty()
}
}
@@ -17,7 +17,7 @@ use gateway_core::{
///
/// 组条件采用“任一命中”;同一 Key 可以同时属于多个组,因此不同组集合不能单独证明两个
/// 调度绑定互斥
#[derive(Default)]
#[derive(Clone, Default)]
pub(crate) struct BindingScope {
client_keys: BTreeSet<ClientApiKeyId>,
account_groups: BTreeSet<AccountGroupId>,
@@ -26,6 +26,7 @@ use crate::{RpcSession, callback::PluginCallbacks};
pub(crate) use registration::{prepare, unavailable_entries, validate_bindings};
#[derive(Clone)]
pub(crate) struct AdapterEntry {
instance_id: String,
scope: BindingScope,
@@ -129,6 +129,13 @@ fn observation(value: &ProviderResponseObservation) -> facts::ResponseObservatio
first_text_ms: timings.first_text_ms,
first_token_ms: timings.first_token_ms,
provider_processing_ms: timings.provider_processing_ms,
upstream_response_ms: timings.upstream_response_ms,
upstream_api_overhead_ms: timings.upstream_api_overhead_ms,
upstream_engine_ms: timings.upstream_engine_ms,
upstream_engine_iapi_ttft_ms: timings.upstream_engine_iapi_ttft_ms,
upstream_engine_service_ttft_ms: timings.upstream_engine_service_ttft_ms,
upstream_engine_iapi_tbt_ms: timings.upstream_engine_iapi_tbt_ms,
upstream_engine_service_tbt_ms: timings.upstream_engine_service_tbt_ms,
latency_ms: None,
},
client_headers: value
@@ -280,6 +287,15 @@ fn project_wire(wire: &ProtocolWireEvent) -> Result<WireEvent, PluginFault> {
WirePayload::RawSse {
frame: frame.to_vec(),
}
} else if let Some(message) = wire.raw_websocket_message() {
// 子模型回调采用 SDK 的事件流表达,不把 WebSocket 文本冒充 HTTP body
WirePayload::RawSse {
frame: gateway_protocol::openai::sse::encode_sse_event(
wire.event_type().unwrap_or_default(),
message,
)
.into_bytes(),
}
} else {
return Err(PluginFault::new(
ErrorCode::Fault,
@@ -212,10 +212,11 @@ impl BodyResource {
};
let transformed = frame.transformed();
let (bytes, framing, terminal, envelope) = frame.into_parts();
// canonical-only 事件没有客户端正文,仍须保留并允许读取其完整事实
let facts_only =
framing == MiddlewareFraming::RawBytes && bytes.is_empty() && envelope.is_some();
if (!facts_only && framing != self.expected_framing) || bytes.len() > maximum {
// 宿主原始字节可能没有 JSON 解析视图,包含未知 WS 文本和 canonical-only 事实
// 预期 framing 约束插件改写后的输出,不能阻止插件读取并原样归还来源
if (framing != MiddlewareFraming::RawBytes && framing != self.expected_framing)
|| bytes.len() > maximum
{
return Err(invalid());
}
let source_id = state.next_source_id.checked_add(1).ok_or_else(invalid)?;
@@ -16,7 +16,7 @@ pub(crate) struct ConnectionOwner {
pub credential_revision: u64,
}
/// 每个适配器代次独立持有连接;取出即独占,失败或取消不会归还半完成会话
/// 每个适配器进程身份独立持有连接;取出即独占,失败或取消不会归还半完成会话
#[derive(Default)]
pub(crate) struct ConnectionPool {
idle: Mutex<BTreeMap<String, IdleConnection>>,
@@ -4,10 +4,7 @@ use std::collections::BTreeSet;
use gateway_admin::model::{
AdminError,
plugins::{
instances::{PluginFailurePolicy, PluginInstance},
state::PluginStateConfiguration,
},
plugins::{instances::PluginInstance, state::PluginStateConfiguration},
};
use gateway_plugin_sdk::{Capability, Manifest, Stage};
use secrecy::ExposeSecret as _;
@@ -47,24 +44,8 @@ pub(super) fn validate(
if capability != Capability::Observer && binding.event.is_some() {
return Err(AdminError::invalid("事件订阅只能用于观察绑定"));
}
let frontend_authentication = capability == Capability::FrontendAuthentication;
if frontend_authentication
&& (stage != Stage::Authentication
|| !matches!(
binding.failure_policy,
PluginFailurePolicy::Reject | PluginFailurePolicy::Delegate
)
|| binding.identity_bindings.is_empty()
|| !binding.client_key_ids.is_empty()
|| !binding.account_group_ids.is_empty()
|| !binding.provider_ids.is_empty()
|| !binding.models.is_empty())
if capability != Capability::FrontendAuthentication && !binding.identity_bindings.is_empty()
{
return Err(AdminError::invalid(
"客户端认证绑定必须配置认证阶段、回退策略和身份映射",
));
}
if !frontend_authentication && !binding.identity_bindings.is_empty() {
return Err(AdminError::invalid("身份映射只能用于客户端认证绑定"));
}
}
@@ -21,8 +21,7 @@ impl PluginRuntime {
&self,
instance: PluginInstance,
target_revision: Revision,
) -> Result<PreparedContributions, AdminError> {
let mut sessions = Vec::new();
) -> Result<Arc<PreparedInstance>, AdminError> {
let mut observer_entries = Vec::new();
let mut commands = Vec::new();
let mut management = Vec::new();
@@ -68,6 +67,10 @@ impl PluginRuntime {
&instance.bindings,
)?;
crate::adapter::policy::validate_bindings(package.manifest(), &instance.bindings)?;
crate::adapter::middleware::validate_bindings(
package.manifest(),
&instance.bindings,
)?;
crate::adapter::upstream_adapter::validate_bindings(
package.manifest(),
&instance.bindings,
@@ -174,6 +177,17 @@ impl PluginRuntime {
.await
.map_err(|_| AdminError::unavailable("插件进程握手失败"))?,
);
let mut prepared = PreparedInstance {
instance_id: instance_id.clone(),
artifact_sha256: instance.artifact_sha256.clone(),
revision: instance.revision,
session: session.clone(),
private_state: private_state.clone(),
maintenance: manifest
.contributes
.contains_key(&gateway_plugin_sdk::Capability::Maintenance),
contributions: PreparedContributions::default(),
};
let registration = session
.call(
"plugin.register",
@@ -234,6 +248,13 @@ impl PluginRuntime {
session.clone(),
callbacks.clone(),
)?);
let middleware_entries = crate::adapter::middleware::compile_entries(
&manifest,
&instance_id,
&bindings,
session.clone(),
callbacks.clone(),
)?;
let model_aliases =
crate::adapter::catalog::prepare(&manifest, &instance, &session).await?;
let upstream_entries = crate::adapter::upstream_adapter::prepare(
@@ -243,25 +264,16 @@ impl PluginRuntime {
Arc::clone(&callbacks),
)
.await?;
sessions.push(PreparedInstance {
instance_id,
artifact_sha256: instance.artifact_sha256,
revision: instance.revision,
session,
private_state,
maintenance: manifest
.contributes
.contains_key(&gateway_plugin_sdk::Capability::Maintenance),
});
Ok(PreparedContributions {
sessions,
prepared.contributions = PreparedContributions {
observer_entries,
commands,
management,
policy_entries,
middleware_entries,
upstream_entries,
authentication_entries,
model_aliases,
})
};
Ok(Arc::new(prepared))
}
}
@@ -1,7 +1,10 @@
//! 插件运行时配置、端口装配与整体关闭;候选和调用均复用同一状态
use super::super::restart_circuit::{PluginRestartCircuitConfig, RestartCircuits, RestartIdentity};
use super::{diagnostics::PreparationDiagnostic, set::PreparedSet};
use super::{
diagnostics::PreparationDiagnostic,
set::{PreparedInstance, PreparedSet},
};
use crate::{
PackageLimits, RpcLimits, RpcSession,
callback::{
@@ -54,6 +57,7 @@ pub struct PluginRuntime {
pub(super) config: PluginRuntimeConfig,
pub(super) prepare_lock: Mutex<()>,
pub(super) prepared: Mutex<BTreeMap<String, Weak<PreparedSet>>>,
pub(super) prepared_instances: Mutex<BTreeMap<String, Weak<PreparedInstance>>>,
pub(super) preparation_diagnostics: SyncMutex<BTreeMap<u64, PreparationDiagnostic>>,
pub(super) restart_circuits: Arc<RestartCircuits>,
pub(super) shutting_down: Arc<AtomicBool>,
@@ -95,6 +99,7 @@ impl PluginRuntime {
config,
prepare_lock: Mutex::new(()),
prepared: Mutex::new(BTreeMap::new()),
prepared_instances: Mutex::new(BTreeMap::new()),
preparation_diagnostics: SyncMutex::new(BTreeMap::new()),
restart_circuits: RestartCircuits::new(restart_circuit),
shutting_down: Arc::new(AtomicBool::new(false)),
@@ -177,6 +182,7 @@ impl PluginRuntime {
prepared.clear();
sets
};
self.prepared_instances.lock().await.clear();
let mut sessions = Vec::new();
for set in &sets {
for instance in &set.sessions {
@@ -30,7 +30,7 @@ pub(super) struct PreparedSet {
pub(super) _execution: Option<Arc<ExecutionExtensionPlans>>,
pub(super) _authentication:
Option<Arc<dyn gateway_core::engine::authentication::FrontendAuthenticationPlan>>,
pub(super) sessions: Vec<PreparedInstance>,
pub(super) sessions: Vec<Arc<PreparedInstance>>,
pub(super) failures: BTreeMap<String, PluginInstanceRuntimeFailure>,
pub(super) shutting_down: Arc<AtomicBool>,
pub(super) commands: Vec<Arc<crate::adapter::command_line::PluginCommand>>,
@@ -38,12 +38,12 @@ pub(super) struct PreparedSet {
pub(super) model_aliases: Vec<gateway_core::routing::ContributedModelAlias>,
}
#[derive(Default)]
#[derive(Clone, Default)]
pub(super) struct PreparedContributions {
pub(super) sessions: Vec<PreparedInstance>,
pub(super) observer_entries: Vec<crate::adapter::observer::ObserverEntry>,
pub(super) commands: Vec<Arc<crate::adapter::command_line::PluginCommand>>,
pub(super) management: Vec<crate::adapter::management::ManagementEntry>,
pub(super) middleware_entries: Vec<crate::adapter::middleware::MiddlewareEntry>,
pub(super) policy_entries: Vec<crate::adapter::policy::PolicyEntry>,
pub(super) upstream_entries: Vec<crate::adapter::upstream_adapter::AdapterEntry>,
pub(super) authentication_entries:
@@ -53,12 +53,12 @@ pub(super) struct PreparedContributions {
impl PreparedContributions {
pub(super) fn append(&mut self, other: Self) {
self.sessions.extend(other.sessions);
self.observer_entries.extend(other.observer_entries);
self.commands.extend(other.commands);
self.management.extend(other.management);
self.model_aliases.extend(other.model_aliases);
self.policy_entries.extend(other.policy_entries);
self.middleware_entries.extend(other.middleware_entries);
self.upstream_entries.extend(other.upstream_entries);
self.authentication_entries
.extend(other.authentication_entries);
@@ -72,6 +72,7 @@ pub(super) struct PreparedInstance {
pub(super) session: Arc<RpcSession>,
pub(super) private_state: Arc<PluginPrivateState>,
pub(super) maintenance: bool,
pub(super) contributions: PreparedContributions,
}
impl PreparedSet {
@@ -97,16 +98,16 @@ impl ExtensionSetLease for PreparedSet {
}
}
impl Drop for PreparedSet {
impl Drop for PreparedInstance {
fn drop(&mut self) {
for instance in self.sessions.drain(..) {
if let Ok(handle) = tokio::runtime::Handle::try_current() {
handle.spawn(async move {
let session = instance.session;
session.quiesce();
session.shutdown(Duration::from_secs(1)).await;
});
}
// 发布集合只共享同一配置的实例,最后一个实例持有者释放后才关闭进程
// 先同步停止新调用,随后有界等待在途 I/O 并回收进程
self.session.quiesce();
if let Ok(handle) = tokio::runtime::Handle::try_current() {
let session = Arc::clone(&self.session);
handle.spawn(async move {
session.shutdown(Duration::from_secs(1)).await;
});
}
}
}
@@ -3,7 +3,7 @@
use super::{
PluginRuntime,
diagnostics::{PreparationDiagnostic, runtime_failure_from_admin},
set::{PreparedContributions, PreparedSet, fingerprint},
set::{PreparedContributions, PreparedSet, fingerprint, instance_fingerprint},
};
use crate::ValidatedPackage;
use async_trait::async_trait;
@@ -88,7 +88,12 @@ impl PluginRuntime {
}
return Ok(ExtensionSetReference::new(set.id.clone(), set));
}
self.prepared_instances
.lock()
.await
.retain(|_, instance| instance.strong_count() > 0);
let mut contributions = PreparedContributions::default();
let mut sessions = Vec::new();
let mut failures = BTreeMap::new();
let mut identities = BTreeSet::new();
for instance in snapshot
@@ -99,15 +104,34 @@ impl PluginRuntime {
if !identities.insert(instance.id.clone()) {
return Err(AdminError::invalid("插件实例 ID 重复"));
}
let result = tokio::time::timeout(
// 沿用准备阶段原有的 30 秒预算;打包校验与能力恢复也在预算内
Duration::from_secs(30),
self.prepare_instance(instance.clone(), snapshot.config_revision),
)
.await
.unwrap_or_else(|_| Err(AdminError::unavailable("插件启动超时")));
let identity = instance_fingerprint(instance)?;
let reusable = {
let instances = self.prepared_instances.lock().await;
instances
.get(&identity)
.and_then(Weak::upgrade)
.filter(|instance| instance.session.is_ready())
};
let result = if let Some(instance) = reusable {
Ok(instance)
} else {
tokio::time::timeout(
// 沿用准备阶段原有的 30 秒预算;打包校验与能力恢复也在预算内
Duration::from_secs(30),
self.prepare_instance(instance.clone(), snapshot.config_revision),
)
.await
.unwrap_or_else(|_| Err(AdminError::unavailable("插件启动超时")))
};
match result {
Ok(candidate) => contributions.append(candidate),
Ok(candidate) => {
self.prepared_instances
.lock()
.await
.insert(identity, Arc::downgrade(&candidate));
contributions.append(candidate.contributions.clone());
sessions.push(candidate);
}
Err(error) if required_revision == Some(instance.revision) => return Err(error),
Err(error) => {
// 目录没有拒绝请求用的 binding,不能在恢复失败时悄悄撤销仍启用的别名
@@ -132,6 +156,9 @@ impl PluginRuntime {
contributions
.policy_entries
.extend(crate::adapter::policy::unavailable_entries(instance)?);
contributions
.middleware_entries
.extend(crate::adapter::middleware::unavailable_entries(instance)?);
contributions.upstream_entries.extend(
crate::adapter::upstream_adapter::unavailable_entries(instance)?,
);
@@ -145,11 +172,11 @@ impl PluginRuntime {
}
}
let PreparedContributions {
sessions,
observer_entries,
commands,
management,
policy_entries,
middleware_entries,
upstream_entries,
authentication_entries,
model_aliases,
@@ -177,23 +204,21 @@ impl PluginRuntime {
.map_err(|_| AdminError::invalid("插件观察计划注册冲突"))
})
.transpose()?;
let policy_plan = crate::adapter::policy::PluginRequestPolicyPlan::compile(
let policies = crate::adapter::policy::PluginRequestPolicyPlan::compile(
policy_entries,
self.config.rpc_limits.maximum_call_timeout,
)?;
let policies = policy_plan
.as_ref()
.filter(|plan| plan.has_request_policy())
.cloned()
.map(|plan| {
self.policies
.register(id.clone(), plan)
.map_err(|_| AdminError::invalid("插件请求策略计划注册冲突"))
})
.transpose()?;
let middleware = policy_plan
.filter(|plan| plan.has_middleware())
.map(|plan| plan as Arc<dyn gateway_core::engine::middleware::MiddlewarePlan>);
)?
.map(|plan| {
self.policies
.register(id.clone(), plan)
.map_err(|_| AdminError::invalid("插件请求策略计划注册冲突"))
})
.transpose()?;
let middleware = crate::adapter::middleware::PluginMiddlewarePlan::compile(
middleware_entries,
self.config.rpc_limits.maximum_call_timeout,
)
.map(|plan| plan as Arc<dyn gateway_core::engine::middleware::MiddlewarePlan>);
let upstream_adapters =
crate::adapter::upstream_adapter::PluginUpstreamAdapterPlan::compile(upstream_entries)?;
let execution = if middleware.is_some() || upstream_adapters.is_some() {
@@ -297,6 +322,7 @@ impl PluginPreparation for PluginRuntime {
let (_, state) = super::super::configuration::validate(&instance, package.manifest())?;
crate::adapter::observer::validate_bindings(package.manifest(), &instance.bindings)?;
crate::adapter::policy::validate_bindings(package.manifest(), &instance.bindings)?;
crate::adapter::middleware::validate_bindings(package.manifest(), &instance.bindings)?;
crate::adapter::catalog::validate_bindings(package.manifest(), &instance.bindings)?;
crate::adapter::upstream_adapter::validate_bindings(
package.manifest(),
@@ -9,8 +9,8 @@ use gateway_admin::{
AdminError,
plugins::{
InspectedPluginArtifact, PluginArtifactIcon, PluginArtifactIconResource,
PluginArtifactIconVariants, PluginArtifactMetadata, PluginContribution,
PluginIconTheme,
PluginArtifactIconVariants, PluginArtifactMetadata, PluginCompatibilityRequirements,
PluginContribution, PluginIconTheme,
},
},
ports::plugins::PluginPackageInspector,
@@ -33,37 +33,12 @@ impl PackageInspector {
slots: Arc::new(tokio::sync::Semaphore::new(2)),
}
}
}
#[async_trait]
impl PluginPackageInspector for PackageInspector {
async fn compatibility_warning(
&self,
archive: Arc<[u8]>,
expected_sha256: String,
) -> Result<Option<String>, AdminError> {
let requirements = self.compatibility(archive, expected_sha256).await?;
super::compatibility::warning(&requirements, &self.host_version)
}
fn api_deprecations(
&self,
metadata: &PluginArtifactMetadata,
) -> Result<Vec<gateway_admin::model::plugins::instances::PluginApiDeprecation>, AdminError>
{
crate::compatibility::warnings(
metadata
.contributes
.iter()
.map(|(capability, declaration)| (capability.as_str(), declaration.version)),
)
}
async fn inspect(
async fn inspect_package(
&self,
archive: Arc<[u8]>,
expected_sha256: Option<String>,
) -> Result<InspectedPluginArtifact, AdminError> {
) -> Result<(InspectedPluginArtifact, PluginCompatibilityRequirements), AdminError> {
let permit = Arc::clone(&self.slots)
.try_acquire_owned()
.map_err(|_| AdminError::unavailable("插件包校验繁忙,请稍后重试"))?;
@@ -129,14 +104,53 @@ impl PluginPackageInspector for PackageInspector {
state_namespaces: crate::callback::private_state::configuration(manifest)?
.namespaces,
};
Ok(InspectedPluginArtifact {
metadata,
archive: package.archive(),
})
let requirements = super::compatibility::requirements(manifest)?;
Ok((
InspectedPluginArtifact {
metadata,
archive: package.archive(),
},
requirements,
))
})
.await
.map_err(|_| AdminError::internal("插件包校验任务失败"))?
}
}
#[async_trait]
impl PluginPackageInspector for PackageInspector {
async fn compatibility_warning(
&self,
archive: Arc<[u8]>,
expected_sha256: String,
) -> Result<Option<String>, AdminError> {
let (_, requirements) = self.inspect_package(archive, Some(expected_sha256)).await?;
super::compatibility::warning(&requirements, &self.host_version)
}
fn api_deprecations(
&self,
metadata: &PluginArtifactMetadata,
) -> Result<Vec<gateway_admin::model::plugins::instances::PluginApiDeprecation>, AdminError>
{
crate::compatibility::warnings(
metadata
.contributes
.iter()
.map(|(capability, declaration)| (capability.as_str(), declaration.version)),
)
}
async fn inspect(
&self,
archive: Arc<[u8]>,
expected_sha256: Option<String>,
) -> Result<InspectedPluginArtifact, AdminError> {
self.inspect_package(archive, expected_sha256)
.await
.map(|(artifact, _)| artifact)
}
async fn icon(
&self,
@@ -167,3 +167,75 @@ async fn enabled_bindings_must_reference_the_exact_declared_contribution_and_sta
);
}
}
#[tokio::test]
async fn frontend_authentication_constraints_apply_to_validation_and_preparation() {
use gateway_admin::model::plugins::instances::PluginFrontendIdentityBinding;
let (cache, store, runtime) =
super::setup_with_contributions(Contributions::from([crate::support::contribution(
Capability::FrontendAuthentication,
vec![Stage::Authentication],
vec![],
vec![],
)]))
.await;
let mut valid = store.snapshot.lock().unwrap().clone();
valid.instances[0].bindings = vec![PluginCapabilityBinding {
contribution: "test.example.frontendAuthentication".into(),
stage: "authentication".into(),
order: 0,
failure_policy: PluginFailurePolicy::Reject,
client_key_ids: vec![],
account_group_ids: vec![],
provider_ids: vec![],
models: vec![],
event: None,
identity_bindings: vec![PluginFrontendIdentityBinding {
principal: "principal".into(),
client_key_id: "key-auth".into(),
}],
}];
assert!(runtime.validate(valid.instances[0].clone()).await.is_ok());
for scenario in [
"empty",
"duplicate",
"principal",
"key",
"policy",
"stage",
"client_scope",
"group_scope",
"provider_scope",
"model_scope",
] {
let mut candidate = valid.clone();
let binding = &mut candidate.instances[0].bindings[0];
match scenario {
"empty" => binding.identity_bindings.clear(),
"duplicate" => binding
.identity_bindings
.push(binding.identity_bindings[0].clone()),
"principal" => binding.identity_bindings[0].principal = " invalid ".into(),
"key" => binding.identity_bindings[0].client_key_id.clear(),
"policy" => binding.failure_policy = PluginFailurePolicy::Observe,
"stage" => binding.stage = "request".into(),
"client_scope" => binding.client_key_ids.push("key-auth".into()),
"group_scope" => binding.account_group_ids.push("group".into()),
"provider_scope" => binding.provider_ids.push("openai".into()),
"model_scope" => binding.models.push("model".into()),
_ => unreachable!(),
}
assert!(
runtime
.validate(candidate.instances[0].clone())
.await
.is_err(),
"validate {scenario}"
);
assert!(
runtime.prepare(candidate).await.is_err(),
"prepare {scenario}"
);
}
assert_eq!(std::fs::read_dir(cache.path()).unwrap().count(), 0);
}
@@ -790,6 +790,8 @@ async fn completed_observation_contains_terminal_cost_timings_and_failure() {
)
.with_timings(ModelRequestTimings {
first_text_ms: Some(12),
upstream_response_ms: Some(7_000),
upstream_engine_iapi_tbt_ms: Some(2.450638),
latency_ms: Some(34),
..ModelRequestTimings::default()
}),
@@ -802,6 +804,11 @@ async fn completed_observation_contains_terminal_cost_timings_and_failure() {
assert_eq!(observed["usage"]["cost"]["total"]["amount"], "0.0123");
assert_eq!(observed["usage"]["cost"]["total"]["currency"], "USD");
assert_eq!(observed["usage"]["timings"]["first_text_ms"], 12);
assert_eq!(observed["usage"]["timings"]["upstream_response_ms"], 7_000);
assert_eq!(
observed["usage"]["timings"]["upstream_engine_iapi_tbt_ms"],
2.450638
);
assert_eq!(observed["usage"]["timings"]["latency_ms"], 34);
assert_eq!(observed["usage"]["failure"]["outcome"], "failed");
assert_eq!(observed["usage"]["failure"]["attempt_count"], 2);
@@ -21,6 +21,92 @@ use gateway_core::{
upstream::{OpaqueUpstreamValue, UpstreamTransport},
};
#[tokio::test]
async fn reading_websocket_facts_preserves_original_text_and_opaque_messages() {
let records = tempfile::tempdir().unwrap();
let marker = records.path().join("facts.jsonl");
let worker = std::fs::read(env!("CARGO_BIN_EXE_gateway-plugin-test-middleware")).unwrap();
let package = crate::support::package_with_contributions(
&worker,
Contributions::from([crate::support::contribution(
Capability::Middleware,
vec![Stage::Attempt],
vec!["openai".into()],
vec!["openai".into()],
)]),
);
let (_cache, runtime) = setup_package(vec![InstanceFixture {
id: "facts-reader",
configuration: serde_json::json!({"mode":"facts", "attempt":true, "facts_marker":marker}),
bindings: vec![binding(MIDDLEWARE_CONTRIBUTION, "attempt", 0, PluginFailurePolicy::Reject)],
}], package).await;
let generation = prepare(&runtime).await;
let plan = runtime
.execution_registry()
.middleware(&generation)
.unwrap();
let messages = [
"{ \"type\" : \"future.event\", \"number\" : 1e3, \"text\" : \"\\u0061\" }".to_owned(),
format!(
"{{\"type\":\"future.deep\",\"value\":{}0{}}}",
"[".repeat(140),
"]".repeat(140)
),
"future non-JSON text".to_owned(),
];
let events: Vec<_> = messages
.iter()
.map(|raw| {
let wire = match serde_json::from_str::<serde_json::Value>(raw) {
Ok(data) => ProtocolWireEvent::json("openai", None, data)
.unwrap()
.with_raw_websocket_message(raw.as_str()),
Err(_) => ProtocolWireEvent::raw_websocket("openai", raw.as_str()).unwrap(),
};
Ok(ProviderEvent::wire(wire))
})
.collect();
let mut stream = execute_attempt_middleware(
Some(&plan),
attempt_middleware_context_for_transport(Arc::default(), ClientTransport::WebSocket),
operation(),
ClientTransport::WebSocket,
Box::new(move |_, _| {
Box::pin(async move {
Ok(ProviderStream::new(
ProviderCallMetadata::new(
ProviderKind::new("openai").unwrap(),
UpstreamModelId::new("native-model").unwrap(),
ProviderAccountId::new("acct_original").unwrap(),
UpstreamTransport::new("websocket").unwrap(),
),
futures::stream::iter(events),
(),
))
})
}),
)
.await
.unwrap();
let mut delivered = Vec::new();
while let Some(event) = stream.next().await {
let event = event.unwrap();
assert!(!event.middleware_transformed());
delivered.push(
event
.wire_event()
.unwrap()
.raw_websocket_message()
.unwrap()
.to_owned(),
);
}
assert_eq!(delivered, messages);
assert_eq!(marker_lines(&marker, 4).await.len(), 4);
drop(stream);
runtime.shutdown().await;
}
#[tokio::test]
async fn facts_are_readable_without_replacing_or_duplicating_the_original_execution() {
let records = tempfile::tempdir().unwrap();
@@ -99,6 +185,8 @@ async fn facts_are_readable_without_replacing_or_duplicating_the_original_execut
)
.with_timings(ProviderResponseTimings {
headers_ms: Some(19),
upstream_response_ms: Some(7_000),
upstream_engine_iapi_tbt_ms: Some(2.450638),
..Default::default()
}),
);
@@ -184,6 +272,14 @@ async fn facts_are_readable_without_replacing_or_duplicating_the_original_execut
"{\"private_field\":\"kept\"}"
);
assert_eq!(facts["observation"]["timings"]["headers_ms"], 19);
assert_eq!(
facts["observation"]["timings"]["upstream_engine_iapi_tbt_ms"],
2.450638
);
assert_eq!(
facts["observation"]["timings"]["upstream_response_ms"],
7_000
);
assert_eq!(
facts["session_update"]["payload"]["cursor"],
"private-cursor"
@@ -0,0 +1,126 @@
//! 验证跨集合复用实例时旧请求和宿主回调仍持有各自的冻结计划
use super::*;
use gateway_admin::ports::plugins::PluginPreparation;
#[tokio::test]
async fn partial_update_preserves_inflight_middleware_and_uses_new_configuration_for_new_calls() {
let (cache, store, runtime) = super::super::setup_with_contributions(Contributions::from([
crate::support::contribution(Capability::Middleware, vec![Stage::Request], vec![], vec![]),
]))
.await;
let markers = tempfile::tempdir().unwrap();
let mut snapshot = store.snapshot.lock().unwrap().clone();
let template = snapshot.instances[0].clone();
snapshot.instances = ["a", "b", "c"]
.into_iter()
.enumerate()
.map(|(order, id)| {
let mut instance = template.clone();
instance.id = id.into();
instance.configuration = serde_json::json!({"startup_marker": markers.path().join(id)});
instance.bindings = vec![binding(
MIDDLEWARE_CONTRIBUTION,
"request",
order as i32,
PluginFailurePolicy::Reject,
)];
instance
})
.collect();
let old = PluginPreparation::prepare(&runtime, snapshot.clone())
.await
.unwrap();
let old_plan = runtime.execution_registry().middleware(&old).unwrap();
assert!(runtime.policy_registry().resolve(&old).is_none());
let (started, waiting) = tokio::sync::oneshot::channel();
let (resume, blocked) = tokio::sync::oneshot::channel();
let inflight = tokio::spawn(async move {
old_plan
.handle(
middleware_context(ClientTransport::HttpJson),
MiddlewareRequest::new(
"openai",
middleware_headers(),
Bytes::from_static(br#"{"input":"hello"}"#),
),
gateway_core::middleware::compose(Vec::new(), move |request| {
Box::pin(async move {
started.send(()).unwrap();
blocked.await.unwrap();
Downstream {
calls: Arc::default(),
closes: Arc::default(),
reads: Arc::default(),
error: false,
}
.into_next()
.run(request)
.await
})
}),
)
.await
});
waiting.await.unwrap();
snapshot.config_revision = Revision::new(2).unwrap();
snapshot.instances[1].revision = snapshot.config_revision;
snapshot.instances[1].configuration["middleware_map_body"] = true.into();
let new = PluginPreparation::prepare(&runtime, snapshot)
.await
.unwrap();
drop(old);
let new_plan = runtime.execution_registry().middleware(&new).unwrap();
let changed = new_plan
.handle(
middleware_context(ClientTransport::HttpJson),
MiddlewareRequest::new(
"openai",
middleware_headers(),
Bytes::from_static(br#"{"input":"hello"}"#),
),
Downstream {
calls: Arc::default(),
closes: Arc::default(),
reads: Arc::default(),
error: false,
}
.into_next(),
)
.await;
let (_, _, _, mut changed_body, _) = changed.unwrap().into_parts();
assert_eq!(
changed_body
.next_frame()
.await
.unwrap()
.unwrap()
.into_bytes(),
Bytes::from_static(b"{\"downstream\":true} ")
);
changed_body.close().await;
resume.send(()).unwrap();
let response = inflight
.await
.unwrap()
.expect("the old request completes through A/B/C callbacks");
let (_, _, _, mut body, _) = response.into_parts();
assert_eq!(
body.next_frame().await.unwrap().unwrap().into_bytes(),
Bytes::from_static(br#"{"downstream":true}"#)
);
body.close().await;
assert!(new.is_ready());
for (id, count) in [("a", 1), ("b", 2), ("c", 1)] {
assert_eq!(
std::fs::read_to_string(markers.path().join(id))
.unwrap()
.lines()
.count(),
count
);
}
drop(new_plan);
drop(new);
super::super::wait_until_empty(cache.path()).await;
}
@@ -3,6 +3,7 @@
mod compatibility;
mod facts;
mod http;
mod lifecycle;
mod mounts;
mod service;
mod websocket;
@@ -704,6 +705,13 @@ fn middleware_context(transport: ClientTransport) -> MiddlewareContext {
}
fn attempt_middleware_context(effects: Arc<ExecutionEffects>) -> MiddlewareContext {
attempt_middleware_context_for_transport(effects, ClientTransport::HttpJson)
}
fn attempt_middleware_context_for_transport(
effects: Arc<ExecutionEffects>,
transport: ClientTransport,
) -> MiddlewareContext {
MiddlewareContext::new(
MiddlewareTarget {
request_id: ModelRequestId::new("req_attempt_http").unwrap(),
@@ -711,7 +719,7 @@ fn attempt_middleware_context(effects: Arc<ExecutionEffects>) -> MiddlewareConte
attempt_index: Some(NonZeroU32::MIN),
operation: Some(OperationKind::Generate),
endpoint: "/v1/responses".into(),
transport: ClientTransport::HttpJson,
transport,
provider: Some(ProviderKind::new("openai").unwrap()),
model: Some("public-a".into()),
account_id: Some(ProviderAccountId::new("acct_openai").unwrap()),
@@ -296,71 +296,71 @@ async fn restart_circuit_counts_a_started_process_that_exits_during_handshake()
}
#[tokio::test]
async fn restart_circuit_does_not_let_an_older_generation_clear_newer_failures() {
async fn reused_session_failure_counts_once_across_multiple_generations() {
let restart_circuit = gateway_plugin_runtime::PluginRestartCircuitConfig {
maximum_failures: NonZeroUsize::new(3).unwrap(),
stability_window: Duration::from_millis(50),
};
let (cache, store, runtime) = super::super::setup_with_restart_circuit(restart_circuit).await;
let controls = tempfile::tempdir().unwrap();
let marker = controls.path().join("overlapping-generations.jsonl");
let old_exit = controls.path().join("old-generation-exit");
let marker = controls.path().join("shared-generations.jsonl");
let exit = controls.path().join("shared-exit");
let mut snapshot = store.snapshot.lock().unwrap().clone();
snapshot.instances[0].configuration = serde_json::json!({
"startup_marker":marker,
"startup_failures":[false,true,true,true],
"exit_after_ready_signals":[old_exit],
"startup_marker": marker,
"startup_failures": [false, true, true, true],
"exit_after_ready_signals": [exit],
});
let mut other = snapshot.instances[0].clone();
other.id = "other-instance".into();
other.name = "Other".into();
other.configuration = serde_json::json!({});
snapshot.instances.push(other);
let published = PluginPreparation::prepare(&runtime, snapshot.clone())
.await
.expect("published generation");
assert!(published.is_ready());
// 旧代次明确越过稳定窗口;新代次在握手前失败,运行时间固定为零
// 不依赖 15 ms 退出与注册完成的竞速,也不让调度延迟重置新失败预算
tokio::time::sleep(restart_circuit.stability_window).await;
let mut failed_candidate = None;
.unwrap();
let mut retained = Vec::new();
for revision in 2..=4 {
snapshot.config_revision = Revision::new(revision).unwrap();
snapshot.instances[1].revision = Revision::new(revision).unwrap();
snapshot.instances[1].configuration = serde_json::json!({"revision":revision});
snapshot.instances[1].revision = snapshot.config_revision;
retained.push(
PluginPreparation::prepare(&runtime, snapshot.clone())
.await
.unwrap(),
);
}
assert_eq!(
startup_count(&marker),
1,
"unrelated edits share the healthy process"
);
tokio::time::sleep(restart_circuit.stability_window).await;
std::fs::write(&exit, b"exit").unwrap();
wait_until_unready(&published).await;
assert!(retained.iter().all(|set| !set.is_ready()));
// 同一进程故障只扣一次预算;两个新的握手失败后才熔断
for revision in 5..=6 {
snapshot.config_revision = Revision::new(revision).unwrap();
snapshot.instances[1].revision = snapshot.config_revision;
let candidate = PluginPreparation::prepare(&runtime, snapshot.clone())
.await
.expect("the unrelated instance starts while the newer incarnation fails");
assert_eq!(startup_count(&marker), revision as usize);
drop(failed_candidate.replace(candidate));
assert!(published.is_ready(), "the older generation remains healthy");
.unwrap();
assert_eq!(startup_count(&marker), (revision - 3) as usize);
drop(candidate);
}
drop(failed_candidate.take());
std::fs::write(&old_exit, b"exit").unwrap();
wait_until_unready(&published).await;
snapshot.config_revision = Revision::new(5).unwrap();
snapshot.instances[1].revision = Revision::new(5).unwrap();
snapshot.instances[1].configuration = serde_json::json!({"revision":5});
let circuit = gateway_admin::ports::plugins::PluginRuntimeDiagnostics::runtime_diagnostics(
&runtime, &snapshot, None, None,
)
.await
.unwrap();
assert_eq!(
circuit["instance-one"]
.failure
.as_ref()
.map(|failure| failure.code.as_str()),
Some("restart_circuit_open"),
);
snapshot.config_revision = Revision::new(7).unwrap();
snapshot.instances[1].revision = snapshot.config_revision;
let recovered = PluginPreparation::prepare(&runtime, snapshot.clone())
.await
.expect("an older failed instance is isolated during unrelated recovery");
.unwrap();
assert_eq!(
startup_count(&marker),
3,
"the open circuit prevents another start"
);
let diagnostics = gateway_admin::ports::plugins::PluginRuntimeDiagnostics::runtime_diagnostics(
&runtime,
&snapshot,
Some(5),
Some(7),
Some(&recovered),
)
.await
@@ -370,18 +370,16 @@ async fn restart_circuit_does_not_let_an_older_generation_clear_newer_failures()
.failure
.as_ref()
.map(|failure| failure.code.as_str()),
Some("unavailable"),
"the older healthy session must not erase newer failures for the same identity"
Some("unavailable")
);
assert_eq!(startup_count(&marker), 4);
drop(retained);
drop(recovered);
drop(published);
super::super::wait_until_empty(cache.path()).await;
}
#[tokio::test]
async fn restart_circuit_restores_an_older_failure_after_candidate_shutdown() {
async fn abandoned_candidate_keeps_shared_process_failure_in_restart_budget() {
let restart_circuit = gateway_plugin_runtime::PluginRestartCircuitConfig {
maximum_failures: NonZeroUsize::MIN,
stability_window: Duration::from_secs(1),
@@ -411,10 +409,11 @@ async fn restart_circuit_restores_an_older_failure_after_candidate_shutdown() {
let candidate = PluginPreparation::prepare(&runtime, snapshot.clone())
.await
.expect("healthy replacement candidate");
assert_eq!(startup_count(&marker), 1);
drop(candidate);
assert!(published.is_ready());
std::fs::write(&old_exit, b"exit").unwrap();
wait_until_unready(&published).await;
assert!(candidate.is_ready());
drop(candidate);
tokio::time::timeout(Duration::from_secs(2), async {
loop {
@@ -461,7 +460,7 @@ async fn restart_circuit_restores_an_older_failure_after_candidate_shutdown() {
.map(|failure| failure.code.as_str()),
Some("unavailable"),
);
assert_eq!(startup_count(&marker), 2);
assert_eq!(startup_count(&marker), 1);
drop(recovered);
drop(published);
@@ -32,3 +32,145 @@ async fn candidate_is_reused_after_commit_and_released_when_no_generation_owner_
drop(published);
super::super::wait_until_empty(cache.path()).await;
}
fn starts(path: &std::path::Path) -> usize {
std::fs::read_to_string(path).unwrap().lines().count()
}
#[tokio::test]
async fn updating_b_reuses_a_and_c_until_their_last_set_is_released() {
// 两种释放顺序都必须保住另一集合仍使用的实例
for release_old_first in [true, false] {
let (cache, store, runtime) = super::super::setup().await;
let markers = tempfile::tempdir().unwrap();
let mut snapshot = store.snapshot.lock().unwrap().clone();
let template = snapshot.instances[0].clone();
snapshot.instances = ["a", "b", "c"]
.into_iter()
.map(|id| {
let mut instance = template.clone();
instance.id = id.into();
instance.configuration =
serde_json::json!({"startup_marker": markers.path().join(id)});
instance
})
.collect();
let old = PluginPreparation::prepare(&runtime, snapshot.clone())
.await
.unwrap();
snapshot.config_revision = Revision::new(2).unwrap();
snapshot.instances[1].revision = snapshot.config_revision;
let new = PluginPreparation::prepare(&runtime, snapshot)
.await
.unwrap();
assert_ne!(old.id(), new.id());
assert_eq!(std::fs::read_dir(cache.path()).unwrap().count(), 4);
assert_eq!(
[
starts(&markers.path().join("a")),
starts(&markers.path().join("b")),
starts(&markers.path().join("c"))
],
[1, 2, 1]
);
assert!(old.is_ready());
assert!(new.is_ready());
let retained = if release_old_first {
drop(old);
new
} else {
drop(new);
old
};
// 被释放集合的独用 B 关闭并移除缓存,A/C 与另一份 B 仍在
tokio::time::timeout(std::time::Duration::from_secs(3), async {
while std::fs::read_dir(cache.path()).unwrap().count() != 3 {
tokio::time::sleep(std::time::Duration::from_millis(10)).await;
}
})
.await
.unwrap();
assert!(retained.is_ready());
drop(retained);
super::super::wait_until_empty(cache.path()).await;
}
}
#[tokio::test]
async fn failed_partial_candidate_does_not_close_reused_instances() {
let (cache, store, runtime) =
super::super::setup_with_contributions(gateway_plugin_sdk::Contributions::from([
crate::support::contribution(
gateway_plugin_sdk::Capability::Observer,
vec![gateway_plugin_sdk::Stage::Observation],
vec![],
vec![],
),
]))
.await;
let markers = tempfile::tempdir().unwrap();
let mut snapshot = store.snapshot.lock().unwrap().clone();
let marker = markers.path().join("a");
snapshot.instances[0].configuration = serde_json::json!({"startup_marker": marker});
let old = PluginPreparation::prepare(&runtime, snapshot.clone())
.await
.unwrap();
let mut other = snapshot.instances[0].clone();
other.id = "b".into();
other.configuration = serde_json::json!({"registration_mismatch": true});
other.revision = Revision::new(2).unwrap();
snapshot.config_revision = other.revision;
snapshot.instances.push(other);
assert!(
PluginPreparation::prepare(&runtime, snapshot.clone())
.await
.is_err()
);
assert_eq!(starts(&marker), 1);
assert!(old.is_ready());
snapshot.instances[1].configuration = serde_json::json!({});
let repaired = PluginPreparation::prepare(&runtime, snapshot)
.await
.unwrap();
assert_eq!(starts(&marker), 1);
drop(old);
assert!(repaired.is_ready());
drop(repaired);
super::super::wait_until_empty(cache.path()).await;
}
#[tokio::test]
async fn targeted_quiescence_stops_the_shared_revision_without_stopping_its_replacement() {
use gateway_admin::ports::plugins::PluginStateLifecycle;
let (cache, store, runtime) = super::super::setup().await;
let mut snapshot = store.snapshot.lock().unwrap().clone();
let previous = snapshot.instances[0].clone();
let old = PluginPreparation::prepare(&runtime, snapshot.clone())
.await
.unwrap();
let mut other = previous.clone();
other.id = "other".into();
other.revision = Revision::new(2).unwrap();
snapshot.config_revision = other.revision;
snapshot.instances.push(other);
let shared = PluginPreparation::prepare(&runtime, snapshot.clone())
.await
.unwrap();
snapshot.config_revision = Revision::new(3).unwrap();
snapshot.instances[0].revision = snapshot.config_revision;
let new = PluginPreparation::prepare(&runtime, snapshot.clone())
.await
.unwrap();
runtime
.quiesce_instance(&previous.id, &previous.artifact_sha256, previous.revision)
.await;
assert!(!old.is_ready());
assert!(!shared.is_ready());
assert!(new.is_ready());
runtime.shutdown().await;
assert!(!new.is_ready());
drop(old);
drop(shared);
drop(new);
super::super::wait_until_empty(cache.path()).await;
}
@@ -137,3 +137,21 @@ async fn host_and_capability_version_mismatches_are_warnings_only() {
assert!(warning.contains("middleware v99"));
assert!(warning.contains("3.19.0"));
}
#[tokio::test]
async fn compatibility_warning_rejects_invalid_packages_and_digest_mismatches() {
let inspector = PackageInspector::new(PackageLimits::default(), semver::Version::new(1, 0, 0));
let archive = crate::support::package_with_contributions(b"fixture", Contributions::new());
let error = inspector
.compatibility_warning(archive, "0".repeat(64))
.await
.unwrap_err();
assert_eq!(error.kind(), gateway_admin::model::AdminErrorKind::Invalid);
let invalid: std::sync::Arc<[u8]> = b"invalid archive".as_slice().into();
let digest = hex::encode(Sha256::digest(invalid.as_ref()));
let error = inspector
.compatibility_warning(invalid, digest)
.await
.unwrap_err();
assert_eq!(error.kind(), gateway_admin::model::AdminErrorKind::Invalid);
}
@@ -746,7 +746,7 @@ impl Environment {
pub async fn bound_model_requests(&self, client_key_id: &str) -> Vec<(String, String)> {
sqlx::query_as(sqlx::AssertSqlSafe(format!(
"select request_kind, subagent_kind from {}.model_requests
"select request_kind, subagent_kind from {}.model_request_observations
where client_api_key_ref = $1 order by started_at, id",
self.schema
)))
+5 -7
View File
@@ -1,7 +1,7 @@
# Gateway Plugin SDK
用于编写 Codex Proxy 插件的 Rust 合同与可选会话辅助。插件只依赖本包,不依赖宿主 Core、Admin、Store 或 Host。
SDK 仍处于实验阶段;能力是否可用取决于宿主支持、清单声明与实际注册,
SDK 处于实验阶段;能力是否可用取决于宿主支持、清单声明与实际注册,
而不只是枚举中存在该名称
| 版本 | 当前值 | 用途 |
@@ -90,14 +90,12 @@ sequenceDiagram
元数据保留 64 KiB 上限;正文总量不由插件传输层额外限制。传输内部以 256 KiB 为单次 I/O 块,
不逐块发起 RPC,也不要求插件配置与宿主同步。读取按进度扩展缓冲区;截断正文必须报错,不能作为完整消息交付
`read_frame` / `write_frame` 使用协议 v2。帧写入队列之前先用 `validate_frame` 校验元数据,
帧写入队列前先用 `validate_frame` 校验元数据,
避免本地输入错误关闭共享传输。协议校验不会自动重试业务调用;部分读写中断后不能把当前位置当作新帧边界。
底层 `read_frame` / `write_frame` 不是取消安全的;自行编排会话时,不能在 `select!` 的其他分支完成后
重建半途中的帧读写。`PluginSession` 会在处理调用完成通知时保留同一个读取 future,直到完整帧到达或会话关闭
会话由宿主发送 `Hello`,插件校验协议后返回携带相同 incarnation 的 `Ready`。
注册结果必须与清单中的能力一致。一次 `Call` 的结果、错误和流帧按 ID 关联;回调必须
携带当前父调用 ID。实例、代次、阶段、账号或资源 ID 用于关联调用和管理资源生命周期
插件须校验 `Hello` 的协议版本。一次 `Call` 的结果、错误和流帧按 ID 关联;实例、代次、阶段、账号或资源 ID 用于关联调用和管理资源生命周期
## 本地验证
@@ -117,6 +115,6 @@ RUST_MIN_STACK=16777216 cargo +1.97.0 test --manifest-path backend/Cargo.toml -p
RUST_MIN_STACK=16777216 RUSTDOCFLAGS='-D warnings' cargo +1.97.0 doc --manifest-path backend/Cargo.toml -p gateway-plugin-sdk --all-features --no-deps --locked
```
源码中的类型和字段是当前合同;宿主接入、目录边界与 Runtime 集成验证见
[系统架构](../../../../docs/architecture.md)。Provider 固定为宿主内置的 OpenAI 与 xAI;插件实现只依赖公开 SDK 合同。
源码中的类型和字段是当前合同;宿主接入、目录边界、内置 Provider 与 Runtime 集成验证见
[系统架构](../../../../docs/architecture.md#31-插件扩展)。
单元测试与文档构建不代替目标平台上的安装、配置及实际能力验证
@@ -46,7 +46,7 @@ session.run(plugin).await?;
## 能力与方法
Provider 固定为宿主内置的 OpenAI 与 xAI。当前宿主开放以下 11 类扩展能力;新插件的 `middleware` 使用 v4,`upstream_adapter` 使用 v2,其余使用 v1;旧合同见[弃用窗口](manifest.md#接口弃用):
Provider 固定为宿主内置的 OpenAI 与 xAI。宿主开放以下 11 类扩展能力,版本与声明规则见[扩展项简写](manifest.md#扩展项简写):
| 能力 | 类型化方法 |
| --- | --- |
@@ -227,7 +227,7 @@ SDK 不自动重试。超时或断连不能证明写入未提交;重试上限
### 洋葱中间件
新插件使用 `middleware.version: 4`,所有挂载使用同一个 `ctx + next` 组合器。清单显式选择处理边界,绑定决定顺序;
`middleware.version: 4` 的所有挂载使用同一个 `ctx + next` 组合器。清单显式选择处理边界,绑定决定顺序;
不按 URL、消息类型或每个内部步骤增加处理器。仅处理一个边界可直接声明其调用类型;跨边界插件使用 `MiddlewareCall`
在一个处理器内匹配类型化视图,不感兴趣的调用交给 `call.forward().await`
@@ -353,7 +353,7 @@ Responses 终端重新解码插件正文时读取改写后的正文上限;atte
更早的改写使用上述 HTTP 请求的 `settings`。`request.head.settings_sources` 使用相同的来源结构,`execution.input` 表示进入当前 Key 作用域时的有效值。
原生 Responses WebSocket 每轮请求读取当前宿主快照,重新认证并解析 Key 策略,同时保留握手阶段的显式改写;已开始的执行保持自己的冻结值
middleware v3 / v4 均可读取和改写完整请求、响应 headers、正文与响应帧,不要求
中间件可读取和改写完整请求、响应 headers、正文与响应帧,不要求
`requests` 权限,也不按字段名称过滤认证、Cookie、会话或连接头。安装者承担插件读取与修改这些数据的风险。
类型、HTTP 格式、大小及资源生命周期仍需符合合同
@@ -18,8 +18,7 @@
作者清单不能包含 `package`。使用 `Manifest::from_author_slice()` 校验并规范化;直接反序列化的 `Manifest`
不会执行作者清单的 ID 与固定阶段推导,声明的可选字段仍使用其默认值。打包后,`main` 和所有资源都必须进入 `package.files`;宿主用
`package_for(host, os, architecture)` 检查版本与平台。运行模式仅为 `trustedProcess`:插件拥有与宿主
相同的系统身份,**不是进程沙箱**
`package_for(host, os, architecture)` 检查版本与平台。运行模式为 `trustedProcess`,信任边界见[完整信任](#完整信任)
宿主实际开放的合同由 [plugin-host-compatibility.json](../../runtime/plugin-host-compatibility.json)
声明,SDK 的 `Capability::contract_versions()` 只表示 SDK 能描述的行为版本,不能代替宿主支持检查
@@ -33,28 +32,22 @@
`capabilities` 使用扩展能力标识;Provider、模型和账号 ID 不属于这个集合
当前清单、进程协议和宿主兼容声明的格式版本均为 `2`。清单不接受 `permissions` 字段;使用其他格式版本的包须用匹配的 SDK 和 CLI 重新构建
清单、进程协议和宿主兼容声明的格式版本均为 `2`;清单不接受 `permissions` 字段
安装包清单校验与已安装元数据读取是两个边界:清单拒绝未知字段,并检查必需字段、类型和版本;
数据库元数据允许的字段增减见[持久化规则](../../../../../docs/architecture.md#发布与执行边界),不改变清单或 RPC 合同。
插件依赖新增接口或字段时,应通过 `engines.codex-proxy-rs` 声明所需宿主版本。宿主范围和能力版本不匹配时持续提示风险,仍可尝试启动;包结构、平台与 RPC 封装校验失败仍阻止加载。CLI 的作者校验保持严格,不支持的能力合同须使用匹配的 SDK 构建
数据库元数据的读取规则见[制品与发布](../../../../../docs/architecture.md#制品配置与发布),不改变清单或 RPC 合同。
`engines.codex-proxy-rs` 声明插件所需的宿主范围。范围或能力版本不匹配时持续提示风险,允许尝试启动;包结构、平台与 RPC 封装校验失败阻止加载。CLI 严格校验作者声明是否符合 SDK 支持的合同
## 接口弃用
新合同与旧合同并行提供,管理端和插件加载日志会提示旧合同及迁移方式。新合同首次正式发布后,旧合同至少再兼容 7 个正式版本;主版本、次版本和补丁版本各计一次,alpha、beta、rc、exp 及同版本重试不计
当前支持范围由[宿主兼容声明](../../runtime/plugin-host-compatibility.json)给出,弃用状态、剩余兼容窗口及作者说明以[弃用清单](../../runtime/plugin-api-deprecations.json)为准,管理端和加载日志展示同一提示
剩余窗口由宿主发行物中的 [弃用清单](../../runtime/plugin-api-deprecations.json) 给出,不通过版本号差值推算。窗口内提供旧合同转换;窗口结束后移除转换和支持声明,所有旧合同插件都应升级 SDK、处理器与清单再重新打包。仍允许尝试启动,但不再保证兼容,不按实际读取过哪些字段判断
| 旧合同 | 替代合同 | 迁移内容 |
| --- | --- | --- |
| middleware v3 | middleware v4 | 执行设置及 `settings_sources.execution` 中的 `disable_fast` 改为 `fast_mode` 三态 |
| upstream_adapter v1 | upstream_adapter v2 | 使用新版 `UpstreamAdapterRequest.fast_mode`,同步更新 SDK 和贡献版本 |
旧布尔投影仅在 `disabled` 时为 `true`,`default` 和 `enabled` 均为 `false`。旧中间件原样回传布尔值时保留实际三态,修改其他设置不会丢失 `enabled`;实际从 `true` 改为 `false` 时设为 `default`。旧接口不能表达强制开启,需升级合同使用三态
兼容窗口自替代合同首次正式发布起至少覆盖 7 个后续正式版本;主版本、次版本和补丁版本各计一次,预发行与同标签重试不计。
窗口内提供适配,窗口外不保证兼容;是否使用过特定字段不改变判断,版本号差值也不能代替清单中的计数
## 扩展项简写
作者声明可以省略 `id` 和固定阶段;默认版本为 `1`,新中间件显式选择版本 `4`,新版上游适配器 SDK 使用版本 `2`:
作者声明可以省略 `id` 和固定阶段;普通能力的版本默认为 `1`,`middleware` 显式声明版本 `4`,`upstream_adapter` 显式声明版本 `2`:
```json
{
@@ -71,7 +64,7 @@
}
```
默认扩展项 ID 为 `<publisher>.<name>.<capability-kebab>`。middleware v3 与 upstream_adapter v1 在[弃用窗口](#接口弃用)内仍可加载。除 `middleware` 外,阶段由
默认扩展项 ID 为 `<publisher>.<name>.<capability-kebab>`。除 `middleware` 外,阶段由
capability 固定并由工具生成:
| 阶段 | 能力 |
@@ -83,7 +76,7 @@ capability 固定并由工具生成:
| `management` / `command_line` | `management` / `command_line` |
| `maintenance` | `maintenance` |
`observer` 使用一个处理器接收完成与上游 WebSocket 事件,实例绑定按 `event` 选择订阅类型;具体合同见[观察事件](capabilities.md#路由调度与观察)。宿主仅接受 `observer` 声明;使用 `request_lifecycle`、`usage` 或 `web_socket_observer` 声明的插件须更新清单、处理器与绑定后重新打包
`observer` 使用一个处理器接收完成与上游 WebSocket 事件,实例绑定按 `event` 选择订阅类型;具体合同见[观察事件](capabilities.md#路由调度与观察)
`middleware` 必须从 `http`、`websocket`、`service`、`request`、`attempt` 中显式选择挂载;同一处理器可覆盖多个边界,协议格式等真实业务选择也不能省略。
安装清单若携带不同的固定阶段会被拒绝,而不是在加载时静默改写
@@ -27,7 +27,7 @@
同一请求不能同时命中两个适配器;配置准备时拒绝重叠,执行时再次复核。适配器未命中时仍使用原生上游,
已命中但不可用时拒绝请求。安装、配置、启停和升级使用已有插件管理
upstream_adapter v1 在[弃用窗口](manifest.md#接口弃用)内仍使用旧布尔投影,不能仅修改清单版本而继续使用旧 DTO。能力版本、清单版本、进程 RPC 版本和宿主兼容声明格式版本分别判断,见[清单](manifest.md)。
能力、清单、进程 RPC 和宿主兼容声明分别校验各自版本,见[清单](manifest.md)。
Provider、业务协议和传输名称不能写入宿主兼容清单的 capability / RPC 版本位置
## 执行与受管出站
@@ -91,6 +91,7 @@ let (message, ()) = tokio::try_join!(
- `persisted`:续接材料随宿主会话保存,仍受实例、代次、Key、账号和凭据版本限制
- `connection_local`:还必须保有本次已建立的 WebSocket;宿主在成功终态后保存同一连接,续接时独占取回
此处代次指插件进程的握手身份;配置未变化的实例在宿主发布集合之间复用时,该身份保持不变。
连接内状态不能跨账号、Key、凭据版本、插件进程或代次复用,失效时拒绝续接,不重新建立连接假装原生续接成功。
每个适配器代次最多保留 64 条闲置连接,闲置期限 30 分钟;保留连接不占用上一轮账号租约。
显式 `close()` 放弃连接内续接。普通成功、错误、取消与退出释放未保留连接

Some files were not shown because too many files have changed in this diff Show More