Compare commits

...
35 Commits
Author SHA1 Message Date
ZyyandCodex 6a5300a7b5 chore(release): v3.32.0
Co-authored-by: Codex <noreply@openai.com>
2026-10-25 23:34:39 +08:00
2c5b8b5c1a feat(openai): configure residency and account routing headers (#515)
Accept optional deployment strings for residency and account routing,
discard downstream values when unset, and isolate WebSocket connections
by the configured handshake headers.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-25 23:19:11 +08:00
0a87ac7f76 fix(plugins)!: honor declared stage call budgets (#513)
Resolve scenario defaults and plugin declarations consistently across the SDK
and runtime. Preserve maintenance across unrelated publications and cancel
long preparation when the runtime shuts down.

BREAKING CHANGE: Plugin RPC requires protocol v6. Rebuild plugins with the
updated SDK; previous protocol versions are not supported.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-25 22:39:23 +08:00
ZyyandCodex 3ac9d5f6f8 ci: keep repository checks independent
Remove automatic cross-repository source integration from host CI while
preserving host checks against locked dependencies. Keep source integration
available for explicit local version combinations and align the documentation.

Co-authored-by: Codex <noreply@openai.com>
2026-10-25 22:34:48 +08:00
ZyyandCodex e7e5b659cd chore(release): v3.31.3
Co-authored-by: Codex <noreply@openai.com>
2026-10-25 17:31:19 +08:00
ZyyandCodex b225b3732d fix(accounts): unify Codex labels and tighten plan badge spacing
Co-authored-by: Codex <noreply@openai.com>
2026-10-25 17:28:06 +08:00
ZyyandCodex 9dca763ea4 fix(ui): polish plugin management and API key labels
Co-authored-by: Codex <noreply@openai.com>
2026-10-25 16:57:42 +08:00
ZyyandCodex d26bc36db5 chore(release): v3.31.2
Co-authored-by: Codex <noreply@openai.com>
2026-10-25 15:59:42 +08:00
ZyyandCodex 96d5ab674a feat(ui): refine forms, navigation and theme loading
Co-authored-by: Codex <noreply@openai.com>
2026-10-25 15:59:34 +08:00
ZyyandCodex 84c0db2613 fix(ui): simplify token label and center reasoning effort
Co-authored-by: Codex <noreply@openai.com>
2026-10-25 04:58:06 +08:00
ZyyandCodex d819315d8b fix(ci): allow independent UI dependency versions
Keep frozen installs, source compatibility builds, and lockfile checks
without requiring every consumer to pin the UI submodule revision.

Co-authored-by: Codex <noreply@openai.com>
2026-10-25 04:48:28 +08:00
ZyyandCodex 06e1d8f0ac chore(release): v3.31.1
Co-authored-by: Codex <noreply@openai.com>
2026-10-25 04:40:32 +08:00
ZyyandCodex bdc8274b36 chore(ui): upgrade component library to v0.6.1
Co-authored-by: Codex <noreply@openai.com>
2026-10-25 04:38:49 +08:00
ZyyandCodex 3f6607209b refactor(usage): remove reported model tracking and polish UI
Remove upstream response model tracking across providers, storage, APIs,
and plugin facts without migrating historical records.

Refine sidebar hints, locale loading indicators, model columns, and usage
page labels and filters.

Co-authored-by: Codex <noreply@openai.com>
2026-10-25 04:03:27 +08:00
ZyyandCodex 3c05bc8f07 refactor(plugins): remove provider-specific request parameters
Use request, attempt, and outbound middleware for plugin-owned identity
selection and complete request rewriting. Preserve native account identity
and partition WebSocket connections through existing outbound opening changes.

Remove the construction parameter channel and its obsolete tests and docs.
Keep middleware at v6 without a compatibility shim.

Co-authored-by: Codex <noreply@openai.com>
2026-10-25 03:30:52 +08:00
ZyyandCodex 9ccd84f1a0 test(store): fix pg_dump lifecycle probe on Ubuntu 26.04
Redirect the mock process stdout to /dev/null explicitly so the wait-stage probe works when sleep reopens closed standard descriptors. Preserve cancellation, process reaping, and partial archive cleanup assertions.

Co-authored-by: Codex <noreply@openai.com>
2026-10-25 01:35:59 +08:00
ZyyandCodex a8f183707e chore(build): refresh base images and pin Ubuntu runners
Co-authored-by: Codex <noreply@openai.com>
2026-10-25 01:02:46 +08:00
ZyyandCodex 3f20af61ff chore(plugins): update workbench to v0.7.0
Co-authored-by: Codex <noreply@openai.com>
2026-10-25 00:23:55 +08:00
ZyyandCodex efaed4c3a6 fix(web): refine health loading and quota labels
Hide request health granularity until analytics are loaded and remove the placeholder daily granularity. Use compact uppercase quota badges and localized five-hour labels.

Co-authored-by: Codex <noreply@openai.com>
2026-10-25 00:08:42 +08:00
ZyyandCodex a36137bafb chore(release): v3.31.0
Co-authored-by: Codex <noreply@openai.com>
2026-10-24 23:22:06 +08:00
2e7a39e1e8 feat(plugins): add provider request composition middleware (#511)
* feat(plugins): add provider request composition middleware

Expose per-attempt provider parameters and a final outbound preparation chain using middleware v6. Preserve native transport behavior when no outbound scope matches.

Co-authored-by: Codex <noreply@openai.com>

* fix(plugins): sync workbench with middleware v6

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-24 22:58:06 +08:00
ZyyandCodex 6648198df4 perf(responses): reduce WebSocket copies and trace completion latency
Move owned request bodies back into WebSocket requests and forward validated
UTF-8 frames without copying their payloads. Record quota, account feedback,
and cyber policy durations while preserving terminal delivery ordering.

Co-authored-by: Codex <noreply@openai.com>
2026-10-24 21:05:22 +08:00
ZyyandCodex 2eed9b20ed fix(web): persist table sorting preferences
Co-authored-by: Codex <noreply@openai.com>
2026-10-24 20:54:37 +08:00
16aa08091c feat(plugins): select upstream adapters by account (#509)
* feat(plugins): select upstream adapters within account scopes

* fix(plugins): defer adapter decisions until account selection

Keep scoped adapters pending until an account is known so unavailable
fallbacks cannot mask a healthy earlier match. Filter native xAI accounts
when only an adapter can accept the request, preserving normal selection
and cold-stream behavior.

Cover fallback precedence and mixed-pool prewarm eligibility regressions.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-24 20:20:47 +08:00
ZyyandCodex abae4a631b fix(skills): enforce required domain guidance
Co-authored-by: Codex <noreply@openai.com>
2026-10-24 19:53:37 +08:00
8983eb4ce9 fix(openai): preserve session affinity for managed upstream adapters (#508)
* fix(openai): preserve session affinity for managed upstream adapters

* test(openai): avoid coupling wire invariance to adapter lookup count

* fix(openai): preserve plugin control over adapter request identity

Keep affinity and turn aliases tied to the admitted request before attempt middleware. Pass plugin rewrites to the adapter without native identity revalidation.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-24 19:51:48 +08:00
ZyyandCodex cd605b850e chore(deps): pin UI dependency to v0.6.0 release tag
Reference the v0.6.0 release tag and synchronize the pnpm lockfile.
The verified remote tag resolves to the existing UI commit.

Co-authored-by: Codex <noreply@openai.com>
2026-10-24 19:02:45 +08:00
ZyyandCodex 7beebb2340 fix(deps): patch crypto-bigint and tokio security advisories
Update crypto-bigint to 0.7.6 for RUSTSEC-2026-0387 and Tokio to
1.53.3 for RUSTSEC-2026-0386. Preserve the exact Tokio constraint
and verify existing transport fingerprint and cancellation contracts.

Co-authored-by: Codex <noreply@openai.com>
2026-10-24 19:02:41 +08:00
ZyyandCodex 31de0bd83c perf(store): move client budget settlement off request path
Track pending charges in memory and settle them with a supervised worker.
Include pending costs in budget checks without retrying historical writes
at admission, and reject new admissions when the backlog reaches its limit.

Use idempotent per-key batches with write deadlines, adaptive batch sizes,
and bounded shutdown draining. Keep load tests outside the regular suite.

Co-authored-by: Codex <noreply@openai.com>
2026-10-24 18:45:12 +08:00
ZyyandCodex 5cfc6dee10 fix(openai): abort stalled websocket pumps during error cleanup
Stop the send pump before returning terminal errors or discarding an
exchange. Preserve socket cleanup ordering and cover stalled data and
control writes.

Co-authored-by: Codex <noreply@openai.com>
2026-10-24 18:45:04 +08:00
ZyyandCodex ef5b4d7dbd fix(openai): reuse reasoning content registrations
Honor reasoning summary indexes and reuse registered content parts while
preserving upstream wire events and type conflict diagnostics.

Co-authored-by: Codex <noreply@openai.com>
2026-10-24 18:45:00 +08:00
ZyyandCodex 31974bc289 fix(diagnostics): align trace snapshot persistence limits
Co-authored-by: Codex <noreply@openai.com>
2026-10-24 14:19:22 +08:00
ZyyandCodex e033e21994 fix(frontend): remove default concurrency input icon
Co-authored-by: Codex <noreply@openai.com>
2026-10-24 00:05:43 +08:00
ZyyandCodex 2818edb9fc perf(i18n): load language resources on demand
Load and cache the selected language with synchronized UI messages. Keep the latest selection when loads overlap and render missing messages as empty text without language fallback.

Co-authored-by: Codex <noreply@openai.com>
2026-10-24 00:05:38 +08:00
ZyyandCodex 1d1f34ba7e fix(frontend): refine admin copy and responsive layouts
Shorten contextual actions and loading messages while preserving explanatory text. Improve quota metadata, profile metrics, theme controls and usage layouts, and add collapsed navigation hints.

Use spaced lowercase time units and remove chart tooltip dividers.

Co-authored-by: Codex <noreply@openai.com>
2026-10-23 23:10:15 +08:00
357 changed files with 10165 additions and 2634 deletions
+14 -10
View File
@@ -1,28 +1,32 @@
---
name: cpr-dev-guide
description: Codex Proxy RS 前后端开发、排障、代码审查和文档修改入口;改页面、弹窗、样式、文案,修接口、调度、数据库、Rust、宿主 SDK 或 Runtime 时均使用;PR、Issue、独立网关插件与发版另配对应技能
description: Codex Proxy RS 开发、修复、排障、代码审查和文档修改入口;页面、样式、文案、接口、调度、数据库及宿主 SDK、Runtime、插件接入均适用;PR、Issue、独立网关插件与发版叠加对应技能
---
# 开发指南
## 先匹配任务与文件
## 先确定领域与必读规范
按用户要改的行为初判,定位后按实际修改或审查的文件复核,不要求用户说出“前端”“后端”或技能名。**命中多行就同时执行,先读取对应规范再修改或给出审查结论**;任务范围变化时补齐新命中项,已有上下文直接复用
按用户意图、受影响行为与对象初判,定位后按实际文件复核,不要求用户说出“前端”“后端”或技能名。**所有命中行共同构成本次必读规范,先读完再确定方案、修改或给出审查结论**;跨领域叠加,不能只选最像当前步骤的一行
| 命中条件,操作或文件满足其一即可 | 必须执行 |
职责边界决定问题由谁修复;即使代码位于账号、调度或 API 模块,只要影响插件接入或为插件增加处理,也命中插件边界。确定方案时说明相关业务事实的归属及已有实现的复用依据,不能以功能恢复或测试通过替代职责判断
| 领域或操作,行为影响与文件路径满足其一即可 | 确定方案前读取与执行 |
| --- | --- |
| 调整页面、弹窗、表单、按钮、文案、样式或交互;目标在 `frontend/`、`modules/ui/`、插件前端,或是 `.vue` 文件 | [前端流程](references/frontend.md),覆盖组件复用、文案、Token、样式、视觉状态和运行证据;局部修改同样执行 |
| 修改服务端接口、路由、调度、账号与用量规则、持久化、后台任务或协议行为;目标是 `backend/` 中的实现、配置、迁移,或 Rust 源码 | [Workspace 边界](../../../docs/architecture.md#3-workspace-边界)、所属模块、[后端自审](../../../docs/architecture.md#后端自审);涉及 Rust 时使用 `$rust-best-practices` 和下方后端注释要求,验证时读[命令](../../../docs/architecture.md#验证命令) |
| 开发、排查或审查插件接入、宿主 SDK、Runtime、扩展接口或独立网关插件;目标在 `backend/crates/gateway-plugin/`、`modules/plugins/` | 完整读取[插件职责边界](references/plugin-boundaries.md)及所用 [SDK](../../../backend/crates/gateway-plugin/sdk/README.md) 类型与能力合同;宿主维护自身合同、资源与稳定性,插件业务错误在插件侧修复,不在宿主、SDK 或 Runtime 中补偿;独立插件开发、排查、审查另用 [cpr-plugin-dev](../cpr-plugin-dev/SKILL.md) |
| 开发、排查或审查服务端接口、路由、调度、账号与用量规则、持久化、后台任务或协议行为;目标是 `backend/` 中的实现、配置、迁移,或 Rust 源码 | [Workspace 边界](../../../docs/architecture.md#3-workspace-边界)的模块与依赖规则、所属模块及[后端自审](../../../docs/architecture.md#后端自审);Rust 使用 `$rust-best-practices` 和下方后端注释要求,宿主 Rust 同时读取 [Rust 模块组织约定](../../../docs/architecture.md#32-rust-模块组织约定) |
| 修改页面接口调用,或开发、排查、审查面向管理端或模型客户端的 HTTP/WS API 合同 | [API](../../../docs/api.md)对应章节;优先复用已有接口,核对调用方 → 服务端解析 → 实际查询或执行,不能只凭前端类型认定参数生效;按实际改动叠加前后端规范 |
| 模块或调用链变更 | [Workspace 边界](../../../docs/architecture.md#3-workspace-边界)及所属模块章节,不通读整份架构 |
| 新增、调整、清理或审查测试 | 下方[测试维护](#测试维护),按当前行为、职责归属与已有覆盖判断保留、合并或删除 |
| 修改部署行为、`deploy/` 配置或数据库迁移 | 按范围读取[部署](../../../deploy/README.md)、[迁移](../../../backend/migrations/README.md)对应章节 |
| 开发、排查或审查宿主 SDK、Runtime、扩展接口或独立插件;目标在 `backend/crates/gateway-plugin/` | [插件职责边界](references/plugin-boundaries.md)及所用 [SDK](../../../backend/crates/gateway-plugin/sdk/README.md) 类型与能力合同;宿主守住自身合同、资源与稳定性,不猜测或补偿插件业务错误;仅独立插件创作、排查另用 [cpr-plugin-dev](../cpr-plugin-dev/SKILL.md) |
| 修改或审查文档 | [文档检查](references/documentation.md)及目标章节,不默认读取代码开发、页面验收或发布流程 |
| 普通代码审查 | [审查标准](../../../CONTRIBUTING.md#审查标准);审查默认只读 |
| 代码审查,包括 PR 审查 | [审查标准](../../../CONTRIBUTING.md#审查标准),叠加被审查行为所属领域的规范;审查默认只读 |
| PR / Issue / 发版 | 按当前意图选 [PR](../cpr-github-pr/SKILL.md)、[Issue](../cpr-github-issue/SKILL.md)或[发版](../cpr-release/SKILL.md),普通提交不创建 PR |
纯文档任务按文档分支执行,不因文档所在目录或提及的路径、语言、功能加载开发流程。每次补读只解决当前未知项;文档链接带 `#章节` 不代表工具自动截取,先用 `rg -n '^#{1,4} ' 文件` 找边界,只读命中章节,不预读整份 CONTRIBUTING、架构、API、主题或全部 references,全文审查除外
纯文档任务按文档分支执行;核实文中的合同与职责时读对应领域条款,不因提及代码路径或语言加载实现、构建与页面验收流程
命中的流程引用完整读取,并落实其中指定的必读章节;长文档先用 `rg -n '^#{1,4} ' 文件` 定位,读取指定章节的完整内容。`#章节` 不会自动截取,“按需”仅用于未命中的领域和后续操作资料,不能省略必读规范;已有内容只在当前上下文可用且仍适用时复用,不循环重读技能
## 参考仓库的优先级
@@ -31,9 +35,9 @@ description: Codex Proxy RS 前后端开发、排障、代码审查和文档修
## 执行与收尾
1. 遵守已提供的 [AGENTS.md](../../../AGENTS.md),按其 CodeGraph 约定定位代码;确认工作区、任务范围与已有改动,找到事实归属、同类实现和调用链,补齐上表命中规范及[项目约定](../../../CONTRIBUTING.md#项目约定)对应条目后再修改
2. 自审完整差异,处理本次引入的职责越界、重复规则、冗余状态与无依据分支;删除或替换功能时一并清理失效的配置、类型、调用、测试与文档,已部署数据和公开合同仍遵守所属迁移及兼容规则;仅审查时报告,不自行修复
2. 以最终差异重新匹配上表,补齐新增领域的规范并检查实际实现,再处理本次引入的职责越界、重复规则、冗余状态与无依据分支;删除或替换功能时一并清理失效的配置、类型、调用、测试与文档,已部署数据和公开合同仍遵守所属迁移及兼容规则;仅审查时报告,不自行修复
3. 常规文档只写当前状态;有文档差异时执行[文档检查](references/documentation.md),没有时只确认现有说明是否失真,普通开发不自动修改 `release/notes.md`
4. 按变更范围执行[验证](../../../CONTRIBUTING.md#验证)及其停止条件,沿用本轮有效证据与用户已确定的验收分工,区分通过、失败、跳过与未执行;提交时补读[提交约定](../../../CONTRIBUTING.md#项目约定)
4. 按变更范围执行[验证](../../../CONTRIBUTING.md#验证)及其停止条件,后端命令见[验证命令](../../../docs/architecture.md#验证命令);沿用本轮有效证据与用户已确定的验收分工,区分通过、失败、跳过与未执行;提交时补读[提交约定](../../../CONTRIBUTING.md#项目约定)
5. 交付实际结果与验证缺口;前端按[前端流程](references/frontend.md)核对最终差异和运行证据,后端按上表检查生命周期,不把构建通过当作行为验收
## 测试维护
@@ -1,6 +1,6 @@
interface:
display_name: "cpr-dev-guide"
short_description: "前后端开发、排障与审查入口,按任务和文件匹配规范并验证"
default_prompt: "使用 $cpr-dev-guide 按请求和实际文件匹配前后端规范,读取命中项后完成已授权的开发或审查,并自审、验证和报告结果。"
short_description: "开发、排障与审查入口,按行为影响匹配领域边界并验证实现"
default_prompt: "使用 $cpr-dev-guide 按请求、行为影响和实际文件匹配领域必读规范,读完职责边界再确定方案,完成已授权的开发或审查、自审与验证。"
policy:
allow_implicit_invocation: true
@@ -1,10 +1,10 @@
# 文档检查
修改或审查文档时读取,职责与行文统一按 [文档职责与更新条件](../../../../CONTRIBUTING.md#文档职责与更新条件),只读该节
修改或审查文档时完整读取本流程,并读取 [文档职责与更新条件](../../../../CONTRIBUTING.md#文档职责与更新条件)的完整章节,覆盖内容归属与层次行文
## 写之前
先确定读者、必须说明的当前事实与所属章节,再决定是否需要修改及详略;文档职责与更新条件按上节核对
先确定读者、必须说明的当前事实与所属章节,再决定是否需要修改及详略;文档职责与更新条件按上节核对。涉及合同或职责说明时,补读所属架构、API 或 SDK 的对应条款,核实事实;不因文档提及实现而执行无关的开发与构建流程
## 逐段验收
@@ -2,9 +2,11 @@
适用于页面、组件、弹窗、表单及局部文案和样式修改,不以是否创建 PR 为触发条件;仅审查时核对和报告,不自行修改
## 开始前
## 领域必读
读取[前端职责](../../../../docs/architecture.md#34-前端模块职责)、[文案与信息层级](../../../../docs/theme.md#界面文案与信息层级)和[界面验证](../../../../CONTRIBUTING.md#界面验证),明确本次用户操作、原页面、最接近的同类实现及可复用基础组件;按本轮验收分工留存所需截图
确定页面方案或审查结论前,读取[前端职责](../../../../docs/architecture.md#34-前端模块职责)、[文案与信息层级](../../../../docs/theme.md#界面文案与信息层级)和[界面验证](../../../../CONTRIBUTING.md#界面验证)的完整章节;只改文案或局部样式也执行。明确本次用户操作、原页面、最接近的同类实现及可复用基础组件,沿用既有设计与请求层,按本轮验收分工留存所需截图
## 按改动补齐规范
用户要求看效果或自行验收时,按[源码联调](../../../../docs/development.md#源码联调)提供当前工作区的可访问预览与页面入口,确认热更新和依赖来源;需要模拟数据时用于实际页面并标明,不把静态稿、旧构建或仅有本机文件路径当作可用预览
@@ -12,7 +14,7 @@
| 改动 | 规范入口 |
| --- | --- |
| Vue 逻辑、请求或状态 | [项目约定](../../../../CONTRIBUTING.md#项目约定)的前端条目及上述前端职责 |
| Vue 组件、逻辑、请求或状态 | 使用 `$vue-best-practices`,读取[项目约定](../../../../CONTRIBUTING.md#项目约定)的前端条目;请求与接口合同同时执行开发入口的 API 分支 |
| 颜色、间距、字体、边框、阴影、局部 CSS 或交互样式 | [通用颜色消费](../../../../docs/theme.md#通用颜色消费)、[样式与命名约定](../../../../docs/theme.md#样式与命名约定),核对无边常态及 Hover、Focus 等视觉状态 |
| 主题算法、编辑器或共用组件视觉行为 | 所改主题章节及[扩大验证范围](../../../../docs/theme.md#验证) |
| 明确指定设计技能,或建立、重塑视觉方向 | 使用指定技能或 `$frontend-design`,设计取舍仍以项目主题、用户任务和已有页面为依据 |
@@ -1,7 +1,7 @@
# 插件职责边界
仅在开发或审查 SDK、Runtime、宿主扩展接口及独立网关插件时读取
先看[插件扩展](../../../../docs/architecture.md#31-插件扩展)中的职责与执行边界,不通读整份架构
开发、排查或审查插件接入、SDK、Runtime、宿主扩展接口及独立网关插件时完整读取,不以文件位于插件目录或正在选择能力为条件
确定方案前读取[职责与源码入口](../../../../docs/architecture.md#职责与源码入口)、[发布与执行边界](../../../../docs/architecture.md#发布与执行边界)及[信任与资源边界](../../../../docs/architecture.md#信任与资源边界),明确问题归属、必要宿主保护与插件责任
**宿主提供可组合能力,守住自身合同、资源与稳定性;插件业务正确性和兼容修复由插件负责**
+5 -3
View File
@@ -13,13 +13,15 @@ description: 草拟、创建、补充或回复 GitHub Issue,包括缺陷、功
补充已有 Issue 先读原帖和讨论,按授权更新或追加,不重复新建;无法推断的目标或关键事实才向用户补齐
## 按需取材
## 按动作读取与核实
- 新建 Issue 时,本仓库只读[提交 Issue](../../../CONTRIBUTING.md#提交-issue)及本次匹配的一张表单:[接口问题](../../../.github/ISSUE_TEMPLATE/api-bug-report.yml)、[常规问题](../../../.github/ISSUE_TEMPLATE/general-bug-report.yml)或[功能建议](../../../.github/ISSUE_TEMPLATE/feature-request.yml);协作规则提案可用普通 Issue
目标、授权、事实与脱敏要求适用于草拟、提交和回复;按下列动作读取必需规则与证据,再组织正文。症状反馈可保留未知根因,进入技术排查或方案判断时补齐对应规范
- 新建 Issue 时,读取本仓库的[提交 Issue](../../../CONTRIBUTING.md#提交-issue)及本次匹配的一张表单:[接口问题](../../../.github/ISSUE_TEMPLATE/api-bug-report.yml)、[常规问题](../../../.github/ISSUE_TEMPLATE/general-bug-report.yml)或[功能建议](../../../.github/ISSUE_TEMPLATE/feature-request.yml);协作规则提案可用普通 Issue
- 回复时读取原帖、相关讨论与支撑答复的证据,按下方回复口吻组织内容,不套用新建 Issue 的表单
- 其他仓库读取它自己的相关规则与实际模板,长文档先定位标题再截取,不预读全部模板或 CONTRIBUTING
- 搜索同场景 open/closed Issues、相关 PR 和发布记录,区分主分支修复与已发布修复,不凭标题判断重复;没有新证据时返回已有链接,用户仍要求新建则按其要求处理
- 仅当需要额外排查或修改时使用 [cpr-dev-guide](../cpr-dev-guide/SKILL.md),提交症状反馈不以找到根因为前提
- 需要技术排查、判断实现方案或修改时执行 [cpr-dev-guide](../cpr-dev-guide/SKILL.md)的领域必读,不能仅凭症状建议跨职责补偿;提交症状反馈不以找到根因为前提
- 引用上游行为或提出方案时执行[问题与方案依据](../../../CONTRIBUTING.md#问题与方案依据),注明适用版本、来源与缺口
## 正文与证据
+2 -2
View File
@@ -19,7 +19,7 @@ description: 准备、创建、更新或审查 GitHub PR,含回复讨论与处
## 按模式读取
只加载当前模式,已有上下文不重读;长文档先定位标题再截取对应章节,不因链接存在而加载全文或所有 references
先完成上述范围判断,再按当前动作与差异涉及的领域叠加下表。命中的流程引用完整读取,长文档读取指定章节;只复用已实际读取且仍适用的内容,单纯整理正文或回复不扩展为完整实现审查
| 当前动作 | 读取与执行 |
| --- | --- |
@@ -27,7 +27,7 @@ description: 准备、创建、更新或审查 GitHub PR,含回复讨论与处
| 创建或更新 PR | [创建流程](references/create.md)、[PR 流程](../../../CONTRIBUTING.md#pr-流程)与[规则文件保护](../../../CONTRIBUTING.md#规则文件保护) |
| 审查 PR | [审查流程](references/review.md)、[审查标准](../../../CONTRIBUTING.md#审查标准) |
| 回复讨论或审查意见 | 目标评论、相关上下文与实际处理结果,按下方回复口吻答复;仅回复不加载创建或完整审查流程 |
| 需要修改代码或文档 | 使用 [cpr-dev-guide](../cpr-dev-guide/SKILL.md)的对应分支,完成自审与验证 |
| 实现自审、审查或修改代码与文档 | 执行 [cpr-dev-guide](../cpr-dev-guide/SKILL.md)的领域必读与对应检查,包括后端、插件、测试和 API 等命中项;只读审查同样执行,不能仅核对 PR 正文与 CI |
| 含前端改动 | [前端流程](../cpr-dev-guide/references/frontend.md),核对适用规范与当前实现的运行证据 |
| 含文档改动 | [文档检查](../cpr-dev-guide/references/documentation.md) |
@@ -1,6 +1,6 @@
interface:
display_name: "cpr-github-pr"
short_description: "提交、更新与审查 PR,核对问题、自审、页面截图和 AI 模型披露"
default_prompt: "使用 $cpr-github-pr 按当前请求准备、提交、更新或审查 PR,核实问题与方案,检查自审、验证、页面截图和 AI 模型披露。"
default_prompt: "使用 $cpr-github-pr 按当前请求准备、提交、更新或审查 PR,实现自审与审查叠加开发指南的领域规范,核实问题、方案、验证、页面截图和 AI 模型披露。"
policy:
allow_implicit_invocation: true
@@ -4,6 +4,7 @@
1. 获取 PR 的 base/head SHA、描述、文件差异、已有讨论及 CI 状态。记录实际审查的 head,使用 merge-base 差异。
需要源码上下文时先遵守目标仓库的定位规则;需检出时使用隔离工作区,保留用户当前分支和运行环境。
按实际行为与文件执行 [cpr-dev-guide](../../cpr-dev-guide/SKILL.md)的领域必读,再判断方案与实现;只读审查也核对职责边界与生命周期。
2. 按入口公共核对在目标分支基线核实问题,区分原有问题、环境或配置差异与 PR 新引入的问题,说明主分支已覆盖的部分。
3. 按[问题与方案依据](../../../../CONTRIBUTING.md#问题与方案依据)评估解决方向,涉及上游时说明官方处理方式及本次方案关系;缺少必要性依据的偏离明确指出,不因测试通过就认定方案合理。
4. 按[审查标准](../../../../CONTRIBUTING.md#审查标准)追踪完整调用链,找到共享业务事实、同类实现及已有验证。
+11 -7
View File
@@ -1,13 +1,17 @@
---
name: cpr-plugin-dev
description: 开发、排查或打包 Codex Proxy RS 独立网关插件;仅改宿主 Runtime、安装器或管理端时使用 cpr-dev-guide,不适用于 Codex 插件或 Vue 插件
description: 开发、修复、排查、审查或打包 Codex Proxy RS 独立网关插件,含处理器、清单与插件页面;仅改宿主能力时使用 cpr-dev-guide 的插件边界,不适用于 Codex 插件或 Vue 插件
---
# 网关插件开发
插件在独立项目中通过公开 SDK 实现业务,不向宿主业务模块添加专属流程
## 先读职责边界
同时使用 [cpr-dev-guide](../cpr-dev-guide/SKILL.md)的对应任务分支,已加载则复用,不递归重读
插件在独立项目中通过公开 SDK 实现业务,插件业务错误在插件侧修复;宿主维护自身合同、资源与稳定性,不猜测插件意图、不补造业务数据,不在宿主、SDK 或 Runtime 中增加插件专属补偿
开发、排查和审查先完整读取[插件职责边界](../cpr-dev-guide/references/plugin-boundaries.md),再选择实现步骤;已有处理器修复和页面改动也执行,不以是否新增能力为条件。需要新增宿主能力时,先界定可独立使用的基础接口及其职责;必要的协议保护与资源回收仍由宿主负责
同时执行 [cpr-dev-guide](../cpr-dev-guide/SKILL.md)的领域必读与收尾要求;已实际读取且仍适用的内容复用,不递归重读入口。仅构建、打包或分发已有插件时按对应操作读取,若需诊断或修改行为,再补齐开发分支
## 范围与合同
@@ -18,13 +22,13 @@ description: 开发、排查或打包 Codex Proxy RS 独立网关插件;仅改
## 按当前步骤读取
只读取命中行与链接的指定章节,长文档先列标题再截取;不预读全部 SDK、示例、前端和打包资料
完成上述适用边界后,按当前步骤读取下表;同时涉及多项时叠加。命中的实现或页面参考完整读取,长文档读取指定章节,示例、前端和打包资料在对应步骤加载
| 当前工作 | 入口 |
| --- | --- |
| 选择能力或确认能力缺口 | [能力与方法](../../../backend/crates/gateway-plugin/sdk/docs/capabilities.md#能力与方法)、所需能力章节、[插件职责边界](../cpr-dev-guide/references/plugin-boundaries.md) |
| 清单与 Rust 处理器 | [实现参考](references/implementation.md)、所用 SDK 类型;只有修改清单字段时读[清单](../../../backend/crates/gateway-plugin/sdk/docs/manifest.md)对应章节 |
| 创建或修改页面 | [页面参考](references/frontend.md),不需要页面的插件不创建前端 |
| 选择、使用或审查能力 | [能力与方法](../../../backend/crates/gateway-plugin/sdk/docs/capabilities.md#能力与方法)及所用能力章节,不以已有调用能编译代替合同核对 |
| 开发、排查或审查清单与 Rust 处理器 | [实现参考](references/implementation.md)、所用 SDK 类型;涉及清单字段时读[清单](../../../backend/crates/gateway-plugin/sdk/docs/manifest.md)对应章节 |
| 创建、修改或审查页面 | [页面参考](references/frontend.md),不需要页面的插件不创建前端 |
| 准备工程或联调依赖 | [工程与依赖](references/development.md#工程与依赖),需要示例时再读[复用示例](references/development.md#复用示例) |
| 构建或打包 | [构建与打包](references/development.md#构建与打包) |
| 实际验收 | [验证与交付](references/development.md#验证与交付),按实现范围验证 |
@@ -1,6 +1,6 @@
interface:
display_name: "cpr-plugin-dev"
short_description: "网关插件开发、接入排障、SDK 实现、页面与打包验证"
default_prompt: "使用 $cpr-plugin-dev 按当前请求开发、排查或打包 Codex Proxy RS 网关插件,复用公开 SDK 与已有页面并验证实际行为。"
short_description: "独立网关插件开发、排障与审查,先核对宿主边界再实现和验证"
default_prompt: "使用 $cpr-plugin-dev 按当前请求开发、排查、审查或打包独立网关插件,先读取适用职责边界,插件业务错误在插件侧修复,再按步骤读取实现与验证资料。"
policy:
allow_implicit_invocation: true
@@ -1,6 +1,6 @@
# 插件页面
仅在创建或修改插件页面时读取,先执行[前端流程](../../cpr-dev-guide/references/frontend.md),再核对以下插件特有合同
创建、修改或审查插件页面时完整读取,先完成[插件入口](../SKILL.md#先读职责边界)的边界判断并执行[前端流程](../../cpr-dev-guide/references/frontend.md),再核对以下插件特有合同
- 优先参考独立示例仓库的 `examples/workbench/frontend/src/api/`:业务路由放在 `modules/`,`request.ts` 封装公开宿主桥;按需使用 `@codex-proxy/ui` 包出口,不跨仓导入宿主或 UI 的内部源码
- Vue 页面保留 SFC,逻辑、脚本和配置使用 TypeScript;只实现业务内容,标题和副标题由 `ManagementPage` 交给宿主显示
@@ -1,7 +1,7 @@
# 清单与处理器实现
仅在编写或排查插件清单、Rust 处理器时读取,涉及 Rust 时使用[开发入口](../../cpr-dev-guide/SKILL.md)的 Rust 分支
选择能力时只读 SDK 的[能力与方法](../../../../backend/crates/gateway-plugin/sdk/docs/capabilities.md#能力与方法)及所需能力章节
开发、排查或审查插件清单、Rust 处理器时完整读取,先完成[插件入口](../SKILL.md#先读职责边界)的边界判断,涉及 Rust 时执行[开发入口](../../cpr-dev-guide/SKILL.md)的 Rust 分支
使用能力前读取 SDK 的[能力与方法](../../../../backend/crates/gateway-plugin/sdk/docs/capabilities.md#能力与方法)及所用能力章节,已有实现的修复与审查也核对合同
## 清单与处理器
+14 -14
View File
@@ -15,20 +15,6 @@ description: Codex Proxy RS 的版本规划、发布说明、正式或预发行
其他仓库使用其发布入口与规则,不套用本仓库路径;规则已在上下文中时不重复读取
## 按任务读取
只读当前阶段和指定章节,先定位标题再截取,不预读整份 CONTRIBUTING、工作流或所有参考文件;实际发布前仍须核对发布入口的完整副作用
| 任务 | 读取范围 |
| --- | --- |
| 推荐版本号 | [基线与版本](references/preparation.md#基线与版本) |
| 编写或翻译发布说明 | [基线与版本](references/preparation.md#基线与版本)、[发布说明](references/preparation.md#发布说明) |
| 检查发行条件 | [发行准备检查](references/preparation.md#发行准备检查) |
| 实际发布 | 完成准备,再读[通道核对](references/operations.md#发行通道核对)和[执行发版](references/operations.md#执行发版) |
| 失败恢复或验收 | [恢复与验收](references/operations.md#恢复与验收),先确定已完成步骤,不直接重跑入口 |
| 实验版 | 在对应阶段补读[实验流程](references/experimental.md)的版本、文案或验收章节 |
| 修复代码、工作流或常规文档 | 使用 [cpr-dev-guide](../cpr-dev-guide/SKILL.md),有 PR 任务再使用 [cpr-github-pr](../cpr-github-pr/SKILL.md) |
## 必须保持的边界
- Codex 兼容性变化按[问题与方案依据](../../../CONTRIBUTING.md#问题与方案依据)核对已有依据与验证范围,缺少依据列为发行风险,不以 CI 通过代替
@@ -37,3 +23,17 @@ description: Codex Proxy RS 的版本规划、发布说明、正式或预发行
- 预发行不得覆盖 GitHub Latest 或镜像 `latest`,完成后回读核对
- `release/notes.md` 只承载当前发行的说明;常规文档修改另按开发入口执行[文档检查](../cpr-dev-guide/references/documentation.md)
- 交付版本、提交、Release/工作流链接与实际结果,未完成构建或缺失产物时不称发布成功
## 按任务读取
先落实上述范围与边界,再读取当前阶段的完整指定章节;实际发布包含准备与验收,不能只读执行命令。阶段变化时补齐读取,准备文案不加载执行流程,实际发布前须核对发布入口的完整副作用
| 任务 | 读取范围 |
| --- | --- |
| 推荐版本号 | [基线与版本](references/preparation.md#基线与版本) |
| 编写或翻译发布说明 | [基线与版本](references/preparation.md#基线与版本)、[发布说明](references/preparation.md#发布说明) |
| 检查发行条件 | [发行准备检查](references/preparation.md#发行准备检查) |
| 实际发布 | 先完成[基线与版本](references/preparation.md#基线与版本)、[发布说明](references/preparation.md#发布说明)、[发行准备检查](references/preparation.md#发行准备检查),再执行[通道核对](references/operations.md#发行通道核对)、[执行发版](references/operations.md#执行发版)与[恢复与验收](references/operations.md#恢复与验收) |
| 失败恢复或验收 | [恢复与验收](references/operations.md#恢复与验收),先确定已完成步骤,不直接重跑入口 |
| 实验版 | 在对应阶段补读[实验流程](references/experimental.md);实际发布必须覆盖分支与版本、发布能力、文案和执行与验收 |
| 修复代码、工作流或常规文档 | 使用 [cpr-dev-guide](../cpr-dev-guide/SKILL.md),有 PR 任务再使用 [cpr-github-pr](../cpr-github-pr/SKILL.md) |
+1 -1
View File
@@ -14,7 +14,7 @@ permissions:
jobs:
container:
runs-on: ubuntu-latest
runs-on: ubuntu-26.04
timeout-minutes: 45
env:
CPR_IMAGE: codex-proxy-rs:ci-${{ inputs.ref }}
+8 -5
View File
@@ -24,7 +24,7 @@ permissions:
jobs:
backend:
if: inputs.run_backend
runs-on: ubuntu-latest
runs-on: ubuntu-26.04
timeout-minutes: 30
env:
CARGO_TERM_COLOR: always
@@ -34,7 +34,7 @@ jobs:
CPR_TEST_REDIS_URL: redis://127.0.0.1:6379
services:
postgres:
image: postgres:18-bookworm@sha256:d9c83446333daec3f0588cc709adb80c26090b7f9f0f7ec8d43c243385d79818
image: postgres:18-bookworm@sha256:bbf83594798e144bff365a9c191c5d779f181fb3135902064ab8ff11918020fc
env:
POSTGRES_DB: codex_proxy
POSTGRES_USER: codex_proxy
@@ -48,7 +48,7 @@ jobs:
--health-timeout 5s
--health-retries 10
redis:
image: redis:8-alpine@sha256:9d317178eceac8454a2284a9e6df2466b93c745529947f0cd42a0fa9609d7005
image: redis:8-alpine@sha256:0bf6577d7774e592b29d093e3ac9390eb91e13d72c21e506afce736de5e0eeb0
ports:
- 6379:6379
options: >-
@@ -199,7 +199,7 @@ jobs:
frontend:
if: inputs.run_frontend
runs-on: ubuntu-latest
runs-on: ubuntu-26.04
timeout-minutes: 15
env:
CI: 'true'
@@ -213,7 +213,7 @@ jobs:
uses: ./.github/actions/build-frontend
workflow-lint:
runs-on: ubuntu-latest
runs-on: ubuntu-26.04
timeout-minutes: 5
steps:
- name: Checkout
@@ -223,6 +223,8 @@ jobs:
# GitHub 已支持 queue: max,actionlint 尚未支持该字段
# https://github.com/rhysd/actionlint/issues/657
# GitHub 已提供 ubuntu-26.04,actionlint 尚未收录该标签
# https://github.com/actions/runner-images#available-images
- name: Lint GitHub Actions workflows
run: >-
docker run --rm
@@ -231,6 +233,7 @@ jobs:
rhysd/actionlint:1.7.7@sha256:887a259a5a534f3c4f36cb02dca341673c6089431057242cdc931e9f133147e9
-color
-ignore '^unexpected key "queue" for "concurrency" section\. expected one of "cancel-in-progress", "group"$'
-ignore '^label "ubuntu-26\.04" is unknown\.'
.github/workflows/*.yml
- name: Check workflow script syntax
+2 -77
View File
@@ -20,7 +20,7 @@ concurrency:
jobs:
changes:
runs-on: ubuntu-latest
runs-on: ubuntu-26.04
timeout-minutes: 5
permissions:
contents: read
@@ -29,7 +29,6 @@ jobs:
backend: ${{ steps.filter.outputs.backend }}
frontend: ${{ steps.filter.outputs.frontend }}
container: ${{ steps.filter.outputs.container }}
source: ${{ steps.filter.outputs.source }}
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -41,7 +40,7 @@ jobs:
id: filter
uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v4.0.3
with:
# 目录规则取并集,再统一排除说明文档,保留隐藏配置与子模块指针
# 目录规则取并集,再统一排除说明文档,保留隐藏配置
predicate-quantifier: some-with-excludes
filters: |
backend:
@@ -61,17 +60,6 @@ jobs:
- '.github/workflows/ci.yml'
- '.github/workflows/_quality.yml'
- '!**/*.md'
source:
- '.github/scripts/toolchains.rb'
- 'backend/rust-toolchain.toml'
- 'deploy/Dockerfile'
- '.gitmodules'
- 'modules/ui'
- 'modules/plugins'
- 'frontend/**'
- 'backend/crates/gateway-plugin/sdk/**'
- '.github/workflows/ci.yml'
- '!**/*.md'
container:
- 'backend/**'
- 'frontend/**'
@@ -101,66 +89,3 @@ jobs:
uses: ./.github/workflows/_container.yml
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
source-development:
needs: changes
if: needs.changes.outputs.source == 'true'
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Checkout with development modules
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
submodules: recursive
- name: Setup pnpm
uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413 # v6.1.0
with:
package_json_file: frontend/package.json
- name: Read build toolchains
id: toolchains
run: ruby .github/scripts/toolchains.rb >> "$GITHUB_OUTPUT"
- name: Setup Node
uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: ${{ steps.toolchains.outputs.node }}
- name: Setup Rust
uses: dtolnay/rust-toolchain@683b588c487a69d01f6c6ddf40f036ff682cc2c8 # master
with:
toolchain: ${{ steps.toolchains.outputs.rust }}
- name: Check dependency pins
run: |
ui_revision="$(git -C modules/ui rev-parse HEAD)"
for lockfile in frontend/pnpm-lock.yaml modules/plugins/examples/workbench/frontend/pnpm-lock.yaml; do
locked_revision="$(sed -nE 's@.*codeload.github.com/zyycn/codex-proxy-ui/tar.gz/([a-f0-9]{40}).*@\1@p' "$lockfile" | sort -u)"
if [ "$locked_revision" != "$ui_revision" ]; then
echo "::error file=$lockfile::UI dependency must match modules/ui ($ui_revision)"
exit 1
fi
done
- name: Install development dependencies
run: |
pnpm --dir modules/ui install --frozen-lockfile
pnpm --dir frontend install --frozen-lockfile
pnpm --dir modules/plugins/examples/workbench/frontend install --frozen-lockfile
- name: Check UI source integration
run: |
pnpm --dir frontend build:source
pnpm --dir modules/plugins/examples/workbench/frontend build:source
- name: Check local SDK integration
env:
RUST_MIN_STACK: 16777216
CARGO_TARGET_DIR: ${{ github.workspace }}/backend/target/plugin-development
run: |
sdk_check_dir="$RUNNER_TEMP/plugin-source"
mkdir -p "$sdk_check_dir/backend"
cp modules/plugins/examples/workbench/backend/Cargo.{toml,lock} "$sdk_check_dir/backend/"
cp -R modules/plugins/examples/workbench/backend/{src,tests} "$sdk_check_dir/backend/"
cp modules/plugins/examples/workbench/plugin.json "$sdk_check_dir/"
cargo test --manifest-path "$sdk_check_dir/backend/Cargo.toml" \
--config "patch.\"https://github.com/zyycn/codex-proxy-rs.git\".gateway-plugin-sdk.path=\"$GITHUB_WORKSPACE/backend/crates/gateway-plugin/sdk\""
- name: Ensure source checks preserve lockfiles
run: |
git diff --exit-code -- frontend/package.json frontend/pnpm-lock.yaml backend/Cargo.lock
git -C modules/plugins diff --exit-code -- examples/workbench/frontend/package.json examples/workbench/frontend/pnpm-lock.yaml examples/workbench/backend/Cargo.lock
+4 -4
View File
@@ -21,7 +21,7 @@ env:
jobs:
release-info:
runs-on: ubuntu-latest
runs-on: ubuntu-26.04
timeout-minutes: 10
outputs:
tag: ${{ steps.version.outputs.tag }}
@@ -73,7 +73,7 @@ jobs:
frontend:
needs: release-info
runs-on: ubuntu-latest
runs-on: ubuntu-26.04
timeout-minutes: 15
env:
CI: 'true'
@@ -205,7 +205,7 @@ jobs:
package-assets:
needs: [release-info, frontend, build-docker-binary, build-binary]
runs-on: ubuntu-latest
runs-on: ubuntu-26.04
timeout-minutes: 20
permissions:
contents: read
@@ -375,7 +375,7 @@ jobs:
queue: max
# 产物准备可与测试重叠,版本标签和 GitHub Release 必须等全部质量与镜像验证通过
needs: [release-info, quality, package-assets, build-docker-image]
runs-on: ubuntu-latest
runs-on: ubuntu-26.04
timeout-minutes: 20
permissions:
contents: write
+1 -1
View File
@@ -18,7 +18,7 @@ concurrency:
jobs:
guard:
runs-on: ubuntu-latest
runs-on: ubuntu-26.04
timeout-minutes: 5
steps:
# 只查询 PR 元数据,禁止检出或执行贡献分支中的代码。
+2 -2
View File
@@ -19,7 +19,7 @@ concurrency:
jobs:
backend-security:
runs-on: ubuntu-latest
runs-on: ubuntu-26.04
timeout-minutes: 20
steps:
- name: Checkout
@@ -38,7 +38,7 @@ jobs:
cargo-audit audit --file backend/Cargo.lock
frontend-security:
runs-on: ubuntu-latest
runs-on: ubuntu-26.04
timeout-minutes: 15
steps:
- name: Checkout
+5 -3
View File
@@ -5,14 +5,16 @@
| 任务意图与常见说法 | 必用技能 |
| --- | --- |
| 开发、修复、排障、代码审查、改文档;包括“改弹窗/样式/文案”“修接口/调度/数据库”等省略请求 | [cpr-dev-guide](.agents/skills/cpr-dev-guide/SKILL.md),按实际操作和文件匹配规范,跨前后端同时执行 |
| 提交/提个 PR、准备或更新 PR、看下 PR、回复 PR 讨论、处理 PR 审查意见 | [cpr-github-pr](.agents/skills/cpr-github-pr/SKILL.md);需要修改时同时执行 cpr-dev-guide |
| 提交/提个 PR、准备或更新 PR、看下 PR、回复 PR 讨论、处理 PR 审查意见 | [cpr-github-pr](.agents/skills/cpr-github-pr/SKILL.md);实现自审、审查或修改同时执行 cpr-dev-guide |
| 提 Issue、反馈到仓库、把问题或建议整理成 Issue、补充或回复 Issue | [cpr-github-issue](.agents/skills/cpr-github-issue/SKILL.md) |
| 写网关插件、修改插件能力、排查插件接入、打包插件 | [cpr-plugin-dev](.agents/skills/cpr-plugin-dev/SKILL.md),同时执行 cpr-dev-guide |
| 写、修或审查独立网关插件,排查插件接入、打包插件 | [cpr-plugin-dev](.agents/skills/cpr-plugin-dev/SKILL.md),同时执行 cpr-dev-guide;仅改宿主能力时由 cpr-dev-guide 加载插件边界 |
| 发版、准备版本、发布 beta/实验版、整理发布说明、打发行 tag | [cpr-release](.agents/skills/cpr-release/SKILL.md) |
结合上下文识别“提交吧”“看看这个问题”等省略说法:普通 Git 提交不自动变成 PR,代码排查不自动变成 Issue;审查与发版准备不代表获准远程写入
只加载当前任务命中的技能、模式和文档章节,复用已有上下文,进入新阶段再补读;链接是按需入口,不是全文或全部参考文件的必读清单
先完整读取命中技能的 `SKILL.md`,再读取其领域必读规范,之后按当前步骤补充实现、命令和示例。技能名称、描述或入口摘要不能代替正文与必读引用;引用中明确要求“先读”“必须读取”或“执行”的规范也要落实,其他资料按需读取
按用户意图、行为影响和实际文件共同匹配,跨领域叠加;确定方案或给出审查结论前完成对应领域必读,不能因改动小、路径未落在该模块或自认为熟悉而跳过。只复用当前上下文中已实际读取且仍适用的内容;输出截断、上下文压缩后缺失的必要内容须补读,范围或阶段变化时重新匹配
执行所选流程中的问题核实、自审、验证与交付要求,协作流程和审查标准见 [CONTRIBUTING.md](CONTRIBUTING.md)
+1 -1
View File
@@ -228,7 +228,7 @@ git diff --check
## 自动检查与合并条件
[CI](.github/workflows/ci.yml) 按变更范围调用质量检查,Markdown 不触发后端、前端、容器和源码联调任务,工作流语法检查独立执行
[CI](.github/workflows/ci.yml) 按变更范围调用质量检查,Markdown 不触发后端、前端和容器任务,工作流语法检查独立执行
后端 CI 使用 nextest 执行 `main` 集成测试,默认 Redis 密码模式执行完整测试集,ACL 用户模式复用同一批构建产物执行 Store 的 Redis 测试;本地验证命令见 [架构文档](docs/architecture.md#验证命令)
+4 -4
View File
@@ -1129,9 +1129,9 @@ checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17"
[[package]]
name = "crypto-bigint"
version = "0.7.5"
version = "0.7.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1a52aa3fcda4e6302a9f48734f234d35d4721b96f8fe07d073f07ce9df4f0271"
checksum = "6bb08299a6046833a879ace115697f12fd2bd6c8d3e93c909ffc9e0738e7cae0"
dependencies = [
"cpubits",
"ctutils",
@@ -4655,9 +4655,9 @@ checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20"
[[package]]
name = "tokio"
version = "1.52.3"
version = "1.53.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fc7f01b389ac15039e4dc9531aa973a135d7a4135281b12d7c1bc79fd57fffe"
checksum = "ce3335fa71841cda333a58d7615b03901380ecf09d59b3296d21f8bbac0dde4e"
dependencies = [
"bytes",
"libc",
+21 -3
View File
@@ -370,10 +370,28 @@ fn config_loader_should_reject_zero_server_port() {
}
#[test]
fn config_loader_should_validate_openai_residency() {
fn config_loader_should_accept_optional_openai_routing_strings() {
for value in [
serde_json::Value::Null,
serde_json::json!("us"),
serde_json::json!("custom-value"),
serde_json::json!(""),
] {
let mut document = valid_config_document();
document["openai"]["residency"] = value.clone();
document["openai"]["account_routing_override"] = value;
parse_config(&document.to_string()).expect("optional routing strings");
}
let mut document = valid_config_document();
document["openai"]["residency"] = serde_json::json!("invalid");
assert_rejected(document.to_string());
document["openai"]
.as_object_mut()
.unwrap()
.remove("residency");
document["openai"]
.as_object_mut()
.unwrap()
.remove("account_routing_override");
parse_config(&document.to_string()).expect("omitted routing strings");
}
fn assert_rejected(config: String) {
+1 -1
View File
@@ -250,7 +250,7 @@ fn source_manifest() -> &'static [u8] {
"main":"bin/plugin",
"runtime":"trustedProcess",
"contributes":{
"middleware":{"version":5,"stages":["request"],"inputFormats":["openai"],"outputFormats":["openai"]},
"middleware":{"version":6,"stages":["request"],"inputFormats":["openai"],"outputFormats":["openai"]},
"management":{"version":2}
},
@@ -740,7 +740,6 @@ pub struct UsageListRecord {
pub provider_account_authentication_kind: Option<String>,
pub upstream_model_id: Option<String>,
pub upstream_transport: Option<String>,
pub upstream_response_model: Option<String>,
pub service_tier: Option<String>,
pub input_tokens: Option<u64>,
pub output_tokens: Option<u64>,
@@ -802,7 +801,6 @@ pub struct UsageRecord {
pub upstream_transport: Option<String>,
pub http_version: Option<String>,
pub websocket_pool: Option<String>,
pub upstream_response_model: Option<String>,
pub service_tier: Option<String>,
/// Provider 已筛选的专有观测 JSON;管理领域保持不透明
pub provider_metadata_json: Option<String>,
@@ -1361,7 +1361,6 @@ fn total_record(
provider_account_authentication_kind: None,
upstream_model_id: Some("gpt-5.5".to_owned()),
upstream_transport: None,
upstream_response_model: None,
service_tier: None,
input_tokens: Some(800),
output_tokens: Some(200),
@@ -28,13 +28,12 @@ pub(super) fn build(document: &Value) -> io::Result<Vec<u8>> {
let detail = &evidence["detail"]["request"];
writeln!(
archive,
"\n结果:{};状态码:{};尝试数:{}\n模型:请求={},发送={},上游报告={}\n传输:{} → {}\n时延:{}",
"\n结果:{};状态码:{};尝试数:{}\n模型:请求={},发送={}\n传输:{} → {}\n时延:{}",
detail["logicalOutcome"],
detail["statusCode"],
detail["attemptCount"],
detail["requestedModel"],
detail["upstreamModel"],
detail["upstreamResponseModel"],
detail["clientTransport"],
detail["upstreamTransport"],
detail["latencyDetails"]
@@ -279,7 +279,6 @@ pub(crate) fn usage_list_record_view(
model,
requested_model: record.requested_model_id,
upstream_model: record.upstream_model_id,
upstream_response_model: record.upstream_response_model,
service_tier: record.service_tier,
client_transport: record.client_transport,
upstream_transport: record.upstream_transport,
@@ -377,7 +376,6 @@ pub(crate) fn usage_record_view(
model,
requested_model: record.requested_model_id,
upstream_model: record.upstream_model_id,
upstream_response_model: record.upstream_response_model,
service_tier: record.service_tier,
status_code,
client_transport: record.client_transport,
@@ -103,7 +103,6 @@ pub struct UsageListRecordView {
pub model: Option<String>,
pub requested_model: Option<String>,
pub upstream_model: Option<String>,
pub upstream_response_model: Option<String>,
pub service_tier: Option<String>,
pub client_transport: String,
pub upstream_transport: Option<String>,
@@ -142,7 +141,6 @@ pub struct UsageRecordView {
pub model: Option<String>,
pub requested_model: Option<String>,
pub upstream_model: Option<String>,
pub upstream_response_model: Option<String>,
pub service_tier: Option<String>,
pub status_code: Option<i64>,
pub client_transport: String,
@@ -891,7 +891,7 @@ impl StreamingExecutionBody {
}
continue;
}
// 已提交的流在确认 Core 终结后才交付成功终态,不能先输出 completed 再报结算失败
// 已提交的流先确认 Core 终结,避免后续 EOF 验证推翻已交付的成功终态
if self.awaiting_terminal_eof && self.committed {
self.verify_terminal_eof().await?;
encoded.push_back(done_frame());
@@ -5,7 +5,7 @@ use super::{
pump::run_pump,
state::{ConnectionCommand, ConnectionStats},
};
use axum::extract::ws::{CloseFrame, Message, WebSocket, close_code};
use axum::extract::ws::{CloseFrame, Message, Utf8Bytes, WebSocket, close_code};
use futures::{Sink, Stream};
use gateway_core::{
engine::middleware::{FrozenMiddlewarePlan, MiddlewareHeader},
@@ -142,7 +142,7 @@ impl ResponsesWebSocketConnection {
/// pump 已关闭、写入超时或 transport 失败时返回稳定错误
pub async fn send_text(
&mut self,
payload: String,
payload: impl Into<Utf8Bytes>,
context: WriteContext,
) -> Result<WriteOutcome, ConnectionWriteError> {
self.send(Message::Text(payload.into()), context).await
@@ -3,7 +3,7 @@
use std::collections::VecDeque;
use std::sync::{Arc, Mutex};
use axum::http::StatusCode;
use axum::{extract::ws::Utf8Bytes, http::StatusCode};
use bytes::Bytes;
use futures::future::BoxFuture;
use gateway_core::engine::CommitRequirement;
@@ -268,7 +268,7 @@ pub(super) async fn forward_response(
return fail_body(connection, body, MiddlewareError::InvalidState, &request_id).await;
}
let terminal = frame.terminal();
let Ok(message) = String::from_utf8(frame.into_bytes().to_vec()) else {
let Ok(message) = Utf8Bytes::try_from(frame.into_bytes()) else {
return fail_body(connection, body, MiddlewareError::InvalidState, &request_id).await;
};
let phase = match (first_frame_written, terminal) {
@@ -764,7 +764,6 @@ pub(super) fn usage_record_with_account(
upstream_transport: Some("http_sse".to_owned()),
http_version: Some("h2".to_owned()),
websocket_pool: None,
upstream_response_model: None,
service_tier: None,
provider_metadata_json: None,
attempt_count: 1,
@@ -909,7 +908,6 @@ async fn usage_route_should_expose_table_facts_without_detail_payload() {
provider_account_authentication_kind: Some("oauth".to_owned()),
upstream_model_id: Some("grok-4.5".to_owned()),
upstream_transport: Some("http_sse".to_owned()),
upstream_response_model: Some("grok-4.6-build".to_owned()),
service_tier: Some("default".to_owned()),
input_tokens: Some(1),
output_tokens: Some(1),
@@ -1059,7 +1057,6 @@ async fn usage_route_should_expose_table_facts_without_detail_payload() {
"imageOutputTokens": value["data"]["items"][0]["tokenDetails"]["imageOutputTokens"],
"requestedModel": value["data"]["items"][0]["requestedModel"],
"upstreamModel": value["data"]["items"][0]["upstreamModel"],
"upstreamResponseModel": value["data"]["items"][0]["upstreamResponseModel"],
"reasoningEffort": value["data"]["items"][0]["reasoningEffort"],
"reasoningPreset": value["data"]["items"][0]["reasoningPreset"],
"subagentKind": value["data"]["items"][0]["subagentKind"],
@@ -1083,7 +1080,6 @@ async fn usage_route_should_expose_table_facts_without_detail_payload() {
"imageOutputTokens": 9,
"requestedModel": "grok-4.5",
"upstreamModel": "grok-4.5",
"upstreamResponseModel": "grok-4.6-build",
"reasoningEffort": "max",
"reasoningPreset": "ultra",
"subagentKind": "worker",
@@ -110,7 +110,6 @@ fn usage_record() -> UsageListRecord {
provider_account_authentication_kind: Some("private-sentinel".to_owned()),
upstream_model_id: Some("private-sentinel".to_owned()),
upstream_transport: Some("websocket".to_owned()),
upstream_response_model: Some("private-sentinel".to_owned()),
service_tier: Some("private-sentinel".to_owned()),
input_tokens: Some(1000),
output_tokens: Some(100),
@@ -9,4 +9,4 @@ mod trace;
pub use capture::{body_fingerprint, diagnostic_headers, diagnostic_json, json_fingerprint};
pub use failure::{OperationalDiagnostics, OperationalFailure};
pub use stream::{StreamCapture, StreamFormat};
pub use trace::TraceContext;
pub use trace::{MAX_TRACE_SNAPSHOT_BYTES, TraceContext};
@@ -15,13 +15,16 @@ use crate::error::ProviderError;
use super::capture::{bounded, diagnostic_event_json, diagnostic_event_type};
use super::{body_fingerprint, diagnostic_headers};
/// 完整诊断快照的编码上限,生产者与持久化适配器共用
pub const MAX_TRACE_SNAPSHOT_BYTES: usize = 128 * 1024;
const MAX_EVENTS: usize = 128;
const HEAD_EVENTS: usize = 8;
const MAX_DATA_BYTES: usize = 4096;
const MAX_BUFFER_BYTES: usize = 64 * 1024;
const MAX_BUFFER_BYTES: usize = MAX_TRACE_SNAPSHOT_BYTES / 2;
const MAX_EVIDENCE_EVENTS: usize = 64;
const MAX_EVIDENCE_DATA_BYTES: usize = 16 * 1024;
const MAX_EVIDENCE_BUFFER_BYTES: usize = 64 * 1024;
const MAX_EVIDENCE_BUFFER_BYTES: usize = MAX_TRACE_SNAPSHOT_BYTES / 2;
/// 显式传给异步任务的关联上下文;默认值禁用捕获,避免全局请求映射
#[derive(Clone, Default)]
@@ -284,27 +287,22 @@ impl TraceContext {
/// 与请求终态原子持久化;旧记录没有此字段,不伪造历史时间线
#[must_use]
pub fn snapshot(&self) -> Option<Value> {
let state = self
let mut state = self
.state
.as_ref()?
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
Some(json!({
"schemaVersion": 3, "requestId": state.request_id,
"startedAtUnixMs": state.started_at_ms, "totalEvents": state.sequence,
"droppedEvents": state.dropped_events, "maxEvents": MAX_EVENTS,
"captureMode": "sanitized", "events": state.events,
"wireDumpEnabled": tracing::enabled!(target: "request_dump", tracing::Level::INFO),
"wireFrames": state.wire_frames, "wireBytes": state.wire_bytes,
"maxBufferBytes": MAX_BUFFER_BYTES + MAX_EVIDENCE_BUFFER_BYTES, "headEvents": HEAD_EVENTS,
"maxEvidenceEvents": MAX_EVIDENCE_EVENTS,
"maxEvidenceDataBytes": MAX_EVIDENCE_DATA_BYTES,
"maxEvidenceBufferBytes": MAX_EVIDENCE_BUFFER_BYTES,
"droppedEvidenceEvents": state.dropped_evidence_events,
"stageCounts": state.stage_counts,
"stageCountOverflow": state.stage_count_overflow,
"snapshotBoundary": "execution_finalization",
}))
loop {
let snapshot = state.snapshot();
if snapshot.to_string().len() <= MAX_TRACE_SNAPSHOT_BYTES {
return Some(snapshot);
}
// 完整编码还包含计数与外层字段,超限时先淘汰普通事件并保留丢弃计数
// 元数据本身无法容纳时省略诊断,不能让辅助观测阻断请求终态写入
let index =
regular_eviction_index(&state).or_else(|| evidence_eviction_index(&state))?;
remove_event(&mut state, index);
}
}
fn push(&self, stage: &'static str, data: Value, coalesce: bool) {
@@ -439,26 +437,14 @@ impl TraceContext {
if state.evidence_bytes > MAX_EVIDENCE_BUFFER_BYTES
|| state.evidence_events > MAX_EVIDENCE_EVENTS
{
if let Some(index) = state
.events
.iter()
.position(|event| event.evidence && event.stage != "request.routed")
.or_else(|| state.events.iter().position(|event| event.evidence))
{
if let Some(index) = evidence_eviction_index(&state) {
remove_event(&mut state, index);
}
continue;
}
// 相比流中间的常规事件,优先保留失败与重试决策
// 关键事实使用独立预算,只有自身超限才会淘汰
let index = (HEAD_EVENTS..state.events.len().saturating_sub(8))
.find(|index| !state.events[*index].evidence)
.or_else(|| {
(HEAD_EVENTS..state.events.len())
.chain(0..HEAD_EVENTS.min(state.events.len()))
.find(|index| !state.events[*index].evidence)
});
if let Some(index) = index {
if let Some(index) = regular_eviction_index(&state) {
remove_event(&mut state, index);
} else {
break;
@@ -467,6 +453,45 @@ impl TraceContext {
}
}
impl TraceState {
fn snapshot(&self) -> Value {
json!({
"schemaVersion": 3, "requestId": self.request_id,
"startedAtUnixMs": self.started_at_ms, "totalEvents": self.sequence,
"droppedEvents": self.dropped_events, "maxEvents": MAX_EVENTS,
"captureMode": "sanitized", "events": self.events,
"wireDumpEnabled": tracing::enabled!(target: "request_dump", tracing::Level::INFO),
"wireFrames": self.wire_frames, "wireBytes": self.wire_bytes,
"maxBufferBytes": MAX_BUFFER_BYTES + MAX_EVIDENCE_BUFFER_BYTES, "headEvents": HEAD_EVENTS,
"maxEvidenceEvents": MAX_EVIDENCE_EVENTS,
"maxEvidenceDataBytes": MAX_EVIDENCE_DATA_BYTES,
"maxEvidenceBufferBytes": MAX_EVIDENCE_BUFFER_BYTES,
"droppedEvidenceEvents": self.dropped_evidence_events,
"stageCounts": self.stage_counts,
"stageCountOverflow": self.stage_count_overflow,
"snapshotBoundary": "execution_finalization",
})
}
}
fn regular_eviction_index(state: &TraceState) -> Option<usize> {
(HEAD_EVENTS..state.events.len().saturating_sub(8))
.find(|index| !state.events[*index].evidence)
.or_else(|| {
(HEAD_EVENTS..state.events.len())
.chain(0..HEAD_EVENTS.min(state.events.len()))
.find(|index| !state.events[*index].evidence)
})
}
fn evidence_eviction_index(state: &TraceState) -> Option<usize> {
state
.events
.iter()
.position(|event| event.evidence && event.stage != "request.routed")
.or_else(|| state.events.iter().position(|event| event.evidence))
}
fn remove_event(state: &mut TraceState, index: usize) {
if let Some(event) = state.events.remove(index) {
state.bytes = state.bytes.saturating_sub(event.bytes);
@@ -44,9 +44,9 @@ pub struct ClientBudgetCharge {
pub struct ClientBudgetError(#[source] pub Option<crate::error::ErrorSource>);
pub trait ClientBudgetPort: Send + Sync {
/// 原子检查当前限额与已用金额,不创建预扣费或待结算记录
/// 检查当前限额、已用金额与已接管的费用,不预占未来费用
fn admit(&self, key_id: ClientApiKeyId) -> BoxFuture<'_, Result<(), GatewayError>>;
/// 按网关请求 ID 幂等累计已取得费用;写入失败由 Store 重试
/// 按网关请求 ID 幂等接管已取得费用;运行时入队后返回,由 Store 后台落库和重试
fn settle(&self, charge: ClientBudgetCharge) -> BoxFuture<'_, Result<(), ClientBudgetError>>;
}
@@ -2302,12 +2302,6 @@ where
http_version,
websocket_pool,
service_tier,
upstream_response_model: self
.current
.as_ref()
.and_then(|current| current.response_observation.as_ref())
.and_then(ProviderResponseObservation::upstream_response_model)
.map(str::to_owned),
provider_metadata_json,
diagnostic_trace_json: self.trace.snapshot().map(|value| value.to_string()),
error: None,
@@ -2490,12 +2484,6 @@ where
http_version,
websocket_pool,
service_tier,
upstream_response_model: self
.current
.as_ref()
.and_then(|current| current.response_observation.as_ref())
.and_then(ProviderResponseObservation::upstream_response_model)
.map(str::to_owned),
provider_metadata_json,
diagnostic_trace_json: self.trace.snapshot().map(|value| value.to_string()),
error: Some(finalization.error),
@@ -32,7 +32,7 @@ pub(super) async fn settle_budget(
diagnostics,
&request_id,
"settle_client_budget",
"Client budget settlement failed; storage will retry on the next request",
"Client budget settlement could not be accepted",
error,
)
.await;
@@ -43,6 +43,8 @@ pub enum MiddlewareMount {
Request,
/// 受管 Provider attempt 内层;每次真实 retry 都重新建立
Attempt,
/// Provider 完成序列化后、提交实际发送前的请求准备
Outbound,
}
/// 中间件正文块的宿主边界
@@ -2,6 +2,7 @@
mod contract;
pub mod http;
pub mod outbound;
mod plan;
pub mod service;
pub mod websocket;
@@ -0,0 +1,140 @@
//! Provider 发送前的完整请求准备;续体只组合值,不执行网络操作
use std::fmt;
use bytes::Bytes;
use futures::FutureExt;
use super::{FrozenMiddlewarePlan, MiddlewareContext, MiddlewareError, MiddlewareHeader};
use crate::error::{ProviderError, ProviderErrorKind};
use crate::upstream::UpstreamSendState;
/// 当前发送所用的传输类型与目标;准备过程中不能切换传输类型
#[derive(Clone, PartialEq, Eq)]
pub enum Target {
Http {
method: http::Method,
uri: http::Uri,
},
WebSocket {
uri: http::Uri,
},
}
impl fmt::Debug for Target {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Http { method, .. } => formatter
.debug_struct("Http")
.field("method", method)
.finish_non_exhaustive(),
Self::WebSocket { .. } => formatter.debug_struct("WebSocket").finish_non_exhaustive(),
}
}
}
/// HTTP 正文已完成序列化与内容编码;WebSocket 正文是一条完整文本消息
///
/// WebSocket headers 表示候选 opening,不能改写已经建立的连接握手
#[derive(Clone, PartialEq, Eq)]
pub struct Request {
pub target: Target,
pub headers: Vec<MiddlewareHeader>,
pub body: Bytes,
}
impl Request {
/// 转换实际发送的 HTTP 头,保留多值与原始字节,不修复插件输出
pub fn header_map(&self) -> Result<http::HeaderMap, ProviderError> {
let mut headers = http::HeaderMap::try_with_capacity(self.headers.len())
.map_err(|_| preparation_error(ProviderErrorKind::InvalidRequest))?;
for header in &self.headers {
let name = http::HeaderName::from_bytes(header.name().as_bytes())
.map_err(|_| preparation_error(ProviderErrorKind::InvalidRequest))?;
let value = http::HeaderValue::from_bytes(header.value())
.map_err(|_| preparation_error(ProviderErrorKind::InvalidRequest))?;
headers
.try_append(name, value)
.map_err(|_| preparation_error(ProviderErrorKind::InvalidRequest))?;
}
Ok(headers)
}
}
impl fmt::Debug for Request {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("Request")
.field("target", &self.target)
.field("headers", &self.headers.len())
.field("body_bytes", &self.body.len())
.finish()
}
}
pub type Next = crate::middleware::Next<Request, Request, MiddlewareError>;
/// 保活本次 attempt 的冻结发布代次、归属和资源上下文
#[derive(Clone, Debug)]
pub struct Pipeline {
plan: FrozenMiddlewarePlan,
context: MiddlewareContext,
}
impl Pipeline {
pub(in crate::engine) const fn new(
plan: FrozenMiddlewarePlan,
context: MiddlewareContext,
) -> Self {
Self { plan, context }
}
/// 完整返回洋葱链最终输出后,Provider 才能提交实际发送
pub async fn prepare(&self, request: Request) -> Result<Request, ProviderError> {
if self.context.cancellation().is_cancelled() {
return Err(preparation_error(ProviderErrorKind::Cancelled));
}
if self.context.deadline().is_elapsed() {
return Err(preparation_error(ProviderErrorKind::Timeout));
}
let target_kind = std::mem::discriminant(&request.target);
let next =
crate::middleware::compose(Vec::new(), |request| Box::pin(async { Ok(request) }));
let prepare = self
.plan
.handle_outbound(self.context.clone(), request, next)
.fuse();
let cancelled = self.context.cancellation().cancelled().fuse();
let expired = self.context.deadline().wait().fuse();
futures::pin_mut!(prepare, cancelled, expired);
let prepared = futures::select_biased! {
() = cancelled => return Err(preparation_error(ProviderErrorKind::Cancelled)),
() = expired => return Err(preparation_error(ProviderErrorKind::Timeout)),
result = prepare => result.map_err(|error| {
crate::engine::provider::middleware_error(error, UpstreamSendState::NotSent)
.with_retry_prohibited()
})?,
};
if self.context.cancellation().is_cancelled() {
return Err(preparation_error(ProviderErrorKind::Cancelled));
}
if self.context.deadline().is_elapsed() {
return Err(preparation_error(ProviderErrorKind::Timeout));
}
if std::mem::discriminant(&prepared.target) != target_kind
|| self
.context
.request_body_limit()
.is_some_and(|limit| prepared.body.len() > limit)
{
return Err(preparation_error(ProviderErrorKind::InvalidRequest));
}
Ok(prepared)
}
}
fn preparation_error(kind: ProviderErrorKind) -> ProviderError {
ProviderError::new(kind, UpstreamSendState::NotSent)
.with_retry_prohibited()
.redact_sensitive_context("invalid outbound request preparation")
}
@@ -9,6 +9,18 @@ use std::{fmt, sync::Arc};
/// 一个发布代次冻结的统一数据面中间件计划
pub trait MiddlewarePlan: Send + Sync + fmt::Debug {
fn has_outbound(&self, _context: &MiddlewareContext) -> bool {
false
}
fn handle_outbound(
&self,
_context: MiddlewareContext,
request: super::outbound::Request,
next: super::outbound::Next,
) -> BoxFuture<'static, Result<super::outbound::Request, MiddlewareError>> {
next.run(request)
}
fn has_service(&self) -> bool {
false
}
@@ -73,6 +85,20 @@ pub struct FrozenMiddlewarePlan {
}
impl FrozenMiddlewarePlan {
#[must_use]
pub fn has_outbound(&self, context: &MiddlewareContext) -> bool {
self.plan.has_outbound(context)
}
pub fn handle_outbound(
&self,
mut context: MiddlewareContext,
request: super::outbound::Request,
next: super::outbound::Next,
) -> BoxFuture<'static, Result<super::outbound::Request, MiddlewareError>> {
context.plan = Some(self.clone());
self.plan.handle_outbound(context, request, next)
}
#[must_use]
pub fn has_service(&self) -> bool {
self.plan.has_service()
+87 -14
View File
@@ -575,11 +575,48 @@ impl AttemptContext {
&self,
provider: &ProviderKind,
model: &UpstreamModelId,
) -> Result<upstream_adapter::UpstreamAdapterSelection, ProviderError> {
let account = self
.required_account()
.or_else(|| match self.continuation_attempt() {
ContinuationAttempt::Native => self
.continuation()
.and_then(continuation::ContinuationBinding::pinned)
.map(|pin| pin.account()),
ContinuationAttempt::ReplayOwner => self
.account_state_owner()
.filter(|owner| owner.provider() == provider)
.map(|owner| owner.account()),
ContinuationAttempt::None | ContinuationAttempt::ReplayAny => None,
});
self.request.upstream_adapters.as_ref().map_or(
Ok(upstream_adapter::UpstreamAdapterSelection::Native),
|plan| plan.select(self, provider, model, account),
)
}
/// 账号候选资格检查和最终执行解析使用同一冻结计划,不调用插件进程
pub fn upstream_adapter_for_account(
&self,
provider: &ProviderKind,
model: &UpstreamModelId,
account: &ProviderAccountId,
) -> Result<Option<Arc<dyn upstream_adapter::UpstreamAdapter>>, ProviderError> {
self.request
.upstream_adapters
.as_ref()
.map_or(Ok(None), |plan| plan.select(self, provider, model))
.map_or(Ok(None), |plan| {
match plan.select(self, provider, model, Some(account))? {
upstream_adapter::UpstreamAdapterSelection::Native => Ok(None),
upstream_adapter::UpstreamAdapterSelection::Adapter(adapter) => {
Ok(Some(adapter))
}
// 已知账号的解析必须给出最终执行端,不能绕过资格检查进入原生路径
upstream_adapter::UpstreamAdapterSelection::PendingAccount => Err(
ProviderError::new(ProviderErrorKind::Protocol, UpstreamSendState::NotSent),
),
}
})
}
#[must_use]
@@ -740,12 +777,57 @@ impl AttemptContext {
} else {
terminal
};
let context = middleware::MiddlewareContext::new(
let context = self.middleware_context(
middleware::MiddlewareMount::Attempt,
Some(operation.kind()),
provider,
model,
account_id,
);
provider::execute_attempt_middleware(
self.request.middleware.as_ref(),
context,
operation,
self.request.client_transport,
terminal,
)
.await
}
/// 为当前实际账号冻结出站准备链,无此阶段时保留 Provider 原生路径
#[must_use]
pub fn outbound_middleware(
&self,
provider: ProviderKind,
model: Option<String>,
account_id: ProviderAccountId,
) -> Option<middleware::outbound::Pipeline> {
let plan = self.request.middleware.as_ref()?;
let context = self.middleware_context(
middleware::MiddlewareMount::Outbound,
None,
provider,
model,
account_id,
);
plan.has_outbound(&context)
.then(|| middleware::outbound::Pipeline::new(plan.clone(), context))
}
fn middleware_context(
&self,
mount: middleware::MiddlewareMount,
operation: Option<crate::operation::OperationKind>,
provider: ProviderKind,
model: Option<String>,
account_id: ProviderAccountId,
) -> middleware::MiddlewareContext {
middleware::MiddlewareContext::new(
middleware::MiddlewareTarget {
request_id: self.request.request_id.clone(),
mount: middleware::MiddlewareMount::Attempt,
mount,
attempt_index: Some(self.attempt_index),
operation: Some(operation.kind()),
operation,
endpoint: self.request.endpoint.clone(),
transport: self.request.client_transport,
provider: Some(provider),
@@ -761,15 +843,7 @@ impl AttemptContext {
execution_effects: self.request.execution_effects.as_ref().map(Arc::clone),
},
)
.with_request_body_limit(self.request.request_body_limit);
provider::execute_attempt_middleware(
self.request.middleware.as_ref(),
context,
operation,
self.request.client_transport,
terminal,
)
.await
.with_request_body_limit(self.request.request_body_limit)
}
#[must_use]
@@ -996,7 +1070,6 @@ pub struct ModelRequestFinalization {
pub http_version: Option<String>,
pub websocket_pool: Option<String>,
pub service_tier: Option<String>,
pub upstream_response_model: Option<String>,
/// Provider 已筛选的专有观测 JSON;Core 不解释字段
pub provider_metadata_json: Option<String>,
/// 请求全程的有界诊断快照,跨 Provider 与重试保留
@@ -249,7 +249,6 @@ pub struct RequestObservation {
provider: Option<ProviderKind>,
account_id: Option<ProviderAccountId>,
upstream_model: Option<crate::routing::UpstreamModelId>,
response_model: Option<String>,
service_tier: Option<String>,
outcome: RequestObservationOutcome,
send_state: UpstreamSendState,
@@ -287,7 +286,6 @@ impl RequestObservation {
provider: None,
account_id: None,
upstream_model: None,
response_model: None,
service_tier: None,
outcome,
send_state,
@@ -341,11 +339,6 @@ impl RequestObservation {
self.upstream_model.as_ref()
}
#[must_use]
pub fn response_model(&self) -> Option<&str> {
self.response_model.as_deref()
}
#[must_use]
pub fn service_tier(&self) -> Option<&str> {
self.service_tier.as_deref()
@@ -626,7 +619,6 @@ impl RequestObservationDispatch {
outcome: RequestObservationOutcome::Rejected,
account_id: None,
upstream_model: None,
response_model: None,
service_tier: None,
send_state: UpstreamSendState::NotSent,
attempt_count: 0,
@@ -699,7 +691,6 @@ impl RequestObservationDispatch {
provider,
account_id: attempt.map(|attempt| attempt.provider_account_id().clone()),
upstream_model: attempt.and_then(|attempt| attempt.upstream_model().cloned()),
response_model: finalization.upstream_response_model.clone(),
service_tier: finalization.service_tier.clone(),
outcome,
send_state: finalization.send_state,
@@ -893,7 +893,10 @@ fn unprepared_middleware_response(
}
}
fn middleware_error(error: MiddlewareError, send_state: UpstreamSendState) -> ProviderError {
pub(super) fn middleware_error(
error: MiddlewareError,
send_state: UpstreamSendState,
) -> ProviderError {
match error {
MiddlewareError::Provider(error) => error,
MiddlewareError::Rejected | MiddlewareError::Remote { rejected: true, .. } => {
@@ -31,7 +31,16 @@ pub trait UpstreamAdapterPlan: fmt::Debug + Send + Sync {
context: &AttemptContext,
provider: &ProviderKind,
model: &UpstreamModelId,
) -> Result<Option<Arc<dyn UpstreamAdapter>>, ProviderError>;
account: Option<&ProviderAccountId>,
) -> Result<UpstreamAdapterSelection, ProviderError>;
}
/// 未确定账号时不能把账号范围声明视为原生执行或继续命中后置兜底
#[derive(Debug)]
pub enum UpstreamAdapterSelection {
Native,
Adapter(Arc<dyn UpstreamAdapter>),
PendingAccount,
}
/// 请求同时持有计划及发布集合,防止旧集合先排空进程而中断在途调用
@@ -58,8 +67,9 @@ impl FrozenUpstreamAdapterPlan {
context: &AttemptContext,
provider: &ProviderKind,
model: &UpstreamModelId,
) -> Result<Option<Arc<dyn UpstreamAdapter>>, ProviderError> {
self.plan.select(context, provider, model)
account: Option<&ProviderAccountId>,
) -> Result<UpstreamAdapterSelection, ProviderError> {
self.plan.select(context, provider, model, account)
}
}
@@ -71,6 +81,9 @@ pub trait UpstreamAdapter: fmt::Debug + Send + Sync {
/// 注册时声明的上游传输,用于宿主登记 attempt
fn transport(&self) -> &str;
/// 选号前复核注册的认证资格,发送前仍由执行端校验当前凭据
fn supports_authentication(&self, kind: &str) -> bool;
}
/// 一次执行的宿主事实;账号及 Provider 归属不可由插件返回值覆盖
-17
View File
@@ -624,7 +624,6 @@ pub struct ProviderResponseObservation {
status_code: Option<u16>,
request_id: Option<OpaqueUpstreamValue>,
service_tier: Option<String>,
upstream_response_model: Option<String>,
timings: ProviderResponseTimings,
// 流式事件分别持有快照,共享不可变头集合,避免逐事件复制头名和集合
client_headers: Arc<[ProviderResponseHeader]>,
@@ -641,7 +640,6 @@ impl ProviderResponseObservation {
status_code: None,
request_id: None,
service_tier: None,
upstream_response_model: None,
timings: ProviderResponseTimings::default(),
client_headers: Arc::default(),
provider_metadata: None,
@@ -697,21 +695,6 @@ impl ProviderResponseObservation {
self
}
/// 记录 Provider 从上游响应明确取得的模型,不改写发送模型或计费合同
#[must_use]
pub fn with_upstream_response_model_if_valid(mut self, model: &str) -> Self {
let model = model.trim();
if validate_text(model, 256, false, None).is_ok() {
self.upstream_response_model = Some(model.to_owned());
}
self
}
#[must_use]
pub fn upstream_response_model(&self) -> Option<&str> {
self.upstream_response_model.as_deref()
}
#[must_use]
pub const fn with_timings(mut self, timings: ProviderResponseTimings) -> Self {
self.timings = timings;
@@ -5,7 +5,7 @@ use std::{
sync::{Arc, Mutex},
};
use gateway_core::diagnostics::{TraceContext, body_fingerprint};
use gateway_core::diagnostics::{MAX_TRACE_SNAPSHOT_BYTES, TraceContext, body_fingerprint};
use gateway_core::error::{ProviderDiagnostic, ProviderError, ProviderErrorKind, RawUpstreamError};
use gateway_core::upstream::UpstreamSendState;
use serde_json::json;
@@ -47,6 +47,7 @@ fn critical_evidence_survives_stream_pressure_and_preserves_stage_coverage() {
);
assert_eq!(snapshot["droppedEvidenceEvents"], 0);
assert!(snapshot["droppedEvents"].as_u64().unwrap() > 0);
assert!(snapshot.to_string().len() <= MAX_TRACE_SNAPSHOT_BYTES);
}
#[test]
@@ -67,8 +68,42 @@ fn critical_evidence_has_an_independent_bound_and_reports_its_own_loss() {
events.len() as u64 + snapshot["droppedEvents"].as_u64().unwrap(),
100
);
assert!(snapshot.to_string().len() < 128 * 1024);
assert!(snapshot.to_string().len() <= MAX_TRACE_SNAPSHOT_BYTES);
}
#[test]
fn snapshot_budget_includes_stage_coverage_and_event_envelopes() {
let trace = TraceContext::new("req_snapshot_budget");
trace.record("request.routed", json!({"scope": "all"}));
for index in 0..10 {
trace.record_evidence(
"provider.request.input",
json!({"index": index, "hashes": "a".repeat(6000)}),
);
}
for index in 0..64 {
let stage = Box::leak(format!("stage.{index}.{}", "x".repeat(512)).into_boxed_str());
trace.record(stage, json!({"detail": "x".repeat(2000)}));
}
trace.record("request.finished", json!({"outcome": "succeeded"}));
let snapshot = trace.snapshot().unwrap();
let events = snapshot["events"].as_array().unwrap();
assert!(snapshot.to_string().len() <= MAX_TRACE_SNAPSHOT_BYTES);
assert_eq!(events.first().unwrap()["stage"], "request.routed");
assert_eq!(events.last().unwrap()["stage"], "request.finished");
assert_eq!(snapshot["droppedEvidenceEvents"], 0);
assert!(snapshot["droppedEvents"].as_u64().unwrap() > 0);
let retained: u64 = events
.iter()
.map(|event| event["count"].as_u64().unwrap())
.sum();
assert_eq!(
retained + snapshot["droppedEvents"].as_u64().unwrap(),
snapshot["totalEvents"]
);
assert_eq!(trace.snapshot(), Some(snapshot));
}
#[test]
fn bounded_history_preserves_start_failure_and_final_result() {
let trace = TraceContext::new("req_trace");
@@ -491,7 +491,6 @@ struct FinalState {
http_version: Option<String>,
websocket_pool: Option<String>,
service_tier: Option<String>,
upstream_response_model: Option<String>,
upstream_request_id: Option<String>,
upstream_status_code: Option<u16>,
transport_decision_wait_ms: Option<u64>,
@@ -682,7 +681,6 @@ impl ExecutionStore for FakeStore {
http_version: finalization.http_version,
websocket_pool: finalization.websocket_pool,
service_tier: finalization.service_tier,
upstream_response_model: finalization.upstream_response_model,
upstream_request_id: finalization.upstream_request_id,
upstream_status_code: finalization.upstream_status_code,
transport_decision_wait_ms: finalization.timings.transport_decision_wait_ms,
@@ -1764,7 +1762,6 @@ fn response_observation_is_persisted_but_never_delivered() {
.with_websocket_pool(WebSocketPoolKind::New)
.with_status_code(200)
.with_request_id(OpaqueUpstreamValue::new("upstream-observed"))
.with_upstream_response_model_if_valid("gpt-returned")
.try_with_service_tier("priority")
.expect("service tier")
.with_timings(ProviderResponseTimings {
@@ -1804,10 +1801,6 @@ fn response_observation_is_persisted_but_never_delivered() {
assert_eq!(finalization.http_version.as_deref(), Some("HTTP/2"));
assert_eq!(finalization.websocket_pool.as_deref(), Some("new"));
assert_eq!(finalization.service_tier.as_deref(), Some("priority"));
assert_eq!(
finalization.upstream_response_model.as_deref(),
Some("gpt-returned")
);
assert_eq!(
finalization.upstream_request_id.as_deref(),
Some("upstream-observed")
@@ -2083,7 +2076,6 @@ fn discarded_attempt_observation_does_not_leak_into_retry_result() {
.with_http_version(UpstreamHttpVersion::Http11)
.with_status_code(503)
.with_request_id(OpaqueUpstreamValue::new("discarded-request"))
.with_upstream_response_model_if_valid("discarded-model")
.with_timings(ProviderResponseTimings {
transport_decision_wait_ms: Some(7),
connect_ms: Some(11),
@@ -2146,7 +2138,6 @@ fn discarded_attempt_observation_does_not_leak_into_retry_result() {
let state = store.state.lock().expect("store lock");
let finalization = &state.finalizations[0];
assert_eq!(finalization.upstream_response_model, None);
let trace: Value =
serde_json::from_str(finalization.diagnostic_trace_json.as_deref().unwrap()).unwrap();
let events = trace["events"].as_array().unwrap();
@@ -1,4 +1,6 @@
//! 执行服务的准入、预算、请求记录与终结行为测试
mod outbound;
use gateway_core::routing::extensions::{ExtensionSetId, ExtensionSetLease, ExtensionSetReference};
use std::{
@@ -2280,7 +2282,6 @@ impl Provider for ObservationProvider {
gateway_core::event::ProviderResponseObservation::new(
UpstreamTransport::new("http_sse").unwrap(),
)
.with_upstream_response_model_if_valid("gpt-reported")
.with_service_tier_if_valid("priority"),
);
Ok(ProviderStream::new(
@@ -2613,7 +2614,6 @@ fn final_observation_is_emitted_once_for_success_rejection_and_detached_cancella
.map(UpstreamModelId::as_str),
Some("gpt-start")
);
assert_eq!(observations[0].response_model(), Some("gpt-reported"));
assert_eq!(observations[0].service_tier(), Some("priority"));
assert_eq!(
observations[0].outcome(),
@@ -0,0 +1,223 @@
//! 出站插件的外部副作用在错误改写后仍约束 Engine 重放
use super::*;
use gateway_core::engine::extensions::{ExecutionExtensionIndex, ExecutionExtensionPlans};
use gateway_core::engine::middleware::{
MiddlewareContext, MiddlewareError, MiddlewareMount, MiddlewareNext, MiddlewarePlan,
MiddlewareRequest, MiddlewareResponse, outbound,
};
#[derive(Debug)]
struct EffectfulOutbound {
catch_error: bool,
caught: Arc<AtomicUsize>,
}
impl MiddlewarePlan for EffectfulOutbound {
fn has_outbound(&self, _context: &MiddlewareContext) -> bool {
true
}
fn handle_outbound(
&self,
context: MiddlewareContext,
_: outbound::Request,
_: outbound::Next,
) -> BoxFuture<'static, Result<outbound::Request, MiddlewareError>> {
Box::pin(async move {
// 对应 Runtime 确认受管 HTTP 已发送后的唯一记录入口
context.execution_effects().unwrap().observe();
Err(MiddlewareError::Fault)
})
}
fn handle(
&self,
context: MiddlewareContext,
request: MiddlewareRequest,
next: MiddlewareNext,
) -> BoxFuture<'static, Result<MiddlewareResponse, MiddlewareError>> {
let catch_error = self.catch_error && context.mount() == MiddlewareMount::Attempt;
if !catch_error {
return next.run(request);
}
let caught = Arc::clone(&self.caught);
Box::pin(async move {
let response = next.run(request).await?;
let (_, _, _, mut body, _) = response.into_parts();
body.next_frame()
.await
.expect_err("outbound preparation fails before a frame");
body.close().await;
caught.fetch_add(1, Ordering::SeqCst);
Err(MiddlewareError::Provider(ProviderError::new(
ProviderErrorKind::Transport,
UpstreamSendState::NotSent,
)))
})
}
}
struct PreparingProvider {
calls: AtomicUsize,
}
#[async_trait]
impl Provider for PreparingProvider {
fn name(&self) -> &'static str {
"openai"
}
fn catalog_generation(&self) -> ProviderCatalogGeneration {
ProviderCatalogGeneration::default()
}
async fn query_model_capabilities(
&self,
) -> Result<Vec<ProviderModelCapabilities>, ProviderError> {
Ok(vec![])
}
async fn execute(
self: Arc<Self>,
request: ProviderRequest,
context: AttemptContext,
) -> Result<ProviderStream, ProviderError> {
self.calls.fetch_add(1, Ordering::SeqCst);
let provider = request.candidate().provider().clone();
let model = request
.candidate()
.upstream_model()
.map(|model| model.as_str().to_owned());
let account = ProviderAccountId::new("acct_openai").unwrap();
let pipeline = context
.outbound_middleware(provider.clone(), model.clone(), account.clone())
.unwrap();
let metadata = ProviderCallMetadata::new(
provider.clone(),
request.candidate().upstream_model().unwrap().clone(),
account.clone(),
UpstreamTransport::new("http").unwrap(),
);
context
.execute_middleware(
request.operation().clone(),
provider,
model,
account,
Box::new(move |_, _| {
Box::pin(async move {
let events = futures::stream::once(async move {
Err::<ProviderEvent, _>(
pipeline
.prepare(outbound::Request {
target: outbound::Target::Http {
method: http::Method::POST,
uri: "https://example.test/".parse().unwrap(),
},
headers: Vec::new(),
body: bytes::Bytes::from_static(b"request"),
})
.await
.unwrap_err(),
)
});
Ok(ProviderStream::new(metadata, events, ()))
})
}),
)
.await
}
}
struct Fixture {
service: DefaultExecutionService,
provider: Arc<PreparingProvider>,
observer: Arc<RecordingRequestObserver>,
caught: Arc<AtomicUsize>,
_extensions: Arc<ExecutionExtensionPlans>,
}
fn fixture(catch_error: bool) -> Fixture {
let provider = Arc::new(PreparingProvider {
calls: AtomicUsize::new(0),
});
let generation = super::super::extensions::reference("outbound-effect");
let extension_index = ExecutionExtensionIndex::default();
let caught = Arc::new(AtomicUsize::new(0));
let extension_owner = extension_index
.register(
generation.id().clone(),
Arc::new(ExecutionExtensionPlans {
middleware: Some(Arc::new(EffectfulOutbound {
catch_error,
caught: Arc::clone(&caught),
})),
upstream_adapters: None,
}),
)
.unwrap();
let observer_index = RequestObserverExtensionIndex::default();
let observer = Arc::new(RecordingRequestObserver::default());
let _observer_owner = observer_index
.register(generation.id().clone(), observer.clone())
.unwrap();
let providers = ProviderRegistry::new([provider.clone() as Arc<dyn Provider>]).unwrap();
let service = DefaultExecutionService::new(
RuntimeSnapshotHandle::new(fallback_observation_snapshot(generation)),
Arc::new(TrackingExecutionStore::default()),
providers,
Arc::new(Admissions::default()),
Arc::new(UnusedContinuation),
Arc::new(RecordingClientApiKeyUsage::default()),
Arc::new(crate::RecordingDiagnostics::default()),
)
.with_execution_extensions(extension_index)
.with_request_observers(observer_index);
Fixture {
service,
provider,
observer,
caught,
_extensions: extension_owner,
}
}
#[test]
fn outbound_external_effect_is_preserved_when_preparation_fails() {
block_on(async {
let fixture = fixture(false);
let mut started = fixture
.service
.start(request(&fixture.service, ClientTransport::HttpJson))
.await
.unwrap();
assert!(
matches!(started.session.collect_uncommitted().await, Err(EngineError::Provider(error)) if error.kind() == ProviderErrorKind::Protocol)
);
started.session.detach_finalize().await;
assert_eq!(fixture.provider.calls.load(Ordering::SeqCst), 1);
let observations = fixture.observer.observations.lock().unwrap();
assert_eq!(observations[0].send_state(), UpstreamSendState::Ambiguous);
assert_eq!(observations[0].attempt_count(), 1);
});
}
#[test]
fn attempt_cannot_hide_outbound_external_effect_with_retryable_not_sent_error() {
block_on(async {
let fixture = fixture(true);
let mut started = fixture
.service
.start(request(&fixture.service, ClientTransport::HttpJson))
.await
.unwrap();
assert!(
matches!(started.session.collect_uncommitted().await, Err(EngineError::Provider(error)) if error.kind() == ProviderErrorKind::Transport && !error.retry_is_prohibited())
);
started.session.detach_finalize().await;
assert_eq!(fixture.caught.load(Ordering::SeqCst), 1);
assert_eq!(fixture.provider.calls.load(Ordering::SeqCst), 1);
let observations = fixture.observer.observations.lock().unwrap();
assert_eq!(observations[0].send_state(), UpstreamSendState::Ambiguous);
assert_eq!(observations[0].attempt_count(), 1);
});
}
@@ -3,7 +3,7 @@
use std::sync::Arc;
use gateway_core::engine::extensions::{ExecutionExtensionIndex, ExecutionExtensionPlans};
use gateway_core::engine::upstream_adapter::{UpstreamAdapter, UpstreamAdapterPlan};
use gateway_core::engine::upstream_adapter::{UpstreamAdapterPlan, UpstreamAdapterSelection};
use gateway_core::routing::extensions::{ExtensionSetId, ExtensionSetLease, ExtensionSetReference};
use gateway_core::{
engine::AttemptContext, error::ProviderError, identity::ProviderKind, routing::UpstreamModelId,
@@ -31,8 +31,9 @@ impl UpstreamAdapterPlan for EmptyAdapterPlan {
_: &AttemptContext,
_: &ProviderKind,
_: &UpstreamModelId,
) -> Result<Option<Arc<dyn UpstreamAdapter>>, ProviderError> {
Ok(None)
_: Option<&gateway_core::account::ProviderAccountId>,
) -> Result<UpstreamAdapterSelection, ProviderError> {
Ok(UpstreamAdapterSelection::Native)
}
}
@@ -89,3 +90,89 @@ fn execution_index_does_not_own_registered_generations() {
assert!(index.middleware(&reference).is_none());
assert!(index.upstream_adapters(&reference).is_none());
}
#[derive(Debug, Default)]
struct AccountSelectionProbe(std::sync::Mutex<Vec<Option<String>>>);
impl UpstreamAdapterPlan for AccountSelectionProbe {
fn select(
&self,
_: &AttemptContext,
_: &ProviderKind,
_: &UpstreamModelId,
account: Option<&gateway_core::account::ProviderAccountId>,
) -> Result<UpstreamAdapterSelection, ProviderError> {
self.0
.lock()
.unwrap()
.push(account.map(|account| account.as_str().into()));
Ok(UpstreamAdapterSelection::Native)
}
}
#[test]
fn adapter_preselection_follows_only_the_current_attempt_account_owner() {
use gateway_core::{
account::{AccountSelectionPolicy, ProviderAccountId, RotationStrategy},
engine::continuation::{ContinuationBinding, NativeContinuationPin, PreviousResponseId},
engine::upstream_adapter::FrozenUpstreamAdapterPlan,
engine::{
AccountAttemptContext, ContinuationAttempt, ModelRequestId, RequestAttemptContext,
},
lifecycle::CancellationToken,
policy::ClientApiKeyId,
};
use std::{
num::NonZeroU32,
time::{Duration, SystemTime},
};
let provider = ProviderKind::new("openai").unwrap();
let model = UpstreamModelId::new("model").unwrap();
let key = ClientApiKeyId::new("key").unwrap();
let pin = NativeContinuationPin::new(
PreviousResponseId::new("public"),
PreviousResponseId::new("upstream"),
key.clone(),
provider.clone(),
ProviderAccountId::new("acct_pinned").unwrap(),
);
for (attempt, required, expected) in [
(ContinuationAttempt::Native, None, Some("acct_pinned")),
(ContinuationAttempt::ReplayAny, None, None),
(ContinuationAttempt::None, None, None),
(
ContinuationAttempt::Native,
Some("acct_required"),
Some("acct_required"),
),
] {
let probe = Arc::new(AccountSelectionProbe::default());
let context = AttemptContext::new(
RequestAttemptContext::new(
ModelRequestId::new("req_account_plan").unwrap(),
key.clone(),
)
.with_upstream_adapters(Some(FrozenUpstreamAdapterPlan::new(
probe.clone(),
reference("account-plan"),
))),
NonZeroU32::MIN,
SystemTime::now() + Duration::from_secs(5),
AccountSelectionPolicy::new(RotationStrategy::RoundRobin, NonZeroU32::MIN),
AccountAttemptContext::new(
Default::default(),
required.map(|id| ProviderAccountId::new(id).unwrap()),
None,
),
Some(ContinuationBinding::Pinned(pin.clone())),
CancellationToken::new(),
)
.with_continuation_attempt(attempt);
assert!(matches!(
context.upstream_adapter(&provider, &model).unwrap(),
UpstreamAdapterSelection::Native
));
assert_eq!(*probe.0.lock().unwrap(), vec![expected.map(str::to_owned)]);
}
}
@@ -1,6 +1,7 @@
//! 请求中间件的改写、交付状态与发布代次持有测试
mod cold;
mod outbound;
use std::num::NonZeroU32;
use std::sync::Arc;
@@ -0,0 +1,303 @@
//! 出站准备链的返回顺序、预算、生命周期与不可修改归属
use super::*;
use gateway_core::account::{AccountSelectionPolicy, RotationStrategy};
use gateway_core::engine::middleware::outbound::{Pipeline, Request, Target};
use gateway_core::engine::{AccountAttemptContext, AttemptContext, RequestAttemptContext};
use gateway_core::error::ProviderErrorKind;
use gateway_core::upstream::UpstreamSendState;
#[derive(Debug, Clone, Copy)]
enum Action {
Forward,
Rewrite,
Oversize,
SwitchTransport,
Reject,
Pending,
Cancel,
}
#[derive(Debug)]
struct PreparingPlan {
action: Action,
calls: Arc<AtomicUsize>,
}
impl MiddlewarePlan for PreparingPlan {
fn has_outbound(&self, context: &MiddlewareContext) -> bool {
context.model() == Some("gpt-test")
}
fn handle_outbound(
&self,
context: MiddlewareContext,
request: Request,
next: gateway_core::engine::middleware::outbound::Next,
) -> BoxFuture<'static, Result<Request, MiddlewareError>> {
assert_eq!(context.mount(), MiddlewareMount::Outbound);
assert_eq!(context.provider().unwrap().as_str(), "openai");
assert_eq!(context.account_id().unwrap().as_str(), "acct_middleware");
assert_eq!(context.model(), Some("gpt-test"));
assert_eq!(context.attempt_index(), NonZeroU32::new(2));
assert_eq!(context.request_id().as_str(), "req_outbound");
assert_eq!(context.client_key_id().as_str(), "key_outbound");
assert!(context.plan().is_some());
self.calls.fetch_add(1, Ordering::SeqCst);
let action = self.action;
Box::pin(async move {
let mut request = next.run(request).await?;
match action {
Action::Forward => (),
Action::Rewrite => {
let mut body = request.body.to_vec();
body.extend_from_slice(b"-after-next");
request.body = body.into();
request
.headers
.push(MiddlewareHeader::new("x-raw", Bytes::from_static(&[0x80])));
}
Action::Oversize => request.body = Bytes::from_static(b"too large"),
Action::SwitchTransport => {
request.target = Target::WebSocket {
uri: "wss://example.test/".parse().unwrap(),
}
}
Action::Reject => return Err(MiddlewareError::Rejected),
Action::Pending => return futures::future::pending().await,
Action::Cancel => {
context.cancellation().cancel();
return futures::future::pending().await;
}
}
Ok(request)
})
}
fn handle(
&self,
_: MiddlewareContext,
request: MiddlewareRequest,
next: MiddlewareNext,
) -> BoxFuture<'static, Result<MiddlewareResponse, MiddlewareError>> {
next.run(request)
}
}
fn pipeline(
action: Action,
limit: usize,
cancellation: CancellationToken,
deadline: SystemTime,
) -> (Pipeline, Arc<AtomicUsize>) {
let (attempt, calls) = attempt(action, limit, cancellation, deadline);
(
attempt
.outbound_middleware(
ProviderKind::new("openai").unwrap(),
Some("gpt-test".into()),
ProviderAccountId::new("acct_middleware").unwrap(),
)
.unwrap(),
calls,
)
}
fn attempt(
action: Action,
limit: usize,
cancellation: CancellationToken,
deadline: SystemTime,
) -> (AttemptContext, Arc<AtomicUsize>) {
let calls = Arc::new(AtomicUsize::new(0));
let plan = FrozenMiddlewarePlan::new(
Arc::new(PreparingPlan {
action,
calls: Arc::clone(&calls),
}),
ExtensionSetReference::new(
ExtensionSetId::new("outbound-test".into()).unwrap(),
Arc::new(ReadyGeneration),
),
);
let request = RequestAttemptContext::new(
ModelRequestId::new("req_outbound").unwrap(),
ClientApiKeyId::new("key_outbound").unwrap(),
)
.with_request_body_limit(limit)
.with_middleware(
Some(plan),
Arc::from([]),
"/v1/responses".into(),
ClientTransport::HttpJson,
);
let attempt = AttemptContext::new(
request,
NonZeroU32::new(2).unwrap(),
deadline,
AccountSelectionPolicy::new(RotationStrategy::Smart, NonZeroU32::new(1).unwrap()),
AccountAttemptContext::new(Default::default(), None, None),
None,
cancellation,
);
(attempt, calls)
}
fn request() -> Request {
Request {
target: Target::Http {
method: http::Method::POST,
uri: "https://example.test/path?secret=private".parse().unwrap(),
},
headers: vec![MiddlewareHeader::new("x-raw", Bytes::from_static(b"first"))],
body: Bytes::from_static(&[0, 255, 0, 128]),
}
}
#[test]
fn outbound_returns_outer_rewrite_after_next_without_decoding_body() {
let (pipeline, calls) = pipeline(
Action::Rewrite,
128,
CancellationToken::new(),
SystemTime::now() + Duration::from_secs(1),
);
let prepared = futures::executor::block_on(pipeline.prepare(request())).unwrap();
assert_eq!(
prepared.body.as_ref(),
&[
0, 255, 0, 128, b'-', b'a', b'f', b't', b'e', b'r', b'-', b'n', b'e', b'x', b't'
]
);
let headers = prepared.header_map().unwrap();
assert_eq!(
headers
.get_all("x-raw")
.iter()
.map(http::HeaderValue::as_bytes)
.collect::<Vec<_>>(),
vec![b"first".as_slice(), &[0x80]]
);
assert_eq!(calls.load(Ordering::SeqCst), 1);
}
#[test]
fn outbound_checks_final_body_limit_without_repair_or_retry() {
let (pipeline, _) = pipeline(
Action::Oversize,
4,
CancellationToken::new(),
SystemTime::now() + Duration::from_secs(1),
);
let error = futures::executor::block_on(pipeline.prepare(request())).unwrap_err();
assert_eq!(error.kind(), ProviderErrorKind::InvalidRequest);
assert_eq!(error.send_state(), UpstreamSendState::NotSent);
assert!(error.retry_is_prohibited());
}
#[test]
fn outbound_cannot_switch_transport_kind() {
let (pipeline, _) = pipeline(
Action::SwitchTransport,
128,
CancellationToken::new(),
SystemTime::now() + Duration::from_secs(1),
);
let error = futures::executor::block_on(pipeline.prepare(request())).unwrap_err();
assert_eq!(error.kind(), ProviderErrorKind::InvalidRequest);
assert!(error.retry_is_prohibited());
}
#[test]
fn outbound_rejection_is_terminal_and_not_sent() {
let (pipeline, _) = pipeline(
Action::Reject,
128,
CancellationToken::new(),
SystemTime::now() + Duration::from_secs(1),
);
let error = futures::executor::block_on(pipeline.prepare(request())).unwrap_err();
assert_eq!(error.kind(), ProviderErrorKind::RequestPolicyDenied);
assert_eq!(error.send_state(), UpstreamSendState::NotSent);
assert!(error.retry_is_prohibited());
}
#[test]
fn outbound_cancelled_attempt_never_enters_middleware() {
let cancelled = CancellationToken::new();
cancelled.cancel();
let (pipeline, calls) = pipeline(
Action::Forward,
128,
cancelled,
SystemTime::now() + Duration::from_secs(1),
);
let error = futures::executor::block_on(pipeline.prepare(request())).unwrap_err();
assert_eq!(error.kind(), ProviderErrorKind::Cancelled);
assert_eq!(calls.load(Ordering::SeqCst), 0);
}
#[test]
fn outbound_deadline_drops_a_pending_preparation() {
let (pipeline, calls) = pipeline(
Action::Pending,
128,
CancellationToken::new(),
SystemTime::now() + Duration::from_secs(1),
);
let error = futures::executor::block_on(pipeline.prepare(request())).unwrap_err();
assert_eq!(error.kind(), ProviderErrorKind::Timeout);
assert_eq!(calls.load(Ordering::SeqCst), 1);
assert!(error.retry_is_prohibited());
}
#[test]
fn outbound_header_conversion_rejects_invalid_bytes_without_retry() {
let mut request = request();
request.headers.push(MiddlewareHeader::new(
"x-invalid",
Bytes::from_static(b"bad\r\nvalue"),
));
let error = request.header_map().unwrap_err();
assert_eq!(error.send_state(), UpstreamSendState::NotSent);
assert!(error.retry_is_prohibited());
}
#[test]
fn outbound_debug_does_not_expose_uri_headers_or_body() {
let debug = format!("{:?}", request());
assert!(!debug.contains("private"));
assert!(!debug.contains("first"));
assert!(!debug.contains("255"));
}
#[test]
fn outbound_cancellation_stops_an_in_flight_preparation() {
let (pipeline, calls) = pipeline(
Action::Cancel,
128,
CancellationToken::new(),
SystemTime::now() + Duration::from_secs(1),
);
let error = futures::executor::block_on(pipeline.prepare(request())).unwrap_err();
assert_eq!(error.kind(), ProviderErrorKind::Cancelled);
assert_eq!(calls.load(Ordering::SeqCst), 1);
assert!(error.retry_is_prohibited());
}
#[test]
fn outbound_scope_miss_keeps_the_provider_on_its_native_path() {
let (attempt, calls) = attempt(
Action::Reject,
0,
CancellationToken::new(),
SystemTime::now() + Duration::from_secs(1),
);
let pipeline = attempt.outbound_middleware(
ProviderKind::new("openai").unwrap(),
Some("unmatched".into()),
ProviderAccountId::new("acct_middleware").unwrap(),
);
assert!(pipeline.is_none());
assert_eq!(calls.load(Ordering::SeqCst), 0);
}
@@ -1,15 +1,15 @@
{
"schema_version": 2,
"manifest_schema_versions": [2],
"protocol_versions": [5],
"protocol_versions": [6],
"capabilities": [
{ "capability": "frontend_authentication", "versions": [2] },
{ "capability": "scheduler", "versions": [3] },
{ "capability": "model_router", "versions": [2] },
{ "capability": "model_catalog", "versions": [1] },
{ "capability": "retry_policy", "versions": [2] },
{ "capability": "middleware", "versions": [5] },
{ "capability": "upstream_adapter", "versions": [4] },
{ "capability": "middleware", "versions": [6] },
{ "capability": "upstream_adapter", "versions": [5] },
{ "capability": "observer", "versions": [2] },
{ "capability": "command_line", "versions": [1] },
{ "capability": "management", "versions": [2] },
@@ -1,7 +1,5 @@
//! 编译插件模型目录贡献并校验能力绑定
use std::time::Duration;
use gateway_admin::model::{
AdminError,
plugins::instances::{PluginCapabilityBinding, PluginInstance},
@@ -23,7 +21,7 @@ pub(crate) async fn prepare(
let reply = session
.call(
"model_catalog.register",
session.context(Stage::Registration, Duration::from_secs(5)),
session.context(Stage::Registration, session.budget(Stage::Registration)),
serde_json::json!({}),
Vec::new(),
)
@@ -48,12 +48,13 @@ impl PluginCommandSession {
let payload = serde_json::to_vec(&invocation)
.map_err(|_| AdminError::invalid("插件命令参数无法编码"))?;
let mut saved_accounts = 0;
let deadline = tokio::time::Instant::now() + command.session.budget(Stage::CommandLine);
let result = tokio::select! {
biased;
() = cancellation.cancelled() => Ok(Err("调用已取消")),
result = tokio::time::timeout(
self.limits.maximum_call_timeout,
command.execute(payload, self.limits, &mut saved_accounts),
result = tokio::time::timeout_at(
deadline,
command.execute(payload, self.limits, deadline, &mut saved_accounts),
) => result,
};
match result {
@@ -76,11 +77,14 @@ impl PluginCommand {
&self,
payload: Vec<u8>,
limits: RpcLimits,
deadline: tokio::time::Instant,
saved_accounts: &mut usize,
) -> Result<PluginCommandOutput, &'static str> {
let mut context = self
.session
.context(Stage::CommandLine, limits.maximum_call_timeout);
let remaining = deadline.saturating_duration_since(tokio::time::Instant::now());
if remaining.is_zero() {
return Err("超过命令总期限");
}
let mut context = self.session.context(Stage::CommandLine, remaining);
context.request_id = Some(uuid::Uuid::new_v4().to_string());
let scope = self
.callbacks
@@ -135,6 +139,7 @@ fn rpc_failure(error: RpcError) -> &'static str {
}
RpcError::Capacity | RpcError::Overloaded => "插件调用容量不足",
RpcError::Context(_) => "调用上下文无效",
RpcError::Budget(_) => "调用预算冲突",
RpcError::Remote(_) => "插件拒绝命令",
}
}
@@ -4,7 +4,7 @@ use super::PluginCommand;
use crate::{RpcSession, callback::PluginCallbacks};
use gateway_admin::model::{AdminError, plugins::instances::PluginInstance};
use gateway_plugin_sdk::{Capability, Manifest, Stage, call::management::CommandRegistration};
use std::{sync::Arc, time::Duration};
use std::sync::Arc;
const MAXIMUM_REGISTRATION_BYTES: usize = 64 * 1024;
@@ -23,7 +23,7 @@ pub(crate) async fn prepare(
let reply = session
.call(
"command_line.register",
session.context(Stage::Registration, Duration::from_secs(5)),
session.context(Stage::Registration, session.budget(Stage::Registration)),
serde_json::json!({}),
Vec::new(),
)
@@ -1,6 +1,6 @@
//! 编译插件客户端认证绑定,并将认证结果映射为宿主身份
use std::{collections::BTreeMap, sync::Arc, time::Duration};
use std::{collections::BTreeMap, sync::Arc};
use futures::future::BoxFuture;
use gateway_admin::model::{
@@ -26,8 +26,6 @@ use gateway_plugin_sdk::{
use crate::{RpcSession, callback::PluginCallbacks};
const IDENTIFIER_TIMEOUT: Duration = Duration::from_secs(5);
#[derive(Clone)]
pub(crate) struct FrontendAuthenticationEntry {
identities: BTreeMap<String, ClientApiKeyId>,
@@ -47,7 +45,7 @@ pub(crate) async fn prepare_entry(
let reply = session
.call(
"frontend_auth.identifier",
session.context(Stage::Registration, IDENTIFIER_TIMEOUT),
session.context(Stage::Registration, session.budget(Stage::Registration)),
serde_json::json!({}),
Vec::new(),
)
@@ -167,19 +165,16 @@ fn valid_identifier(value: &str) -> bool {
pub(crate) struct PluginFrontendAuthenticationPlan {
entry: FrontendAuthenticationEntry,
timeout: Duration,
}
impl PluginFrontendAuthenticationPlan {
pub(crate) fn compile(
mut entries: Vec<FrontendAuthenticationEntry>,
maximum_call_timeout: Duration,
) -> Result<Option<Arc<Self>>, AdminError> {
match entries.len() {
0 => Ok(None),
1 => Ok(Some(Arc::new(Self {
entry: entries.pop().expect("one entry was checked"),
timeout: maximum_call_timeout.min(Duration::from_secs(30)),
}))),
_ => Err(AdminError::invalid("同一发布集合只能启用一个入口认证插件")),
}
@@ -197,7 +192,8 @@ impl FrontendAuthenticationPlan for PluginFrontendAuthenticationPlan {
.invocation
.as_ref()
.ok_or(FrontendAuthenticationError)?;
let mut context = session.context(Stage::Authentication, self.timeout);
let mut context =
session.context(Stage::Authentication, session.budget(Stage::Authentication));
context.request = crate::adapter::request_context(request.request_context());
let _scope = callbacks
.prepare_frontend_authentication(&context, request.shared_request_context())
@@ -165,9 +165,10 @@ impl ManagementEntry {
{
return Err(invalid_state());
}
let mut context = self
.session
.context(Stage::PublicManagement, limits.maximum_call_timeout);
let mut context = self.session.context(
Stage::PublicManagement,
self.session.budget(Stage::PublicManagement),
);
context.request = crate::adapter::request_context(request.request_context.as_deref());
context.request_id = Some(request.request_id);
let _scope = self
@@ -98,7 +98,7 @@ impl ManagementEntry {
.map_err(|_| AdminError::invalid("插件管理请求无法编码"))?;
let mut context = self
.session
.context(Stage::Management, limits.maximum_call_timeout);
.context(Stage::Management, self.session.budget(Stage::Management));
context.request = crate::adapter::request_context(request.request_context.as_deref());
context.request_id = Some(request.request_id);
let _scope = self
@@ -1,6 +1,6 @@
//! 将插件管理声明编译为页面、路由与资源入口
use std::{collections::BTreeMap, sync::Arc, time::Duration};
use std::{collections::BTreeMap, sync::Arc};
use gateway_admin::model::{
AdminError,
@@ -34,7 +34,7 @@ pub(crate) async fn prepare(
let reply = session
.call(
"management.register",
session.context(Stage::Registration, Duration::from_secs(5)),
session.context(Stage::Registration, session.budget(Stage::Registration)),
serde_json::json!({}),
Vec::new(),
)
@@ -7,7 +7,7 @@ use gateway_core::engine::middleware::{
MiddlewareRequest, MiddlewareResponse,
};
use gateway_core::middleware::{Middleware, Next};
use std::{sync::Arc, time::Duration};
use std::sync::Arc;
// 协议适配只提供选择条件与单层调用;顺序、上下文捕获和续体拼接共用这一个入口
type Invoke<Context, Input, Output, Error> = fn(
@@ -15,7 +15,6 @@ type Invoke<Context, Input, Output, Error> = fn(
Context,
Input,
Next<Input, Output, Error>,
Duration,
) -> BoxFuture<'static, Result<Output, Error>>;
impl PluginMiddlewarePlan {
@@ -42,8 +41,7 @@ impl PluginMiddlewarePlan {
.map(|entry| -> Middleware<Input, Output, Error> {
let entry = Arc::clone(entry);
let context = context.clone();
let timeout = self.middleware_timeout;
Box::new(move |input, next| invoke(entry, context, input, next, timeout))
Box::new(move |input, next| invoke(entry, context, input, next))
})
.collect();
gateway_core::middleware::compose(layers, move |input| next.run(input))
@@ -51,6 +49,33 @@ impl PluginMiddlewarePlan {
}
impl MiddlewarePlan for PluginMiddlewarePlan {
fn has_outbound(&self, context: &MiddlewareContext) -> bool {
self.middleware
.get(&MiddlewareMount::Outbound)
.is_some_and(|entries| entries.iter().any(|entry| outbound_matches(entry, context)))
}
fn handle_outbound(
&self,
context: MiddlewareContext,
request: gateway_core::engine::middleware::outbound::Request,
next: gateway_core::engine::middleware::outbound::Next,
) -> BoxFuture<
'static,
Result<gateway_core::engine::middleware::outbound::Request, MiddlewareError>,
> {
self.chain(
MiddlewareMount::Outbound,
context,
outbound_matches,
|entry, context, request, next| {
Box::pin(super::outbound::invoke(entry, context, request, next))
},
next,
)
.run(request)
}
fn has_service(&self) -> bool {
self.middleware.contains_key(&MiddlewareMount::Service)
}
@@ -70,8 +95,8 @@ impl MiddlewarePlan for PluginMiddlewarePlan {
MiddlewareMount::Service,
context,
|entry, context| !context.extensions.contains(&entry.instance_id),
|entry, context, input, next, timeout| {
Box::pin(super::service::invoke(entry, context, input, next, timeout))
|entry, context, input, next| {
Box::pin(super::service::invoke(entry, context, input, next))
},
next,
)
@@ -94,10 +119,8 @@ impl MiddlewarePlan for PluginMiddlewarePlan {
MiddlewareMount::WebSocket,
context,
|_entry, _context| true,
|entry, context, message, next, timeout| {
Box::pin(super::websocket::invoke(
entry, context, message, next, timeout,
))
|entry, context, message, next| {
Box::pin(super::websocket::invoke(entry, context, message, next))
},
next,
)
@@ -118,8 +141,8 @@ impl MiddlewarePlan for PluginMiddlewarePlan {
MiddlewareMount::Http,
context,
|entry, context| !context.extensions.contains(&entry.instance_id),
|entry, context, request, next, timeout| {
Box::pin(super::http::invoke(entry, context, request, next, timeout))
|entry, context, request, next| {
Box::pin(super::http::invoke(entry, context, request, next))
},
next,
)
@@ -144,13 +167,22 @@ impl MiddlewarePlan for PluginMiddlewarePlan {
context.model(),
)
},
|entry, context, request, next, timeout| {
Box::pin(super::request::invoke(
entry, context, request, next, timeout,
))
|entry, context, request, next| {
Box::pin(super::request::invoke(entry, context, request, next))
},
next,
)
.run(request)
}
}
// 存在性检查与实际组合使用同一选择规则,未命中时不进入原生报文准备
fn outbound_matches(entry: &MiddlewareEntry, context: &MiddlewareContext) -> bool {
!context.extension_scope().contains(&entry.instance_id)
&& entry.scope.matches_processing(
context.client_key_id(),
context.account_group_ids(),
context.provider(),
context.model(),
)
}
@@ -20,7 +20,6 @@ pub(super) async fn invoke(
context: core::Context,
request: core::Request,
next: core::Next,
timeout: std::time::Duration,
) -> Result<core::Response, MiddlewareError> {
let Some(ports) = &entry.invocation else {
return if entry.failure_policy == PluginFailurePolicy::Delegate {
@@ -35,7 +34,9 @@ pub(super) async fn invoke(
.and_then(|settings| settings.runtime.as_ref())
.map_or(serde_json::Value::Null, |settings| settings.inspect());
let (invocation, request) = Invocation::new(request, next, entry.instance_id.clone())?;
let mut call = ports.session.context(Stage::Http, timeout);
let mut call = ports
.session
.context(Stage::Http, ports.session.budget(Stage::Http));
call.request = crate::adapter::request_context(context.extensions.request());
call.request_id = Some(context.request_id.clone());
call.resource_stream = true;
@@ -2,6 +2,7 @@
mod execution;
mod http;
mod outbound;
mod request;
mod service;
mod websocket;
@@ -17,7 +18,6 @@ use std::{
collections::{BTreeMap, BTreeSet},
fmt,
sync::Arc,
time::Duration,
};
#[derive(Clone)]
@@ -56,11 +56,16 @@ pub(crate) fn validate_bindings(
}
if !matches!(
stage,
Stage::Http | Stage::WebSocket | Stage::Service | Stage::Request | Stage::Attempt
Stage::Http
| Stage::WebSocket
| Stage::Service
| Stage::Request
| Stage::Attempt
| Stage::Outbound
) || !stages.insert(stage)
{
return Err(AdminError::invalid(
"同一插件实例的中间件只能在 http/websocket/service/request/attempt 各绑定一次",
"同一插件实例的中间件只能在 http/websocket/service/request/attempt/outbound 各绑定一次",
));
}
if matches!(stage, Stage::Http | Stage::WebSocket | Stage::Service)
@@ -117,7 +122,7 @@ pub(crate) fn unavailable_entries(
.filter(|binding| {
matches!(
binding.stage.as_str(),
"http" | "websocket" | "service" | "request" | "attempt"
"http" | "websocket" | "service" | "request" | "attempt" | "outbound"
)
})
.map(|binding| {
@@ -151,6 +156,7 @@ fn compile_entry(
"websocket" => MiddlewareMount::WebSocket,
"request" => MiddlewareMount::Request,
"attempt" => MiddlewareMount::Attempt,
"outbound" => MiddlewareMount::Outbound,
_ => return Err(AdminError::invalid("插件中间件阶段无效")),
},
invocation,
@@ -161,7 +167,6 @@ fn compile_entry(
pub(crate) struct PluginMiddlewarePlan {
middleware: BTreeMap<MiddlewareMount, Box<[Arc<MiddlewareEntry>]>>,
middleware_timeout: Duration,
}
impl fmt::Debug for PluginMiddlewarePlan {
@@ -174,10 +179,7 @@ impl fmt::Debug for PluginMiddlewarePlan {
}
impl PluginMiddlewarePlan {
pub(crate) fn compile(
mut entries: Vec<MiddlewareEntry>,
maximum_call_timeout: Duration,
) -> Option<Arc<Self>> {
pub(crate) fn compile(mut entries: Vec<MiddlewareEntry>) -> Option<Arc<Self>> {
if entries.is_empty() {
return None;
}
@@ -201,7 +203,6 @@ impl PluginMiddlewarePlan {
.into_iter()
.map(|(mount, entries)| (mount, entries.into_boxed_slice()))
.collect(),
middleware_timeout: maximum_call_timeout,
}))
}
}
@@ -0,0 +1,103 @@
//! 在 Provider 发送前组合完整出站请求,并沿用模型数据面的调用资源
use std::sync::Arc;
use gateway_admin::model::plugins::instances::PluginFailurePolicy;
use gateway_core::engine::middleware::{MiddlewareContext, MiddlewareError, outbound as core};
use gateway_plugin_sdk::{
Stage,
call::middleware::{HANDLE_METHOD, outbound as wire},
};
use super::MiddlewareEntry;
use crate::callback::outbound::{self, Invocation};
pub(super) async fn invoke(
entry: Arc<MiddlewareEntry>,
context: MiddlewareContext,
request: core::Request,
next: core::Next,
) -> Result<core::Request, MiddlewareError> {
let Some(ports) = &entry.invocation else {
return if entry.failure_policy == PluginFailurePolicy::Delegate {
next.run(request).await
} else {
Err(MiddlewareError::Fault)
};
};
let invocation = Invocation::new(next);
let input = wire::CallHead {
request: outbound::head(&request),
metadata: wire::Metadata {
request_id: context.request_id().as_str().to_owned(),
client_key_id: context.client_key_id().as_str().to_owned(),
account_group_ids: context
.account_group_ids()
.iter()
.map(|id| id.as_str().to_owned())
.collect(),
provider: context
.provider()
.ok_or(MiddlewareError::InvalidState)?
.as_str()
.to_owned(),
model: context.model().map(str::to_owned),
account_id: context
.account_id()
.ok_or(MiddlewareError::InvalidState)?
.as_str()
.to_owned(),
attempt_index: context
.attempt_index()
.ok_or(MiddlewareError::InvalidState)?
.get(),
},
};
let mut call = ports.session.context(
Stage::Outbound,
context
.deadline()
.bounded(ports.session.budget(Stage::Outbound)),
);
call.request = crate::adapter::request_context(context.extension_scope().request());
call.request_id = Some(input.metadata.request_id.clone());
call.attempt_id = Some(format!(
"{}:{}",
input.metadata.request_id, input.metadata.attempt_index
));
call.account_id = Some(input.metadata.account_id.clone());
let scope = ports
.callbacks
.prepare_data_plane(
&call,
context.execution_effects(),
context.extension_scope().clone(),
context.plan(),
context.cancellation().clone(),
context.request_body_limit(),
)
.map_err(|_| MiddlewareError::Fault)?;
let _binding =
ports
.callbacks
.bind_middleware(call.resource_scope_id.clone(), invocation, scope)?;
let reply = tokio::select! {
biased;
() = context.cancellation().cancelled() => return Err(MiddlewareError::Fault),
() = context.deadline().wait() => return Err(gateway_core::error::GatewayError::new(
gateway_core::error::GatewayErrorKind::Timeout,
"request deadline elapsed",
).into()),
reply = ports.session.call(
HANDLE_METHOD,
call,
serde_json::to_value(input).map_err(|_| MiddlewareError::InvalidState)?,
request.body.into(),
) => reply.map_err(crate::callback::error::rpc_middleware)?,
};
let head = ports
.session
.decode_response(Stage::Outbound, reply.result)
.map_err(|_| MiddlewareError::InvalidState)?;
outbound::decode(head, reply.payload).map_err(|_| MiddlewareError::InvalidState)
}
@@ -1,9 +1,6 @@
//! 请求与尝试中间件的 RPC 调用、失败回退和响应正文生命周期
use std::{
sync::{Arc, Mutex},
time::Duration,
};
use std::sync::{Arc, Mutex};
use bytes::Bytes;
use futures::future::BoxFuture;
@@ -37,7 +34,6 @@ pub(super) async fn invoke(
context: MiddlewareContext,
request: MiddlewareRequest,
next: MiddlewareNext,
maximum_timeout: Duration,
) -> Result<MiddlewareResponse, MiddlewareError> {
let Some(invocation_ports) = &entry.invocation else {
return if entry.failure_policy
@@ -48,10 +44,7 @@ pub(super) async fn invoke(
Err(MiddlewareError::Fault)
};
};
if maximum_timeout.is_zero()
|| context.deadline().is_elapsed()
|| context.cancellation().is_cancelled()
{
if context.deadline().is_elapsed() || context.cancellation().is_cancelled() {
return Err(MiddlewareError::Fault);
}
let invocation = MiddlewareInvocation::new(&context, request, next, entry.instance_id.clone());
@@ -59,15 +52,21 @@ pub(super) async fn invoke(
Ok(projection) => projection,
Err(_) => return recover_rpc(&entry, &invocation, RpcError::Context(None)).await,
};
let stage = match entry.mount {
MiddlewareMount::Http
| MiddlewareMount::WebSocket
| MiddlewareMount::Service
| MiddlewareMount::Outbound => {
return Err(MiddlewareError::InvalidState);
}
MiddlewareMount::Request => Stage::Request,
MiddlewareMount::Attempt => Stage::Attempt,
};
let mut call_context = invocation_ports.session.context(
match entry.mount {
MiddlewareMount::Http | MiddlewareMount::WebSocket | MiddlewareMount::Service => {
return Err(MiddlewareError::InvalidState);
}
MiddlewareMount::Request => Stage::Request,
MiddlewareMount::Attempt => Stage::Attempt,
},
maximum_timeout,
stage,
context
.deadline()
.bounded(invocation_ports.session.budget(stage)),
);
call_context.resource_stream = true;
call_context.request = crate::adapter::request_context(context.extension_scope().request());
@@ -107,7 +106,10 @@ pub(super) async fn invoke(
.collect(),
request_id: context.request_id().as_str().to_owned(),
mount: match context.mount() {
MiddlewareMount::Http | MiddlewareMount::WebSocket | MiddlewareMount::Service => {
MiddlewareMount::Http
| MiddlewareMount::WebSocket
| MiddlewareMount::Service
| MiddlewareMount::Outbound => {
return Err(MiddlewareError::InvalidState);
}
MiddlewareMount::Request => WireMiddlewareMount::Request,
@@ -16,7 +16,6 @@ pub(super) async fn invoke(
context: core::Context,
input: core::Value,
next: core::Next,
timeout: std::time::Duration,
) -> Result<core::Value, core::Error> {
let Some(ports) = &entry.invocation else {
return if entry.failure_policy == PluginFailurePolicy::Delegate {
@@ -26,7 +25,9 @@ pub(super) async fn invoke(
};
};
let invocation = Invocation::new(next);
let mut call = ports.session.context(Stage::Service, timeout);
let mut call = ports
.session
.context(Stage::Service, ports.session.budget(Stage::Service));
call.request = crate::adapter::request_context(context.extensions.request());
call.request_id = Some(context.request_id.clone());
let _binding = ports
@@ -16,7 +16,6 @@ pub(super) async fn invoke(
context: core::Context,
message: core::Message,
next: core::Next,
timeout: std::time::Duration,
) -> Result<Option<core::Message>, MiddlewareError> {
let Some(ports) = &entry.invocation else {
return if entry.failure_policy == PluginFailurePolicy::Delegate {
@@ -44,7 +43,9 @@ pub(super) async fn invoke(
.encode(message)
.map_err(|_| MiddlewareError::Fault)?,
};
let mut call = ports.session.context(Stage::WebSocket, timeout);
let mut call = ports
.session
.context(Stage::WebSocket, ports.session.budget(Stage::WebSocket));
call.request = crate::adapter::request_context(context.extensions.request());
call.request_id = Some(context.connection_id.clone());
let _binding = ports
@@ -3,7 +3,7 @@
use std::{
collections::BTreeMap,
sync::{Arc, Mutex},
time::{Duration, SystemTime, UNIX_EPOCH},
time::{SystemTime, UNIX_EPOCH},
};
use gateway_admin::model::{
@@ -31,7 +31,6 @@ use tokio::sync::{OwnedSemaphorePermit, Semaphore, mpsc};
use crate::{RpcSession, adapter::scope::BindingScope};
const OBSERVATION_TIMEOUT: Duration = Duration::from_secs(2);
const OBSERVATION_ENVELOPE_RESERVE: usize = 16 * 1024;
#[derive(Clone)]
@@ -57,10 +56,11 @@ impl ObserverEntry {
event: Event,
payload: Vec<u8>,
extensions: gateway_core::engine::extensions::ExtensionCallScope,
timeout: Duration,
) {
let request_id = event.request_id();
let mut context = self.session.context(Stage::Observation, timeout);
let mut context = self
.session
.context(Stage::Observation, self.session.budget(Stage::Observation));
context.request = crate::adapter::request_context(extensions.request());
context.request_id = Some(request_id.to_owned());
let Ok(_scope) = self.callbacks.prepare_extensions(&context, extensions) else {
@@ -166,7 +166,6 @@ pub(crate) struct PluginObserverPlan {
websocket_queues: Arc<Mutex<BTreeMap<String, mpsc::Sender<QueuedWebSocketObservation>>>>,
websocket_queue_capacity: usize,
maximum_websocket_payload_bytes: usize,
timeout: Duration,
}
struct QueuedWebSocketObservation {
@@ -180,7 +179,6 @@ impl PluginObserverPlan {
pub(crate) fn compile(
mut entries: Vec<ObserverEntry>,
maximum_calls: usize,
maximum_call_timeout: Duration,
maximum_buffered_body_bytes: usize,
) -> Option<Arc<Self>> {
if entries.is_empty() {
@@ -205,7 +203,6 @@ impl PluginObserverPlan {
websocket_queues: Arc::new(Mutex::new(BTreeMap::new())),
websocket_queue_capacity: maximum_calls.clamp(1, 256),
maximum_websocket_payload_bytes,
timeout: maximum_call_timeout.min(OBSERVATION_TIMEOUT),
}))
}
@@ -326,7 +323,6 @@ impl RequestObserverPlan for PluginObserverPlan {
return;
};
let entries = Arc::clone(&self.entries);
let timeout = self.timeout;
handle.spawn(async move {
let _generation = generation;
let _slot = slot;
@@ -342,7 +338,6 @@ impl RequestObserverPlan for PluginObserverPlan {
Event::RequestCompleted(Box::new(wire_observation(&observation))),
Vec::new(),
observation.extension_scope().clone(),
timeout,
)
.await;
}
@@ -372,10 +367,8 @@ impl PluginObserverPlan {
let (sender, receiver) = mpsc::channel(self.websocket_queue_capacity);
queues.insert(request_id.to_owned(), sender.clone());
let entries = Arc::clone(&self.entries);
let timeout = self.timeout;
handle.spawn(dispatch_websocket_queue(
entries,
timeout,
generation,
worker_slot,
receiver,
@@ -386,7 +379,6 @@ impl PluginObserverPlan {
async fn dispatch_websocket_queue(
entries: Arc<[ObserverEntry]>,
timeout: Duration,
generation: ExtensionSetReference,
worker_slot: OwnedSemaphorePermit,
mut receiver: mpsc::Receiver<QueuedWebSocketObservation>,
@@ -422,7 +414,6 @@ async fn dispatch_websocket_queue(
Event::WebSocketResponse(Box::new(request)),
queued.payload.to_vec(),
queued.observation.extension_scope().clone(),
timeout,
)
.await;
}
@@ -455,7 +446,6 @@ fn wire_observation(observation: &RequestObservation) -> RequestCompleted {
upstream_model: observation
.upstream_model()
.map(|model| model.as_str().to_owned()),
response_model: observation.response_model().map(str::to_owned),
service_tier: observation.service_tier().map(str::to_owned),
requested_model: observation
.requested_model()
@@ -3,7 +3,7 @@
mod retry;
mod route_schedule;
use std::{fmt, sync::Arc, time::Duration};
use std::{fmt, sync::Arc};
use gateway_admin::model::{
AdminError,
@@ -13,8 +13,6 @@ use gateway_plugin_sdk::{Capability, Manifest, Stage};
use crate::{RpcSession, adapter::scope::BindingScope, callback::PluginCallbacks};
const POLICY_TIMEOUT: Duration = Duration::from_secs(2);
#[derive(Clone)]
pub(crate) enum PolicyEntry {
Router(ModelRouterEntry),
@@ -219,7 +217,6 @@ pub(crate) struct PluginRequestPolicyPlan {
routers: Arc<[ModelRouterEntry]>,
schedulers: Arc<[AccountSchedulerEntry]>,
retries: Arc<[RetryEntry]>,
policy_timeout: Duration,
}
impl fmt::Debug for PluginRequestPolicyPlan {
@@ -234,10 +231,7 @@ impl fmt::Debug for PluginRequestPolicyPlan {
}
impl PluginRequestPolicyPlan {
pub(crate) fn compile(
entries: Vec<PolicyEntry>,
maximum_call_timeout: Duration,
) -> Result<Option<Arc<Self>>, AdminError> {
pub(crate) fn compile(entries: Vec<PolicyEntry>) -> Result<Option<Arc<Self>>, AdminError> {
let mut routers = Vec::new();
let mut schedulers = Vec::new();
let mut retries = Vec::new();
@@ -277,7 +271,6 @@ impl PluginRequestPolicyPlan {
routers: routers.into(),
schedulers: schedulers.into(),
retries: retries.into(),
policy_timeout: maximum_call_timeout.min(POLICY_TIMEOUT),
})))
}
}
@@ -17,11 +17,10 @@ use super::RetryEntry;
pub(super) fn decide(
entries: Arc<[RetryEntry]>,
input: RetryInput,
timeout: Duration,
) -> BoxFuture<'static, Result<RetryDecision, RequestPolicyFault>> {
Box::pin(async move {
let started = std::time::Instant::now();
for entry in entries.iter().filter(|entry| {
let started = tokio::time::Instant::now();
let matched = entries.iter().filter(|entry| {
!input.extension_scope.contains(&entry.instance_id)
&& entry.scope.matches_processing(
&input.client_key_id,
@@ -29,23 +28,32 @@ pub(super) fn decide(
Some(&input.facts.provider),
input.facts.model.as_deref(),
)
}) {
});
let timeout = matched
.clone()
.filter_map(|entry| entry.invocation.as_ref())
.map(|invocation| invocation.session.budget(Stage::Retry))
.max()
.unwrap_or_default()
.min(input.facts.remaining_deadline);
for entry in matched {
// 整条委托链共享预算,不能每个插件重新取得完整等待时间
let remaining = timeout
.min(input.facts.remaining_deadline)
.saturating_sub(started.elapsed());
let remaining = timeout.saturating_sub(started.elapsed());
if remaining.is_zero() {
break;
}
match call(entry, &input, remaining).await {
Ok(RetryDecision::Delegate) => {}
Ok(decision) => return Ok(decision),
Err(()) => tracing::warn!(
plugin_id = entry.plugin_id,
instance_id = entry.instance_id,
request_id = input.request_id.as_str(),
"插件重试策略无效或调用失败,委托后续策略"
),
match tokio::time::timeout(remaining, call(entry, &input, remaining)).await {
Err(_) => break,
Ok(result) => match result {
Ok(RetryDecision::Delegate) => {}
Ok(decision) => return Ok(decision),
Err(()) => tracing::warn!(
plugin_id = entry.plugin_id,
instance_id = entry.instance_id,
request_id = input.request_id.as_str(),
"插件重试策略无效或调用失败,委托后续策略"
),
},
}
}
Ok(RetryDecision::Delegate)
@@ -79,7 +87,10 @@ async fn call(
.unwrap_or(u64::MAX),
allowed_actions,
};
let mut context = invocation.session.context(Stage::Retry, timeout);
let mut context = invocation.session.context(
Stage::Retry,
timeout.min(invocation.session.budget(Stage::Retry)),
);
context.request = crate::adapter::request_context(input.extension_scope.request());
context.request_id = Some(request.request_id.clone());
context.attempt_id = Some(format!("{}:{}", request.request_id, request.attempt_index));
@@ -1,6 +1,6 @@
//! 插件模型路由与账号调度调用,以及请求事实的线协议投影
use std::time::{Duration, SystemTime, UNIX_EPOCH};
use std::time::{SystemTime, UNIX_EPOCH};
use base64::{Engine as _, engine::general_purpose::STANDARD as BASE64};
use futures::future::BoxFuture;
@@ -38,7 +38,7 @@ impl RequestPolicyPlan for PluginRequestPolicyPlan {
input: gateway_core::engine::policy::RetryInput,
) -> BoxFuture<'static, Result<gateway_core::engine::policy::RetryDecision, RequestPolicyFault>>
{
super::retry::decide(self.retries.clone(), input, self.policy_timeout)
super::retry::decide(self.retries.clone(), input)
}
fn route_model(
@@ -46,12 +46,11 @@ impl RequestPolicyPlan for PluginRequestPolicyPlan {
input: ModelRouteInput,
) -> BoxFuture<'static, Result<ModelRouteDecision, RequestPolicyFault>> {
let routers = std::sync::Arc::clone(&self.routers);
let timeout = self.policy_timeout;
Box::pin(async move {
for entry in routers.iter().filter(|entry| {
!input.suppresses_plugin(&entry.instance_id) && entry.scope.matches_route(&input)
}) {
let result = route_with_entry(entry, &input, timeout).await;
let result = route_with_entry(entry, &input).await;
match result {
Ok(ModelRouteDecision::Unhandled) => continue,
Ok(decision) => return Ok(decision),
@@ -78,14 +77,13 @@ impl RequestPolicyPlan for PluginRequestPolicyPlan {
input: AccountScheduleInput,
) -> BoxFuture<'static, Result<AccountScheduleDecision, RequestPolicyFault>> {
let schedulers = std::sync::Arc::clone(&self.schedulers);
let timeout = self.policy_timeout;
Box::pin(async move {
let Some(entry) = schedulers.iter().find(|entry| {
!input.suppresses_plugin(&entry.instance_id) && entry.scope.matches_schedule(&input)
}) else {
return Ok(AccountScheduleDecision::Delegate);
};
match schedule_with_entry(entry, &input, timeout).await {
match schedule_with_entry(entry, &input).await {
Ok(decision) => Ok(decision),
Err(())
if entry.failure_policy
@@ -108,7 +106,6 @@ impl RequestPolicyPlan for PluginRequestPolicyPlan {
async fn route_with_entry(
entry: &ModelRouterEntry,
input: &ModelRouteInput,
timeout: Duration,
) -> Result<ModelRouteDecision, ()> {
let invocation = entry.invocation.as_ref().ok_or(())?;
let projection = operation_projection(input.operation())?;
@@ -126,7 +123,9 @@ async fn route_with_entry(
.collect(),
headers,
};
let mut context = invocation.session.context(Stage::Routing, timeout);
let mut context = invocation
.session
.context(Stage::Routing, invocation.session.budget(Stage::Routing));
context.request = crate::adapter::request_context(input.extension_scope().request());
context.request_id = Some(input.request_id().as_str().to_owned());
let _network_scope = invocation
@@ -172,7 +171,6 @@ async fn route_with_entry(
async fn schedule_with_entry(
entry: &AccountSchedulerEntry,
input: &AccountScheduleInput,
timeout: Duration,
) -> Result<AccountScheduleDecision, ()> {
let invocation = entry.invocation.as_ref().ok_or(())?;
let candidates = input.candidates();
@@ -197,7 +195,10 @@ async fn schedule_with_entry(
})
.collect(),
};
let mut context = invocation.session.context(Stage::Scheduling, timeout);
let mut context = invocation.session.context(
Stage::Scheduling,
invocation.session.budget(Stage::Scheduling),
);
context.request = crate::adapter::request_context(input.extension_scope().request());
context.request_id = Some(input.request_id().as_str().to_owned());
context.attempt_id = Some(format!(
@@ -152,6 +152,11 @@ impl Execution {
if invocation.operation.protocol() != declaration.protocol
|| invocation.metadata.provider().as_str() != declaration.provider.as_str()
|| invocation.metadata.provider_account_id() != invocation.account.account_id()
|| (!declaration.account_ids.is_empty()
&& !declaration
.account_ids
.iter()
.any(|id| id == invocation.account.account_id().as_str()))
|| !declaration
.authentication_kinds
.iter()
@@ -162,7 +167,7 @@ impl Execution {
let timeout = invocation
.context
.deadline()
.bounded(self.adapter.session.maximum_call_timeout());
.bounded(self.adapter.session.budget(Stage::Upstream));
if timeout.is_zero() || invocation.context.cancellation().is_cancelled() {
return Err(ProviderError::new(
ProviderErrorKind::Cancelled,
@@ -1,4 +1,4 @@
//! 将插件上游适配贡献编译为可按请求选择的执行计划
//! 将插件上游适配贡献编译为按请求与账号范围解析的执行计划
mod event;
mod execution;
@@ -10,9 +10,13 @@ pub(crate) mod target;
use std::{fmt, sync::Arc};
use gateway_core::{
account::ProviderAccountId,
engine::{
AttemptContext,
upstream_adapter::{UpstreamAdapter, UpstreamAdapterInvocation, UpstreamAdapterPlan},
upstream_adapter::{
UpstreamAdapter, UpstreamAdapterInvocation, UpstreamAdapterPlan,
UpstreamAdapterSelection,
},
},
error::{ProviderError, ProviderErrorKind},
identity::ProviderKind,
@@ -58,6 +62,13 @@ impl UpstreamAdapter for PluginUpstreamAdapter {
self.declaration.transport.as_str()
}
fn supports_authentication(&self, kind: &str) -> bool {
self.declaration
.authentication_kinds
.iter()
.any(|candidate| candidate == kind)
}
fn execute(
self: Arc<Self>,
invocation: UpstreamAdapterInvocation,
@@ -91,7 +102,8 @@ impl UpstreamAdapterPlan for PluginUpstreamAdapterPlan {
context: &AttemptContext,
provider: &ProviderKind,
model: &UpstreamModelId,
) -> Result<Option<Arc<dyn UpstreamAdapter>>, ProviderError> {
account: Option<&ProviderAccountId>,
) -> Result<UpstreamAdapterSelection, ProviderError> {
let model = context
.requested_model()
.map_or(model.as_str(), |model| model.as_str());
@@ -115,13 +127,29 @@ impl UpstreamAdapterPlan for PluginUpstreamAdapterPlan {
.iter()
.any(|candidate| candidate == model))
{
if !adapter.declaration.account_ids.is_empty() {
let Some(account) = account else {
// 后置声明只覆盖未命中此账号范围的候选,不能提前以其故障拒绝整个请求
return Ok(UpstreamAdapterSelection::PendingAccount);
};
if !adapter
.declaration
.account_ids
.iter()
.any(|id| id == account.as_str())
{
continue;
}
}
if !adapter.session.is_ready() {
return Err(unavailable());
}
return Ok(Some(Arc::clone(adapter) as Arc<dyn UpstreamAdapter>));
return Ok(UpstreamAdapterSelection::Adapter(
Arc::clone(adapter) as Arc<dyn UpstreamAdapter>
));
}
}
Ok(None)
Ok(UpstreamAdapterSelection::Native)
}
}
@@ -47,12 +47,6 @@ pub(super) fn merge(
.try_with_service_tier(value)
.map_err(|source| invalid(sent).with_source(source))?;
}
if let Some(value) = update.upstream_response_model {
current = current.with_upstream_response_model_if_valid(&value);
if current.upstream_response_model() != Some(value.trim()) {
return Err(invalid(sent));
}
}
if !update.client_headers.is_empty() {
current = current.with_client_headers(
update
@@ -1,6 +1,6 @@
//! 校验并准备插件上游适配声明与能力绑定
use std::{sync::Arc, time::Duration};
use std::sync::Arc;
use gateway_admin::model::{
AdminError,
@@ -56,7 +56,7 @@ pub(crate) async fn prepare(
let reply = session
.call(
REGISTER_METHOD,
session.context(Stage::Registration, Duration::from_secs(5)),
session.context(Stage::Registration, session.budget(Stage::Registration)),
serde_json::json!({}),
vec![],
)
@@ -119,7 +119,6 @@ fn observation(value: &ProviderResponseObservation) -> facts::ResponseObservatio
status_code: value.status_code(),
request_id: value.request_id().map(|value| value.as_str().to_owned()),
service_tier: value.service_tier().map(str::to_owned),
upstream_response_model: value.upstream_response_model().map(str::to_owned),
timings: facts::ResponseTimings {
transport_decision_wait_ms: timings.transport_decision_wait_ms,
connect_ms: timings.connect_ms,
@@ -14,6 +14,7 @@ mod keys;
mod log;
mod middleware;
mod model;
pub(crate) mod outbound;
pub(crate) mod private_state;
pub(crate) mod request_limits;
mod resources;
@@ -197,6 +198,7 @@ impl PluginCallbacks {
context.stage,
gateway_plugin_sdk::Stage::Request
| gateway_plugin_sdk::Stage::Attempt
| gateway_plugin_sdk::Stage::Outbound
| gateway_plugin_sdk::Stage::Routing
| gateway_plugin_sdk::Stage::Scheduling
| gateway_plugin_sdk::Stage::Retry
@@ -0,0 +1,98 @@
//! 出站准备的单次续体与完整请求编解码,不发起网络请求
use std::sync::{Arc, Mutex};
use futures::future::BoxFuture;
use gateway_core::engine::middleware::{MiddlewareHeader, outbound as core};
use gateway_plugin_sdk::{
PluginFault,
call::middleware::{MiddlewareHeader as WireHeader, NEXT_METHOD, outbound as wire},
};
use super::{MiddlewareCallback, invalid};
use crate::RpcReply;
pub(crate) struct Invocation(Mutex<Option<core::Next>>);
impl Invocation {
pub(crate) fn new(next: core::Next) -> Arc<Self> {
Arc::new(Self(Mutex::new(Some(next))))
}
}
impl MiddlewareCallback for Invocation {
fn invoke(
self: Arc<Self>,
method: String,
params: serde_json::Value,
payload: Vec<u8>,
_maximum_payload: usize,
) -> BoxFuture<'static, Result<RpcReply, PluginFault>> {
Box::pin(async move {
if method != NEXT_METHOD {
return Err(invalid());
}
let request_head = serde_json::from_value(params).map_err(|_| invalid())?;
let request = decode(request_head, payload)?;
let next = self
.0
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.take()
.ok_or_else(invalid)?;
let request = next
.run(request)
.await
.map_err(|error| super::error::middleware(&error))?;
Ok(RpcReply {
result: serde_json::to_value(head(&request)).map_err(|_| invalid())?,
payload: request.body.into(),
})
})
}
}
pub(crate) fn head(request: &core::Request) -> wire::RequestHead {
wire::RequestHead {
target: match &request.target {
core::Target::Http { method, uri } => wire::Target::Http {
method: method.to_string(),
uri: uri.to_string(),
},
core::Target::WebSocket { uri } => wire::Target::WebSocket {
uri: uri.to_string(),
},
},
headers: request
.headers
.iter()
.map(|header| WireHeader {
name: header.name().to_owned(),
value: header.value().to_vec(),
})
.collect(),
}
}
pub(crate) fn decode(
head: wire::RequestHead,
payload: Vec<u8>,
) -> Result<core::Request, PluginFault> {
Ok(core::Request {
target: match head.target {
wire::Target::Http { method, uri } => core::Target::Http {
method: method.parse().map_err(|_| invalid())?,
uri: uri.parse().map_err(|_| invalid())?,
},
wire::Target::WebSocket { uri } => core::Target::WebSocket {
uri: uri.parse().map_err(|_| invalid())?,
},
},
headers: head
.headers
.into_iter()
.map(|header| MiddlewareHeader::new(header.name, header.value.into()))
.collect(),
body: payload.into(),
})
}
@@ -16,18 +16,22 @@ use gateway_admin::{
use gateway_plugin_sdk::{Handshake, Stage, call::registration::Registration};
use std::{sync::Arc, time::Duration};
const PROCESS_PREPARATION_TIMEOUT: Duration = Duration::from_secs(30);
struct StartedInstance {
instance: PluginInstance,
package: Arc<crate::PreparedPackage>,
callbacks: Arc<PluginCallbacks>,
prepared: PreparedInstance,
}
impl PluginRuntime {
pub(super) async fn prepare_instance(
async fn start_instance(
&self,
instance: PluginInstance,
target_revision: Revision,
request_body_limit: Option<usize>,
) -> Result<Arc<PreparedInstance>, AdminError> {
let mut observer_entries = Vec::new();
let mut commands = Vec::new();
let mut management = Vec::new();
let mut policy_entries = Vec::new();
let mut authentication_entries = Vec::new();
) -> Result<StartedInstance, AdminError> {
let authorization_binding = instance_fingerprint(&instance)?;
if self.restart_circuit_is_open(&RestartIdentity::new(
instance.id.clone(),
@@ -109,7 +113,6 @@ impl PluginRuntime {
crate::compatibility::report(&manifest, &instance_id)?;
let restart_identity =
RestartIdentity::new(instance_id.clone(), authorization_binding.clone());
let bindings = instance.bindings.clone();
let owner = self
.state
.load_owner(
@@ -181,23 +184,69 @@ impl PluginRuntime {
lifecycle,
)
.await
.map_err(|_| AdminError::unavailable("插件进程握手失败"))?,
.map_err(|error| match error {
crate::RpcError::Budget(gateway_plugin_sdk::CallBudgetError::Invalid {
stage,
milliseconds,
}) => AdminError::invalid(format!(
"插件阶段 {stage:?} 的调用预算无效:{milliseconds} 毫秒"
)),
_ => AdminError::unavailable("插件进程握手失败"),
})?,
);
let mut prepared = PreparedInstance {
let prepared = PreparedInstance {
instance_id: instance_id.clone(),
artifact_sha256: instance.artifact_sha256.clone(),
revision: instance.revision,
session: session.clone(),
session,
private_state: private_state.clone(),
maintenance: manifest
.contributes
.contains_key(&gateway_plugin_sdk::Capability::Maintenance),
contributions: PreparedContributions::default(),
};
Ok(StartedInstance {
instance,
package,
callbacks,
prepared,
})
}
pub(super) async fn prepare_instance(
&self,
instance: PluginInstance,
target_revision: Revision,
request_body_limit: Option<usize>,
) -> Result<Arc<PreparedInstance>, AdminError> {
// 启动控制期限只包住制品准备与握手,Ready 后每次注册使用协商预算
let StartedInstance {
instance,
package,
callbacks,
mut prepared,
} = tokio::time::timeout(
PROCESS_PREPARATION_TIMEOUT,
self.start_instance(instance, target_revision, request_body_limit),
)
.await
.map_err(|_| AdminError::unavailable("插件启动超时"))??;
let session = &prepared.session;
let manifest = package.package().manifest();
let plugin_id = manifest
.plugin_id()
.map_err(|_| AdminError::invalid("插件身份无效"))?;
let instance_id = &instance.id;
let bindings = &instance.bindings;
let mut observer_entries = Vec::new();
let mut commands = Vec::new();
let mut management = Vec::new();
let mut policy_entries = Vec::new();
let mut authentication_entries = Vec::new();
let registration = session
.call(
"plugin.register",
session.context(Stage::Registration, Duration::from_secs(5)),
session.context(Stage::Registration, session.budget(Stage::Registration)),
serde_json::json!({}),
vec![],
)
@@ -211,7 +260,7 @@ impl PluginRuntime {
commands.extend(
crate::adapter::command_line::prepare(
&instance,
&manifest,
manifest,
session.clone(),
callbacks.clone(),
)
@@ -228,8 +277,8 @@ impl PluginRuntime {
management.push(entry);
}
if let Some(entry) = crate::adapter::frontend_authentication::prepare_entry(
&manifest,
&bindings,
manifest,
bindings,
session.clone(),
callbacks.clone(),
)
@@ -238,35 +287,34 @@ impl PluginRuntime {
authentication_entries.push(entry);
}
if let Some(entry) = crate::adapter::observer::compile_entry(
&manifest,
manifest,
&plugin_id,
&instance_id,
&bindings,
instance_id,
bindings,
session.clone(),
callbacks.clone(),
)? {
observer_entries.push(entry);
}
policy_entries.extend(crate::adapter::policy::compile_entries(
&manifest,
&instance_id,
&bindings,
manifest,
instance_id,
bindings,
session.clone(),
callbacks.clone(),
)?);
let middleware_entries = crate::adapter::middleware::compile_entries(
&manifest,
&instance_id,
&bindings,
manifest,
instance_id,
bindings,
session.clone(),
callbacks.clone(),
)?;
let model_aliases =
crate::adapter::catalog::prepare(&manifest, &instance, &session).await?;
let model_aliases = crate::adapter::catalog::prepare(manifest, &instance, session).await?;
let upstream_entries = crate::adapter::upstream_adapter::prepare(
&manifest,
manifest,
&instance,
Arc::clone(&session),
Arc::clone(session),
Arc::clone(&callbacks),
)
.await?;
@@ -1,8 +1,11 @@
//! 对账只从已发布集合开始;通知合并,实例串行,失败和停用不阻塞其他实例
use super::{PluginRuntime, set::PreparedSet};
use super::{
PluginRuntime,
set::{PreparedInstance, PreparedSet},
};
use crate::RpcSession;
use futures::{StreamExt as _, future::BoxFuture};
use futures::{StreamExt as _, future::BoxFuture, stream::FuturesUnordered};
use gateway_core::{
lifecycle::CancellationToken,
runtime::RuntimeSnapshotHandle,
@@ -13,12 +16,51 @@ use gateway_core::{
},
};
use gateway_plugin_sdk::Stage;
use std::{sync::Arc, time::Duration};
use tokio::{sync::Notify, task::JoinSet};
use std::{collections::BTreeMap, sync::Arc, time::Duration};
use tokio::{sync::Notify, task::JoinHandle};
const RECONCILE_INTERVAL: Duration = Duration::from_secs(30);
const RETRY_DELAY: Duration = Duration::from_secs(5);
const CALL_TIMEOUT: Duration = Duration::from_secs(30);
struct InstanceTask {
instance: Arc<PreparedInstance>,
dirty: Arc<Notify>,
cancellation: CancellationToken,
task: JoinHandle<()>,
}
impl InstanceTask {
fn start(instance: Arc<PreparedInstance>) -> Self {
let dirty = Arc::new(Notify::new());
dirty.notify_one();
let cancellation = CancellationToken::new();
let task = tokio::spawn(reconcile(
Arc::clone(&instance.session),
Arc::clone(&dirty),
cancellation.clone(),
));
Self {
instance,
dirty,
cancellation,
task,
}
}
async fn stop(mut self) {
self.cancellation.cancel();
let _ = (&mut self.task).await;
// 宿主资源在取消时同步撤销,替换任务还须等对端确认或进程退出
self.instance.session.settle_cancellations().await;
}
}
impl Drop for InstanceTask {
fn drop(&mut self) {
self.cancellation.cancel();
self.task.abort();
}
}
struct MaintenanceTask {
runtime: Arc<PluginRuntime>,
@@ -59,18 +101,12 @@ impl DaemonTask for MaintenanceTask {
let mut publications = self.snapshots.subscribe_publications();
let mut active: Option<Arc<PreparedSet>> = None;
let mut revision = None;
let mut stop = CancellationToken::new();
let mut tasks = JoinSet::new();
let mut notifications = Vec::new();
let mut tasks = BTreeMap::<String, InstanceTask>::new();
loop {
let snapshot = self.snapshots.acquire().ok();
let reference = snapshot.as_ref().and_then(|s| s.extensions());
let same_set = active.as_ref().map(|s| &s.id) == reference.map(|s| s.id());
if !same_set {
stop.cancel();
while tasks.join_next().await.is_some() {}
notifications.clear();
stop = CancellationToken::new();
active = match reference {
Some(reference) => {
Some(self.runtime.prepared_set(reference).await.map_err(|_| {
@@ -79,17 +115,37 @@ impl DaemonTask for MaintenanceTask {
}
None => None,
};
if let Some(set) = &active {
for instance in set.sessions.iter().filter(|i| i.maintenance) {
let dirty = Arc::new(Notify::new());
dirty.notify_one();
notifications.push(dirty.clone());
tasks.spawn(reconcile(instance.session.clone(), dirty, stop.clone()));
}
let desired = active
.iter()
.flat_map(|set| &set.sessions)
.filter(|instance| instance.maintenance)
.map(|instance| (instance.instance_id.clone(), Arc::clone(instance)))
.collect::<BTreeMap<_, _>>();
let retired = tasks
.extract_if(.., |id, task| {
// 发布层只复用完全相同且就绪的实例,Arc 身份也包含会话 incarnation
!desired
.get(id)
.is_some_and(|instance| Arc::ptr_eq(instance, &task.instance))
})
.map(|(_, task)| task)
.collect::<Vec<_>>();
for task in &retired {
task.cancellation.cancel();
}
} else if revision != snapshot.as_ref().map(|s| s.revision()) {
for dirty in &notifications {
dirty.notify_one();
futures::future::join_all(retired.into_iter().map(InstanceTask::stop)).await;
for task in tasks.values() {
task.dirty.notify_one();
}
for (id, instance) in desired {
tasks
.entry(id)
.or_insert_with(|| InstanceTask::start(instance));
}
}
if same_set && revision != snapshot.as_ref().map(|s| s.revision()) {
for task in tasks.values() {
task.dirty.notify_one();
}
}
revision = snapshot.as_ref().map(|s| s.revision());
@@ -101,22 +157,39 @@ impl DaemonTask for MaintenanceTask {
biased;
() = cancellation.cancelled() => break,
changed = publications.next() => if changed.is_none() { break; },
result = tasks.join_next(), if !tasks.is_empty() => {
if result.is_some() {
stop.cancel();
while tasks.join_next().await.is_some() {}
result = next_finished(&mut tasks), if !tasks.is_empty() => {
if let Some(instance_id) = result {
tasks.remove(&instance_id);
stop_all(tasks).await;
return Err(WorkerTaskError::safe("plugin maintenance task stopped unexpectedly"));
}
}
}
}
stop.cancel();
while tasks.join_next().await.is_some() {}
stop_all(tasks).await;
Ok(())
})
}
}
async fn next_finished(tasks: &mut BTreeMap<String, InstanceTask>) -> Option<String> {
let mut pending = tasks
.iter_mut()
.map(|(id, task)| async move {
let _ = (&mut task.task).await;
id.clone()
})
.collect::<FuturesUnordered<_>>();
pending.next().await
}
async fn stop_all(tasks: BTreeMap<String, InstanceTask>) {
for task in tasks.values() {
task.cancellation.cancel();
}
futures::future::join_all(tasks.into_values().map(InstanceTask::stop)).await;
}
async fn reconcile(session: Arc<RpcSession>, dirty: Arc<Notify>, cancellation: CancellationToken) {
loop {
tokio::select! {
@@ -125,21 +198,38 @@ async fn reconcile(session: Arc<RpcSession>, dirty: Arc<Notify>, cancellation: C
() = dirty.notified() => {},
() = tokio::time::sleep(RECONCILE_INTERVAL) => {},
}
let context = session.context(Stage::Maintenance, CALL_TIMEOUT);
let context = session.context(Stage::Maintenance, session.budget(Stage::Maintenance));
let instance_id = context.instance_id.clone();
let resource_scope_id = context.resource_scope_id.clone();
let budget_ms = context.timeout_ms;
let started = tokio::time::Instant::now();
let result = tokio::select! {
biased;
() = cancellation.cancelled() => return,
() = cancellation.cancelled() => {
tracing::info!(target: "plugin.maintenance", %instance_id, %resource_scope_id, budget_ms,
budget_source = ?session.budget_source(Stage::Maintenance),
elapsed_ms = started.elapsed().as_millis() as u64,
reason = "cancelled", "plugin maintenance ended");
return;
},
result = session.call("plugin.reconcile", context, serde_json::json!({}), Vec::new()) => result,
};
if result.is_err() {
tracing::warn!(%instance_id, "plugin maintenance failed; retrying after bounded delay");
if let Err(error) = result {
tracing::warn!(target: "plugin.maintenance", %instance_id, %resource_scope_id, budget_ms,
budget_source = ?session.budget_source(Stage::Maintenance),
elapsed_ms = started.elapsed().as_millis() as u64,
reason = error.diagnostic().0, "plugin maintenance failed; retrying after bounded delay");
tokio::select! {
biased;
() = cancellation.cancelled() => return,
() = tokio::time::sleep(RETRY_DELAY) => {},
}
dirty.notify_one();
} else {
tracing::info!(target: "plugin.maintenance", %instance_id, %resource_scope_id, budget_ms,
budget_source = ?session.budget_source(Stage::Maintenance),
elapsed_ms = started.elapsed().as_millis() as u64,
reason = "completed", "plugin maintenance ended");
}
}
}
@@ -27,15 +27,13 @@ use gateway_core::{
observation::RequestObserverExtensionIndex,
policy::RequestPolicyExtensionIndex,
},
lifecycle::CancellationToken,
provider_ports::OAuthPendingFlowPort,
};
use std::{
collections::BTreeMap,
path::PathBuf,
sync::{
Arc, Mutex as SyncMutex, Weak,
atomic::{AtomicBool, Ordering},
},
sync::{Arc, Mutex as SyncMutex, Weak},
time::Duration,
};
use tokio::sync::{Mutex, Semaphore};
@@ -62,7 +60,7 @@ pub struct PluginRuntime {
pub(super) prepared_instances: Mutex<BTreeMap<String, Weak<PreparedInstance>>>,
pub(super) preparation_diagnostics: SyncMutex<BTreeMap<u64, PreparationDiagnostic>>,
pub(super) restart_circuits: Arc<RestartCircuits>,
pub(super) shutting_down: Arc<AtomicBool>,
pub(super) shutting_down: CancellationToken,
shutdown_lock: Mutex<()>,
pub(super) processes: Arc<gateway_host::process::ProcessSupervisor>,
pub(super) validators: Arc<Semaphore>,
@@ -105,7 +103,7 @@ impl PluginRuntime {
prepared_instances: Mutex::new(BTreeMap::new()),
preparation_diagnostics: SyncMutex::new(BTreeMap::new()),
restart_circuits: RestartCircuits::new(restart_circuit),
shutting_down: Arc::new(AtomicBool::new(false)),
shutting_down: CancellationToken::new(),
shutdown_lock: Mutex::new(()),
processes,
validators: Arc::new(Semaphore::new(MAXIMUM_PARALLEL_PREPARATIONS)),
@@ -182,7 +180,7 @@ impl PluginRuntime {
/// 调用方必须先停止接收 HTTP 与 Worker;这里拒绝新候选并等待全部插件 I/O 退出
pub async fn shutdown(&self) {
self.shutting_down.store(true, Ordering::Release);
self.shutting_down.cancel();
let _shutdown = self.shutdown_lock.lock().await;
let _prepare = self.prepare_lock.lock().await;
let sets = {
@@ -10,18 +10,12 @@ use gateway_core::{
extensions::ExecutionExtensionPlans, observation::RequestObserverPlan,
policy::RequestPolicyPlan,
},
lifecycle::CancellationToken,
routing::extensions::{ExtensionSetId, ExtensionSetLease},
};
use secrecy::ExposeSecret as _;
use sha2::{Digest as _, Sha256};
use std::{
collections::BTreeMap,
sync::{
Arc,
atomic::{AtomicBool, Ordering},
},
time::Duration,
};
use std::{collections::BTreeMap, sync::Arc, time::Duration};
pub(super) struct PreparedSet {
pub(super) id: ExtensionSetId,
@@ -32,7 +26,7 @@ pub(super) struct PreparedSet {
Option<Arc<dyn gateway_core::engine::authentication::FrontendAuthenticationPlan>>,
pub(super) sessions: Vec<Arc<PreparedInstance>>,
pub(super) failures: BTreeMap<String, PluginInstanceRuntimeFailure>,
pub(super) shutting_down: Arc<AtomicBool>,
pub(super) shutting_down: CancellationToken,
pub(super) commands: Vec<Arc<crate::adapter::command_line::PluginCommand>>,
pub(super) management: Vec<crate::adapter::management::ManagementEntry>,
pub(super) model_aliases: Vec<gateway_core::routing::ContributedModelAlias>,
@@ -94,7 +88,7 @@ impl ExtensionSetLease for PreparedSet {
fn can_serve(&self) -> bool {
// 发布的是可用能力和故障绑定组成的完整计划;单个进程退出不能使原生转发失效
!self.shutting_down.load(Ordering::Acquire)
!self.shutting_down.is_cancelled()
}
}
@@ -26,8 +26,7 @@ use gateway_core::{
};
use std::{
collections::{BTreeMap, BTreeSet},
sync::{Arc, Weak, atomic::Ordering},
time::Duration,
sync::{Arc, Weak},
};
impl PluginRuntime {
@@ -39,9 +38,11 @@ impl PluginRuntime {
) -> Result<ExtensionSetReference, AdminError> {
let target_revision = snapshot.config_revision.get();
self.record_preparation(target_revision, PreparationDiagnostic::Preparing);
let result = self
.prepare_inner(snapshot, required_revision, request_body_limit)
.await;
let result = tokio::select! {
biased;
() = self.shutting_down.cancelled() => Err(AdminError::unavailable("插件运行时正在关闭")),
result = self.prepare_inner(snapshot, required_revision, request_body_limit) => result,
};
match &result {
Ok(reference) => self.record_preparation(
target_revision,
@@ -63,7 +64,7 @@ impl PluginRuntime {
required_revision: Option<Revision>,
request_body_limit: Option<usize>,
) -> Result<ExtensionSetReference, AdminError> {
if self.shutting_down.load(Ordering::Acquire) {
if self.shutting_down.is_cancelled() {
return Err(AdminError::unavailable("插件运行时正在关闭"));
}
if snapshot.instances.len() > 64 {
@@ -72,7 +73,7 @@ impl PluginRuntime {
let fingerprint = fingerprint(&snapshot)?;
// 准备串行化与弱索引分锁;诊断和已发布请求不会等待外部 I/O
let _prepare = self.prepare_lock.lock().await;
if self.shutting_down.load(Ordering::Acquire) {
if self.shutting_down.is_cancelled() {
return Err(AdminError::unavailable("插件运行时正在关闭"));
}
let existing = {
@@ -125,17 +126,12 @@ impl PluginRuntime {
let result = if let Some(instance) = reusable {
Ok(instance)
} else {
tokio::time::timeout(
// 沿用准备阶段原有的 30 秒预算;打包校验与能力恢复也在预算内
Duration::from_secs(30),
self.prepare_instance(
instance.clone(),
snapshot.config_revision,
request_body_limit,
),
self.prepare_instance(
instance.clone(),
snapshot.config_revision,
request_body_limit,
)
.await
.unwrap_or_else(|_| Err(AdminError::unavailable("插件启动超时")))
};
(instance, identity, result)
},
@@ -208,7 +204,6 @@ impl PluginRuntime {
.rpc_limits
.maximum_calls
.min(self.config.rpc_limits.maximum_executing_calls),
self.config.rpc_limits.maximum_call_timeout,
self.config.rpc_limits.maximum_buffered_body_bytes,
)
.map(|plan| {
@@ -217,21 +212,16 @@ impl PluginRuntime {
.map_err(|_| AdminError::invalid("插件观察计划注册冲突"))
})
.transpose()?;
let policies = crate::adapter::policy::PluginRequestPolicyPlan::compile(
policy_entries,
self.config.rpc_limits.maximum_call_timeout,
)?
.map(|plan| {
self.policies
.register(id.clone(), plan)
.map_err(|_| AdminError::invalid("插件请求策略计划注册冲突"))
})
.transpose()?;
let middleware = crate::adapter::middleware::PluginMiddlewarePlan::compile(
middleware_entries,
self.config.rpc_limits.maximum_call_timeout,
)
.map(|plan| plan as Arc<dyn gateway_core::engine::middleware::MiddlewarePlan>);
let policies = crate::adapter::policy::PluginRequestPolicyPlan::compile(policy_entries)?
.map(|plan| {
self.policies
.register(id.clone(), plan)
.map_err(|_| AdminError::invalid("插件请求策略计划注册冲突"))
})
.transpose()?;
let middleware =
crate::adapter::middleware::PluginMiddlewarePlan::compile(middleware_entries)
.map(|plan| plan as Arc<dyn gateway_core::engine::middleware::MiddlewarePlan>);
let upstream_adapters =
crate::adapter::upstream_adapter::PluginUpstreamAdapterPlan::compile(upstream_entries);
let execution = if middleware.is_some() || upstream_adapters.is_some() {
@@ -252,7 +242,6 @@ impl PluginRuntime {
let authentication =
crate::adapter::frontend_authentication::PluginFrontendAuthenticationPlan::compile(
authentication_entries,
self.config.rpc_limits.maximum_call_timeout,
)?
.map(|plan| {
self.authentications
@@ -90,9 +90,10 @@ impl PluginRuntime {
.session
.call(
"plugin.state.migrate",
instance
.session
.context(Stage::Configuration, Duration::from_secs(30)),
instance.session.context(
Stage::Configuration,
instance.session.budget(Stage::Configuration),
),
serde_json::json!({}),
serde_json::to_vec(&request)
.map_err(|_| AdminError::internal("状态迁移批次无法编码"))?,
@@ -207,7 +208,11 @@ impl gateway_admin::ports::plugins::PluginStateLifecycle for PluginRuntime {
prepared: &ExtensionSetReference,
transition: PluginStateTransition,
) -> Result<(), AdminError> {
self.run_state_migration(prepared, transition).await
tokio::select! {
biased;
() = self.shutting_down.cancelled() => Err(AdminError::unavailable("插件运行时正在关闭")),
result = self.run_state_migration(prepared, transition) => result,
}
}
}
pub(super) fn map_state_admin_error(
@@ -12,10 +12,13 @@ use std::{
use futures::future::BoxFuture;
use gateway_host::process::{ProcessControl, ProcessSpec, ProcessStartError, ProcessSupervisor};
use gateway_plugin_sdk::{
CallContext, Frame, Handshake, Message, PROTOCOL_VERSION, PluginFault, Stage,
CallBudgetError, CallBudgetSource, CallContext, Frame, Handshake, Message, PROTOCOL_VERSION,
PluginFault, ResolvedCallBudgets, Stage,
client::{CallDeadline, CallDeadlinePause, EncodedFrame, read_frame, write_frame},
};
use tokio::sync::{Mutex as AsyncMutex, OwnedSemaphorePermit, Semaphore, mpsc, oneshot, watch};
use tokio::sync::{
Mutex as AsyncMutex, Notify, OwnedSemaphorePermit, Semaphore, mpsc, oneshot, watch,
};
use crate::{PreparedPackage, RpcStream, stream::StreamIngress};
@@ -32,7 +35,6 @@ pub struct RpcLimits {
pub maximum_queue_timeout: Duration,
pub maximum_callbacks: usize,
pub handshake_timeout: Duration,
pub maximum_call_timeout: Duration,
pub maximum_stderr_bytes: usize,
}
@@ -46,7 +48,6 @@ impl Default for RpcLimits {
maximum_queue_timeout: Duration::from_secs(30),
maximum_callbacks: 64,
handshake_timeout: Duration::from_secs(5),
maximum_call_timeout: Duration::from_secs(120),
maximum_stderr_bytes: 64 * 1024,
}
}
@@ -58,6 +59,8 @@ pub enum RpcError {
Start(#[source] ProcessStartError),
#[error("plugin handshake does not match the prepared instance")]
Handshake(#[source] Option<gateway_core::error::ErrorSource>),
#[error("plugin call budget negotiation failed: {0}")]
Budget(#[from] CallBudgetError),
#[error("plugin protocol is invalid")]
Protocol(#[source] Option<gateway_core::error::ErrorSource>),
#[error("plugin returned an invalid response at {0:?}")]
@@ -89,6 +92,7 @@ impl RpcError {
match self {
RpcError::Start(_) => ("process_start", "插件进程启动失败"),
RpcError::Handshake(_) => ("handshake", "插件握手失败"),
RpcError::Budget(_) => ("call_budget", "插件调用预算协商失败"),
RpcError::Protocol(_) => ("protocol", "插件协议错误"),
RpcError::InvalidResponse(stage, _) => (
"invalid_response",
@@ -104,6 +108,7 @@ impl RpcError {
Stage::WebSocket => "WebSocket 中间件:插件返回了无效响应",
Stage::Request => "请求中间件:插件返回了无效响应",
Stage::Attempt => "尝试中间件:插件返回了无效响应",
Stage::Outbound => "出站准备中间件:插件返回了无效响应",
Stage::Upstream => "上游适配阶段:插件返回了无效响应",
Stage::Observation => "观察阶段:插件返回了无效响应",
Stage::Management | Stage::PublicManagement => "管理阶段:插件返回了无效响应",
@@ -190,6 +195,7 @@ pub trait CallbackHandler: Send + Sync {
pub struct RpcSession {
shared: Arc<Shared>,
handshake: Handshake,
budgets: ResolvedCallBudgets,
data: mpsc::Sender<EncodedFrame>,
slots: Arc<Semaphore>,
next_call: AsyncMutex<u64>,
@@ -221,6 +227,7 @@ pub(crate) struct Shared {
callback_slots: Arc<Semaphore>,
next_slots: Arc<Semaphore>,
execution: ExecutionCapacity,
cancellations_changed: Notify,
}
struct State {
@@ -294,6 +301,7 @@ impl Shared {
}
self.stopped.send_replace(true);
self.process.stop();
self.cancellations_changed.notify_waiters();
}
fn expect_stop(&self) {
@@ -511,6 +519,8 @@ impl Shared {
return Err(RpcError::Protocol(None));
}
state.calls.remove(&id);
drop(state);
self.cancellations_changed.notify_waiters();
Ok(())
}
@@ -567,8 +577,36 @@ impl Shared {
}
impl RpcSession {
pub(crate) const fn maximum_call_timeout(&self) -> Duration {
self.limits.maximum_call_timeout
pub fn budget(&self, stage: Stage) -> Duration {
self.budgets.budget(stage)
}
#[must_use]
pub(crate) fn budget_source(&self, stage: Stage) -> CallBudgetSource {
self.budgets.source(stage)
}
/// 替换维护任务前等待已取消的调用确认;未确认仍沿用会话的有界故障回收
pub(crate) async fn settle_cancellations(&self) {
let ids: Vec<_> = self
.shared
.state()
.calls
.iter()
.filter_map(|(id, call)| call.cancellation.is_some().then_some(*id))
.collect();
loop {
let changed = self.shared.cancellations_changed.notified();
tokio::pin!(changed);
changed.as_mut().enable();
if !ids.iter().any(|id| self.shared.is_cancelling(*id)) {
break;
}
changed.await;
}
if !self.is_ready() {
self.shared.process.exited().await;
}
}
pub async fn start(
@@ -619,7 +657,6 @@ impl RpcSession {
|| limits.maximum_buffered_body_bytes < 1024
|| limits.maximum_buffered_body_bytes > 16 * 1024 * 1024
|| limits.handshake_timeout.is_zero()
|| limits.maximum_call_timeout.is_zero()
{
return Err(RpcError::Context(None));
}
@@ -690,11 +727,12 @@ impl RpcSession {
Message::Ready {
protocol_version,
incarnation,
call_budgets,
} if protocol_version == handshake.protocol_version
&& incarnation == handshake.incarnation
&& reply.payload.is_empty() =>
{
Ok(())
ResolvedCallBudgets::resolve(&call_budgets).map_err(RpcError::Budget)
}
_ => Err(RpcError::Handshake(None)),
}
@@ -702,15 +740,18 @@ impl RpcSession {
.await
.map_err(|_| RpcError::Timeout)
.and_then(|result| result);
if let Err(error) = ready {
if let Some(lifecycle) = &lifecycle {
// 只有 Ready 之后的存活才可复位预算;长时间卡在握手不等于稳定运行
lifecycle.fail(Duration::ZERO);
let budgets = match ready {
Ok(budgets) => budgets,
Err(error) => {
if let Some(lifecycle) = &lifecycle {
// 只有 Ready 之后的存活才可复位预算;长时间卡在握手不等于稳定运行
lifecycle.fail(Duration::ZERO);
}
process.stop();
process.exited().await;
return Err(error);
}
process.stop();
process.exited().await;
return Err(error);
}
};
let (data, data_receiver) = mpsc::channel(limits.maximum_calls);
let (control, control_receiver) =
mpsc::channel(limits.maximum_callbacks + 2 * limits.maximum_calls + 8);
@@ -736,6 +777,7 @@ impl RpcSession {
limits.maximum_queued_calls,
limits.maximum_queue_timeout,
),
cancellations_changed: Notify::new(),
});
dispatch::start(
input,
@@ -759,6 +801,7 @@ impl RpcSession {
Ok(Self {
shared,
handshake,
budgets,
data,
slots: Arc::new(Semaphore::new(limits.maximum_calls)),
next_call: AsyncMutex::new(1),
@@ -798,8 +841,8 @@ impl RpcSession {
#[must_use]
pub fn context(&self, stage: gateway_plugin_sdk::Stage, timeout: Duration) -> CallContext {
// 宿主生成的上下文受本会话预算约束;固定注册期限不能使较小的总预算无法启动
let timeout = timeout.min(self.limits.maximum_call_timeout);
// 调用方可以传入更短的外部剩余期限,不能扩大实例已协商的阶段预算
let timeout = timeout.min(self.budget(stage));
CallContext {
request: None,
call_id: 0,
@@ -822,7 +865,7 @@ impl RpcSession {
|| context.generation != self.handshake.generation
|| context.incarnation != self.handshake.incarnation
|| context.timeout_ms == 0
|| Duration::from_millis(context.timeout_ms) > self.limits.maximum_call_timeout
|| Duration::from_millis(context.timeout_ms) > self.budget(context.stage)
|| method.is_empty()
|| method.len() > 128
{
@@ -6,6 +6,120 @@ use serde_json::json;
use crate::support::environment::Environment;
#[derive(Default)]
struct BudgetAccountAccess {
calls: std::sync::atomic::AtomicUsize,
first_started: tokio::sync::Notify,
second_started: tokio::sync::Notify,
}
#[async_trait::async_trait]
impl gateway_admin::ports::plugin_accounts::PluginAccountAccess for BudgetAccountAccess {
async fn list(
&self,
_: gateway_admin::model::provider_credentials::PluginAccountListQuery,
) -> Result<
gateway_admin::model::provider_credentials::PluginAccountPage,
gateway_admin::model::AdminError,
> {
panic!("账号提交不应读取账号列表")
}
async fn get_runtime(
&self,
_: &gateway_core::account::ProviderAccountId,
) -> Result<gateway_admin::model::accounts::AccountRecord, gateway_admin::model::AdminError>
{
panic!("创建账号不应读取已有账号")
}
async fn get_credential(
&self,
_: &gateway_core::account::ProviderAccountId,
) -> Result<
gateway_admin::model::provider_credentials::PluginAccountCredential,
gateway_admin::model::AdminError,
> {
panic!("账号提交不应读取凭据")
}
async fn save(
&self,
_: gateway_admin::model::provider_credentials::PreparedPluginAccountSave,
_: &gateway_admin::model::MutationContext,
) -> Result<
gateway_admin::model::provider_credentials::PluginAccountSaveResult,
gateway_admin::model::AdminError,
> {
if self.calls.fetch_add(1, std::sync::atomic::Ordering::SeqCst) == 0 {
self.first_started.notify_one();
tokio::time::sleep(std::time::Duration::from_secs(70)).await;
Ok(
gateway_admin::model::provider_credentials::PluginAccountSaveResult {
config_revision: gateway_admin::model::Revision::new(2).unwrap(),
account_id: gateway_core::account::ProviderAccountId::new(
"acct_command_budget",
)
.unwrap(),
credential_revision: gateway_admin::model::Revision::new(1).unwrap(),
},
)
} else {
self.second_started.notify_one();
std::future::pending().await
}
}
}
#[tokio::test]
async fn account_commits_share_the_commands_declared_deadline_and_report_partial_completion() {
use gateway_admin::ports::plugin_accounts::PluginAccountAccess;
use std::{sync::Arc, time::Duration};
let account = json!({"action":"create","provider_id":"openai","facts":{
"name":"预算测试","authentication_kind":"api_key","material":{"key":"fixture-only"}
}});
let config = json!({
"call_budgets_ms":{"command_line":180_000},
"command_registration":{"commands":[{"name":"save","description":"提交账号"}]},
"command_result":{"stdout":"","stderr":"","exit_code":0,"accounts":[account, account]},
});
let (_directory, _store, runtime) = super::setup(config).await;
let probe = Arc::new(BudgetAccountAccess::default());
let accounts: Arc<dyn PluginAccountAccess> = probe.clone();
runtime.bind_account_ports(&accounts).unwrap();
let commands = runtime.prepare_command_line().await.unwrap();
let task = tokio::spawn(async move {
let result = commands.execute("commands", "save", &[]).await;
(commands, result)
});
tokio::time::timeout(Duration::from_secs(5), probe.first_started.notified())
.await
.unwrap();
tokio::time::pause();
tokio::time::advance(Duration::from_secs(121)).await;
probe.second_started.notified().await;
assert!(
!task.is_finished(),
"账号提交沿用声明预算,不能被默认期限截断"
);
tokio::time::advance(Duration::from_secs(60)).await;
let (commands, result) = task.await.unwrap();
tokio::time::resume();
assert!(
matches!(
result,
Err(PluginCommandError::Incomplete {
saved_accounts: 1,
reason: "超过命令总期限",
})
),
"后续账号提交不能重新获得完整预算"
);
commands.shutdown().await;
}
#[tokio::test]
async fn model_requests_have_a_body_limit_independent_of_the_response_buffer() {
use gateway_plugin_sdk::call::model::request_payload_limit;
@@ -174,8 +174,7 @@ async fn failed_or_cancelled_commands_are_not_replayed_and_children_are_reaped()
} else {
// 正常完成注册并确认命令已进入子进程后,只推进执行期限
tokio::time::pause();
tokio::time::advance(limits.maximum_call_timeout + Duration::from_millis(1))
.await;
tokio::time::advance(Duration::from_secs(120) + Duration::from_millis(1)).await;
}
}
};
@@ -209,3 +208,46 @@ async fn failed_or_cancelled_commands_are_not_replayed_and_children_are_reaped()
}
}
}
#[tokio::test]
async fn command_declared_budget_survives_the_default_and_expires_at_its_own_deadline() {
use std::time::Duration;
let mut config = configuration(json!([]));
config["command_wait"] = json!(true);
config["call_budgets_ms"] = json!({"command_line":180_000});
let (directory, store, runtime) = setup(config).await;
let marker = directory.path().join("command-budget.jsonl");
store.snapshot.lock().unwrap().instances[0].configuration["command_marker"] = json!(marker);
let commands = runtime.prepare_command_line().await.unwrap();
let task = tokio::spawn(async move {
let result = commands.execute("commands", "inspect", &[]).await;
(commands, result)
});
tokio::time::timeout(Duration::from_secs(5), async {
while !marker.exists() {
tokio::time::sleep(Duration::from_millis(5)).await;
}
})
.await
.unwrap();
tokio::time::pause();
tokio::time::advance(Duration::from_secs(121)).await;
tokio::task::yield_now().await;
assert!(!task.is_finished(), "声明的命令预算不受默认 120 秒截断");
tokio::time::advance(Duration::from_secs(60)).await;
let (commands, result) = task.await.unwrap();
tokio::time::resume();
let error = result.err().unwrap();
assert!(matches!(
error,
gateway_plugin_runtime::PluginCommandError::Incomplete {
saved_accounts: 0,
..
}
));
assert!(error.to_string().contains("超时") || error.to_string().contains("总期限"));
commands.shutdown().await;
assert_eq!(std::fs::read_to_string(marker).unwrap().lines().count(), 1);
}
@@ -189,7 +189,6 @@ async fn facts_are_readable_without_replacing_or_duplicating_the_original_execut
.with_status_code(200)
.with_request_id(OpaqueUpstreamValue::new("response-request-id"))
.with_service_tier_if_valid("priority")
.with_upstream_response_model_if_valid("returned-model")
.with_client_headers(vec![ProviderResponseHeader::new(
"set-cookie",
Bytes::from_static(b"fixture=secret"),
@@ -41,6 +41,24 @@ async fn http_plugin_with_policy(
PluginRuntime,
ExtensionSetReference,
gateway_core::engine::middleware::FrozenMiddlewarePlan,
) {
http_plugin_with_configuration(
serde_json::json!({"http":true,"mode":mode}),
limits,
failure_policy,
)
.await
}
async fn http_plugin_with_configuration(
configuration: serde_json::Value,
limits: RpcLimits,
failure_policy: PluginFailurePolicy,
) -> (
tempfile::TempDir,
PluginRuntime,
ExtensionSetReference,
gateway_core::engine::middleware::FrozenMiddlewarePlan,
) {
let worker = std::fs::read(env!("CARGO_BIN_EXE_gateway-plugin-test-middleware")).unwrap();
let package = crate::support::package_with_contributions(
@@ -55,7 +73,7 @@ async fn http_plugin_with_policy(
let (cache, runtime) = setup_package_with_limits(
vec![InstanceFixture {
id: "http-plugin",
configuration: serde_json::json!({"http":true,"mode":mode}),
configuration,
bindings: vec![binding(MIDDLEWARE_CONTRIBUTION, "http", 0, failure_policy)],
}],
package,
@@ -64,14 +64,11 @@ async fn session_model_and_network_callbacks_outlive_handshake_and_each_executio
let key = format!("key_{}", uuid::Uuid::new_v4().simple());
environment.client_key(&key, "sk-session-fixture").await;
environment
.install_plugin(serde_json::json!({"http":true,"mode":"callbacks"}))
.await;
let (runtime, bundle) = environment
.runtime_with_limits(RpcLimits {
maximum_call_timeout: Duration::from_millis(500),
..RpcLimits::default()
})
.install_plugin(
serde_json::json!({"http":true,"mode":"callbacks", "call_budgets_ms":{"http":500}}),
)
.await;
let (runtime, bundle) = environment.runtime_with_limits(RpcLimits::default()).await;
environment.store.start_command_line_writes().unwrap();
let snapshot = bundle.snapshots().snapshot_for_diagnostics().unwrap();
let plan = runtime
@@ -185,12 +182,10 @@ async fn session_model_and_network_callbacks_outlive_handshake_and_each_executio
#[tokio::test]
async fn real_plugin_controls_upgrade_and_concurrent_duplex_with_large_binary_frames() {
let (_cache, runtime, generation, plan) = http_plugin_with_limits(
"upgrade",
RpcLimits {
maximum_call_timeout: Duration::from_millis(500),
..RpcLimits::default()
},
let (_cache, runtime, generation, plan) = http_plugin_with_configuration(
serde_json::json!({"http":true,"mode":"upgrade", "call_budgets_ms":{"http":500}}),
RpcLimits::default(),
PluginFailurePolicy::Reject,
)
.await;
let (send, incoming) = tokio::sync::mpsc::channel(1);
@@ -147,7 +147,8 @@ async fn sdk_budget_runtime(stage: Stage, count: usize) -> (tempfile::TempDir, P
.enumerate()
.map(|(order, id)| InstanceFixture {
id,
configuration: serde_json::json!({"mode":"map", "attempt":stage == Stage::Attempt}),
configuration: serde_json::json!({"mode":"map", "attempt":stage == Stage::Attempt,
"call_budgets_ms":{"request":300,"attempt":300}}),
bindings: vec![binding(
MIDDLEWARE_CONTRIBUTION,
mount,
@@ -156,15 +157,7 @@ async fn sdk_budget_runtime(stage: Stage, count: usize) -> (tempfile::TempDir, P
)],
})
.collect();
setup_package_with_limits(
instances,
package,
RpcLimits {
maximum_call_timeout: Duration::from_millis(300),
..RpcLimits::default()
},
)
.await
setup_package_with_limits(instances, package, RpcLimits::default()).await
}
fn budget_request() -> MiddlewareRequest {
@@ -5,6 +5,7 @@ mod facts;
mod http;
mod lifecycle;
mod mounts;
mod outbound;
mod retry;
mod scheduling;
mod service;
@@ -368,54 +369,85 @@ async fn retry_policy_preserves_decisions_and_leaves_replay_safety_to_core() {
}
#[tokio::test]
async fn retry_policy_timeout_exhausts_the_chain_budget_and_delegates() {
async fn retry_policy_combines_instance_budgets_with_one_request_bounded_chain() {
use gateway_core::{
engine::policy::{RetryDecision, RetryFacts},
error::ProviderErrorKind,
upstream::UpstreamSendState,
};
let records = tempfile::tempdir().unwrap();
let second_marker = records.path().join("second.jsonl");
let (_cache, runtime) = setup(
vec![
for (
first_budget,
first_delay,
second_budget,
second_delay,
request_budget,
expected,
reached_second,
) in [
(100, 500, 1000, 100, 2000, RetryDecision::Stop, true),
(1000, 100, 50, 300, 2000, RetryDecision::Delegate, true),
(1000, 300, 1000, 800, 2000, RetryDecision::Delegate, true),
(1000, 500, 1000, 0, 100, RetryDecision::Delegate, false),
(3000, 2100, 3000, 0, 5000, RetryDecision::Stop, true),
] {
let records = tempfile::tempdir().unwrap();
let second_marker = records.path().join("second.jsonl");
let (_cache, runtime) = setup(vec![
InstanceFixture {
id: "retry-delayed",
configuration: serde_json::json!({"retry_delay_ms":500,"retry_decision":{"decision":"stop"}}),
bindings: vec![binding("test.example.retryPolicy", "retry", 0, PluginFailurePolicy::Delegate)],
configuration: serde_json::json!({
"call_budgets_ms":{"retry":first_budget},
"retry_delay_ms":first_delay,
"retry_decision":{"decision":"delegate"},
}),
bindings: vec![binding(
"test.example.retryPolicy",
"retry",
0,
PluginFailurePolicy::Delegate,
)],
},
InstanceFixture {
id: "retry-next",
configuration: serde_json::json!({"retry_marker":second_marker,"retry_decision":{"decision":"stop"}}),
bindings: vec![binding("test.example.retryPolicy", "retry", 1, PluginFailurePolicy::Delegate)],
configuration: serde_json::json!({
"call_budgets_ms":{"retry":second_budget},
"retry_delay_ms":second_delay,
"retry_marker":second_marker,
"retry_decision":{"decision":"stop"},
}),
bindings: vec![binding(
"test.example.retryPolicy",
"retry",
1,
PluginFailurePolicy::Delegate,
)],
},
],
).await;
let context = policy_context(&runtime, prepare(&runtime).await, "req_retry_timeout");
let decision = tokio::time::timeout(
Duration::from_secs(2),
context.retry_decision(RetryFacts {
attempt_index: NonZeroU32::MIN,
provider: ProviderKind::new("openai").unwrap(),
model: Some("gpt-test".into()),
error_kind: ProviderErrorKind::RateLimited,
upstream_status: Some(429),
send_state: UpstreamSendState::NotSent,
remaining_routing_attempts: 1,
remaining_deadline: Duration::from_millis(100),
retry_allowed: true,
}),
)
.await
.expect("the shared deadline bounds the whole retry chain");
assert_eq!(decision, RetryDecision::Delegate);
assert!(
!second_marker.exists(),
"later policies cannot restart an exhausted budget"
);
runtime.shutdown().await;
])
.await;
let context = policy_context(&runtime, prepare(&runtime).await, "req_retry_timeout");
let decision = tokio::time::timeout(
Duration::from_secs(6),
context.retry_decision(RetryFacts {
attempt_index: NonZeroU32::MIN,
provider: ProviderKind::new("openai").unwrap(),
model: Some("gpt-test".into()),
error_kind: ProviderErrorKind::RateLimited,
upstream_status: Some(429),
send_state: UpstreamSendState::NotSent,
remaining_routing_attempts: 1,
remaining_deadline: Duration::from_millis(request_budget),
retry_allowed: true,
}),
)
.await
.expect("重试链不能重置总预算");
assert_eq!(
decision, expected,
"{first_budget}/{second_budget}/{request_budget}"
);
assert_eq!(second_marker.exists(), reached_second);
runtime.shutdown().await;
}
}
async fn marker_lines(path: &std::path::Path, count: usize) -> Vec<serde_json::Value> {
@@ -1308,7 +1340,7 @@ async fn real_request_plugins_rewrite_settings_in_onion_order() {
InstanceFixture { id: "settings-first", configuration: serde_json::json!({"mode":"settings","expected_settings":baseline_json,"settings":first}), bindings: vec![binding(MIDDLEWARE_CONTRIBUTION,"request",0,PluginFailurePolicy::Reject)] },
InstanceFixture { id: "settings-second", configuration: serde_json::json!({"mode":"settings","expected_settings":first,"settings":last}), bindings: vec![binding(MIDDLEWARE_CONTRIBUTION,"request",1,PluginFailurePolicy::Reject)] },
InstanceFixture { id: "settings-pass", configuration: serde_json::json!({"mode":"settings","expected_settings":last}), bindings: vec![binding(MIDDLEWARE_CONTRIBUTION,"request",2,PluginFailurePolicy::Reject)] },
], package).await;
], package.clone()).await;
let generation = prepare(&runtime).await;
let plan = runtime
.execution_registry()
@@ -1369,6 +1401,43 @@ async fn real_request_plugins_rewrite_settings_in_onion_order() {
drop(plan);
drop(generation);
super::wait_until_empty(cache.path()).await;
// 根对象 null 也是显式替换,必须由设置合同拒绝,不能悄悄沿用原配置
let (cache, runtime) = setup_package(vec![InstanceFixture {
id: "settings-null",
configuration: serde_json::json!({"mode":"settings", "expected_settings":baseline_json, "settings":null}),
bindings: vec![binding(MIDDLEWARE_CONTRIBUTION, "request", 0, PluginFailurePolicy::Reject)],
}], package).await;
let generation = prepare(&runtime).await;
let plan = runtime
.execution_registry()
.middleware(&generation)
.unwrap();
let calls = Arc::new(AtomicUsize::new(0));
assert!(
plan.handle(
middleware_context(ClientTransport::HttpJson),
MiddlewareRequest::new(
"openai",
middleware_headers(),
Bytes::from_static(br#"{"input":"hello"}"#)
)
.with_settings(configuration),
(Downstream {
calls: Arc::clone(&calls),
closes: Arc::default(),
reads: Arc::default(),
error: false
})
.into_next(),
)
.await
.is_err()
);
assert_eq!(calls.load(Ordering::SeqCst), 0);
drop(plan);
drop(generation);
super::wait_until_empty(cache.path()).await;
}
#[tokio::test]
@@ -1593,7 +1662,7 @@ impl CapabilitiesNext {
#[tokio::test]
async fn sdk_capability_declarations_require_request_stage() {
for (version, attempt) in [(5, false), (5, true)] {
for (version, attempt) in [(6, false), (6, true)] {
let stage = if attempt {
Stage::Attempt
} else {
@@ -0,0 +1,344 @@
//! 验证真实插件出站准备的组合、资源预算和终止边界
use super::*;
use gateway_core::engine::middleware::outbound::{Next, Request, Target};
async fn runtime(
instances: Vec<InstanceFixture>,
limits: RpcLimits,
) -> (tempfile::TempDir, PluginRuntime) {
let worker = std::fs::read(env!("CARGO_BIN_EXE_gateway-plugin-test-middleware")).unwrap();
let package = crate::support::package_with_contributions(
&worker,
Contributions::from([crate::support::contribution(
Capability::Middleware,
vec![Stage::Outbound],
vec!["outbound".into()],
vec!["outbound".into()],
)]),
);
setup_package_with_limits(instances, package, limits).await
}
fn instance(id: &'static str, order: i32, mut configuration: serde_json::Value) -> InstanceFixture {
configuration["outbound"] = true.into();
InstanceFixture {
id,
configuration,
bindings: vec![binding(
MIDDLEWARE_CONTRIBUTION,
"outbound",
order,
PluginFailurePolicy::Reject,
)],
}
}
fn context() -> MiddlewareContext {
context_for_provider("openai")
}
fn context_for_provider(provider: &str) -> MiddlewareContext {
MiddlewareContext::new(
MiddlewareTarget {
request_id: ModelRequestId::new("req_outbound").unwrap(),
mount: MiddlewareMount::Outbound,
attempt_index: Some(NonZeroU32::MIN),
operation: Some(OperationKind::Generate),
endpoint: "/v1/responses".into(),
transport: ClientTransport::HttpJson,
provider: Some(ProviderKind::new(provider).unwrap()),
model: Some("public-a".into()),
account_id: Some(ProviderAccountId::new("acct_openai").unwrap()),
},
MiddlewareAuthority {
client_key_id: ClientApiKeyId::new("key-middleware").unwrap(),
account_group_ids: Arc::from([AccountGroupId::new(
"grp_11111111111111111111111111111111",
)
.unwrap()]),
cancellation: CancellationToken::new(),
deadline: (SystemTime::now() + Duration::from_secs(5)).into(),
extension_scope: ExtensionCallScope::default(),
execution_effects: Some(Arc::default()),
},
)
}
fn request() -> Request {
Request {
target: Target::Http {
method: ::http::Method::POST,
uri: "https://example.test/original".parse().unwrap(),
},
headers: vec![MiddlewareHeader::new(
"x-delete",
Bytes::from_static(b"original"),
)],
body: Bytes::from_static(&[0, 255]),
}
}
fn terminal() -> Next {
gateway_core::middleware::compose(Vec::new(), |request| Box::pin(async { Ok(request) }))
}
#[tokio::test]
async fn outbound_plugins_compose_http_and_websocket_with_full_replacement() {
for websocket in [false, true] {
let records = tempfile::NamedTempFile::new().unwrap();
let initial_body = if websocket {
b"msg".to_vec()
} else {
vec![0, 255]
};
let target = if websocket {
serde_json::json!({"kind":"websocket", "uri":"wss://rewritten.test/socket"})
} else {
serde_json::json!({"kind":"http", "method":"PUT", "uri":"https://rewritten.test/path?opaque=1"})
};
let mut outer = instance(
"outer",
0,
serde_json::json!({
"records":records.path(), "enter_byte":65, "exit_byte":68,
"expected_body":initial_body, "target":target,
"headers":[{"name":"x-multi","value":[97]}, {"name":"x-multi","value":[255]}],
"second_next":true,
}),
);
outer.bindings[0].client_key_ids = vec!["key-middleware".into()];
outer.bindings[0].account_group_ids = vec!["grp_11111111111111111111111111111111".into()];
outer.bindings[0].provider_ids = vec!["openai".into()];
outer.bindings[0].models = vec!["public-a".into()];
let mut inner_body = initial_body.clone();
inner_body.push(65);
let mut inner = instance(
"inner",
1,
serde_json::json!({
"records":records.path(), "enter_byte":66, "exit_byte":67, "expected_body":inner_body,
}),
);
inner.bindings[0].provider_ids = vec!["openai".into()];
let mut excluded = instance("excluded", 2, serde_json::json!({"fault":true}));
excluded.bindings[0].provider_ids = vec!["xai".into()];
let (cache, runtime) = runtime(vec![outer, inner, excluded], RpcLimits::default()).await;
let generation = prepare(&runtime).await;
let plan = runtime
.execution_registry()
.middleware(&generation)
.unwrap();
assert!(plan.has_outbound(&context()));
assert!(!plan.has_outbound(&context_for_provider("unmatched")));
let mut input = request();
input.body = initial_body.clone().into();
if websocket {
input.target = Target::WebSocket {
uri: "wss://example.test/socket".parse().unwrap(),
};
}
let result = plan
.handle_outbound(context(), input, terminal())
.await
.unwrap();
let mut expected_body = initial_body;
expected_body.extend([65, 66, 67, 68]);
assert_eq!(result.body.as_ref(), expected_body);
let headers = result.header_map().unwrap();
assert!(!headers.contains_key("x-delete"));
assert_eq!(
headers
.get_all("x-multi")
.iter()
.map(|value| value.as_bytes())
.collect::<Vec<_>>(),
[b"a".as_slice(), &[255]]
);
match result.target {
Target::Http { method, uri } => {
assert_eq!(method, ::http::Method::PUT);
assert_eq!(uri.to_string(), "https://rewritten.test/path?opaque=1");
}
Target::WebSocket { uri } => assert_eq!(uri.to_string(), "wss://rewritten.test/socket"),
}
assert_eq!(
std::fs::read_to_string(records.path()).unwrap(),
"enter:outer\nenter:inner\nexit:inner\nexit:outer\n"
);
drop(plan);
drop(generation);
super::super::wait_until_empty(cache.path()).await;
}
}
#[tokio::test]
async fn outbound_plugin_can_return_directly_without_running_next() {
let (cache, runtime) = runtime(
vec![instance(
"direct",
0,
serde_json::json!({"direct":true,"exit_byte":42}),
)],
RpcLimits::default(),
)
.await;
let generation = prepare(&runtime).await;
let plan = runtime
.execution_registry()
.middleware(&generation)
.unwrap();
let result = plan
.handle_outbound(
context(),
request(),
gateway_core::middleware::compose(Vec::new(), |_| {
panic!("direct preparation must not consume next")
}),
)
.await
.unwrap();
assert_eq!(result.body.as_ref(), [0, 255, 42]);
drop(plan);
drop(generation);
super::super::wait_until_empty(cache.path()).await;
}
#[tokio::test]
async fn outbound_http_callbacks_use_the_frozen_body_budget_and_propagate_failure() {
for body_len in [3, 4] {
let server = MockServer::start().await;
Mock::given(any())
.respond_with(ResponseTemplate::new(204))
.expect(if body_len == 3 { 1 } else { 0 })
.mount(&server)
.await;
let mut entry = instance(
"callback",
0,
serde_json::json!({"http_url":server.uri(),"http_body_len":body_len,"fault":true}),
);
entry.bindings[0].failure_policy = PluginFailurePolicy::Delegate;
let (cache, runtime) = runtime(vec![entry], RpcLimits::default()).await;
let generation = prepare(&runtime).await;
let plan = runtime
.execution_registry()
.middleware(&generation)
.unwrap();
let result = plan
.handle_outbound(
context().with_request_body_limit(Some(3)),
request(),
gateway_core::middleware::compose(Vec::new(), |_| {
panic!("plugin failure must not silently replay preparation")
}),
)
.await;
assert!(result.is_err());
server.verify().await;
drop(plan);
drop(generation);
super::super::wait_until_empty(cache.path()).await;
}
}
#[tokio::test]
async fn outbound_invalid_method_is_not_repaired_or_delegated() {
let mut entry = instance(
"invalid",
0,
serde_json::json!({"direct":true,"target":{"kind":"http","method":"BAD METHOD","uri":"https://example.test/"}}),
);
entry.bindings[0].failure_policy = PluginFailurePolicy::Delegate;
let (cache, runtime) = runtime(vec![entry], RpcLimits::default()).await;
let generation = prepare(&runtime).await;
let plan = runtime
.execution_registry()
.middleware(&generation)
.unwrap();
let result = plan.handle_outbound(context(), request(), terminal()).await;
assert!(matches!(result, Err(MiddlewareError::InvalidState)));
drop(plan);
drop(generation);
super::super::wait_until_empty(cache.path()).await;
}
#[tokio::test]
async fn outbound_next_wait_pauses_plugin_budget_and_cancel_releases_the_continuation() {
for cancel in [false, true] {
let (cache, runtime) = runtime(
vec![
instance(
"outer",
0,
serde_json::json!({"call_budgets_ms":{"outbound":300}}),
),
instance(
"inner",
1,
serde_json::json!({"call_budgets_ms":{"outbound":300}}),
),
],
RpcLimits::default(),
)
.await;
let generation = prepare(&runtime).await;
let plan = runtime
.execution_registry()
.middleware(&generation)
.unwrap();
let context = context();
let cancellation = context.cancellation().clone();
let (started, entered) = tokio::sync::oneshot::channel();
let (closed, dropped) = tokio::sync::oneshot::channel();
let pending = tokio::spawn(async move {
plan.handle_outbound(
context,
request(),
gateway_core::middleware::compose(Vec::new(), move |request| {
Box::pin(async move {
let _closed = NotifyOnDrop(Some(closed));
started.send(()).unwrap();
if cancel {
std::future::pending::<()>().await;
} else {
tokio::time::sleep(Duration::from_millis(900)).await;
}
Ok(request)
})
}),
)
.await
});
tokio::time::timeout(Duration::from_secs(2), entered)
.await
.unwrap()
.unwrap();
tokio::time::sleep(Duration::from_millis(600)).await;
assert!(!pending.is_finished());
if cancel {
cancellation.cancel();
}
let result = tokio::time::timeout(Duration::from_secs(2), pending)
.await
.unwrap()
.unwrap();
assert_eq!(result.is_err(), cancel);
tokio::time::timeout(Duration::from_secs(2), dropped)
.await
.unwrap()
.unwrap();
drop(generation);
super::super::wait_until_empty(cache.path()).await;
}
}
struct NotifyOnDrop(Option<tokio::sync::oneshot::Sender<()>>);
impl Drop for NotifyOnDrop {
fn drop(&mut self) {
if let Some(sender) = self.0.take() {
let _ = sender.send(());
}
}
}
@@ -136,6 +136,7 @@ async fn retry_callback_effects_survive_success_failure_and_cancellation() {
"retry_http_return_marker":(outcome == "early_result").then_some(marker),
"retry_service_request":is_service.then_some(service_request),
"retry_service_marker":service_marker,
"call_budgets_ms":{"retry":500},
}))
.await;
let key = "sk-retry-effects-fixture";
@@ -146,10 +147,7 @@ async fn retry_callback_effects_survive_success_failure_and_cancellation() {
});
let (runtime, core) = environment
.runtime_with_providers(
RpcLimits {
maximum_call_timeout: Duration::from_millis(500),
..RpcLimits::default()
},
RpcLimits::default(),
ProviderRegistry::new([provider.clone() as Arc<dyn Provider>]).unwrap(),
)
.await;
@@ -552,3 +552,55 @@ async fn a_host_version_mismatch_still_restores_the_enabled_plugin() {
runtime.shutdown().await;
super::super::wait_until_empty(cache.path()).await;
}
#[tokio::test]
async fn declared_registration_budget_is_not_cut_short_by_process_preparation_timeout() {
let (cache, store, runtime) = super::super::setup().await;
let mut snapshot = store.snapshot.lock().unwrap().clone();
snapshot.instances[0].configuration = serde_json::json!({
"registration_delay_ms":31_000,
"call_budgets_ms":{"registration":40_000},
});
let candidate = tokio::time::timeout(
Duration::from_secs(45),
PluginPreparation::prepare(&runtime, snapshot),
)
.await
.expect("registration stays bounded by its declared budget")
.expect("ready process may register beyond the startup control deadline");
assert!(candidate.is_ready());
drop(candidate);
super::super::wait_until_empty(cache.path()).await;
}
#[tokio::test]
async fn shutdown_cancels_long_registration_and_reclaims_the_unpublished_candidate() {
let (cache, store, runtime) = super::super::setup().await;
let runtime = std::sync::Arc::new(runtime);
let markers = tempfile::tempdir().unwrap();
let started = markers.path().join("registration");
let mut snapshot = store.snapshot.lock().unwrap().clone();
snapshot.instances[0].configuration = serde_json::json!({
"registration_started_marker":started,
"registration_delay_ms":600_000,
"call_budgets_ms":{"registration":600_000},
});
let preparing = tokio::spawn({
let runtime = std::sync::Arc::clone(&runtime);
async move { PluginPreparation::prepare(runtime.as_ref(), snapshot).await }
});
tokio::time::timeout(Duration::from_secs(5), async {
while !started.exists() {
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await
.unwrap();
tokio::time::timeout(Duration::from_secs(3), runtime.shutdown())
.await
.expect(
"shutdown must cancel candidate preparation instead of waiting for its long budget",
);
assert!(preparing.await.unwrap().is_err());
super::super::wait_until_empty(cache.path()).await;
}
@@ -0,0 +1,157 @@
//! 验证维护任务跨发布保活、通知合并及实例替换后的取消顺序
use gateway_admin::{
model::{Revision, plugins::instances::PluginInstanceSnapshot},
ports::plugins::PluginPreparation,
};
use gateway_core::{
lifecycle::CancellationToken,
routing::{ConfigRevision, RuntimeSnapshot},
runtime::RuntimeSnapshotHandle,
task::{WorkerContribution, WorkerDaemonContext, WorkerRunnable},
};
use gateway_plugin_runtime::PluginRuntime;
use gateway_plugin_sdk::{Capability, Contributions, Stage};
use serde_json::{Value, json};
use std::{path::Path, sync::Arc, time::Duration};
async fn publish(
runtime: &PluginRuntime,
snapshots: &RuntimeSnapshotHandle,
instances: PluginInstanceSnapshot,
) {
let revision = ConfigRevision::new(instances.config_revision.get()).unwrap();
let prepared = PluginPreparation::prepare(runtime, instances)
.await
.unwrap();
snapshots.publish(
RuntimeSnapshot::new(
revision,
gateway_core::settings::SettingsValues::new(1, "smart", Default::default(), None, None),
vec![],
vec![],
vec![],
)
.unwrap()
.with_extensions(Some(prepared)),
);
}
fn records(path: &Path) -> Vec<Value> {
std::fs::read_to_string(path)
.unwrap_or_default()
.lines()
.filter_map(|line| serde_json::from_str(line).ok())
.collect()
}
async fn wait_records(path: &Path, count: usize) -> Vec<Value> {
tokio::time::timeout(Duration::from_secs(5), async {
loop {
let entries = records(path);
if entries.len() >= count {
return entries;
}
tokio::time::sleep(Duration::from_millis(10)).await;
}
})
.await
.unwrap_or_else(|error| panic!("missing maintenance records: {error}; {:?}", records(path)))
}
#[tokio::test]
async fn unrelated_publications_preserve_inflight_maintenance_and_replacement_waits_for_cancel() {
let (cache, store, runtime) = super::super::setup_with_contributions(Contributions::from([
crate::support::contribution(
Capability::Maintenance,
vec![Stage::Maintenance],
vec![],
vec![],
),
]))
.await;
let runtime = Arc::new(runtime);
let markers = tempfile::tempdir().unwrap();
let mut instances = store.snapshot.lock().unwrap().clone();
let template = instances.instances[0].clone();
instances.instances = ["a", "b"]
.into_iter()
.map(|id| {
let mut instance = template.clone();
instance.id = id.into();
instance.configuration = json!({
"maintenance_marker":markers.path().join(id),
"execution_cancelled_marker":markers.path().join(id),
"maintenance_wait_marker":markers.path().join(format!("release-{id}")),
"call_context_marker":markers.path().join(format!("context-{id}")),
"call_budgets_ms":{"maintenance":600_000},
});
instance
})
.collect();
let snapshots = RuntimeSnapshotHandle::default();
publish(&runtime, &snapshots, instances.clone()).await;
let WorkerContribution::Registration(registration) =
runtime.maintenance_worker(snapshots.clone()).unwrap()
else {
panic!("maintenance worker registration")
};
let WorkerRunnable::Daemon { task, .. } = registration.runnable else {
panic!("maintenance daemon")
};
let stop = CancellationToken::new();
let running = tokio::spawn({
let stop = stop.clone();
async move { task.run(WorkerDaemonContext::new(stop)).await.unwrap() }
});
let a = markers.path().join("a");
let b = markers.path().join("b");
assert_eq!(wait_records(&a, 1).await[0]["phase"], "start");
assert_eq!(wait_records(&b, 1).await[0]["phase"], "start");
let contexts = records(&markers.path().join("context-a"));
let maintenance = contexts
.iter()
.find(|entry| entry["method"] == "plugin.reconcile")
.expect("maintenance call context");
assert_eq!(maintenance["context"]["timeout_ms"], 600_000);
instances.config_revision = Revision::new(2).unwrap();
instances.instances[1].revision = instances.config_revision;
publish(&runtime, &snapshots, instances.clone()).await;
let changed = wait_records(&b, 3).await;
assert!(
changed[1].get("call_id").is_some(),
"旧维护必须先完成取消确认"
);
assert_eq!(changed[2]["phase"], "start");
assert_eq!(records(&a).len(), 1, "无关实例发布不得重启 A 的在途维护");
instances.config_revision = Revision::new(3).unwrap();
instances.instances[1].revision = instances.config_revision;
instances.instances[1].enabled = false;
publish(&runtime, &snapshots, instances.clone()).await;
assert!(wait_records(&b, 4).await[3].get("call_id").is_some());
instances.config_revision = Revision::new(4).unwrap();
instances.instances[1].revision = instances.config_revision;
instances.instances[1].enabled = true;
publish(&runtime, &snapshots, instances.clone()).await;
assert_eq!(wait_records(&b, 5).await[4]["phase"], "start");
assert_eq!(records(&a).len(), 1, "停用与恢复 B 不得取消 A");
// 多次发布通知合并为一次后续对账,首轮仍使用原调用 ID 完成
std::fs::write(markers.path().join("release-a"), []).unwrap();
let completed = wait_records(&a, 4).await;
assert_eq!(completed[1]["phase"], "done");
assert_eq!(completed[1]["id"], completed[0]["id"]);
assert_eq!(completed[2]["phase"], "start");
assert_eq!(completed[3]["phase"], "done");
assert_ne!(completed[2]["id"], completed[0]["id"]);
assert_eq!(completed.len(), 4);
stop.cancel();
running.await.unwrap();
assert!(wait_records(&b, 6).await[5].get("call_id").is_some());
snapshots.suspend();
runtime.shutdown().await;
super::super::wait_until_empty(cache.path()).await;
}

Some files were not shown because too many files have changed in this diff Show More