mirror of
https://github.com/zyycn/codex-proxy-rs.git
synced 2026-11-07 16:05:27 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a36137bafb | ||
|
|
2e7a39e1e8 | ||
|
|
6648198df4 | ||
|
|
2eed9b20ed | ||
|
|
16aa08091c | ||
|
|
abae4a631b | ||
|
|
8983eb4ce9 | ||
|
|
cd605b850e | ||
|
|
7beebb2340 | ||
|
|
31de0bd83c | ||
|
|
5cfc6dee10 | ||
|
|
ef5b4d7dbd | ||
|
|
31974bc289 | ||
|
|
e033e21994 | ||
|
|
2818edb9fc | ||
|
|
1d1f34ba7e |
@@ -1,28 +1,32 @@
|
||||
---
|
||||
name: cpr-dev-guide
|
||||
description: Codex Proxy RS 前后端开发、排障、代码审查和文档修改入口;改页面、弹窗、样式、文案,修接口、调度、数据库、Rust、宿主 SDK 或 Runtime 时均使用;PR、Issue、独立网关插件与发版另配对应技能
|
||||
description: Codex Proxy RS 开发、修复、排障、代码审查和文档修改入口;页面、样式、文案、接口、调度、数据库及宿主 SDK、Runtime、插件接入均适用;PR、Issue、独立网关插件与发版叠加对应技能
|
||||
---
|
||||
|
||||
# 开发指南
|
||||
|
||||
## 先匹配任务与文件
|
||||
## 先确定领域与必读规范
|
||||
|
||||
按用户要改的行为初判,定位后按实际修改或审查的文件复核,不要求用户说出“前端”“后端”或技能名。**命中多行就同时执行,先读取对应规范再修改或给出审查结论**;任务范围变化时补齐新命中项,已有上下文直接复用
|
||||
按用户意图、受影响行为与对象初判,定位后按实际文件复核,不要求用户说出“前端”“后端”或技能名。**所有命中行共同构成本次必读规范,先读完再确定方案、修改或给出审查结论**;跨领域叠加,不能只选最像当前步骤的一行
|
||||
|
||||
| 命中条件,操作或文件满足其一即可 | 必须执行 |
|
||||
职责边界决定问题由谁修复;即使代码位于账号、调度或 API 模块,只要影响插件接入或为插件增加处理,也命中插件边界。确定方案时说明相关业务事实的归属及已有实现的复用依据,不能以功能恢复或测试通过替代职责判断
|
||||
|
||||
| 领域或操作,行为影响与文件路径满足其一即可 | 确定方案前读取与执行 |
|
||||
| --- | --- |
|
||||
| 调整页面、弹窗、表单、按钮、文案、样式或交互;目标在 `frontend/`、`modules/ui/`、插件前端,或是 `.vue` 文件 | [前端流程](references/frontend.md),覆盖组件复用、文案、Token、样式、视觉状态和运行证据;局部修改同样执行 |
|
||||
| 修改服务端接口、路由、调度、账号与用量规则、持久化、后台任务或协议行为;目标是 `backend/` 中的实现、配置、迁移,或 Rust 源码 | [Workspace 边界](../../../docs/architecture.md#3-workspace-边界)、所属模块、[后端自审](../../../docs/architecture.md#后端自审);涉及 Rust 时使用 `$rust-best-practices` 和下方后端注释要求,验证时读[命令](../../../docs/architecture.md#验证命令) |
|
||||
| 开发、排查或审查插件接入、宿主 SDK、Runtime、扩展接口或独立网关插件;目标在 `backend/crates/gateway-plugin/`、`modules/plugins/` | 完整读取[插件职责边界](references/plugin-boundaries.md)及所用 [SDK](../../../backend/crates/gateway-plugin/sdk/README.md) 类型与能力合同;宿主维护自身合同、资源与稳定性,插件业务错误在插件侧修复,不在宿主、SDK 或 Runtime 中补偿;独立插件开发、排查、审查另用 [cpr-plugin-dev](../cpr-plugin-dev/SKILL.md) |
|
||||
| 开发、排查或审查服务端接口、路由、调度、账号与用量规则、持久化、后台任务或协议行为;目标是 `backend/` 中的实现、配置、迁移,或 Rust 源码 | [Workspace 边界](../../../docs/architecture.md#3-workspace-边界)的模块与依赖规则、所属模块及[后端自审](../../../docs/architecture.md#后端自审);Rust 使用 `$rust-best-practices` 和下方后端注释要求,宿主 Rust 同时读取 [Rust 模块组织约定](../../../docs/architecture.md#32-rust-模块组织约定) |
|
||||
| 修改页面接口调用,或开发、排查、审查面向管理端或模型客户端的 HTTP/WS API 合同 | [API](../../../docs/api.md)对应章节;优先复用已有接口,核对调用方 → 服务端解析 → 实际查询或执行,不能只凭前端类型认定参数生效;按实际改动叠加前后端规范 |
|
||||
| 模块或调用链变更 | [Workspace 边界](../../../docs/architecture.md#3-workspace-边界)及所属模块章节,不通读整份架构 |
|
||||
| 新增、调整、清理或审查测试 | 下方[测试维护](#测试维护),按当前行为、职责归属与已有覆盖判断保留、合并或删除 |
|
||||
| 修改部署行为、`deploy/` 配置或数据库迁移 | 按范围读取[部署](../../../deploy/README.md)、[迁移](../../../backend/migrations/README.md)对应章节 |
|
||||
| 开发、排查或审查宿主 SDK、Runtime、扩展接口或独立插件;目标在 `backend/crates/gateway-plugin/` | [插件职责边界](references/plugin-boundaries.md)及所用 [SDK](../../../backend/crates/gateway-plugin/sdk/README.md) 类型与能力合同;宿主守住自身合同、资源与稳定性,不猜测或补偿插件业务错误;仅独立插件创作、排查另用 [cpr-plugin-dev](../cpr-plugin-dev/SKILL.md) |
|
||||
| 修改或审查文档 | [文档检查](references/documentation.md)及目标章节,不默认读取代码开发、页面验收或发布流程 |
|
||||
| 普通代码审查 | [审查标准](../../../CONTRIBUTING.md#审查标准);审查默认只读 |
|
||||
| 代码审查,包括 PR 审查 | [审查标准](../../../CONTRIBUTING.md#审查标准),叠加被审查行为所属领域的规范;审查默认只读 |
|
||||
| PR / Issue / 发版 | 按当前意图选 [PR](../cpr-github-pr/SKILL.md)、[Issue](../cpr-github-issue/SKILL.md)或[发版](../cpr-release/SKILL.md),普通提交不创建 PR |
|
||||
|
||||
纯文档任务按文档分支执行,不因文档所在目录或提及的路径、语言、功能加载开发流程。每次补读只解决当前未知项;文档链接带 `#章节` 不代表工具自动截取,先用 `rg -n '^#{1,4} ' 文件` 找边界,只读命中章节,不预读整份 CONTRIBUTING、架构、API、主题或全部 references,全文审查除外
|
||||
纯文档任务按文档分支执行;核实文中的合同与职责时读对应领域条款,不因提及代码路径或语言加载实现、构建与页面验收流程
|
||||
|
||||
命中的流程引用完整读取,并落实其中指定的必读章节;长文档先用 `rg -n '^#{1,4} ' 文件` 定位,读取指定章节的完整内容。`#章节` 不会自动截取,“按需”仅用于未命中的领域和后续操作资料,不能省略必读规范;已有内容只在当前上下文可用且仍适用时复用,不循环重读技能
|
||||
|
||||
## 参考仓库的优先级
|
||||
|
||||
@@ -31,9 +35,9 @@ description: Codex Proxy RS 前后端开发、排障、代码审查和文档修
|
||||
## 执行与收尾
|
||||
|
||||
1. 遵守已提供的 [AGENTS.md](../../../AGENTS.md),按其 CodeGraph 约定定位代码;确认工作区、任务范围与已有改动,找到事实归属、同类实现和调用链,补齐上表命中规范及[项目约定](../../../CONTRIBUTING.md#项目约定)对应条目后再修改
|
||||
2. 自审完整差异,处理本次引入的职责越界、重复规则、冗余状态与无依据分支;删除或替换功能时一并清理失效的配置、类型、调用、测试与文档,已部署数据和公开合同仍遵守所属迁移及兼容规则;仅审查时报告,不自行修复
|
||||
2. 以最终差异重新匹配上表,补齐新增领域的规范并检查实际实现,再处理本次引入的职责越界、重复规则、冗余状态与无依据分支;删除或替换功能时一并清理失效的配置、类型、调用、测试与文档,已部署数据和公开合同仍遵守所属迁移及兼容规则;仅审查时报告,不自行修复
|
||||
3. 常规文档只写当前状态;有文档差异时执行[文档检查](references/documentation.md),没有时只确认现有说明是否失真,普通开发不自动修改 `release/notes.md`
|
||||
4. 按变更范围执行[验证](../../../CONTRIBUTING.md#验证)及其停止条件,沿用本轮有效证据与用户已确定的验收分工,区分通过、失败、跳过与未执行;提交时补读[提交约定](../../../CONTRIBUTING.md#项目约定)
|
||||
4. 按变更范围执行[验证](../../../CONTRIBUTING.md#验证)及其停止条件,后端命令见[验证命令](../../../docs/architecture.md#验证命令);沿用本轮有效证据与用户已确定的验收分工,区分通过、失败、跳过与未执行;提交时补读[提交约定](../../../CONTRIBUTING.md#项目约定)
|
||||
5. 交付实际结果与验证缺口;前端按[前端流程](references/frontend.md)核对最终差异和运行证据,后端按上表检查生命周期,不把构建通过当作行为验收
|
||||
|
||||
## 测试维护
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
interface:
|
||||
display_name: "cpr-dev-guide"
|
||||
short_description: "前后端开发、排障与审查入口,按任务和文件匹配规范并验证"
|
||||
default_prompt: "使用 $cpr-dev-guide 按请求和实际文件匹配前后端规范,读取命中项后完成已授权的开发或审查,并自审、验证和报告结果。"
|
||||
short_description: "开发、排障与审查入口,按行为影响匹配领域边界并验证实现"
|
||||
default_prompt: "使用 $cpr-dev-guide 按请求、行为影响和实际文件匹配领域必读规范,读完职责边界再确定方案,完成已授权的开发或审查、自审与验证。"
|
||||
policy:
|
||||
allow_implicit_invocation: true
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# 文档检查
|
||||
|
||||
修改或审查文档时读取,职责与行文统一按 [文档职责与更新条件](../../../../CONTRIBUTING.md#文档职责与更新条件),只读该节
|
||||
修改或审查文档时完整读取本流程,并读取 [文档职责与更新条件](../../../../CONTRIBUTING.md#文档职责与更新条件)的完整章节,覆盖内容归属与层次行文
|
||||
|
||||
## 写之前
|
||||
|
||||
先确定读者、必须说明的当前事实与所属章节,再决定是否需要修改及详略;文档职责与更新条件按上节核对
|
||||
先确定读者、必须说明的当前事实与所属章节,再决定是否需要修改及详略;文档职责与更新条件按上节核对。涉及合同或职责说明时,补读所属架构、API 或 SDK 的对应条款,核实事实;不因文档提及实现而执行无关的开发与构建流程
|
||||
|
||||
## 逐段验收
|
||||
|
||||
|
||||
@@ -2,9 +2,11 @@
|
||||
|
||||
适用于页面、组件、弹窗、表单及局部文案和样式修改,不以是否创建 PR 为触发条件;仅审查时核对和报告,不自行修改
|
||||
|
||||
## 开始前
|
||||
## 领域必读
|
||||
|
||||
读取[前端职责](../../../../docs/architecture.md#34-前端模块职责)、[文案与信息层级](../../../../docs/theme.md#界面文案与信息层级)和[界面验证](../../../../CONTRIBUTING.md#界面验证),明确本次用户操作、原页面、最接近的同类实现及可复用基础组件;按本轮验收分工留存所需截图
|
||||
确定页面方案或审查结论前,读取[前端职责](../../../../docs/architecture.md#34-前端模块职责)、[文案与信息层级](../../../../docs/theme.md#界面文案与信息层级)和[界面验证](../../../../CONTRIBUTING.md#界面验证)的完整章节;只改文案或局部样式也执行。明确本次用户操作、原页面、最接近的同类实现及可复用基础组件,沿用既有设计与请求层,按本轮验收分工留存所需截图
|
||||
|
||||
## 按改动补齐规范
|
||||
|
||||
用户要求看效果或自行验收时,按[源码联调](../../../../docs/development.md#源码联调)提供当前工作区的可访问预览与页面入口,确认热更新和依赖来源;需要模拟数据时用于实际页面并标明,不把静态稿、旧构建或仅有本机文件路径当作可用预览
|
||||
|
||||
@@ -12,7 +14,7 @@
|
||||
|
||||
| 改动 | 规范入口 |
|
||||
| --- | --- |
|
||||
| Vue 逻辑、请求或状态 | [项目约定](../../../../CONTRIBUTING.md#项目约定)的前端条目及上述前端职责 |
|
||||
| Vue 组件、逻辑、请求或状态 | 使用 `$vue-best-practices`,读取[项目约定](../../../../CONTRIBUTING.md#项目约定)的前端条目;请求与接口合同同时执行开发入口的 API 分支 |
|
||||
| 颜色、间距、字体、边框、阴影、局部 CSS 或交互样式 | [通用颜色消费](../../../../docs/theme.md#通用颜色消费)、[样式与命名约定](../../../../docs/theme.md#样式与命名约定),核对无边常态及 Hover、Focus 等视觉状态 |
|
||||
| 主题算法、编辑器或共用组件视觉行为 | 所改主题章节及[扩大验证范围](../../../../docs/theme.md#验证) |
|
||||
| 明确指定设计技能,或建立、重塑视觉方向 | 使用指定技能或 `$frontend-design`,设计取舍仍以项目主题、用户任务和已有页面为依据 |
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# 插件职责边界
|
||||
|
||||
仅在开发或审查 SDK、Runtime、宿主扩展接口及独立网关插件时读取
|
||||
先看[插件扩展](../../../../docs/architecture.md#31-插件扩展)中的职责与执行边界,不通读整份架构
|
||||
开发、排查或审查插件接入、SDK、Runtime、宿主扩展接口及独立网关插件时完整读取,不以文件位于插件目录或正在选择能力为条件
|
||||
确定方案前读取[职责与源码入口](../../../../docs/architecture.md#职责与源码入口)、[发布与执行边界](../../../../docs/architecture.md#发布与执行边界)及[信任与资源边界](../../../../docs/architecture.md#信任与资源边界),明确问题归属、必要宿主保护与插件责任
|
||||
|
||||
**宿主提供可组合能力,守住自身合同、资源与稳定性;插件业务正确性和兼容修复由插件负责**
|
||||
|
||||
|
||||
@@ -13,13 +13,15 @@ description: 草拟、创建、补充或回复 GitHub Issue,包括缺陷、功
|
||||
|
||||
补充已有 Issue 先读原帖和讨论,按授权更新或追加,不重复新建;无法推断的目标或关键事实才向用户补齐
|
||||
|
||||
## 按需取材
|
||||
## 按动作读取与核实
|
||||
|
||||
- 新建 Issue 时,本仓库只读[提交 Issue](../../../CONTRIBUTING.md#提交-issue)及本次匹配的一张表单:[接口问题](../../../.github/ISSUE_TEMPLATE/api-bug-report.yml)、[常规问题](../../../.github/ISSUE_TEMPLATE/general-bug-report.yml)或[功能建议](../../../.github/ISSUE_TEMPLATE/feature-request.yml);协作规则提案可用普通 Issue
|
||||
目标、授权、事实与脱敏要求适用于草拟、提交和回复;按下列动作读取必需规则与证据,再组织正文。症状反馈可保留未知根因,进入技术排查或方案判断时补齐对应规范
|
||||
|
||||
- 新建 Issue 时,读取本仓库的[提交 Issue](../../../CONTRIBUTING.md#提交-issue)及本次匹配的一张表单:[接口问题](../../../.github/ISSUE_TEMPLATE/api-bug-report.yml)、[常规问题](../../../.github/ISSUE_TEMPLATE/general-bug-report.yml)或[功能建议](../../../.github/ISSUE_TEMPLATE/feature-request.yml);协作规则提案可用普通 Issue
|
||||
- 回复时读取原帖、相关讨论与支撑答复的证据,按下方回复口吻组织内容,不套用新建 Issue 的表单
|
||||
- 其他仓库读取它自己的相关规则与实际模板,长文档先定位标题再截取,不预读全部模板或 CONTRIBUTING
|
||||
- 搜索同场景 open/closed Issues、相关 PR 和发布记录,区分主分支修复与已发布修复,不凭标题判断重复;没有新证据时返回已有链接,用户仍要求新建则按其要求处理
|
||||
- 仅当需要额外排查或修改时使用 [cpr-dev-guide](../cpr-dev-guide/SKILL.md),提交症状反馈不以找到根因为前提
|
||||
- 需要技术排查、判断实现方案或修改时执行 [cpr-dev-guide](../cpr-dev-guide/SKILL.md)的领域必读,不能仅凭症状建议跨职责补偿;提交症状反馈不以找到根因为前提
|
||||
- 引用上游行为或提出方案时执行[问题与方案依据](../../../CONTRIBUTING.md#问题与方案依据),注明适用版本、来源与缺口
|
||||
|
||||
## 正文与证据
|
||||
|
||||
@@ -19,7 +19,7 @@ description: 准备、创建、更新或审查 GitHub PR,含回复讨论与处
|
||||
|
||||
## 按模式读取
|
||||
|
||||
只加载当前模式,已有上下文不重读;长文档先定位标题再截取对应章节,不因链接存在而加载全文或所有 references
|
||||
先完成上述范围判断,再按当前动作与差异涉及的领域叠加下表。命中的流程引用完整读取,长文档读取指定章节;只复用已实际读取且仍适用的内容,单纯整理正文或回复不扩展为完整实现审查
|
||||
|
||||
| 当前动作 | 读取与执行 |
|
||||
| --- | --- |
|
||||
@@ -27,7 +27,7 @@ description: 准备、创建、更新或审查 GitHub PR,含回复讨论与处
|
||||
| 创建或更新 PR | [创建流程](references/create.md)、[PR 流程](../../../CONTRIBUTING.md#pr-流程)与[规则文件保护](../../../CONTRIBUTING.md#规则文件保护) |
|
||||
| 审查 PR | [审查流程](references/review.md)、[审查标准](../../../CONTRIBUTING.md#审查标准) |
|
||||
| 回复讨论或审查意见 | 目标评论、相关上下文与实际处理结果,按下方回复口吻答复;仅回复不加载创建或完整审查流程 |
|
||||
| 需要修改代码或文档 | 使用 [cpr-dev-guide](../cpr-dev-guide/SKILL.md)的对应分支,完成自审与验证 |
|
||||
| 实现自审、审查或修改代码与文档 | 执行 [cpr-dev-guide](../cpr-dev-guide/SKILL.md)的领域必读与对应检查,包括后端、插件、测试和 API 等命中项;只读审查同样执行,不能仅核对 PR 正文与 CI |
|
||||
| 含前端改动 | [前端流程](../cpr-dev-guide/references/frontend.md),核对适用规范与当前实现的运行证据 |
|
||||
| 含文档改动 | [文档检查](../cpr-dev-guide/references/documentation.md) |
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
interface:
|
||||
display_name: "cpr-github-pr"
|
||||
short_description: "提交、更新与审查 PR,核对问题、自审、页面截图和 AI 模型披露"
|
||||
default_prompt: "使用 $cpr-github-pr 按当前请求准备、提交、更新或审查 PR,核实问题与方案,检查自审、验证、页面截图和 AI 模型披露。"
|
||||
default_prompt: "使用 $cpr-github-pr 按当前请求准备、提交、更新或审查 PR,实现自审与审查叠加开发指南的领域规范,核实问题、方案、验证、页面截图和 AI 模型披露。"
|
||||
policy:
|
||||
allow_implicit_invocation: true
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
|
||||
1. 获取 PR 的 base/head SHA、描述、文件差异、已有讨论及 CI 状态。记录实际审查的 head,使用 merge-base 差异。
|
||||
需要源码上下文时先遵守目标仓库的定位规则;需检出时使用隔离工作区,保留用户当前分支和运行环境。
|
||||
按实际行为与文件执行 [cpr-dev-guide](../../cpr-dev-guide/SKILL.md)的领域必读,再判断方案与实现;只读审查也核对职责边界与生命周期。
|
||||
2. 按入口公共核对在目标分支基线核实问题,区分原有问题、环境或配置差异与 PR 新引入的问题,说明主分支已覆盖的部分。
|
||||
3. 按[问题与方案依据](../../../../CONTRIBUTING.md#问题与方案依据)评估解决方向,涉及上游时说明官方处理方式及本次方案关系;缺少必要性依据的偏离明确指出,不因测试通过就认定方案合理。
|
||||
4. 按[审查标准](../../../../CONTRIBUTING.md#审查标准)追踪完整调用链,找到共享业务事实、同类实现及已有验证。
|
||||
|
||||
@@ -1,13 +1,17 @@
|
||||
---
|
||||
name: cpr-plugin-dev
|
||||
description: 开发、排查或打包 Codex Proxy RS 独立网关插件;仅改宿主 Runtime、安装器或管理端时使用 cpr-dev-guide,不适用于 Codex 插件或 Vue 插件
|
||||
description: 开发、修复、排查、审查或打包 Codex Proxy RS 独立网关插件,含处理器、清单与插件页面;仅改宿主能力时使用 cpr-dev-guide 的插件边界,不适用于 Codex 插件或 Vue 插件
|
||||
---
|
||||
|
||||
# 网关插件开发
|
||||
|
||||
插件在独立项目中通过公开 SDK 实现业务,不向宿主业务模块添加专属流程
|
||||
## 先读职责边界
|
||||
|
||||
同时使用 [cpr-dev-guide](../cpr-dev-guide/SKILL.md)的对应任务分支,已加载则复用,不递归重读
|
||||
插件在独立项目中通过公开 SDK 实现业务,插件业务错误在插件侧修复;宿主维护自身合同、资源与稳定性,不猜测插件意图、不补造业务数据,不在宿主、SDK 或 Runtime 中增加插件专属补偿
|
||||
|
||||
开发、排查和审查先完整读取[插件职责边界](../cpr-dev-guide/references/plugin-boundaries.md),再选择实现步骤;已有处理器修复和页面改动也执行,不以是否新增能力为条件。需要新增宿主能力时,先界定可独立使用的基础接口及其职责;必要的协议保护与资源回收仍由宿主负责
|
||||
|
||||
同时执行 [cpr-dev-guide](../cpr-dev-guide/SKILL.md)的领域必读与收尾要求;已实际读取且仍适用的内容复用,不递归重读入口。仅构建、打包或分发已有插件时按对应操作读取,若需诊断或修改行为,再补齐开发分支
|
||||
|
||||
## 范围与合同
|
||||
|
||||
@@ -18,13 +22,13 @@ description: 开发、排查或打包 Codex Proxy RS 独立网关插件;仅改
|
||||
|
||||
## 按当前步骤读取
|
||||
|
||||
只读取命中行与链接的指定章节,长文档先列标题再截取;不预读全部 SDK、示例、前端和打包资料
|
||||
完成上述适用边界后,按当前步骤读取下表;同时涉及多项时叠加。命中的实现或页面参考完整读取,长文档读取指定章节,示例、前端和打包资料在对应步骤加载
|
||||
|
||||
| 当前工作 | 入口 |
|
||||
| --- | --- |
|
||||
| 选择能力或确认能力缺口 | [能力与方法](../../../backend/crates/gateway-plugin/sdk/docs/capabilities.md#能力与方法)、所需能力章节、[插件职责边界](../cpr-dev-guide/references/plugin-boundaries.md) |
|
||||
| 清单与 Rust 处理器 | [实现参考](references/implementation.md)、所用 SDK 类型;只有修改清单字段时读[清单](../../../backend/crates/gateway-plugin/sdk/docs/manifest.md)对应章节 |
|
||||
| 创建或修改页面 | [页面参考](references/frontend.md),不需要页面的插件不创建前端 |
|
||||
| 选择、使用或审查能力 | [能力与方法](../../../backend/crates/gateway-plugin/sdk/docs/capabilities.md#能力与方法)及所用能力章节,不以已有调用能编译代替合同核对 |
|
||||
| 开发、排查或审查清单与 Rust 处理器 | [实现参考](references/implementation.md)、所用 SDK 类型;涉及清单字段时读[清单](../../../backend/crates/gateway-plugin/sdk/docs/manifest.md)对应章节 |
|
||||
| 创建、修改或审查页面 | [页面参考](references/frontend.md),不需要页面的插件不创建前端 |
|
||||
| 准备工程或联调依赖 | [工程与依赖](references/development.md#工程与依赖),需要示例时再读[复用示例](references/development.md#复用示例) |
|
||||
| 构建或打包 | [构建与打包](references/development.md#构建与打包) |
|
||||
| 实际验收 | [验证与交付](references/development.md#验证与交付),按实现范围验证 |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
interface:
|
||||
display_name: "cpr-plugin-dev"
|
||||
short_description: "网关插件开发、接入排障、SDK 实现、页面与打包验证"
|
||||
default_prompt: "使用 $cpr-plugin-dev 按当前请求开发、排查或打包 Codex Proxy RS 网关插件,复用公开 SDK 与已有页面并验证实际行为。"
|
||||
short_description: "独立网关插件开发、排障与审查,先核对宿主边界再实现和验证"
|
||||
default_prompt: "使用 $cpr-plugin-dev 按当前请求开发、排查、审查或打包独立网关插件,先读取适用职责边界,插件业务错误在插件侧修复,再按步骤读取实现与验证资料。"
|
||||
policy:
|
||||
allow_implicit_invocation: true
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# 插件页面
|
||||
|
||||
仅在创建或修改插件页面时读取,先执行[前端流程](../../cpr-dev-guide/references/frontend.md),再核对以下插件特有合同
|
||||
创建、修改或审查插件页面时完整读取,先完成[插件入口](../SKILL.md#先读职责边界)的边界判断并执行[前端流程](../../cpr-dev-guide/references/frontend.md),再核对以下插件特有合同
|
||||
|
||||
- 优先参考独立示例仓库的 `examples/workbench/frontend/src/api/`:业务路由放在 `modules/`,`request.ts` 封装公开宿主桥;按需使用 `@codex-proxy/ui` 包出口,不跨仓导入宿主或 UI 的内部源码
|
||||
- Vue 页面保留 SFC,逻辑、脚本和配置使用 TypeScript;只实现业务内容,标题和副标题由 `ManagementPage` 交给宿主显示
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# 清单与处理器实现
|
||||
|
||||
仅在编写或排查插件清单、Rust 处理器时读取,涉及 Rust 时使用[开发入口](../../cpr-dev-guide/SKILL.md)的 Rust 分支
|
||||
选择能力时只读 SDK 的[能力与方法](../../../../backend/crates/gateway-plugin/sdk/docs/capabilities.md#能力与方法)及所需能力章节
|
||||
开发、排查或审查插件清单、Rust 处理器时完整读取,先完成[插件入口](../SKILL.md#先读职责边界)的边界判断,涉及 Rust 时执行[开发入口](../../cpr-dev-guide/SKILL.md)的 Rust 分支
|
||||
使用能力前读取 SDK 的[能力与方法](../../../../backend/crates/gateway-plugin/sdk/docs/capabilities.md#能力与方法)及所用能力章节,已有实现的修复与审查也核对合同
|
||||
|
||||
## 清单与处理器
|
||||
|
||||
|
||||
@@ -15,20 +15,6 @@ description: Codex Proxy RS 的版本规划、发布说明、正式或预发行
|
||||
|
||||
其他仓库使用其发布入口与规则,不套用本仓库路径;规则已在上下文中时不重复读取
|
||||
|
||||
## 按任务读取
|
||||
|
||||
只读当前阶段和指定章节,先定位标题再截取,不预读整份 CONTRIBUTING、工作流或所有参考文件;实际发布前仍须核对发布入口的完整副作用
|
||||
|
||||
| 任务 | 读取范围 |
|
||||
| --- | --- |
|
||||
| 推荐版本号 | [基线与版本](references/preparation.md#基线与版本) |
|
||||
| 编写或翻译发布说明 | [基线与版本](references/preparation.md#基线与版本)、[发布说明](references/preparation.md#发布说明) |
|
||||
| 检查发行条件 | [发行准备检查](references/preparation.md#发行准备检查) |
|
||||
| 实际发布 | 完成准备,再读[通道核对](references/operations.md#发行通道核对)和[执行发版](references/operations.md#执行发版) |
|
||||
| 失败恢复或验收 | [恢复与验收](references/operations.md#恢复与验收),先确定已完成步骤,不直接重跑入口 |
|
||||
| 实验版 | 在对应阶段补读[实验流程](references/experimental.md)的版本、文案或验收章节 |
|
||||
| 修复代码、工作流或常规文档 | 使用 [cpr-dev-guide](../cpr-dev-guide/SKILL.md),有 PR 任务再使用 [cpr-github-pr](../cpr-github-pr/SKILL.md) |
|
||||
|
||||
## 必须保持的边界
|
||||
|
||||
- Codex 兼容性变化按[问题与方案依据](../../../CONTRIBUTING.md#问题与方案依据)核对已有依据与验证范围,缺少依据列为发行风险,不以 CI 通过代替
|
||||
@@ -37,3 +23,17 @@ description: Codex Proxy RS 的版本规划、发布说明、正式或预发行
|
||||
- 预发行不得覆盖 GitHub Latest 或镜像 `latest`,完成后回读核对
|
||||
- `release/notes.md` 只承载当前发行的说明;常规文档修改另按开发入口执行[文档检查](../cpr-dev-guide/references/documentation.md)
|
||||
- 交付版本、提交、Release/工作流链接与实际结果,未完成构建或缺失产物时不称发布成功
|
||||
|
||||
## 按任务读取
|
||||
|
||||
先落实上述范围与边界,再读取当前阶段的完整指定章节;实际发布包含准备与验收,不能只读执行命令。阶段变化时补齐读取,准备文案不加载执行流程,实际发布前须核对发布入口的完整副作用
|
||||
|
||||
| 任务 | 读取范围 |
|
||||
| --- | --- |
|
||||
| 推荐版本号 | [基线与版本](references/preparation.md#基线与版本) |
|
||||
| 编写或翻译发布说明 | [基线与版本](references/preparation.md#基线与版本)、[发布说明](references/preparation.md#发布说明) |
|
||||
| 检查发行条件 | [发行准备检查](references/preparation.md#发行准备检查) |
|
||||
| 实际发布 | 先完成[基线与版本](references/preparation.md#基线与版本)、[发布说明](references/preparation.md#发布说明)、[发行准备检查](references/preparation.md#发行准备检查),再执行[通道核对](references/operations.md#发行通道核对)、[执行发版](references/operations.md#执行发版)与[恢复与验收](references/operations.md#恢复与验收) |
|
||||
| 失败恢复或验收 | [恢复与验收](references/operations.md#恢复与验收),先确定已完成步骤,不直接重跑入口 |
|
||||
| 实验版 | 在对应阶段补读[实验流程](references/experimental.md);实际发布必须覆盖分支与版本、发布能力、文案和执行与验收 |
|
||||
| 修复代码、工作流或常规文档 | 使用 [cpr-dev-guide](../cpr-dev-guide/SKILL.md),有 PR 任务再使用 [cpr-github-pr](../cpr-github-pr/SKILL.md) |
|
||||
|
||||
@@ -5,14 +5,16 @@
|
||||
| 任务意图与常见说法 | 必用技能 |
|
||||
| --- | --- |
|
||||
| 开发、修复、排障、代码审查、改文档;包括“改弹窗/样式/文案”“修接口/调度/数据库”等省略请求 | [cpr-dev-guide](.agents/skills/cpr-dev-guide/SKILL.md),按实际操作和文件匹配规范,跨前后端同时执行 |
|
||||
| 提交/提个 PR、准备或更新 PR、看下 PR、回复 PR 讨论、处理 PR 审查意见 | [cpr-github-pr](.agents/skills/cpr-github-pr/SKILL.md);需要修改时同时执行 cpr-dev-guide |
|
||||
| 提交/提个 PR、准备或更新 PR、看下 PR、回复 PR 讨论、处理 PR 审查意见 | [cpr-github-pr](.agents/skills/cpr-github-pr/SKILL.md);实现自审、审查或修改同时执行 cpr-dev-guide |
|
||||
| 提 Issue、反馈到仓库、把问题或建议整理成 Issue、补充或回复 Issue | [cpr-github-issue](.agents/skills/cpr-github-issue/SKILL.md) |
|
||||
| 写网关插件、修改插件能力、排查插件接入、打包插件 | [cpr-plugin-dev](.agents/skills/cpr-plugin-dev/SKILL.md),同时执行 cpr-dev-guide |
|
||||
| 写、修或审查独立网关插件,排查插件接入、打包插件 | [cpr-plugin-dev](.agents/skills/cpr-plugin-dev/SKILL.md),同时执行 cpr-dev-guide;仅改宿主能力时由 cpr-dev-guide 加载插件边界 |
|
||||
| 发版、准备版本、发布 beta/实验版、整理发布说明、打发行 tag | [cpr-release](.agents/skills/cpr-release/SKILL.md) |
|
||||
|
||||
结合上下文识别“提交吧”“看看这个问题”等省略说法:普通 Git 提交不自动变成 PR,代码排查不自动变成 Issue;审查与发版准备不代表获准远程写入
|
||||
|
||||
只加载当前任务命中的技能、模式和文档章节,复用已有上下文,进入新阶段再补读;链接是按需入口,不是全文或全部参考文件的必读清单
|
||||
先完整读取命中技能的 `SKILL.md`,再读取其领域必读规范,之后按当前步骤补充实现、命令和示例。技能名称、描述或入口摘要不能代替正文与必读引用;引用中明确要求“先读”“必须读取”或“执行”的规范也要落实,其他资料按需读取
|
||||
|
||||
按用户意图、行为影响和实际文件共同匹配,跨领域叠加;确定方案或给出审查结论前完成对应领域必读,不能因改动小、路径未落在该模块或自认为熟悉而跳过。只复用当前上下文中已实际读取且仍适用的内容;输出截断、上下文压缩后缺失的必要内容须补读,范围或阶段变化时重新匹配
|
||||
|
||||
执行所选流程中的问题核实、自审、验证与交付要求,协作流程和审查标准见 [CONTRIBUTING.md](CONTRIBUTING.md)
|
||||
|
||||
|
||||
Generated
+4
-4
@@ -1129,9 +1129,9 @@ checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17"
|
||||
|
||||
[[package]]
|
||||
name = "crypto-bigint"
|
||||
version = "0.7.5"
|
||||
version = "0.7.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1a52aa3fcda4e6302a9f48734f234d35d4721b96f8fe07d073f07ce9df4f0271"
|
||||
checksum = "6bb08299a6046833a879ace115697f12fd2bd6c8d3e93c909ffc9e0738e7cae0"
|
||||
dependencies = [
|
||||
"cpubits",
|
||||
"ctutils",
|
||||
@@ -4655,9 +4655,9 @@ checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20"
|
||||
|
||||
[[package]]
|
||||
name = "tokio"
|
||||
version = "1.52.3"
|
||||
version = "1.53.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8fc7f01b389ac15039e4dc9531aa973a135d7a4135281b12d7c1bc79fd57fffe"
|
||||
checksum = "ce3335fa71841cda333a58d7615b03901380ecf09d59b3296d21f8bbac0dde4e"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"libc",
|
||||
|
||||
@@ -250,7 +250,7 @@ fn source_manifest() -> &'static [u8] {
|
||||
"main":"bin/plugin",
|
||||
"runtime":"trustedProcess",
|
||||
"contributes":{
|
||||
"middleware":{"version":5,"stages":["request"],"inputFormats":["openai"],"outputFormats":["openai"]},
|
||||
"middleware":{"version":6,"stages":["request"],"inputFormats":["openai"],"outputFormats":["openai"]},
|
||||
"management":{"version":2}
|
||||
},
|
||||
|
||||
|
||||
@@ -891,7 +891,7 @@ impl StreamingExecutionBody {
|
||||
}
|
||||
continue;
|
||||
}
|
||||
// 已提交的流在确认 Core 终结后才交付成功终态,不能先输出 completed 再报结算失败
|
||||
// 已提交的流先确认 Core 终结,避免后续 EOF 验证推翻已交付的成功终态
|
||||
if self.awaiting_terminal_eof && self.committed {
|
||||
self.verify_terminal_eof().await?;
|
||||
encoded.push_back(done_frame());
|
||||
|
||||
@@ -5,7 +5,7 @@ use super::{
|
||||
pump::run_pump,
|
||||
state::{ConnectionCommand, ConnectionStats},
|
||||
};
|
||||
use axum::extract::ws::{CloseFrame, Message, WebSocket, close_code};
|
||||
use axum::extract::ws::{CloseFrame, Message, Utf8Bytes, WebSocket, close_code};
|
||||
use futures::{Sink, Stream};
|
||||
use gateway_core::{
|
||||
engine::middleware::{FrozenMiddlewarePlan, MiddlewareHeader},
|
||||
@@ -142,7 +142,7 @@ impl ResponsesWebSocketConnection {
|
||||
/// pump 已关闭、写入超时或 transport 失败时返回稳定错误
|
||||
pub async fn send_text(
|
||||
&mut self,
|
||||
payload: String,
|
||||
payload: impl Into<Utf8Bytes>,
|
||||
context: WriteContext,
|
||||
) -> Result<WriteOutcome, ConnectionWriteError> {
|
||||
self.send(Message::Text(payload.into()), context).await
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
use std::collections::VecDeque;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use axum::http::StatusCode;
|
||||
use axum::{extract::ws::Utf8Bytes, http::StatusCode};
|
||||
use bytes::Bytes;
|
||||
use futures::future::BoxFuture;
|
||||
use gateway_core::engine::CommitRequirement;
|
||||
@@ -268,7 +268,7 @@ pub(super) async fn forward_response(
|
||||
return fail_body(connection, body, MiddlewareError::InvalidState, &request_id).await;
|
||||
}
|
||||
let terminal = frame.terminal();
|
||||
let Ok(message) = String::from_utf8(frame.into_bytes().to_vec()) else {
|
||||
let Ok(message) = Utf8Bytes::try_from(frame.into_bytes()) else {
|
||||
return fail_body(connection, body, MiddlewareError::InvalidState, &request_id).await;
|
||||
};
|
||||
let phase = match (first_frame_written, terminal) {
|
||||
|
||||
@@ -9,4 +9,4 @@ mod trace;
|
||||
pub use capture::{body_fingerprint, diagnostic_headers, diagnostic_json, json_fingerprint};
|
||||
pub use failure::{OperationalDiagnostics, OperationalFailure};
|
||||
pub use stream::{StreamCapture, StreamFormat};
|
||||
pub use trace::TraceContext;
|
||||
pub use trace::{MAX_TRACE_SNAPSHOT_BYTES, TraceContext};
|
||||
|
||||
@@ -15,13 +15,16 @@ use crate::error::ProviderError;
|
||||
use super::capture::{bounded, diagnostic_event_json, diagnostic_event_type};
|
||||
use super::{body_fingerprint, diagnostic_headers};
|
||||
|
||||
/// 完整诊断快照的编码上限,生产者与持久化适配器共用
|
||||
pub const MAX_TRACE_SNAPSHOT_BYTES: usize = 128 * 1024;
|
||||
|
||||
const MAX_EVENTS: usize = 128;
|
||||
const HEAD_EVENTS: usize = 8;
|
||||
const MAX_DATA_BYTES: usize = 4096;
|
||||
const MAX_BUFFER_BYTES: usize = 64 * 1024;
|
||||
const MAX_BUFFER_BYTES: usize = MAX_TRACE_SNAPSHOT_BYTES / 2;
|
||||
const MAX_EVIDENCE_EVENTS: usize = 64;
|
||||
const MAX_EVIDENCE_DATA_BYTES: usize = 16 * 1024;
|
||||
const MAX_EVIDENCE_BUFFER_BYTES: usize = 64 * 1024;
|
||||
const MAX_EVIDENCE_BUFFER_BYTES: usize = MAX_TRACE_SNAPSHOT_BYTES / 2;
|
||||
|
||||
/// 显式传给异步任务的关联上下文;默认值禁用捕获,避免全局请求映射
|
||||
#[derive(Clone, Default)]
|
||||
@@ -284,27 +287,22 @@ impl TraceContext {
|
||||
/// 与请求终态原子持久化;旧记录没有此字段,不伪造历史时间线
|
||||
#[must_use]
|
||||
pub fn snapshot(&self) -> Option<Value> {
|
||||
let state = self
|
||||
let mut state = self
|
||||
.state
|
||||
.as_ref()?
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
Some(json!({
|
||||
"schemaVersion": 3, "requestId": state.request_id,
|
||||
"startedAtUnixMs": state.started_at_ms, "totalEvents": state.sequence,
|
||||
"droppedEvents": state.dropped_events, "maxEvents": MAX_EVENTS,
|
||||
"captureMode": "sanitized", "events": state.events,
|
||||
"wireDumpEnabled": tracing::enabled!(target: "request_dump", tracing::Level::INFO),
|
||||
"wireFrames": state.wire_frames, "wireBytes": state.wire_bytes,
|
||||
"maxBufferBytes": MAX_BUFFER_BYTES + MAX_EVIDENCE_BUFFER_BYTES, "headEvents": HEAD_EVENTS,
|
||||
"maxEvidenceEvents": MAX_EVIDENCE_EVENTS,
|
||||
"maxEvidenceDataBytes": MAX_EVIDENCE_DATA_BYTES,
|
||||
"maxEvidenceBufferBytes": MAX_EVIDENCE_BUFFER_BYTES,
|
||||
"droppedEvidenceEvents": state.dropped_evidence_events,
|
||||
"stageCounts": state.stage_counts,
|
||||
"stageCountOverflow": state.stage_count_overflow,
|
||||
"snapshotBoundary": "execution_finalization",
|
||||
}))
|
||||
loop {
|
||||
let snapshot = state.snapshot();
|
||||
if snapshot.to_string().len() <= MAX_TRACE_SNAPSHOT_BYTES {
|
||||
return Some(snapshot);
|
||||
}
|
||||
// 完整编码还包含计数与外层字段,超限时先淘汰普通事件并保留丢弃计数
|
||||
// 元数据本身无法容纳时省略诊断,不能让辅助观测阻断请求终态写入
|
||||
let index =
|
||||
regular_eviction_index(&state).or_else(|| evidence_eviction_index(&state))?;
|
||||
remove_event(&mut state, index);
|
||||
}
|
||||
}
|
||||
|
||||
fn push(&self, stage: &'static str, data: Value, coalesce: bool) {
|
||||
@@ -439,26 +437,14 @@ impl TraceContext {
|
||||
if state.evidence_bytes > MAX_EVIDENCE_BUFFER_BYTES
|
||||
|| state.evidence_events > MAX_EVIDENCE_EVENTS
|
||||
{
|
||||
if let Some(index) = state
|
||||
.events
|
||||
.iter()
|
||||
.position(|event| event.evidence && event.stage != "request.routed")
|
||||
.or_else(|| state.events.iter().position(|event| event.evidence))
|
||||
{
|
||||
if let Some(index) = evidence_eviction_index(&state) {
|
||||
remove_event(&mut state, index);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
// 相比流中间的常规事件,优先保留失败与重试决策
|
||||
// 关键事实使用独立预算,只有自身超限才会淘汰
|
||||
let index = (HEAD_EVENTS..state.events.len().saturating_sub(8))
|
||||
.find(|index| !state.events[*index].evidence)
|
||||
.or_else(|| {
|
||||
(HEAD_EVENTS..state.events.len())
|
||||
.chain(0..HEAD_EVENTS.min(state.events.len()))
|
||||
.find(|index| !state.events[*index].evidence)
|
||||
});
|
||||
if let Some(index) = index {
|
||||
if let Some(index) = regular_eviction_index(&state) {
|
||||
remove_event(&mut state, index);
|
||||
} else {
|
||||
break;
|
||||
@@ -467,6 +453,45 @@ impl TraceContext {
|
||||
}
|
||||
}
|
||||
|
||||
impl TraceState {
|
||||
fn snapshot(&self) -> Value {
|
||||
json!({
|
||||
"schemaVersion": 3, "requestId": self.request_id,
|
||||
"startedAtUnixMs": self.started_at_ms, "totalEvents": self.sequence,
|
||||
"droppedEvents": self.dropped_events, "maxEvents": MAX_EVENTS,
|
||||
"captureMode": "sanitized", "events": self.events,
|
||||
"wireDumpEnabled": tracing::enabled!(target: "request_dump", tracing::Level::INFO),
|
||||
"wireFrames": self.wire_frames, "wireBytes": self.wire_bytes,
|
||||
"maxBufferBytes": MAX_BUFFER_BYTES + MAX_EVIDENCE_BUFFER_BYTES, "headEvents": HEAD_EVENTS,
|
||||
"maxEvidenceEvents": MAX_EVIDENCE_EVENTS,
|
||||
"maxEvidenceDataBytes": MAX_EVIDENCE_DATA_BYTES,
|
||||
"maxEvidenceBufferBytes": MAX_EVIDENCE_BUFFER_BYTES,
|
||||
"droppedEvidenceEvents": self.dropped_evidence_events,
|
||||
"stageCounts": self.stage_counts,
|
||||
"stageCountOverflow": self.stage_count_overflow,
|
||||
"snapshotBoundary": "execution_finalization",
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
fn regular_eviction_index(state: &TraceState) -> Option<usize> {
|
||||
(HEAD_EVENTS..state.events.len().saturating_sub(8))
|
||||
.find(|index| !state.events[*index].evidence)
|
||||
.or_else(|| {
|
||||
(HEAD_EVENTS..state.events.len())
|
||||
.chain(0..HEAD_EVENTS.min(state.events.len()))
|
||||
.find(|index| !state.events[*index].evidence)
|
||||
})
|
||||
}
|
||||
|
||||
fn evidence_eviction_index(state: &TraceState) -> Option<usize> {
|
||||
state
|
||||
.events
|
||||
.iter()
|
||||
.position(|event| event.evidence && event.stage != "request.routed")
|
||||
.or_else(|| state.events.iter().position(|event| event.evidence))
|
||||
}
|
||||
|
||||
fn remove_event(state: &mut TraceState, index: usize) {
|
||||
if let Some(event) = state.events.remove(index) {
|
||||
state.bytes = state.bytes.saturating_sub(event.bytes);
|
||||
|
||||
@@ -44,9 +44,9 @@ pub struct ClientBudgetCharge {
|
||||
pub struct ClientBudgetError(#[source] pub Option<crate::error::ErrorSource>);
|
||||
|
||||
pub trait ClientBudgetPort: Send + Sync {
|
||||
/// 原子检查当前限额与已用金额,不创建预扣费或待结算记录
|
||||
/// 检查当前限额、已用金额与已接管的费用,不预占未来费用
|
||||
fn admit(&self, key_id: ClientApiKeyId) -> BoxFuture<'_, Result<(), GatewayError>>;
|
||||
|
||||
/// 按网关请求 ID 幂等累计已取得费用;写入失败由 Store 重试
|
||||
/// 按网关请求 ID 幂等接管已取得费用;运行时入队后返回,由 Store 后台落库和重试
|
||||
fn settle(&self, charge: ClientBudgetCharge) -> BoxFuture<'_, Result<(), ClientBudgetError>>;
|
||||
}
|
||||
|
||||
@@ -32,7 +32,7 @@ pub(super) async fn settle_budget(
|
||||
diagnostics,
|
||||
&request_id,
|
||||
"settle_client_budget",
|
||||
"Client budget settlement failed; storage will retry on the next request",
|
||||
"Client budget settlement could not be accepted",
|
||||
error,
|
||||
)
|
||||
.await;
|
||||
|
||||
@@ -43,6 +43,8 @@ pub enum MiddlewareMount {
|
||||
Request,
|
||||
/// 受管 Provider attempt 内层;每次真实 retry 都重新建立
|
||||
Attempt,
|
||||
/// Provider 完成序列化后、提交实际发送前的请求准备
|
||||
Outbound,
|
||||
}
|
||||
|
||||
/// 中间件正文块的宿主边界
|
||||
@@ -102,6 +104,7 @@ impl fmt::Debug for MiddlewareHeader {
|
||||
#[derive(Clone, PartialEq, Eq)]
|
||||
pub struct MiddlewareRequest {
|
||||
settings: Option<RequestSettings>,
|
||||
provider_parameters: Option<serde_json::Map<String, serde_json::Value>>,
|
||||
protocol: String,
|
||||
headers: Vec<MiddlewareHeader>,
|
||||
body: Bytes,
|
||||
@@ -123,6 +126,7 @@ impl MiddlewareRequest {
|
||||
let protocol = protocol.into();
|
||||
Self {
|
||||
settings: None,
|
||||
provider_parameters: None,
|
||||
original_protocol: protocol.clone(),
|
||||
original_body: body.clone(),
|
||||
capabilities: None,
|
||||
@@ -143,6 +147,27 @@ impl MiddlewareRequest {
|
||||
self.settings.as_ref()
|
||||
}
|
||||
|
||||
/// Provider 提供的本次构造参数;Core 不解释其中的业务字段
|
||||
#[must_use]
|
||||
pub fn provider_parameters(&self) -> Option<&serde_json::Map<String, serde_json::Value>> {
|
||||
self.provider_parameters.as_ref()
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn with_provider_parameters(
|
||||
mut self,
|
||||
parameters: serde_json::Map<String, serde_json::Value>,
|
||||
) -> Self {
|
||||
self.provider_parameters = Some(parameters);
|
||||
self
|
||||
}
|
||||
|
||||
pub fn take_provider_parameters(
|
||||
&mut self,
|
||||
) -> Option<serde_json::Map<String, serde_json::Value>> {
|
||||
self.provider_parameters.take()
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn protocol(&self) -> &str {
|
||||
&self.protocol
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
mod contract;
|
||||
pub mod http;
|
||||
pub mod outbound;
|
||||
mod plan;
|
||||
pub mod service;
|
||||
pub mod websocket;
|
||||
|
||||
@@ -0,0 +1,140 @@
|
||||
//! Provider 发送前的完整请求准备;续体只组合值,不执行网络操作
|
||||
|
||||
use std::fmt;
|
||||
|
||||
use bytes::Bytes;
|
||||
use futures::FutureExt;
|
||||
|
||||
use super::{FrozenMiddlewarePlan, MiddlewareContext, MiddlewareError, MiddlewareHeader};
|
||||
use crate::error::{ProviderError, ProviderErrorKind};
|
||||
use crate::upstream::UpstreamSendState;
|
||||
|
||||
/// 当前发送所用的传输类型与目标;准备过程中不能切换传输类型
|
||||
#[derive(Clone, PartialEq, Eq)]
|
||||
pub enum Target {
|
||||
Http {
|
||||
method: http::Method,
|
||||
uri: http::Uri,
|
||||
},
|
||||
WebSocket {
|
||||
uri: http::Uri,
|
||||
},
|
||||
}
|
||||
|
||||
impl fmt::Debug for Target {
|
||||
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
match self {
|
||||
Self::Http { method, .. } => formatter
|
||||
.debug_struct("Http")
|
||||
.field("method", method)
|
||||
.finish_non_exhaustive(),
|
||||
Self::WebSocket { .. } => formatter.debug_struct("WebSocket").finish_non_exhaustive(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// HTTP 正文已完成序列化与内容编码;WebSocket 正文是一条完整文本消息
|
||||
///
|
||||
/// WebSocket headers 表示候选 opening,不能改写已经建立的连接握手
|
||||
#[derive(Clone, PartialEq, Eq)]
|
||||
pub struct Request {
|
||||
pub target: Target,
|
||||
pub headers: Vec<MiddlewareHeader>,
|
||||
pub body: Bytes,
|
||||
}
|
||||
|
||||
impl Request {
|
||||
/// 转换实际发送的 HTTP 头,保留多值与原始字节,不修复插件输出
|
||||
pub fn header_map(&self) -> Result<http::HeaderMap, ProviderError> {
|
||||
let mut headers = http::HeaderMap::try_with_capacity(self.headers.len())
|
||||
.map_err(|_| preparation_error(ProviderErrorKind::InvalidRequest))?;
|
||||
for header in &self.headers {
|
||||
let name = http::HeaderName::from_bytes(header.name().as_bytes())
|
||||
.map_err(|_| preparation_error(ProviderErrorKind::InvalidRequest))?;
|
||||
let value = http::HeaderValue::from_bytes(header.value())
|
||||
.map_err(|_| preparation_error(ProviderErrorKind::InvalidRequest))?;
|
||||
headers
|
||||
.try_append(name, value)
|
||||
.map_err(|_| preparation_error(ProviderErrorKind::InvalidRequest))?;
|
||||
}
|
||||
Ok(headers)
|
||||
}
|
||||
}
|
||||
|
||||
impl fmt::Debug for Request {
|
||||
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
formatter
|
||||
.debug_struct("Request")
|
||||
.field("target", &self.target)
|
||||
.field("headers", &self.headers.len())
|
||||
.field("body_bytes", &self.body.len())
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
pub type Next = crate::middleware::Next<Request, Request, MiddlewareError>;
|
||||
|
||||
/// 保活本次 attempt 的冻结发布代次、归属和资源上下文
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Pipeline {
|
||||
plan: FrozenMiddlewarePlan,
|
||||
context: MiddlewareContext,
|
||||
}
|
||||
|
||||
impl Pipeline {
|
||||
pub(in crate::engine) const fn new(
|
||||
plan: FrozenMiddlewarePlan,
|
||||
context: MiddlewareContext,
|
||||
) -> Self {
|
||||
Self { plan, context }
|
||||
}
|
||||
|
||||
/// 完整返回洋葱链最终输出后,Provider 才能提交实际发送
|
||||
pub async fn prepare(&self, request: Request) -> Result<Request, ProviderError> {
|
||||
if self.context.cancellation().is_cancelled() {
|
||||
return Err(preparation_error(ProviderErrorKind::Cancelled));
|
||||
}
|
||||
if self.context.deadline().is_elapsed() {
|
||||
return Err(preparation_error(ProviderErrorKind::Timeout));
|
||||
}
|
||||
let target_kind = std::mem::discriminant(&request.target);
|
||||
let next =
|
||||
crate::middleware::compose(Vec::new(), |request| Box::pin(async { Ok(request) }));
|
||||
let prepare = self
|
||||
.plan
|
||||
.handle_outbound(self.context.clone(), request, next)
|
||||
.fuse();
|
||||
let cancelled = self.context.cancellation().cancelled().fuse();
|
||||
let expired = self.context.deadline().wait().fuse();
|
||||
futures::pin_mut!(prepare, cancelled, expired);
|
||||
let prepared = futures::select_biased! {
|
||||
() = cancelled => return Err(preparation_error(ProviderErrorKind::Cancelled)),
|
||||
() = expired => return Err(preparation_error(ProviderErrorKind::Timeout)),
|
||||
result = prepare => result.map_err(|error| {
|
||||
crate::engine::provider::middleware_error(error, UpstreamSendState::NotSent)
|
||||
.with_retry_prohibited()
|
||||
})?,
|
||||
};
|
||||
if self.context.cancellation().is_cancelled() {
|
||||
return Err(preparation_error(ProviderErrorKind::Cancelled));
|
||||
}
|
||||
if self.context.deadline().is_elapsed() {
|
||||
return Err(preparation_error(ProviderErrorKind::Timeout));
|
||||
}
|
||||
if std::mem::discriminant(&prepared.target) != target_kind
|
||||
|| self
|
||||
.context
|
||||
.request_body_limit()
|
||||
.is_some_and(|limit| prepared.body.len() > limit)
|
||||
{
|
||||
return Err(preparation_error(ProviderErrorKind::InvalidRequest));
|
||||
}
|
||||
Ok(prepared)
|
||||
}
|
||||
}
|
||||
|
||||
fn preparation_error(kind: ProviderErrorKind) -> ProviderError {
|
||||
ProviderError::new(kind, UpstreamSendState::NotSent)
|
||||
.with_retry_prohibited()
|
||||
.redact_sensitive_context("invalid outbound request preparation")
|
||||
}
|
||||
@@ -9,6 +9,18 @@ use std::{fmt, sync::Arc};
|
||||
|
||||
/// 一个发布代次冻结的统一数据面中间件计划
|
||||
pub trait MiddlewarePlan: Send + Sync + fmt::Debug {
|
||||
fn has_outbound(&self, _context: &MiddlewareContext) -> bool {
|
||||
false
|
||||
}
|
||||
fn handle_outbound(
|
||||
&self,
|
||||
_context: MiddlewareContext,
|
||||
request: super::outbound::Request,
|
||||
next: super::outbound::Next,
|
||||
) -> BoxFuture<'static, Result<super::outbound::Request, MiddlewareError>> {
|
||||
next.run(request)
|
||||
}
|
||||
|
||||
fn has_service(&self) -> bool {
|
||||
false
|
||||
}
|
||||
@@ -73,6 +85,20 @@ pub struct FrozenMiddlewarePlan {
|
||||
}
|
||||
|
||||
impl FrozenMiddlewarePlan {
|
||||
#[must_use]
|
||||
pub fn has_outbound(&self, context: &MiddlewareContext) -> bool {
|
||||
self.plan.has_outbound(context)
|
||||
}
|
||||
pub fn handle_outbound(
|
||||
&self,
|
||||
mut context: MiddlewareContext,
|
||||
request: super::outbound::Request,
|
||||
next: super::outbound::Next,
|
||||
) -> BoxFuture<'static, Result<super::outbound::Request, MiddlewareError>> {
|
||||
context.plan = Some(self.clone());
|
||||
self.plan.handle_outbound(context, request, next)
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn has_service(&self) -> bool {
|
||||
self.plan.has_service()
|
||||
|
||||
@@ -575,11 +575,48 @@ impl AttemptContext {
|
||||
&self,
|
||||
provider: &ProviderKind,
|
||||
model: &UpstreamModelId,
|
||||
) -> Result<upstream_adapter::UpstreamAdapterSelection, ProviderError> {
|
||||
let account = self
|
||||
.required_account()
|
||||
.or_else(|| match self.continuation_attempt() {
|
||||
ContinuationAttempt::Native => self
|
||||
.continuation()
|
||||
.and_then(continuation::ContinuationBinding::pinned)
|
||||
.map(|pin| pin.account()),
|
||||
ContinuationAttempt::ReplayOwner => self
|
||||
.account_state_owner()
|
||||
.filter(|owner| owner.provider() == provider)
|
||||
.map(|owner| owner.account()),
|
||||
ContinuationAttempt::None | ContinuationAttempt::ReplayAny => None,
|
||||
});
|
||||
self.request.upstream_adapters.as_ref().map_or(
|
||||
Ok(upstream_adapter::UpstreamAdapterSelection::Native),
|
||||
|plan| plan.select(self, provider, model, account),
|
||||
)
|
||||
}
|
||||
|
||||
/// 账号候选资格检查和最终执行解析使用同一冻结计划,不调用插件进程
|
||||
pub fn upstream_adapter_for_account(
|
||||
&self,
|
||||
provider: &ProviderKind,
|
||||
model: &UpstreamModelId,
|
||||
account: &ProviderAccountId,
|
||||
) -> Result<Option<Arc<dyn upstream_adapter::UpstreamAdapter>>, ProviderError> {
|
||||
self.request
|
||||
.upstream_adapters
|
||||
.as_ref()
|
||||
.map_or(Ok(None), |plan| plan.select(self, provider, model))
|
||||
.map_or(Ok(None), |plan| {
|
||||
match plan.select(self, provider, model, Some(account))? {
|
||||
upstream_adapter::UpstreamAdapterSelection::Native => Ok(None),
|
||||
upstream_adapter::UpstreamAdapterSelection::Adapter(adapter) => {
|
||||
Ok(Some(adapter))
|
||||
}
|
||||
// 已知账号的解析必须给出最终执行端,不能绕过资格检查进入原生路径
|
||||
upstream_adapter::UpstreamAdapterSelection::PendingAccount => Err(
|
||||
ProviderError::new(ProviderErrorKind::Protocol, UpstreamSendState::NotSent),
|
||||
),
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
@@ -718,6 +755,7 @@ impl AttemptContext {
|
||||
provider: ProviderKind,
|
||||
model: Option<String>,
|
||||
account_id: ProviderAccountId,
|
||||
parameters: Option<serde_json::Map<String, serde_json::Value>>,
|
||||
terminal: provider::ProviderMiddlewareTerminal,
|
||||
) -> Result<provider::ProviderStream, ProviderError> {
|
||||
// 入口已校验原请求,此处复查中间件最终提交的 operation,每次重试均生效
|
||||
@@ -725,7 +763,7 @@ impl AttemptContext {
|
||||
.account_scope()
|
||||
.is_some_and(|scope| !scope.allows_image_generation())
|
||||
{
|
||||
Box::new(move |operation, headers| {
|
||||
Box::new(move |operation, headers, parameters| {
|
||||
Box::pin(async move {
|
||||
image_generation::check_image_generation(false, &operation).map_err(|_| {
|
||||
ProviderError::new(
|
||||
@@ -734,18 +772,64 @@ impl AttemptContext {
|
||||
)
|
||||
.with_retry_prohibited()
|
||||
})?;
|
||||
terminal(operation, headers).await
|
||||
terminal(operation, headers, parameters).await
|
||||
})
|
||||
})
|
||||
} else {
|
||||
terminal
|
||||
};
|
||||
let context = middleware::MiddlewareContext::new(
|
||||
let context = self.middleware_context(
|
||||
middleware::MiddlewareMount::Attempt,
|
||||
Some(operation.kind()),
|
||||
provider,
|
||||
model,
|
||||
account_id,
|
||||
);
|
||||
provider::execute_attempt_middleware(
|
||||
self.request.middleware.as_ref(),
|
||||
context,
|
||||
operation,
|
||||
self.request.client_transport,
|
||||
parameters,
|
||||
terminal,
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
/// 为当前实际账号冻结出站准备链,无此阶段时保留 Provider 原生路径
|
||||
#[must_use]
|
||||
pub fn outbound_middleware(
|
||||
&self,
|
||||
provider: ProviderKind,
|
||||
model: Option<String>,
|
||||
account_id: ProviderAccountId,
|
||||
) -> Option<middleware::outbound::Pipeline> {
|
||||
let plan = self.request.middleware.as_ref()?;
|
||||
let context = self.middleware_context(
|
||||
middleware::MiddlewareMount::Outbound,
|
||||
None,
|
||||
provider,
|
||||
model,
|
||||
account_id,
|
||||
);
|
||||
plan.has_outbound(&context)
|
||||
.then(|| middleware::outbound::Pipeline::new(plan.clone(), context))
|
||||
}
|
||||
|
||||
fn middleware_context(
|
||||
&self,
|
||||
mount: middleware::MiddlewareMount,
|
||||
operation: Option<crate::operation::OperationKind>,
|
||||
provider: ProviderKind,
|
||||
model: Option<String>,
|
||||
account_id: ProviderAccountId,
|
||||
) -> middleware::MiddlewareContext {
|
||||
middleware::MiddlewareContext::new(
|
||||
middleware::MiddlewareTarget {
|
||||
request_id: self.request.request_id.clone(),
|
||||
mount: middleware::MiddlewareMount::Attempt,
|
||||
mount,
|
||||
attempt_index: Some(self.attempt_index),
|
||||
operation: Some(operation.kind()),
|
||||
operation,
|
||||
endpoint: self.request.endpoint.clone(),
|
||||
transport: self.request.client_transport,
|
||||
provider: Some(provider),
|
||||
@@ -761,15 +845,7 @@ impl AttemptContext {
|
||||
execution_effects: self.request.execution_effects.as_ref().map(Arc::clone),
|
||||
},
|
||||
)
|
||||
.with_request_body_limit(self.request.request_body_limit);
|
||||
provider::execute_attempt_middleware(
|
||||
self.request.middleware.as_ref(),
|
||||
context,
|
||||
operation,
|
||||
self.request.client_transport,
|
||||
terminal,
|
||||
)
|
||||
.await
|
||||
.with_request_body_limit(self.request.request_body_limit)
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
|
||||
@@ -471,6 +471,7 @@ pub type ProviderMiddlewareTerminal = Box<
|
||||
dyn FnOnce(
|
||||
Operation,
|
||||
Vec<MiddlewareHeader>,
|
||||
Option<serde_json::Map<String, serde_json::Value>>,
|
||||
) -> BoxFuture<'static, Result<ProviderStream, ProviderError>>
|
||||
+ Send
|
||||
+ 'static,
|
||||
@@ -489,18 +490,19 @@ fn provider_middleware_next(
|
||||
activate: oneshot::Receiver<()>,
|
||||
observation: Arc<Mutex<MiddlewareSendObservation>>,
|
||||
) -> MiddlewareNext {
|
||||
crate::middleware::compose(Vec::new(), move |request| {
|
||||
crate::middleware::compose(Vec::new(), move |mut request| {
|
||||
Box::pin(async move {
|
||||
let fallback_protocol = operation.protocol().to_owned();
|
||||
if request.has_capability_declaration() {
|
||||
return Err(MiddlewareError::InvalidState);
|
||||
}
|
||||
let parameters = request.take_provider_parameters();
|
||||
let (protocol, headers, body) = request.into_parts();
|
||||
validate_attempt_headers(&headers)?;
|
||||
let operation = operation
|
||||
.replace_middleware_wire(protocol, body)
|
||||
.map_err(|_| MiddlewareError::InvalidState)?;
|
||||
let stream = terminal(operation, headers)
|
||||
let stream = terminal(operation, headers, parameters)
|
||||
.await
|
||||
.map_err(MiddlewareError::Provider)?;
|
||||
let metadata = stream.metadata().clone();
|
||||
@@ -734,18 +736,22 @@ pub async fn execute_attempt_middleware(
|
||||
context: MiddlewareContext,
|
||||
operation: Operation,
|
||||
transport: ClientTransport,
|
||||
parameters: Option<serde_json::Map<String, serde_json::Value>>,
|
||||
terminal: ProviderMiddlewareTerminal,
|
||||
) -> Result<ProviderStream, ProviderError> {
|
||||
let Some(plan) = plan else {
|
||||
return terminal(operation, Vec::new()).await;
|
||||
return terminal(operation, Vec::new(), parameters).await;
|
||||
};
|
||||
let request = MiddlewareRequest::new(
|
||||
let mut request = MiddlewareRequest::new(
|
||||
operation.protocol(),
|
||||
Vec::<MiddlewareHeader>::new(),
|
||||
operation
|
||||
.middleware_body()
|
||||
.map_err(|_| middleware_protocol_error(UpstreamSendState::NotSent))?,
|
||||
);
|
||||
if let Some(parameters) = parameters {
|
||||
request = request.with_provider_parameters(parameters);
|
||||
}
|
||||
let (prepared, metadata) = oneshot::channel();
|
||||
let (activate, activation) = oneshot::channel();
|
||||
let observation = Arc::new(Mutex::new(MiddlewareSendObservation {
|
||||
@@ -893,7 +899,10 @@ fn unprepared_middleware_response(
|
||||
}
|
||||
}
|
||||
|
||||
fn middleware_error(error: MiddlewareError, send_state: UpstreamSendState) -> ProviderError {
|
||||
pub(super) fn middleware_error(
|
||||
error: MiddlewareError,
|
||||
send_state: UpstreamSendState,
|
||||
) -> ProviderError {
|
||||
match error {
|
||||
MiddlewareError::Provider(error) => error,
|
||||
MiddlewareError::Rejected | MiddlewareError::Remote { rejected: true, .. } => {
|
||||
|
||||
@@ -31,7 +31,16 @@ pub trait UpstreamAdapterPlan: fmt::Debug + Send + Sync {
|
||||
context: &AttemptContext,
|
||||
provider: &ProviderKind,
|
||||
model: &UpstreamModelId,
|
||||
) -> Result<Option<Arc<dyn UpstreamAdapter>>, ProviderError>;
|
||||
account: Option<&ProviderAccountId>,
|
||||
) -> Result<UpstreamAdapterSelection, ProviderError>;
|
||||
}
|
||||
|
||||
/// 未确定账号时不能把账号范围声明视为原生执行或继续命中后置兜底
|
||||
#[derive(Debug)]
|
||||
pub enum UpstreamAdapterSelection {
|
||||
Native,
|
||||
Adapter(Arc<dyn UpstreamAdapter>),
|
||||
PendingAccount,
|
||||
}
|
||||
|
||||
/// 请求同时持有计划及发布集合,防止旧集合先排空进程而中断在途调用
|
||||
@@ -58,8 +67,9 @@ impl FrozenUpstreamAdapterPlan {
|
||||
context: &AttemptContext,
|
||||
provider: &ProviderKind,
|
||||
model: &UpstreamModelId,
|
||||
) -> Result<Option<Arc<dyn UpstreamAdapter>>, ProviderError> {
|
||||
self.plan.select(context, provider, model)
|
||||
account: Option<&ProviderAccountId>,
|
||||
) -> Result<UpstreamAdapterSelection, ProviderError> {
|
||||
self.plan.select(context, provider, model, account)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -71,6 +81,9 @@ pub trait UpstreamAdapter: fmt::Debug + Send + Sync {
|
||||
|
||||
/// 注册时声明的上游传输,用于宿主登记 attempt
|
||||
fn transport(&self) -> &str;
|
||||
|
||||
/// 选号前复核注册的认证资格,发送前仍由执行端校验当前凭据
|
||||
fn supports_authentication(&self, kind: &str) -> bool;
|
||||
}
|
||||
|
||||
/// 一次执行的宿主事实;账号及 Provider 归属不可由插件返回值覆盖
|
||||
|
||||
@@ -5,7 +5,7 @@ use std::{
|
||||
sync::{Arc, Mutex},
|
||||
};
|
||||
|
||||
use gateway_core::diagnostics::{TraceContext, body_fingerprint};
|
||||
use gateway_core::diagnostics::{MAX_TRACE_SNAPSHOT_BYTES, TraceContext, body_fingerprint};
|
||||
use gateway_core::error::{ProviderDiagnostic, ProviderError, ProviderErrorKind, RawUpstreamError};
|
||||
use gateway_core::upstream::UpstreamSendState;
|
||||
use serde_json::json;
|
||||
@@ -47,6 +47,7 @@ fn critical_evidence_survives_stream_pressure_and_preserves_stage_coverage() {
|
||||
);
|
||||
assert_eq!(snapshot["droppedEvidenceEvents"], 0);
|
||||
assert!(snapshot["droppedEvents"].as_u64().unwrap() > 0);
|
||||
assert!(snapshot.to_string().len() <= MAX_TRACE_SNAPSHOT_BYTES);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -67,8 +68,42 @@ fn critical_evidence_has_an_independent_bound_and_reports_its_own_loss() {
|
||||
events.len() as u64 + snapshot["droppedEvents"].as_u64().unwrap(),
|
||||
100
|
||||
);
|
||||
assert!(snapshot.to_string().len() < 128 * 1024);
|
||||
assert!(snapshot.to_string().len() <= MAX_TRACE_SNAPSHOT_BYTES);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn snapshot_budget_includes_stage_coverage_and_event_envelopes() {
|
||||
let trace = TraceContext::new("req_snapshot_budget");
|
||||
trace.record("request.routed", json!({"scope": "all"}));
|
||||
for index in 0..10 {
|
||||
trace.record_evidence(
|
||||
"provider.request.input",
|
||||
json!({"index": index, "hashes": "a".repeat(6000)}),
|
||||
);
|
||||
}
|
||||
for index in 0..64 {
|
||||
let stage = Box::leak(format!("stage.{index}.{}", "x".repeat(512)).into_boxed_str());
|
||||
trace.record(stage, json!({"detail": "x".repeat(2000)}));
|
||||
}
|
||||
trace.record("request.finished", json!({"outcome": "succeeded"}));
|
||||
let snapshot = trace.snapshot().unwrap();
|
||||
let events = snapshot["events"].as_array().unwrap();
|
||||
assert!(snapshot.to_string().len() <= MAX_TRACE_SNAPSHOT_BYTES);
|
||||
assert_eq!(events.first().unwrap()["stage"], "request.routed");
|
||||
assert_eq!(events.last().unwrap()["stage"], "request.finished");
|
||||
assert_eq!(snapshot["droppedEvidenceEvents"], 0);
|
||||
assert!(snapshot["droppedEvents"].as_u64().unwrap() > 0);
|
||||
let retained: u64 = events
|
||||
.iter()
|
||||
.map(|event| event["count"].as_u64().unwrap())
|
||||
.sum();
|
||||
assert_eq!(
|
||||
retained + snapshot["droppedEvents"].as_u64().unwrap(),
|
||||
snapshot["totalEvents"]
|
||||
);
|
||||
assert_eq!(trace.snapshot(), Some(snapshot));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bounded_history_preserves_start_failure_and_final_result() {
|
||||
let trace = TraceContext::new("req_trace");
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
//! 执行服务的准入、预算、请求记录与终结行为测试
|
||||
|
||||
mod outbound;
|
||||
use gateway_core::routing::extensions::{ExtensionSetId, ExtensionSetLease, ExtensionSetReference};
|
||||
|
||||
use std::{
|
||||
|
||||
@@ -0,0 +1,224 @@
|
||||
//! 出站插件的外部副作用在错误改写后仍约束 Engine 重放
|
||||
|
||||
use super::*;
|
||||
use gateway_core::engine::extensions::{ExecutionExtensionIndex, ExecutionExtensionPlans};
|
||||
use gateway_core::engine::middleware::{
|
||||
MiddlewareContext, MiddlewareError, MiddlewareMount, MiddlewareNext, MiddlewarePlan,
|
||||
MiddlewareRequest, MiddlewareResponse, outbound,
|
||||
};
|
||||
|
||||
#[derive(Debug)]
|
||||
struct EffectfulOutbound {
|
||||
catch_error: bool,
|
||||
caught: Arc<AtomicUsize>,
|
||||
}
|
||||
|
||||
impl MiddlewarePlan for EffectfulOutbound {
|
||||
fn has_outbound(&self, _context: &MiddlewareContext) -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
fn handle_outbound(
|
||||
&self,
|
||||
context: MiddlewareContext,
|
||||
_: outbound::Request,
|
||||
_: outbound::Next,
|
||||
) -> BoxFuture<'static, Result<outbound::Request, MiddlewareError>> {
|
||||
Box::pin(async move {
|
||||
// 对应 Runtime 确认受管 HTTP 已发送后的唯一记录入口
|
||||
context.execution_effects().unwrap().observe();
|
||||
Err(MiddlewareError::Fault)
|
||||
})
|
||||
}
|
||||
|
||||
fn handle(
|
||||
&self,
|
||||
context: MiddlewareContext,
|
||||
request: MiddlewareRequest,
|
||||
next: MiddlewareNext,
|
||||
) -> BoxFuture<'static, Result<MiddlewareResponse, MiddlewareError>> {
|
||||
let catch_error = self.catch_error && context.mount() == MiddlewareMount::Attempt;
|
||||
if !catch_error {
|
||||
return next.run(request);
|
||||
}
|
||||
let caught = Arc::clone(&self.caught);
|
||||
Box::pin(async move {
|
||||
let response = next.run(request).await?;
|
||||
let (_, _, _, mut body, _) = response.into_parts();
|
||||
body.next_frame()
|
||||
.await
|
||||
.expect_err("outbound preparation fails before a frame");
|
||||
body.close().await;
|
||||
caught.fetch_add(1, Ordering::SeqCst);
|
||||
Err(MiddlewareError::Provider(ProviderError::new(
|
||||
ProviderErrorKind::Transport,
|
||||
UpstreamSendState::NotSent,
|
||||
)))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
struct PreparingProvider {
|
||||
calls: AtomicUsize,
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
impl Provider for PreparingProvider {
|
||||
fn name(&self) -> &'static str {
|
||||
"openai"
|
||||
}
|
||||
fn catalog_generation(&self) -> ProviderCatalogGeneration {
|
||||
ProviderCatalogGeneration::default()
|
||||
}
|
||||
async fn query_model_capabilities(
|
||||
&self,
|
||||
) -> Result<Vec<ProviderModelCapabilities>, ProviderError> {
|
||||
Ok(vec![])
|
||||
}
|
||||
|
||||
async fn execute(
|
||||
self: Arc<Self>,
|
||||
request: ProviderRequest,
|
||||
context: AttemptContext,
|
||||
) -> Result<ProviderStream, ProviderError> {
|
||||
self.calls.fetch_add(1, Ordering::SeqCst);
|
||||
let provider = request.candidate().provider().clone();
|
||||
let model = request
|
||||
.candidate()
|
||||
.upstream_model()
|
||||
.map(|model| model.as_str().to_owned());
|
||||
let account = ProviderAccountId::new("acct_openai").unwrap();
|
||||
let pipeline = context
|
||||
.outbound_middleware(provider.clone(), model.clone(), account.clone())
|
||||
.unwrap();
|
||||
let metadata = ProviderCallMetadata::new(
|
||||
provider.clone(),
|
||||
request.candidate().upstream_model().unwrap().clone(),
|
||||
account.clone(),
|
||||
UpstreamTransport::new("http").unwrap(),
|
||||
);
|
||||
context
|
||||
.execute_middleware(
|
||||
request.operation().clone(),
|
||||
provider,
|
||||
model,
|
||||
account,
|
||||
None,
|
||||
Box::new(move |_, _, _| {
|
||||
Box::pin(async move {
|
||||
let events = futures::stream::once(async move {
|
||||
Err::<ProviderEvent, _>(
|
||||
pipeline
|
||||
.prepare(outbound::Request {
|
||||
target: outbound::Target::Http {
|
||||
method: http::Method::POST,
|
||||
uri: "https://example.test/".parse().unwrap(),
|
||||
},
|
||||
headers: Vec::new(),
|
||||
body: bytes::Bytes::from_static(b"request"),
|
||||
})
|
||||
.await
|
||||
.unwrap_err(),
|
||||
)
|
||||
});
|
||||
Ok(ProviderStream::new(metadata, events, ()))
|
||||
})
|
||||
}),
|
||||
)
|
||||
.await
|
||||
}
|
||||
}
|
||||
|
||||
struct Fixture {
|
||||
service: DefaultExecutionService,
|
||||
provider: Arc<PreparingProvider>,
|
||||
observer: Arc<RecordingRequestObserver>,
|
||||
caught: Arc<AtomicUsize>,
|
||||
_extensions: Arc<ExecutionExtensionPlans>,
|
||||
}
|
||||
|
||||
fn fixture(catch_error: bool) -> Fixture {
|
||||
let provider = Arc::new(PreparingProvider {
|
||||
calls: AtomicUsize::new(0),
|
||||
});
|
||||
let generation = super::super::extensions::reference("outbound-effect");
|
||||
let extension_index = ExecutionExtensionIndex::default();
|
||||
let caught = Arc::new(AtomicUsize::new(0));
|
||||
let extension_owner = extension_index
|
||||
.register(
|
||||
generation.id().clone(),
|
||||
Arc::new(ExecutionExtensionPlans {
|
||||
middleware: Some(Arc::new(EffectfulOutbound {
|
||||
catch_error,
|
||||
caught: Arc::clone(&caught),
|
||||
})),
|
||||
upstream_adapters: None,
|
||||
}),
|
||||
)
|
||||
.unwrap();
|
||||
let observer_index = RequestObserverExtensionIndex::default();
|
||||
let observer = Arc::new(RecordingRequestObserver::default());
|
||||
let _observer_owner = observer_index
|
||||
.register(generation.id().clone(), observer.clone())
|
||||
.unwrap();
|
||||
let providers = ProviderRegistry::new([provider.clone() as Arc<dyn Provider>]).unwrap();
|
||||
let service = DefaultExecutionService::new(
|
||||
RuntimeSnapshotHandle::new(fallback_observation_snapshot(generation)),
|
||||
Arc::new(TrackingExecutionStore::default()),
|
||||
providers,
|
||||
Arc::new(Admissions::default()),
|
||||
Arc::new(UnusedContinuation),
|
||||
Arc::new(RecordingClientApiKeyUsage::default()),
|
||||
Arc::new(crate::RecordingDiagnostics::default()),
|
||||
)
|
||||
.with_execution_extensions(extension_index)
|
||||
.with_request_observers(observer_index);
|
||||
Fixture {
|
||||
service,
|
||||
provider,
|
||||
observer,
|
||||
caught,
|
||||
_extensions: extension_owner,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outbound_external_effect_is_preserved_when_preparation_fails() {
|
||||
block_on(async {
|
||||
let fixture = fixture(false);
|
||||
let mut started = fixture
|
||||
.service
|
||||
.start(request(&fixture.service, ClientTransport::HttpJson))
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(
|
||||
matches!(started.session.collect_uncommitted().await, Err(EngineError::Provider(error)) if error.kind() == ProviderErrorKind::Protocol)
|
||||
);
|
||||
started.session.detach_finalize().await;
|
||||
assert_eq!(fixture.provider.calls.load(Ordering::SeqCst), 1);
|
||||
let observations = fixture.observer.observations.lock().unwrap();
|
||||
assert_eq!(observations[0].send_state(), UpstreamSendState::Ambiguous);
|
||||
assert_eq!(observations[0].attempt_count(), 1);
|
||||
});
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn attempt_cannot_hide_outbound_external_effect_with_retryable_not_sent_error() {
|
||||
block_on(async {
|
||||
let fixture = fixture(true);
|
||||
let mut started = fixture
|
||||
.service
|
||||
.start(request(&fixture.service, ClientTransport::HttpJson))
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(
|
||||
matches!(started.session.collect_uncommitted().await, Err(EngineError::Provider(error)) if error.kind() == ProviderErrorKind::Transport && !error.retry_is_prohibited())
|
||||
);
|
||||
started.session.detach_finalize().await;
|
||||
assert_eq!(fixture.caught.load(Ordering::SeqCst), 1);
|
||||
assert_eq!(fixture.provider.calls.load(Ordering::SeqCst), 1);
|
||||
let observations = fixture.observer.observations.lock().unwrap();
|
||||
assert_eq!(observations[0].send_state(), UpstreamSendState::Ambiguous);
|
||||
assert_eq!(observations[0].attempt_count(), 1);
|
||||
});
|
||||
}
|
||||
@@ -3,7 +3,7 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use gateway_core::engine::extensions::{ExecutionExtensionIndex, ExecutionExtensionPlans};
|
||||
use gateway_core::engine::upstream_adapter::{UpstreamAdapter, UpstreamAdapterPlan};
|
||||
use gateway_core::engine::upstream_adapter::{UpstreamAdapterPlan, UpstreamAdapterSelection};
|
||||
use gateway_core::routing::extensions::{ExtensionSetId, ExtensionSetLease, ExtensionSetReference};
|
||||
use gateway_core::{
|
||||
engine::AttemptContext, error::ProviderError, identity::ProviderKind, routing::UpstreamModelId,
|
||||
@@ -31,8 +31,9 @@ impl UpstreamAdapterPlan for EmptyAdapterPlan {
|
||||
_: &AttemptContext,
|
||||
_: &ProviderKind,
|
||||
_: &UpstreamModelId,
|
||||
) -> Result<Option<Arc<dyn UpstreamAdapter>>, ProviderError> {
|
||||
Ok(None)
|
||||
_: Option<&gateway_core::account::ProviderAccountId>,
|
||||
) -> Result<UpstreamAdapterSelection, ProviderError> {
|
||||
Ok(UpstreamAdapterSelection::Native)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -89,3 +90,89 @@ fn execution_index_does_not_own_registered_generations() {
|
||||
assert!(index.middleware(&reference).is_none());
|
||||
assert!(index.upstream_adapters(&reference).is_none());
|
||||
}
|
||||
|
||||
#[derive(Debug, Default)]
|
||||
struct AccountSelectionProbe(std::sync::Mutex<Vec<Option<String>>>);
|
||||
|
||||
impl UpstreamAdapterPlan for AccountSelectionProbe {
|
||||
fn select(
|
||||
&self,
|
||||
_: &AttemptContext,
|
||||
_: &ProviderKind,
|
||||
_: &UpstreamModelId,
|
||||
account: Option<&gateway_core::account::ProviderAccountId>,
|
||||
) -> Result<UpstreamAdapterSelection, ProviderError> {
|
||||
self.0
|
||||
.lock()
|
||||
.unwrap()
|
||||
.push(account.map(|account| account.as_str().into()));
|
||||
Ok(UpstreamAdapterSelection::Native)
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn adapter_preselection_follows_only_the_current_attempt_account_owner() {
|
||||
use gateway_core::{
|
||||
account::{AccountSelectionPolicy, ProviderAccountId, RotationStrategy},
|
||||
engine::continuation::{ContinuationBinding, NativeContinuationPin, PreviousResponseId},
|
||||
engine::upstream_adapter::FrozenUpstreamAdapterPlan,
|
||||
engine::{
|
||||
AccountAttemptContext, ContinuationAttempt, ModelRequestId, RequestAttemptContext,
|
||||
},
|
||||
lifecycle::CancellationToken,
|
||||
policy::ClientApiKeyId,
|
||||
};
|
||||
use std::{
|
||||
num::NonZeroU32,
|
||||
time::{Duration, SystemTime},
|
||||
};
|
||||
|
||||
let provider = ProviderKind::new("openai").unwrap();
|
||||
let model = UpstreamModelId::new("model").unwrap();
|
||||
let key = ClientApiKeyId::new("key").unwrap();
|
||||
let pin = NativeContinuationPin::new(
|
||||
PreviousResponseId::new("public"),
|
||||
PreviousResponseId::new("upstream"),
|
||||
key.clone(),
|
||||
provider.clone(),
|
||||
ProviderAccountId::new("acct_pinned").unwrap(),
|
||||
);
|
||||
for (attempt, required, expected) in [
|
||||
(ContinuationAttempt::Native, None, Some("acct_pinned")),
|
||||
(ContinuationAttempt::ReplayAny, None, None),
|
||||
(ContinuationAttempt::None, None, None),
|
||||
(
|
||||
ContinuationAttempt::Native,
|
||||
Some("acct_required"),
|
||||
Some("acct_required"),
|
||||
),
|
||||
] {
|
||||
let probe = Arc::new(AccountSelectionProbe::default());
|
||||
let context = AttemptContext::new(
|
||||
RequestAttemptContext::new(
|
||||
ModelRequestId::new("req_account_plan").unwrap(),
|
||||
key.clone(),
|
||||
)
|
||||
.with_upstream_adapters(Some(FrozenUpstreamAdapterPlan::new(
|
||||
probe.clone(),
|
||||
reference("account-plan"),
|
||||
))),
|
||||
NonZeroU32::MIN,
|
||||
SystemTime::now() + Duration::from_secs(5),
|
||||
AccountSelectionPolicy::new(RotationStrategy::RoundRobin, NonZeroU32::MIN),
|
||||
AccountAttemptContext::new(
|
||||
Default::default(),
|
||||
required.map(|id| ProviderAccountId::new(id).unwrap()),
|
||||
None,
|
||||
),
|
||||
Some(ContinuationBinding::Pinned(pin.clone())),
|
||||
CancellationToken::new(),
|
||||
)
|
||||
.with_continuation_attempt(attempt);
|
||||
assert!(matches!(
|
||||
context.upstream_adapter(&provider, &model).unwrap(),
|
||||
UpstreamAdapterSelection::Native
|
||||
));
|
||||
assert_eq!(*probe.0.lock().unwrap(), vec![expected.map(str::to_owned)]);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -88,7 +88,7 @@ fn plan(action: Action) -> FrozenMiddlewarePlan {
|
||||
}
|
||||
|
||||
fn terminal(polls: Arc<AtomicUsize>, released: Arc<AtomicUsize>) -> ProviderMiddlewareTerminal {
|
||||
Box::new(move |_, _| {
|
||||
Box::new(move |_, _, _| {
|
||||
Box::pin(async move {
|
||||
let metadata = ProviderCallMetadata::new(
|
||||
ProviderKind::new("openai").unwrap(),
|
||||
@@ -127,6 +127,7 @@ fn inspecting_a_response_waits_for_the_outer_stream_and_keeps_send_facts() {
|
||||
context(),
|
||||
operation(),
|
||||
ClientTransport::HttpJson,
|
||||
None,
|
||||
terminal(polls.clone(), released.clone()),
|
||||
)
|
||||
.await
|
||||
@@ -165,6 +166,7 @@ fn rejecting_metadata_drops_the_unactivated_handler_and_account_lease() {
|
||||
context(),
|
||||
operation(),
|
||||
ClientTransport::HttpJson,
|
||||
None,
|
||||
terminal(polls.clone(), released.clone()),
|
||||
)
|
||||
.await
|
||||
@@ -194,7 +196,8 @@ fn middleware_rejection_before_next_does_not_wait_for_activation() {
|
||||
context(),
|
||||
operation(),
|
||||
ClientTransport::HttpJson,
|
||||
Box::new(|_, _| panic!("rejected middleware cannot prepare upstream")),
|
||||
None,
|
||||
Box::new(|_, _, _| panic!("rejected middleware cannot prepare upstream")),
|
||||
)
|
||||
.await
|
||||
.err()
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
//! 请求中间件的改写、交付状态与发布代次持有测试
|
||||
|
||||
mod cold;
|
||||
mod outbound;
|
||||
|
||||
use std::num::NonZeroU32;
|
||||
use std::sync::Arc;
|
||||
@@ -339,7 +340,7 @@ fn attempt_middleware_consumes_owned_next_once_and_preserves_host_envelopes() {
|
||||
|
||||
let terminal: ProviderMiddlewareTerminal = {
|
||||
let terminal_calls = Arc::clone(&terminal_calls);
|
||||
Box::new(move |operation, headers| {
|
||||
Box::new(move |operation, headers, _| {
|
||||
terminal_calls.fetch_add(1, Ordering::SeqCst);
|
||||
Box::pin(async move {
|
||||
assert!(headers.is_empty());
|
||||
@@ -391,6 +392,7 @@ fn attempt_middleware_consumes_owned_next_once_and_preserves_host_envelopes() {
|
||||
context(),
|
||||
operation(),
|
||||
ClientTransport::HttpJson,
|
||||
None,
|
||||
terminal,
|
||||
)
|
||||
.await
|
||||
@@ -485,7 +487,7 @@ fn attempt_passthrough_preserves_parsed_wire_for_every_client_transport() {
|
||||
payload.clone(),
|
||||
)
|
||||
.unwrap();
|
||||
let terminal: ProviderMiddlewareTerminal = Box::new(move |_, _| {
|
||||
let terminal: ProviderMiddlewareTerminal = Box::new(move |_, _, _| {
|
||||
Box::pin(async move {
|
||||
Ok(ProviderStream::new(
|
||||
empty_provider_stream().metadata().clone(),
|
||||
@@ -499,6 +501,7 @@ fn attempt_passthrough_preserves_parsed_wire_for_every_client_transport() {
|
||||
context(),
|
||||
operation(),
|
||||
transport,
|
||||
None,
|
||||
terminal,
|
||||
)
|
||||
.await
|
||||
@@ -566,7 +569,7 @@ fn attempt_sse_encoding_keeps_opaque_metadata_from_creating_extra_frames() {
|
||||
Some(10),
|
||||
)
|
||||
.unwrap();
|
||||
let terminal: ProviderMiddlewareTerminal = Box::new(move |_, _| {
|
||||
let terminal: ProviderMiddlewareTerminal = Box::new(move |_, _, _| {
|
||||
Box::pin(async move {
|
||||
Ok(ProviderStream::new(
|
||||
empty_provider_stream().metadata().clone(),
|
||||
@@ -581,6 +584,7 @@ fn attempt_sse_encoding_keeps_opaque_metadata_from_creating_extra_frames() {
|
||||
context(),
|
||||
operation(),
|
||||
ClientTransport::HttpSse,
|
||||
None,
|
||||
terminal,
|
||||
)
|
||||
.await
|
||||
@@ -595,7 +599,7 @@ fn attempt_middleware_preserves_multi_value_and_authentication_headers() {
|
||||
let allowed_calls = Arc::new(AtomicUsize::new(0));
|
||||
let allowed_terminal: ProviderMiddlewareTerminal = {
|
||||
let allowed_calls = Arc::clone(&allowed_calls);
|
||||
Box::new(move |_operation, headers| {
|
||||
Box::new(move |_operation, headers, _| {
|
||||
allowed_calls.fetch_add(1, Ordering::SeqCst);
|
||||
Box::pin(async move {
|
||||
assert_eq!(headers.len(), 2);
|
||||
@@ -623,6 +627,7 @@ fn attempt_middleware_preserves_multi_value_and_authentication_headers() {
|
||||
context(),
|
||||
operation(),
|
||||
ClientTransport::HttpJson,
|
||||
None,
|
||||
allowed_terminal,
|
||||
)
|
||||
.await
|
||||
@@ -632,7 +637,7 @@ fn attempt_middleware_preserves_multi_value_and_authentication_headers() {
|
||||
let calls = Arc::new(AtomicUsize::new(0));
|
||||
let terminal: ProviderMiddlewareTerminal = {
|
||||
let calls = calls.clone();
|
||||
Box::new(move |_, headers| {
|
||||
Box::new(move |_, headers, _| {
|
||||
calls.fetch_add(1, Ordering::SeqCst);
|
||||
Box::pin(async move {
|
||||
assert_eq!(headers[0].name(), "authorization");
|
||||
@@ -656,6 +661,7 @@ fn attempt_middleware_preserves_multi_value_and_authentication_headers() {
|
||||
context(),
|
||||
operation(),
|
||||
ClientTransport::HttpJson,
|
||||
None,
|
||||
terminal,
|
||||
)
|
||||
.await
|
||||
@@ -735,7 +741,8 @@ fn attempt_middleware_cannot_add_image_generation_to_a_denied_scope() {
|
||||
ProviderKind::new("openai").unwrap(),
|
||||
Some("gpt-test".into()),
|
||||
ProviderAccountId::new("acct_middleware").unwrap(),
|
||||
Box::new(move |operation, _| {
|
||||
None,
|
||||
Box::new(move |operation, _, _| {
|
||||
Box::pin(async move {
|
||||
assert!(operation.image_generation_requested());
|
||||
count.fetch_add(1, Ordering::SeqCst);
|
||||
|
||||
@@ -0,0 +1,303 @@
|
||||
//! 出站准备链的返回顺序、预算、生命周期与不可修改归属
|
||||
|
||||
use super::*;
|
||||
use gateway_core::account::{AccountSelectionPolicy, RotationStrategy};
|
||||
use gateway_core::engine::middleware::outbound::{Pipeline, Request, Target};
|
||||
use gateway_core::engine::{AccountAttemptContext, AttemptContext, RequestAttemptContext};
|
||||
use gateway_core::error::ProviderErrorKind;
|
||||
use gateway_core::upstream::UpstreamSendState;
|
||||
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
enum Action {
|
||||
Forward,
|
||||
Rewrite,
|
||||
Oversize,
|
||||
SwitchTransport,
|
||||
Reject,
|
||||
Pending,
|
||||
Cancel,
|
||||
}
|
||||
|
||||
#[derive(Debug)]
|
||||
struct PreparingPlan {
|
||||
action: Action,
|
||||
calls: Arc<AtomicUsize>,
|
||||
}
|
||||
|
||||
impl MiddlewarePlan for PreparingPlan {
|
||||
fn has_outbound(&self, context: &MiddlewareContext) -> bool {
|
||||
context.model() == Some("gpt-test")
|
||||
}
|
||||
|
||||
fn handle_outbound(
|
||||
&self,
|
||||
context: MiddlewareContext,
|
||||
request: Request,
|
||||
next: gateway_core::engine::middleware::outbound::Next,
|
||||
) -> BoxFuture<'static, Result<Request, MiddlewareError>> {
|
||||
assert_eq!(context.mount(), MiddlewareMount::Outbound);
|
||||
assert_eq!(context.provider().unwrap().as_str(), "openai");
|
||||
assert_eq!(context.account_id().unwrap().as_str(), "acct_middleware");
|
||||
assert_eq!(context.model(), Some("gpt-test"));
|
||||
assert_eq!(context.attempt_index(), NonZeroU32::new(2));
|
||||
assert_eq!(context.request_id().as_str(), "req_outbound");
|
||||
assert_eq!(context.client_key_id().as_str(), "key_outbound");
|
||||
assert!(context.plan().is_some());
|
||||
self.calls.fetch_add(1, Ordering::SeqCst);
|
||||
let action = self.action;
|
||||
Box::pin(async move {
|
||||
let mut request = next.run(request).await?;
|
||||
match action {
|
||||
Action::Forward => (),
|
||||
Action::Rewrite => {
|
||||
let mut body = request.body.to_vec();
|
||||
body.extend_from_slice(b"-after-next");
|
||||
request.body = body.into();
|
||||
request
|
||||
.headers
|
||||
.push(MiddlewareHeader::new("x-raw", Bytes::from_static(&[0x80])));
|
||||
}
|
||||
Action::Oversize => request.body = Bytes::from_static(b"too large"),
|
||||
Action::SwitchTransport => {
|
||||
request.target = Target::WebSocket {
|
||||
uri: "wss://example.test/".parse().unwrap(),
|
||||
}
|
||||
}
|
||||
Action::Reject => return Err(MiddlewareError::Rejected),
|
||||
Action::Pending => return futures::future::pending().await,
|
||||
Action::Cancel => {
|
||||
context.cancellation().cancel();
|
||||
return futures::future::pending().await;
|
||||
}
|
||||
}
|
||||
Ok(request)
|
||||
})
|
||||
}
|
||||
|
||||
fn handle(
|
||||
&self,
|
||||
_: MiddlewareContext,
|
||||
request: MiddlewareRequest,
|
||||
next: MiddlewareNext,
|
||||
) -> BoxFuture<'static, Result<MiddlewareResponse, MiddlewareError>> {
|
||||
next.run(request)
|
||||
}
|
||||
}
|
||||
|
||||
fn pipeline(
|
||||
action: Action,
|
||||
limit: usize,
|
||||
cancellation: CancellationToken,
|
||||
deadline: SystemTime,
|
||||
) -> (Pipeline, Arc<AtomicUsize>) {
|
||||
let (attempt, calls) = attempt(action, limit, cancellation, deadline);
|
||||
(
|
||||
attempt
|
||||
.outbound_middleware(
|
||||
ProviderKind::new("openai").unwrap(),
|
||||
Some("gpt-test".into()),
|
||||
ProviderAccountId::new("acct_middleware").unwrap(),
|
||||
)
|
||||
.unwrap(),
|
||||
calls,
|
||||
)
|
||||
}
|
||||
|
||||
fn attempt(
|
||||
action: Action,
|
||||
limit: usize,
|
||||
cancellation: CancellationToken,
|
||||
deadline: SystemTime,
|
||||
) -> (AttemptContext, Arc<AtomicUsize>) {
|
||||
let calls = Arc::new(AtomicUsize::new(0));
|
||||
let plan = FrozenMiddlewarePlan::new(
|
||||
Arc::new(PreparingPlan {
|
||||
action,
|
||||
calls: Arc::clone(&calls),
|
||||
}),
|
||||
ExtensionSetReference::new(
|
||||
ExtensionSetId::new("outbound-test".into()).unwrap(),
|
||||
Arc::new(ReadyGeneration),
|
||||
),
|
||||
);
|
||||
let request = RequestAttemptContext::new(
|
||||
ModelRequestId::new("req_outbound").unwrap(),
|
||||
ClientApiKeyId::new("key_outbound").unwrap(),
|
||||
)
|
||||
.with_request_body_limit(limit)
|
||||
.with_middleware(
|
||||
Some(plan),
|
||||
Arc::from([]),
|
||||
"/v1/responses".into(),
|
||||
ClientTransport::HttpJson,
|
||||
);
|
||||
let attempt = AttemptContext::new(
|
||||
request,
|
||||
NonZeroU32::new(2).unwrap(),
|
||||
deadline,
|
||||
AccountSelectionPolicy::new(RotationStrategy::Smart, NonZeroU32::new(1).unwrap()),
|
||||
AccountAttemptContext::new(Default::default(), None, None),
|
||||
None,
|
||||
cancellation,
|
||||
);
|
||||
(attempt, calls)
|
||||
}
|
||||
|
||||
fn request() -> Request {
|
||||
Request {
|
||||
target: Target::Http {
|
||||
method: http::Method::POST,
|
||||
uri: "https://example.test/path?secret=private".parse().unwrap(),
|
||||
},
|
||||
headers: vec![MiddlewareHeader::new("x-raw", Bytes::from_static(b"first"))],
|
||||
body: Bytes::from_static(&[0, 255, 0, 128]),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outbound_returns_outer_rewrite_after_next_without_decoding_body() {
|
||||
let (pipeline, calls) = pipeline(
|
||||
Action::Rewrite,
|
||||
128,
|
||||
CancellationToken::new(),
|
||||
SystemTime::now() + Duration::from_secs(1),
|
||||
);
|
||||
let prepared = futures::executor::block_on(pipeline.prepare(request())).unwrap();
|
||||
assert_eq!(
|
||||
prepared.body.as_ref(),
|
||||
&[
|
||||
0, 255, 0, 128, b'-', b'a', b'f', b't', b'e', b'r', b'-', b'n', b'e', b'x', b't'
|
||||
]
|
||||
);
|
||||
let headers = prepared.header_map().unwrap();
|
||||
assert_eq!(
|
||||
headers
|
||||
.get_all("x-raw")
|
||||
.iter()
|
||||
.map(http::HeaderValue::as_bytes)
|
||||
.collect::<Vec<_>>(),
|
||||
vec![b"first".as_slice(), &[0x80]]
|
||||
);
|
||||
assert_eq!(calls.load(Ordering::SeqCst), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outbound_checks_final_body_limit_without_repair_or_retry() {
|
||||
let (pipeline, _) = pipeline(
|
||||
Action::Oversize,
|
||||
4,
|
||||
CancellationToken::new(),
|
||||
SystemTime::now() + Duration::from_secs(1),
|
||||
);
|
||||
let error = futures::executor::block_on(pipeline.prepare(request())).unwrap_err();
|
||||
assert_eq!(error.kind(), ProviderErrorKind::InvalidRequest);
|
||||
assert_eq!(error.send_state(), UpstreamSendState::NotSent);
|
||||
assert!(error.retry_is_prohibited());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outbound_cannot_switch_transport_kind() {
|
||||
let (pipeline, _) = pipeline(
|
||||
Action::SwitchTransport,
|
||||
128,
|
||||
CancellationToken::new(),
|
||||
SystemTime::now() + Duration::from_secs(1),
|
||||
);
|
||||
let error = futures::executor::block_on(pipeline.prepare(request())).unwrap_err();
|
||||
assert_eq!(error.kind(), ProviderErrorKind::InvalidRequest);
|
||||
assert!(error.retry_is_prohibited());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outbound_rejection_is_terminal_and_not_sent() {
|
||||
let (pipeline, _) = pipeline(
|
||||
Action::Reject,
|
||||
128,
|
||||
CancellationToken::new(),
|
||||
SystemTime::now() + Duration::from_secs(1),
|
||||
);
|
||||
let error = futures::executor::block_on(pipeline.prepare(request())).unwrap_err();
|
||||
assert_eq!(error.kind(), ProviderErrorKind::RequestPolicyDenied);
|
||||
assert_eq!(error.send_state(), UpstreamSendState::NotSent);
|
||||
assert!(error.retry_is_prohibited());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outbound_cancelled_attempt_never_enters_middleware() {
|
||||
let cancelled = CancellationToken::new();
|
||||
cancelled.cancel();
|
||||
let (pipeline, calls) = pipeline(
|
||||
Action::Forward,
|
||||
128,
|
||||
cancelled,
|
||||
SystemTime::now() + Duration::from_secs(1),
|
||||
);
|
||||
let error = futures::executor::block_on(pipeline.prepare(request())).unwrap_err();
|
||||
assert_eq!(error.kind(), ProviderErrorKind::Cancelled);
|
||||
assert_eq!(calls.load(Ordering::SeqCst), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outbound_deadline_drops_a_pending_preparation() {
|
||||
let (pipeline, calls) = pipeline(
|
||||
Action::Pending,
|
||||
128,
|
||||
CancellationToken::new(),
|
||||
SystemTime::now() + Duration::from_secs(1),
|
||||
);
|
||||
let error = futures::executor::block_on(pipeline.prepare(request())).unwrap_err();
|
||||
assert_eq!(error.kind(), ProviderErrorKind::Timeout);
|
||||
assert_eq!(calls.load(Ordering::SeqCst), 1);
|
||||
assert!(error.retry_is_prohibited());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outbound_header_conversion_rejects_invalid_bytes_without_retry() {
|
||||
let mut request = request();
|
||||
request.headers.push(MiddlewareHeader::new(
|
||||
"x-invalid",
|
||||
Bytes::from_static(b"bad\r\nvalue"),
|
||||
));
|
||||
let error = request.header_map().unwrap_err();
|
||||
assert_eq!(error.send_state(), UpstreamSendState::NotSent);
|
||||
assert!(error.retry_is_prohibited());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outbound_debug_does_not_expose_uri_headers_or_body() {
|
||||
let debug = format!("{:?}", request());
|
||||
assert!(!debug.contains("private"));
|
||||
assert!(!debug.contains("first"));
|
||||
assert!(!debug.contains("255"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outbound_cancellation_stops_an_in_flight_preparation() {
|
||||
let (pipeline, calls) = pipeline(
|
||||
Action::Cancel,
|
||||
128,
|
||||
CancellationToken::new(),
|
||||
SystemTime::now() + Duration::from_secs(1),
|
||||
);
|
||||
let error = futures::executor::block_on(pipeline.prepare(request())).unwrap_err();
|
||||
assert_eq!(error.kind(), ProviderErrorKind::Cancelled);
|
||||
assert_eq!(calls.load(Ordering::SeqCst), 1);
|
||||
assert!(error.retry_is_prohibited());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outbound_scope_miss_keeps_the_provider_on_its_native_path() {
|
||||
let (attempt, calls) = attempt(
|
||||
Action::Reject,
|
||||
0,
|
||||
CancellationToken::new(),
|
||||
SystemTime::now() + Duration::from_secs(1),
|
||||
);
|
||||
let pipeline = attempt.outbound_middleware(
|
||||
ProviderKind::new("openai").unwrap(),
|
||||
Some("unmatched".into()),
|
||||
ProviderAccountId::new("acct_middleware").unwrap(),
|
||||
);
|
||||
assert!(pipeline.is_none());
|
||||
assert_eq!(calls.load(Ordering::SeqCst), 0);
|
||||
}
|
||||
@@ -8,8 +8,8 @@
|
||||
{ "capability": "model_router", "versions": [2] },
|
||||
{ "capability": "model_catalog", "versions": [1] },
|
||||
{ "capability": "retry_policy", "versions": [2] },
|
||||
{ "capability": "middleware", "versions": [5] },
|
||||
{ "capability": "upstream_adapter", "versions": [4] },
|
||||
{ "capability": "middleware", "versions": [6] },
|
||||
{ "capability": "upstream_adapter", "versions": [5] },
|
||||
{ "capability": "observer", "versions": [2] },
|
||||
{ "capability": "command_line", "versions": [1] },
|
||||
{ "capability": "management", "versions": [2] },
|
||||
|
||||
@@ -51,6 +51,35 @@ impl PluginMiddlewarePlan {
|
||||
}
|
||||
|
||||
impl MiddlewarePlan for PluginMiddlewarePlan {
|
||||
fn has_outbound(&self, context: &MiddlewareContext) -> bool {
|
||||
self.middleware
|
||||
.get(&MiddlewareMount::Outbound)
|
||||
.is_some_and(|entries| entries.iter().any(|entry| outbound_matches(entry, context)))
|
||||
}
|
||||
|
||||
fn handle_outbound(
|
||||
&self,
|
||||
context: MiddlewareContext,
|
||||
request: gateway_core::engine::middleware::outbound::Request,
|
||||
next: gateway_core::engine::middleware::outbound::Next,
|
||||
) -> BoxFuture<
|
||||
'static,
|
||||
Result<gateway_core::engine::middleware::outbound::Request, MiddlewareError>,
|
||||
> {
|
||||
self.chain(
|
||||
MiddlewareMount::Outbound,
|
||||
context,
|
||||
outbound_matches,
|
||||
|entry, context, request, next, timeout| {
|
||||
Box::pin(super::outbound::invoke(
|
||||
entry, context, request, next, timeout,
|
||||
))
|
||||
},
|
||||
next,
|
||||
)
|
||||
.run(request)
|
||||
}
|
||||
|
||||
fn has_service(&self) -> bool {
|
||||
self.middleware.contains_key(&MiddlewareMount::Service)
|
||||
}
|
||||
@@ -154,3 +183,14 @@ impl MiddlewarePlan for PluginMiddlewarePlan {
|
||||
.run(request)
|
||||
}
|
||||
}
|
||||
|
||||
// 存在性检查与实际组合使用同一选择规则,未命中时不进入原生报文准备
|
||||
fn outbound_matches(entry: &MiddlewareEntry, context: &MiddlewareContext) -> bool {
|
||||
!context.extension_scope().contains(&entry.instance_id)
|
||||
&& entry.scope.matches_processing(
|
||||
context.client_key_id(),
|
||||
context.account_group_ids(),
|
||||
context.provider(),
|
||||
context.model(),
|
||||
)
|
||||
}
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
mod execution;
|
||||
mod http;
|
||||
mod outbound;
|
||||
mod request;
|
||||
mod service;
|
||||
mod websocket;
|
||||
@@ -56,11 +57,16 @@ pub(crate) fn validate_bindings(
|
||||
}
|
||||
if !matches!(
|
||||
stage,
|
||||
Stage::Http | Stage::WebSocket | Stage::Service | Stage::Request | Stage::Attempt
|
||||
Stage::Http
|
||||
| Stage::WebSocket
|
||||
| Stage::Service
|
||||
| Stage::Request
|
||||
| Stage::Attempt
|
||||
| Stage::Outbound
|
||||
) || !stages.insert(stage)
|
||||
{
|
||||
return Err(AdminError::invalid(
|
||||
"同一插件实例的中间件只能在 http/websocket/service/request/attempt 各绑定一次",
|
||||
"同一插件实例的中间件只能在 http/websocket/service/request/attempt/outbound 各绑定一次",
|
||||
));
|
||||
}
|
||||
if matches!(stage, Stage::Http | Stage::WebSocket | Stage::Service)
|
||||
@@ -117,7 +123,7 @@ pub(crate) fn unavailable_entries(
|
||||
.filter(|binding| {
|
||||
matches!(
|
||||
binding.stage.as_str(),
|
||||
"http" | "websocket" | "service" | "request" | "attempt"
|
||||
"http" | "websocket" | "service" | "request" | "attempt" | "outbound"
|
||||
)
|
||||
})
|
||||
.map(|binding| {
|
||||
@@ -151,6 +157,7 @@ fn compile_entry(
|
||||
"websocket" => MiddlewareMount::WebSocket,
|
||||
"request" => MiddlewareMount::Request,
|
||||
"attempt" => MiddlewareMount::Attempt,
|
||||
"outbound" => MiddlewareMount::Outbound,
|
||||
_ => return Err(AdminError::invalid("插件中间件阶段无效")),
|
||||
},
|
||||
invocation,
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
//! 在 Provider 发送前组合完整出站请求,并沿用模型数据面的调用资源
|
||||
|
||||
use std::{sync::Arc, time::Duration};
|
||||
|
||||
use gateway_admin::model::plugins::instances::PluginFailurePolicy;
|
||||
use gateway_core::engine::middleware::{MiddlewareContext, MiddlewareError, outbound as core};
|
||||
use gateway_plugin_sdk::{
|
||||
Stage,
|
||||
call::middleware::{HANDLE_METHOD, outbound as wire},
|
||||
};
|
||||
|
||||
use super::MiddlewareEntry;
|
||||
use crate::callback::outbound::{self, Invocation};
|
||||
|
||||
pub(super) async fn invoke(
|
||||
entry: Arc<MiddlewareEntry>,
|
||||
context: MiddlewareContext,
|
||||
request: core::Request,
|
||||
next: core::Next,
|
||||
timeout: Duration,
|
||||
) -> Result<core::Request, MiddlewareError> {
|
||||
let Some(ports) = &entry.invocation else {
|
||||
return if entry.failure_policy == PluginFailurePolicy::Delegate {
|
||||
next.run(request).await
|
||||
} else {
|
||||
Err(MiddlewareError::Fault)
|
||||
};
|
||||
};
|
||||
let invocation = Invocation::new(next);
|
||||
let input = wire::CallHead {
|
||||
request: outbound::head(&request),
|
||||
metadata: wire::Metadata {
|
||||
request_id: context.request_id().as_str().to_owned(),
|
||||
client_key_id: context.client_key_id().as_str().to_owned(),
|
||||
account_group_ids: context
|
||||
.account_group_ids()
|
||||
.iter()
|
||||
.map(|id| id.as_str().to_owned())
|
||||
.collect(),
|
||||
provider: context
|
||||
.provider()
|
||||
.ok_or(MiddlewareError::InvalidState)?
|
||||
.as_str()
|
||||
.to_owned(),
|
||||
model: context.model().map(str::to_owned),
|
||||
account_id: context
|
||||
.account_id()
|
||||
.ok_or(MiddlewareError::InvalidState)?
|
||||
.as_str()
|
||||
.to_owned(),
|
||||
attempt_index: context
|
||||
.attempt_index()
|
||||
.ok_or(MiddlewareError::InvalidState)?
|
||||
.get(),
|
||||
},
|
||||
};
|
||||
let mut call = ports.session.context(Stage::Outbound, timeout);
|
||||
call.request = crate::adapter::request_context(context.extension_scope().request());
|
||||
call.request_id = Some(input.metadata.request_id.clone());
|
||||
call.attempt_id = Some(format!(
|
||||
"{}:{}",
|
||||
input.metadata.request_id, input.metadata.attempt_index
|
||||
));
|
||||
call.account_id = Some(input.metadata.account_id.clone());
|
||||
let scope = ports
|
||||
.callbacks
|
||||
.prepare_data_plane(
|
||||
&call,
|
||||
context.execution_effects(),
|
||||
context.extension_scope().clone(),
|
||||
context.plan(),
|
||||
context.cancellation().clone(),
|
||||
context.request_body_limit(),
|
||||
)
|
||||
.map_err(|_| MiddlewareError::Fault)?;
|
||||
let _binding =
|
||||
ports
|
||||
.callbacks
|
||||
.bind_middleware(call.resource_scope_id.clone(), invocation, scope)?;
|
||||
let reply = tokio::select! {
|
||||
biased;
|
||||
() = context.cancellation().cancelled() => return Err(MiddlewareError::Fault),
|
||||
() = context.deadline().wait() => return Err(gateway_core::error::GatewayError::new(
|
||||
gateway_core::error::GatewayErrorKind::Timeout,
|
||||
"request deadline elapsed",
|
||||
).into()),
|
||||
reply = ports.session.call(
|
||||
HANDLE_METHOD,
|
||||
call,
|
||||
serde_json::to_value(input).map_err(|_| MiddlewareError::InvalidState)?,
|
||||
request.body.into(),
|
||||
) => reply.map_err(crate::callback::error::rpc_middleware)?,
|
||||
};
|
||||
let head = ports
|
||||
.session
|
||||
.decode_response(Stage::Outbound, reply.result)
|
||||
.map_err(|_| MiddlewareError::InvalidState)?;
|
||||
outbound::decode(head, reply.payload).map_err(|_| MiddlewareError::InvalidState)
|
||||
}
|
||||
@@ -61,7 +61,10 @@ pub(super) async fn invoke(
|
||||
};
|
||||
let mut call_context = invocation_ports.session.context(
|
||||
match entry.mount {
|
||||
MiddlewareMount::Http | MiddlewareMount::WebSocket | MiddlewareMount::Service => {
|
||||
MiddlewareMount::Http
|
||||
| MiddlewareMount::WebSocket
|
||||
| MiddlewareMount::Service
|
||||
| MiddlewareMount::Outbound => {
|
||||
return Err(MiddlewareError::InvalidState);
|
||||
}
|
||||
MiddlewareMount::Request => Stage::Request,
|
||||
@@ -95,6 +98,7 @@ pub(super) async fn invoke(
|
||||
scope,
|
||||
)?;
|
||||
let head = MiddlewareRequestHead {
|
||||
provider_parameters: invocation.provider_parameters(),
|
||||
settings_sources: invocation.settings_sources(),
|
||||
settings: invocation
|
||||
.settings()
|
||||
@@ -107,7 +111,10 @@ pub(super) async fn invoke(
|
||||
.collect(),
|
||||
request_id: context.request_id().as_str().to_owned(),
|
||||
mount: match context.mount() {
|
||||
MiddlewareMount::Http | MiddlewareMount::WebSocket | MiddlewareMount::Service => {
|
||||
MiddlewareMount::Http
|
||||
| MiddlewareMount::WebSocket
|
||||
| MiddlewareMount::Service
|
||||
| MiddlewareMount::Outbound => {
|
||||
return Err(MiddlewareError::InvalidState);
|
||||
}
|
||||
MiddlewareMount::Request => WireMiddlewareMount::Request,
|
||||
|
||||
@@ -152,6 +152,11 @@ impl Execution {
|
||||
if invocation.operation.protocol() != declaration.protocol
|
||||
|| invocation.metadata.provider().as_str() != declaration.provider.as_str()
|
||||
|| invocation.metadata.provider_account_id() != invocation.account.account_id()
|
||||
|| (!declaration.account_ids.is_empty()
|
||||
&& !declaration
|
||||
.account_ids
|
||||
.iter()
|
||||
.any(|id| id == invocation.account.account_id().as_str()))
|
||||
|| !declaration
|
||||
.authentication_kinds
|
||||
.iter()
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
//! 将插件上游适配贡献编译为可按请求选择的执行计划
|
||||
//! 将插件上游适配贡献编译为按请求与账号范围解析的执行计划
|
||||
|
||||
mod event;
|
||||
mod execution;
|
||||
@@ -10,9 +10,13 @@ pub(crate) mod target;
|
||||
use std::{fmt, sync::Arc};
|
||||
|
||||
use gateway_core::{
|
||||
account::ProviderAccountId,
|
||||
engine::{
|
||||
AttemptContext,
|
||||
upstream_adapter::{UpstreamAdapter, UpstreamAdapterInvocation, UpstreamAdapterPlan},
|
||||
upstream_adapter::{
|
||||
UpstreamAdapter, UpstreamAdapterInvocation, UpstreamAdapterPlan,
|
||||
UpstreamAdapterSelection,
|
||||
},
|
||||
},
|
||||
error::{ProviderError, ProviderErrorKind},
|
||||
identity::ProviderKind,
|
||||
@@ -58,6 +62,13 @@ impl UpstreamAdapter for PluginUpstreamAdapter {
|
||||
self.declaration.transport.as_str()
|
||||
}
|
||||
|
||||
fn supports_authentication(&self, kind: &str) -> bool {
|
||||
self.declaration
|
||||
.authentication_kinds
|
||||
.iter()
|
||||
.any(|candidate| candidate == kind)
|
||||
}
|
||||
|
||||
fn execute(
|
||||
self: Arc<Self>,
|
||||
invocation: UpstreamAdapterInvocation,
|
||||
@@ -91,7 +102,8 @@ impl UpstreamAdapterPlan for PluginUpstreamAdapterPlan {
|
||||
context: &AttemptContext,
|
||||
provider: &ProviderKind,
|
||||
model: &UpstreamModelId,
|
||||
) -> Result<Option<Arc<dyn UpstreamAdapter>>, ProviderError> {
|
||||
account: Option<&ProviderAccountId>,
|
||||
) -> Result<UpstreamAdapterSelection, ProviderError> {
|
||||
let model = context
|
||||
.requested_model()
|
||||
.map_or(model.as_str(), |model| model.as_str());
|
||||
@@ -115,13 +127,29 @@ impl UpstreamAdapterPlan for PluginUpstreamAdapterPlan {
|
||||
.iter()
|
||||
.any(|candidate| candidate == model))
|
||||
{
|
||||
if !adapter.declaration.account_ids.is_empty() {
|
||||
let Some(account) = account else {
|
||||
// 后置声明只覆盖未命中此账号范围的候选,不能提前以其故障拒绝整个请求
|
||||
return Ok(UpstreamAdapterSelection::PendingAccount);
|
||||
};
|
||||
if !adapter
|
||||
.declaration
|
||||
.account_ids
|
||||
.iter()
|
||||
.any(|id| id == account.as_str())
|
||||
{
|
||||
continue;
|
||||
}
|
||||
}
|
||||
if !adapter.session.is_ready() {
|
||||
return Err(unavailable());
|
||||
}
|
||||
return Ok(Some(Arc::clone(adapter) as Arc<dyn UpstreamAdapter>));
|
||||
return Ok(UpstreamAdapterSelection::Adapter(
|
||||
Arc::clone(adapter) as Arc<dyn UpstreamAdapter>
|
||||
));
|
||||
}
|
||||
}
|
||||
Ok(None)
|
||||
Ok(UpstreamAdapterSelection::Native)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -135,6 +135,15 @@ pub(crate) enum MiddlewareCompletionBody {
|
||||
}
|
||||
|
||||
impl MiddlewareInvocation {
|
||||
pub(crate) fn provider_parameters(&self) -> Option<serde_json::Map<String, serde_json::Value>> {
|
||||
self.state
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||
.original_request
|
||||
.provider_parameters()
|
||||
.cloned()
|
||||
}
|
||||
|
||||
pub(crate) fn settings(&self) -> Result<serde_json::Value, PluginFault> {
|
||||
let state = self
|
||||
.state
|
||||
@@ -334,6 +343,12 @@ impl MiddlewareInvocation {
|
||||
request: MiddlewareNextRequest,
|
||||
payload: Vec<u8>,
|
||||
) -> Result<MiddlewareRequest, PluginFault> {
|
||||
if let Some(parameters) = request.provider_parameters {
|
||||
if original.provider_parameters().is_none() {
|
||||
return Err(invalid());
|
||||
}
|
||||
original = original.with_provider_parameters(parameters);
|
||||
}
|
||||
if let Some(settings) = request.settings {
|
||||
let current = original.settings().ok_or_else(|| {
|
||||
PluginFault::new(
|
||||
|
||||
@@ -14,6 +14,7 @@ mod keys;
|
||||
mod log;
|
||||
mod middleware;
|
||||
mod model;
|
||||
pub(crate) mod outbound;
|
||||
pub(crate) mod private_state;
|
||||
pub(crate) mod request_limits;
|
||||
mod resources;
|
||||
@@ -197,6 +198,7 @@ impl PluginCallbacks {
|
||||
context.stage,
|
||||
gateway_plugin_sdk::Stage::Request
|
||||
| gateway_plugin_sdk::Stage::Attempt
|
||||
| gateway_plugin_sdk::Stage::Outbound
|
||||
| gateway_plugin_sdk::Stage::Routing
|
||||
| gateway_plugin_sdk::Stage::Scheduling
|
||||
| gateway_plugin_sdk::Stage::Retry
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
//! 出站准备的单次续体与完整请求编解码,不发起网络请求
|
||||
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
use futures::future::BoxFuture;
|
||||
use gateway_core::engine::middleware::{MiddlewareHeader, outbound as core};
|
||||
use gateway_plugin_sdk::{
|
||||
PluginFault,
|
||||
call::middleware::{MiddlewareHeader as WireHeader, NEXT_METHOD, outbound as wire},
|
||||
};
|
||||
|
||||
use super::{MiddlewareCallback, invalid};
|
||||
use crate::RpcReply;
|
||||
|
||||
pub(crate) struct Invocation(Mutex<Option<core::Next>>);
|
||||
|
||||
impl Invocation {
|
||||
pub(crate) fn new(next: core::Next) -> Arc<Self> {
|
||||
Arc::new(Self(Mutex::new(Some(next))))
|
||||
}
|
||||
}
|
||||
|
||||
impl MiddlewareCallback for Invocation {
|
||||
fn invoke(
|
||||
self: Arc<Self>,
|
||||
method: String,
|
||||
params: serde_json::Value,
|
||||
payload: Vec<u8>,
|
||||
_maximum_payload: usize,
|
||||
) -> BoxFuture<'static, Result<RpcReply, PluginFault>> {
|
||||
Box::pin(async move {
|
||||
if method != NEXT_METHOD {
|
||||
return Err(invalid());
|
||||
}
|
||||
let request_head = serde_json::from_value(params).map_err(|_| invalid())?;
|
||||
let request = decode(request_head, payload)?;
|
||||
let next = self
|
||||
.0
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||
.take()
|
||||
.ok_or_else(invalid)?;
|
||||
let request = next
|
||||
.run(request)
|
||||
.await
|
||||
.map_err(|error| super::error::middleware(&error))?;
|
||||
Ok(RpcReply {
|
||||
result: serde_json::to_value(head(&request)).map_err(|_| invalid())?,
|
||||
payload: request.body.into(),
|
||||
})
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn head(request: &core::Request) -> wire::RequestHead {
|
||||
wire::RequestHead {
|
||||
target: match &request.target {
|
||||
core::Target::Http { method, uri } => wire::Target::Http {
|
||||
method: method.to_string(),
|
||||
uri: uri.to_string(),
|
||||
},
|
||||
core::Target::WebSocket { uri } => wire::Target::WebSocket {
|
||||
uri: uri.to_string(),
|
||||
},
|
||||
},
|
||||
headers: request
|
||||
.headers
|
||||
.iter()
|
||||
.map(|header| WireHeader {
|
||||
name: header.name().to_owned(),
|
||||
value: header.value().to_vec(),
|
||||
})
|
||||
.collect(),
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn decode(
|
||||
head: wire::RequestHead,
|
||||
payload: Vec<u8>,
|
||||
) -> Result<core::Request, PluginFault> {
|
||||
Ok(core::Request {
|
||||
target: match head.target {
|
||||
wire::Target::Http { method, uri } => core::Target::Http {
|
||||
method: method.parse().map_err(|_| invalid())?,
|
||||
uri: uri.parse().map_err(|_| invalid())?,
|
||||
},
|
||||
wire::Target::WebSocket { uri } => core::Target::WebSocket {
|
||||
uri: uri.parse().map_err(|_| invalid())?,
|
||||
},
|
||||
},
|
||||
headers: head
|
||||
.headers
|
||||
.into_iter()
|
||||
.map(|header| MiddlewareHeader::new(header.name, header.value.into()))
|
||||
.collect(),
|
||||
body: payload.into(),
|
||||
})
|
||||
}
|
||||
@@ -104,6 +104,7 @@ impl RpcError {
|
||||
Stage::WebSocket => "WebSocket 中间件:插件返回了无效响应",
|
||||
Stage::Request => "请求中间件:插件返回了无效响应",
|
||||
Stage::Attempt => "尝试中间件:插件返回了无效响应",
|
||||
Stage::Outbound => "出站准备中间件:插件返回了无效响应",
|
||||
Stage::Upstream => "上游适配阶段:插件返回了无效响应",
|
||||
Stage::Observation => "观察阶段:插件返回了无效响应",
|
||||
Stage::Management | Stage::PublicManagement => "管理阶段:插件返回了无效响应",
|
||||
|
||||
@@ -71,7 +71,8 @@ async fn reading_websocket_facts_preserves_original_text_and_opaque_messages() {
|
||||
attempt_middleware_context_for_transport(Arc::default(), ClientTransport::WebSocket),
|
||||
operation(),
|
||||
ClientTransport::WebSocket,
|
||||
Box::new(move |_, _| {
|
||||
None,
|
||||
Box::new(move |_, _, _| {
|
||||
Box::pin(async move {
|
||||
Ok(ProviderStream::new(
|
||||
ProviderCallMetadata::new(
|
||||
@@ -227,7 +228,8 @@ async fn facts_are_readable_without_replacing_or_duplicating_the_original_execut
|
||||
attempt_middleware_context(Arc::default()),
|
||||
operation(),
|
||||
ClientTransport::HttpJson,
|
||||
Box::new(move |_, _| {
|
||||
None,
|
||||
Box::new(move |_, _, _| {
|
||||
Box::pin(async move {
|
||||
Ok(ProviderStream::new(
|
||||
metadata,
|
||||
|
||||
@@ -3,6 +3,72 @@
|
||||
use super::*;
|
||||
use gateway_admin::ports::plugins::PluginPreparation;
|
||||
|
||||
#[tokio::test]
|
||||
async fn provider_parameters_compose_through_sdk_without_merging_deleted_fields() {
|
||||
let worker = std::fs::read(env!("CARGO_BIN_EXE_gateway-plugin-test-middleware")).unwrap();
|
||||
let package = crate::support::package_with_contributions(
|
||||
&worker,
|
||||
Contributions::from([crate::support::contribution(
|
||||
Capability::Middleware,
|
||||
vec![Stage::Attempt],
|
||||
vec!["openai".into()],
|
||||
vec!["openai".into()],
|
||||
)]),
|
||||
);
|
||||
let original = serde_json::json!({"installation_id":"account-default", "future":{"keep":true}});
|
||||
let first = serde_json::json!({"installation_id":"plugin-first"});
|
||||
let last = serde_json::json!({});
|
||||
let instances = [
|
||||
("parameters-first", original.clone(), first.clone()),
|
||||
("parameters-last", first, serde_json::Value::Null),
|
||||
]
|
||||
.into_iter()
|
||||
.enumerate()
|
||||
.map(|(index, (id, expected, parameters))| InstanceFixture {
|
||||
id,
|
||||
configuration: serde_json::json!({"mode":"parameters", "attempt":true,
|
||||
"expected_parameters":expected, "parameters":parameters}),
|
||||
bindings: vec![binding(
|
||||
MIDDLEWARE_CONTRIBUTION,
|
||||
"attempt",
|
||||
index as i32,
|
||||
PluginFailurePolicy::Reject,
|
||||
)],
|
||||
})
|
||||
.collect();
|
||||
let (cache, runtime) = setup_package(instances, package).await;
|
||||
let generation = prepare(&runtime).await;
|
||||
let plan = runtime
|
||||
.execution_registry()
|
||||
.middleware(&generation)
|
||||
.unwrap();
|
||||
let downstream = Downstream {
|
||||
calls: Arc::default(),
|
||||
closes: Arc::default(),
|
||||
reads: Arc::default(),
|
||||
error: false,
|
||||
};
|
||||
let response = plan
|
||||
.handle(
|
||||
attempt_middleware_context(Arc::default()),
|
||||
budget_request().with_provider_parameters(original.as_object().unwrap().clone()),
|
||||
gateway_core::middleware::compose(Vec::new(), move |request| {
|
||||
assert_eq!(
|
||||
request.provider_parameters().unwrap(),
|
||||
last.as_object().unwrap()
|
||||
);
|
||||
downstream.into_next().run(request)
|
||||
}),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let (_, _, _, body, _) = response.into_parts();
|
||||
body.close().await;
|
||||
drop(plan);
|
||||
drop(generation);
|
||||
super::super::wait_until_empty(cache.path()).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn partial_update_preserves_inflight_middleware_and_uses_new_configuration_for_new_calls() {
|
||||
let (cache, store, runtime) = super::super::setup_with_contributions(Contributions::from([
|
||||
|
||||
@@ -5,6 +5,7 @@ mod facts;
|
||||
mod http;
|
||||
mod lifecycle;
|
||||
mod mounts;
|
||||
mod outbound;
|
||||
mod retry;
|
||||
mod scheduling;
|
||||
mod service;
|
||||
@@ -1308,7 +1309,7 @@ async fn real_request_plugins_rewrite_settings_in_onion_order() {
|
||||
InstanceFixture { id: "settings-first", configuration: serde_json::json!({"mode":"settings","expected_settings":baseline_json,"settings":first}), bindings: vec![binding(MIDDLEWARE_CONTRIBUTION,"request",0,PluginFailurePolicy::Reject)] },
|
||||
InstanceFixture { id: "settings-second", configuration: serde_json::json!({"mode":"settings","expected_settings":first,"settings":last}), bindings: vec![binding(MIDDLEWARE_CONTRIBUTION,"request",1,PluginFailurePolicy::Reject)] },
|
||||
InstanceFixture { id: "settings-pass", configuration: serde_json::json!({"mode":"settings","expected_settings":last}), bindings: vec![binding(MIDDLEWARE_CONTRIBUTION,"request",2,PluginFailurePolicy::Reject)] },
|
||||
], package).await;
|
||||
], package.clone()).await;
|
||||
let generation = prepare(&runtime).await;
|
||||
let plan = runtime
|
||||
.execution_registry()
|
||||
@@ -1369,6 +1370,43 @@ async fn real_request_plugins_rewrite_settings_in_onion_order() {
|
||||
drop(plan);
|
||||
drop(generation);
|
||||
super::wait_until_empty(cache.path()).await;
|
||||
|
||||
// 根对象 null 也是显式替换,必须由设置合同拒绝,不能悄悄沿用原配置
|
||||
let (cache, runtime) = setup_package(vec![InstanceFixture {
|
||||
id: "settings-null",
|
||||
configuration: serde_json::json!({"mode":"settings", "expected_settings":baseline_json, "settings":null}),
|
||||
bindings: vec![binding(MIDDLEWARE_CONTRIBUTION, "request", 0, PluginFailurePolicy::Reject)],
|
||||
}], package).await;
|
||||
let generation = prepare(&runtime).await;
|
||||
let plan = runtime
|
||||
.execution_registry()
|
||||
.middleware(&generation)
|
||||
.unwrap();
|
||||
let calls = Arc::new(AtomicUsize::new(0));
|
||||
assert!(
|
||||
plan.handle(
|
||||
middleware_context(ClientTransport::HttpJson),
|
||||
MiddlewareRequest::new(
|
||||
"openai",
|
||||
middleware_headers(),
|
||||
Bytes::from_static(br#"{"input":"hello"}"#)
|
||||
)
|
||||
.with_settings(configuration),
|
||||
(Downstream {
|
||||
calls: Arc::clone(&calls),
|
||||
closes: Arc::default(),
|
||||
reads: Arc::default(),
|
||||
error: false
|
||||
})
|
||||
.into_next(),
|
||||
)
|
||||
.await
|
||||
.is_err()
|
||||
);
|
||||
assert_eq!(calls.load(Ordering::SeqCst), 0);
|
||||
drop(plan);
|
||||
drop(generation);
|
||||
super::wait_until_empty(cache.path()).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -1593,7 +1631,7 @@ impl CapabilitiesNext {
|
||||
|
||||
#[tokio::test]
|
||||
async fn sdk_capability_declarations_require_request_stage() {
|
||||
for (version, attempt) in [(5, false), (5, true)] {
|
||||
for (version, attempt) in [(6, false), (6, true)] {
|
||||
let stage = if attempt {
|
||||
Stage::Attempt
|
||||
} else {
|
||||
|
||||
@@ -0,0 +1,339 @@
|
||||
//! 验证真实插件出站准备的组合、资源预算和终止边界
|
||||
|
||||
use super::*;
|
||||
use gateway_core::engine::middleware::outbound::{Next, Request, Target};
|
||||
|
||||
async fn runtime(
|
||||
instances: Vec<InstanceFixture>,
|
||||
limits: RpcLimits,
|
||||
) -> (tempfile::TempDir, PluginRuntime) {
|
||||
let worker = std::fs::read(env!("CARGO_BIN_EXE_gateway-plugin-test-middleware")).unwrap();
|
||||
let package = crate::support::package_with_contributions(
|
||||
&worker,
|
||||
Contributions::from([crate::support::contribution(
|
||||
Capability::Middleware,
|
||||
vec![Stage::Outbound],
|
||||
vec!["outbound".into()],
|
||||
vec!["outbound".into()],
|
||||
)]),
|
||||
);
|
||||
setup_package_with_limits(instances, package, limits).await
|
||||
}
|
||||
|
||||
fn instance(id: &'static str, order: i32, mut configuration: serde_json::Value) -> InstanceFixture {
|
||||
configuration["outbound"] = true.into();
|
||||
InstanceFixture {
|
||||
id,
|
||||
configuration,
|
||||
bindings: vec![binding(
|
||||
MIDDLEWARE_CONTRIBUTION,
|
||||
"outbound",
|
||||
order,
|
||||
PluginFailurePolicy::Reject,
|
||||
)],
|
||||
}
|
||||
}
|
||||
|
||||
fn context() -> MiddlewareContext {
|
||||
context_for_provider("openai")
|
||||
}
|
||||
|
||||
fn context_for_provider(provider: &str) -> MiddlewareContext {
|
||||
MiddlewareContext::new(
|
||||
MiddlewareTarget {
|
||||
request_id: ModelRequestId::new("req_outbound").unwrap(),
|
||||
mount: MiddlewareMount::Outbound,
|
||||
attempt_index: Some(NonZeroU32::MIN),
|
||||
operation: Some(OperationKind::Generate),
|
||||
endpoint: "/v1/responses".into(),
|
||||
transport: ClientTransport::HttpJson,
|
||||
provider: Some(ProviderKind::new(provider).unwrap()),
|
||||
model: Some("public-a".into()),
|
||||
account_id: Some(ProviderAccountId::new("acct_openai").unwrap()),
|
||||
},
|
||||
MiddlewareAuthority {
|
||||
client_key_id: ClientApiKeyId::new("key-middleware").unwrap(),
|
||||
account_group_ids: Arc::from([AccountGroupId::new(
|
||||
"grp_11111111111111111111111111111111",
|
||||
)
|
||||
.unwrap()]),
|
||||
cancellation: CancellationToken::new(),
|
||||
deadline: (SystemTime::now() + Duration::from_secs(5)).into(),
|
||||
extension_scope: ExtensionCallScope::default(),
|
||||
execution_effects: Some(Arc::default()),
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
fn request() -> Request {
|
||||
Request {
|
||||
target: Target::Http {
|
||||
method: ::http::Method::POST,
|
||||
uri: "https://example.test/original".parse().unwrap(),
|
||||
},
|
||||
headers: vec![MiddlewareHeader::new(
|
||||
"x-delete",
|
||||
Bytes::from_static(b"original"),
|
||||
)],
|
||||
body: Bytes::from_static(&[0, 255]),
|
||||
}
|
||||
}
|
||||
|
||||
fn terminal() -> Next {
|
||||
gateway_core::middleware::compose(Vec::new(), |request| Box::pin(async { Ok(request) }))
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn outbound_plugins_compose_http_and_websocket_with_full_replacement() {
|
||||
for websocket in [false, true] {
|
||||
let records = tempfile::NamedTempFile::new().unwrap();
|
||||
let initial_body = if websocket {
|
||||
b"msg".to_vec()
|
||||
} else {
|
||||
vec![0, 255]
|
||||
};
|
||||
let target = if websocket {
|
||||
serde_json::json!({"kind":"websocket", "uri":"wss://rewritten.test/socket"})
|
||||
} else {
|
||||
serde_json::json!({"kind":"http", "method":"PUT", "uri":"https://rewritten.test/path?opaque=1"})
|
||||
};
|
||||
let mut outer = instance(
|
||||
"outer",
|
||||
0,
|
||||
serde_json::json!({
|
||||
"records":records.path(), "enter_byte":65, "exit_byte":68,
|
||||
"expected_body":initial_body, "target":target,
|
||||
"headers":[{"name":"x-multi","value":[97]}, {"name":"x-multi","value":[255]}],
|
||||
"second_next":true,
|
||||
}),
|
||||
);
|
||||
outer.bindings[0].client_key_ids = vec!["key-middleware".into()];
|
||||
outer.bindings[0].account_group_ids = vec!["grp_11111111111111111111111111111111".into()];
|
||||
outer.bindings[0].provider_ids = vec!["openai".into()];
|
||||
outer.bindings[0].models = vec!["public-a".into()];
|
||||
let mut inner_body = initial_body.clone();
|
||||
inner_body.push(65);
|
||||
let mut inner = instance(
|
||||
"inner",
|
||||
1,
|
||||
serde_json::json!({
|
||||
"records":records.path(), "enter_byte":66, "exit_byte":67, "expected_body":inner_body,
|
||||
}),
|
||||
);
|
||||
inner.bindings[0].provider_ids = vec!["openai".into()];
|
||||
let mut excluded = instance("excluded", 2, serde_json::json!({"fault":true}));
|
||||
excluded.bindings[0].provider_ids = vec!["xai".into()];
|
||||
let (cache, runtime) = runtime(vec![outer, inner, excluded], RpcLimits::default()).await;
|
||||
let generation = prepare(&runtime).await;
|
||||
let plan = runtime
|
||||
.execution_registry()
|
||||
.middleware(&generation)
|
||||
.unwrap();
|
||||
assert!(plan.has_outbound(&context()));
|
||||
assert!(!plan.has_outbound(&context_for_provider("unmatched")));
|
||||
let mut input = request();
|
||||
input.body = initial_body.clone().into();
|
||||
if websocket {
|
||||
input.target = Target::WebSocket {
|
||||
uri: "wss://example.test/socket".parse().unwrap(),
|
||||
};
|
||||
}
|
||||
let result = plan
|
||||
.handle_outbound(context(), input, terminal())
|
||||
.await
|
||||
.unwrap();
|
||||
let mut expected_body = initial_body;
|
||||
expected_body.extend([65, 66, 67, 68]);
|
||||
assert_eq!(result.body.as_ref(), expected_body);
|
||||
let headers = result.header_map().unwrap();
|
||||
assert!(!headers.contains_key("x-delete"));
|
||||
assert_eq!(
|
||||
headers
|
||||
.get_all("x-multi")
|
||||
.iter()
|
||||
.map(|value| value.as_bytes())
|
||||
.collect::<Vec<_>>(),
|
||||
[b"a".as_slice(), &[255]]
|
||||
);
|
||||
match result.target {
|
||||
Target::Http { method, uri } => {
|
||||
assert_eq!(method, ::http::Method::PUT);
|
||||
assert_eq!(uri.to_string(), "https://rewritten.test/path?opaque=1");
|
||||
}
|
||||
Target::WebSocket { uri } => assert_eq!(uri.to_string(), "wss://rewritten.test/socket"),
|
||||
}
|
||||
assert_eq!(
|
||||
std::fs::read_to_string(records.path()).unwrap(),
|
||||
"enter:outer\nenter:inner\nexit:inner\nexit:outer\n"
|
||||
);
|
||||
drop(plan);
|
||||
drop(generation);
|
||||
super::super::wait_until_empty(cache.path()).await;
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn outbound_plugin_can_return_directly_without_running_next() {
|
||||
let (cache, runtime) = runtime(
|
||||
vec![instance(
|
||||
"direct",
|
||||
0,
|
||||
serde_json::json!({"direct":true,"exit_byte":42}),
|
||||
)],
|
||||
RpcLimits::default(),
|
||||
)
|
||||
.await;
|
||||
let generation = prepare(&runtime).await;
|
||||
let plan = runtime
|
||||
.execution_registry()
|
||||
.middleware(&generation)
|
||||
.unwrap();
|
||||
let result = plan
|
||||
.handle_outbound(
|
||||
context(),
|
||||
request(),
|
||||
gateway_core::middleware::compose(Vec::new(), |_| {
|
||||
panic!("direct preparation must not consume next")
|
||||
}),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(result.body.as_ref(), [0, 255, 42]);
|
||||
drop(plan);
|
||||
drop(generation);
|
||||
super::super::wait_until_empty(cache.path()).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn outbound_http_callbacks_use_the_frozen_body_budget_and_propagate_failure() {
|
||||
for body_len in [3, 4] {
|
||||
let server = MockServer::start().await;
|
||||
Mock::given(any())
|
||||
.respond_with(ResponseTemplate::new(204))
|
||||
.expect(if body_len == 3 { 1 } else { 0 })
|
||||
.mount(&server)
|
||||
.await;
|
||||
let mut entry = instance(
|
||||
"callback",
|
||||
0,
|
||||
serde_json::json!({"http_url":server.uri(),"http_body_len":body_len,"fault":true}),
|
||||
);
|
||||
entry.bindings[0].failure_policy = PluginFailurePolicy::Delegate;
|
||||
let (cache, runtime) = runtime(vec![entry], RpcLimits::default()).await;
|
||||
let generation = prepare(&runtime).await;
|
||||
let plan = runtime
|
||||
.execution_registry()
|
||||
.middleware(&generation)
|
||||
.unwrap();
|
||||
let result = plan
|
||||
.handle_outbound(
|
||||
context().with_request_body_limit(Some(3)),
|
||||
request(),
|
||||
gateway_core::middleware::compose(Vec::new(), |_| {
|
||||
panic!("plugin failure must not silently replay preparation")
|
||||
}),
|
||||
)
|
||||
.await;
|
||||
assert!(result.is_err());
|
||||
server.verify().await;
|
||||
drop(plan);
|
||||
drop(generation);
|
||||
super::super::wait_until_empty(cache.path()).await;
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn outbound_invalid_method_is_not_repaired_or_delegated() {
|
||||
let mut entry = instance(
|
||||
"invalid",
|
||||
0,
|
||||
serde_json::json!({"direct":true,"target":{"kind":"http","method":"BAD METHOD","uri":"https://example.test/"}}),
|
||||
);
|
||||
entry.bindings[0].failure_policy = PluginFailurePolicy::Delegate;
|
||||
let (cache, runtime) = runtime(vec![entry], RpcLimits::default()).await;
|
||||
let generation = prepare(&runtime).await;
|
||||
let plan = runtime
|
||||
.execution_registry()
|
||||
.middleware(&generation)
|
||||
.unwrap();
|
||||
let result = plan.handle_outbound(context(), request(), terminal()).await;
|
||||
assert!(matches!(result, Err(MiddlewareError::InvalidState)));
|
||||
drop(plan);
|
||||
drop(generation);
|
||||
super::super::wait_until_empty(cache.path()).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn outbound_next_wait_pauses_plugin_budget_and_cancel_releases_the_continuation() {
|
||||
for cancel in [false, true] {
|
||||
let (cache, runtime) = runtime(
|
||||
vec![
|
||||
instance("outer", 0, serde_json::json!({})),
|
||||
instance("inner", 1, serde_json::json!({})),
|
||||
],
|
||||
RpcLimits {
|
||||
maximum_call_timeout: Duration::from_millis(300),
|
||||
..RpcLimits::default()
|
||||
},
|
||||
)
|
||||
.await;
|
||||
let generation = prepare(&runtime).await;
|
||||
let plan = runtime
|
||||
.execution_registry()
|
||||
.middleware(&generation)
|
||||
.unwrap();
|
||||
let context = context();
|
||||
let cancellation = context.cancellation().clone();
|
||||
let (started, entered) = tokio::sync::oneshot::channel();
|
||||
let (closed, dropped) = tokio::sync::oneshot::channel();
|
||||
let pending = tokio::spawn(async move {
|
||||
plan.handle_outbound(
|
||||
context,
|
||||
request(),
|
||||
gateway_core::middleware::compose(Vec::new(), move |request| {
|
||||
Box::pin(async move {
|
||||
let _closed = NotifyOnDrop(Some(closed));
|
||||
started.send(()).unwrap();
|
||||
if cancel {
|
||||
std::future::pending::<()>().await;
|
||||
} else {
|
||||
tokio::time::sleep(Duration::from_millis(900)).await;
|
||||
}
|
||||
Ok(request)
|
||||
})
|
||||
}),
|
||||
)
|
||||
.await
|
||||
});
|
||||
tokio::time::timeout(Duration::from_secs(2), entered)
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
tokio::time::sleep(Duration::from_millis(600)).await;
|
||||
assert!(!pending.is_finished());
|
||||
if cancel {
|
||||
cancellation.cancel();
|
||||
}
|
||||
let result = tokio::time::timeout(Duration::from_secs(2), pending)
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
assert_eq!(result.is_err(), cancel);
|
||||
tokio::time::timeout(Duration::from_secs(2), dropped)
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
drop(generation);
|
||||
super::super::wait_until_empty(cache.path()).await;
|
||||
}
|
||||
}
|
||||
|
||||
struct NotifyOnDrop(Option<tokio::sync::oneshot::Sender<()>>);
|
||||
|
||||
impl Drop for NotifyOnDrop {
|
||||
fn drop(&mut self) {
|
||||
if let Some(sender) = self.0.take() {
|
||||
let _ = sender.send(());
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -18,7 +18,9 @@ use gateway_core::{
|
||||
AccountAttemptContext, AttemptContext, ModelRequestId, RequestAttemptContext,
|
||||
middleware::MiddlewareHeader,
|
||||
provider::{EventStream, ProviderCallMetadata},
|
||||
upstream_adapter::{UpstreamAccountConnection, UpstreamAdapterInvocation},
|
||||
upstream_adapter::{
|
||||
UpstreamAccountConnection, UpstreamAdapterInvocation, UpstreamAdapterSelection,
|
||||
},
|
||||
},
|
||||
error::{ProviderError, ProviderErrorKind},
|
||||
event::GatewayEvent,
|
||||
@@ -86,7 +88,7 @@ impl UpstreamAccountConnection for Account {
|
||||
|
||||
fn configuration(base: &str, streaming: bool) -> Value {
|
||||
json!({
|
||||
"upstream_registration":{"adapters":[{"id":"example","provider":"openai","base_url":format!("{base}/adapter/"),"paths":[{"path":"responses","purpose":"inference"}],"authentication_kinds":["oauth"],"transport":if streaming {"http_sse"} else {"http_json"},"protocol":"openai"}]},
|
||||
"upstream_registration":{"adapters":[{"id":"example","provider":"openai","base_url":format!("{base}/adapter/"),"paths":[{"path":"responses","purpose":"inference"}],"authentication_kinds":["oauth"],"account_ids":[],"transport":if streaming {"http_sse"} else {"http_json"},"protocol":"openai"}]},
|
||||
"upstream_callbacks":[{"method":if streaming {"host.upstream.http.do_stream"} else {"host.upstream.http.do"},"params":{"method":"POST","path":"responses","headers":[["content-type",b"application/json".to_vec()]]},"body":"request"}],
|
||||
"upstream_events":[
|
||||
{"event":{"facts":[{"type":"started","id":"response-one","model":"native-model"}]}},
|
||||
@@ -113,12 +115,13 @@ async fn setup_with_limits(
|
||||
Arc<super::Store>,
|
||||
gateway_plugin_runtime::PluginRuntime,
|
||||
) {
|
||||
let contributes = Contributions::from([crate::support::contribution(
|
||||
let (capability, contribution) = crate::support::contribution(
|
||||
Capability::UpstreamAdapter,
|
||||
vec![Stage::Upstream],
|
||||
vec!["openai".into()],
|
||||
vec!["openai".into()],
|
||||
)]);
|
||||
);
|
||||
let contributes = Contributions::from([(capability, contribution)]);
|
||||
let (cache, store, runtime) =
|
||||
super::setup_with_limits(contributes, Default::default(), limits).await;
|
||||
{
|
||||
@@ -209,7 +212,7 @@ fn execute_with_context(
|
||||
let provider = ProviderKind::new("openai").unwrap();
|
||||
let model = UpstreamModelId::new("native-model").unwrap();
|
||||
let adapter = context
|
||||
.upstream_adapter(&provider, &model)
|
||||
.upstream_adapter_for_account(&provider, &model, &account.id)
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
let metadata = ProviderCallMetadata::new(
|
||||
@@ -240,6 +243,144 @@ fn execute_with_context(
|
||||
})
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn account_scoped_preselection_keeps_fallback_failures_within_matching_accounts() {
|
||||
for fallback_first in [false, true] {
|
||||
let mut config = configuration("https://example.invalid", false);
|
||||
config["upstream_registration"]["adapters"][0]["account_ids"] = json!(["acct_one"]);
|
||||
let (cache, store, runtime) = setup(config).await;
|
||||
{
|
||||
let mut snapshot = store.snapshot.lock().unwrap();
|
||||
let mut fallback = snapshot.instances[0].clone();
|
||||
fallback.id = "instance-two".into();
|
||||
fallback.name = "Fallback".into();
|
||||
fallback.configuration = configuration("https://fallback.invalid", false);
|
||||
fallback.configuration["upstream_registration"] = json!(false);
|
||||
if fallback_first {
|
||||
snapshot.instances.insert(0, fallback);
|
||||
} else {
|
||||
snapshot.instances.push(fallback);
|
||||
}
|
||||
}
|
||||
let generation =
|
||||
ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
{
|
||||
let context = context(&runtime, &generation, "key-one");
|
||||
let provider = ProviderKind::new("openai").unwrap();
|
||||
let model = UpstreamModelId::new("native-model").unwrap();
|
||||
let preselection = context.upstream_adapter(&provider, &model);
|
||||
let selected = context.upstream_adapter_for_account(
|
||||
&provider,
|
||||
&model,
|
||||
&ProviderAccountId::new("acct_one").unwrap(),
|
||||
);
|
||||
if !fallback_first {
|
||||
assert!(matches!(
|
||||
preselection.unwrap(),
|
||||
UpstreamAdapterSelection::PendingAccount
|
||||
));
|
||||
assert!(selected.unwrap().is_some());
|
||||
} else {
|
||||
assert_eq!(
|
||||
preselection.unwrap_err().kind(),
|
||||
ProviderErrorKind::Unavailable
|
||||
);
|
||||
assert_eq!(selected.unwrap_err().kind(), ProviderErrorKind::Unavailable);
|
||||
}
|
||||
assert_eq!(
|
||||
context
|
||||
.upstream_adapter_for_account(
|
||||
&provider,
|
||||
&model,
|
||||
&ProviderAccountId::new("acct_native").unwrap()
|
||||
)
|
||||
.unwrap_err()
|
||||
.kind(),
|
||||
ProviderErrorKind::Unavailable
|
||||
);
|
||||
}
|
||||
drop(generation);
|
||||
runtime.shutdown().await;
|
||||
super::wait_until_empty(cache.path()).await;
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn account_scoped_adapters_share_a_model_without_sharing_their_targets() {
|
||||
let first = MockServer::start().await;
|
||||
let second = MockServer::start().await;
|
||||
for server in [&first, &second] {
|
||||
Mock::given(method("POST"))
|
||||
.and(path("/adapter/responses"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_string("{}"))
|
||||
.expect(1)
|
||||
.mount(server)
|
||||
.await;
|
||||
}
|
||||
let mut config = configuration(&first.uri(), false);
|
||||
let mut second_declaration = config["upstream_registration"]["adapters"][0].clone();
|
||||
config["upstream_registration"]["adapters"][0]["account_ids"] = json!(["acct_one"]);
|
||||
second_declaration["id"] = json!("second");
|
||||
second_declaration["base_url"] = json!(format!("{}/adapter/", second.uri()));
|
||||
second_declaration["account_ids"] = json!(["acct_two"]);
|
||||
config["upstream_registration"]["adapters"]
|
||||
.as_array_mut()
|
||||
.unwrap()
|
||||
.push(second_declaration);
|
||||
let (cache, _, runtime) = setup(config).await;
|
||||
let generation = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
let provider = ProviderKind::new("openai").unwrap();
|
||||
let model = UpstreamModelId::new("native-model").unwrap();
|
||||
{
|
||||
let context = context(&runtime, &generation, "key-one");
|
||||
assert!(matches!(
|
||||
context.upstream_adapter(&provider, &model).unwrap(),
|
||||
UpstreamAdapterSelection::PendingAccount
|
||||
));
|
||||
assert!(
|
||||
context
|
||||
.upstream_adapter_for_account(
|
||||
&provider,
|
||||
&model,
|
||||
&ProviderAccountId::new("acct_native").unwrap()
|
||||
)
|
||||
.unwrap()
|
||||
.is_none()
|
||||
);
|
||||
}
|
||||
for (account, expected) in [("acct_one", "example"), ("acct_two", "second")] {
|
||||
let mut stream = execute(
|
||||
&runtime,
|
||||
&generation,
|
||||
Account::with_identity(account, 1),
|
||||
"key-one",
|
||||
None,
|
||||
);
|
||||
let mut completed = 0;
|
||||
let mut adapter = None;
|
||||
while let Some(event) = stream.next().await {
|
||||
let event = event.unwrap();
|
||||
completed += event
|
||||
.canonical_facts()
|
||||
.iter()
|
||||
.filter(|fact| matches!(fact, GatewayEvent::Completed(_)))
|
||||
.count();
|
||||
if let Some(state) = event.session_update() {
|
||||
adapter = Some(state.extension_owner().unwrap().adapter_id.clone());
|
||||
}
|
||||
}
|
||||
assert_eq!(completed, 1);
|
||||
assert_eq!(adapter.as_deref(), Some(expected));
|
||||
}
|
||||
drop(generation);
|
||||
runtime.shutdown().await;
|
||||
super::wait_until_empty(cache.path()).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn adapter_preserves_three_state_fast_in_request_metadata() {
|
||||
use gateway_core::account::FastMode;
|
||||
@@ -852,32 +993,38 @@ async fn adapter_does_not_publish_completed_when_rpc_fails_after_terminal_frame(
|
||||
|
||||
#[tokio::test]
|
||||
async fn overlapping_adapter_declarations_keep_first_match_without_blocking_preparation() {
|
||||
let server = MockServer::start().await;
|
||||
Mock::given(method("POST"))
|
||||
.and(path("/adapter/responses"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_string("{}"))
|
||||
.expect(1)
|
||||
.mount(&server)
|
||||
.await;
|
||||
let mut config = configuration(&server.uri(), false);
|
||||
let mut duplicate = config["upstream_registration"]["adapters"][0].clone();
|
||||
duplicate["base_url"] = json!("https://unused.invalid/adapter/");
|
||||
config["upstream_registration"]["adapters"]
|
||||
.as_array_mut()
|
||||
.unwrap()
|
||||
.push(duplicate);
|
||||
let (cache, _, runtime) = setup(config).await;
|
||||
let generation = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
let mut stream = execute(&runtime, &generation, Account::new(), "key-one", None);
|
||||
while let Some(event) = stream.next().await {
|
||||
event.unwrap();
|
||||
for scoped in [false, true] {
|
||||
let server = MockServer::start().await;
|
||||
Mock::given(method("POST"))
|
||||
.and(path("/adapter/responses"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_string("{}"))
|
||||
.expect(1)
|
||||
.mount(&server)
|
||||
.await;
|
||||
let mut config = configuration(&server.uri(), false);
|
||||
if scoped {
|
||||
config["upstream_registration"]["adapters"][0]["account_ids"] = json!(["acct_one"]);
|
||||
}
|
||||
let mut duplicate = config["upstream_registration"]["adapters"][0].clone();
|
||||
duplicate["base_url"] = json!("https://unused.invalid/adapter/");
|
||||
config["upstream_registration"]["adapters"]
|
||||
.as_array_mut()
|
||||
.unwrap()
|
||||
.push(duplicate);
|
||||
let (cache, _, runtime) = setup(config).await;
|
||||
let generation =
|
||||
ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
let mut stream = execute(&runtime, &generation, Account::new(), "key-one", None);
|
||||
while let Some(event) = stream.next().await {
|
||||
event.unwrap();
|
||||
}
|
||||
drop(stream);
|
||||
drop(generation);
|
||||
runtime.shutdown().await;
|
||||
super::wait_until_empty(cache.path()).await;
|
||||
}
|
||||
drop(stream);
|
||||
drop(generation);
|
||||
runtime.shutdown().await;
|
||||
super::wait_until_empty(cache.path()).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -912,6 +1059,78 @@ async fn adapter_registration_preserves_plugin_targets_with_typed_provider_decod
|
||||
runtime.shutdown().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn adapter_registration_requires_explicit_account_scope() {
|
||||
let mut config = configuration("https://example.invalid", false);
|
||||
config["upstream_registration"]["adapters"][0]
|
||||
.as_object_mut()
|
||||
.unwrap()
|
||||
.remove("account_ids");
|
||||
let (cache, store, runtime) = setup(config).await;
|
||||
let snapshot = store.snapshot.lock().unwrap().clone();
|
||||
assert!(
|
||||
gateway_admin::ports::plugins::PluginPreparation::prepare(&runtime, snapshot)
|
||||
.await
|
||||
.is_err()
|
||||
);
|
||||
runtime.shutdown().await;
|
||||
super::wait_until_empty(cache.path()).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn account_scoped_adapter_rechecks_the_invocation_account_before_sending() {
|
||||
let server = MockServer::start().await;
|
||||
let mut config = configuration(&server.uri(), false);
|
||||
config["upstream_registration"]["adapters"][0]["account_ids"] = json!(["acct_one"]);
|
||||
let (cache, _, runtime) = setup(config).await;
|
||||
let generation = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
let context = context(&runtime, &generation, "key-one");
|
||||
let provider = ProviderKind::new("openai").unwrap();
|
||||
let model = UpstreamModelId::new("native-model").unwrap();
|
||||
let adapter = context
|
||||
.upstream_adapter_for_account(
|
||||
&provider,
|
||||
&model,
|
||||
&ProviderAccountId::new("acct_one").unwrap(),
|
||||
)
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
let account = Account::with_identity("acct_other", 1);
|
||||
let metadata = ProviderCallMetadata::new(
|
||||
provider,
|
||||
model,
|
||||
account.id.clone(),
|
||||
UpstreamTransport::new(adapter.transport()).unwrap(),
|
||||
);
|
||||
let operation = Operation::Generate(GenerateRequest::from_protocol_payload(
|
||||
ProtocolPayload::json_object(
|
||||
"openai",
|
||||
json!({"model":"native-model","input":"hello"})
|
||||
.as_object()
|
||||
.unwrap()
|
||||
.clone(),
|
||||
)
|
||||
.unwrap(),
|
||||
));
|
||||
let mut stream = adapter.execute(UpstreamAdapterInvocation {
|
||||
operation,
|
||||
headers: vec![],
|
||||
context,
|
||||
metadata,
|
||||
account: account.clone(),
|
||||
});
|
||||
let error = stream.next().await.unwrap().unwrap_err();
|
||||
assert_eq!(error.send_state(), UpstreamSendState::NotSent);
|
||||
assert_eq!(account.failures.load(Ordering::SeqCst), 1);
|
||||
assert!(server.received_requests().await.unwrap().is_empty());
|
||||
drop(stream);
|
||||
drop(generation);
|
||||
runtime.shutdown().await;
|
||||
super::wait_until_empty(cache.path()).await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
#[expect(
|
||||
clippy::result_large_err,
|
||||
@@ -1172,177 +1391,191 @@ async fn usage_without_response_model_still_uses_native_account_pricing() {
|
||||
#[tokio::test]
|
||||
async fn core_published_adapter_uses_normal_attempt_usage_and_cost_ledger() {
|
||||
use crate::support::environment::Environment;
|
||||
let Some(mut environment) = Environment::create_command().await else {
|
||||
eprintln!("SKIP: plugin integration environment absent");
|
||||
return;
|
||||
};
|
||||
let server = MockServer::start().await;
|
||||
Mock::given(method("POST"))
|
||||
.and(header("authorization", "Bearer fixture-native-token"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_string("{}"))
|
||||
.expect(1)
|
||||
.mount(&server)
|
||||
.await;
|
||||
let account = environment.account(None).await;
|
||||
let group = environment.account_group_with_account(&account).await;
|
||||
let key_id = "key_adapter_ledger";
|
||||
environment
|
||||
.client_key_with_limits_and_groups(
|
||||
key_id,
|
||||
"sk-adapter-ledger-fixture",
|
||||
gateway_core::policy::RateLimits::unlimited(),
|
||||
vec![group],
|
||||
)
|
||||
.await;
|
||||
environment
|
||||
.install_plugin(configuration(&server.uri(), false))
|
||||
.await;
|
||||
environment.install_plugin(json!({
|
||||
"plugin_id":"test.adapter-ledger-caller",
|
||||
"command_registration":{"commands":[{"name":"run","description":"执行模型"}]},
|
||||
"command_nested_model_fixture":{
|
||||
"request":{"client_key_id":key_id,"model":crate::support::native::MODEL,"protocol":"openai","operation":"generate","provider":"openai","account_id":account.as_str()},
|
||||
"body":{"model":crate::support::native::MODEL,"input":"adapter ledger"}
|
||||
},
|
||||
"command_result":{"stdout":"ok","stderr":"","exit_code":0}
|
||||
})).await;
|
||||
let (runtime, core) = environment.command_plane().await;
|
||||
let snapshot = environment
|
||||
.store
|
||||
.admin_ports()
|
||||
.plugins()
|
||||
.load_instances()
|
||||
.await
|
||||
.unwrap();
|
||||
let caller = snapshot
|
||||
.instances
|
||||
.iter()
|
||||
.find(|instance| instance.configuration.get("command_registration").is_some())
|
||||
.unwrap();
|
||||
let commands = runtime.prepare_command_line().await.unwrap();
|
||||
environment.store.start_command_line_writes().unwrap();
|
||||
let result = commands.execute(&caller.id, "run", &[]).await.unwrap();
|
||||
assert_eq!(result.stdout, "ok");
|
||||
commands.shutdown().await;
|
||||
environment
|
||||
.store
|
||||
.shutdown_command_line_writes()
|
||||
.await
|
||||
.unwrap();
|
||||
environment
|
||||
.assert_upstream_accounting(key_id, account.as_str())
|
||||
.await;
|
||||
drop(core);
|
||||
drop(runtime);
|
||||
environment.close().await;
|
||||
for scoped in [false, true] {
|
||||
let Some(mut environment) = Environment::create_command().await else {
|
||||
eprintln!("SKIP: plugin integration environment absent");
|
||||
return;
|
||||
};
|
||||
let server = MockServer::start().await;
|
||||
Mock::given(method("POST"))
|
||||
.and(header("authorization", "Bearer fixture-native-token"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_string("{}"))
|
||||
.expect(1)
|
||||
.mount(&server)
|
||||
.await;
|
||||
let account = environment.account(None).await;
|
||||
let group = environment.account_group_with_account(&account).await;
|
||||
let key_id = "key_adapter_ledger";
|
||||
environment
|
||||
.client_key_with_limits_and_groups(
|
||||
key_id,
|
||||
"sk-adapter-ledger-fixture",
|
||||
gateway_core::policy::RateLimits::unlimited(),
|
||||
vec![group],
|
||||
)
|
||||
.await;
|
||||
let mut adapter_config = configuration(&server.uri(), false);
|
||||
if scoped {
|
||||
adapter_config["upstream_registration"]["adapters"][0]["account_ids"] =
|
||||
json!([account.as_str()]);
|
||||
}
|
||||
environment.install_plugin(adapter_config).await;
|
||||
environment.install_plugin(json!({
|
||||
"plugin_id":"test.adapter-ledger-caller",
|
||||
"command_registration":{"commands":[{"name":"run","description":"执行模型"}]},
|
||||
"command_nested_model_fixture":{
|
||||
"request":{"client_key_id":key_id,"model":crate::support::native::MODEL,"protocol":"openai","operation":"generate","provider":"openai","account_id":account.as_str()},
|
||||
"body":{"model":crate::support::native::MODEL,"input":"adapter ledger"}
|
||||
},
|
||||
"command_result":{"stdout":"ok","stderr":"","exit_code":0}
|
||||
})).await;
|
||||
let (runtime, core) = environment.command_plane().await;
|
||||
let snapshot = environment
|
||||
.store
|
||||
.admin_ports()
|
||||
.plugins()
|
||||
.load_instances()
|
||||
.await
|
||||
.unwrap();
|
||||
let caller = snapshot
|
||||
.instances
|
||||
.iter()
|
||||
.find(|instance| instance.configuration.get("command_registration").is_some())
|
||||
.unwrap();
|
||||
let commands = runtime.prepare_command_line().await.unwrap();
|
||||
environment.store.start_command_line_writes().unwrap();
|
||||
let result = commands.execute(&caller.id, "run", &[]).await.unwrap();
|
||||
assert_eq!(result.stdout, "ok");
|
||||
commands.shutdown().await;
|
||||
environment
|
||||
.store
|
||||
.shutdown_command_line_writes()
|
||||
.await
|
||||
.unwrap();
|
||||
environment
|
||||
.assert_upstream_accounting(key_id, account.as_str())
|
||||
.await;
|
||||
drop(core);
|
||||
drop(runtime);
|
||||
environment.close().await;
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn adapter_reconfiguration_disable_and_rollback_keep_inflight_generation_and_reap_workers() {
|
||||
let original = MockServer::start().await;
|
||||
let replacement = MockServer::start().await;
|
||||
for (server, expected) in [(&original, 2), (&replacement, 1)] {
|
||||
Mock::given(method("POST"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_string("{}"))
|
||||
.expect(expected)
|
||||
.mount(server)
|
||||
.await;
|
||||
}
|
||||
let (cache, store, runtime) = setup(configuration(&original.uri(), false)).await;
|
||||
let old = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
let account = Account::new();
|
||||
let mut old_stream = execute(&runtime, &old, account.clone(), "key-one", None);
|
||||
old_stream.next().await.unwrap().unwrap();
|
||||
{
|
||||
let mut snapshot = store.snapshot.lock().unwrap();
|
||||
snapshot.config_revision = gateway_admin::model::Revision::new(2).unwrap();
|
||||
snapshot.instances[0].revision = snapshot.config_revision;
|
||||
snapshot.instances[0].configuration = configuration(&replacement.uri(), false);
|
||||
}
|
||||
let new = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(2).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
assert_ne!(old.id(), new.id());
|
||||
let mut new_stream = execute(&runtime, &new, account.clone(), "key-one", None);
|
||||
new_stream.next().await.unwrap().unwrap();
|
||||
drop(old);
|
||||
let old_state = old_stream
|
||||
.next()
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap()
|
||||
.take_session_update()
|
||||
.unwrap();
|
||||
assert!(old_stream.next().await.is_none());
|
||||
drop(old_stream);
|
||||
{
|
||||
let mut snapshot = store.snapshot.lock().unwrap();
|
||||
snapshot.config_revision = gateway_admin::model::Revision::new(3).unwrap();
|
||||
snapshot.instances[0].revision = snapshot.config_revision;
|
||||
snapshot.instances[0].enabled = false;
|
||||
}
|
||||
let disabled = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(3).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(
|
||||
context(&runtime, &disabled, "key-one")
|
||||
.upstream_adapter(
|
||||
&ProviderKind::new("openai").unwrap(),
|
||||
&UpstreamModelId::new("native-model").unwrap()
|
||||
)
|
||||
.unwrap()
|
||||
.is_none()
|
||||
);
|
||||
drop(new);
|
||||
assert!(
|
||||
new_stream
|
||||
for scoped in [false, true] {
|
||||
let scoped_configuration = |base: &str| {
|
||||
let mut config = configuration(base, false);
|
||||
if scoped {
|
||||
config["upstream_registration"]["adapters"][0]["account_ids"] = json!(["acct_one"]);
|
||||
}
|
||||
config
|
||||
};
|
||||
let original = MockServer::start().await;
|
||||
let replacement = MockServer::start().await;
|
||||
for (server, expected) in [(&original, 2), (&replacement, 1)] {
|
||||
Mock::given(method("POST"))
|
||||
.respond_with(ResponseTemplate::new(200).set_body_string("{}"))
|
||||
.expect(expected)
|
||||
.mount(server)
|
||||
.await;
|
||||
}
|
||||
let (cache, store, runtime) = setup(scoped_configuration(&original.uri())).await;
|
||||
let old = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
let account = Account::new();
|
||||
let mut old_stream = execute(&runtime, &old, account.clone(), "key-one", None);
|
||||
old_stream.next().await.unwrap().unwrap();
|
||||
{
|
||||
let mut snapshot = store.snapshot.lock().unwrap();
|
||||
snapshot.config_revision = gateway_admin::model::Revision::new(2).unwrap();
|
||||
snapshot.instances[0].revision = snapshot.config_revision;
|
||||
snapshot.instances[0].configuration = scoped_configuration(&replacement.uri());
|
||||
}
|
||||
let new = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(2).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
assert_ne!(old.id(), new.id());
|
||||
let mut new_stream = execute(&runtime, &new, account.clone(), "key-one", None);
|
||||
new_stream.next().await.unwrap().unwrap();
|
||||
drop(old);
|
||||
let old_state = old_stream
|
||||
.next()
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap()
|
||||
.canonical_facts()
|
||||
.iter()
|
||||
.any(|fact| matches!(fact, GatewayEvent::Completed(_)))
|
||||
);
|
||||
assert!(new_stream.next().await.is_none());
|
||||
drop(new_stream);
|
||||
drop(disabled);
|
||||
super::wait_until_empty(cache.path()).await;
|
||||
assert_eq!(Arc::strong_count(&account), 1);
|
||||
{
|
||||
let mut snapshot = store.snapshot.lock().unwrap();
|
||||
snapshot.config_revision = gateway_admin::model::Revision::new(4).unwrap();
|
||||
snapshot.instances[0].revision = snapshot.config_revision;
|
||||
snapshot.instances[0].enabled = true;
|
||||
snapshot.instances[0].configuration = configuration(&original.uri(), false);
|
||||
}
|
||||
let restored = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(4).unwrap())
|
||||
.take_session_update()
|
||||
.unwrap();
|
||||
assert!(old_stream.next().await.is_none());
|
||||
drop(old_stream);
|
||||
{
|
||||
let mut snapshot = store.snapshot.lock().unwrap();
|
||||
snapshot.config_revision = gateway_admin::model::Revision::new(3).unwrap();
|
||||
snapshot.instances[0].revision = snapshot.config_revision;
|
||||
snapshot.instances[0].enabled = false;
|
||||
}
|
||||
let disabled = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(3).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
assert!(matches!(
|
||||
context(&runtime, &disabled, "key-one")
|
||||
.upstream_adapter(
|
||||
&ProviderKind::new("openai").unwrap(),
|
||||
&UpstreamModelId::new("native-model").unwrap()
|
||||
)
|
||||
.unwrap(),
|
||||
UpstreamAdapterSelection::Native
|
||||
));
|
||||
drop(new);
|
||||
assert!(
|
||||
new_stream
|
||||
.next()
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap()
|
||||
.canonical_facts()
|
||||
.iter()
|
||||
.any(|fact| matches!(fact, GatewayEvent::Completed(_)))
|
||||
);
|
||||
assert!(new_stream.next().await.is_none());
|
||||
drop(new_stream);
|
||||
drop(disabled);
|
||||
super::wait_until_empty(cache.path()).await;
|
||||
assert_eq!(Arc::strong_count(&account), 1);
|
||||
{
|
||||
let mut snapshot = store.snapshot.lock().unwrap();
|
||||
snapshot.config_revision = gateway_admin::model::Revision::new(4).unwrap();
|
||||
snapshot.instances[0].revision = snapshot.config_revision;
|
||||
snapshot.instances[0].enabled = true;
|
||||
snapshot.instances[0].configuration = scoped_configuration(&original.uri());
|
||||
}
|
||||
let restored = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(4).unwrap())
|
||||
.await
|
||||
.unwrap();
|
||||
// 恢复同一配置仍是新代次,不能复活旧进程中的续接身份
|
||||
let error = execute(
|
||||
&runtime,
|
||||
&restored,
|
||||
account.clone(),
|
||||
"key-one",
|
||||
Some(old_state),
|
||||
)
|
||||
.next()
|
||||
.await
|
||||
.unwrap();
|
||||
// 恢复同一配置仍是新代次,不能复活旧进程中的续接身份
|
||||
let error = execute(
|
||||
&runtime,
|
||||
&restored,
|
||||
account.clone(),
|
||||
"key-one",
|
||||
Some(old_state),
|
||||
)
|
||||
.next()
|
||||
.await
|
||||
.unwrap()
|
||||
.unwrap_err();
|
||||
assert_eq!(error.send_state(), UpstreamSendState::NotSent);
|
||||
let mut fresh = execute(&runtime, &restored, account.clone(), "key-one", None);
|
||||
while let Some(event) = fresh.next().await {
|
||||
event.unwrap();
|
||||
.unwrap()
|
||||
.unwrap_err();
|
||||
assert_eq!(error.send_state(), UpstreamSendState::NotSent);
|
||||
let mut fresh = execute(&runtime, &restored, account.clone(), "key-one", None);
|
||||
while let Some(event) = fresh.next().await {
|
||||
event.unwrap();
|
||||
}
|
||||
drop(fresh);
|
||||
drop(restored);
|
||||
runtime.shutdown().await;
|
||||
super::wait_until_empty(cache.path()).await;
|
||||
assert_eq!(Arc::strong_count(&account), 1);
|
||||
}
|
||||
drop(fresh);
|
||||
drop(restored);
|
||||
runtime.shutdown().await;
|
||||
super::wait_until_empty(cache.path()).await;
|
||||
assert_eq!(Arc::strong_count(&account), 1);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -1514,7 +1747,7 @@ async fn large_adapter_requests_use_the_frozen_budget_for_both_http_capabilities
|
||||
let provider = ProviderKind::new("openai").unwrap();
|
||||
let model = UpstreamModelId::new("native-model").unwrap();
|
||||
let adapter = context
|
||||
.upstream_adapter(&provider, &model)
|
||||
.upstream_adapter_for_account(&provider, &model, &account.id)
|
||||
.unwrap()
|
||||
.unwrap();
|
||||
let metadata = ProviderCallMetadata::new(
|
||||
|
||||
@@ -63,7 +63,7 @@ async fn package_inspector_returns_static_requirements_without_starting_the_plug
|
||||
requirements.protocol_version,
|
||||
gateway_plugin_sdk::PROTOCOL_VERSION
|
||||
);
|
||||
assert_eq!(requirements.capabilities, vec![("middleware".into(), 5)]);
|
||||
assert_eq!(requirements.capabilities, vec![("middleware".into(), 6)]);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
//! 中间件测试插件子进程入口及异常退出清理辅助
|
||||
|
||||
mod http_resources;
|
||||
mod outbound;
|
||||
mod usage_data;
|
||||
|
||||
use gateway_plugin_sdk::{
|
||||
@@ -26,7 +27,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
Capability::Middleware,
|
||||
ContributionDeclaration {
|
||||
id: "test.example.middleware".into(),
|
||||
version: 5,
|
||||
version: 6,
|
||||
stages: vec![
|
||||
Stage::Http,
|
||||
Stage::WebSocket,
|
||||
@@ -88,7 +89,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
id: "test.example.middleware".into(),
|
||||
version: configuration["middleware_version"]
|
||||
.as_u64()
|
||||
.unwrap_or(5)
|
||||
.unwrap_or(6)
|
||||
.try_into()?,
|
||||
stages: vec![if configuration["service"] == true {
|
||||
Stage::Service
|
||||
@@ -98,6 +99,8 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
Stage::WebSocket
|
||||
} else if configuration["attempt"] == true {
|
||||
Stage::Attempt
|
||||
} else if configuration["outbound"] == true {
|
||||
Stage::Outbound
|
||||
} else {
|
||||
Stage::Request
|
||||
}],
|
||||
@@ -107,6 +110,8 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
"http".into()
|
||||
} else if configuration["websocket"] == true {
|
||||
"websocket".into()
|
||||
} else if configuration["outbound"] == true {
|
||||
"outbound".into()
|
||||
} else {
|
||||
"openai".into()
|
||||
}],
|
||||
@@ -116,11 +121,23 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
"http".into()
|
||||
} else if configuration["websocket"] == true {
|
||||
"websocket".into()
|
||||
} else if configuration["outbound"] == true {
|
||||
"outbound".into()
|
||||
} else {
|
||||
"openai".into()
|
||||
}],
|
||||
},
|
||||
)]);
|
||||
if configuration["outbound"] == true {
|
||||
let plugin = MiddlewarePlugin::new(
|
||||
&contributes,
|
||||
move |call: gateway_plugin_sdk::client::OutboundCall| {
|
||||
outbound::handle(call, configuration.clone())
|
||||
},
|
||||
)?;
|
||||
session.run(plugin).await?;
|
||||
return Ok(());
|
||||
}
|
||||
if configuration["service"] == true {
|
||||
let plugin = MiddlewarePlugin::new(
|
||||
&contributes,
|
||||
@@ -528,6 +545,19 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
call.request.head.settings = settings.clone();
|
||||
}
|
||||
}
|
||||
if mode == "parameters" {
|
||||
assert_eq!(
|
||||
serde_json::to_value(&call.request.head.provider_parameters).unwrap(),
|
||||
configuration["expected_parameters"],
|
||||
);
|
||||
if let Some(parameters) = configuration.get("parameters") {
|
||||
call.request.head.provider_parameters = if parameters.is_null() {
|
||||
None
|
||||
} else {
|
||||
Some(parameters.as_object().unwrap().clone())
|
||||
};
|
||||
}
|
||||
}
|
||||
if mode == "inspect" && !call.request.body.is_empty() {
|
||||
// 读取和解析只能作用于副本,不能把解析结果写回未改写的正文
|
||||
assert!(serde_json::from_slice::<serde_json::Value>(&call.request.body).is_ok());
|
||||
@@ -590,7 +620,7 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
Ok(vec![frame])
|
||||
})?;
|
||||
}
|
||||
"passthrough" | "declare" | "settings" => {}
|
||||
"passthrough" | "declare" | "settings" | "parameters" => {}
|
||||
"inspect" => {
|
||||
response.body = response.body.inspect_frames(|frame| {
|
||||
assert!(!frame.payload.is_empty());
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
//! 真实出站准备插件,记录洋葱顺序并测试完整请求与调用资源
|
||||
|
||||
use gateway_plugin_sdk::{
|
||||
ErrorCode, PluginFault,
|
||||
call::middleware::NEXT_METHOD,
|
||||
client::{OutboundCall, OutboundRequest, SessionError},
|
||||
};
|
||||
|
||||
pub(super) async fn handle(
|
||||
mut call: OutboundCall,
|
||||
configuration: serde_json::Value,
|
||||
) -> Result<OutboundRequest, PluginFault> {
|
||||
assert_eq!(call.metadata.provider, "openai");
|
||||
assert_eq!(call.metadata.account_id, "acct_openai");
|
||||
assert_eq!(call.metadata.attempt_index, 1);
|
||||
assert_eq!(call.metadata.client_key_id, "key-middleware");
|
||||
let record = |phase| {
|
||||
if let Some(path) = configuration["records"].as_str() {
|
||||
use std::io::Write as _;
|
||||
let mut file = std::fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.append(true)
|
||||
.open(path)
|
||||
.unwrap();
|
||||
writeln!(file, "{phase}:{}", call.context.instance_id).unwrap();
|
||||
}
|
||||
};
|
||||
record("enter");
|
||||
if let Some(expected) = configuration.get("expected_body") {
|
||||
assert_eq!(serde_json::json!(call.request.body), *expected);
|
||||
}
|
||||
if let Some(headers) = configuration.get("headers") {
|
||||
call.request.head.headers = serde_json::from_value(headers.clone()).unwrap();
|
||||
}
|
||||
if let Some(target) = configuration.get("target") {
|
||||
call.request.head.target = serde_json::from_value(target.clone()).unwrap();
|
||||
}
|
||||
if let Some(url) = configuration["http_url"].as_str() {
|
||||
let body_len = configuration["http_body_len"].as_u64().unwrap_or(0) as usize;
|
||||
call.host
|
||||
.call(
|
||||
"host.http.do",
|
||||
serde_json::json!({"method":"POST", "url":url}),
|
||||
vec![42; body_len],
|
||||
)
|
||||
.await
|
||||
.map_err(SessionError::into_plugin_fault)?;
|
||||
}
|
||||
if configuration["fault"] == true {
|
||||
return Err(PluginFault::new(ErrorCode::Fault, "outbound fixture error"));
|
||||
}
|
||||
if let Some(byte) = configuration["enter_byte"].as_u64() {
|
||||
call.request.body.push(byte as u8);
|
||||
}
|
||||
let mut request = if configuration["direct"] == true {
|
||||
call.request
|
||||
} else {
|
||||
call.next.run(call.request).await?
|
||||
};
|
||||
if configuration["second_next"] == true {
|
||||
let result = call
|
||||
.host
|
||||
.call(
|
||||
NEXT_METHOD,
|
||||
serde_json::to_value(&request.head).unwrap(),
|
||||
request.body.clone(),
|
||||
)
|
||||
.await;
|
||||
let Err(error) = result else {
|
||||
panic!("next can only be consumed once")
|
||||
};
|
||||
assert_eq!(error.into_plugin_fault().code, ErrorCode::InvalidInput);
|
||||
}
|
||||
if let Some(byte) = configuration["exit_byte"].as_u64() {
|
||||
request.body.push(byte as u8);
|
||||
}
|
||||
record("exit");
|
||||
Ok(request)
|
||||
}
|
||||
@@ -75,7 +75,8 @@ impl Provider for NativeProvider {
|
||||
let model = candidate.upstream_model().cloned().ok_or_else(|| {
|
||||
ProviderError::new(ProviderErrorKind::Unavailable, UpstreamSendState::NotSent)
|
||||
})?;
|
||||
let adapter = context.upstream_adapter(candidate.provider(), &model)?;
|
||||
let adapter =
|
||||
context.upstream_adapter_for_account(candidate.provider(), &model, &account)?;
|
||||
let metadata = ProviderCallMetadata::new(
|
||||
candidate.provider().clone(),
|
||||
model,
|
||||
|
||||
@@ -1046,6 +1046,7 @@ impl Peer {
|
||||
id,
|
||||
NEXT_METHOD,
|
||||
serde_json::to_value(MiddlewareNextRequest {
|
||||
provider_parameters: None,
|
||||
settings: None,
|
||||
protocol: None,
|
||||
header_mutations: Vec::new(),
|
||||
|
||||
@@ -18,7 +18,7 @@ SDK 包版本为 `0.1.0`,独立于网关版本,处于实验阶段。能力
|
||||
| 根门面 | 清单、能力声明、调用上下文、消息与帧、公开错误 |
|
||||
| `call::frontend_authentication` | 数据面认证信封、认证器标识与外部 principal 结果 |
|
||||
| `call::model` | 宿主模型调用的 canonical 事实、原生 wire 与有界事件编解码 |
|
||||
| `call::middleware` | HTTP、WebSocket 消息、公开服务、模型请求及 attempt 的洋葱调用、单次 next 与惰性正文合同 |
|
||||
| `call::middleware` | HTTP、WebSocket 消息、公开服务、模型请求、attempt 与上游出站准备的洋葱调用、单次 next 与惰性正文合同 |
|
||||
| `call::services` | 公开服务的类型化操作、输入输出及错误合同;`usage` 提供[用量维护与计价](docs/usage-data.md) |
|
||||
| `call::upstream_adapter` | 内置 OpenAI / xAI 的[受管上游适配器](docs/upstream-adapters.md)注册、执行、出站与续接合同 |
|
||||
| `call::policy` | 模型路由、账号调度与重试决策合同 |
|
||||
@@ -31,7 +31,7 @@ SDK 包版本为 `0.1.0`,独立于网关版本,处于实验阶段。能力
|
||||
|
||||
`call/` 按业务能力组织数据,`message.rs` 定义消息与帧。已有请求链的请求/响应改写、协议转换和思考参数映射使用
|
||||
`call::middleware`;需要指定业务上游并解析结果时使用 `call::upstream_adapter`,作为洋葱链终端复用宿主账号与结算。
|
||||
下游 WebSocket 改写使用 `websocket` 中间件;上游 WebSocket 的观察接口保留只读合同
|
||||
下游 WebSocket 改写使用 `websocket` 中间件;上游发送前的请求准备使用 `outbound` 中间件,响应观察接口保留只读合同
|
||||
|
||||
插件使用的导入路径例如:
|
||||
|
||||
|
||||
@@ -216,7 +216,7 @@ SDK 不自动重试。超时或断连不能证明写入未提交;重试上限
|
||||
`invalid_input`,不启动模型执行。上游适配器的请求 `GPAQ` 也使用请求预算,响应事件仍受
|
||||
原有 8 MiB 编码上限、Runtime 缓冲预算及流控窗口限制
|
||||
|
||||
HTTP 和 request 中间件中的模型回调沿用有效请求设置;attempt、路由、调度、重试和上游适配器
|
||||
HTTP 和 request 中间件中的模型回调沿用有效请求设置;attempt、outbound、路由、调度、重试和上游适配器
|
||||
沿用该次模型请求冻结的正文预算。显式 Key 模型调用的预检与执行使用同一绑定快照,
|
||||
同一父调用内复用 Key 时不受后续配置发布影响;父调用结束后重新绑定才读取新配置,无需重新启用插件
|
||||
|
||||
@@ -245,7 +245,7 @@ HTTP 和 request 中间件中的模型回调沿用有效请求设置;attempt
|
||||
`host.http.do/do_stream` 和流读取/关闭方法提供受管 HTTP。正文使用独立二进制载荷,Host 统一执行代理、超时和背压,不按插件身份限制目标地址段。DNS 由宿主解析,连接保留目标 Host 与 TLS 域名校验;重定向不会自动跟随。
|
||||
受管 HTTP(包括 `host.upstream.http.do[_stream]`)的请求正文按有效设置
|
||||
`responses_max_decompressed_body_bytes` 检查。HTTP/request 中间件使用有效请求设置;
|
||||
attempt、路由、调度、重试和上游适配器使用执行冻结预算,其余调用读取当前发布配置。
|
||||
attempt、outbound、路由、调度、重试和上游适配器使用执行冻结预算,其余调用读取当前发布配置。
|
||||
超限返回带 HTTP 413 的 `invalid_input` 和 `not_sent`,不向目标发送请求。
|
||||
注册阶段沿用正在编译的配置预算,首次发布前也可发送非空正文。
|
||||
非空正文无法取得有效配置时拒绝调用;无正文查询不消耗正文预算。
|
||||
@@ -258,7 +258,7 @@ attempt、路由、调度、重试和上游适配器使用执行冻结预算,
|
||||
|
||||
### 洋葱中间件
|
||||
|
||||
`middleware.version: 5` 的所有挂载使用同一个 `ctx + next` 组合器。清单显式选择处理边界,绑定决定顺序;
|
||||
`middleware.version: 6` 的所有挂载使用同一个 `ctx + next` 组合器。清单显式选择处理边界,绑定决定顺序;
|
||||
不按 URL、消息类型或每个内部步骤增加处理器。仅处理一个边界可直接声明其调用类型;跨边界插件使用 `MiddlewareCall`
|
||||
在一个处理器内匹配类型化视图,不感兴趣的调用交给 `call.forward().await`
|
||||
|
||||
@@ -269,6 +269,7 @@ attempt、路由、调度、重试和上游适配器使用执行冻结预算,
|
||||
| `service` | `ServiceCall` / `TypedServiceCall<O>` | 已登记的公开业务服务 |
|
||||
| `request` | `RequestCall` | 一次逻辑模型请求 |
|
||||
| `attempt` | `RequestCall` | 本次已选 Provider 和账号的执行 |
|
||||
| `outbound` | `OutboundCall` | 原样返回准备结果,不发送网络 |
|
||||
|
||||
`PluginBuilder::middleware` 与 `MiddlewarePlugin::new` 共用 `MiddlewareInput` / `MiddlewareOutput`。
|
||||
`next.run(input)` 消费自身,最多执行一次;结果按相反顺序经过外层。重试或独立子请求通过宿主执行服务发起
|
||||
@@ -381,6 +382,45 @@ Ping 的自动 Pong 与关闭握手属于底层 WebSocket 协议事实;插件
|
||||
重写消息不能撤销已经发生的发送。文本必须是 UTF-8,控制帧长度和关闭格式仍遵守 WebSocket 协议。
|
||||
每条消息组合有独立 RPC 期限;自定义会话持有一个受管流式调用。连接结束会取消仍在处理的消息与发送
|
||||
|
||||
#### 上游出站准备
|
||||
|
||||
`outbound` 在 Provider 完成请求序列化后、提交发送前执行;无绑定或未命中范围时保留原生请求构造,不进入出站准备链。
|
||||
`OutboundCall.request` 包含
|
||||
`head.target`、Provider 已生成的完整 `head.headers` 和二进制 `body`;`target` 为 `Http { method, uri }`
|
||||
或 `WebSocket { uri }`,不能在准备过程中切换传输类型。HTTP 正文是已经序列化、完成内容编码的实际字节,
|
||||
插件自行处理压缩与 `Content-Encoding` 等头部的一致性;宿主不解码正文、补回字段或修复请求
|
||||
|
||||
`OutboundCall.metadata` 提供真实请求、Client Key、分组、Provider、模型、账号与 attempt 序号。
|
||||
这些归属由宿主冻结,返回值只有请求;修改 URL、认证头或正文不改变账号租约、凭据版本与计费归属。
|
||||
宿主不从最终字节重建模型、选号或续接事实;需要改变这些业务含义时,插件应在相应的 request / attempt 阶段保持一致。
|
||||
头部保留重复项与原始字节,正文不经 JSON 编码。
|
||||
这些头部是交给 transport 的字段集合;原生 HTTP 客户端 reqwest 会在缺少 `Accept` 时发送 `Accept: */*`,本能力不支持禁用该默认行为。
|
||||
最终结果仍须符合原生 transport 的可编码格式与目标约束;xAI 的内置 transport 只接受固定官方推理端点,
|
||||
当前原生 WebSocket 客户端不接受非 ASCII 握手头值,不可表示的值在发送前报错
|
||||
|
||||
`next.run(request)` 只进入下层准备链并返回完整请求,不打开连接、不发送消息;外层仍可修改返回值。
|
||||
所有外层处理结束后,Provider 才把结果交给 transport 发送。插件错误、非法传输值或正文超限在原生发送前结束,
|
||||
不自动跳过、重放或切换传输;取消、请求截止与回调资源沿用当前 attempt
|
||||
|
||||
WebSocket 的 `body` 是准备发送的一条完整 UTF-8 文本消息,`headers` 和 `uri` 是候选 opening。
|
||||
复用连接时不能改写已发生的握手;宿主不为插件主动重连、重放消息或重建续接。
|
||||
插件需要握手身份与消息内容一致时,应结合实际连接与续接语义选择改写
|
||||
|
||||
```rust
|
||||
use gateway_plugin_sdk::client::{OutboundCall, PluginBuilder};
|
||||
|
||||
let plugin = PluginBuilder::from_json(manifest_json)?
|
||||
.middleware(|call: OutboundCall| async move {
|
||||
let mut request = call.next.run(call.request).await?;
|
||||
request.head.headers.retain(|header| header.name != "x-optional");
|
||||
Ok(request)
|
||||
})?
|
||||
.build()?;
|
||||
```
|
||||
|
||||
该处理器的清单声明 `middleware.version: 6` 和 `stages: ["outbound"]`,不需要单独注册网络发送方法。
|
||||
当前内置 Provider 接入其模型执行所用的 HTTP 与 WebSocket 文本发送边界;OAuth、额度刷新等后台网络不属于模型出站准备
|
||||
|
||||
#### 模型请求与 attempt
|
||||
|
||||
`RequestCall.request.head` 包含完整 Client Key 与分组归属。request 中间件改写正文后,OpenAI 在进入 attempt 前应用宿主 Fast 策略;
|
||||
@@ -389,10 +429,22 @@ request 与 attempt 阶段通过 `append_header` 添加的请求头会进入上
|
||||
OpenAI 与 xAI 的上游 headers 先由账号和画像生成,插件同名字段覆盖基线,同名多值按插件顺序保留;不存在按认证、Cookie 或会话名称划分的保护列表。
|
||||
实际账号租约、凭据版本与计费归属仍记录真实选中事实,不随插件修改 header 名称而伪造
|
||||
|
||||
middleware v5 的 request 阶段 `request.head.settings` 是本次调用的有效设置对象,包含 `runtime`、`fast_mode`、`client_limits` 和 `timeout_ms`。
|
||||
attempt 的 `request.head.provider_parameters` 携带 Provider 提供的本次构造参数;没有该能力的执行不提供此对象。
|
||||
OpenAI 原生 Responses、Images、Search 与 Live 创建请求提供 `installation_id` 字符串,默认取自选中账号,
|
||||
用于原生安装身份、已有 turn metadata 身份字段及会话、缓存关联标识的派生。独立 `upstream_adapter` 自行构造其请求。
|
||||
|
||||
插件直接修改此对象,SDK 在值改变时提交完整替换;空对象明确删除全部参数,将原本存在的对象设为 `None` 同样提交空对象。
|
||||
线协议的 `next.provider_parameters` 省略时保留原值,显式 `null` 或非对象类型拒绝。
|
||||
后续插件读取前层结果,Core / Runtime 不解释业务字段、不深合并;缺失必需字段或类型错误在发送前失败且禁止重试,不补回账号默认身份。
|
||||
|
||||
身份选择、Key 与账号配对、UUID 生成与持久化由插件负责,可复用已有状态服务;参数不写回账号凭据、全局设置或真实计费事实。
|
||||
WS 新响应链按构造身份区分连接,精确续接仍使用实际持有该响应链的原连接;Live sideband / hangup 不继承创建请求的临时参数
|
||||
|
||||
request 阶段 `request.head.settings` 是本次调用的有效设置对象,包含 `runtime`、`fast_mode`、`client_limits` 和 `timeout_ms`。
|
||||
`fast_mode` 使用 `default`、`enabled`、`disabled` 三态,具体档位语义见[账号分组](../../../../../docs/api.md#6-账号分组)。
|
||||
`runtime` 使用宿主快照的设置字段,画像已合并 Client Key 设置;可以修改映射、画像、调度、位置及计价等执行参数。
|
||||
SDK 仅在对象改变时提交完整替换,`false`、`0`、空映射和合同允许的 `null` 不表示跳过;不做隐式深合并。
|
||||
线协议省略 `next.settings` 才表示保留;显式 `null` 仍按完整替换校验,因不符合设置对象合同而拒绝。
|
||||
多个插件按入站顺序读取前层改写后的值,终端在执行前从同一配置代次重算,改写不发布全局配置。
|
||||
`timeout_ms` 默认为 `null`,表示不限制模型请求总时长;数值从本次模型请求开始计时,`0` 表示立即到期,
|
||||
改写不会清零已用时间。HTTP 处理期限与 RPC 操作期限仍各自生效
|
||||
@@ -455,6 +507,7 @@ restore_and_validate_response(response).await
|
||||
| --- | --- |
|
||||
| 请求头/正文改写、提前拒绝、响应头处理 | `middleware` 包裹一次 `next.run(request)` |
|
||||
| 响应逐帧转换、观察 | 返回惰性 `map_frames` / `inspect_frames`,不能在 `next` 返回时就把流视为结束 |
|
||||
| Provider 已生成请求的最终 URL、头部或字节改写 | `outbound` 返回最终请求,Provider 随后发送 |
|
||||
| 新上游路径、协议编码、错误及用量解析 | `upstream_adapter` 终端,复用已选账号和原生结算 |
|
||||
| 账号代理、默认凭据注入、网络发送状态 | 宿主网络外层,插件可覆盖默认请求头,普通 HTTP 与账号 HTTP 共用发送和正文资源实现 |
|
||||
| 路由、调度、重试决策 | 各自的决策端口,Core 保持最终裁决和预算 |
|
||||
|
||||
@@ -53,9 +53,9 @@ RPC v5 的帧布局、编码与会话生命周期见[通信边界](../README.md#
|
||||
|
||||
| 能力 | 当前版本 |
|
||||
| --- | --- |
|
||||
| `middleware` | `5` |
|
||||
| `middleware` | `6` |
|
||||
| `scheduler` | `3` |
|
||||
| `upstream_adapter` | `4` |
|
||||
| `upstream_adapter` | `5` |
|
||||
| `frontend_authentication`、`model_router`、`retry_policy`、`observer`、`management` | `2` |
|
||||
| `model_catalog`、`command_line`、`maintenance` | `1` |
|
||||
|
||||
@@ -67,7 +67,7 @@ RPC v5 的帧布局、编码与会话生命周期见[通信边界](../README.md#
|
||||
"management": { "version": 2 },
|
||||
"command_line": {},
|
||||
"middleware": {
|
||||
"version": 5,
|
||||
"version": 6,
|
||||
"stages": ["request"],
|
||||
"inputFormats": ["openai"],
|
||||
"outputFormats": ["openai"]
|
||||
@@ -90,7 +90,7 @@ capability 固定并由工具生成:
|
||||
|
||||
`observer` 使用一个处理器接收完成与上游 WebSocket 事件,实例绑定按 `event` 选择订阅类型;具体合同见[观察事件](capabilities.md#路由调度与观察)
|
||||
|
||||
`middleware` 必须从 `http`、`websocket`、`service`、`request`、`attempt` 中显式选择挂载;同一处理器可覆盖多个边界,协议格式等真实业务选择也不能省略。
|
||||
`middleware` 必须从 `http`、`websocket`、`service`、`request`、`attempt`、`outbound` 中显式选择挂载;同一处理器可覆盖多个边界,协议格式等真实业务选择也不能省略。
|
||||
安装清单若携带不同的固定阶段会被拒绝,而不是在加载时静默改写
|
||||
|
||||
## 完整信任
|
||||
|
||||
@@ -2,13 +2,13 @@
|
||||
|
||||
[返回 SDK](../README.md) · [清单声明](manifest.md#扩展项简写) · [能力与回调](capabilities.md)
|
||||
|
||||
`upstream_adapter` v4 是现有生成请求洋葱链的终端,只挂到内置 `openai` / `xai`,不替换独立图像或压缩端点。
|
||||
`upstream_adapter` v5 是现有生成请求洋葱链的终端,只挂到内置 `openai` / `xai`,不替换独立图像或压缩端点。
|
||||
插件定义 Base URL、业务路径、请求编码和响应解析;宿主继续持有身份、ACL、选号、租约、凭据、代理、重试和账本。
|
||||
普通请求或响应加工使用 [middleware](capabilities.md#洋葱中间件)
|
||||
|
||||
## 声明与绑定
|
||||
|
||||
清单声明 `upstream_adapter`、版本 `4`,以及匹配的 `inputFormats` / `outputFormats`;固定阶段 `upstream` 由作者清单规范化入口生成。
|
||||
清单声明 `upstream_adapter`、版本 `5`,以及匹配的 `inputFormats` / `outputFormats`;固定阶段 `upstream` 由作者清单规范化入口生成。
|
||||
使用 `PluginBuilder::on` 注册
|
||||
`methods::UPSTREAM_ADAPTER_REGISTER` 与 `methods::UPSTREAM_ADAPTER_EXECUTE`
|
||||
|
||||
@@ -24,11 +24,21 @@
|
||||
| `transport` | `http_json`、`http_sse` 或 `websocket` |
|
||||
| `protocol` | 请求和客户端 wire 使用的协议 |
|
||||
| `models` | 公开请求模型的精确匹配,空集合沿用绑定范围 |
|
||||
| `account_ids` | 上游账号 ID 范围,必填;空数组匹配所有账号 |
|
||||
|
||||
实例绑定沿用 Key、账号组、Provider、模型范围,只允许一次 `upstream` 绑定和 `reject` 故障策略。
|
||||
多个声明命中时按发布顺序选择首个适配器,插件负责声明的业务含义。适配器未命中时仍使用原生上游,
|
||||
已命中但不可用时拒绝请求。安装、配置、启停和升级使用已有插件管理
|
||||
|
||||
账号范围只收窄绑定,不能扩大 Key、分组或模型权限。宿主先排除适配器不可用或认证类型不兼容的账号,
|
||||
再沿用原调度、亲和和租约;取得账号后从同一发布代次解析适配器。不同账号可以为同一公开模型选择不同适配器,
|
||||
未匹配账号保持原生执行,重叠声明仍按发布顺序取首个匹配。
|
||||
前置账号范围已匹配的可用适配器不受后置兜底声明故障影响。
|
||||
xAI 原生编码不支持的请求仍可选择匹配的适配器账号;原生账号会在取得租约前被排除
|
||||
|
||||
账号范围由插件注册结果提供,不改变管理端 binding 中 `accountGroupIds` 的请求作用域语义。
|
||||
停用或卸载实例会移除其绑定;维护时须先停用依赖该适配器的账号并排空请求,账号范围不提供永久依赖约束。
|
||||
|
||||
能力、清单、进程 RPC 和宿主兼容声明分别校验各自版本,见[清单](manifest.md)。
|
||||
Provider、业务协议和传输名称不能写入宿主兼容清单的 capability / RPC 版本位置
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
//! 统一请求/响应中间件的跨进程数据合同
|
||||
|
||||
pub mod http;
|
||||
pub mod outbound;
|
||||
pub mod websocket;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
@@ -69,6 +70,9 @@ pub struct MiddlewareRequestHead {
|
||||
/// 当前请求的完整有效执行设置;尚未建立可改写设置的边界使用 null
|
||||
#[serde(default)]
|
||||
pub settings: serde_json::Value,
|
||||
/// 本次 Provider 构造参数;仅原生 attempt 提供
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub provider_parameters: Option<serde_json::Map<String, serde_json::Value>>,
|
||||
pub request_id: String,
|
||||
pub client_key_id: String,
|
||||
pub account_group_ids: Vec<String>,
|
||||
@@ -121,8 +125,19 @@ pub struct CapabilityDeclaration {
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct MiddlewareNextRequest {
|
||||
/// None 保留原设置;Some 是完整替换,字段删除必须符合对应设置类型
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
#[serde(
|
||||
default,
|
||||
skip_serializing_if = "Option::is_none",
|
||||
deserialize_with = "deserialize_replacement"
|
||||
)]
|
||||
pub settings: Option<serde_json::Value>,
|
||||
/// 完整替换构造参数;空对象仍表示显式替换,不隐式合并或补回删除字段
|
||||
#[serde(
|
||||
default,
|
||||
skip_serializing_if = "Option::is_none",
|
||||
deserialize_with = "deserialize_replacement"
|
||||
)]
|
||||
pub provider_parameters: Option<serde_json::Map<String, serde_json::Value>>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub protocol: Option<String>,
|
||||
#[serde(default)]
|
||||
@@ -132,6 +147,15 @@ pub struct MiddlewareNextRequest {
|
||||
pub capabilities: Option<CapabilityDeclaration>,
|
||||
}
|
||||
|
||||
fn deserialize_replacement<'de, T, D>(deserializer: D) -> Result<Option<T>, D::Error>
|
||||
where
|
||||
T: Deserialize<'de>,
|
||||
D: serde::Deserializer<'de>,
|
||||
{
|
||||
// 只有缺省表示保留;显式值按字段类型解码,不能被 Option 当作省略
|
||||
T::deserialize(deserializer).map(Some)
|
||||
}
|
||||
|
||||
/// 受父调用约束的不透明响应正文句柄
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct MiddlewareBodyHandle {
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
//! 上游发送前的准备合同;正文由 RPC 二进制载荷承载
|
||||
|
||||
use std::fmt;
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use super::MiddlewareHeader;
|
||||
|
||||
/// HTTP 发送目标或 WebSocket 候选 opening,返回值必须保留传输类型
|
||||
#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(tag = "kind", rename_all = "snake_case", deny_unknown_fields)]
|
||||
pub enum Target {
|
||||
Http {
|
||||
method: String,
|
||||
uri: String,
|
||||
},
|
||||
#[serde(rename = "websocket")]
|
||||
WebSocket {
|
||||
uri: String,
|
||||
},
|
||||
}
|
||||
|
||||
impl fmt::Debug for Target {
|
||||
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
match self {
|
||||
Self::Http { method, .. } => formatter
|
||||
.debug_struct("Http")
|
||||
.field("method", method)
|
||||
.finish_non_exhaustive(),
|
||||
Self::WebSocket { .. } => formatter.debug_struct("WebSocket").finish_non_exhaustive(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// HTTP 正文是实际内容编码后的字节;WebSocket 头是候选 opening
|
||||
#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct RequestHead {
|
||||
pub target: Target,
|
||||
pub headers: Vec<MiddlewareHeader>,
|
||||
}
|
||||
|
||||
impl fmt::Debug for RequestHead {
|
||||
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
formatter
|
||||
.debug_struct("RequestHead")
|
||||
.field("target", &self.target)
|
||||
.field("headers", &self.headers.len())
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
/// 宿主冻结的实际业务归属;改写请求不会改变账号、调用关联或计量归属
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct Metadata {
|
||||
pub request_id: String,
|
||||
pub client_key_id: String,
|
||||
pub account_group_ids: Vec<String>,
|
||||
pub provider: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub model: Option<String>,
|
||||
pub account_id: String,
|
||||
pub attempt_index: u32,
|
||||
}
|
||||
|
||||
/// 入口请求与宿主归属;最终响应只返回 RequestHead 和正文载荷
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct CallHead {
|
||||
pub request: RequestHead,
|
||||
pub metadata: Metadata,
|
||||
}
|
||||
@@ -66,6 +66,8 @@ pub struct UpstreamAdapterDeclaration {
|
||||
/// 公开请求模型的匹配范围;空集合沿用功能绑定范围
|
||||
#[serde(default)]
|
||||
pub models: Vec<String>,
|
||||
/// 显式上游账号范围,空集合匹配所有账号,不扩大请求的账号权限
|
||||
pub account_ids: Vec<String>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
|
||||
@@ -26,14 +26,14 @@ impl Capability {
|
||||
#[must_use]
|
||||
pub const fn contract_versions(self) -> &'static [u32] {
|
||||
match self {
|
||||
Self::Middleware => &[5],
|
||||
Self::Middleware => &[6],
|
||||
Self::FrontendAuthentication
|
||||
| Self::ModelRouter
|
||||
| Self::RetryPolicy
|
||||
| Self::Observer
|
||||
| Self::Management => &[2],
|
||||
Self::Scheduler => &[3],
|
||||
Self::UpstreamAdapter => &[4],
|
||||
Self::UpstreamAdapter => &[5],
|
||||
Self::ModelCatalog | Self::CommandLine | Self::Maintenance => &[1],
|
||||
}
|
||||
}
|
||||
@@ -91,6 +91,8 @@ pub enum Stage {
|
||||
WebSocket,
|
||||
Request,
|
||||
Attempt,
|
||||
/// Provider 发送前的完整请求准备,不执行上游发送
|
||||
Outbound,
|
||||
/// Core 登记已选账号的 attempt 后才启动的受管上游执行
|
||||
Upstream,
|
||||
Observation,
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
//! 多边界插件只注册一次处理器,按类型选择当前调用视图
|
||||
use super::super::session::{CallReply, PluginCall};
|
||||
use super::{
|
||||
HttpCall, HttpResponse, MiddlewareInput, MiddlewareOutput, MiddlewareResponse, RequestCall,
|
||||
ServiceCall, ServiceResponse, WebSocketCall, WebSocketMessage,
|
||||
HttpCall, HttpResponse, MiddlewareInput, MiddlewareOutput, MiddlewareResponse, OutboundCall,
|
||||
OutboundRequest, RequestCall, ServiceCall, ServiceResponse, WebSocketCall, WebSocketMessage,
|
||||
};
|
||||
use crate::{CallContext, PluginFault, Stage};
|
||||
|
||||
@@ -11,12 +11,14 @@ pub enum MiddlewareCall {
|
||||
Http(Box<HttpCall>),
|
||||
WebSocket(Box<WebSocketCall>),
|
||||
Request(Box<RequestCall>),
|
||||
Outbound(Box<OutboundCall>),
|
||||
}
|
||||
pub enum MiddlewareResult {
|
||||
Service(ServiceResponse),
|
||||
Http(HttpResponse),
|
||||
WebSocket(Option<WebSocketMessage>),
|
||||
Request(MiddlewareResponse),
|
||||
Outbound(OutboundRequest),
|
||||
}
|
||||
impl MiddlewareCall {
|
||||
#[must_use]
|
||||
@@ -26,11 +28,17 @@ impl MiddlewareCall {
|
||||
Self::Http(call) => &call.context,
|
||||
Self::WebSocket(call) => &call.context,
|
||||
Self::Request(call) => &call.context,
|
||||
Self::Outbound(call) => &call.context,
|
||||
}
|
||||
}
|
||||
/// 原样进入下游,供当前处理器不感兴趣的调用复用
|
||||
pub async fn forward(self) -> Result<MiddlewareResult, PluginFault> {
|
||||
match self {
|
||||
Self::Outbound(call) => call
|
||||
.next
|
||||
.run(call.request)
|
||||
.await
|
||||
.map(MiddlewareResult::Outbound),
|
||||
Self::Service(call) => call.forward().await.map(MiddlewareResult::Service),
|
||||
Self::Http(call) => call
|
||||
.next
|
||||
@@ -57,9 +65,13 @@ impl MiddlewareInput for MiddlewareCall {
|
||||
|| HttpCall::accepts(stage)
|
||||
|| WebSocketCall::accepts(stage)
|
||||
|| RequestCall::accepts(stage)
|
||||
|| OutboundCall::accepts(stage)
|
||||
}
|
||||
fn decode(call: PluginCall) -> Result<Self, PluginFault> {
|
||||
match call.context.stage {
|
||||
Stage::Outbound => {
|
||||
OutboundCall::decode(call).map(|call| Self::Outbound(Box::new(call)))
|
||||
}
|
||||
Stage::Service => ServiceCall::decode(call).map(Box::new).map(Self::Service),
|
||||
Stage::Http => HttpCall::decode(call).map(Box::new).map(Self::Http),
|
||||
Stage::WebSocket => WebSocketCall::decode(call)
|
||||
@@ -76,6 +88,7 @@ impl MiddlewareOutput for MiddlewareResult {
|
||||
Self::Http(response) => response.encode(),
|
||||
Self::WebSocket(response) => response.encode(),
|
||||
Self::Request(response) => response.encode(),
|
||||
Self::Outbound(request) => request.encode(),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -100,3 +113,9 @@ impl From<ServiceResponse> for MiddlewareResult {
|
||||
Self::Service(response)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<OutboundRequest> for MiddlewareResult {
|
||||
fn from(request: OutboundRequest) -> Self {
|
||||
Self::Outbound(request)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
mod body;
|
||||
mod call;
|
||||
mod http;
|
||||
mod outbound;
|
||||
mod plugin;
|
||||
mod service;
|
||||
mod websocket;
|
||||
@@ -10,6 +11,7 @@ mod websocket;
|
||||
pub use body::{MiddlewareBody, MiddlewareBodySender};
|
||||
pub use call::{MiddlewareCall, MiddlewareResult};
|
||||
pub use http::{HttpBody, HttpCall, HttpFrame, HttpNext, HttpRequest, HttpResponse};
|
||||
pub use outbound::{OutboundCall, OutboundNext, OutboundRequest};
|
||||
pub use plugin::MiddlewarePlugin;
|
||||
pub use service::{ServiceCall, ServiceNext, ServiceResponse, TypedServiceCall};
|
||||
pub use websocket::{
|
||||
@@ -37,6 +39,7 @@ pub struct MiddlewareRequest {
|
||||
pub body: Vec<u8>,
|
||||
original_protocol: String,
|
||||
original_settings: serde_json::Value,
|
||||
original_provider_parameters: Option<serde_json::Map<String, serde_json::Value>>,
|
||||
original_headers: Vec<MiddlewareHeader>,
|
||||
original_body: Vec<u8>,
|
||||
body_replaced: bool,
|
||||
@@ -96,6 +99,9 @@ impl MiddlewareRequest {
|
||||
};
|
||||
(
|
||||
MiddlewareNextRequest {
|
||||
provider_parameters: (self.head.provider_parameters
|
||||
!= self.original_provider_parameters)
|
||||
.then(|| self.head.provider_parameters.unwrap_or_default()),
|
||||
settings: (self.head.settings != self.original_settings)
|
||||
.then_some(self.head.settings),
|
||||
protocol,
|
||||
@@ -339,6 +345,7 @@ impl RequestCall {
|
||||
context: call.context,
|
||||
request: MiddlewareRequest {
|
||||
original_settings: head.settings.clone(),
|
||||
original_provider_parameters: head.provider_parameters.clone(),
|
||||
body: call.payload,
|
||||
original_body,
|
||||
body_replaced: false,
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
//! 出站请求的类型化准备入口;一次 next 只返回准备结果
|
||||
|
||||
use std::fmt;
|
||||
|
||||
use crate::call::middleware::{HANDLE_METHOD, NEXT_METHOD, outbound as wire};
|
||||
use crate::client::{CallCancellation, CallReply, HostClient, PluginCall, SessionError};
|
||||
use crate::{CallContext, PluginFault, Stage};
|
||||
|
||||
use super::{MiddlewareInput, MiddlewareOutput, invalid_input};
|
||||
|
||||
/// HTTP 正文为内容编码后的实际字节,WebSocket 正文为完整 UTF-8 消息
|
||||
#[derive(Clone, PartialEq, Eq)]
|
||||
pub struct OutboundRequest {
|
||||
pub head: wire::RequestHead,
|
||||
pub body: Vec<u8>,
|
||||
}
|
||||
|
||||
impl fmt::Debug for OutboundRequest {
|
||||
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
formatter
|
||||
.debug_struct("OutboundRequest")
|
||||
.field("head", &self.head)
|
||||
.field("body_bytes", &self.body.len())
|
||||
.finish()
|
||||
}
|
||||
}
|
||||
|
||||
/// 当前 attempt 的准备输入与单次续体,归属不从返回请求重建
|
||||
pub struct OutboundCall {
|
||||
pub context: CallContext,
|
||||
pub metadata: wire::Metadata,
|
||||
pub request: OutboundRequest,
|
||||
pub next: OutboundNext,
|
||||
pub host: HostClient,
|
||||
pub cancellation: CallCancellation,
|
||||
}
|
||||
|
||||
impl MiddlewareInput for OutboundCall {
|
||||
type Output = OutboundRequest;
|
||||
|
||||
fn accepts(stage: Stage) -> bool {
|
||||
stage == Stage::Outbound
|
||||
}
|
||||
|
||||
fn decode(call: PluginCall) -> Result<Self, PluginFault> {
|
||||
if call.method != HANDLE_METHOD || !Self::accepts(call.context.stage) {
|
||||
return Err(invalid_input());
|
||||
}
|
||||
let head: wire::CallHead =
|
||||
serde_json::from_value(call.params).map_err(|_| invalid_input())?;
|
||||
Ok(Self {
|
||||
context: call.context,
|
||||
metadata: head.metadata,
|
||||
request: OutboundRequest {
|
||||
head: head.request,
|
||||
body: call.payload,
|
||||
},
|
||||
next: OutboundNext {
|
||||
host: call.host.clone(),
|
||||
},
|
||||
host: call.host,
|
||||
cancellation: call.cancellation,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl MiddlewareOutput for OutboundRequest {
|
||||
fn encode(self) -> Result<CallReply, PluginFault> {
|
||||
Ok(CallReply::unary(
|
||||
serde_json::to_value(self.head).map_err(|_| invalid_input())?,
|
||||
self.body,
|
||||
))
|
||||
}
|
||||
}
|
||||
|
||||
/// 消费续体只能进入下一层一次,不发送网络;外层仍可改写返回的完整请求
|
||||
pub struct OutboundNext {
|
||||
host: HostClient,
|
||||
}
|
||||
|
||||
impl OutboundNext {
|
||||
pub async fn run(self, request: OutboundRequest) -> Result<OutboundRequest, PluginFault> {
|
||||
let reply = self
|
||||
.host
|
||||
.call(
|
||||
NEXT_METHOD,
|
||||
serde_json::to_value(request.head).map_err(|_| invalid_input())?,
|
||||
request.body,
|
||||
)
|
||||
.await
|
||||
.map_err(SessionError::into_plugin_fault)?;
|
||||
Ok(OutboundRequest {
|
||||
head: serde_json::from_value(reply.result).map_err(|_| invalid_input())?,
|
||||
body: reply.payload,
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -27,7 +27,7 @@ use super::{
|
||||
///
|
||||
/// let contributes = Contributions::from([(Capability::Middleware, ContributionDeclaration {
|
||||
/// id: "acme.request-tags.tagRequest".into(),
|
||||
/// version: 5,
|
||||
/// version: 6,
|
||||
/// stages: vec![Stage::Request],
|
||||
/// input_formats: vec!["openai".into()],
|
||||
/// output_formats: vec!["openai".into()],
|
||||
@@ -67,7 +67,7 @@ where
|
||||
.contract_versions()
|
||||
.contains(&declaration.version)
|
||||
|| declaration.stages.is_empty()
|
||||
|| declaration.stages.len() > 5
|
||||
|| declaration.stages.len() > 6
|
||||
|| declaration.stages.iter().any(|stage| {
|
||||
!matches!(
|
||||
stage,
|
||||
@@ -76,6 +76,7 @@ where
|
||||
| Stage::Service
|
||||
| Stage::Request
|
||||
| Stage::Attempt
|
||||
| Stage::Outbound
|
||||
) || !C::accepts(*stage)
|
||||
})
|
||||
|| declaration
|
||||
|
||||
@@ -19,10 +19,10 @@ pub use http::{HostHttpBody, HostHttpResponse};
|
||||
pub use middleware::{
|
||||
HttpBody, HttpCall, HttpFrame, HttpNext, HttpRequest, HttpResponse, MiddlewareBody,
|
||||
MiddlewareBodySender, MiddlewareCall, MiddlewareInput, MiddlewareNext, MiddlewareOutput,
|
||||
MiddlewarePlugin, MiddlewareRequest, MiddlewareResponse, MiddlewareResult, RequestCall,
|
||||
ServiceCall, ServiceNext, ServiceResponse, TypedServiceCall, WebSocketCall, WebSocketDirection,
|
||||
WebSocketKind, WebSocketMessage, WebSocketNext, WebSocketPayload, WebSocketSender,
|
||||
WebSocketSession,
|
||||
MiddlewarePlugin, MiddlewareRequest, MiddlewareResponse, MiddlewareResult, OutboundCall,
|
||||
OutboundNext, OutboundRequest, RequestCall, ServiceCall, ServiceNext, ServiceResponse,
|
||||
TypedServiceCall, WebSocketCall, WebSocketDirection, WebSocketKind, WebSocketMessage,
|
||||
WebSocketNext, WebSocketPayload, WebSocketSender, WebSocketSession,
|
||||
};
|
||||
pub use plugin::{
|
||||
AuthorError, ComposedPlugin, Empty, Method, PluginBuilder, TypedCall, TypedReply, methods,
|
||||
|
||||
@@ -315,6 +315,7 @@ impl Manifest {
|
||||
| Stage::Service
|
||||
| Stage::Request
|
||||
| Stage::Attempt
|
||||
| Stage::Outbound
|
||||
)
|
||||
})))
|
||||
|| (*capability != Capability::Middleware
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
//! 验证中间件挂载、正文模式、增量头部与二进制帧的线协议合同
|
||||
|
||||
mod outbound;
|
||||
|
||||
use gateway_plugin_sdk::{
|
||||
Capability, Stage,
|
||||
call::middleware::{
|
||||
@@ -11,6 +13,47 @@ use gateway_plugin_sdk::{
|
||||
};
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn provider_parameters_distinguish_preserve_empty_and_invalid_replacement() {
|
||||
let preserved: MiddlewareNextRequest =
|
||||
serde_json::from_value(json!({"body":"preserve"})).unwrap();
|
||||
assert!(preserved.provider_parameters.is_none());
|
||||
for replacement in [json!({}), json!({"installation_id":null})] {
|
||||
let request: MiddlewareNextRequest = serde_json::from_value(json!({
|
||||
"body":"preserve", "provider_parameters":replacement,
|
||||
}))
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
request.provider_parameters.as_ref(),
|
||||
replacement.as_object()
|
||||
);
|
||||
}
|
||||
for invalid in [
|
||||
json!(null),
|
||||
json!([]),
|
||||
json!("invalid"),
|
||||
json!(42),
|
||||
json!(false),
|
||||
] {
|
||||
assert!(
|
||||
serde_json::from_value::<MiddlewareNextRequest>(json!({
|
||||
"body":"preserve", "provider_parameters":invalid,
|
||||
}))
|
||||
.is_err()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn settings_replacement_preserves_explicit_null_for_host_validation() {
|
||||
let preserved: MiddlewareNextRequest =
|
||||
serde_json::from_value(json!({"body":"preserve"})).unwrap();
|
||||
assert!(preserved.settings.is_none());
|
||||
let replaced: MiddlewareNextRequest =
|
||||
serde_json::from_value(json!({"body":"preserve", "settings":null})).unwrap();
|
||||
assert_eq!(replaced.settings, Some(serde_json::Value::Null));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn middleware_capability_and_mounts_have_one_stable_wire_vocabulary() {
|
||||
assert_eq!(
|
||||
@@ -32,6 +75,7 @@ fn middleware_capability_and_mounts_have_one_stable_wire_vocabulary() {
|
||||
#[test]
|
||||
fn middleware_request_keeps_body_out_of_json_and_rejects_unknown_fields() {
|
||||
let request = MiddlewareRequestHead {
|
||||
provider_parameters: None,
|
||||
settings_sources: serde_json::Value::Null,
|
||||
settings: serde_json::Value::Null,
|
||||
client_key_id: "fixture-key".into(),
|
||||
@@ -69,6 +113,7 @@ fn middleware_request_keeps_body_out_of_json_and_rejects_unknown_fields() {
|
||||
#[test]
|
||||
fn next_distinguishes_preserve_from_replace_empty_and_uses_incremental_headers() {
|
||||
let preserve = MiddlewareNextRequest {
|
||||
provider_parameters: None,
|
||||
settings: None,
|
||||
protocol: None,
|
||||
header_mutations: Vec::new(),
|
||||
@@ -81,6 +126,7 @@ fn next_distinguishes_preserve_from_replace_empty_and_uses_incremental_headers()
|
||||
);
|
||||
|
||||
let replace = MiddlewareNextRequest {
|
||||
provider_parameters: None,
|
||||
settings: None,
|
||||
protocol: Some("xai".into()),
|
||||
header_mutations: vec![
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
//! 出站候选目标和可信归属的独立 wire 合同
|
||||
|
||||
use gateway_plugin_sdk::call::middleware::outbound::{CallHead, RequestHead, Target};
|
||||
use serde_json::json;
|
||||
|
||||
#[test]
|
||||
fn outbound_websocket_head_keeps_duplicate_header_bytes() {
|
||||
let head: RequestHead = serde_json::from_value(json!({
|
||||
"target":{"kind":"websocket","uri":"wss://example.test/"},
|
||||
"headers":[{"name":"x-value","value":[128]},{"name":"x-value","value":[255]}]
|
||||
}))
|
||||
.unwrap();
|
||||
assert!(matches!(head.target, Target::WebSocket { .. }));
|
||||
assert_eq!(
|
||||
head.headers
|
||||
.iter()
|
||||
.map(|header| header.value.clone())
|
||||
.collect::<Vec<_>>(),
|
||||
vec![vec![128], vec![255]]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn outbound_return_cannot_replace_host_metadata() {
|
||||
let value = json!({"target":{"kind":"http","method":"POST","uri":"https://example.test/"},"headers":[],"metadata":{"account_id":"forged"}});
|
||||
assert!(serde_json::from_value::<RequestHead>(value).is_err());
|
||||
assert!(serde_json::from_value::<CallHead>(json!({"request":{},"metadata":{}})).is_err());
|
||||
}
|
||||
@@ -1,3 +1,25 @@
|
||||
//! 上游适配器调用合同的测试入口
|
||||
//! 验证上游适配器注册与调用合同
|
||||
|
||||
mod codec;
|
||||
|
||||
#[test]
|
||||
fn registration_round_trip_preserves_explicit_account_scopes() {
|
||||
use gateway_plugin_sdk::call::upstream_adapter::UpstreamAdapterRegistration;
|
||||
use serde_json::json;
|
||||
|
||||
for accounts in [json!([]), json!(["acct_one", "acct_two"])] {
|
||||
let wire = json!({"adapters":[{
|
||||
"id":"example", "provider":"openai", "base_url":"https://example.invalid/",
|
||||
"paths":[{"path":"responses","purpose":"inference"}],
|
||||
"authentication_kinds":["oauth"], "transport":"http_sse", "protocol":"openai",
|
||||
"account_ids":accounts
|
||||
}]});
|
||||
let registration: UpstreamAdapterRegistration = serde_json::from_value(wire).unwrap();
|
||||
let serialized = serde_json::to_value(registration).unwrap();
|
||||
let round_trip: UpstreamAdapterRegistration = serde_json::from_value(serialized).unwrap();
|
||||
assert_eq!(
|
||||
serde_json::to_value(&round_trip.adapters[0].account_ids).unwrap(),
|
||||
accounts
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
//! 验证类型化插件处理器的组合注册、分派与能力声明一致性
|
||||
|
||||
mod outbound;
|
||||
|
||||
use std::{
|
||||
collections::BTreeMap,
|
||||
sync::{
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
//! 出站准备处理器经真实 SDK 会话传递二进制正文并等待外层返回
|
||||
|
||||
use super::*;
|
||||
use gateway_plugin_sdk::call::middleware::{
|
||||
HANDLE_METHOD, MiddlewareHeader, NEXT_METHOD, outbound,
|
||||
};
|
||||
use gateway_plugin_sdk::client::{MiddlewareCall, MiddlewareResult, OutboundRequest};
|
||||
|
||||
#[tokio::test]
|
||||
async fn outbound_builder_preserves_binary_payload_and_outer_changes_after_next() {
|
||||
let manifest = gateway_plugin_sdk::Manifest::from_author_slice(&author_manifest(
|
||||
json!({"middleware":{"version":6,"stages":["outbound"]}}),
|
||||
json!([]),
|
||||
))
|
||||
.unwrap();
|
||||
let contributes = manifest.contributes.clone();
|
||||
let plugin = PluginBuilder::from_manifest(manifest)
|
||||
.unwrap()
|
||||
.middleware(|call: MiddlewareCall| async move {
|
||||
let MiddlewareCall::Outbound(call) = call else {
|
||||
return call.forward().await;
|
||||
};
|
||||
assert_eq!(call.metadata.account_id, "actual-account");
|
||||
assert_eq!(call.metadata.attempt_index, 3);
|
||||
let mut request = call.request;
|
||||
request.body.push(0x80);
|
||||
let mut request = call.next.run(request).await?;
|
||||
request.body.push(0xff);
|
||||
Ok(MiddlewareResult::from(request))
|
||||
})
|
||||
.unwrap()
|
||||
.build()
|
||||
.unwrap();
|
||||
let (mut host, task) = start_session(plugin, contributes).await;
|
||||
let request = outbound::RequestHead {
|
||||
target: outbound::Target::Http {
|
||||
method: "POST".into(),
|
||||
uri: "https://example.test/native?token=private".into(),
|
||||
},
|
||||
headers: vec![
|
||||
MiddlewareHeader {
|
||||
name: "x-raw".into(),
|
||||
value: vec![0x80],
|
||||
},
|
||||
MiddlewareHeader {
|
||||
name: "x-raw".into(),
|
||||
value: b"second".to_vec(),
|
||||
},
|
||||
],
|
||||
};
|
||||
let call = outbound::CallHead {
|
||||
request: request.clone(),
|
||||
metadata: outbound::Metadata {
|
||||
request_id: "request".into(),
|
||||
client_key_id: "key".into(),
|
||||
account_group_ids: vec!["group".into()],
|
||||
provider: "openai".into(),
|
||||
model: Some("model".into()),
|
||||
account_id: "actual-account".into(),
|
||||
attempt_index: 3,
|
||||
},
|
||||
};
|
||||
send_call(
|
||||
&mut host,
|
||||
1,
|
||||
HANDLE_METHOD,
|
||||
Stage::Outbound,
|
||||
serde_json::to_value(call).unwrap(),
|
||||
vec![0, 255, 0],
|
||||
)
|
||||
.await;
|
||||
let callback = receive(&mut host).await;
|
||||
let Message::Callback {
|
||||
id,
|
||||
parent_id,
|
||||
method,
|
||||
params,
|
||||
} = callback.message
|
||||
else {
|
||||
panic!("expected next callback")
|
||||
};
|
||||
assert_eq!((parent_id, method.as_str()), (1, NEXT_METHOD));
|
||||
assert_eq!(
|
||||
serde_json::from_value::<outbound::RequestHead>(params).unwrap(),
|
||||
request
|
||||
);
|
||||
assert_eq!(callback.payload, vec![0, 255, 0, 128]);
|
||||
write_frame(
|
||||
&mut host.writer,
|
||||
&Frame {
|
||||
message: Message::Result {
|
||||
id,
|
||||
result: serde_json::to_value(&request).unwrap(),
|
||||
},
|
||||
payload: callback.payload,
|
||||
},
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let (head, body) = unwrap_result(receive(&mut host).await, 1);
|
||||
assert_eq!(
|
||||
serde_json::from_value::<outbound::RequestHead>(head).unwrap(),
|
||||
request
|
||||
);
|
||||
assert_eq!(body, vec![0, 255, 0, 128, 255]);
|
||||
let debug = format!(
|
||||
"{:?}",
|
||||
OutboundRequest {
|
||||
head: request,
|
||||
body
|
||||
}
|
||||
);
|
||||
assert!(!debug.contains("private"));
|
||||
assert!(!debug.contains("second"));
|
||||
assert!(!debug.contains("255"));
|
||||
shutdown(&mut host, task).await;
|
||||
}
|
||||
@@ -308,7 +308,7 @@ fn middleware_contributions() -> Contributions {
|
||||
Capability::Middleware,
|
||||
ContributionDeclaration {
|
||||
id: "test.example.middleware".into(),
|
||||
version: 5,
|
||||
version: 6,
|
||||
stages: vec![Stage::Request],
|
||||
input_formats: vec!["openai".into()],
|
||||
output_formats: vec!["openai".into()],
|
||||
@@ -391,6 +391,7 @@ fn middleware_context(id: u64) -> CallContext {
|
||||
|
||||
pub(super) fn middleware_request() -> MiddlewareRequestHead {
|
||||
MiddlewareRequestHead {
|
||||
provider_parameters: None,
|
||||
settings_sources: serde_json::Value::Null,
|
||||
settings: serde_json::Value::Null,
|
||||
client_key_id: "fixture-key".into(),
|
||||
|
||||
@@ -23,7 +23,7 @@ fn source_manifest() -> Manifest {
|
||||
"contributes": {
|
||||
"middleware": {
|
||||
"id": "9acme.request-tags.tagRequest",
|
||||
"version": 5,
|
||||
"version": 6,
|
||||
"stages": ["request"],
|
||||
"inputFormats": ["openai"],
|
||||
"outputFormats": ["openai"]
|
||||
@@ -96,7 +96,7 @@ fn author_manifest_fills_default_ids_versions_and_fixed_stages() {
|
||||
"main": "bin/worker",
|
||||
"runtime": "trustedProcess",
|
||||
"contributes": {
|
||||
"middleware": {"version": 5, "stages": ["attempt"]},
|
||||
"middleware": {"version": 6, "stages": ["attempt"]},
|
||||
"management": {"version": 2}
|
||||
}
|
||||
});
|
||||
@@ -105,7 +105,7 @@ fn author_manifest_fills_default_ids_versions_and_fixed_stages() {
|
||||
let management = &manifest.contributes[&Capability::Management];
|
||||
|
||||
assert_eq!(middleware.id, "9acme.request-tags.middleware");
|
||||
assert_eq!(middleware.version, 5);
|
||||
assert_eq!(middleware.version, 6);
|
||||
assert_eq!(middleware.stages, [Stage::Attempt]);
|
||||
assert_eq!(management.id, "9acme.request-tags.management");
|
||||
assert_eq!(management.version, 2);
|
||||
@@ -133,9 +133,9 @@ fn installation_checks_protocol_engine_and_target() {
|
||||
#[test]
|
||||
fn capability_contract_versions_are_checked_independently_of_rpc_version() {
|
||||
for (capability, stage, supported, needs_formats) in [
|
||||
(Capability::Middleware, Stage::Request, 5, true),
|
||||
(Capability::Middleware, Stage::Request, 6, true),
|
||||
(Capability::Management, Stage::Management, 2, false),
|
||||
(Capability::UpstreamAdapter, Stage::Upstream, 4, true),
|
||||
(Capability::UpstreamAdapter, Stage::Upstream, 5, true),
|
||||
(Capability::Scheduler, Stage::Scheduling, 3, false),
|
||||
] {
|
||||
for mut manifest in [source_manifest(), packaged_manifest()] {
|
||||
@@ -377,7 +377,7 @@ fn manifest_round_trips_public_fields_and_rejects_unknown_fields() {
|
||||
#[test]
|
||||
fn contributes_rejects_duplicate_capability_keys_in_direct_json_parsing() {
|
||||
let declaration = r#"{
|
||||
"id":"test.example.middleware","version":5,"stages":["request"],
|
||||
"id":"test.example.middleware","version":6,"stages":["request"],
|
||||
"inputFormats":["openai"],"outputFormats":["openai"]
|
||||
}"#;
|
||||
let manifest = format!(
|
||||
@@ -412,7 +412,7 @@ fn contributes_rejects_duplicate_capability_keys_in_direct_json_parsing() {
|
||||
fn contribution_declaration_uses_camel_case_description_fields() {
|
||||
let declaration = ContributionDeclaration {
|
||||
id: "test.example.middleware".into(),
|
||||
version: 5,
|
||||
version: 6,
|
||||
stages: vec![Stage::Request],
|
||||
input_formats: vec!["openai".into()],
|
||||
output_formats: vec!["openai".into()],
|
||||
@@ -421,7 +421,7 @@ fn contribution_declaration_uses_camel_case_description_fields() {
|
||||
serde_json::to_value(declaration).unwrap(),
|
||||
json!({
|
||||
"id":"test.example.middleware",
|
||||
"version":5,
|
||||
"version":6,
|
||||
"stages":["request"],
|
||||
"inputFormats":["openai"],
|
||||
"outputFormats":["openai"]
|
||||
|
||||
@@ -241,6 +241,10 @@ async fn connect(
|
||||
)?);
|
||||
let (admissions, admission_release_writer) =
|
||||
redis::BufferedClientAdmissionPort::new(admissions, diagnostics.clone());
|
||||
let (budgets, budget_writer) = postgres::BufferedClientBudgetStore::new(
|
||||
postgres::PgClientBudgetStore::new(pool.clone()).with_timezone(config.timezone),
|
||||
diagnostics.clone(),
|
||||
);
|
||||
let core_ports = CoreStorePorts::new(
|
||||
execution,
|
||||
(
|
||||
@@ -260,9 +264,7 @@ async fn connect(
|
||||
Arc::new(client_key_usage),
|
||||
diagnostics.clone(),
|
||||
)
|
||||
.with_budget(Arc::new(
|
||||
postgres::PgClientBudgetStore::new(pool.clone()).with_timezone(config.timezone),
|
||||
))
|
||||
.with_budget(Arc::new(budgets))
|
||||
.with_session_affinity(Arc::clone(&provider_session_affinity))
|
||||
.with_client_retry_store(Arc::new(redis::RedisClientRetryRepository::new(
|
||||
redis_connection.clone(),
|
||||
@@ -304,6 +306,7 @@ async fn connect(
|
||||
execution_writer,
|
||||
client_key_usage_writer,
|
||||
admission_release_writer,
|
||||
budget_writer,
|
||||
)?,
|
||||
None,
|
||||
),
|
||||
@@ -313,6 +316,7 @@ async fn connect(
|
||||
execution: execution_writer,
|
||||
client_key_usage: client_key_usage_writer,
|
||||
admission_release: admission_release_writer,
|
||||
budget: budget_writer,
|
||||
}),
|
||||
),
|
||||
};
|
||||
|
||||
@@ -0,0 +1,344 @@
|
||||
//! 费用登记与后台落库分离,准入同时检查尚未落库的精确费用
|
||||
|
||||
use std::{
|
||||
collections::{BTreeMap, VecDeque},
|
||||
num::NonZeroUsize,
|
||||
sync::{Arc, Mutex},
|
||||
time::{Duration, Instant},
|
||||
};
|
||||
|
||||
use futures::future::BoxFuture;
|
||||
use gateway_core::{
|
||||
diagnostics::{OperationalDiagnostics, OperationalFailure},
|
||||
engine::budget::{ClientBudgetCharge, ClientBudgetError, ClientBudgetPort},
|
||||
error::{ErrorDetails, GatewayError},
|
||||
policy::ClientApiKeyId,
|
||||
task::{DaemonTask, WorkerDaemonContext, WorkerTaskError},
|
||||
};
|
||||
use tokio::sync::Notify;
|
||||
|
||||
use super::{PgClientBudgetStore, client_budgets::unavailable};
|
||||
|
||||
const DEFAULT_PENDING_CAPACITY: NonZeroUsize =
|
||||
NonZeroUsize::new(4_096).expect("non-zero queue capacity");
|
||||
const ADMISSION_TIMEOUT: Duration = Duration::from_secs(1);
|
||||
const WRITE_TIMEOUT: Duration = Duration::from_secs(5);
|
||||
const DRAIN_TIMEOUT: Duration = Duration::from_secs(3);
|
||||
const WRITE_BATCH_SIZE: usize = 32;
|
||||
|
||||
#[derive(Default)]
|
||||
struct PendingCharges {
|
||||
by_key: BTreeMap<String, BTreeMap<String, ClientBudgetCharge>>,
|
||||
ready: VecDeque<(String, String)>,
|
||||
count: usize,
|
||||
closing: bool,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
struct BudgetQueue {
|
||||
pending: Mutex<PendingCharges>,
|
||||
changed: Notify,
|
||||
}
|
||||
|
||||
/// 单副本运行时先接管费用,再由受监督的 writer 落库和重试
|
||||
pub struct BufferedClientBudgetStore {
|
||||
inner: Arc<PgClientBudgetStore>,
|
||||
queue: Arc<BudgetQueue>,
|
||||
capacity: NonZeroUsize,
|
||||
}
|
||||
|
||||
impl BufferedClientBudgetStore {
|
||||
#[must_use]
|
||||
pub fn new(
|
||||
inner: PgClientBudgetStore,
|
||||
diagnostics: Arc<dyn OperationalDiagnostics>,
|
||||
) -> (Self, ClientBudgetWriter) {
|
||||
Self::with_capacity(inner, DEFAULT_PENDING_CAPACITY, diagnostics)
|
||||
}
|
||||
|
||||
#[must_use]
|
||||
pub fn with_capacity(
|
||||
inner: PgClientBudgetStore,
|
||||
capacity: NonZeroUsize,
|
||||
diagnostics: Arc<dyn OperationalDiagnostics>,
|
||||
) -> (Self, ClientBudgetWriter) {
|
||||
let inner = Arc::new(inner);
|
||||
let queue = Arc::new(BudgetQueue::default());
|
||||
(
|
||||
Self {
|
||||
inner: inner.clone(),
|
||||
queue: queue.clone(),
|
||||
capacity,
|
||||
},
|
||||
ClientBudgetWriter {
|
||||
inner,
|
||||
queue,
|
||||
diagnostics,
|
||||
batch_limits: tokio::sync::Mutex::new(BTreeMap::new()),
|
||||
},
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
impl ClientBudgetPort for BufferedClientBudgetStore {
|
||||
fn admit(&self, key_id: ClientApiKeyId) -> BoxFuture<'_, Result<(), GatewayError>> {
|
||||
Box::pin(async move {
|
||||
let pending = {
|
||||
let state = self.queue.pending.lock().map_err(|_| unavailable())?;
|
||||
if state.closing || state.count >= self.capacity.get() {
|
||||
return Err(unavailable());
|
||||
}
|
||||
state
|
||||
.by_key
|
||||
.get(key_id.as_str())
|
||||
.map(|charges| charges.values().cloned().collect::<Vec<_>>())
|
||||
.unwrap_or_default()
|
||||
};
|
||||
tokio::time::timeout(
|
||||
ADMISSION_TIMEOUT,
|
||||
self.inner.admit_with_pending(key_id, &pending),
|
||||
)
|
||||
.await
|
||||
.map_err(|source| unavailable().with_source(source))?
|
||||
})
|
||||
}
|
||||
|
||||
fn settle(&self, charge: ClientBudgetCharge) -> BoxFuture<'_, Result<(), ClientBudgetError>> {
|
||||
Box::pin(async move {
|
||||
let mut state = self
|
||||
.queue
|
||||
.pending
|
||||
.lock()
|
||||
.map_err(|_| ClientBudgetError(None))?;
|
||||
let key = charge.key_id.as_str().to_owned();
|
||||
let request = charge.request_id.as_str().to_owned();
|
||||
let charges = state.by_key.entry(key.clone()).or_default();
|
||||
if charges.contains_key(&request) {
|
||||
return Ok(());
|
||||
}
|
||||
charges.insert(request.clone(), charge);
|
||||
state.ready.push_back((key, request));
|
||||
state.count += 1;
|
||||
// 达到容量后拒绝新准入,但已准入请求的费用必须接收,不能丢弃或反向等待写入
|
||||
self.queue.changed.notify_one();
|
||||
if state.closing {
|
||||
return Err(ClientBudgetError(None));
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// 每次仅占一个写入任务;任务重启、取消和超时后仍保留同一笔费用
|
||||
pub struct ClientBudgetWriter {
|
||||
inner: Arc<PgClientBudgetStore>,
|
||||
queue: Arc<BudgetQueue>,
|
||||
diagnostics: Arc<dyn OperationalDiagnostics>,
|
||||
// 独占 writer,同时保留各 Key 跨任务重启的缩批进度
|
||||
batch_limits: tokio::sync::Mutex<BTreeMap<String, usize>>,
|
||||
}
|
||||
|
||||
struct PendingWrite {
|
||||
queue: Arc<BudgetQueue>,
|
||||
key: String,
|
||||
charges: Vec<ClientBudgetCharge>,
|
||||
}
|
||||
|
||||
impl PendingWrite {
|
||||
fn complete(&self) -> Result<bool, WorkerTaskError> {
|
||||
let mut state = self.queue.pending.lock().map_err(|_| queue_error())?;
|
||||
if let Some(charges) = state.by_key.get_mut(&self.key) {
|
||||
let before = charges.len();
|
||||
for charge in &self.charges {
|
||||
charges.remove(charge.request_id.as_str());
|
||||
}
|
||||
let removed = before - charges.len();
|
||||
let empty = charges.is_empty();
|
||||
state.count -= removed;
|
||||
if empty {
|
||||
state.by_key.remove(&self.key);
|
||||
}
|
||||
}
|
||||
Ok(!state.by_key.contains_key(&self.key))
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for PendingWrite {
|
||||
fn drop(&mut self) {
|
||||
let Ok(mut state) = self.queue.pending.lock() else {
|
||||
return;
|
||||
};
|
||||
// 只有提交确认后才移除;future 被取消时重排队,由请求 ID 处理提交结果未知
|
||||
for charge in &self.charges {
|
||||
let request = charge.request_id.as_str();
|
||||
if state
|
||||
.by_key
|
||||
.get(&self.key)
|
||||
.is_some_and(|charges| charges.contains_key(request))
|
||||
{
|
||||
state
|
||||
.ready
|
||||
.push_back((self.key.clone(), request.to_owned()));
|
||||
self.queue.changed.notify_one();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl ClientBudgetWriter {
|
||||
fn next(
|
||||
&self,
|
||||
batch_limits: &BTreeMap<String, usize>,
|
||||
) -> Result<Option<PendingWrite>, WorkerTaskError> {
|
||||
let mut state = self.queue.pending.lock().map_err(|_| queue_error())?;
|
||||
let Some((key, _)) = state.ready.front() else {
|
||||
return Ok(None);
|
||||
};
|
||||
let key = key.clone();
|
||||
let limit = batch_limits.get(&key).copied().unwrap_or(WRITE_BATCH_SIZE);
|
||||
let mut requests = Vec::with_capacity(limit);
|
||||
// 混合 Key 的请求也能合批;限制单批数量,并保持其他 Key 的相对顺序
|
||||
state.ready.retain(|(candidate, request)| {
|
||||
if candidate == &key && requests.len() < limit {
|
||||
requests.push(request.clone());
|
||||
false
|
||||
} else {
|
||||
true
|
||||
}
|
||||
});
|
||||
let charges = requests
|
||||
.iter()
|
||||
.map(|request| {
|
||||
state
|
||||
.by_key
|
||||
.get(&key)
|
||||
.and_then(|charges| charges.get(request))
|
||||
.cloned()
|
||||
.ok_or_else(queue_error)
|
||||
})
|
||||
.collect::<Result<Vec<_>, _>>()?;
|
||||
Ok(Some(PendingWrite {
|
||||
queue: self.queue.clone(),
|
||||
key,
|
||||
charges,
|
||||
}))
|
||||
}
|
||||
|
||||
async fn write(
|
||||
&self,
|
||||
batch_limits: &mut BTreeMap<String, usize>,
|
||||
pending: PendingWrite,
|
||||
) -> Result<(), WorkerTaskError> {
|
||||
let started = Instant::now();
|
||||
let result =
|
||||
match tokio::time::timeout(WRITE_TIMEOUT, self.inner.settle_charges(&pending.charges))
|
||||
.await
|
||||
{
|
||||
Ok(result) => result,
|
||||
Err(source) => Err(ClientBudgetError(Some(source.into()))),
|
||||
};
|
||||
if let Err(source) = result {
|
||||
// 失败后只缩小当前 Key,避免固定大批次持续超过写入时限
|
||||
batch_limits.insert(pending.key.clone(), (pending.charges.len() / 2).max(1));
|
||||
let mut failure = OperationalFailure::new(
|
||||
"store",
|
||||
"settle_client_budget",
|
||||
"budget_unavailable",
|
||||
"Client budget settlement failed; background writer will retry",
|
||||
);
|
||||
failure.correlation_id = pending
|
||||
.charges
|
||||
.first()
|
||||
.map(|charge| charge.request_id.as_str().to_owned());
|
||||
failure.details = ErrorDetails::capture(Some(&source.clone().into()), None, false);
|
||||
let _ = self.diagnostics.record_failure(failure).await;
|
||||
return Err(
|
||||
WorkerTaskError::safe("client budget settlement failed").with_source(source)
|
||||
);
|
||||
}
|
||||
if pending.complete()? {
|
||||
batch_limits.remove(&pending.key);
|
||||
} else if let Some(limit) = batch_limits.get_mut(&pending.key)
|
||||
&& pending.charges.len() == *limit
|
||||
&& started.elapsed() < WRITE_TIMEOUT / 4
|
||||
{
|
||||
// 满批写入足够快才恢复吞吐,保留余量避免慢库反复放大到超时
|
||||
*limit = (*limit * 2).min(WRITE_BATCH_SIZE);
|
||||
if *limit == WRITE_BATCH_SIZE {
|
||||
batch_limits.remove(&pending.key);
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn drain(
|
||||
&self,
|
||||
batch_limits: &mut BTreeMap<String, usize>,
|
||||
) -> Result<(), WorkerTaskError> {
|
||||
self.queue
|
||||
.pending
|
||||
.lock()
|
||||
.map_err(|_| queue_error())?
|
||||
.closing = true;
|
||||
let result = tokio::time::timeout(DRAIN_TIMEOUT, async {
|
||||
while let Some(pending) = self.next(batch_limits)? {
|
||||
self.write(batch_limits, pending).await?;
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
.await;
|
||||
let result = result
|
||||
.map_err(|source| {
|
||||
WorkerTaskError::safe("client budget shutdown drain timed out").with_source(source)
|
||||
})
|
||||
.and_then(std::convert::identity);
|
||||
if let Err(error) = &result {
|
||||
tracing::warn!(operation = "settle_client_budget", %error, "费用队列关闭排空未完成");
|
||||
}
|
||||
result
|
||||
}
|
||||
}
|
||||
|
||||
impl DaemonTask for ClientBudgetWriter {
|
||||
fn run(&self, context: WorkerDaemonContext) -> BoxFuture<'_, Result<(), WorkerTaskError>> {
|
||||
Box::pin(async move {
|
||||
let mut batch_limits = self.batch_limits.lock().await;
|
||||
loop {
|
||||
let write = async {
|
||||
match self.next(&batch_limits)? {
|
||||
Some(pending) => {
|
||||
self.write(&mut batch_limits, pending).await?;
|
||||
context.succeeded();
|
||||
}
|
||||
None => self.queue.changed.notified().await,
|
||||
}
|
||||
Ok::<(), WorkerTaskError>(())
|
||||
};
|
||||
tokio::select! {
|
||||
biased;
|
||||
() = context.cancellation().cancelled() => return self.drain(&mut batch_limits).await,
|
||||
result = write => result?,
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for ClientBudgetWriter {
|
||||
fn drop(&mut self) {
|
||||
if let Ok(mut state) = self.queue.pending.lock() {
|
||||
state.closing = true;
|
||||
if state.count > 0 {
|
||||
tracing::warn!(
|
||||
operation = "settle_client_budget",
|
||||
pending_charges = state.count,
|
||||
"费用写入任务已销毁,仍有未落库费用"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn queue_error() -> WorkerTaskError {
|
||||
WorkerTaskError::safe("client budget settlement queue is unavailable")
|
||||
}
|
||||
@@ -1,7 +1,7 @@
|
||||
//! 按 Key 串行检查限额,并幂等累计已取得的 USD 费用
|
||||
//! 读取 Key 预算快照,并按 Key 幂等累计已取得的 USD 费用
|
||||
|
||||
use gateway_admin::model::audit::MutationAuditOperation;
|
||||
use std::{collections::BTreeMap, sync::Mutex, time::Duration};
|
||||
use std::{collections::BTreeMap, time::Duration};
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
use futures::future::BoxFuture;
|
||||
@@ -129,7 +129,6 @@ async fn reset_client_key_budget_in_transaction(
|
||||
pub struct PgClientBudgetStore {
|
||||
timezone: gateway_core::time::DeploymentTimeZone,
|
||||
pool: PgPool,
|
||||
retry: Mutex<BTreeMap<String, ClientBudgetCharge>>,
|
||||
}
|
||||
|
||||
impl PgClientBudgetStore {
|
||||
@@ -138,7 +137,6 @@ impl PgClientBudgetStore {
|
||||
Self {
|
||||
timezone: Default::default(),
|
||||
pool,
|
||||
retry: Mutex::new(BTreeMap::new()),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -148,88 +146,103 @@ impl PgClientBudgetStore {
|
||||
self
|
||||
}
|
||||
|
||||
async fn admit_inner(&self, key_id: ClientApiKeyId) -> Result<(), GatewayError> {
|
||||
// 短暂存储故障后按原金额重试;进程退出丢失的费用不转成人工核账或阻断 Key
|
||||
let retries = self
|
||||
.retry
|
||||
.lock()
|
||||
.map_err(|_| unavailable())?
|
||||
.values()
|
||||
.filter(|charge| charge.key_id == key_id)
|
||||
.cloned()
|
||||
pub(super) async fn admit_with_pending(
|
||||
&self,
|
||||
key_id: ClientApiKeyId,
|
||||
pending: &[ClientBudgetCharge],
|
||||
) -> Result<(), GatewayError> {
|
||||
// 同一语句快照同时读取窗口和已落库请求,避免提交与队列移除之间重复计算费用
|
||||
// 普通准入只读,不等待后台结算持有的 Key 行锁
|
||||
let ids = pending
|
||||
.iter()
|
||||
.map(|charge| charge.request_id.as_str())
|
||||
.collect::<Vec<_>>();
|
||||
for charge in retries {
|
||||
self.settle(charge)
|
||||
.await
|
||||
.map_err(|source| unavailable().with_source(source))?;
|
||||
}
|
||||
let mut tx = self
|
||||
.pool
|
||||
.begin()
|
||||
.await
|
||||
.map_err(|source| unavailable().with_source(source))?;
|
||||
let row = sqlx::query(
|
||||
"select daily_limit_usd::text, weekly_limit_usd::text, enabled
|
||||
from client_api_keys where id = $1 for update",
|
||||
"select k.enabled, k.daily_limit_usd::text, k.weekly_limit_usd::text,
|
||||
w.daily_start, w.daily_end, w.weekly_start, w.weekly_end,
|
||||
coalesce(w.daily_used_usd, 0)::text as daily_used,
|
||||
coalesce(w.weekly_used_usd, 0)::text as weekly_used,
|
||||
case when k.daily_limit_usd = 0 and k.weekly_limit_usd = 0 then array[]::text[]
|
||||
else array(select request_id from client_api_key_charge_events where request_id = any($2))
|
||||
end as persisted
|
||||
from client_api_keys k left join client_api_key_budget_windows w on w.client_api_key_id = k.id
|
||||
where k.id = $1",
|
||||
)
|
||||
.bind(key_id.as_str())
|
||||
.fetch_optional(&mut *tx)
|
||||
.await
|
||||
.bind(key_id.as_str()).bind(ids).fetch_optional(&self.pool).await
|
||||
.map_err(|source| unavailable().with_source(source))?
|
||||
.ok_or_else(|| {
|
||||
GatewayError::new(
|
||||
GatewayErrorKind::Unauthorized,
|
||||
"client API key no longer exists",
|
||||
)
|
||||
})?;
|
||||
.ok_or_else(|| GatewayError::new(GatewayErrorKind::Unauthorized, "client API key no longer exists"))?;
|
||||
if !row.get::<bool, _>("enabled") {
|
||||
return Err(GatewayError::new(
|
||||
GatewayErrorKind::PolicyDenied,
|
||||
"client API key is disabled",
|
||||
));
|
||||
}
|
||||
let parse = |field: &str| -> Result<Decimal, GatewayError> {
|
||||
row.get::<String, _>(field)
|
||||
.parse()
|
||||
.map_err(|source| unavailable().with_source(source))
|
||||
};
|
||||
let limits = ClientBudgetLimits {
|
||||
daily_usd: row
|
||||
.get::<String, _>("daily_limit_usd")
|
||||
.parse()
|
||||
.map_err(|source| unavailable().with_source(source))?,
|
||||
weekly_usd: row
|
||||
.get::<String, _>("weekly_limit_usd")
|
||||
.parse()
|
||||
.map_err(|source| unavailable().with_source(source))?,
|
||||
daily_usd: parse("daily_limit_usd")?,
|
||||
weekly_usd: parse("weekly_limit_usd")?,
|
||||
};
|
||||
let now = Utc::now();
|
||||
advance_windows(&mut tx, key_id.as_str(), now, now, self.timezone)
|
||||
.await
|
||||
.map_err(|source| unavailable().with_source(source))?;
|
||||
let day = self.timezone.day_start(now).ok_or_else(unavailable)?;
|
||||
let window = |start: &str, end: &str, used: &str, days| -> Result<_, GatewayError> {
|
||||
let start = row.get::<Option<DateTime<Utc>>, _>(start);
|
||||
let end = row.get::<Option<DateTime<Utc>>, _>(end);
|
||||
let reopen = end.is_none_or(|end| end <= now);
|
||||
if reopen {
|
||||
Ok((
|
||||
day.max(start.unwrap_or(day)).max(end.unwrap_or(day)),
|
||||
self.timezone
|
||||
.days_after(now, days)
|
||||
.ok_or_else(unavailable)?,
|
||||
Decimal::ZERO,
|
||||
true,
|
||||
))
|
||||
} else {
|
||||
Ok((
|
||||
start.ok_or_else(unavailable)?,
|
||||
end.ok_or_else(unavailable)?,
|
||||
parse(used)?,
|
||||
false,
|
||||
))
|
||||
}
|
||||
};
|
||||
let (daily_start, daily_end, mut daily, open_daily) =
|
||||
window("daily_start", "daily_end", "daily_used", 1)?;
|
||||
let (weekly_start, weekly_end, mut weekly, open_weekly) =
|
||||
window("weekly_start", "weekly_end", "weekly_used", 7)?;
|
||||
if limits.is_limited() {
|
||||
let window = sqlx::query(
|
||||
"select daily_used_usd::text, weekly_used_usd::text, daily_end, weekly_end
|
||||
from client_api_key_budget_windows where client_api_key_id = $1",
|
||||
)
|
||||
.bind(key_id.as_str())
|
||||
.fetch_one(&mut *tx)
|
||||
.await
|
||||
.map_err(|source| unavailable().with_source(source))?;
|
||||
let daily: Decimal = window
|
||||
.get::<String, _>("daily_used_usd")
|
||||
.parse()
|
||||
.map_err(|source| unavailable().with_source(source))?;
|
||||
let weekly: Decimal = window
|
||||
.get::<String, _>("weekly_used_usd")
|
||||
.parse()
|
||||
.map_err(|source| unavailable().with_source(source))?;
|
||||
let persisted = row
|
||||
.get::<Vec<String>, _>("persisted")
|
||||
.into_iter()
|
||||
.collect::<std::collections::BTreeSet<_>>();
|
||||
for charge in pending {
|
||||
if persisted.contains(charge.request_id.as_str()) {
|
||||
continue;
|
||||
}
|
||||
let completed_at = DateTime::<Utc>::from(charge.completed_at);
|
||||
if completed_at >= daily_start && completed_at < daily_end {
|
||||
daily = daily
|
||||
.checked_add(charge.amount_usd)
|
||||
.ok_or_else(unavailable)?;
|
||||
}
|
||||
if completed_at >= weekly_start && completed_at < weekly_end {
|
||||
weekly = weekly
|
||||
.checked_add(charge.amount_usd)
|
||||
.ok_or_else(unavailable)?;
|
||||
}
|
||||
}
|
||||
let daily_exceeded = limits.daily_usd != Decimal::ZERO && daily >= limits.daily_usd;
|
||||
let weekly_exceeded = limits.weekly_usd != Decimal::ZERO && weekly >= limits.weekly_usd;
|
||||
if daily_exceeded || weekly_exceeded {
|
||||
let daily_end: DateTime<Utc> = window.get("daily_end");
|
||||
let weekly_end: DateTime<Utc> = window.get("weekly_end");
|
||||
let reset = if weekly_exceeded {
|
||||
weekly_end
|
||||
} else {
|
||||
daily_end
|
||||
};
|
||||
let retry = (reset - now).to_std().unwrap_or(Duration::from_secs(1));
|
||||
return Err(GatewayError::new(
|
||||
GatewayErrorKind::RateLimited,
|
||||
"client API key budget is exhausted",
|
||||
@@ -239,30 +252,59 @@ impl PgClientBudgetStore {
|
||||
} else {
|
||||
"key_daily_budget_exceeded"
|
||||
})
|
||||
.with_retry_after(retry));
|
||||
.with_retry_after((reset - now).to_std().unwrap_or(Duration::from_secs(1))));
|
||||
}
|
||||
}
|
||||
tx.commit()
|
||||
.await
|
||||
.map_err(|source| unavailable().with_source(source))
|
||||
// 首次使用或窗口到期才写入;与重置、结算保持相同锁序
|
||||
if open_daily || open_weekly {
|
||||
let mut tx = self
|
||||
.pool
|
||||
.begin()
|
||||
.await
|
||||
.map_err(|source| unavailable().with_source(source))?;
|
||||
sqlx::query("select id from client_api_keys where id = $1 for update")
|
||||
.bind(key_id.as_str())
|
||||
.fetch_optional(&mut *tx)
|
||||
.await
|
||||
.map_err(|source| unavailable().with_source(source))?
|
||||
.ok_or_else(|| {
|
||||
GatewayError::new(
|
||||
GatewayErrorKind::Unauthorized,
|
||||
"client API key no longer exists",
|
||||
)
|
||||
})?;
|
||||
advance_windows(&mut tx, key_id.as_str(), now, now, self.timezone)
|
||||
.await
|
||||
.map_err(|source| unavailable().with_source(source))?;
|
||||
tx.commit()
|
||||
.await
|
||||
.map_err(|source| unavailable().with_source(source))?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
async fn settle_inner(&self, charge: &ClientBudgetCharge) -> Result<(), ClientBudgetError> {
|
||||
pub(super) async fn settle_charges(
|
||||
&self,
|
||||
charges: &[ClientBudgetCharge],
|
||||
) -> Result<(), ClientBudgetError> {
|
||||
let Some(first) = charges.first() else {
|
||||
return Ok(());
|
||||
};
|
||||
let mut tx = self
|
||||
.pool
|
||||
.begin()
|
||||
.await
|
||||
.map_err(|source| ClientBudgetError(Some(source.into())))?;
|
||||
// 与准入统一先锁 Key,再写窗口和费用,串行化同一 Key 的并发结算
|
||||
// 与窗口开启和重置统一先锁 Key,再写窗口和费用
|
||||
let key = sqlx::query_scalar::<_, String>(
|
||||
"select id from client_api_keys where id = $1 for update",
|
||||
)
|
||||
.bind(charge.key_id.as_str())
|
||||
.bind(first.key_id.as_str())
|
||||
.fetch_optional(&mut *tx)
|
||||
.await
|
||||
.map_err(|source| ClientBudgetError(Some(source.into())))?;
|
||||
let Some(key) = key else { return Ok(()) }; // 删除 Key 时也会删除其费用记录
|
||||
settle_in_transaction(&mut tx, &key, charge, self.timezone)
|
||||
settle_in_transaction(&mut tx, &key, charges, self.timezone)
|
||||
.await
|
||||
.map_err(|source| ClientBudgetError(Some(source.into())))?;
|
||||
tx.commit()
|
||||
@@ -274,58 +316,65 @@ impl PgClientBudgetStore {
|
||||
async fn settle_in_transaction(
|
||||
tx: &mut Transaction<'_, Postgres>,
|
||||
key: &str,
|
||||
charge: &ClientBudgetCharge,
|
||||
charges: &[ClientBudgetCharge],
|
||||
timezone: gateway_core::time::DeploymentTimeZone,
|
||||
) -> Result<(), sqlx::Error> {
|
||||
let completed_at = DateTime::<Utc>::from(charge.completed_at);
|
||||
// 仅在请求结束时写入费用;请求 ID 冲突时不重复累计
|
||||
let changed = sqlx::query(
|
||||
let ids = charges
|
||||
.iter()
|
||||
.map(|charge| charge.request_id.as_str())
|
||||
.collect::<Vec<_>>();
|
||||
let amounts = charges
|
||||
.iter()
|
||||
.map(|charge| charge.amount_usd.canonical())
|
||||
.collect::<Vec<_>>();
|
||||
let times = charges
|
||||
.iter()
|
||||
.map(|charge| DateTime::<Utc>::from(charge.completed_at))
|
||||
.collect::<Vec<_>>();
|
||||
// 同一 Key 的小批次共用事务,只累计本次真正插入的事件
|
||||
let inserted = sqlx::query(
|
||||
"insert into client_api_key_charge_events (request_id, client_api_key_id, amount_usd, completed_at)
|
||||
values ($1, $2, $3::text::numeric, $4)
|
||||
on conflict (request_id) do nothing",
|
||||
select request_id, $1, amount::numeric, completed_at
|
||||
from unnest($2::text[], $3::text[], $4::timestamptz[]) as charges(request_id, amount, completed_at)
|
||||
on conflict (request_id) do nothing returning amount_usd::text as amount, completed_at",
|
||||
).bind(key).bind(ids).bind(amounts).bind(times).fetch_all(&mut **tx).await?;
|
||||
let amounts = inserted
|
||||
.iter()
|
||||
.map(|row| row.get::<String, _>("amount"))
|
||||
.collect::<Vec<_>>();
|
||||
let times = inserted
|
||||
.iter()
|
||||
.map(|row| row.get::<DateTime<Utc>, _>("completed_at"))
|
||||
.collect::<Vec<_>>();
|
||||
let Some(latest) = times.iter().max().copied() else {
|
||||
return Ok(());
|
||||
};
|
||||
advance_windows(tx, key, Utc::now(), latest, timezone).await?;
|
||||
sqlx::query(
|
||||
"update client_api_key_budget_windows w set
|
||||
daily_used_usd = daily_used_usd + coalesce((select sum(amount::numeric)
|
||||
from unnest($2::text[], $3::timestamptz[]) as charges(amount, completed_at)
|
||||
where completed_at >= w.daily_start and completed_at < w.daily_end), 0),
|
||||
weekly_used_usd = weekly_used_usd + coalesce((select sum(amount::numeric)
|
||||
from unnest($2::text[], $3::timestamptz[]) as charges(amount, completed_at)
|
||||
where completed_at >= w.weekly_start and completed_at < w.weekly_end), 0)
|
||||
where client_api_key_id = $1",
|
||||
)
|
||||
.bind(charge.request_id.as_str())
|
||||
.bind(key)
|
||||
.bind(charge.amount_usd.canonical())
|
||||
.bind(completed_at)
|
||||
.bind(amounts)
|
||||
.bind(times)
|
||||
.execute(&mut **tx)
|
||||
.await?
|
||||
.rows_affected();
|
||||
if changed == 1 {
|
||||
advance_windows(tx, key, Utc::now(), completed_at, timezone).await?;
|
||||
sqlx::query("update client_api_key_budget_windows set
|
||||
daily_used_usd = daily_used_usd + case when $3 >= daily_start and $3 < daily_end then $2::text::numeric else 0 end,
|
||||
weekly_used_usd = weekly_used_usd + case when $3 >= weekly_start and $3 < weekly_end then $2::text::numeric else 0 end
|
||||
where client_api_key_id = $1")
|
||||
.bind(key).bind(charge.amount_usd.canonical()).bind(completed_at)
|
||||
.execute(&mut **tx).await?;
|
||||
}
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
impl ClientBudgetPort for PgClientBudgetStore {
|
||||
fn admit(&self, key_id: ClientApiKeyId) -> BoxFuture<'_, Result<(), GatewayError>> {
|
||||
Box::pin(async move { self.admit_inner(key_id).await })
|
||||
Box::pin(async move { self.admit_with_pending(key_id, &[]).await })
|
||||
}
|
||||
|
||||
fn settle(&self, charge: ClientBudgetCharge) -> BoxFuture<'_, Result<(), ClientBudgetError>> {
|
||||
Box::pin(async move {
|
||||
let result = self.settle_inner(&charge).await;
|
||||
// 重试缓存的锁异常不能覆盖已捕获的账本失败
|
||||
let mut retry = self.retry.lock().map_err(|_| {
|
||||
result
|
||||
.as_ref()
|
||||
.err()
|
||||
.cloned()
|
||||
.unwrap_or(ClientBudgetError(None))
|
||||
})?;
|
||||
if result.is_err() {
|
||||
retry.insert(charge.request_id.as_str().to_owned(), charge);
|
||||
} else {
|
||||
retry.remove(charge.request_id.as_str());
|
||||
}
|
||||
result
|
||||
})
|
||||
Box::pin(async move { self.settle_charges(std::slice::from_ref(&charge)).await })
|
||||
}
|
||||
}
|
||||
|
||||
@@ -346,7 +395,7 @@ async fn advance_windows(
|
||||
let weekly_end = timezone
|
||||
.days_after(now, 7)
|
||||
.ok_or_else(|| sqlx::Error::Protocol("invalid budget week end".to_owned()))?;
|
||||
// Key 行锁仍串行化准入和结算;未滚动的窗口无需产生新元组与 WAL
|
||||
// 调用方持有 Key 行锁;未滚动的窗口无需产生新元组与 WAL
|
||||
sqlx::query("insert into client_api_key_budget_windows
|
||||
(client_api_key_id, daily_start, daily_end, weekly_start, weekly_end)
|
||||
select $1, $4, $5, $4, $6
|
||||
@@ -426,7 +475,7 @@ pub(super) async fn load_client_key_budgets(
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn unavailable() -> GatewayError {
|
||||
pub(super) fn unavailable() -> GatewayError {
|
||||
GatewayError::new(
|
||||
GatewayErrorKind::ProviderInfrastructureUnavailable,
|
||||
"client budget service is temporarily unavailable",
|
||||
|
||||
@@ -8,6 +8,7 @@ use serde_json::Value;
|
||||
use sqlx::PgPool;
|
||||
use uuid::Uuid;
|
||||
|
||||
use gateway_core::diagnostics::MAX_TRACE_SNAPSHOT_BYTES;
|
||||
use gateway_core::engine::{
|
||||
AttemptRecord as CoreAttemptRecord, ExecutionStore, IntermediateFailure,
|
||||
ModelRequestFinalization as CoreModelRequestFinalization, ModelRequestId,
|
||||
@@ -326,13 +327,11 @@ impl ModelRequestFinalization {
|
||||
{
|
||||
return Err(invalid("provider observation must be a JSON object"));
|
||||
}
|
||||
if self
|
||||
.diagnostic_trace_json
|
||||
.as_ref()
|
||||
.is_some_and(|trace| !trace.is_object() || trace.to_string().len() > 64 * 1024)
|
||||
{
|
||||
if self.diagnostic_trace_json.as_ref().is_some_and(|trace| {
|
||||
!trace.is_object() || trace.to_string().len() > MAX_TRACE_SNAPSHOT_BYTES
|
||||
}) {
|
||||
return Err(invalid(
|
||||
"diagnostic trace must be a JSON object within 64 KiB",
|
||||
"diagnostic trace must be a JSON object within the snapshot size limit",
|
||||
));
|
||||
}
|
||||
validate_optional_stable_reason(
|
||||
|
||||
@@ -4,6 +4,7 @@ mod account_groups;
|
||||
mod admin_security_audit;
|
||||
mod admission_recovery;
|
||||
mod backup;
|
||||
mod client_budget_buffer;
|
||||
mod client_budgets;
|
||||
mod client_keys;
|
||||
mod connection;
|
||||
@@ -27,6 +28,7 @@ pub use account_groups::*;
|
||||
pub use admin_security_audit::*;
|
||||
pub use admission_recovery::*;
|
||||
pub use backup::*;
|
||||
pub use client_budget_buffer::{BufferedClientBudgetStore, ClientBudgetWriter};
|
||||
pub use client_budgets::PgClientBudgetStore;
|
||||
pub use client_keys::*;
|
||||
pub use connection::connect_and_migrate;
|
||||
|
||||
@@ -11,9 +11,10 @@ pub(crate) struct CommandStoreWriters {
|
||||
pub(crate) execution: postgres::ExecutionObservationWriter<postgres::PgExecutionStore>,
|
||||
pub(crate) client_key_usage: postgres::PgClientApiKeyUsageWriter,
|
||||
pub(crate) admission_release: redis::ClientAdmissionReleaseWriter,
|
||||
pub(crate) budget: postgres::ClientBudgetWriter,
|
||||
}
|
||||
|
||||
/// 短生命周期 CLI 只运行数据面必需的三个写泵,不注册恢复、保留或维护 Worker
|
||||
/// 短生命周期 CLI 只运行数据面必需的四个写泵,不注册恢复、保留或维护 Worker
|
||||
pub struct CommandStoreDrain {
|
||||
cancellation: gateway_core::lifecycle::CancellationToken,
|
||||
tasks: Vec<tokio::task::JoinHandle<Result<(), WorkerTaskError>>>,
|
||||
@@ -31,12 +32,14 @@ impl CommandStoreWriters {
|
||||
execution,
|
||||
client_key_usage,
|
||||
admission_release,
|
||||
budget,
|
||||
} = self;
|
||||
let execution_idle = execution.idle();
|
||||
let tasks = vec![
|
||||
spawn_command_writer(execution, cancellation.child_token()),
|
||||
spawn_command_writer(client_key_usage, cancellation.child_token()),
|
||||
spawn_command_writer(admission_release, cancellation.child_token()),
|
||||
spawn_command_writer(budget, cancellation.child_token()),
|
||||
];
|
||||
CommandStoreDrain {
|
||||
cancellation,
|
||||
@@ -101,6 +104,7 @@ pub(crate) fn store_worker_contributions(
|
||||
execution_writer: postgres::ExecutionObservationWriter<postgres::PgExecutionStore>,
|
||||
client_key_usage_writer: postgres::PgClientApiKeyUsageWriter,
|
||||
admission_release_writer: redis::ClientAdmissionReleaseWriter,
|
||||
budget_writer: postgres::ClientBudgetWriter,
|
||||
) -> StoreResult<Vec<WorkerContribution>> {
|
||||
let stale_id = WorkerId::try_new(WorkerKind::StaleModelRequestRecovery, "postgres")
|
||||
.map_err(worker_definition_error)?;
|
||||
@@ -111,10 +115,22 @@ pub(crate) fn store_worker_contributions(
|
||||
.map_err(worker_definition_error)?;
|
||||
let admission_flush_id = WorkerId::try_new(WorkerKind::OpsFlush, "redis_admission")
|
||||
.map_err(worker_definition_error)?;
|
||||
let budget_flush_id = WorkerId::try_new(WorkerKind::OpsFlush, "postgres_client_budget")
|
||||
.map_err(worker_definition_error)?;
|
||||
let ops_flush_restart =
|
||||
DaemonRestartPolicy::try_new(Duration::from_secs(1), Duration::from_secs(60))
|
||||
.map_err(worker_definition_error)?;
|
||||
Ok(vec![
|
||||
WorkerContribution::Registration(
|
||||
WorkerRegistration::try_new(
|
||||
budget_flush_id,
|
||||
WorkerRunnable::Daemon {
|
||||
restart: ops_flush_restart,
|
||||
task: Box::new(budget_writer),
|
||||
},
|
||||
)
|
||||
.map_err(worker_definition_error)?,
|
||||
),
|
||||
WorkerContribution::Registration(scheduled_worker(
|
||||
stale_id,
|
||||
Duration::from_secs(30),
|
||||
|
||||
@@ -0,0 +1,368 @@
|
||||
//! 验证慢结算隔离、待结算额度、幂等重试和后台写入生命周期
|
||||
|
||||
use std::{num::NonZeroUsize, sync::Arc, time::Duration};
|
||||
|
||||
use gateway_admin::{
|
||||
model::client_keys::{
|
||||
ClientKeyBudgetMutationOrigin, ClientKeyBudgetPeriod, ResetClientKeyBudget,
|
||||
},
|
||||
ports::store::ClientKeyStore as _,
|
||||
};
|
||||
use gateway_core::{
|
||||
engine::budget::ClientBudgetPort,
|
||||
lifecycle::CancellationToken,
|
||||
task::{DaemonTask, WorkerDaemonContext},
|
||||
};
|
||||
use gateway_store::postgres::{
|
||||
BufferedClientBudgetStore, ClientBudgetWriter, PgAdminClientKeyStore, PgClientBudgetStore,
|
||||
};
|
||||
|
||||
use super::{
|
||||
TestDatabase,
|
||||
client_budgets::{charge, context, key_id, seed, status},
|
||||
};
|
||||
|
||||
fn buffered(pool: sqlx::PgPool) -> (BufferedClientBudgetStore, Arc<ClientBudgetWriter>) {
|
||||
let (store, writer) = BufferedClientBudgetStore::new(
|
||||
PgClientBudgetStore::new(pool),
|
||||
Arc::new(crate::RecordingDiagnostics::default()),
|
||||
);
|
||||
(store, Arc::new(writer))
|
||||
}
|
||||
|
||||
async fn wait_for_writes(context: &WorkerDaemonContext, expected: u64) {
|
||||
tokio::time::timeout(Duration::from_secs(5), async {
|
||||
loop {
|
||||
let completed = context.completed_cycles();
|
||||
if completed >= expected {
|
||||
break;
|
||||
}
|
||||
context.progress_since(completed).await;
|
||||
}
|
||||
})
|
||||
.await
|
||||
.expect("background budget writes must complete");
|
||||
}
|
||||
|
||||
async fn drain(writer: &ClientBudgetWriter) {
|
||||
let cancellation = CancellationToken::new();
|
||||
cancellation.cancel();
|
||||
writer
|
||||
.run(WorkerDaemonContext::new(cancellation))
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn slow_settlement_does_not_block_receipt_or_concurrent_admission() {
|
||||
let Some(database) = TestDatabase::create("budget_buffer_slow").await else {
|
||||
return;
|
||||
};
|
||||
let direct = PgClientBudgetStore::new(database.pool.clone());
|
||||
for (key, daily) in [("allowed", "10"), ("exhausted", "1")] {
|
||||
seed(&database, key, daily, "0").await;
|
||||
direct.admit(key_id(key)).await.unwrap();
|
||||
}
|
||||
// 控制连接独立于数据面连接池,保证测试只阻塞结算所需行锁
|
||||
let pool = sqlx::postgres::PgPoolOptions::new()
|
||||
.max_connections(4)
|
||||
.connect_with((*database.pool.connect_options()).clone())
|
||||
.await
|
||||
.unwrap();
|
||||
let (store, writer) = buffered(pool.clone());
|
||||
let mut blocker = database.pool.begin().await.unwrap();
|
||||
sqlx::query("select id from client_api_keys for update")
|
||||
.fetch_all(&mut *blocker)
|
||||
.await
|
||||
.unwrap();
|
||||
for key in ["allowed", "exhausted"] {
|
||||
let fee = charge(key, key, "1.25");
|
||||
tokio::time::timeout(Duration::from_millis(250), async {
|
||||
store.settle(fee.clone()).await.unwrap();
|
||||
store.settle(fee).await.unwrap();
|
||||
})
|
||||
.await
|
||||
.expect("completion must only enqueue, including duplicate receipts");
|
||||
}
|
||||
let worker_context = WorkerDaemonContext::new(CancellationToken::new());
|
||||
let running_context = worker_context.clone();
|
||||
let task = tokio::spawn(async move { writer.run(running_context).await });
|
||||
tokio::time::timeout(Duration::from_millis(500), async {
|
||||
for result in
|
||||
futures::future::join_all((0..4).map(|_| store.admit(key_id("allowed")))).await
|
||||
{
|
||||
result.unwrap();
|
||||
}
|
||||
let error = store.admit(key_id("exhausted")).await.unwrap_err();
|
||||
assert_eq!(error.client_error_code(), Some("key_daily_budget_exceeded"));
|
||||
})
|
||||
.await
|
||||
.expect("admission must not wait for settlement row locks");
|
||||
let before: i64 = sqlx::query_scalar("select count(*) from client_api_key_charge_events")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(before, 0);
|
||||
blocker.commit().await.unwrap();
|
||||
wait_for_writes(&worker_context, 2).await;
|
||||
worker_context.cancellation().cancel();
|
||||
task.await.unwrap().unwrap();
|
||||
assert_eq!(
|
||||
status(&database, "allowed")
|
||||
.await
|
||||
.daily_used_usd
|
||||
.canonical(),
|
||||
"1.25"
|
||||
);
|
||||
assert_eq!(
|
||||
status(&database, "exhausted")
|
||||
.await
|
||||
.daily_used_usd
|
||||
.canonical(),
|
||||
"1.25"
|
||||
);
|
||||
pool.close().await;
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn pending_cost_respects_committed_ids_and_manual_reset_boundaries() {
|
||||
let Some(database) = TestDatabase::create("budget_buffer_reset").await else {
|
||||
return;
|
||||
};
|
||||
seed(&database, "key", "2", "0").await;
|
||||
let direct = PgClientBudgetStore::new(database.pool.clone());
|
||||
let (store, writer) = buffered(database.pool.clone());
|
||||
store.admit(key_id("key")).await.unwrap();
|
||||
let committed = charge("key", "committed", "1.25");
|
||||
store.settle(committed.clone()).await.unwrap();
|
||||
direct.settle(committed).await.unwrap();
|
||||
// 模拟提交成功、队列尚未移除,金额不能按 2.5 重复计算
|
||||
store.admit(key_id("key")).await.unwrap();
|
||||
store
|
||||
.settle(charge("key", "before-reset", "3"))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
store
|
||||
.admit(key_id("key"))
|
||||
.await
|
||||
.unwrap_err()
|
||||
.client_error_code(),
|
||||
Some("key_daily_budget_exceeded")
|
||||
);
|
||||
PgAdminClientKeyStore::new(database.pool.clone())
|
||||
.reset_client_key_budget(
|
||||
ResetClientKeyBudget {
|
||||
id: key_id("key"),
|
||||
period: ClientKeyBudgetPeriod::All,
|
||||
},
|
||||
ClientKeyBudgetMutationOrigin::Admin,
|
||||
&context(),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
store.admit(key_id("key")).await.unwrap();
|
||||
store
|
||||
.settle(charge("key", "after-reset", "2.5"))
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
store
|
||||
.admit(key_id("key"))
|
||||
.await
|
||||
.unwrap_err()
|
||||
.client_error_code(),
|
||||
Some("key_daily_budget_exceeded")
|
||||
);
|
||||
drain(&writer).await;
|
||||
assert_eq!(
|
||||
status(&database, "key").await.daily_used_usd.canonical(),
|
||||
"2.5"
|
||||
);
|
||||
assert_eq!(
|
||||
status(&database, "key").await.weekly_used_usd.canonical(),
|
||||
"2.5"
|
||||
);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn failed_write_retains_cost_and_retries_without_another_request() {
|
||||
let Some(database) = TestDatabase::create("budget_buffer_retry").await else {
|
||||
return;
|
||||
};
|
||||
seed(&database, "key", "1", "0").await;
|
||||
let (store, writer) = buffered(database.pool.clone());
|
||||
store.admit(key_id("key")).await.unwrap();
|
||||
sqlx::raw_sql(
|
||||
"create sequence write_attempts;
|
||||
create function reject_budget_update() returns trigger language plpgsql as $$ begin
|
||||
perform nextval('write_attempts');
|
||||
raise exception 'injected settlement failure';
|
||||
end $$;
|
||||
create trigger reject_budget_update before update on client_api_key_budget_windows
|
||||
for each row execute function reject_budget_update();",
|
||||
)
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
for (request, amount) in [("retry-a", "0.25"), ("retry-b", "0.5"), ("retry-c", "0.5")] {
|
||||
store.settle(charge("key", request, amount)).await.unwrap();
|
||||
}
|
||||
writer
|
||||
.run(WorkerDaemonContext::new(CancellationToken::new()))
|
||||
.await
|
||||
.unwrap_err();
|
||||
for result in futures::future::join_all((0..4).map(|_| store.admit(key_id("key")))).await {
|
||||
assert_eq!(
|
||||
result.unwrap_err().client_error_code(),
|
||||
Some("key_daily_budget_exceeded")
|
||||
);
|
||||
}
|
||||
let attempts: i64 = sqlx::query_scalar("select last_value from write_attempts")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(attempts, 1, "admission never retries writes");
|
||||
let rows: i64 = sqlx::query_scalar("select count(*) from client_api_key_charge_events")
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(rows, 0, "the entire batch rolls back");
|
||||
assert_eq!(
|
||||
status(&database, "key").await.daily_used_usd.canonical(),
|
||||
"0"
|
||||
);
|
||||
sqlx::query("drop trigger reject_budget_update on client_api_key_budget_windows")
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
let worker_context = WorkerDaemonContext::new(CancellationToken::new());
|
||||
let running_context = worker_context.clone();
|
||||
let task = tokio::spawn(async move { writer.run(running_context).await });
|
||||
wait_for_writes(&worker_context, 1).await;
|
||||
worker_context.cancellation().cancel();
|
||||
task.await.unwrap().unwrap();
|
||||
assert_eq!(
|
||||
status(&database, "key").await.daily_used_usd.canonical(),
|
||||
"1.25"
|
||||
);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cancelled_writer_requeues_inflight_cost_for_shutdown_drain() {
|
||||
let Some(database) = TestDatabase::create("budget_buffer_cancel").await else {
|
||||
return;
|
||||
};
|
||||
seed(&database, "key", "0", "0").await;
|
||||
let name = format!("budget_cancel_{}", uuid::Uuid::new_v4().simple());
|
||||
let pool = sqlx::postgres::PgPoolOptions::new()
|
||||
.max_connections(2)
|
||||
.connect_with(
|
||||
(*database.pool.connect_options())
|
||||
.clone()
|
||||
.application_name(&name),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
let (store, writer) = buffered(pool.clone());
|
||||
store.admit(key_id("key")).await.unwrap();
|
||||
let mut blocker = database.pool.begin().await.unwrap();
|
||||
sqlx::query("select id from client_api_keys for update")
|
||||
.fetch_all(&mut *blocker)
|
||||
.await
|
||||
.unwrap();
|
||||
store
|
||||
.settle(charge("key", "cancelled", "0.75"))
|
||||
.await
|
||||
.unwrap();
|
||||
let running = writer.clone();
|
||||
let task = tokio::spawn(async move {
|
||||
running
|
||||
.run(WorkerDaemonContext::new(CancellationToken::new()))
|
||||
.await
|
||||
});
|
||||
tokio::time::timeout(Duration::from_secs(2), async {
|
||||
loop {
|
||||
let blocked: bool = sqlx::query_scalar("select exists(select 1 from pg_stat_activity where application_name=$1 and wait_event_type='Lock')")
|
||||
.bind(&name).fetch_one(&database.pool).await.unwrap();
|
||||
if blocked { break; }
|
||||
tokio::time::sleep(Duration::from_millis(5)).await;
|
||||
}
|
||||
}).await.unwrap();
|
||||
task.abort();
|
||||
assert!(task.await.unwrap_err().is_cancelled());
|
||||
blocker.commit().await.unwrap();
|
||||
drain(&writer).await;
|
||||
assert_eq!(
|
||||
status(&database, "key").await.daily_used_usd.canonical(),
|
||||
"0.75"
|
||||
);
|
||||
pool.close().await;
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn full_queue_rejects_new_admission_but_keeps_already_completed_costs() {
|
||||
let Some(database) = TestDatabase::create("budget_buffer_full").await else {
|
||||
return;
|
||||
};
|
||||
for key in ["key", "other"] {
|
||||
seed(&database, key, "0", "0").await;
|
||||
}
|
||||
let (store, writer) = BufferedClientBudgetStore::with_capacity(
|
||||
PgClientBudgetStore::new(database.pool.clone()),
|
||||
NonZeroUsize::new(2).unwrap(),
|
||||
Arc::new(crate::RecordingDiagnostics::default()),
|
||||
);
|
||||
for (i, (key, amount)) in [("key", "1"), ("other", "0.25"), ("key", "2")]
|
||||
.into_iter()
|
||||
.enumerate()
|
||||
{
|
||||
store
|
||||
.settle(charge(key, &format!("fee-{i}"), amount))
|
||||
.await
|
||||
.unwrap();
|
||||
}
|
||||
assert_eq!(
|
||||
store
|
||||
.admit(key_id("key"))
|
||||
.await
|
||||
.unwrap_err()
|
||||
.client_error_code(),
|
||||
Some("key_budget_unavailable")
|
||||
);
|
||||
drain(&writer).await;
|
||||
assert_eq!(
|
||||
status(&database, "key").await.daily_used_usd.canonical(),
|
||||
"3"
|
||||
);
|
||||
assert_eq!(
|
||||
status(&database, "other").await.daily_used_usd.canonical(),
|
||||
"0.25"
|
||||
);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn stalled_budget_read_has_a_total_admission_deadline() {
|
||||
let Some(database) = TestDatabase::create("budget_buffer_deadline").await else {
|
||||
return;
|
||||
};
|
||||
seed(&database, "key", "0", "0").await;
|
||||
let (store, _writer) = buffered(database.pool.clone());
|
||||
let mut blocker = database.pool.begin().await.unwrap();
|
||||
sqlx::query("lock table client_api_keys in access exclusive mode")
|
||||
.execute(&mut *blocker)
|
||||
.await
|
||||
.unwrap();
|
||||
let error = tokio::time::timeout(Duration::from_millis(1_500), store.admit(key_id("key")))
|
||||
.await
|
||||
.expect("budget query must have a total deadline")
|
||||
.unwrap_err();
|
||||
assert_eq!(error.client_error_code(), Some("key_budget_unavailable"));
|
||||
blocker.rollback().await.unwrap();
|
||||
database.close().await;
|
||||
}
|
||||
@@ -33,11 +33,11 @@ use gateway_store::postgres::{
|
||||
|
||||
use super::TestDatabase;
|
||||
|
||||
fn key_id(key: &str) -> ClientApiKeyId {
|
||||
pub(super) fn key_id(key: &str) -> ClientApiKeyId {
|
||||
ClientApiKeyId::new(key).unwrap()
|
||||
}
|
||||
|
||||
fn charge(key: &str, request: &str, amount: &str) -> ClientBudgetCharge {
|
||||
pub(super) fn charge(key: &str, request: &str, amount: &str) -> ClientBudgetCharge {
|
||||
ClientBudgetCharge {
|
||||
key_id: key_id(key),
|
||||
request_id: ModelRequestId::new(format!("req_{request}")).unwrap(),
|
||||
@@ -46,7 +46,7 @@ fn charge(key: &str, request: &str, amount: &str) -> ClientBudgetCharge {
|
||||
}
|
||||
}
|
||||
|
||||
async fn seed(database: &TestDatabase, key: &str, daily: &str, weekly: &str) {
|
||||
pub(super) async fn seed(database: &TestDatabase, key: &str, daily: &str, weekly: &str) {
|
||||
sqlx::query(
|
||||
"insert into client_api_keys (id, name, key, daily_limit_usd, weekly_limit_usd, created_at, updated_at)
|
||||
values ($1, $1, $2, $3::text::numeric, $4::text::numeric, now(), now())",
|
||||
@@ -60,7 +60,7 @@ async fn seed(database: &TestDatabase, key: &str, daily: &str, weekly: &str) {
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
async fn status(database: &TestDatabase, key: &str) -> ClientBudgetStatus {
|
||||
pub(super) async fn status(database: &TestDatabase, key: &str) -> ClientBudgetStatus {
|
||||
PgClientApiKeyRepository::new(database.pool.clone())
|
||||
.get_client_api_key(key)
|
||||
.await
|
||||
@@ -69,7 +69,7 @@ async fn status(database: &TestDatabase, key: &str) -> ClientBudgetStatus {
|
||||
.budget
|
||||
}
|
||||
|
||||
fn context() -> MutationContext {
|
||||
pub(super) fn context() -> MutationContext {
|
||||
MutationContext {
|
||||
actor: MutationActor::System,
|
||||
request_id: "budget-test".to_owned(),
|
||||
@@ -687,7 +687,7 @@ async fn zero_cost_and_interrupted_requests_never_block_limited_keys() {
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn transient_settlement_failure_rolls_back_and_retries_exact_cost_before_admission() {
|
||||
async fn settlement_failure_rolls_back_and_explicit_retry_is_idempotent() {
|
||||
let Some(database) = TestDatabase::create("budgets_retry").await else {
|
||||
return;
|
||||
};
|
||||
@@ -721,6 +721,7 @@ async fn transient_settlement_failure_rolls_back_and_retries_exact_cost_before_a
|
||||
.execute(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
store.settle(charge("key", "retry", "1.25")).await.unwrap();
|
||||
let error = store.admit(key_id("key")).await.unwrap_err();
|
||||
assert_eq!(error.client_error_code(), Some("key_daily_budget_exceeded"));
|
||||
store.settle(charge("key", "retry", "1.25")).await.unwrap();
|
||||
|
||||
@@ -8,7 +8,7 @@ use chrono::{DateTime, Duration, Utc};
|
||||
use gateway_admin::{
|
||||
model::observability as admin_observability, ports::store::ObservabilityStore as _,
|
||||
};
|
||||
use gateway_core::diagnostics::TraceContext;
|
||||
use gateway_core::diagnostics::{MAX_TRACE_SNAPSHOT_BYTES, TraceContext};
|
||||
use gateway_core::engine::{
|
||||
ExecutionOutcome, ExecutionStore, ModelRequestFailureObservation,
|
||||
ModelRequestFinalization as CoreModelRequestFinalization, ModelRequestId,
|
||||
@@ -1429,54 +1429,100 @@ async fn seed_running_request(pool: &sqlx::PgPool, id: &str) -> Result<(), sqlx:
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn diagnostic_trace_is_finalized_atomically_and_available_for_failed_request_details() {
|
||||
async fn diagnostic_trace_pressure_preserves_terminal_outcomes_and_usage() {
|
||||
let Some(database) = TestDatabase::create("diagnostic_trace").await else {
|
||||
return;
|
||||
};
|
||||
seed_running_request(&database.pool, "req_diagnostic_failed")
|
||||
.await
|
||||
.unwrap();
|
||||
let store = PgExecutionStore::new(database.pool.clone());
|
||||
let mut finalization = successful_core_finalization("req_diagnostic_failed");
|
||||
finalization.outcome = ExecutionOutcome::Failed;
|
||||
finalization.client_status_code = Some(502);
|
||||
let diagnostic = gateway_core::error::ProviderDiagnostic::new(
|
||||
"OpenAI WebSocket receive idle timeout after 300s",
|
||||
)
|
||||
.with_classification("receive", "receive_idle_timeout");
|
||||
let provider_error = gateway_core::error::ProviderError::new(
|
||||
gateway_core::error::ProviderErrorKind::Timeout,
|
||||
UpstreamSendState::Ambiguous,
|
||||
)
|
||||
.with_diagnostic(diagnostic);
|
||||
finalization.error = Some(GatewayError::from_provider(&provider_error));
|
||||
let trace = serde_json::json!({"schemaVersion": 1, "requestId": "req_diagnostic_failed", "events": [
|
||||
{"attemptIndex": 1, "stage": "attempt.failed", "data": {"diagnostic": {
|
||||
"stage": "receive", "code": "receive_idle_timeout", "message": "OpenAI WebSocket receive idle timeout after 300s"
|
||||
}}},
|
||||
]});
|
||||
finalization.diagnostic_trace_json = Some(trace.to_string());
|
||||
ExecutionStore::finalize_model_request(&store, finalization)
|
||||
.await
|
||||
.unwrap();
|
||||
let persisted: (String, serde_json::Value, String) = sqlx::query_as(
|
||||
"select outcome, diagnostic_trace_json, error_message from model_request_observations where id = 'req_diagnostic_failed'"
|
||||
).fetch_one(&database.pool).await.unwrap();
|
||||
assert_eq!(
|
||||
persisted,
|
||||
(
|
||||
"failed".to_owned(),
|
||||
trace.clone(),
|
||||
"OpenAI WebSocket receive idle timeout after 300s".to_owned()
|
||||
)
|
||||
);
|
||||
let repository = super::observability_repository(&database.pool);
|
||||
let detail = repository
|
||||
.usage_record_detail("req_diagnostic_failed")
|
||||
for (id, outcome, expected_outcome) in [
|
||||
(
|
||||
"req_diagnostic_success",
|
||||
ExecutionOutcome::Succeeded,
|
||||
"succeeded",
|
||||
),
|
||||
("req_diagnostic_failed", ExecutionOutcome::Failed, "failed"),
|
||||
] {
|
||||
seed_running_request(&database.pool, id).await.unwrap();
|
||||
let mut finalization = successful_core_finalization(id);
|
||||
finalization.outcome = outcome;
|
||||
finalization.usage.input_tokens = Some(100_000);
|
||||
finalization.usage.output_tokens = Some(2000);
|
||||
finalization.usage.total_tokens = Some(102_000);
|
||||
finalization.cost = CalculatedCost::from_usd_ticks(10_000_000_000)
|
||||
.unwrap()
|
||||
.into_estimate();
|
||||
finalization.timings.first_token_ms = Some(120);
|
||||
finalization.timings.latency_ms = Some(42_000);
|
||||
let trace = TraceContext::new(id);
|
||||
trace.record("request.routed", json!({"scope": "all"}));
|
||||
for stage in ["provider.request.input", "provider.request.outbound"] {
|
||||
trace.record_evidence(stage, json!({"hashes": "a".repeat(6000)}));
|
||||
}
|
||||
for index in 0..500 {
|
||||
trace.record(
|
||||
"stream.sample",
|
||||
json!({"index": index, "detail": "x".repeat(2000)}),
|
||||
);
|
||||
}
|
||||
let error_message = if outcome == ExecutionOutcome::Failed {
|
||||
finalization.client_status_code = Some(502);
|
||||
let message = "OpenAI WebSocket receive idle timeout after 300s";
|
||||
let provider_error = gateway_core::error::ProviderError::new(
|
||||
gateway_core::error::ProviderErrorKind::Timeout,
|
||||
UpstreamSendState::Ambiguous,
|
||||
)
|
||||
.with_diagnostic(
|
||||
gateway_core::error::ProviderDiagnostic::new(message)
|
||||
.with_classification("receive", "receive_idle_timeout"),
|
||||
);
|
||||
trace.record_provider_failure(&provider_error);
|
||||
finalization.error = Some(GatewayError::from_provider(&provider_error));
|
||||
Some(message)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
trace.record("request.finished", json!({"outcome": expected_outcome}));
|
||||
let snapshot = trace.snapshot().unwrap();
|
||||
let encoded = snapshot.to_string();
|
||||
assert!(encoded.len() > 64 * 1024);
|
||||
assert!(encoded.len() <= MAX_TRACE_SNAPSHOT_BYTES);
|
||||
assert!(snapshot["droppedEvents"].as_u64().unwrap() > 0);
|
||||
finalization.diagnostic_trace_json = Some(encoded);
|
||||
ExecutionStore::finalize_model_request(&store, finalization)
|
||||
.await
|
||||
.expect("trace pressure must not discard the terminal result");
|
||||
let persisted: Value = sqlx::query_scalar(
|
||||
"select jsonb_build_object(
|
||||
'outcome', outcome, 'input', input_tokens, 'output', output_tokens,
|
||||
'cost', cost_amount::text, 'firstToken', first_token_ms,
|
||||
'latency', latency_ms, 'error', error_message)
|
||||
from model_request_observations where id = $1",
|
||||
)
|
||||
.bind(id)
|
||||
.fetch_one(&database.pool)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(detail.trace, Some(trace));
|
||||
assert!(detail.related_requests.is_empty());
|
||||
assert_eq!(
|
||||
persisted,
|
||||
json!({
|
||||
"outcome": expected_outcome, "input": 100_000, "output": 2000,
|
||||
"cost": "1.0000000000", "firstToken": 120, "latency": 42_000,
|
||||
"error": error_message,
|
||||
})
|
||||
);
|
||||
let detail = repository.usage_record_detail(id).await.unwrap();
|
||||
assert_eq!(detail.trace, Some(snapshot));
|
||||
assert!(detail.related_requests.is_empty());
|
||||
}
|
||||
assert_eq!(
|
||||
store
|
||||
.recover_expired_model_requests(Utc::now() + Duration::minutes(20))
|
||||
.await
|
||||
.unwrap()
|
||||
.requests,
|
||||
0,
|
||||
);
|
||||
database.close().await;
|
||||
}
|
||||
|
||||
@@ -1784,7 +1830,7 @@ async fn zero_attempt_finalization_rejects_sent_or_missing_attempt_facts_atomica
|
||||
for invalid_trace in [
|
||||
"not JSON".to_owned(),
|
||||
"[]".to_owned(),
|
||||
json!({"data": "x".repeat(65_536)}).to_string(),
|
||||
json!({"data": "x".repeat(MAX_TRACE_SNAPSHOT_BYTES)}).to_string(),
|
||||
] {
|
||||
let mut finalization = early_failure(&request);
|
||||
finalization.diagnostic_trace_json = Some(invalid_trace);
|
||||
|
||||
@@ -20,6 +20,7 @@ mod analytics_buckets;
|
||||
mod analytics_reset;
|
||||
mod backup;
|
||||
mod backup_export;
|
||||
mod client_budget_buffer;
|
||||
mod client_budgets;
|
||||
mod client_keys;
|
||||
mod connection;
|
||||
|
||||
@@ -51,7 +51,7 @@ sha2 = "0.11.0"
|
||||
subtle = "2.6.1"
|
||||
tar = { version = "0.4.46", default-features = false }
|
||||
thiserror = "2.0.18"
|
||||
tokio = { version = "=1.52.3", default-features = false, features = ["fs", "macros", "rt", "sync", "time"] }
|
||||
tokio = { version = "=1.53.3", default-features = false, features = ["fs", "macros", "rt", "sync", "time"] }
|
||||
tokio-rustls = { version = "=0.26.4", default-features = false }
|
||||
tokio-util = { version = "0.7.18", features = ["rt"] }
|
||||
tower-service = "0.3.3"
|
||||
@@ -76,7 +76,7 @@ openssl-sys = { version = "=0.9.111", features = ["vendored"] }
|
||||
[dev-dependencies]
|
||||
allocation-counter = "0.8.1"
|
||||
tempfile = "3.27.0"
|
||||
tokio = { version = "1.52.3", default-features = false, features = ["rt-multi-thread", "test-util"] }
|
||||
tokio = { version = "1.53.3", default-features = false, features = ["rt-multi-thread", "test-util"] }
|
||||
tracing-subscriber = { version = "0.3.23", features = ["json"] }
|
||||
wiremock = "0.6.5"
|
||||
|
||||
|
||||
@@ -165,6 +165,7 @@ impl<'a> CodexSelectionModel<'a> {
|
||||
}
|
||||
|
||||
struct CredentialSelectionInput<'a> {
|
||||
managed_upstream: bool,
|
||||
requires_websocket: bool,
|
||||
requires_oauth: bool,
|
||||
request_url: &'a Url,
|
||||
@@ -350,6 +351,7 @@ impl CodexCredentialSelector {
|
||||
request: &SelectCodexCredential<'_>,
|
||||
) -> Result<CodexCredentialLease, CredentialSelectionError> {
|
||||
let input = CredentialSelectionInput {
|
||||
managed_upstream: false,
|
||||
requires_websocket: false,
|
||||
requires_oauth: false,
|
||||
request_url: request.request_url,
|
||||
@@ -375,6 +377,7 @@ impl CodexCredentialSelector {
|
||||
guardian: bool,
|
||||
) -> Result<CodexCredentialLease, CredentialSelectionError> {
|
||||
let input = CredentialSelectionInput {
|
||||
managed_upstream: true,
|
||||
requires_websocket,
|
||||
requires_oauth: false,
|
||||
request_url: request.request_url,
|
||||
@@ -450,6 +453,7 @@ impl CodexCredentialSelector {
|
||||
request: &SelectCodexProviderEndpointCredential<'_>,
|
||||
) -> Result<CodexCredentialLease, CredentialSelectionError> {
|
||||
let input = CredentialSelectionInput {
|
||||
managed_upstream: false,
|
||||
requires_websocket: false,
|
||||
requires_oauth: request.requires_oauth,
|
||||
request_url: request.request_url,
|
||||
@@ -506,6 +510,13 @@ impl CodexCredentialSelector {
|
||||
model: CodexSelectionModel<'_>,
|
||||
follow_only: bool,
|
||||
) -> Result<CodexCredentialLease, CredentialSelectionError> {
|
||||
let adapter_model = match model {
|
||||
CodexSelectionModel::Requested(model) if request.managed_upstream => Some(
|
||||
gateway_core::routing::UpstreamModelId::new(model)
|
||||
.map_err(|_| CredentialSelectionError::NoEligibleCredential)?,
|
||||
),
|
||||
_ => None,
|
||||
};
|
||||
let diagnostic = request.attempt.is_diagnostic_required_account();
|
||||
let binding_key = request.session_affinity_key.filter(|_| !diagnostic);
|
||||
let affinity_mode = request
|
||||
@@ -633,7 +644,26 @@ impl CodexCredentialSelector {
|
||||
.collect::<Vec<_>>();
|
||||
let mut eligible = Vec::with_capacity(accounts.len());
|
||||
for account in accounts {
|
||||
let adapter = if let Some(model) = adapter_model.as_ref() {
|
||||
match request.attempt.upstream_adapter_for_account(
|
||||
&self.provider_kind,
|
||||
model,
|
||||
account.id(),
|
||||
) {
|
||||
Ok(adapter) => adapter,
|
||||
// 已匹配但不可用的适配器不能回退到该账号的原生目标
|
||||
Err(_) => continue,
|
||||
}
|
||||
} else {
|
||||
None
|
||||
};
|
||||
if adapter.as_ref().is_some_and(|adapter| {
|
||||
!adapter.supports_authentication(account.authentication_kind())
|
||||
}) {
|
||||
continue;
|
||||
}
|
||||
if request.requires_websocket
|
||||
&& adapter.is_none()
|
||||
&& pinned_account.as_ref().is_none_or(|id| id == account.id())
|
||||
{
|
||||
let runtime = match self.repository.reload_selected_credential(&account).await {
|
||||
|
||||
@@ -179,7 +179,8 @@ impl CodexProvider {
|
||||
provider_kind,
|
||||
requested_model.map(|model| model.as_str().to_owned()),
|
||||
account_id,
|
||||
Box::new(move |operation, middleware_headers| {
|
||||
Some(RequestParameters::baseline(lease.installation_id())),
|
||||
Box::new(move |operation, middleware_headers, parameters| {
|
||||
Box::pin(async move {
|
||||
provider
|
||||
.execute_selected_raw_json_endpoint(
|
||||
@@ -187,8 +188,11 @@ impl CodexProvider {
|
||||
operation,
|
||||
middleware_headers,
|
||||
request,
|
||||
lease,
|
||||
account_selection_wait_ms,
|
||||
SelectedRequest {
|
||||
parameters: RequestParameters::decode(parameters)?,
|
||||
lease,
|
||||
account_selection_wait_ms,
|
||||
},
|
||||
)
|
||||
.await
|
||||
})
|
||||
@@ -203,9 +207,13 @@ impl CodexProvider {
|
||||
operation: Operation,
|
||||
middleware_headers: Vec<MiddlewareHeader>,
|
||||
mut request: RawJsonEndpointRequest,
|
||||
mut lease: CodexCredentialLease,
|
||||
account_selection_wait_ms: u64,
|
||||
selected: SelectedRequest,
|
||||
) -> Result<ProviderStream, ProviderError> {
|
||||
let SelectedRequest {
|
||||
parameters,
|
||||
mut lease,
|
||||
account_selection_wait_ms,
|
||||
} = selected;
|
||||
let affinity = match &operation {
|
||||
Operation::GenerateImage(image) => {
|
||||
let model = endpoint_requested_model(image.payload());
|
||||
@@ -309,6 +317,11 @@ impl CodexProvider {
|
||||
let allows_account_state_mutation = lease.allows_account_state_mutation();
|
||||
let provider_kind = ProviderKind::new(PROVIDER_NAME)
|
||||
.map_err(|_| provider_error(ProviderErrorKind::Protocol, UpstreamSendState::NotSent))?;
|
||||
let outbound_middleware = context.outbound_middleware(
|
||||
provider_kind.clone(),
|
||||
endpoint_requested_model(payload).map(|model| model.as_str().to_owned()),
|
||||
lease.account_id().clone(),
|
||||
);
|
||||
let metadata = ProviderCallMetadata::for_provider_endpoint(
|
||||
provider_kind,
|
||||
lease.account_id().clone(),
|
||||
@@ -324,7 +337,7 @@ impl CodexProvider {
|
||||
lease.capacity_snapshot(),
|
||||
));
|
||||
// Standalone Provider 端点没有可证明的账号 owner;Images/Search metadata 必须按
|
||||
// 跨账号输入收敛到当前 lease,不能沿用下游声明的账号或 installation identity
|
||||
// 跨账号输入收敛到本次构造参数,不能沿用下游声明的账号身份
|
||||
let turn_metadata = payload
|
||||
.context()
|
||||
.get("turn_metadata")
|
||||
@@ -332,17 +345,19 @@ impl CodexProvider {
|
||||
.and_then(|metadata| {
|
||||
crate::transport::request::scope_turn_metadata(
|
||||
metadata,
|
||||
lease.installation_id(),
|
||||
¶meters.installation_id,
|
||||
true,
|
||||
)
|
||||
});
|
||||
let events = cold_json_response_stream(ColdJsonResponse {
|
||||
parameters,
|
||||
client: self
|
||||
.client_for_request(&context)?
|
||||
.for_account(lease.account())
|
||||
.map_err(|error| map_client_error(error, UpstreamSendState::NotSent, false).error)?
|
||||
.with_responses_api_base_url(lease.authentication().responses_api_base_url())
|
||||
.with_middleware_headers(middleware_headers)
|
||||
.with_outbound_middleware(outbound_middleware)
|
||||
.with_privacy(context.privacy(), context.cancellation().clone()),
|
||||
response_origin: request.response_origin,
|
||||
endpoint_path: request.endpoint_path,
|
||||
@@ -377,6 +392,7 @@ struct RawJsonEndpointRequest {
|
||||
}
|
||||
|
||||
pub(super) struct ColdResponse {
|
||||
pub(super) parameters: RequestParameters,
|
||||
pub(super) client: CodexBackendClient,
|
||||
pub(super) response_origin: Url,
|
||||
pub(super) request: CodexResponsesRequest,
|
||||
@@ -396,6 +412,7 @@ pub(super) struct ColdResponse {
|
||||
}
|
||||
|
||||
pub(super) struct ColdJsonResponse {
|
||||
pub(super) parameters: RequestParameters,
|
||||
pub(super) client: CodexBackendClient,
|
||||
pub(super) response_origin: Url,
|
||||
pub(super) endpoint_path: &'static str,
|
||||
@@ -640,7 +657,7 @@ pub(super) fn cold_json_response_stream(request: ColdJsonResponse) -> EventStrea
|
||||
let response = create_json_attempt(
|
||||
&request,
|
||||
&active_account,
|
||||
request.lease.installation_id(),
|
||||
&request.parameters.installation_id,
|
||||
&authorization,
|
||||
cookie_header.as_ref(),
|
||||
account_selection,
|
||||
@@ -691,7 +708,7 @@ pub(super) fn cold_json_response_stream(request: ColdJsonResponse) -> EventStrea
|
||||
.selector
|
||||
.capture_response_cookies(
|
||||
&active_account,
|
||||
&request.response_origin,
|
||||
response.diagnostics.response_origin.as_ref().unwrap_or(&request.response_origin),
|
||||
&response.set_cookie_headers,
|
||||
)
|
||||
.await
|
||||
@@ -759,6 +776,7 @@ fn image_response_metering(
|
||||
|
||||
pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
|
||||
let ColdResponse {
|
||||
parameters,
|
||||
client,
|
||||
response_origin,
|
||||
request,
|
||||
@@ -815,7 +833,7 @@ pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
|
||||
&request,
|
||||
&request_id,
|
||||
&active_account,
|
||||
lease.installation_id(),
|
||||
¶meters.installation_id,
|
||||
&authorization,
|
||||
cookie_header.as_ref(),
|
||||
account_selection,
|
||||
@@ -918,7 +936,7 @@ pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
|
||||
match selector
|
||||
.capture_response_cookies(
|
||||
&active_account,
|
||||
&response_origin,
|
||||
response.diagnostics.response_origin.as_ref().unwrap_or(&response_origin),
|
||||
&response.set_cookie_headers,
|
||||
)
|
||||
.await
|
||||
@@ -1152,26 +1170,42 @@ pub(super) fn cold_response_stream(response: ColdResponse) -> EventStream {
|
||||
session_transport_recovery.websocket_succeeded(key);
|
||||
}
|
||||
if allows_account_state_mutation && (completed || terminal_failure.is_some()) {
|
||||
let started = Instant::now();
|
||||
synchronize_passive_quota(
|
||||
"a,
|
||||
&active_account,
|
||||
passive_quota_observation.rate_limits(),
|
||||
)
|
||||
.await;
|
||||
trace.record("provider.quota_sync", json!({
|
||||
"durationMs": started.elapsed().as_millis(),
|
||||
}));
|
||||
}
|
||||
if let Some((failure, _)) = terminal_failure.as_ref() {
|
||||
let started = Instant::now();
|
||||
apply_failure(&failure_context, &active_account, failure)
|
||||
.await;
|
||||
trace.record("provider.account_failure", json!({
|
||||
"durationMs": started.elapsed().as_millis(),
|
||||
}));
|
||||
}
|
||||
attach_openai_session_update(&mut events, &mut session_capture);
|
||||
if allows_account_state_mutation && completed && terminal_failure.is_none() {
|
||||
// 完成事实交给下游前更新账号健康,绑定已经在发送前完成
|
||||
let started = Instant::now();
|
||||
selector
|
||||
.record_success(&active_account)
|
||||
.await;
|
||||
trace.record("provider.account_success", json!({
|
||||
"durationMs": started.elapsed().as_millis(),
|
||||
}));
|
||||
let started = Instant::now();
|
||||
selector
|
||||
.observe_cyber_policy_success(cyber_policy_scope.as_ref())
|
||||
.await;
|
||||
trace.record("provider.cyber_policy_success", json!({
|
||||
"durationMs": started.elapsed().as_millis(),
|
||||
}));
|
||||
}
|
||||
if (rate_limits_changed
|
||||
|| response_model_changed
|
||||
|
||||
@@ -48,6 +48,7 @@ pub(super) struct MappedProviderFailure {
|
||||
pub(super) cyber_policy_failure: bool,
|
||||
pub(super) upstream_capacity_failure: bool,
|
||||
pub(super) set_cookie_headers: Vec<String>,
|
||||
pub(super) response_origin: Option<Url>,
|
||||
pub(super) rate_limit_headers: Vec<(String, String)>,
|
||||
pub(super) observation: Option<ProviderResponseObservation>,
|
||||
pub(super) capture_response_cookies: bool,
|
||||
@@ -63,6 +64,7 @@ impl MappedProviderFailure {
|
||||
cyber_policy_failure: false,
|
||||
upstream_capacity_failure: false,
|
||||
set_cookie_headers: Vec::new(),
|
||||
response_origin: None,
|
||||
rate_limit_headers: Vec::new(),
|
||||
observation: None,
|
||||
capture_response_cookies: false,
|
||||
@@ -345,7 +347,10 @@ pub(super) async fn apply_failure(
|
||||
.selector
|
||||
.capture_response_cookies(
|
||||
account,
|
||||
context.response_origin,
|
||||
failure
|
||||
.response_origin
|
||||
.as_ref()
|
||||
.unwrap_or(context.response_origin),
|
||||
&failure.set_cookie_headers,
|
||||
)
|
||||
.await
|
||||
@@ -738,6 +743,9 @@ pub(super) fn map_client_error(
|
||||
uncertain_state: UpstreamSendState,
|
||||
observe_transport: bool,
|
||||
) -> MappedProviderFailure {
|
||||
if let CodexClientError::Outbound(error) = error {
|
||||
return MappedProviderFailure::plain(*error);
|
||||
}
|
||||
let local_connection_capacity = crate::transport::connection::is_admission_failure(&error);
|
||||
let diagnostic = if local_connection_capacity {
|
||||
Some(
|
||||
@@ -789,13 +797,15 @@ pub(super) fn map_client_error(
|
||||
let connect_retry = !local_connection_capacity
|
||||
&& matches!(&error, CodexClientError::Http(error) if transient_http_connect(error));
|
||||
let mut failure = match &error {
|
||||
CodexClientError::Privacy(_) => MappedProviderFailure::plain(
|
||||
provider_error(
|
||||
ProviderErrorKind::InvalidRequest,
|
||||
UpstreamSendState::NotSent,
|
||||
CodexClientError::Outbound(_) | CodexClientError::Privacy(_) => {
|
||||
MappedProviderFailure::plain(
|
||||
provider_error(
|
||||
ProviderErrorKind::InvalidRequest,
|
||||
UpstreamSendState::NotSent,
|
||||
)
|
||||
.with_retry_prohibited(),
|
||||
)
|
||||
.with_retry_prohibited(),
|
||||
),
|
||||
}
|
||||
CodexClientError::ConnectionBudgetExhausted => MappedProviderFailure::plain(
|
||||
provider_error(ProviderErrorKind::Timeout, UpstreamSendState::NotSent)
|
||||
.with_connection_retry(gateway_core::engine::AttemptTransport::Fallback),
|
||||
@@ -1021,6 +1031,7 @@ fn client_diagnostic(error: &CodexClientError) -> Option<ProviderDiagnostic> {
|
||||
"invalid_sse",
|
||||
"OpenAI HTTP returned an invalid Responses event stream".to_owned(),
|
||||
),
|
||||
CodexClientError::Outbound(_) => return None,
|
||||
CodexClientError::Privacy(error) => (
|
||||
"prepare",
|
||||
"privacy_policy_rejected",
|
||||
@@ -1468,6 +1479,7 @@ pub(super) fn map_upstream_failure(
|
||||
// 普通 5xx 与未知流内错误不足以证明容量拒绝,不能用于冻结账号
|
||||
upstream_capacity_failure: capacity_unavailable,
|
||||
set_cookie_headers: failure.set_cookie_headers,
|
||||
response_origin: failure.response_origin,
|
||||
rate_limit_headers: failure.rate_limit_headers,
|
||||
observation,
|
||||
capture_response_cookies: !matches!(
|
||||
|
||||
@@ -280,7 +280,8 @@ impl CodexProvider {
|
||||
provider_kind,
|
||||
Some(upstream_model.as_str().to_owned()),
|
||||
account_id,
|
||||
Box::new(move |operation, middleware_headers| {
|
||||
Some(RequestParameters::baseline(lease.installation_id())),
|
||||
Box::new(move |operation, middleware_headers, parameters| {
|
||||
Box::pin(async move {
|
||||
provider
|
||||
.execute_selected_live_call(
|
||||
@@ -288,8 +289,11 @@ impl CodexProvider {
|
||||
operation,
|
||||
upstream_model,
|
||||
middleware_headers,
|
||||
lease,
|
||||
account_selection_wait_ms,
|
||||
SelectedRequest {
|
||||
parameters: RequestParameters::decode(parameters)?,
|
||||
lease,
|
||||
account_selection_wait_ms,
|
||||
},
|
||||
)
|
||||
.await
|
||||
})
|
||||
@@ -304,9 +308,13 @@ impl CodexProvider {
|
||||
operation: Operation,
|
||||
upstream_model: UpstreamModelId,
|
||||
middleware_headers: Vec<MiddlewareHeader>,
|
||||
mut lease: CodexCredentialLease,
|
||||
account_selection_wait_ms: u64,
|
||||
selected: SelectedRequest,
|
||||
) -> Result<ProviderStream, ProviderError> {
|
||||
let SelectedRequest {
|
||||
parameters,
|
||||
mut lease,
|
||||
account_selection_wait_ms,
|
||||
} = selected;
|
||||
let Operation::ProviderHttp(request) = operation else {
|
||||
return Err(provider_error(
|
||||
ProviderErrorKind::InvalidRequest,
|
||||
@@ -333,6 +341,11 @@ impl CodexProvider {
|
||||
let allows_account_state_mutation = lease.allows_account_state_mutation();
|
||||
let provider_kind = ProviderKind::new(PROVIDER_NAME)
|
||||
.map_err(|_| provider_error(ProviderErrorKind::Protocol, UpstreamSendState::NotSent))?;
|
||||
let outbound_middleware = context.outbound_middleware(
|
||||
provider_kind.clone(),
|
||||
Some(upstream_model.as_str().to_owned()),
|
||||
lease.account_id().clone(),
|
||||
);
|
||||
let metadata = ProviderCallMetadata::new(
|
||||
provider_kind,
|
||||
upstream_model.clone(),
|
||||
@@ -361,12 +374,14 @@ impl CodexProvider {
|
||||
.cloned()
|
||||
.collect::<Vec<_>>();
|
||||
let events = cold_live_call_stream(ColdLiveCall {
|
||||
parameters,
|
||||
client: self
|
||||
.client_for_request(&context)?
|
||||
.for_account(lease.account())
|
||||
.map_err(|error| map_client_error(error, UpstreamSendState::NotSent, false).error)?
|
||||
.with_responses_api_base_url(lease.authentication().responses_api_base_url())
|
||||
.with_middleware_headers(middleware_headers),
|
||||
.with_middleware_headers(middleware_headers)
|
||||
.with_outbound_middleware(outbound_middleware),
|
||||
registry: Arc::clone(&self.live_registry),
|
||||
response_origin: self.live_calls_url.clone(),
|
||||
endpoint_query: request.query().map(str::to_owned),
|
||||
@@ -392,6 +407,7 @@ impl CodexProvider {
|
||||
}
|
||||
|
||||
struct ColdLiveCall {
|
||||
parameters: RequestParameters,
|
||||
client: CodexBackendClient,
|
||||
registry: Arc<CodexLiveRegistry>,
|
||||
response_origin: Url,
|
||||
@@ -435,7 +451,7 @@ fn cold_live_call_stream(request: ColdLiveCall) -> EventStream {
|
||||
authorization.expose_secret(),
|
||||
active_account.upstream_account_id(),
|
||||
&request_id,
|
||||
Some(request.lease.installation_id()),
|
||||
Some(&request.parameters.installation_id),
|
||||
);
|
||||
request_context.trace = Some(&trace);
|
||||
request_context.account_selection = CodexAccountSelectionTelemetry::new(
|
||||
|
||||
@@ -16,6 +16,7 @@ use gateway_core::engine::provider::{
|
||||
ContinuationRequestObservation, EventStream, Provider, ProviderCallMetadata, ProviderRequest,
|
||||
ProviderRequestObservation, ProviderSelectionObservation, ProviderStream,
|
||||
};
|
||||
use gateway_core::engine::upstream_adapter::UpstreamAdapterSelection;
|
||||
use gateway_core::engine::{AttemptContext, AttemptTransport, ContinuationAttempt};
|
||||
use gateway_core::error::{
|
||||
ClientVisibleUpstreamError, ClientVisibleUpstreamResponse, ContinuationFailure,
|
||||
@@ -106,6 +107,7 @@ mod execution;
|
||||
mod failure;
|
||||
mod live;
|
||||
mod observation;
|
||||
mod parameters;
|
||||
mod upstream_adapter;
|
||||
mod workers;
|
||||
pub(crate) use live::{CodexLiveGateway, CodexLiveRegistry};
|
||||
@@ -116,6 +118,7 @@ use execution::*;
|
||||
pub use failure::openai_failure_affects_account_score;
|
||||
use failure::*;
|
||||
use observation::*;
|
||||
use parameters::RequestParameters;
|
||||
pub(crate) use workers::worker_contributions;
|
||||
|
||||
const PROVIDER_NAME: &str = "openai";
|
||||
@@ -218,9 +221,14 @@ impl PreparedGenerateRequest {
|
||||
}
|
||||
|
||||
struct SelectedGenerate {
|
||||
request: SelectedRequest,
|
||||
frozen_requirements: CapabilityRequirements,
|
||||
}
|
||||
|
||||
struct SelectedRequest {
|
||||
lease: CodexCredentialLease,
|
||||
account_selection_wait_ms: u64,
|
||||
frozen_requirements: CapabilityRequirements,
|
||||
parameters: RequestParameters,
|
||||
}
|
||||
|
||||
impl CodexProvider {
|
||||
@@ -267,6 +275,38 @@ impl CodexProvider {
|
||||
}
|
||||
}
|
||||
|
||||
async fn remember_generate_turn(
|
||||
&self,
|
||||
generate: &GenerateRequest,
|
||||
headers: &[MiddlewareHeader],
|
||||
context: &AttemptContext,
|
||||
affinity: Option<&CodexSessionAffinity>,
|
||||
) -> Result<(), ProviderError> {
|
||||
if context.is_diagnostic_required_account() {
|
||||
return Ok(());
|
||||
}
|
||||
if let Some(affinity) = affinity
|
||||
&& let Some(turn) = turn_id_with_headers(
|
||||
generate.protocol_payload().body(),
|
||||
generate.protocol_payload().context(),
|
||||
headers,
|
||||
)
|
||||
.and_then(|turn| derive_turn_alias(&turn, context.client_api_key_ref()))
|
||||
{
|
||||
self.selector
|
||||
.remember_turn(
|
||||
&turn,
|
||||
affinity,
|
||||
context
|
||||
.account_selection_policy()
|
||||
.openai_session_affinity_ttl(),
|
||||
)
|
||||
.await
|
||||
.map_err(map_selection_error)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn client_for_request(
|
||||
&self,
|
||||
context: &AttemptContext,
|
||||
@@ -608,7 +648,8 @@ impl Provider for CodexProvider {
|
||||
unreachable!("generate settings keep the operation kind")
|
||||
};
|
||||
let adapter = context.upstream_adapter(candidate.provider(), upstream_model)?;
|
||||
if adapter.is_none()
|
||||
let adapted = matches!(adapter, UpstreamAdapterSelection::Adapter(_));
|
||||
if !adapted
|
||||
&& generate
|
||||
.provider_session_state(PROVIDER_NAME)
|
||||
.is_some_and(|state| state.extension_owner().is_some())
|
||||
@@ -622,18 +663,25 @@ impl Provider for CodexProvider {
|
||||
.transpose()
|
||||
.map_err(map_request_error)?;
|
||||
// 固定调度约束:Guardian 分类先于选号,不能随原生/适配器发送路径改变
|
||||
// 复用编码器的权威 metadata 解析,但只为原生路径准备会话与传输状态
|
||||
// 复用编码器的权威 metadata 解析,但只为原生路径准备续接与传输状态
|
||||
let guardian = upstream
|
||||
.as_ref()
|
||||
.is_some_and(CodexResponsesRequest::is_guardian);
|
||||
let adapter_affinity = upstream.as_ref().filter(|_| adapted).map(|request| {
|
||||
(
|
||||
derive_codex_session_affinity(request, context.client_api_key_ref()),
|
||||
derive_codex_cyber_policy_session_key(request, context.client_api_key_ref()),
|
||||
false,
|
||||
)
|
||||
});
|
||||
let preselection = upstream
|
||||
.filter(|_| adapter.is_none())
|
||||
.filter(|_| !adapted)
|
||||
.map(|upstream| self.prepare_generate_request(generate, upstream, &context));
|
||||
if let Some(prepared) = preselection.as_ref() {
|
||||
prepared.checked_native_continuation_scope(&context)?;
|
||||
}
|
||||
let (selection_session_affinity, selection_cyber_policy_key, requires_websocket) =
|
||||
preselection.map_or((None, None, false), |prepared| {
|
||||
preselection.map_or(adapter_affinity.unwrap_or((None, None, false)), |prepared| {
|
||||
let requires_websocket =
|
||||
transport_requirement(&prepared.upstream).requires_websocket()
|
||||
|| (context.continuation_attempt() == ContinuationAttempt::Native
|
||||
@@ -680,6 +728,21 @@ impl Provider for CodexProvider {
|
||||
};
|
||||
let account_selection_wait_ms =
|
||||
u64::try_from(selection_started_at.elapsed().as_millis()).unwrap_or(u64::MAX);
|
||||
let adapter = context.upstream_adapter_for_account(
|
||||
candidate.provider(),
|
||||
upstream_model,
|
||||
lease.account_id(),
|
||||
)?;
|
||||
if adapter.is_some() {
|
||||
// 适配器自行解释业务改写,宿主轮次关联只使用本次选号已确认的会话
|
||||
self.remember_generate_turn(
|
||||
generate,
|
||||
&[],
|
||||
&context,
|
||||
selection_session_affinity.as_ref(),
|
||||
)
|
||||
.await?;
|
||||
}
|
||||
let provider_kind = ProviderKind::new(PROVIDER_NAME)
|
||||
.map_err(|_| provider_error(ProviderErrorKind::Protocol, UpstreamSendState::NotSent))?;
|
||||
let account_id = lease.account_id().clone();
|
||||
@@ -693,7 +756,10 @@ impl Provider for CodexProvider {
|
||||
provider_kind,
|
||||
Some(upstream_model.as_str().to_owned()),
|
||||
account_id,
|
||||
Box::new(move |operation, middleware_headers| {
|
||||
adapter
|
||||
.is_none()
|
||||
.then(|| RequestParameters::baseline(lease.installation_id())),
|
||||
Box::new(move |operation, middleware_headers, parameters| {
|
||||
Box::pin(async move {
|
||||
if let Some(adapter) = adapter {
|
||||
return provider.execute_upstream_adapter(
|
||||
@@ -714,8 +780,11 @@ impl Provider for CodexProvider {
|
||||
terminal_model,
|
||||
terminal_context,
|
||||
SelectedGenerate {
|
||||
lease,
|
||||
account_selection_wait_ms,
|
||||
request: SelectedRequest {
|
||||
parameters: RequestParameters::decode(parameters)?,
|
||||
lease,
|
||||
account_selection_wait_ms,
|
||||
},
|
||||
frozen_requirements,
|
||||
},
|
||||
)
|
||||
@@ -737,8 +806,12 @@ impl CodexProvider {
|
||||
selected: SelectedGenerate,
|
||||
) -> Result<ProviderStream, ProviderError> {
|
||||
let SelectedGenerate {
|
||||
mut lease,
|
||||
account_selection_wait_ms,
|
||||
request:
|
||||
SelectedRequest {
|
||||
parameters,
|
||||
mut lease,
|
||||
account_selection_wait_ms,
|
||||
},
|
||||
frozen_requirements,
|
||||
} = selected;
|
||||
let Operation::Generate(generate) = operation else {
|
||||
@@ -781,39 +854,24 @@ impl CodexProvider {
|
||||
derive_codex_transport_key(&upstream_request, context.client_api_key_ref());
|
||||
let previous_session = processed.previous_session;
|
||||
let continuation_requested = processed.continuation_requested;
|
||||
let session_affinity = processed.session_affinity;
|
||||
let cyber_policy_session_key = processed.cyber_policy_session_key;
|
||||
if !context.is_diagnostic_required_account() {
|
||||
self.selector
|
||||
.validate_translated_selection(
|
||||
&mut lease,
|
||||
session_affinity.as_ref(),
|
||||
cyber_policy_session_key.as_ref(),
|
||||
processed.session_affinity.as_ref(),
|
||||
processed.cyber_policy_session_key.as_ref(),
|
||||
context.account_selection_policy(),
|
||||
)
|
||||
.await
|
||||
.map_err(map_selection_error)?;
|
||||
}
|
||||
if !context.is_diagnostic_required_account()
|
||||
&& let Some(affinity) = session_affinity.as_ref()
|
||||
&& let Some(turn) = turn_id_with_headers(
|
||||
generate.protocol_payload().body(),
|
||||
generate.protocol_payload().context(),
|
||||
&middleware_headers,
|
||||
)
|
||||
.and_then(|turn| derive_turn_alias(&turn, context.client_api_key_ref()))
|
||||
{
|
||||
self.selector
|
||||
.remember_turn(
|
||||
&turn,
|
||||
affinity,
|
||||
context
|
||||
.account_selection_policy()
|
||||
.openai_session_affinity_ttl(),
|
||||
)
|
||||
.await
|
||||
.map_err(map_selection_error)?;
|
||||
}
|
||||
self.remember_generate_turn(
|
||||
&generate,
|
||||
&middleware_headers,
|
||||
&context,
|
||||
processed.session_affinity.as_ref(),
|
||||
)
|
||||
.await?;
|
||||
let lease = Arc::new(lease);
|
||||
if previous_session.as_ref().is_some_and(|state| {
|
||||
state
|
||||
@@ -893,7 +951,7 @@ impl CodexProvider {
|
||||
.then(|| crate::transport::diagnostics::evidence::request_state(&upstream_request));
|
||||
scope_request_to_account(
|
||||
&mut upstream_request,
|
||||
lease.installation_id(),
|
||||
¶meters.installation_id,
|
||||
account_scope,
|
||||
);
|
||||
if let Some(before) = state_before {
|
||||
@@ -1004,6 +1062,11 @@ impl CodexProvider {
|
||||
));
|
||||
}
|
||||
apply_transport(&mut upstream_request, transport);
|
||||
let outbound_middleware = context.outbound_middleware(
|
||||
provider_kind.clone(),
|
||||
Some(upstream_model.as_str().to_owned()),
|
||||
lease.account_id().clone(),
|
||||
);
|
||||
let metadata = ProviderCallMetadata::new(
|
||||
provider_kind,
|
||||
upstream_model.clone(),
|
||||
@@ -1043,6 +1106,7 @@ impl CodexProvider {
|
||||
AttemptTransport::Default | AttemptTransport::Fallback => 0,
|
||||
};
|
||||
let events = cold_response_stream(ColdResponse {
|
||||
parameters,
|
||||
client: self
|
||||
.client_for_request(&context)?
|
||||
.for_account(lease.account())
|
||||
@@ -1051,6 +1115,7 @@ impl CodexProvider {
|
||||
.with_connection_budget(context.connection_budget().clone())
|
||||
.with_response_control(context.response_control().cloned())
|
||||
.with_middleware_headers(middleware_headers)
|
||||
.with_outbound_middleware(outbound_middleware)
|
||||
.with_privacy(context.privacy(), context.cancellation().clone()),
|
||||
response_origin: self.responses_url.clone(),
|
||||
request: upstream_request,
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
//! OpenAI 原生请求的构造参数;只解释本次值,不维护插件身份策略
|
||||
|
||||
use gateway_core::{
|
||||
error::{ProviderError, ProviderErrorKind},
|
||||
upstream::UpstreamSendState,
|
||||
};
|
||||
use serde::Deserialize;
|
||||
use serde_json::{Map, Value};
|
||||
|
||||
#[derive(Deserialize)]
|
||||
pub(super) struct RequestParameters {
|
||||
pub(super) installation_id: String,
|
||||
}
|
||||
|
||||
impl RequestParameters {
|
||||
pub(super) fn baseline(installation_id: &str) -> Map<String, Value> {
|
||||
Map::from_iter([(
|
||||
"installation_id".into(),
|
||||
Value::String(installation_id.to_owned()),
|
||||
)])
|
||||
}
|
||||
|
||||
pub(super) fn decode(parameters: Option<Map<String, Value>>) -> Result<Self, ProviderError> {
|
||||
let invalid = || {
|
||||
ProviderError::new(
|
||||
ProviderErrorKind::InvalidRequest,
|
||||
UpstreamSendState::NotSent,
|
||||
)
|
||||
.with_retry_prohibited()
|
||||
};
|
||||
// 参数基线已在选号后提供,显式删除和类型错误不能回退为账号默认身份
|
||||
serde_json::from_value(Value::Object(parameters.ok_or_else(invalid)?))
|
||||
.map_err(|_| invalid())
|
||||
}
|
||||
}
|
||||
@@ -901,8 +901,7 @@ impl CodexCanonicalDecoder {
|
||||
output: &mut Vec<GatewayEvent>,
|
||||
) -> Result<(), ProviderError> {
|
||||
self.require_started()?;
|
||||
let output_index = event_index(value, "output_index")?;
|
||||
let part_index = optional_event_index(value, "content_index")?.unwrap_or_default();
|
||||
let index = event_content_index(value)?;
|
||||
let part = value
|
||||
.get("part")
|
||||
.or_else(|| value.get("summary_part"))
|
||||
@@ -912,7 +911,8 @@ impl CodexCanonicalDecoder {
|
||||
Some("summary_text" | "reasoning_text") => ContentKind::Reasoning,
|
||||
_ => return Ok(()),
|
||||
};
|
||||
self.add_content(content_index(output_index, part_index)?, kind, output)
|
||||
// reasoning 项已声明首段,后续 part 复用同一内容;类型冲突仍由共享登记逻辑拒绝
|
||||
self.ensure_content(index, kind, output)
|
||||
}
|
||||
|
||||
fn text_delta(
|
||||
|
||||
@@ -195,6 +195,8 @@ impl fmt::Debug for CodexClientVisibleUpstreamResponse {
|
||||
/// Codex 上游 HTTP 客户端错误
|
||||
#[derive(Error)]
|
||||
pub enum CodexClientError {
|
||||
#[error("outbound request preparation failed")]
|
||||
Outbound(Box<gateway_core::error::ProviderError>),
|
||||
#[error("privacy policy rejected the request: {0}")]
|
||||
Privacy(#[from] gateway_core::settings::privacy::PrivacyError),
|
||||
#[error("connection recovery budget exhausted")]
|
||||
@@ -224,7 +226,6 @@ pub enum CodexClientError {
|
||||
/// 请求头值无效
|
||||
#[error("invalid request header value: {0}")]
|
||||
InvalidHeaderValue(#[from] reqwest::header::InvalidHeaderValue),
|
||||
/// 中间件业务头试图覆盖 Provider 已构造的受管头
|
||||
/// SSE 响应解析失败
|
||||
#[error("invalid upstream SSE response: {0}")]
|
||||
InvalidSse(#[from] SseError),
|
||||
@@ -278,6 +279,7 @@ pub enum CodexClientError {
|
||||
impl fmt::Debug for CodexClientError {
|
||||
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
match self {
|
||||
Self::Outbound(_) => formatter.write_str("CodexClientError::Outbound([REDACTED])"),
|
||||
Self::ConnectionBudgetExhausted => {
|
||||
formatter.write_str("CodexClientError::ConnectionBudgetExhausted")
|
||||
}
|
||||
@@ -348,7 +350,8 @@ impl CodexClientError {
|
||||
Self::Upstream { transport, .. } | Self::ErrorBodyRead { transport, .. } => {
|
||||
Some(*transport)
|
||||
}
|
||||
Self::Privacy(_)
|
||||
Self::Outbound(_)
|
||||
| Self::Privacy(_)
|
||||
| Self::ConnectionBudgetExhausted
|
||||
| Self::CustomCa(_)
|
||||
| Self::InvalidHeaderName(_)
|
||||
@@ -700,6 +703,7 @@ pub struct CodexBackendClient {
|
||||
pub(super) outbound_proxy: Option<gateway_core::account::OutboundProxy>,
|
||||
pub(super) egress_key: String,
|
||||
pub(super) middleware_headers: Vec<MiddlewareHeader>,
|
||||
pub(super) outbound_middleware: Option<gateway_core::engine::middleware::outbound::Pipeline>,
|
||||
}
|
||||
|
||||
impl CodexBackendClient {
|
||||
@@ -961,7 +965,8 @@ pub(super) async fn http_json_upstream_error(
|
||||
) -> CodexClientError {
|
||||
let status = response.status();
|
||||
let headers = response.headers();
|
||||
let diagnostics = super::response_meta::diagnostics(Some(status.as_u16()), headers);
|
||||
let diagnostics = super::response_meta::diagnostics(Some(status.as_u16()), headers)
|
||||
.with_response_origin(Some(response.url().clone()));
|
||||
let set_cookie_headers = super::response_meta::set_cookie_headers(headers);
|
||||
let rate_limit_headers = super::response_meta::rate_limit_headers(headers);
|
||||
let retry_after_seconds = retry_after_seconds(headers, None);
|
||||
|
||||
@@ -76,23 +76,28 @@ impl CodexBackendClient {
|
||||
.cloned()
|
||||
.unwrap_or_default()
|
||||
.exchange("http_json");
|
||||
let outbound = self
|
||||
.prepare_http_request(endpoint_url(&self.base_url, endpoint_path), headers, body)
|
||||
.await?;
|
||||
trace.headers(
|
||||
"upstream.request.headers",
|
||||
serde_json::json!({
|
||||
"method": "POST", "endpoint": endpoint_path,
|
||||
}),
|
||||
headers
|
||||
serde_json::json!({"method": outbound.method().as_str(), "transport": "http_json"}),
|
||||
outbound
|
||||
.headers()
|
||||
.iter()
|
||||
.map(|(name, value)| (name.as_str(), value.as_bytes())),
|
||||
);
|
||||
trace.capture("upstream.request.body", &body);
|
||||
trace.capture(
|
||||
"upstream.request.body",
|
||||
outbound
|
||||
.body()
|
||||
.and_then(reqwest::Body::as_bytes)
|
||||
.unwrap_or_default(),
|
||||
);
|
||||
let headers_started_at = Instant::now();
|
||||
let response = self
|
||||
.client
|
||||
.post(endpoint_url(&self.base_url, endpoint_path))
|
||||
.headers(headers)
|
||||
.body(body)
|
||||
.send()
|
||||
.execute(outbound)
|
||||
.await
|
||||
.map_err(CodexClientError::HttpJson)?;
|
||||
let upstream_headers_ms = elapsed_duration_millis(headers_started_at.elapsed());
|
||||
@@ -124,7 +129,8 @@ impl CodexBackendClient {
|
||||
.await);
|
||||
}
|
||||
|
||||
let diagnostics = response_meta::diagnostics(Some(status.as_u16()), response.headers());
|
||||
let diagnostics = response_meta::diagnostics(Some(status.as_u16()), response.headers())
|
||||
.with_response_origin(Some(response.url().clone()));
|
||||
let set_cookie_headers = response_meta::set_cookie_headers(response.headers());
|
||||
let rate_limit_headers = response_meta::rate_limit_headers(response.headers());
|
||||
let response_metadata = response_meta::response_metadata(response.headers());
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user