Compare commits

...
27 Commits
Author SHA1 Message Date
ZyyandCodex 47d30474af chore(release): v3.34.0
Co-authored-by: Codex <noreply@openai.com>
2026-10-27 16:48:35 +08:00
ZyyandCodex f1d5b73255 fix(ui): refine admin filters, quota colors and batch actions
Simplify group filters and keep date and platform controls aligned. Apply quota warning thresholds independently to each window and move account batch actions into a compact menu.

Co-authored-by: Codex <noreply@openai.com>
2026-10-27 16:20:15 +08:00
ZyyandCodex 18c75132c3 fix(openai): add built-in GPT-6.1 Terra pricing
Include published standard, flex, priority, and long-context prices in the built-in catalog and classify Terra as a reasoning model for preview web search billing.

Co-authored-by: Codex <noreply@openai.com>
2026-10-27 16:20:15 +08:00
ZyyandCodex a48139b4c2 fix(store): support literal substring search in admin lists
Match account names and emails, client key names and labels, and observability search fields by literal substrings. Preserve wildcard escaping and existing case rules.

Co-authored-by: Codex <noreply@openai.com>
2026-10-27 16:20:15 +08:00
蓝芷怡andCodex dd529d07ec fix(store): support pg_dump staging files on Windows (#475)
Co-authored-by: Codex <noreply@openai.com>
2026-10-27 15:13:13 +08:00
19fbb5da75 feat(plugin): support account-scoped adapted model catalogs (#533)
* feat(plugin): publish account-scoped adapted model catalogs

* fix(core): preserve candidate account scope during execution

Apply the narrowed candidate scope before provider account selection, including affinity and account rotation. Cover adapted-only models and same-name models with incompatible tool capabilities through the real OpenAI selector and local HTTP fixtures.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-27 13:37:57 +08:00
ZyyandCodex 9296885cce fix(tests): align middleware fixture with startup compatibility
Co-authored-by: Codex <noreply@openai.com>
2026-10-27 13:17:38 +08:00
Ke WangandCodex af9790ac37 fix(plugin): prove replay safety for managed HTTP 429 rejections (#532)
Seal managed callback observations before Core retry decisions and synchronize RPC resource revocation with stream teardown.

Co-authored-by: Codex <noreply@openai.com>
2026-10-27 13:12:42 +08:00
Ke Wang c58e280739 fix(deps): update zstd safety wrappers (#534)
Preserve Zstd 1.5.7 and compression level 3 while updating the Rust wrappers to resolve security advisories.
2026-10-27 12:53:00 +08:00
ZyyandCodex e33f50ec09 fix(plugins): gate incompatible plugin startup
Keep incompatible plugins out of execution sets during automatic startup.
Require scoped confirmation for manual compatibility overrides and show
the stopped state and upgrade effects in the management UI.

Co-authored-by: Codex <noreply@openai.com>
2026-10-27 12:37:28 +08:00
ZyyandCodex 1db8b0e319 chore(deps): update plugins to v0.8.0
Co-authored-by: Codex <noreply@openai.com>
2026-10-26 19:19:56 +08:00
ZyyandCodex 8c8618aa0f chore(release): v3.33.0
Co-authored-by: Codex <noreply@openai.com>
2026-10-26 18:44:00 +08:00
c43a8872db fix(frontend): highlight selected models missing upstream (#528)
* fix(frontend): highlight unavailable selected models

* fix(frontend): refine model catalog warnings and refresh control

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-26 18:39:08 +08:00
ZyyandCodex 51c5caed0e fix(deploy): disable account routing override in example config
Co-authored-by: Codex <noreply@openai.com>
2026-10-26 18:00:53 +08:00
a7f95c3a78 fix(openai): preserve Retry-After when converting WS errors to SSE (#531)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-26 17:08:57 +08:00
ZyyandCodex 421e37896a fix(tests): avoid RPC admission callback deadlock
Drive the ordinary callback call to completion before consuming its notification so delayed suspension cannot strand an unpolled admission future. Bound next callback startup and resumption waits.

Co-authored-by: Codex <noreply@openai.com>
2026-10-26 17:06:45 +08:00
1d1ccda42a refactor(plugins): consolidate host lifecycle and diagnostics (#529)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-26 17:06:44 +08:00
e38cf31323 refactor(plugins): separate client identity from model execution ports (#527)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-26 13:21:31 +08:00
614c81c375 refactor(plugins)!: separate identity queries and cooperative scheduling (#525)
Resolve identity and frozen access facts without constructing model execution
contexts. Coordinate parent task suspension, resumption and retained callbacks
at the SDK and Runtime boundary, with bounded admission and cancellation.
Expose account HTTP, client verification, access facts and quota details through
the existing host owners.

BREAKING CHANGE: Plugin sessions require RPC v7; rebuild and repackage plugins
with the matching SDK. No legacy protocol negotiation or fallback is provided.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-26 13:05:08 +08:00
f0484c4e11 chore(frontend): upgrade ESLint config to v10 and migrate formatting (#522)
* chore(deps-dev): bump @antfu/eslint-config in /frontend

Bumps [@antfu/eslint-config](https://github.com/antfu/eslint-config) from 9.5.1 to 10.0.0.
- [Release notes](https://github.com/antfu/eslint-config/releases)
- [Commits](https://github.com/antfu/eslint-config/compare/v9.5.1...v10.0.0)

---
updated-dependencies:
- dependency-name: "@antfu/eslint-config"
  dependency-version: 10.0.0
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(frontend): migrate formatting to ESLint v10

Co-authored-by: Codex <noreply@openai.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-26 11:34:32 +08:00
dependabot[bot] dffc417b9f chore(deps): bump rust from e12566d to 48b7f43 in /deploy (#524)
Bumps rust from `e12566d` to `48b7f43`.

---
updated-dependencies:
- dependency-name: rust
  dependency-version: 1.99.1-bookworm
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-26 11:04:28 +08:00
dependabot[bot] 2d4294c3c9 chore(deps): bump debian from 7c7b2c9 to 7ff6ba3 in /deploy (#523)
Bumps debian from `7c7b2c9` to `7ff6ba3`.

---
updated-dependencies:
- dependency-name: debian
  dependency-version: bookworm-slim
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-26 11:04:23 +08:00
dependabot[bot] 3d9906facf chore(deps): bump the minor-and-patch group in /frontend with 5 updates (#520)
Bumps the minor-and-patch group in /frontend with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [@lucide/vue](https://github.com/lucide-icons/lucide/tree/HEAD/packages/vue) | `1.59.0` | `1.61.0` |
| [es-toolkit](https://github.com/toss/es-toolkit) | `1.52.0` | `1.53.0` |
| [eslint-plugin-vuejs-accessibility](https://github.com/vue-a11y/eslint-plugin-vuejs-accessibility) | `2.6.2` | `2.6.3` |
| [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) | `8.3.4` | `8.3.5` |
| [vue-tsc](https://github.com/vuejs/language-tools/tree/HEAD/packages/tsc) | `3.3.11` | `3.3.13` |


Updates `@lucide/vue` from 1.59.0 to 1.61.0
- [Release notes](https://github.com/lucide-icons/lucide/releases)
- [Commits](https://github.com/lucide-icons/lucide/commits/1.61.0/packages/vue)

Updates `es-toolkit` from 1.52.0 to 1.53.0
- [Release notes](https://github.com/toss/es-toolkit/releases)
- [Changelog](https://github.com/toss/es-toolkit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/toss/es-toolkit/compare/v1.52.0...v1.53.0)

Updates `eslint-plugin-vuejs-accessibility` from 2.6.2 to 2.6.3
- [Release notes](https://github.com/vue-a11y/eslint-plugin-vuejs-accessibility/releases)
- [Changelog](https://github.com/vue-a11y/eslint-plugin-vuejs-accessibility/blob/main/CHANGELOG.md)
- [Commits](https://github.com/vue-a11y/eslint-plugin-vuejs-accessibility/compare/v2.6.2...v2.6.3)

Updates `vite` from 8.3.4 to 8.3.5
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.3.5/packages/vite)

Updates `vue-tsc` from 3.3.11 to 3.3.13
- [Release notes](https://github.com/vuejs/language-tools/releases)
- [Changelog](https://github.com/vuejs/language-tools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/vuejs/language-tools/commits/v3.3.13/packages/tsc)

---
updated-dependencies:
- dependency-name: "@lucide/vue"
  dependency-version: 1.61.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: es-toolkit
  dependency-version: 1.53.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: eslint-plugin-vuejs-accessibility
  dependency-version: 2.6.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: vite
  dependency-version: 8.3.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: vue-tsc
  dependency-version: 3.3.13
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-26 11:04:18 +08:00
dependabot[bot] da9c2b92b0 chore(deps): bump jsonschema from 0.58.6 to 0.60.0 in /backend (#519)
Bumps [jsonschema](https://github.com/Stranger6667/jsonschema) from 0.58.6 to 0.60.0.
- [Release notes](https://github.com/Stranger6667/jsonschema/releases)
- [Changelog](https://github.com/Stranger6667/jsonschema/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stranger6667/jsonschema/compare/ruby-v0.58.6...ruby-v0.60.0)

---
updated-dependencies:
- dependency-name: jsonschema
  dependency-version: 0.60.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-26 11:02:43 +08:00
dependabot[bot] de5c10c4e4 chore(deps): bump scraper from 0.27.0 to 0.29.0 in /backend (#518)
Bumps [scraper](https://github.com/rust-scraper/scraper) from 0.27.0 to 0.29.0.
- [Release notes](https://github.com/rust-scraper/scraper/releases)
- [Commits](https://github.com/rust-scraper/scraper/compare/v0.27.0...v0.29.0)

---
updated-dependencies:
- dependency-name: scraper
  dependency-version: 0.29.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-26 11:02:33 +08:00
dependabot[bot] 939cb99985 chore(deps): bump the minor-and-patch group in /backend with 7 updates (#517)
Bumps the minor-and-patch group in /backend with 7 updates:

| Package | From | To |
| --- | --- | --- |
| [zip](https://github.com/zip-rs/zip2) | `9.0.4` | `9.0.5` |
| [async-compression](https://github.com/Nullus157/async-compression) | `0.4.50` | `0.4.51` |
| [aws-credential-types](https://github.com/smithy-lang/smithy-rs) | `1.3.0` | `1.3.1` |
| [aws-sdk-s3](https://github.com/awslabs/aws-sdk-rust) | `1.152.0` | `1.155.0` |
| [aws-smithy-runtime-api](https://github.com/smithy-lang/smithy-rs) | `1.19.0` | `1.19.1` |
| [aws-smithy-types](https://github.com/smithy-lang/smithy-rs) | `1.8.1` | `1.9.0` |
| [crc32fast](https://github.com/srijs/rust-crc32fast) | `1.5.2` | `1.5.3` |


Updates `zip` from 9.0.4 to 9.0.5
- [Release notes](https://github.com/zip-rs/zip2/releases)
- [Changelog](https://github.com/zip-rs/zip2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/zip-rs/zip2/compare/v9.0.4...v9.0.5)

Updates `async-compression` from 0.4.50 to 0.4.51
- [Release notes](https://github.com/Nullus157/async-compression/releases)
- [Commits](https://github.com/Nullus157/async-compression/compare/async-compression-v0.4.50...async-compression-v0.4.51)

Updates `aws-credential-types` from 1.3.0 to 1.3.1
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)

Updates `aws-sdk-s3` from 1.152.0 to 1.155.0
- [Release notes](https://github.com/awslabs/aws-sdk-rust/releases)
- [Commits](https://github.com/awslabs/aws-sdk-rust/commits)

Updates `aws-smithy-runtime-api` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)

Updates `aws-smithy-types` from 1.8.1 to 1.9.0
- [Release notes](https://github.com/smithy-lang/smithy-rs/releases)
- [Changelog](https://github.com/smithy-lang/smithy-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/smithy-lang/smithy-rs/commits)

Updates `crc32fast` from 1.5.2 to 1.5.3
- [Commits](https://github.com/srijs/rust-crc32fast/compare/v1.5.2...v1.5.3)

---
updated-dependencies:
- dependency-name: zip
  dependency-version: 9.0.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: async-compression
  dependency-version: 0.4.51
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: aws-credential-types
  dependency-version: 1.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: aws-sdk-s3
  dependency-version: 1.155.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: aws-smithy-runtime-api
  dependency-version: 1.19.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: aws-smithy-types
  dependency-version: 1.9.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
- dependency-name: crc32fast
  dependency-version: 1.5.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-26 11:02:21 +08:00
dependabot[bot] 2bb8ebc01c chore(deps): bump the minor-and-patch group with 3 updates (#516)
Bumps the minor-and-patch group with 3 updates: [Swatinem/rust-cache](https://github.com/swatinem/rust-cache), [taiki-e/install-action](https://github.com/taiki-e/install-action) and [docker/login-action](https://github.com/docker/login-action).


Updates `Swatinem/rust-cache` from 2.9.2 to 2.9.3
- [Release notes](https://github.com/swatinem/rust-cache/releases)
- [Changelog](https://github.com/Swatinem/rust-cache/blob/master/CHANGELOG.md)
- [Commits](https://github.com/swatinem/rust-cache/compare/6323deb102c322ba6fcbdcafc7e3dddab59af2b6...d1527c8f67ef9af357290475d4af4b0fab8f12e0)

Updates `taiki-e/install-action` from 2.87.29 to 2.87.31
- [Release notes](https://github.com/taiki-e/install-action/releases)
- [Changelog](https://github.com/taiki-e/install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/install-action/compare/8cca0918f8471c33515202b73f56ab27f2bed283...1e29709c0ec63babbe8f266397a465f29a825f91)

Updates `docker/login-action` from 4.6.0 to 4.7.0
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/dbcb813823bdd20940b903addbd779551569679f...81c6f93d4bc991e510b342a00b419b3bb4efc5c6)

---
updated-dependencies:
- dependency-name: Swatinem/rust-cache
  dependency-version: 2.9.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: taiki-e/install-action
  dependency-version: 2.87.31
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: minor-and-patch
- dependency-name: docker/login-action
  dependency-version: 4.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-26 11:01:07 +08:00
414 changed files with 16708 additions and 5673 deletions
+2 -2
View File
@@ -108,7 +108,7 @@ jobs:
components: rustfmt,clippy
- name: Cache Rust
uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2
uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3
with:
workspaces: backend -> target
save-if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
@@ -122,7 +122,7 @@ jobs:
run: cargo clippy --all-targets --all-features --locked -- -D warnings
- name: Setup nextest
uses: taiki-e/install-action@8cca0918f8471c33515202b73f56ab27f2bed283 # v2.87.29
uses: taiki-e/install-action@1e29709c0ec63babbe8f266397a465f29a825f91 # v2.87.31
with:
tool: cargo-nextest@0.9.143
fallback: none
+2 -2
View File
@@ -312,7 +312,7 @@ jobs:
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- name: Login to GHCR
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
@@ -409,7 +409,7 @@ jobs:
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
- name: Login to GHCR
uses: docker/login-action@dbcb813823bdd20940b903addbd779551569679f # v4.6.0
uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
+1 -1
View File
@@ -28,7 +28,7 @@ jobs:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Install cargo-audit
uses: taiki-e/install-action@8cca0918f8471c33515202b73f56ab27f2bed283 # v2.87.29
uses: taiki-e/install-action@1e29709c0ec63babbe8f266397a465f29a825f91 # v2.87.31
with:
tool: cargo-audit@0.22.2
fallback: none
+96 -106
View File
@@ -147,9 +147,9 @@ dependencies = [
[[package]]
name = "async-compression"
version = "0.4.50"
version = "0.4.51"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ee19bd99b43e3691acbad4e840420a4881cea6c0b66a208125a824f8fd53f5a1"
checksum = "9d5755640120b24961f8231e05892e4a4f7e0d04e8aeca161508c9c397946970"
dependencies = [
"compression-codecs",
"compression-core",
@@ -224,9 +224,9 @@ checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
[[package]]
name = "aws-credential-types"
version = "1.3.0"
version = "1.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e93964ffdaf57857f544be3666a5f57570bb699e934700f11b49708f61bb556e"
checksum = "576e8da35751dc85409ec1cd15999f692c277a1fbe9336780fe6f15e49dd637d"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
@@ -260,9 +260,9 @@ dependencies = [
[[package]]
name = "aws-runtime"
version = "1.10.0"
version = "1.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2b8a9911551b4ea6ca13805ef52ed96f7d2bbb43cc3b4a14cb0776a71f33cfaa"
checksum = "9a0cc2144962cc5e358467b16e3d82981b1d424a7c3d34602be0036103814a1a"
dependencies = [
"aws-credential-types",
"aws-sigv4",
@@ -286,9 +286,9 @@ dependencies = [
[[package]]
name = "aws-sdk-s3"
version = "1.152.0"
version = "1.155.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "004d5acbd6850ab58789a43822e0a2ef23927a46b012c17f8fa51253103db9f8"
checksum = "5a542f0e9261b9ce56708d591a62062079fa3c58a362ffbf49a2e48a4f6ced16"
dependencies = [
"arc-swap",
"aws-credential-types",
@@ -322,9 +322,9 @@ dependencies = [
[[package]]
name = "aws-sigv4"
version = "1.6.0"
version = "1.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2312577f088c9fbf4206dfdb884cf1de9407b43e1a923cbed5237775116fc24b"
checksum = "9d3d48355cb86b15ccbb38b8bc6b87500c824ee4ed8cbc10a961b435242b5e16"
dependencies = [
"aws-credential-types",
"aws-smithy-eventstream",
@@ -344,9 +344,9 @@ dependencies = [
[[package]]
name = "aws-smithy-async"
version = "1.3.0"
version = "1.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f02e407fb3b54891734224b9ffac8a71fdd35f542500fa1af95754a6b2beb316"
checksum = "de681a9df7ff79823b759ee17061076ec049cd61a8fa28ece4293d89c9f6964e"
dependencies = [
"futures-util",
"pin-project-lite",
@@ -376,9 +376,9 @@ dependencies = [
[[package]]
name = "aws-smithy-eventstream"
version = "0.61.4"
version = "0.61.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "80c2051c2f1016fb8e6548dd07b8bc2ac9c3fe583721444b92f515e856d31609"
checksum = "14c8a643d0e114cae39e45415e3225793a6396698b7779a0c9e3ede731d8fc71"
dependencies = [
"aws-smithy-types",
"bytes",
@@ -387,9 +387,9 @@ dependencies = [
[[package]]
name = "aws-smithy-http"
version = "0.64.1"
version = "0.64.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "639b4d8f8555f24a9be649811c3eb0b4d4616f4d61daf0c32e28873bc1ea9af1"
checksum = "ba8960956e0ee0ca3f3161e62913a61bc65a7ef638ca3ef1d4c7159f8443e398"
dependencies = [
"aws-smithy-eventstream",
"aws-smithy-runtime-api",
@@ -403,15 +403,14 @@ dependencies = [
"http-body-util",
"percent-encoding",
"pin-project-lite",
"pin-utils",
"tracing",
]
[[package]]
name = "aws-smithy-http-client"
version = "1.5.0"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "51c89cc3f1f281d659a67a519a1b5c6d445b5ce09fa7e5aee40c2c2707e9509d"
checksum = "aeca69f65362f7375891008ea525fd440ad6db1836458ac188c87b126804114e"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api",
@@ -433,9 +432,9 @@ dependencies = [
[[package]]
name = "aws-smithy-json"
version = "0.63.1"
version = "0.65.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3385d469edbe8b60cc72002784652b5efca39178192aa9cc4b44c9875c6bdc18"
checksum = "9f6835fb38bc6fc8c91e584b0366eb4ff595e5c485134dbf4f0bdeaa18856666"
dependencies = [
"aws-smithy-runtime-api",
"aws-smithy-schema",
@@ -453,9 +452,9 @@ dependencies = [
[[package]]
name = "aws-smithy-runtime"
version = "1.16.0"
version = "1.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6e302ac1d88b99652489df31abdec6ac42a2ab2ac3982ad0ac49f64dfaf28ba"
checksum = "79cef0c41e8ca40f74f68ed8ee4d1ddbde12bb847b95ef9b3c025681ddfae58a"
dependencies = [
"aws-smithy-async",
"aws-smithy-http",
@@ -470,16 +469,15 @@ dependencies = [
"http-body",
"http-body-util",
"pin-project-lite",
"pin-utils",
"tokio",
"tracing",
]
[[package]]
name = "aws-smithy-runtime-api"
version = "1.19.0"
version = "1.19.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c0730c16f91124c6a2abb4932c77e299288b3dd9f967ea2e9ec48cc6731e87a4"
checksum = "06788004dea79018413f381891da6dcca23196acd79c4d51b767fbdacc76eae4"
dependencies = [
"aws-smithy-async",
"aws-smithy-runtime-api-macros",
@@ -505,9 +503,9 @@ dependencies = [
[[package]]
name = "aws-smithy-schema"
version = "0.2.1"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8f395d93304280b64b7632fea798d177e74897fe7f063416ce627cd6fa24829"
checksum = "bec8dae1015a6d959a5faf86fbdf77486cfe19e4ccb6ee009d8a46efc45028a4"
dependencies = [
"aws-smithy-runtime-api",
"aws-smithy-types",
@@ -516,9 +514,9 @@ dependencies = [
[[package]]
name = "aws-smithy-types"
version = "1.8.1"
version = "1.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "69bb407740a197147da48238ecc94498493c9e85445732360cec180296ca45f1"
checksum = "55752025c39c0e094962fd73f1c57014b191f678612e517f219468e575af9822"
dependencies = [
"base64-simd",
"bytes",
@@ -530,7 +528,6 @@ dependencies = [
"itoa",
"num-integer",
"pin-project-lite",
"pin-utils",
"ryu",
"serde",
"time",
@@ -540,9 +537,9 @@ dependencies = [
[[package]]
name = "aws-smithy-xml"
version = "0.62.1"
version = "0.64.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0b932c8d6dc127fc980eecd78f8694ae9b9551b69a93a7def2a199c1c0033daf"
checksum = "a6581618a2fd006159016d1c36881d94d61b4a017f4a8d631310875e18252413"
dependencies = [
"aws-smithy-runtime-api",
"aws-smithy-schema",
@@ -552,9 +549,9 @@ dependencies = [
[[package]]
name = "aws-types"
version = "1.6.0"
version = "1.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "209f3a6d82a6e9e5f94abbed94c7a26e1c052341002bf57a5fb5481f625896fc"
checksum = "6ba0d970d3947bb064d2ae1d823efa186c1dbde88f88afa8f6549946e5d8deee"
dependencies = [
"aws-credential-types",
"aws-smithy-async",
@@ -945,9 +942,9 @@ dependencies = [
[[package]]
name = "compression-codecs"
version = "0.4.45"
version = "0.4.46"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "98fc98460ba0ad5317075d3632b8dfc45d0be8c4a49347c2a38272019717614a"
checksum = "cae36683101b2899f48bd31f8db9839437356188e3b1ff23096e73dab09ba8fe"
dependencies = [
"compression-core",
"flate2",
@@ -956,9 +953,9 @@ dependencies = [
[[package]]
name = "compression-core"
version = "0.4.33"
version = "0.4.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e8ccc4ea9f6acc32d102c0f6d471d11d913ad15f20c04de743374861fa1d414"
checksum = "2ebdda89edc534bf8d5dd2857562633b6495557e5846552ade994b99ed13c36a"
[[package]]
name = "config"
@@ -1084,9 +1081,9 @@ dependencies = [
[[package]]
name = "crc32fast"
version = "1.5.2"
version = "1.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "01a7799fd6b852db0e61728dde9a204c423b44d689dbd432522543614b490e78"
checksum = "0ba106e844045ddc02c20768ac4b0b7cb45f2cc2ecbd833ac9f5dacb183c2055"
dependencies = [
"cfg-if",
]
@@ -1166,14 +1163,14 @@ dependencies = [
[[package]]
name = "cssparser"
version = "0.37.0"
version = "0.38.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8c9cdaae01d5ed7882b04d795e7f752f46ff52d2fa3b50a20d28c464510bba98"
checksum = "11119743ad110e8c1bdccd930d7f5c30c99e5fc76a7b63ec9807e84eef0c5f59"
dependencies = [
"cssparser-macros",
"dtoa-short",
"itoa",
"phf 0.13.1",
"phf 0.14.0",
"smallvec",
]
@@ -2125,12 +2122,13 @@ dependencies = [
[[package]]
name = "html5ever"
version = "0.39.0"
version = "0.40.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "46a1761807faccc9a19e86944bbf40610014066306f96edcdedc2fb714bcb7b8"
checksum = "456a1a377e608e555d22ddab27ac0114bc7a7b4199078108e34c2aeae6c9b130"
dependencies = [
"log",
"markup5ever",
"memchr",
]
[[package]]
@@ -2483,9 +2481,9 @@ dependencies = [
[[package]]
name = "jsonschema"
version = "0.58.6"
version = "0.60.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f622c574a29fb3034a0a90645afee194dfedc75fc124228801b6e77041075bd1"
checksum = "7e7305938cac5ee144352ceb71583ffc6cfeac7d20fcb20ef865a4d5c314e499"
dependencies = [
"ahash",
"bytecount",
@@ -2511,23 +2509,24 @@ dependencies = [
[[package]]
name = "jsonschema-regex"
version = "0.58.6"
version = "0.60.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a968fffd387b2ab005bf00c1851f5d26e90df05b6f7ad609ed0d620033cca5e7"
checksum = "6f8a848331614144c03e815fb931f238c9e342bd7ab501b29e35d647e7cd5688"
dependencies = [
"regex-syntax",
]
[[package]]
name = "jsonschema-value"
version = "0.58.6"
version = "0.60.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39b7670345e5ce87371339d13705dbb9276b7e3e934bbe1efa2c6734a1f65a96"
checksum = "5a5a947ef798eeacacb43edf29b1d0ab64bdf5e2c9fbcaad10ee8663d922b544"
dependencies = [
"ahash",
"bytecount",
"fraction",
"getrandom 0.3.4",
"memchr",
"num-cmp",
"num-traits",
"serde_json",
@@ -2642,9 +2641,9 @@ dependencies = [
[[package]]
name = "markup5ever"
version = "0.39.0"
version = "0.40.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7122d987ec5f704ee56f6e5b41a7d93722e9aae27ae07cafa4036c4d3f9757de"
checksum = "0ab3dc68ac4a0f5719e560136778c1ee716e296030d75dbd4484e37e39e3a842"
dependencies = [
"log",
"tendril",
@@ -3101,23 +3100,23 @@ dependencies = [
[[package]]
name = "phf"
version = "0.13.1"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c1562dc717473dbaa4c1f85a36410e03c047b2e7df7f45ee938fbef64ae7fadf"
checksum = "010378780309880b08997fae13be7834dba947d36393bd372f2b1556deb2a2f6"
dependencies = [
"phf_macros 0.13.1",
"phf_shared 0.13.1",
"phf_macros 0.14.0",
"phf_shared 0.14.0",
"serde",
]
[[package]]
name = "phf_codegen"
version = "0.13.1"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "49aa7f9d80421bca176ca8dbfebe668cc7a2684708594ec9f3c0db0805d5d6e1"
checksum = "41b585a510fb76fdebead6897982ef2a03a21d8e6cbcca904999742a4afc6ffe"
dependencies = [
"phf_generator 0.13.1",
"phf_shared 0.13.1",
"phf_generator 0.14.0",
"phf_shared 0.14.0",
]
[[package]]
@@ -3132,12 +3131,12 @@ dependencies = [
[[package]]
name = "phf_generator"
version = "0.13.1"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "135ace3a761e564ec88c03a77317a7c6b80bb7f7135ef2544dbe054243b89737"
checksum = "aeb62e0959d5a1bebc965f4d15d9e2b7cea002b6b0f5ba8cde6cc26738467100"
dependencies = [
"fastrand",
"phf_shared 0.13.1",
"phf_shared 0.14.0",
]
[[package]]
@@ -3155,12 +3154,12 @@ dependencies = [
[[package]]
name = "phf_macros"
version = "0.13.1"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "812f032b54b1e759ccd5f8b6677695d5268c588701effba24601f6932f8269ef"
checksum = "5fa8d0ca26d424d27630da600c6624696e7dec8bf7b3b492b383c5dc49e5e085"
dependencies = [
"phf_generator 0.13.1",
"phf_shared 0.13.1",
"phf_generator 0.14.0",
"phf_shared 0.14.0",
"proc-macro2",
"quote",
"syn 2.0.119",
@@ -3186,9 +3185,9 @@ dependencies = [
[[package]]
name = "phf_shared"
version = "0.13.1"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e57fef6bc5981e38c2ce2d63bfa546861309f875b8a75f092d1d54ae2d64f266"
checksum = "c6fd9027e2d9319be6349febd1db4e8d02aa544921200c9b777720ac34a3aa89"
dependencies = [
"siphasher",
]
@@ -3199,12 +3198,6 @@ version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "pin-utils"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184"
[[package]]
name = "pkcs8"
version = "0.11.0"
@@ -3636,9 +3629,9 @@ dependencies = [
[[package]]
name = "referencing"
version = "0.58.6"
version = "0.60.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "558f3855529c0a7e78a48e409fa653456e55f1b167eab4109c16a255b7727e3f"
checksum = "922268812b6f9e20b4ab0f1efe8e07036958eed7346ceddb31c777b5531fd11f"
dependencies = [
"ahash",
"fluent-uri",
@@ -3875,9 +3868,9 @@ checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"
[[package]]
name = "scraper"
version = "0.27.0"
version = "0.29.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bdd0be4d296f048bfb06dd01bbc80ef789ddd2e55583e8d2e6b804942abfabc2"
checksum = "6844b5f49a5579f12ba4f89d9f7eec6d71889e4aa0bc7181821040ce66f08fff"
dependencies = [
"cssparser",
"ego-tree",
@@ -3885,7 +3878,6 @@ dependencies = [
"html5ever",
"precomputed-hash",
"selectors",
"tendril",
]
[[package]]
@@ -3950,17 +3942,15 @@ dependencies = [
[[package]]
name = "selectors"
version = "0.38.0"
version = "0.41.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8adfa1c298912827b8a28b223b3b874357397ae706e6190acd9bf28cee99114d"
checksum = "8322aebea94a6cc288ff884e8a49bdbceef60206c400c6c46d22e62fc4fd91a6"
dependencies = [
"bitflags",
"cssparser",
"derive_more",
"log",
"new_debug_unreachable",
"phf 0.13.1",
"phf_codegen",
"precomputed-hash",
"rustc-hash",
"servo_arc",
@@ -4067,9 +4057,9 @@ dependencies = [
[[package]]
name = "servo_arc"
version = "0.4.3"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "170fb83ab34de17dc69aa7c67482b22218ddb85da56546f9bd6b929e32a05930"
checksum = "b4eceb519ec34d8290aaf6e46f435102f2bfa63f22b05a144fb16ab266d3b198"
dependencies = [
"stable_deref_trait",
]
@@ -4401,24 +4391,24 @@ checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596"
[[package]]
name = "string_cache"
version = "0.9.0"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a18596f8c785a729f2819c0f6a7eae6ebeebdfffbfe4214ae6b087f690e31901"
checksum = "ffa8a5dbe8b3f0bbe29d4c3225daafaeead63afdc1b65fc4c01a1384166038e6"
dependencies = [
"new_debug_unreachable",
"parking_lot",
"phf_shared 0.13.1",
"phf_shared 0.14.0",
"precomputed-hash",
]
[[package]]
name = "string_cache_codegen"
version = "0.6.1"
version = "0.11.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "585635e46db231059f76c5849798146164652513eb9e8ab2685939dd90f29b69"
checksum = "928dcdf75e47626b3617a976ec205d9f057584c371c1f23b782129268d0e6edc"
dependencies = [
"phf_generator 0.13.1",
"phf_shared 0.13.1",
"phf_generator 0.14.0",
"phf_shared 0.14.0",
"proc-macro2",
"quote",
]
@@ -5214,11 +5204,11 @@ dependencies = [
[[package]]
name = "web_atoms"
version = "0.2.6"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba8b815c1b593dc0baf78dd0f4fc8fdb2de53198fb1163738093e9a311c33fb3"
checksum = "7572660c8890448ba236b7376f27e389c6a7e1c70195622faced601f855c0ada"
dependencies = [
"phf 0.13.1",
"phf 0.14.0",
"phf_codegen",
"string_cache",
"string_cache_codegen",
@@ -5651,9 +5641,9 @@ dependencies = [
[[package]]
name = "zip"
version = "9.0.4"
version = "9.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a954114b47a54df49f83d2722814e86b23d4bb1b54a1b90fb7d92b41619ddc22"
checksum = "6e04976da629d3b04204ba8105b396a193af8ad2583b2c79bfb3ab8af580a11e"
dependencies = [
"crc32fast",
"flate2",
@@ -5677,27 +5667,27 @@ checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"
[[package]]
name = "zstd"
version = "0.13.3"
version = "0.14.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e91ee311a569c327171651566e07972200e76fcfe2242a4fa446149a3881c08a"
checksum = "db3ae9bf90af6e40197bef767eeefc46ad6397420fe873830bd85e5fbbfb3c70"
dependencies = [
"zstd-safe",
]
[[package]]
name = "zstd-safe"
version = "7.2.4"
version = "8.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f49c4d5f0abb602a93fb8736af2a4f4dd9512e36f7f570d66e65ff867ed3b9d"
checksum = "ded520f16b29e1a9b771c722221b2b0a0398c2316d9bb3c42b458a5414d7434b"
dependencies = [
"zstd-sys",
]
[[package]]
name = "zstd-sys"
version = "2.0.16+zstd.1.5.7"
version = "2.1.2+zstd.1.5.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "91e19ebc2adc8f83e43039e79776e3fda8ca919132d68a1fed6a5faca2683748"
checksum = "481ada9c543b2323cbb7bbdbb5249177fbf0188ed959ed2c15e4b22421c84ed3"
dependencies = [
"cc",
"pkg-config",
@@ -103,6 +103,10 @@ async fn execute(
let management = ManagementPorts::new(&store, providers.admin.clone(), core.snapshot_control());
or_shutdown!(plugin_runtime, management.bind(&plugin_runtime));
let core = or_shutdown!(plugin_runtime, core.activate().await);
or_shutdown!(
plugin_runtime,
plugin_runtime.bind_client_identity_port(&core.client_identity_port())
);
let nested_models = core.nested_model_execution_port();
let affinity_lookup = core.affinity_lookup_port();
or_shutdown!(
@@ -38,6 +38,10 @@ pub async fn run() -> Result<(), BootstrapError> {
let management = ManagementPorts::new(&store, providers.admin.clone(), core.snapshot_control());
or_shutdown!(plugin_runtime, management.bind(&plugin_runtime));
let mut core = or_shutdown!(plugin_runtime, core.activate().await);
or_shutdown!(
plugin_runtime,
plugin_runtime.bind_client_identity_port(&core.client_identity_port())
);
let nested_models = core.nested_model_execution_port();
let affinity_lookup = core.affinity_lookup_port();
or_shutdown!(
@@ -84,6 +84,7 @@ pub struct PluginInstanceMutation {
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum PluginInstanceRuntimeStatus {
Disabled,
Unknown,
AwaitingPublication,
Preparing,
Running,
@@ -108,6 +109,21 @@ pub struct PluginInstanceRuntime {
pub actual_artifact_sha256: Option<String>,
pub failure: Option<PluginInstanceRuntimeFailure>,
pub draining_revisions: Vec<u64>,
pub load: Option<PluginInstanceRuntimeLoad>,
}
/// 当前发布会话的资源占用采样,计数可以重叠且不构成原子快照
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PluginInstanceRuntimeLoad {
/// 总在途名额,包含排队、取消确认和仍由响应流持有的调用
pub active_calls: usize,
pub executing_calls: usize,
pub queued_calls: usize,
pub resuming_calls: usize,
/// 包含已返回但尚未由插件消费的回调结果
pub pending_callbacks: usize,
/// 已交付首结果但尚未收到 End 的流
pub active_streams: usize,
}
/// 可继续运行的旧接口提示,与无法启用的兼容性错误分开
@@ -136,6 +152,8 @@ pub struct PluginInstanceView {
#[derive(Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct ConfigurePluginInstance {
/// 只随本次启动请求传递,不写入实例配置
pub compatibility_confirmation: Option<PluginCompatibilityConfirmation>,
/// 创建草稿固定一个 UUID,重试复用同一配置,不能作为编辑已有配置的入口
pub creation_id: Option<String>,
/// 编辑时检查用户看到的配置版本,避免覆盖另一窗口已经保存的内容
@@ -159,6 +177,16 @@ pub struct PluginInstanceReplacement {
pub expected_revision: u64,
}
/// 管理员确认的宿主、实例配置与目标包;不构成永久兼容豁免
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
pub struct PluginCompatibilityConfirmation {
pub host_fingerprint: String,
pub instance_id: String,
pub instance_revision: u64,
pub artifact_sha256: String,
}
/// 恢复已安装旧版本的配置快照,并重新通过目标版本校验
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
@@ -196,6 +224,9 @@ pub struct PluginVersionConfiguration {
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
pub struct PluginVersionPlan {
pub compatibility_warning: Option<String>,
pub compatibility_confirmation: Option<PluginCompatibilityConfirmation>,
pub load_error: Option<String>,
pub instance_revision: u64,
pub artifact_sha256: String,
pub configuration: serde_json::Value,
@@ -62,6 +62,48 @@ impl PluginHostCompatibility {
.iter()
.any(|entry| entry.capability == capability && entry.versions.contains(&version))
}
/// 启动和升级预检共用比较规则,只检查插件声明的能力
#[must_use]
pub fn incompatibilities(
&self,
requirements: &PluginCompatibilityRequirements,
host: &semver::Version,
) -> Vec<String> {
let mut reasons = Vec::new();
if !self
.manifest_schema_versions
.contains(&requirements.manifest_schema_version)
{
reasons.push(format!(
"清单 v{} 不在宿主支持范围内",
requirements.manifest_schema_version
));
}
if !self
.protocol_versions
.contains(&requirements.protocol_version)
{
reasons.push(format!(
"RPC v{} 不在宿主支持范围内",
requirements.protocol_version
));
}
if !semver::VersionReq::parse(&requirements.host_version)
.is_ok_and(|range| range.matches(host))
{
reasons.push(format!(
"插件声明的宿主范围为 {},当前为 {host}",
requirements.host_version
));
}
for (capability, version) in &requirements.capabilities {
if !self.supports_capability(capability, *version) {
reasons.push(format!("{capability} v{version} 不在宿主支持范围内"));
}
}
reasons
}
}
fn valid_versions(versions: &[u32]) -> bool {
@@ -46,6 +46,15 @@ pub trait PluginPreparation: PluginRuntimeDiagnostics + PluginStateLifecycle {
&self,
snapshot: PluginInstanceSnapshot,
) -> Result<ExtensionSetReference, AdminError>;
/// 确认由 Admin 校验配置版本,Runtime 复核宿主并仅放行目标实例
async fn prepare_confirmed(
&self,
_snapshot: PluginInstanceSnapshot,
_confirmation: crate::model::plugins::instances::PluginCompatibilityConfirmation,
) -> Result<ExtensionSetReference, AdminError> {
Err(AdminError::invalid("插件运行时不支持确认启动"))
}
}
/// 运行诊断独立于发布准备;没有进程事实时显式返回 None
@@ -175,6 +184,10 @@ pub trait PluginDistribution: Send + Sync {
/// Runtime 只解释插件包格式;安装事务与来源选择归 Admin
#[async_trait]
pub trait PluginPackageInspector: Send + Sync {
/// 当前宿主版本、兼容声明与平台的稳定身份,不含敏感信息
fn compatibility_host_fingerprint(&self) -> Result<String, AdminError> {
Err(AdminError::internal("宿主兼容声明不可用"))
}
/// 完整校验包并返回版本范围诊断;校验失败与可加载但未经宿主承诺的提醒分开
async fn compatibility_warning(
&self,
@@ -161,53 +161,24 @@ impl PluginSystemUpdatePreflight {
.await,
) {
(_, Err(error)) if error.kind() == AdminErrorKind::Invalid => {
Some("插件包或协议与目标版本不兼容".to_owned())
Some(error.message().to_owned())
}
(_, Err(error)) => return Err(map_inspection_error(error)),
(None, _) => Some("无法确认插件与目标版本兼容".to_owned()),
(Some(host), Ok(requirements)) => {
if !semver::VersionReq::parse(&requirements.host_version)
.is_ok_and(|versions| versions.matches(target))
{
Some(format!("要求网关版本 {}", requirements.host_version))
} else if !host
.manifest_schema_versions
.contains(&requirements.manifest_schema_version)
|| !host
.protocol_versions
.contains(&requirements.protocol_version)
|| requirements
.capabilities
.iter()
.any(|(capability, version)| {
!host.supports_capability(capability, *version)
})
{
Some("插件协议或能力与目标版本不兼容".to_owned())
} else {
None
}
let reasons = host.incompatibilities(&requirements, target);
(!reasons.is_empty()).then(|| reasons.join(","))
}
}
} else {
match self
.inspector
.inspect(
artifact.archive.clone(),
Some(instance.artifact_sha256.clone()),
)
.compatibility_warning(artifact.archive, instance.artifact_sha256.clone())
.await
{
Ok(_) => self
.inspector
.compatibility_warning(
artifact.archive,
instance.artifact_sha256.clone(),
)
.await
.map_err(map_inspection_error)?,
Ok(warning) => warning,
Err(error) if error.kind() == AdminErrorKind::Invalid => {
Some("插件与当前宿主不兼容".to_owned())
Some(error.message().to_owned())
}
Err(error) => return Err(map_inspection_error(error)),
}
@@ -186,6 +186,8 @@ impl PluginsService {
.configuration_ready(probe, &metadata)
.await?;
let status = self.package_status(&metadata.sha256).await?;
let compatible = status.warning.is_none() && status.load_error.is_none();
// 其它管理员写入或同插件的并发安装可能推进全局 revision;稳定 creation ID
// 配合每轮重新读取,既不会复制默认实例,也不会覆盖先完成的另一个版本
for _ in 0..3 {
@@ -207,12 +209,13 @@ impl PluginsService {
}
let input = ConfigurePluginInstance {
compatibility_confirmation: None,
replace_instances: Vec::new(),
creation_id: Some(creation_id.clone()),
expected_revision: None,
name: metadata.display_name.clone(),
artifact_sha256: metadata.sha256.clone(),
enabled: !configuration_required,
enabled: !configuration_required && compatible,
configuration: configuration.clone(),
secrets: None,
bindings: bindings.clone(),
@@ -1,773 +0,0 @@
//! 插件实例的配置、停用、删除与私有状态迁移编排
use std::collections::{BTreeMap, BTreeSet};
use gateway_core::{
policy::ClientApiKeyId,
routing::{AccountGroupId, extensions::ExtensionSetReference},
};
use secrecy::ExposeSecret;
use super::PluginsService;
use crate::{
model::{
AdminError, AdminErrorKind, MutationContext, Revision,
plugins::instances::{
ConfigurePluginInstance, PluginInstance, PluginInstanceMutation,
PluginInstanceReplacement, PluginInstanceRuntime, PluginInstanceRuntimeStatus,
PluginInstanceSnapshot, PluginInstanceView,
},
plugins::state::{PluginStateCommit, PluginStateConfiguration},
},
use_case::{map_store_error, publish_committed},
};
#[derive(Clone, Default)]
pub(super) struct PackageStatus {
warning: Option<String>,
load_error: Option<String>,
}
impl PluginsService {
/// 只校验固定包体,不启动插件;停用实例也需区分兼容性提醒与加载错误
async fn package_status(&self, digest: &str) -> Result<PackageStatus, AdminError> {
// 运行宿主和摘要固定,缓存纯静态结论,避免管理页轮询反复解包
// 只缓存有限数量的确定结果,暂时性错误仍可重试
let mut cache = self.compatibility.lock().await;
if let Some(warning) = cache.get(digest) {
return Ok(warning.clone());
}
let artifact = self
.store
.load_artifact(digest)
.await
.map_err(|error| map_store_error(error, "plugin"))?;
let warning = match self
.inspector
.compatibility_warning(artifact.archive, digest.to_owned())
.await
{
Ok(warning) => PackageStatus {
warning,
load_error: None,
},
Err(error) if error.kind() == AdminErrorKind::Invalid => PackageStatus {
warning: None,
load_error: Some(error.message().to_owned()),
},
Err(error) => return Err(error),
};
if cache.len() < 128 {
cache.insert(digest.to_owned(), warning.clone());
}
Ok(warning)
}
pub async fn instances(&self) -> Result<Vec<PluginInstanceView>, AdminError> {
let snapshot = self
.store
.load_instances()
.await
.map_err(|e| map_store_error(e, "plugin"))?;
let published = self.published.snapshot_for_diagnostics();
let published_revision = published.as_ref().map(|snapshot| snapshot.revision().get());
let published_ready = published
.as_ref()
.is_some_and(|snapshot| snapshot.extensions().is_none_or(|set| set.can_serve()));
let mut diagnostics = self
.preparation
.runtime_diagnostics(
&snapshot,
published_revision,
published
.as_ref()
.and_then(|snapshot| snapshot.extensions()),
)
.await
.unwrap_or_default();
let target_revision = snapshot.config_revision.get();
let artifacts = self
.list()
.await?
.into_iter()
.map(|artifact| (artifact.metadata.sha256.clone(), artifact.metadata))
.collect::<BTreeMap<_, _>>();
let mut views = Vec::with_capacity(snapshot.instances.len());
for instance in snapshot.instances {
let metadata = artifacts
.get(&instance.artifact_sha256)
.ok_or_else(|| AdminError::not_found("插件制品不存在"))?;
let package_status = self.package_status(&instance.artifact_sha256).await?;
let configuration_required = package_status.load_error.is_none()
&& !self
.preparation
.configuration_ready(instance.clone(), metadata)
.await?;
let runtime = diagnostics.remove(&instance.id).unwrap_or_else(|| {
let running = instance.enabled
&& published_ready
&& published_revision == Some(target_revision);
PluginInstanceRuntime {
status: if !instance.enabled {
PluginInstanceRuntimeStatus::Disabled
} else if running {
PluginInstanceRuntimeStatus::Running
} else {
PluginInstanceRuntimeStatus::AwaitingPublication
},
actual_revision: running.then_some(instance.revision.get()),
actual_artifact_sha256: running.then(|| instance.artifact_sha256.clone()),
failure: None,
draining_revisions: Vec::new(),
}
});
views.push(PluginInstanceView {
api_deprecations: if package_status.load_error.is_none() {
self.inspector.api_deprecations(metadata)?
} else {
Vec::new()
},
configuration_required,
compatibility_warning: package_status.warning,
load_error: package_status.load_error,
running: runtime.status == PluginInstanceRuntimeStatus::Running,
published_revision,
runtime,
instance,
});
}
Ok(views)
}
pub async fn configure_instance(
&self,
id: Option<&str>,
input: ConfigurePluginInstance,
context: &MutationContext,
) -> Result<PluginInstanceMutation, AdminError> {
if id.is_some() && input.creation_id.is_some() {
return Err(AdminError::invalid("编辑配置不能使用创建标识"));
}
if id.is_none() && input.expected_revision.is_some() {
return Err(AdminError::invalid("创建配置不能指定已有配置版本"));
}
if let Some(creation_id) = &input.creation_id {
let parsed = uuid::Uuid::parse_str(creation_id)
.map_err(|_| AdminError::invalid("配置创建标识必须是 UUID"))?;
if parsed.to_string() != *creation_id {
return Err(AdminError::invalid("配置创建标识必须使用标准 UUID 格式"));
}
}
let snapshot = self
.store
.load_instances()
.await
.map_err(|e| map_store_error(e, "plugin"))?;
let existing = match id {
Some(id) => Some(
snapshot
.instances
.iter()
.find(|instance| instance.id == id)
.cloned()
.ok_or_else(|| AdminError::not_found("插件实例不存在"))?,
),
None => input.creation_id.as_ref().and_then(|creation_id| {
snapshot
.instances
.iter()
.find(|instance| instance.id == *creation_id)
.cloned()
}),
};
if let Some(existing) = &existing {
if id.is_none() && !same_creation_request(existing, &input) {
return Err(AdminError::conflict("此配置已创建,请加载最新配置后编辑"));
}
if input
.expected_revision
.is_some_and(|revision| revision != existing.revision.get())
{
return Err(AdminError::conflict("插件配置已变更,请加载最新配置后重试"));
}
}
self.configure_from_snapshot(snapshot, existing, input, context)
.await
}
pub(super) async fn configure_from_snapshot(
&self,
mut snapshot: PluginInstanceSnapshot,
existing: Option<PluginInstance>,
input: ConfigurePluginInstance,
context: &MutationContext,
) -> Result<PluginInstanceMutation, AdminError> {
let expected_revision = snapshot.config_revision;
let candidate_revision = next_revision(expected_revision)?;
let artifacts = self
.store
.list_artifacts()
.await
.map_err(|error| map_store_error(error, "plugin"))?;
let artifact = artifacts
.iter()
.find(|artifact| artifact.metadata.sha256 == input.artifact_sha256)
.ok_or_else(|| AdminError::not_found("插件制品不存在"))?;
if artifact.accepted_at.is_none() {
return Err(AdminError::invalid("请先安装并接受该版本声明的权限"));
}
if input.enabled
&& let Some(warning) = self
.package_status(&input.artifact_sha256)
.await?
.load_error
{
return Err(AdminError::invalid(format!(
"{warning},无法启动,请安装兼容版本"
)));
}
let same_plugin = |digest: &str| {
artifacts.iter().any(|item| {
item.metadata.sha256 == digest
&& item.metadata.plugin_id == artifact.metadata.plugin_id
})
};
if existing
.as_ref()
.is_some_and(|previous| !same_plugin(&previous.artifact_sha256))
{
return Err(AdminError::invalid("不能将已有配置切换为另一个插件"));
}
if input.enabled
&& snapshot.instances.iter().any(|other| {
other.enabled
&& existing
.as_ref()
.is_none_or(|current| current.id != other.id)
&& same_plugin(&other.artifact_sha256)
&& !input
.replace_instances
.iter()
.any(|replacement| replacement.id == other.id)
})
{
return Err(AdminError::conflict(
"此插件已有启用配置,请先切换或停用旧配置",
));
}
let saved = if let Some(previous) = &existing
&& previous.artifact_sha256 != input.artifact_sha256
&& input.secrets.is_none()
{
self.store
.load_version_configuration(&previous.id, &input.artifact_sha256)
.await
.map_err(|error| map_store_error(error, "plugin"))?
} else {
None
};
let instance = PluginInstance {
id: existing.as_ref().map_or_else(
|| {
input
.creation_id
.clone()
.unwrap_or_else(|| uuid::Uuid::now_v7().to_string())
},
|instance| instance.id.clone(),
),
name: input.name.trim().to_owned(),
artifact_sha256: input.artifact_sha256,
enabled: input.enabled,
trusted_process: true,
configuration: input.configuration,
secrets: input
.secrets
.or_else(|| saved.map(|saved| saved.secrets))
.unwrap_or_else(|| {
existing
.as_ref()
.map_or_else(Default::default, |instance| instance.secrets.clone())
}),
bindings: input.bindings,
revision: candidate_revision,
};
validate_input(&instance)?;
if existing
.as_ref()
.is_some_and(|previous| previous.artifact_sha256 != instance.artifact_sha256)
&& !self
.preparation
.configuration_ready(instance.clone(), &artifact.metadata)
.await?
{
return Err(AdminError::invalid(
"目标版本仍缺少必填设置,请补充后再切换",
));
}
let replacements = input.replace_instances;
if !replacements.is_empty() {
if !instance.enabled || replacements.len() > 256 {
return Err(AdminError::invalid(
"仅启用配置时可以切换其他配置,单次最多 256 项",
));
}
let mut ids = BTreeSet::new();
for replacement in &replacements {
if replacement.id == instance.id || !ids.insert(&replacement.id) {
return Err(AdminError::invalid("停用列表不能重复或包含当前配置"));
}
let previous = snapshot
.instances
.iter()
.find(|item| item.id == replacement.id)
.ok_or_else(|| AdminError::conflict("待停用配置已变更,请重新确认"))?;
if !previous.enabled || previous.revision.get() != replacement.expected_revision {
return Err(AdminError::conflict("待停用配置已变更,请重新确认"));
}
if !artifacts.iter().any(|item| {
item.metadata.sha256 == previous.artifact_sha256
&& item.metadata.plugin_id == artifact.metadata.plugin_id
}) {
return Err(AdminError::invalid("只能切换同一插件的配置"));
}
}
}
let state = self.preparation.validate(instance.clone()).await?;
if let Some(existing) = &existing
&& self.state.transition_required(&existing.id, &state).await?
{
return self
.configure_instance_with_state_transition(
snapshot,
existing.clone(),
instance,
state,
&replacements,
context,
)
.await;
}
snapshot.instances.retain(|item| item.id != instance.id);
snapshot.instances.push(instance.clone());
snapshot.config_revision = candidate_revision;
apply_replacements(&mut snapshot, &replacements);
let prepared = self.preparation.prepare(snapshot.clone()).await?;
let result = self
.store
.save_instance_replacing(
instance,
expected_revision,
PluginStateCommit {
configuration: state,
transition_id: None,
},
&replacements,
context,
)
.await
.map_err(|e| map_store_error(e, "plugin"))?;
self.publish_instance(snapshot, prepared, &result, &replacements)
.await?;
Ok(result)
}
async fn configure_instance_with_state_transition(
&self,
mut snapshot: PluginInstanceSnapshot,
previous: PluginInstance,
mut target: PluginInstance,
target_state: PluginStateConfiguration,
replacements: &[PluginInstanceReplacement],
context: &MutationContext,
) -> Result<PluginInstanceMutation, AdminError> {
let previous_state = self.preparation.validate(previous.clone()).await?;
let original_revision = snapshot.config_revision;
// 即使最终保持停用,迁移也必须由目标制品的受控候选执行
let mut migration_instance = target.clone();
migration_instance.enabled = true;
snapshot.config_revision = migration_instance.revision;
snapshot
.instances
.retain(|instance| instance.id != migration_instance.id);
snapshot.instances.push(migration_instance);
apply_replacements(&mut snapshot, replacements);
let migration_prepared = self.preparation.prepare(snapshot).await?;
let mut disabled_revision = None;
let disabled = if previous.enabled {
let mut disabled = previous.clone();
disabled.enabled = false;
let mutation = self
.store
.save_instance_with_state(
disabled,
original_revision,
PluginStateCommit {
configuration: previous_state.clone(),
transition_id: None,
},
context,
)
.await
.map_err(|error| map_store_error(error, "plugin"))?;
publish_committed(self.snapshots.as_ref(), mutation.config_revision).await?;
self.preparation
.quiesce_instance(&previous.id, &previous.artifact_sha256, previous.revision)
.await;
disabled_revision = Some(mutation.config_revision);
mutation.instance
} else {
previous.clone()
};
let expected_revision = disabled_revision.unwrap_or(original_revision);
let transition = match self
.state
.begin_transition(
&disabled.id,
disabled.revision,
&target.artifact_sha256,
target_state.clone(),
)
.await
{
Ok(transition) => transition,
Err(error) => {
return Err(self
.recover_failed_state_transition(
None,
&previous,
&previous_state,
disabled_revision,
context,
error,
)
.await);
}
};
// 迁移开始后的所有提交前失败都走同一恢复出口;提交成功后不再回滚旧配置
let committed = async {
self.preparation
.migrate_state(&migration_prepared, transition.clone())
.await?;
let mut current = self
.store
.load_instances()
.await
.map_err(|error| map_store_error(error, "plugin"))?;
if current.config_revision != expected_revision {
return Err(AdminError::conflict("迁移期间插件配置已发生变化"));
}
let candidate_revision = next_revision(expected_revision)?;
target.revision = candidate_revision;
current.instances.retain(|item| item.id != target.id);
current.instances.push(target.clone());
current.config_revision = candidate_revision;
apply_replacements(&mut current, replacements);
let prepared = self.preparation.prepare(current.clone()).await?;
let result = self
.store
.save_instance_replacing(
target,
expected_revision,
PluginStateCommit {
configuration: target_state,
transition_id: Some(transition.id.clone()),
},
replacements,
context,
)
.await
.map_err(|error| map_store_error(error, "plugin"))?;
Ok((current, prepared, result))
}
.await;
let (current, prepared, result) = match committed {
Ok(committed) => committed,
Err(error) => {
return Err(self
.recover_failed_state_transition(
Some(&transition.id),
&previous,
&previous_state,
disabled_revision,
context,
error,
)
.await);
}
};
self.publish_instance(current, prepared, &result, replacements)
.await?;
drop(migration_prepared);
Ok(result)
}
async fn recover_failed_state_transition(
&self,
transition_id: Option<&str>,
previous: &PluginInstance,
previous_state: &PluginStateConfiguration,
disabled_revision: Option<Revision>,
context: &MutationContext,
original_error: AdminError,
) -> AdminError {
if let Some(transition_id) = transition_id {
self.state.abort_transition(transition_id).await;
}
let Some(disabled_revision) = disabled_revision else {
return original_error;
};
let restored = async {
let mut snapshot = self
.store
.load_instances()
.await
.map_err(|error| map_store_error(error, "plugin"))?;
if snapshot.config_revision != disabled_revision {
return Err(AdminError::conflict(
"状态迁移失败;实例已保持停用,且配置已被其他操作修改",
));
}
let candidate_revision = next_revision(disabled_revision)?;
let mut restore = previous.clone();
restore.enabled = true;
restore.revision = candidate_revision;
snapshot.instances.retain(|item| item.id != restore.id);
snapshot.instances.push(restore.clone());
snapshot.config_revision = candidate_revision;
let prepared = self.preparation.prepare(snapshot.clone()).await?;
let mutation = self
.store
.save_instance_with_state(
restore,
disabled_revision,
PluginStateCommit {
configuration: previous_state.clone(),
transition_id: None,
},
context,
)
.await
.map_err(|error| map_store_error(error, "plugin"))?;
self.publish_instance(snapshot, prepared, &mutation, &[])
.await?;
Ok::<(), AdminError>(())
}
.await;
match restored {
Ok(()) => original_error,
Err(error) => {
tracing::warn!(
error_kind = ?error.kind(),
"plugin state migration failed and the previous instance could not be restored"
);
AdminError::conflict("状态迁移失败;为避免覆盖并发配置,插件实例已保持停用")
}
}
}
/// 以事务返回的版本激活并发布,候选强引用必须覆盖整个提交后阶段
async fn publish_instance(
&self,
mut snapshot: PluginInstanceSnapshot,
prepared: ExtensionSetReference,
mutation: &PluginInstanceMutation,
replacements: &[PluginInstanceReplacement],
) -> Result<(), AdminError> {
let prepared = if mutation.config_revision == snapshot.config_revision
&& mutation.instance.revision == snapshot.config_revision
{
prepared
} else {
snapshot.config_revision = mutation.config_revision;
snapshot
.instances
.retain(|item| item.id != mutation.instance.id);
snapshot.instances.push(mutation.instance.clone());
apply_replacements(&mut snapshot, replacements);
self.preparation.prepare(snapshot).await?
};
self.preparation
.activate_state(&prepared, &mutation.instance)
.await?;
publish_committed(self.snapshots.as_ref(), mutation.config_revision).await?;
drop(prepared);
Ok(())
}
pub async fn delete_instance(
&self,
id: &str,
context: &MutationContext,
) -> Result<Revision, AdminError> {
let snapshot = self
.store
.load_instances()
.await
.map_err(|e| map_store_error(e, "plugin"))?;
let instance = snapshot
.instances
.iter()
.find(|instance| instance.id == id)
.ok_or_else(|| AdminError::not_found("插件实例不存在"))?;
if instance.enabled {
return Err(AdminError::conflict("请先停用插件实例"));
}
let revision = self
.store
.delete_instance(id, snapshot.config_revision, context)
.await
.map_err(|e| map_store_error(e, "plugin"))?;
publish_committed(self.snapshots.as_ref(), revision).await?;
Ok(revision)
}
/// 紧急管理修复不要求损坏插件成功准备;提交后沿用现有发布与暂停合同
pub async fn disable_instance(
&self,
id: &str,
context: &MutationContext,
) -> Result<PluginInstanceMutation, AdminError> {
let PluginInstanceSnapshot {
config_revision,
instances,
} = self
.store
.load_instances()
.await
.map_err(|e| map_store_error(e, "plugin"))?;
let mut instance = instances
.into_iter()
.find(|instance| instance.id == id)
.ok_or_else(|| AdminError::not_found("插件实例不存在"))?;
instance.enabled = false;
let result = self
.store
.save_instance(instance, config_revision, context)
.await
.map_err(|e| map_store_error(e, "plugin"))?;
publish_committed(self.snapshots.as_ref(), result.config_revision).await?;
Ok(result)
}
}
fn apply_replacements(
snapshot: &mut PluginInstanceSnapshot,
replacements: &[PluginInstanceReplacement],
) {
for instance in &mut snapshot.instances {
if replacements.iter().any(|item| item.id == instance.id) {
instance.enabled = false;
instance.revision = snapshot.config_revision;
}
}
}
fn same_creation_request(existing: &PluginInstance, input: &ConfigurePluginInstance) -> bool {
let same_secrets = input
.secrets
.as_ref()
.map_or(existing.secrets.is_empty(), |secrets| {
secrets.len() == existing.secrets.len()
&& secrets.iter().all(|(key, value)| {
existing
.secrets
.get(key)
.is_some_and(|current| current.expose_secret() == value.expose_secret())
})
});
existing.name == input.name.trim()
&& existing.artifact_sha256 == input.artifact_sha256
&& existing.enabled == input.enabled
&& existing.configuration == input.configuration
&& existing.bindings == input.bindings
&& same_secrets
}
fn validate_input(instance: &PluginInstance) -> Result<(), AdminError> {
if instance.name.is_empty()
|| instance.name.len() > 128
|| instance.name.chars().any(char::is_control)
|| instance.artifact_sha256.len() != 64
|| !instance
.artifact_sha256
.bytes()
.all(|byte| byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte))
|| !instance.configuration.is_object()
|| serde_json::to_vec(&instance.configuration).map_or(true, |bytes| bytes.len() > 48 * 1024)
|| instance.secrets.len() > 64
|| instance.bindings.len() > 64
{
return Err(AdminError::invalid("插件实例配置不合法或超过大小限制"));
}
use secrecy::ExposeSecret as _;
if instance
.secrets
.iter()
.any(|(key, value)| key.len() > 64 || value.expose_secret().len() > 8192)
|| instance
.secrets
.values()
.map(|value| value.expose_secret().len())
.sum::<usize>()
> 48 * 1024
{
return Err(AdminError::invalid("插件敏感配置超过大小限制"));
}
if instance.bindings.iter().any(|binding| {
binding.contribution.is_empty()
|| binding.contribution.len() > 128
|| !binding
.contribution
.bytes()
.all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'.' | b'-'))
|| binding.client_key_ids.len() > 256
|| binding.account_group_ids.len() > 256
|| binding.client_key_ids.iter().collect::<BTreeSet<_>>().len()
!= binding.client_key_ids.len()
|| binding
.account_group_ids
.iter()
.collect::<BTreeSet<_>>()
.len()
!= binding.account_group_ids.len()
|| binding
.client_key_ids
.iter()
.any(|id| ClientApiKeyId::new(id.clone()).is_err())
|| binding
.account_group_ids
.iter()
.any(|id| AccountGroupId::new(id.clone()).is_err())
|| binding.identity_bindings.len() > 256
|| binding
.identity_bindings
.iter()
.map(|identity| &identity.principal)
.collect::<BTreeSet<_>>()
.len()
!= binding.identity_bindings.len()
|| binding.identity_bindings.iter().any(|identity| {
identity.principal.trim() != identity.principal
|| identity.principal.is_empty()
|| identity.principal.len() > 256
|| identity.principal.chars().any(char::is_control)
|| ClientApiKeyId::new(identity.client_key_id.clone()).is_err()
})
}) {
return Err(AdminError::invalid("插件绑定的范围或入口认证身份不合法"));
}
Ok(())
}
fn next_revision(revision: Revision) -> Result<Revision, AdminError> {
Revision::new(
revision
.get()
.checked_add(1)
.ok_or_else(|| AdminError::internal("配置版本已耗尽"))?,
)
.map_err(|_| AdminError::internal("配置版本不合法"))
}
@@ -0,0 +1,135 @@
//! 插件配置的输入约束、创建幂等比较与候选快照变换
use std::collections::BTreeSet;
use gateway_core::{policy::ClientApiKeyId, routing::AccountGroupId};
use secrecy::ExposeSecret;
use crate::model::{
AdminError, Revision,
plugins::instances::{
ConfigurePluginInstance, PluginInstance, PluginInstanceReplacement, PluginInstanceSnapshot,
},
};
pub(super) fn apply_replacements(
snapshot: &mut PluginInstanceSnapshot,
replacements: &[PluginInstanceReplacement],
) {
for instance in &mut snapshot.instances {
if replacements.iter().any(|item| item.id == instance.id) {
instance.enabled = false;
instance.revision = snapshot.config_revision;
}
}
}
pub(super) fn same_creation_request(
existing: &PluginInstance,
input: &ConfigurePluginInstance,
) -> bool {
let same_secrets = input
.secrets
.as_ref()
.map_or(existing.secrets.is_empty(), |secrets| {
secrets.len() == existing.secrets.len()
&& secrets.iter().all(|(key, value)| {
existing
.secrets
.get(key)
.is_some_and(|current| current.expose_secret() == value.expose_secret())
})
});
existing.name == input.name.trim()
&& existing.artifact_sha256 == input.artifact_sha256
&& existing.enabled == input.enabled
&& existing.configuration == input.configuration
&& existing.bindings == input.bindings
&& same_secrets
}
pub(super) fn validate_input(instance: &PluginInstance) -> Result<(), AdminError> {
if instance.name.is_empty()
|| instance.name.len() > 128
|| instance.name.chars().any(char::is_control)
|| instance.artifact_sha256.len() != 64
|| !instance
.artifact_sha256
.bytes()
.all(|byte| byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte))
|| !instance.configuration.is_object()
|| serde_json::to_vec(&instance.configuration).map_or(true, |bytes| bytes.len() > 48 * 1024)
|| instance.secrets.len() > 64
|| instance.bindings.len() > 64
{
return Err(AdminError::invalid("插件实例配置不合法或超过大小限制"));
}
use secrecy::ExposeSecret as _;
if instance
.secrets
.iter()
.any(|(key, value)| key.len() > 64 || value.expose_secret().len() > 8192)
|| instance
.secrets
.values()
.map(|value| value.expose_secret().len())
.sum::<usize>()
> 48 * 1024
{
return Err(AdminError::invalid("插件敏感配置超过大小限制"));
}
if instance.bindings.iter().any(|binding| {
binding.contribution.is_empty()
|| binding.contribution.len() > 128
|| !binding
.contribution
.bytes()
.all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'.' | b'-'))
|| binding.client_key_ids.len() > 256
|| binding.account_group_ids.len() > 256
|| binding.client_key_ids.iter().collect::<BTreeSet<_>>().len()
!= binding.client_key_ids.len()
|| binding
.account_group_ids
.iter()
.collect::<BTreeSet<_>>()
.len()
!= binding.account_group_ids.len()
|| binding
.client_key_ids
.iter()
.any(|id| ClientApiKeyId::new(id.clone()).is_err())
|| binding
.account_group_ids
.iter()
.any(|id| AccountGroupId::new(id.clone()).is_err())
|| binding.identity_bindings.len() > 256
|| binding
.identity_bindings
.iter()
.map(|identity| &identity.principal)
.collect::<BTreeSet<_>>()
.len()
!= binding.identity_bindings.len()
|| binding.identity_bindings.iter().any(|identity| {
identity.principal.trim() != identity.principal
|| identity.principal.is_empty()
|| identity.principal.len() > 256
|| identity.principal.chars().any(char::is_control)
|| ClientApiKeyId::new(identity.client_key_id.clone()).is_err()
})
}) {
return Err(AdminError::invalid("插件绑定的范围或入口认证身份不合法"));
}
Ok(())
}
pub(super) fn next_revision(revision: Revision) -> Result<Revision, AdminError> {
Revision::new(
revision
.get()
.checked_add(1)
.ok_or_else(|| AdminError::internal("配置版本已耗尽"))?,
)
.map_err(|_| AdminError::internal("配置版本不合法"))
}
@@ -0,0 +1,226 @@
//! 插件私有状态迁移的排空、暂存提交与失败恢复编排
use super::{
PluginsService,
configuration::{apply_replacements, next_revision},
};
use crate::{
model::{
AdminError, MutationContext, Revision,
plugins::{
instances::{
PluginCompatibilityConfirmation, PluginInstance, PluginInstanceMutation,
PluginInstanceReplacement, PluginInstanceSnapshot,
},
state::{PluginStateCommit, PluginStateConfiguration},
},
},
use_case::{map_store_error, publish_committed},
};
impl PluginsService {
pub(super) async fn configure_instance_with_state_transition(
&self,
mut snapshot: PluginInstanceSnapshot,
mut target: PluginInstance,
target_state: PluginStateConfiguration,
replacements: &[PluginInstanceReplacement],
confirmation: Option<&PluginCompatibilityConfirmation>,
context: &MutationContext,
) -> Result<PluginInstanceMutation, AdminError> {
let previous = snapshot
.instances
.iter()
.find(|instance| instance.id == target.id)
.cloned()
.ok_or_else(|| AdminError::not_found("插件实例不存在"))?;
let previous_state = self.preparation.validate(previous.clone()).await?;
let original_revision = snapshot.config_revision;
// 即使最终保持停用,迁移也必须由目标制品的受控候选执行
let mut migration_instance = target.clone();
migration_instance.enabled = true;
snapshot.config_revision = migration_instance.revision;
snapshot
.instances
.retain(|instance| instance.id != migration_instance.id);
snapshot.instances.push(migration_instance);
apply_replacements(&mut snapshot, replacements);
let migration_prepared = self
.prepare_instance_candidate(snapshot, confirmation)
.await?;
let mut disabled_revision = None;
let disabled = if previous.enabled {
let mut disabled = previous.clone();
disabled.enabled = false;
let mutation = self
.store
.save_instance_with_state(
disabled,
original_revision,
PluginStateCommit {
configuration: previous_state.clone(),
transition_id: None,
},
context,
)
.await
.map_err(|error| map_store_error(error, "plugin"))?;
publish_committed(self.snapshots.as_ref(), mutation.config_revision).await?;
self.preparation
.quiesce_instance(&previous.id, &previous.artifact_sha256, previous.revision)
.await;
disabled_revision = Some(mutation.config_revision);
mutation.instance
} else {
previous.clone()
};
let expected_revision = disabled_revision.unwrap_or(original_revision);
let transition = match self
.state
.begin_transition(
&disabled.id,
disabled.revision,
&target.artifact_sha256,
target_state.clone(),
)
.await
{
Ok(transition) => transition,
Err(error) => {
return Err(self
.recover_failed_state_transition(
None,
&previous,
&previous_state,
disabled_revision,
context,
error,
)
.await);
}
};
// 迁移开始后的所有提交前失败都走同一恢复出口;提交成功后不再回滚旧配置
let committed = async {
self.preparation
.migrate_state(&migration_prepared, transition.clone())
.await?;
let mut current = self
.store
.load_instances()
.await
.map_err(|error| map_store_error(error, "plugin"))?;
if current.config_revision != expected_revision {
return Err(AdminError::conflict("迁移期间插件配置已发生变化"));
}
let candidate_revision = next_revision(expected_revision)?;
target.revision = candidate_revision;
current.instances.retain(|item| item.id != target.id);
current.instances.push(target.clone());
current.config_revision = candidate_revision;
apply_replacements(&mut current, replacements);
let prepared = self
.prepare_instance_candidate(current.clone(), confirmation)
.await?;
let result = self
.store
.save_instance_replacing(
target,
expected_revision,
PluginStateCommit {
configuration: target_state,
transition_id: Some(transition.id.clone()),
},
replacements,
context,
)
.await
.map_err(|error| map_store_error(error, "plugin"))?;
Ok((current, prepared, result))
}
.await;
let (current, prepared, result) = match committed {
Ok(committed) => committed,
Err(error) => {
return Err(self
.recover_failed_state_transition(
Some(&transition.id),
&previous,
&previous_state,
disabled_revision,
context,
error,
)
.await);
}
};
self.publish_instance(current, prepared, &result, replacements, confirmation)
.await?;
drop(migration_prepared);
Ok(result)
}
async fn recover_failed_state_transition(
&self,
transition_id: Option<&str>,
previous: &PluginInstance,
previous_state: &PluginStateConfiguration,
disabled_revision: Option<Revision>,
context: &MutationContext,
original_error: AdminError,
) -> AdminError {
if let Some(transition_id) = transition_id {
self.state.abort_transition(transition_id).await;
}
let Some(disabled_revision) = disabled_revision else {
return original_error;
};
let restored = async {
let mut snapshot = self
.store
.load_instances()
.await
.map_err(|error| map_store_error(error, "plugin"))?;
if snapshot.config_revision != disabled_revision {
return Err(AdminError::conflict(
"状态迁移失败;实例已保持停用,且配置已被其他操作修改",
));
}
let candidate_revision = next_revision(disabled_revision)?;
let mut restore = previous.clone();
restore.enabled = true;
restore.revision = candidate_revision;
snapshot.instances.retain(|item| item.id != restore.id);
snapshot.instances.push(restore.clone());
snapshot.config_revision = candidate_revision;
let prepared = self.preparation.prepare(snapshot.clone()).await?;
let mutation = self
.store
.save_instance_with_state(
restore,
disabled_revision,
PluginStateCommit {
configuration: previous_state.clone(),
transition_id: None,
},
context,
)
.await
.map_err(|error| map_store_error(error, "plugin"))?;
self.publish_instance(snapshot, prepared, &mutation, &[], None)
.await?;
Ok::<(), AdminError>(())
}
.await;
match restored {
Ok(()) => original_error,
Err(error) => {
tracing::warn!(
error_kind = ?error.kind(),
"plugin state migration failed and the previous instance could not be restored"
);
AdminError::conflict("状态迁移失败;为避免覆盖并发配置,插件实例已保持停用")
}
}
}
}
@@ -0,0 +1,15 @@
//! 插件实例用例的职责边界与静态检查结果
mod configuration;
mod migration;
mod mutations;
mod publication;
mod queries;
use super::PluginsService;
#[derive(Clone, Default)]
pub(super) struct PackageStatus {
pub(super) warning: Option<String>,
pub(super) load_error: Option<String>,
}
@@ -0,0 +1,333 @@
//! 插件实例的配置提交、发布、停用与删除用例
use std::collections::BTreeSet;
use super::{
PluginsService,
configuration::{apply_replacements, next_revision, same_creation_request, validate_input},
};
use crate::{
model::{
AdminError, MutationContext, Revision,
plugins::{
instances::{
ConfigurePluginInstance, PluginInstance, PluginInstanceMutation,
PluginInstanceSnapshot,
},
state::PluginStateCommit,
},
},
use_case::{map_store_error, publish_committed},
};
impl PluginsService {
pub async fn configure_instance(
&self,
id: Option<&str>,
input: ConfigurePluginInstance,
context: &MutationContext,
) -> Result<PluginInstanceMutation, AdminError> {
if id.is_some() && input.creation_id.is_some() {
return Err(AdminError::invalid("编辑配置不能使用创建标识"));
}
if id.is_none() && input.expected_revision.is_some() {
return Err(AdminError::invalid("创建配置不能指定已有配置版本"));
}
if let Some(creation_id) = &input.creation_id {
let parsed = uuid::Uuid::parse_str(creation_id)
.map_err(|_| AdminError::invalid("配置创建标识必须是 UUID"))?;
if parsed.to_string() != *creation_id {
return Err(AdminError::invalid("配置创建标识必须使用标准 UUID 格式"));
}
}
let snapshot = self
.store
.load_instances()
.await
.map_err(|e| map_store_error(e, "plugin"))?;
let existing = match id {
Some(id) => Some(
snapshot
.instances
.iter()
.find(|instance| instance.id == id)
.cloned()
.ok_or_else(|| AdminError::not_found("插件实例不存在"))?,
),
None => input.creation_id.as_ref().and_then(|creation_id| {
snapshot
.instances
.iter()
.find(|instance| instance.id == *creation_id)
.cloned()
}),
};
if let Some(existing) = &existing {
if id.is_none() && !same_creation_request(existing, &input) {
return Err(AdminError::conflict("此配置已创建,请加载最新配置后编辑"));
}
if input
.expected_revision
.is_some_and(|revision| revision != existing.revision.get())
{
return Err(AdminError::conflict("插件配置已变更,请加载最新配置后重试"));
}
}
self.configure_from_snapshot(snapshot, existing, input, context)
.await
}
pub(in crate::use_case::plugins) async fn configure_from_snapshot(
&self,
mut snapshot: PluginInstanceSnapshot,
existing: Option<PluginInstance>,
input: ConfigurePluginInstance,
context: &MutationContext,
) -> Result<PluginInstanceMutation, AdminError> {
let expected_revision = snapshot.config_revision;
let candidate_revision = next_revision(expected_revision)?;
let artifacts = self
.store
.list_artifacts()
.await
.map_err(|error| map_store_error(error, "plugin"))?;
let artifact = artifacts
.iter()
.find(|artifact| artifact.metadata.sha256 == input.artifact_sha256)
.ok_or_else(|| AdminError::not_found("插件制品不存在"))?;
if artifact.accepted_at.is_none() {
return Err(AdminError::invalid("请先安装并确认完整信任该版本"));
}
let confirmation = input.compatibility_confirmation.as_ref();
if let Some(confirmed) = confirmation
&& (!input.enabled
|| existing.as_ref().is_none_or(|instance| {
instance.id != confirmed.instance_id
|| instance.revision.get() != confirmed.instance_revision
})
|| input.artifact_sha256 != confirmed.artifact_sha256
|| self.inspector.compatibility_host_fingerprint()? != confirmed.host_fingerprint)
{
return Err(AdminError::conflict(
"宿主、插件包或配置已变化,请重新确认启动风险",
));
}
if input.enabled {
let status = self.package_status(&input.artifact_sha256).await?;
if let Some(reason) = status.load_error {
return Err(AdminError::invalid(format!(
"{reason},无法启动,请安装兼容版本"
)));
}
if let Some(reason) = status.warning
&& confirmation.is_none()
{
return Err(AdminError::conflict(format!(
"插件版本不兼容:{reason};请先确认启动风险"
)));
}
}
let same_plugin = |digest: &str| {
artifacts.iter().any(|item| {
item.metadata.sha256 == digest
&& item.metadata.plugin_id == artifact.metadata.plugin_id
})
};
if existing
.as_ref()
.is_some_and(|previous| !same_plugin(&previous.artifact_sha256))
{
return Err(AdminError::invalid("不能将已有配置切换为另一个插件"));
}
if input.enabled
&& snapshot.instances.iter().any(|other| {
other.enabled
&& existing
.as_ref()
.is_none_or(|current| current.id != other.id)
&& same_plugin(&other.artifact_sha256)
&& !input
.replace_instances
.iter()
.any(|replacement| replacement.id == other.id)
})
{
return Err(AdminError::conflict(
"此插件已有启用配置,请先切换或停用旧配置",
));
}
let saved = if let Some(previous) = &existing
&& previous.artifact_sha256 != input.artifact_sha256
&& input.secrets.is_none()
{
self.store
.load_version_configuration(&previous.id, &input.artifact_sha256)
.await
.map_err(|error| map_store_error(error, "plugin"))?
} else {
None
};
let instance = PluginInstance {
id: existing.as_ref().map_or_else(
|| {
input
.creation_id
.clone()
.unwrap_or_else(|| uuid::Uuid::now_v7().to_string())
},
|instance| instance.id.clone(),
),
name: input.name.trim().to_owned(),
artifact_sha256: input.artifact_sha256,
enabled: input.enabled,
trusted_process: true,
configuration: input.configuration,
secrets: input
.secrets
.or_else(|| saved.map(|saved| saved.secrets))
.unwrap_or_else(|| {
existing
.as_ref()
.map_or_else(Default::default, |instance| instance.secrets.clone())
}),
bindings: input.bindings,
revision: candidate_revision,
};
validate_input(&instance)?;
if existing
.as_ref()
.is_some_and(|previous| previous.artifact_sha256 != instance.artifact_sha256)
&& !self
.preparation
.configuration_ready(instance.clone(), &artifact.metadata)
.await?
{
return Err(AdminError::invalid(
"目标版本仍缺少必填设置,请补充后再切换",
));
}
let replacements = input.replace_instances;
if !replacements.is_empty() {
if !instance.enabled || replacements.len() > 256 {
return Err(AdminError::invalid(
"仅启用配置时可以切换其他配置,单次最多 256 项",
));
}
let mut ids = BTreeSet::new();
for replacement in &replacements {
if replacement.id == instance.id || !ids.insert(&replacement.id) {
return Err(AdminError::invalid("停用列表不能重复或包含当前配置"));
}
let previous = snapshot
.instances
.iter()
.find(|item| item.id == replacement.id)
.ok_or_else(|| AdminError::conflict("待停用配置已变更,请重新确认"))?;
if !previous.enabled || previous.revision.get() != replacement.expected_revision {
return Err(AdminError::conflict("待停用配置已变更,请重新确认"));
}
if !artifacts.iter().any(|item| {
item.metadata.sha256 == previous.artifact_sha256
&& item.metadata.plugin_id == artifact.metadata.plugin_id
}) {
return Err(AdminError::invalid("只能切换同一插件的配置"));
}
}
}
let state = self.preparation.validate(instance.clone()).await?;
if let Some(existing) = &existing
&& self.state.transition_required(&existing.id, &state).await?
{
return self
.configure_instance_with_state_transition(
snapshot,
instance,
state,
&replacements,
confirmation,
context,
)
.await;
}
snapshot.instances.retain(|item| item.id != instance.id);
snapshot.instances.push(instance.clone());
snapshot.config_revision = candidate_revision;
apply_replacements(&mut snapshot, &replacements);
let prepared = self
.prepare_instance_candidate(snapshot.clone(), confirmation)
.await?;
let result = self
.store
.save_instance_replacing(
instance,
expected_revision,
PluginStateCommit {
configuration: state,
transition_id: None,
},
&replacements,
context,
)
.await
.map_err(|e| map_store_error(e, "plugin"))?;
self.publish_instance(snapshot, prepared, &result, &replacements, confirmation)
.await?;
Ok(result)
}
pub async fn delete_instance(
&self,
id: &str,
context: &MutationContext,
) -> Result<Revision, AdminError> {
let snapshot = self
.store
.load_instances()
.await
.map_err(|e| map_store_error(e, "plugin"))?;
let instance = snapshot
.instances
.iter()
.find(|instance| instance.id == id)
.ok_or_else(|| AdminError::not_found("插件实例不存在"))?;
if instance.enabled {
return Err(AdminError::conflict("请先停用插件实例"));
}
let revision = self
.store
.delete_instance(id, snapshot.config_revision, context)
.await
.map_err(|e| map_store_error(e, "plugin"))?;
publish_committed(self.snapshots.as_ref(), revision).await?;
Ok(revision)
}
/// 紧急管理修复不要求损坏插件成功准备;提交后沿用现有发布与暂停合同
pub async fn disable_instance(
&self,
id: &str,
context: &MutationContext,
) -> Result<PluginInstanceMutation, AdminError> {
let PluginInstanceSnapshot {
config_revision,
instances,
} = self
.store
.load_instances()
.await
.map_err(|e| map_store_error(e, "plugin"))?;
let mut instance = instances
.into_iter()
.find(|instance| instance.id == id)
.ok_or_else(|| AdminError::not_found("插件实例不存在"))?;
instance.enabled = false;
let result = self
.store
.save_instance(instance, config_revision, context)
.await
.map_err(|e| map_store_error(e, "plugin"))?;
publish_committed(self.snapshots.as_ref(), result.config_revision).await?;
Ok(result)
}
}
@@ -0,0 +1,63 @@
//! 配置提交与迁移恢复共用的状态激活和发布收尾
use gateway_core::routing::extensions::ExtensionSetReference;
use super::{PluginsService, configuration::apply_replacements};
use crate::{
model::{
AdminError,
plugins::instances::{
PluginCompatibilityConfirmation, PluginInstanceMutation, PluginInstanceReplacement,
PluginInstanceSnapshot,
},
},
use_case::publish_committed,
};
impl PluginsService {
pub(super) async fn prepare_instance_candidate(
&self,
snapshot: PluginInstanceSnapshot,
confirmation: Option<&PluginCompatibilityConfirmation>,
) -> Result<ExtensionSetReference, AdminError> {
match confirmation {
Some(confirmation) => {
self.preparation
.prepare_confirmed(snapshot, confirmation.clone())
.await
}
None => self.preparation.prepare(snapshot).await,
}
}
/// 以事务返回的版本激活并发布,候选强引用必须覆盖整个提交后阶段
pub(super) async fn publish_instance(
&self,
mut snapshot: PluginInstanceSnapshot,
prepared: ExtensionSetReference,
mutation: &PluginInstanceMutation,
replacements: &[PluginInstanceReplacement],
confirmation: Option<&PluginCompatibilityConfirmation>,
) -> Result<(), AdminError> {
let prepared = if mutation.config_revision == snapshot.config_revision
&& mutation.instance.revision == snapshot.config_revision
{
prepared
} else {
snapshot.config_revision = mutation.config_revision;
snapshot
.instances
.retain(|item| item.id != mutation.instance.id);
snapshot.instances.push(mutation.instance.clone());
apply_replacements(&mut snapshot, replacements);
self.prepare_instance_candidate(snapshot, confirmation)
.await?
};
self.preparation
.activate_state(&prepared, &mutation.instance)
.await?;
publish_committed(self.snapshots.as_ref(), mutation.config_revision).await?;
drop(prepared);
Ok(())
}
}
@@ -0,0 +1,122 @@
//! 插件实例列表与静态包检查的只读投影,不从发布事实推断进程状态
use std::collections::BTreeMap;
use super::{PackageStatus, PluginsService};
use crate::{
model::{
AdminError, AdminErrorKind,
plugins::instances::{
PluginInstanceRuntime, PluginInstanceRuntimeStatus, PluginInstanceView,
},
},
use_case::map_store_error,
};
impl PluginsService {
/// 只校验固定包体,不启动插件;停用实例也需区分兼容性提醒与加载错误
pub(in crate::use_case::plugins) async fn package_status(
&self,
digest: &str,
) -> Result<PackageStatus, AdminError> {
// 运行宿主和摘要固定,缓存纯静态结论,避免管理页轮询反复解包
// 只缓存有限数量的确定结果,暂时性错误仍可重试
if let Some(status) = self.compatibility.lock().await.get(digest).cloned() {
return Ok(status);
}
let artifact = self
.store
.load_artifact(digest)
.await
.map_err(|error| map_store_error(error, "plugin"))?;
let status = match self
.inspector
.compatibility_warning(artifact.archive, digest.to_owned())
.await
{
Ok(warning) => PackageStatus {
warning,
load_error: None,
},
Err(error) if error.kind() == AdminErrorKind::Invalid => PackageStatus {
warning: None,
load_error: Some(error.message().to_owned()),
},
Err(error) => return Err(error),
};
let mut cache = self.compatibility.lock().await;
if cache.len() < 128 {
cache.insert(digest.to_owned(), status.clone());
}
Ok(status)
}
pub async fn instances(&self) -> Result<Vec<PluginInstanceView>, AdminError> {
let snapshot = self
.store
.load_instances()
.await
.map_err(|e| map_store_error(e, "plugin"))?;
let published = self.published.snapshot_for_diagnostics();
let published_revision = published.as_ref().map(|snapshot| snapshot.revision().get());
let mut diagnostics = self
.preparation
.runtime_diagnostics(
&snapshot,
published_revision,
published
.as_ref()
.and_then(|snapshot| snapshot.extensions()),
)
.await
.unwrap_or_default();
let artifacts = self
.list()
.await?
.into_iter()
.map(|artifact| (artifact.metadata.sha256.clone(), artifact.metadata))
.collect::<BTreeMap<_, _>>();
let mut views = Vec::with_capacity(snapshot.instances.len());
for instance in snapshot.instances {
let metadata = artifacts
.get(&instance.artifact_sha256)
.ok_or_else(|| AdminError::not_found("插件制品不存在"))?;
let package_status = self.package_status(&instance.artifact_sha256).await?;
let configuration_required = package_status.load_error.is_none()
&& !self
.preparation
.configuration_ready(instance.clone(), metadata)
.await?;
let runtime =
diagnostics
.remove(&instance.id)
.unwrap_or_else(|| PluginInstanceRuntime {
status: if !instance.enabled {
PluginInstanceRuntimeStatus::Disabled
} else {
PluginInstanceRuntimeStatus::Unknown
},
actual_revision: None,
actual_artifact_sha256: None,
failure: None,
draining_revisions: Vec::new(),
load: None,
});
views.push(PluginInstanceView {
api_deprecations: if package_status.load_error.is_none() {
self.inspector.api_deprecations(metadata)?
} else {
Vec::new()
},
configuration_required,
compatibility_warning: package_status.warning,
load_error: package_status.load_error,
running: runtime.status == PluginInstanceRuntimeStatus::Running,
published_revision,
runtime,
instance,
});
}
Ok(views)
}
}
@@ -10,9 +10,9 @@ use crate::{
plugins::{
PluginArtifactMetadata,
instances::{
ConfigurePluginInstance, PluginInstance, PluginInstanceMutation,
PluginRollbackPlan, PluginRollbackTarget, PluginVersionConfiguration,
PluginVersionPlan, RollbackPluginInstance,
ConfigurePluginInstance, PluginCompatibilityConfirmation, PluginInstance,
PluginInstanceMutation, PluginRollbackPlan, PluginRollbackTarget,
PluginVersionConfiguration, PluginVersionPlan, RollbackPluginInstance,
},
},
},
@@ -61,6 +61,7 @@ impl PluginsService {
AdminError::invalid("该版本没有可恢复的配置,请通过版本切换检查当前设置")
})?;
let configuration = ConfigurePluginInstance {
compatibility_confirmation: None,
creation_id: None,
expected_revision: None,
replace_instances: Vec::new(),
@@ -142,7 +143,21 @@ impl PluginsService {
.find(|instance| instance.id == id)
.ok_or_else(|| AdminError::not_found("插件实例不存在"))?;
let (configuration, restored) = self.version_configuration(instance, digest).await?;
let status = self.package_status(digest).await?;
let compatibility_confirmation = if status.warning.is_some() {
Some(PluginCompatibilityConfirmation {
host_fingerprint: self.inspector.compatibility_host_fingerprint()?,
instance_id: instance.id.clone(),
instance_revision: instance.revision.get(),
artifact_sha256: digest.to_owned(),
})
} else {
None
};
Ok(PluginVersionPlan {
compatibility_warning: status.warning,
compatibility_confirmation,
load_error: status.load_error,
instance_revision: instance.revision.get(),
artifact_sha256: digest.to_owned(),
configuration: configuration.configuration,
@@ -179,6 +194,7 @@ impl PluginsService {
.version_configuration(&existing, &input.artifact_sha256)
.await?;
let request = ConfigurePluginInstance {
compatibility_confirmation: None,
creation_id: None,
expected_revision: Some(existing.revision.get()),
replace_instances: Vec::new(),
@@ -39,6 +39,7 @@ struct FixtureData {
artifact: InstalledPluginArtifact,
snapshot: PluginInstanceSnapshot,
saves: usize,
compatibility_warning: Option<String>,
publications: Vec<u64>,
}
@@ -61,6 +62,7 @@ impl Fixture {
instances: existing.into_iter().collect(),
},
saves: 0,
compatibility_warning: None,
publications: Vec::new(),
}),
})
@@ -89,6 +91,14 @@ impl SnapshotControl for Fixture {
#[async_trait]
impl gateway_admin::ports::plugins::PluginPackageInspector for Fixture {
async fn compatibility_warning(
&self,
_: Arc<[u8]>,
_: String,
) -> Result<Option<String>, AdminError> {
Ok(self.data.lock().unwrap().compatibility_warning.clone())
}
async fn inspect(
&self,
archive: Arc<[u8]>,
@@ -488,3 +498,20 @@ fn context() -> MutationContext {
fn store_error(kind: AdminStoreErrorKind) -> AdminStoreError {
AdminStoreError::new(kind, "plugin", "install fixture")
}
#[tokio::test]
async fn installing_an_incompatible_package_keeps_the_default_instance_stopped() {
let fixture = Fixture::new(metadata(true), None);
fixture.data.lock().unwrap().compatibility_warning = Some("宿主范围不匹配".into());
let result = fixture
.service()
.accept_artifact(DIGEST, &context())
.await
.unwrap();
assert!(!result.configuration_required);
let data = fixture.data.lock().unwrap();
assert_eq!(data.snapshot.instances.len(), 1);
assert!(!data.snapshot.instances[0].enabled);
assert!(!data.snapshot.instances[0].bindings.is_empty());
assert!(data.artifact.accepted_at.is_some());
}
@@ -49,6 +49,8 @@ use serde_json::json;
use super::TestPluginPorts;
mod queries;
const OLD_ARTIFACT: &str = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
const NEW_ARTIFACT: &str = "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb";
@@ -61,6 +63,10 @@ enum MigrationBehavior {
#[async_trait]
impl gateway_admin::ports::plugins::PluginPackageInspector for LifecycleFixture {
fn compatibility_host_fingerprint(&self) -> Result<String, AdminError> {
Ok("test-host".into())
}
fn api_deprecations(
&self,
metadata: &PluginArtifactMetadata,
@@ -88,6 +94,11 @@ impl gateway_admin::ports::plugins::PluginPackageInspector for LifecycleFixture
archive: Arc<[u8]>,
digest: String,
) -> Result<Option<String>, AdminError> {
let gate = self.inspection_gate.lock().unwrap().clone();
if let Some(gate) = gate.filter(|gate| gate.digest == digest) {
gate.started.notify_one();
gate.release.notified().await;
}
self.inspect(archive, Some(digest)).await?;
Ok(self.data.lock().unwrap().compatibility_warning.clone())
}
@@ -143,11 +154,19 @@ struct LifecycleFixture {
behavior: MigrationBehavior,
data: Mutex<FixtureData>,
next_prepared: AtomicU64,
inspection_gate: Mutex<Option<Arc<InspectionGate>>>,
}
struct InspectionGate {
digest: String,
started: tokio::sync::Notify,
release: tokio::sync::Notify,
}
impl LifecycleFixture {
fn new(behavior: MigrationBehavior) -> Arc<Self> {
Arc::new(Self {
inspection_gate: Mutex::new(None),
behavior,
data: Mutex::new(FixtureData {
snapshot: PluginInstanceSnapshot {
@@ -267,6 +286,22 @@ impl ExtensionSetLease for Lease {
#[async_trait]
impl PluginPreparation for LifecycleFixture {
async fn prepare_confirmed(
&self,
snapshot: PluginInstanceSnapshot,
confirmation: gateway_admin::model::plugins::instances::PluginCompatibilityConfirmation,
) -> Result<ExtensionSetReference, AdminError> {
assert_eq!(confirmation.host_fingerprint, "test-host");
assert!(
snapshot
.instances
.iter()
.any(|instance| instance.id == confirmation.instance_id
&& instance.artifact_sha256 == confirmation.artifact_sha256)
);
self.prepare(snapshot).await
}
async fn configuration_ready(
&self,
instance: PluginInstance,
@@ -716,6 +751,7 @@ fn rollback_input() -> RollbackPluginInstance {
fn input() -> ConfigurePluginInstance {
ConfigurePluginInstance {
compatibility_confirmation: None,
replace_instances: Vec::new(),
creation_id: None,
expected_revision: None,
@@ -970,49 +1006,6 @@ fn state_unavailable<T>() -> PluginStateStoreResult<T> {
))
}
#[tokio::test]
async fn instance_list_uses_runtime_diagnostics_and_keeps_running_compatibility_derived() {
let fixture = LifecycleFixture::new(MigrationBehavior::Succeed);
let expected = PluginInstanceRuntime {
status: PluginInstanceRuntimeStatus::PreparationFailed,
actual_revision: Some(9),
actual_artifact_sha256: Some(OLD_ARTIFACT.into()),
failure: Some(PluginInstanceRuntimeFailure {
code: "unavailable".into(),
message: "插件候选准备失败".into(),
}),
draining_revisions: vec![8],
};
fixture.data.lock().unwrap().diagnostics = Some(BTreeMap::from([(
"00000000-0000-7000-8000-000000000001".into(),
expected.clone(),
)]));
let views = service(fixture, Arc::new(Published::default()))
.instances()
.await
.unwrap();
assert_eq!(views.len(), 1);
assert!(!views[0].running);
assert_eq!(views[0].runtime, expected);
}
#[tokio::test]
async fn instance_list_falls_back_without_inventing_runtime_failures() {
let fixture = LifecycleFixture::new(MigrationBehavior::Succeed);
let views = service(fixture, Arc::new(Published::default()))
.instances()
.await
.unwrap();
assert_eq!(views.len(), 1);
assert_eq!(
views[0].runtime.status,
PluginInstanceRuntimeStatus::AwaitingPublication
);
assert!(!views[0].running);
assert!(views[0].runtime.failure.is_none());
}
#[tokio::test]
async fn incompatible_state_upgrade_quiesces_migrates_and_promotes_the_target() {
let fixture = LifecycleFixture::new(MigrationBehavior::Succeed);
@@ -1644,7 +1637,7 @@ async fn transient_compatibility_check_failure_can_be_retried() {
}
#[tokio::test]
async fn compatibility_warning_allows_enabling_and_persists_after_success() {
async fn incompatible_start_requires_confirmation_and_preserves_the_warning() {
let fixture = LifecycleFixture::new(MigrationBehavior::Succeed);
{
let mut data = fixture.data.lock().unwrap();
@@ -1657,6 +1650,21 @@ async fn compatibility_warning_allows_enabling_and_persists_after_success() {
assert!(view.load_error.is_none());
let mut request = input();
request.artifact_sha256 = OLD_ARTIFACT.into();
let error = service
.configure_instance(Some(&view.instance.id), request, &context())
.await
.err()
.unwrap();
assert_eq!(error.kind(), AdminErrorKind::Conflict);
assert!(fixture.data.lock().unwrap().saves.is_empty());
assert!(fixture.data.lock().unwrap().prepared_snapshots.is_empty());
let plan = service
.version_plan(&view.instance.id, OLD_ARTIFACT)
.await
.unwrap();
let mut request = input();
request.artifact_sha256 = OLD_ARTIFACT.into();
request.compatibility_confirmation = plan.compatibility_confirmation;
let result = service
.configure_instance(Some(&view.instance.id), request, &context())
.await
@@ -1667,3 +1675,54 @@ async fn compatibility_warning_allows_enabling_and_persists_after_success() {
assert!(view.compatibility_warning.is_some());
assert!(view.load_error.is_none());
}
#[tokio::test]
async fn incompatible_start_rejects_stale_confirmations_and_unloadable_packages() {
for mismatch in ["host", "instance", "revision", "artifact", "unloadable"] {
let fixture = LifecycleFixture::new(MigrationBehavior::Succeed);
{
let mut data = fixture.data.lock().unwrap();
data.snapshot.instances[0].enabled = false;
data.compatibility_warning = Some("middleware v2 未声明兼容".into());
if mismatch == "unloadable" {
data.inspection_error = Some(AdminErrorKind::Invalid);
}
}
let service = service(fixture.clone(), Arc::new(Published::default()));
let previous = fixture.data.lock().unwrap().snapshot.instances[0].clone();
let mut confirmation =
gateway_admin::model::plugins::instances::PluginCompatibilityConfirmation {
host_fingerprint: "test-host".into(),
instance_id: previous.id.clone(),
instance_revision: previous.revision.get(),
artifact_sha256: OLD_ARTIFACT.into(),
};
match mismatch {
"host" => confirmation.host_fingerprint = "another-host".into(),
"instance" => confirmation.instance_id = "another-instance".into(),
"revision" => confirmation.instance_revision += 1,
"artifact" => confirmation.artifact_sha256 = NEW_ARTIFACT.into(),
_ => {}
}
let mut request = input();
request.artifact_sha256 = OLD_ARTIFACT.into();
request.compatibility_confirmation = Some(confirmation);
let error = service
.configure_instance(Some(&previous.id), request, &context())
.await
.err()
.unwrap();
assert_eq!(
error.kind(),
if mismatch == "unloadable" {
AdminErrorKind::Invalid
} else {
AdminErrorKind::Conflict
}
);
let data = fixture.data.lock().unwrap();
assert!(data.saves.is_empty(), "{mismatch}");
assert!(data.prepared_snapshots.is_empty(), "{mismatch}");
assert!(!data.snapshot.instances[0].enabled);
}
}
@@ -0,0 +1,151 @@
//! 实例运行诊断与包检查缓存的查询回归测试
use super::*;
use gateway_admin::model::plugins::instances::PluginInstanceRuntimeLoad;
#[tokio::test]
async fn instance_list_uses_runtime_diagnostics_and_keeps_running_compatibility_derived() {
let fixture = LifecycleFixture::new(MigrationBehavior::Succeed);
let expected = PluginInstanceRuntime {
status: PluginInstanceRuntimeStatus::PreparationFailed,
actual_revision: Some(9),
actual_artifact_sha256: Some(OLD_ARTIFACT.into()),
failure: Some(PluginInstanceRuntimeFailure {
code: "unavailable".into(),
message: "插件候选准备失败".into(),
}),
draining_revisions: vec![8],
load: Some(PluginInstanceRuntimeLoad {
active_calls: 3,
executing_calls: 1,
queued_calls: 1,
resuming_calls: 1,
pending_callbacks: 2,
active_streams: 0,
}),
};
fixture.data.lock().unwrap().diagnostics = Some(BTreeMap::from([(
"00000000-0000-7000-8000-000000000001".into(),
expected.clone(),
)]));
let views = service(fixture, Arc::new(Published::default()))
.instances()
.await
.unwrap();
assert_eq!(views.len(), 1);
assert!(!views[0].running);
assert_eq!(views[0].runtime, expected);
}
#[tokio::test]
async fn instance_list_falls_back_without_inventing_runtime_failures() {
let fixture = LifecycleFixture::new(MigrationBehavior::Succeed);
let views = service(fixture, Arc::new(Published::default()))
.instances()
.await
.unwrap();
assert_eq!(views.len(), 1);
assert_eq!(
views[0].runtime.status,
PluginInstanceRuntimeStatus::Unknown
);
assert!(!views[0].running);
assert!(views[0].runtime.failure.is_none());
assert!(views[0].runtime.actual_revision.is_none());
assert!(views[0].runtime.load.is_none());
}
#[tokio::test]
async fn disabled_instance_without_runtime_diagnostics_has_a_known_disabled_status() {
let fixture = LifecycleFixture::new(MigrationBehavior::Succeed);
fixture.data.lock().unwrap().snapshot.instances[0].enabled = false;
let views = service(fixture, Arc::new(Published::default()))
.instances()
.await
.unwrap();
assert_eq!(
views[0].runtime.status,
PluginInstanceRuntimeStatus::Disabled
);
assert!(!views[0].running);
assert!(views[0].runtime.failure.is_none());
assert!(views[0].runtime.load.is_none());
}
#[tokio::test]
async fn matching_publication_without_process_diagnostics_does_not_imply_running() {
for diagnostics in [None, Some(BTreeMap::new())] {
let fixture = LifecycleFixture::new(MigrationBehavior::Succeed);
fixture.data.lock().unwrap().diagnostics = diagnostics;
let revision = fixture.snapshot().config_revision.get();
let published = RuntimeSnapshotHandle::new(
gateway_core::routing::RuntimeSnapshot::new(
ConfigRevision::new(revision).unwrap(),
gateway_core::settings::SettingsValues::new(
1,
"round_robin",
BTreeMap::new(),
None,
None,
),
Vec::new(),
Vec::new(),
Vec::new(),
)
.unwrap(),
);
let service = PluginsService::new(
fixture.clone(),
fixture.clone(),
PluginDistributionPorts::new(Arc::new(TestPluginPorts), Arc::new(TestPluginPorts)),
Arc::new(Published::default()),
fixture.clone(),
published,
fixture,
);
let view = service.instances().await.unwrap().remove(0);
assert_eq!(view.published_revision, Some(revision));
assert_eq!(view.runtime.status, PluginInstanceRuntimeStatus::Unknown);
assert!(!view.running);
assert!(view.runtime.actual_revision.is_none());
assert!(view.runtime.actual_artifact_sha256.is_none());
assert!(view.runtime.failure.is_none());
}
}
#[tokio::test]
async fn cached_package_query_is_not_blocked_by_another_package_inspection() {
let fixture = LifecycleFixture::new(MigrationBehavior::Succeed);
let service = Arc::new(service(fixture.clone(), Arc::new(Published::default())));
service.instances().await.unwrap();
let gate = Arc::new(InspectionGate {
digest: NEW_ARTIFACT.into(),
started: tokio::sync::Notify::new(),
release: tokio::sync::Notify::new(),
});
*fixture.inspection_gate.lock().unwrap() = Some(gate.clone());
fixture.data.lock().unwrap().snapshot.instances[0].artifact_sha256 = NEW_ARTIFACT.into();
let inspecting = {
let service = service.clone();
tokio::spawn(async move { service.instances().await })
};
tokio::time::timeout(std::time::Duration::from_secs(1), gate.started.notified())
.await
.expect("another package inspection started");
fixture.data.lock().unwrap().snapshot.instances[0].artifact_sha256 = OLD_ARTIFACT.into();
let cached =
tokio::time::timeout(std::time::Duration::from_millis(100), service.instances()).await;
gate.release.notify_one();
inspecting.await.unwrap().unwrap();
assert_eq!(
cached
.expect("cached query must not wait for inspection")
.unwrap()[0]
.instance
.artifact_sha256,
OLD_ARTIFACT
);
assert_eq!(fixture.data.lock().unwrap().archive_reads, 2);
}
+1 -1
View File
@@ -37,7 +37,7 @@ tower-http = { version = "0.7.0", features = ["compression-gzip", "cors", "fs",
tracing = "0.1.44"
url = "2.5.8"
uuid = { version = "1.23.3", features = ["v7"] }
zstd = "=0.13.3"
zstd = "=0.14.3"
zip = { version = "9.0.4", default-features = false, features = ["deflate-flate2"] }
[dev-dependencies]
@@ -9,8 +9,8 @@ use axum::{
};
use gateway_admin::model::plugins::instances::{
ConfigurePluginInstance, PluginCapabilityBinding, PluginInstance, PluginInstanceRuntime,
PluginInstanceRuntimeFailure, PluginInstanceRuntimeStatus, PluginInstanceView,
RollbackPluginInstance,
PluginInstanceRuntimeFailure, PluginInstanceRuntimeLoad, PluginInstanceRuntimeStatus,
PluginInstanceView, RollbackPluginInstance,
};
use serde::{Deserialize, Serialize};
@@ -70,6 +70,7 @@ struct InstanceView {
#[serde(rename_all = "snake_case")]
enum InstanceRuntimeStatus {
Disabled,
Unknown,
AwaitingPublication,
Preparing,
Running,
@@ -83,6 +84,7 @@ impl From<PluginInstanceRuntimeStatus> for InstanceRuntimeStatus {
fn from(value: PluginInstanceRuntimeStatus) -> Self {
match value {
PluginInstanceRuntimeStatus::Disabled => Self::Disabled,
PluginInstanceRuntimeStatus::Unknown => Self::Unknown,
PluginInstanceRuntimeStatus::AwaitingPublication => Self::AwaitingPublication,
PluginInstanceRuntimeStatus::Preparing => Self::Preparing,
PluginInstanceRuntimeStatus::Running => Self::Running,
@@ -118,6 +120,31 @@ struct InstanceRuntimeView {
actual_artifact_sha256: Option<String>,
failure: Option<InstanceRuntimeFailure>,
draining_revisions: Vec<u64>,
load: Option<InstanceRuntimeLoad>,
}
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
struct InstanceRuntimeLoad {
active_calls: usize,
executing_calls: usize,
queued_calls: usize,
resuming_calls: usize,
pending_callbacks: usize,
active_streams: usize,
}
impl From<PluginInstanceRuntimeLoad> for InstanceRuntimeLoad {
fn from(value: PluginInstanceRuntimeLoad) -> Self {
Self {
active_calls: value.active_calls,
executing_calls: value.executing_calls,
queued_calls: value.queued_calls,
resuming_calls: value.resuming_calls,
pending_callbacks: value.pending_callbacks,
active_streams: value.active_streams,
}
}
}
impl From<PluginInstanceRuntime> for InstanceRuntimeView {
@@ -128,6 +155,7 @@ impl From<PluginInstanceRuntime> for InstanceRuntimeView {
actual_artifact_sha256: value.actual_artifact_sha256,
failure: value.failure.map(Into::into),
draining_revisions: value.draining_revisions,
load: value.load.map(Into::into),
}
}
}
@@ -322,6 +322,17 @@ async fn instance_list_serializes_safe_runtime_diagnostics() {
assert_eq!(instance["running"], false);
assert_eq!(instance["runtime"]["status"], "preparation_failed");
assert_eq!(instance["runtime"]["actualRevision"], 6);
assert_eq!(
instance["runtime"]["load"],
serde_json::json!({
"activeCalls": 3,
"executingCalls": 1,
"queuedCalls": 1,
"resumingCalls": 1,
"pendingCallbacks": 2,
"activeStreams": 0,
})
);
assert_eq!(
instance["runtime"]["drainingRevisions"],
serde_json::json!([5])
@@ -665,6 +676,14 @@ impl gateway_admin::ports::plugins::PluginRuntimeDiagnostics for TestPluginPorts
message: "插件候选准备失败".into(),
}),
draining_revisions: vec![5],
load: Some(gateway_admin::model::plugins::instances::PluginInstanceRuntimeLoad {
active_calls: 3,
executing_calls: 1,
queued_calls: 1,
resuming_calls: 1,
pending_callbacks: 2,
active_streams: 0,
}),
},
)
})
@@ -2164,109 +2164,132 @@ async fn streaming_upstream_wire_failure_should_not_be_rewritten_as_a_gateway_er
#[tokio::test]
async fn streaming_upstream_error_event_should_be_translated_to_response_failed_for_sse() {
let trace = Arc::new(Trace::default());
let error_data = json!({
"type": "error",
"error": {
"type": "service_unavailable_error",
"code": "server_is_overloaded",
"message": "Our servers are currently overloaded. Please try again later.",
"param": null,
"future_error_field": {"keep": true}
},
"sequence_number": 2,
"future_event_field": {"keep": true}
});
let projected_error_frame = Bytes::from(encode_sse_event("error", &error_data.to_string()));
let error_wire = ProviderEvent::wire(
ProtocolWireEvent::json_with_raw_sse_metadata(
"openai",
Some("error".to_owned()),
error_data,
projected_error_frame,
None,
None,
)
.expect("valid upstream error wire"),
);
let started_wire = ProviderEvent::canonical_with_wire(
vec![started()],
ProtocolWireEvent::json(
"openai",
Some("response.created".to_owned()),
json!({
"type": "response.created",
"response": {
"id": "resp_test",
"model": "public-model",
"status": "in_progress",
"future_response_field": {"keep": true}
}
}),
)
.expect("valid upstream started wire"),
);
let session = FakeSession::streaming(
Arc::clone(&trace),
vec![
NextStep::Event(CoordinatedEvent::single(
started_wire,
CommitRequirement::CommitBeforeDelivery,
)),
NextStep::Event(CoordinatedEvent::single(
error_wire,
CommitRequirement::AlreadyCommitted,
)),
NextStep::Error(EngineError::Provider(ProviderError::new(
ProviderErrorKind::UpstreamCapacityUnavailable,
UpstreamSendState::Sent,
))),
],
);
let response = stream_execution_response(Box::new(session), None).await;
let body = to_bytes(response.into_body(), usize::MAX)
.await
.expect("read SSE body");
let body = String::from_utf8(body.to_vec()).expect("SSE is UTF-8");
let events = parse_sse_events(&body).expect("translated SSE should parse");
let failed = events
.iter()
.find(|event| event.event.as_deref() == Some("response.failed"))
.expect("translated response.failed event");
let failed: Value = serde_json::from_str(&failed.data).expect("response.failed JSON");
assert_eq!(
for (error_type, code, client_code, kind, client_status) in [
(
failed,
body.matches("event: response.failed").count(),
body.contains("event: error\n"),
body.ends_with("data: [DONE]\n\n"),
"service_unavailable_error",
"server_is_overloaded",
"server_error",
ProviderErrorKind::UpstreamCapacityUnavailable,
503,
),
(
json!({
"type": "response.failed",
"response": {
"id": "resp_test",
"model": "public-model",
"status": "failed",
"error": {
"type": "service_unavailable_error",
"code": "server_error",
"message": "Our servers are currently overloaded. Please try again later.",
"param": null,
"future_error_field": {"keep": true}
"rate_limit_error",
"turn_rate_limit_exceeded",
"turn_rate_limit_exceeded",
ProviderErrorKind::RateLimited,
429,
),
] {
let trace = Arc::new(Trace::default());
let error_data = json!({
"type": "error",
"status": 429,
"headers": {"Retry-After": "24"},
"error": {
"type": error_type,
"code": code,
"message": "Our servers are currently overloaded. Please try again later.",
"param": null,
"future_error_field": {"keep": true}
},
"sequence_number": 2,
"future_event_field": {"keep": true}
});
let projected_error_frame = Bytes::from(encode_sse_event("error", &error_data.to_string()));
let error_wire = ProviderEvent::wire(
ProtocolWireEvent::json_with_raw_sse_metadata(
"openai",
Some("error".to_owned()),
error_data,
projected_error_frame,
None,
None,
)
.expect("valid upstream error wire"),
);
let started_wire = ProviderEvent::canonical_with_wire(
vec![started()],
ProtocolWireEvent::json(
"openai",
Some("response.created".to_owned()),
json!({
"type": "response.created",
"response": {
"id": "resp_test",
"model": "public-model",
"status": "in_progress",
"future_response_field": {"keep": true}
}
}),
)
.expect("valid upstream started wire"),
);
let session = FakeSession::streaming(
Arc::clone(&trace),
vec![
NextStep::Event(CoordinatedEvent::single(
started_wire,
CommitRequirement::CommitBeforeDelivery,
)),
NextStep::Event(CoordinatedEvent::single(
error_wire,
CommitRequirement::AlreadyCommitted,
)),
NextStep::Error(EngineError::Provider(ProviderError::new(
kind,
UpstreamSendState::Sent,
))),
],
);
let response = stream_execution_response(Box::new(session), None).await;
assert_eq!(response.status(), StatusCode::OK);
let body = to_bytes(response.into_body(), usize::MAX)
.await
.expect("read SSE body");
let body = String::from_utf8(body.to_vec()).expect("SSE is UTF-8");
let events = parse_sse_events(&body).expect("translated SSE should parse");
let failed = events
.iter()
.find(|event| event.event.as_deref() == Some("response.failed"))
.expect("translated response.failed event");
let failed: Value = serde_json::from_str(&failed.data).expect("response.failed JSON");
assert_eq!(
(
failed,
body.matches("event: response.failed").count(),
body.contains("event: error\n"),
body.ends_with("data: [DONE]\n\n"),
),
(
json!({
"type": "response.failed",
"status": client_status,
"headers": {"Retry-After": "24"},
"response": {
"id": "resp_test",
"model": "public-model",
"status": "failed",
"error": {
"type": error_type,
"code": client_code,
"message": "Our servers are currently overloaded. Please try again later.",
"param": null,
"future_error_field": {"keep": true},
"headers": {"retry-after": "24"}
},
"future_response_field": {"keep": true}
},
"future_response_field": {"keep": true}
},
"sequence_number": 2,
"future_event_field": {"keep": true}
}),
1,
false,
true,
)
);
"sequence_number": 2,
"future_event_field": {"keep": true}
}),
1,
false,
true,
)
);
}
}
#[tokio::test]
@@ -948,8 +948,8 @@ fn transparent_encoder_should_translate_error_wire_to_response_failed_for_websoc
json!({
"type": "error",
"status": 429,
"headers": {"x-request-id": "req_raw"},
"error": {"type": "custom_type", "code": "custom_code", "message": "raw marker"},
"headers": {"x-request-id": "req_raw", "Retry-After": "24"},
"error": {"type": "rate_limit_error", "code": "turn_rate_limit_exceeded", "message": "raw marker"},
"future": {"keep": true}
}),
);
@@ -9,15 +9,38 @@ use std::{
use futures::future::BoxFuture;
use crate::{
error::GatewayError,
policy::{ClientApiKeyId, PlaintextClientApiKey},
routing::extensions::{ExtensionSetId, ExtensionSetReference},
routing::{
extensions::{ExtensionSetId, ExtensionSetReference},
request_settings::RequestSettings,
},
};
use super::execution::AuthenticatedClient;
/// 身份解析与认证只返回冻结权限,不提供模型执行入口
pub trait ClientIdentityPort: Send + Sync {
/// 只解析身份与冻结权限,不建立模型执行图或记录使用事实
fn resolve_client(
&self,
client_key_id: ClientApiKeyId,
settings: Option<RequestSettings>,
) -> BoxFuture<'_, Result<AuthenticatedClient, GatewayError>>;
/// 复用入口认证计划及 principal 映射,不进行模型准入
fn verify_client(
&self,
request: ClientAuthenticationRequest,
) -> BoxFuture<'_, Result<AuthenticatedClient, GatewayError>>;
}
pub const MAXIMUM_AUTHORIZATION_BYTES: usize = 8 * 1024;
/// API 从数据面 Authorization 头构造的有界认证信封
#[derive(Clone, PartialEq, Eq)]
#[derive(Clone)]
pub struct ClientAuthenticationRequest {
invocation: Option<Arc<super::extensions::InvocationOrigin>>,
request_context: Option<Arc<super::extensions::RequestContext>>,
settings: Option<crate::routing::request_settings::RequestSettings>,
authorization: Arc<str>,
@@ -25,6 +48,18 @@ pub struct ClientAuthenticationRequest {
}
impl ClientAuthenticationRequest {
/// 子认证继承受信调用的来源、取消和副作用归属,不据此跳过认证器
#[must_use]
pub fn with_invocation(mut self, origin: super::extensions::InvocationOrigin) -> Self {
self.invocation = Some(Arc::new(origin));
self
}
#[must_use]
pub fn invocation(&self) -> Option<&super::extensions::InvocationOrigin> {
self.invocation.as_deref()
}
#[must_use]
pub fn with_request_context(
mut self,
@@ -35,12 +70,18 @@ impl ClientAuthenticationRequest {
}
#[must_use]
pub fn request_context(&self) -> Option<&super::extensions::RequestContext> {
self.request_context.as_deref()
self.invocation
.as_ref()
.and_then(|origin| origin.extensions.request())
.or(self.request_context.as_deref())
}
#[must_use]
pub fn shared_request_context(&self) -> Option<Arc<super::extensions::RequestContext>> {
self.request_context.clone()
self.invocation
.as_ref()
.and_then(|origin| origin.extensions.shared_request())
.or_else(|| self.request_context.clone())
}
#[must_use]
@@ -79,6 +120,7 @@ impl ClientAuthenticationRequest {
.filter(|value| !value.is_empty())
.and_then(|value| PlaintextClientApiKey::new(value.to_owned()).ok());
Ok(Self {
invocation: None,
settings: None,
request_context: None,
authorization: authorization.into(),
@@ -1110,7 +1110,7 @@ where
pinned_account.clone(),
self.account_state_owner.clone(),
)
.with_account_scope(Arc::clone(self.plan.account_scope())),
.with_account_scope(Arc::clone(candidate.account_scope())),
}
.with_credential_recovery_attempted(pinned_account.as_ref().is_some_and(|account| {
self.credential_recovery_attempted_accounts
@@ -12,7 +12,6 @@ pub use contract::{
StartedExecution,
};
mod scope;
pub use scope::BoundModelExecutionContext;
mod service;
pub use failure::gateway_error_from_engine;
pub use service::DefaultExecutionService;
@@ -15,13 +15,10 @@ use crate::{
lifecycle::{CancellationToken, Deadline},
operation::Operation,
policy::ClientApiKeyId,
routing::{
FrozenAccountScope, PublicModelId, UpstreamModelId, request_settings::RequestSettings,
},
routing::{FrozenAccountScope, PublicModelId, UpstreamModelId},
};
use std::{
collections::BTreeMap,
fmt,
sync::{
Arc, Mutex, Weak,
atomic::{AtomicUsize, Ordering},
@@ -188,57 +185,6 @@ pub(super) struct ActiveRequestLease {
pub(super) authority: Arc<ActiveRequestAuthority>,
}
/// 管理页或 CLI 一次调用持有的显式模型执行身份
///
/// 结构不暴露 Key 明文;Runtime 只能把它原样交回 Core 发起模型请求
#[derive(Clone)]
pub struct BoundModelExecutionContext {
pub(super) authority: Arc<BoundModelExecutionAuthority>,
}
impl BoundModelExecutionContext {
/// 与实际执行共用绑定时的快照,不能回退到后续发布的配置
#[must_use]
pub fn request_body_limit(&self) -> usize {
self.authority
.client
.snapshot
.responses_max_decompressed_body_bytes()
}
#[must_use]
pub fn request_settings(&self) -> Option<&RequestSettings> {
self.authority.client.request_settings()
}
}
pub(super) struct BoundModelExecutionAuthority {
pub(super) client: AuthenticatedClient,
pub(super) account_scope: Arc<FrozenAccountScope>,
pub(super) cancellation: CancellationToken,
pub(super) extension_scope: ExtensionCallScope,
pub(super) initiating_plugin_instance_id: String,
}
impl fmt::Debug for BoundModelExecutionContext {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("BoundModelExecutionContext")
.field("key_id", &self.authority.client.policy.key_id())
.field(
"initiating_plugin_instance_id",
&self.authority.initiating_plugin_instance_id,
)
.finish_non_exhaustive()
}
}
impl Drop for BoundModelExecutionAuthority {
fn drop(&mut self) {
self.cancellation.cancel();
}
}
impl ActiveRequestLease {
pub(super) fn release(self) {
drop(self);
@@ -12,9 +12,8 @@ use super::contract::{
use super::probe_store::{ProbeObservation, TransientExecutionStore};
use super::scope::{
ActiveRequestAuthority, ActiveRequestLease, ActiveRequestRegistry, AuthorizedExecution,
BoundExecutionFacts, BoundModelExecutionAuthority, BoundModelExecutionContext,
ExecutionStartGuard, ExecutionTarget, NestedExecutionFacts, NestedExecutionGraph,
PendingStartExecution, PreparedExecutionStart,
BoundExecutionFacts, ExecutionStartGuard, ExecutionTarget, NestedExecutionFacts,
NestedExecutionGraph, PendingStartExecution, PreparedExecutionStart,
};
use super::{COORDINATION_TIMEOUT, DIAGNOSTIC_TIMEOUT};
use super::{scope::ExecutionAdmission, session::DefaultExecutionSession};
@@ -28,7 +27,7 @@ use crate::{
ClientAdmissionRequest,
},
authentication::{
ClientAuthenticationRequest, FrontendAuthenticationDecision,
ClientAuthenticationRequest, ClientIdentityPort, FrontendAuthenticationDecision,
FrontendAuthenticationExtensionIndex,
},
budget::{ClientBudgetCharge, ClientBudgetPort},
@@ -41,8 +40,8 @@ use crate::{
middleware::FrozenMiddlewarePlan,
nested::{
AffinityLookupPort, AffinityLookupRequest, AffinityLookupResult,
BoundModelExecutionBinding, BoundModelExecutionRequest, ExecutionEffects,
NestedModelExecutionPort, NestedModelExecutionRequest,
BoundModelExecutionRequest, ExecutionEffects, NestedModelExecutionPort,
NestedModelExecutionRequest,
},
observation::{
FrozenRequestObservationContext, RequestObservationDispatch, RequestObservationScope,
@@ -1081,72 +1080,54 @@ impl DefaultExecutionService {
})
}
async fn bind_bound_model_inner(
&self,
binding: BoundModelExecutionBinding,
) -> Result<BoundModelExecutionContext, GatewayError> {
let client = self
.client_for_key_id(&binding.client_key_id, binding.settings)
.await?;
if binding.extension_scope.len() >= ExtensionCallScope::MAXIMUM_DEPTH {
return Err(GatewayError::new(
GatewayErrorKind::PolicyDenied,
"plugin model execution depth exceeded",
));
}
let extension_scope = binding
.extension_scope
.extending(binding.initiating_plugin_instance_id.clone())
.ok_or_else(|| {
GatewayError::new(
GatewayErrorKind::Internal,
"bound plugin execution scope is invalid",
)
})?;
let account_scope = Arc::clone(client.policy.account_scope());
Ok(BoundModelExecutionContext {
authority: Arc::new(BoundModelExecutionAuthority {
client,
account_scope,
cancellation: binding.cancellation,
extension_scope,
initiating_plugin_instance_id: binding.initiating_plugin_instance_id,
}),
})
}
async fn start_bound_model_inner(
&self,
request: BoundModelExecutionRequest,
) -> Result<StartedExecution, GatewayError> {
let BoundModelExecutionRequest {
context,
client,
initiating_plugin_instance_id,
cancellation,
extension_scope,
public_model,
operation,
mut metadata,
provider,
account,
} = request;
let authority = &context.authority;
if authority.cancellation.is_cancelled() {
if cancellation.is_cancelled() {
return Err(GatewayError::new(
GatewayErrorKind::Cancelled,
"bound plugin invocation was cancelled",
));
}
let now = SystemTime::now();
let deadline_at = Deadline::from_timeout(now, authority.client.execution_timeout())
if extension_scope.len() >= ExtensionCallScope::MAXIMUM_DEPTH {
return Err(GatewayError::new(
GatewayErrorKind::PolicyDenied,
"plugin model execution depth exceeded",
));
}
let extension_scope = extension_scope
.extending(initiating_plugin_instance_id.clone())
.ok_or_else(|| {
GatewayError::new(
GatewayErrorKind::PolicyDenied,
"plugin model execution scope is recursive",
)
})?;
let now = SystemTime::now();
let deadline_at =
Deadline::from_timeout(now, client.execution_timeout()).ok_or_else(|| {
GatewayError::new(GatewayErrorKind::Internal, "system clock is invalid")
})?;
let (account_scope, required_provider) = restrict_model_execution_scope(
authority.account_scope.as_ref(),
client.policy.account_scope().as_ref(),
provider,
account.as_ref(),
None,
)?;
// 连接承载多个独立执行;递归预算和副作用只在本次执行的子调用图内共享
// 父连接的取消仍通过 authority 传播,不能因新建执行丢失扩展调用链
// 父调用的取消继续向子执行传播,不能因新建执行丢失扩展调用链
let graph = Arc::new(NestedExecutionGraph::new(Arc::new(
ExecutionEffects::default(),
)));
@@ -1154,12 +1135,12 @@ impl DefaultExecutionService {
let execution_effects_baseline = graph.effects.epoch();
metadata.endpoint = "host.model".to_owned();
metadata.transport = ClientTransport::InternalPlugin;
metadata.inherit_request_origin(&authority.extension_scope);
metadata.inherit_request_origin(&extension_scope);
self.client_api_key_usage
.record_used(authority.client.policy.key_id());
.record_used(client.policy.key_id());
self.start_authorized(
PendingStartExecution {
client: authority.client.clone(),
client,
target: ExecutionTarget::Model(public_model),
operation,
metadata,
@@ -1168,13 +1149,13 @@ impl DefaultExecutionService {
account_scope,
response_control: None,
deadline_at,
cancellation: authority.cancellation.child_token(),
extension_scope: authority.extension_scope.clone(),
cancellation: cancellation.child_token(),
extension_scope,
required_provider,
required_account: account,
nested: None,
bound: Some(BoundExecutionFacts {
initiating_plugin_instance_id: authority.initiating_plugin_instance_id.clone(),
initiating_plugin_instance_id,
}),
graph: Some(graph),
execution_effects_baseline,
@@ -1570,14 +1551,7 @@ impl ExecutionService for DefaultExecutionService {
.query_client_model_catalog(scope, protocol, client_version)
.await?
else {
for profile in client.snapshot.public_model_profiles_for_provider(kind) {
if !client.snapshot.catalog_model_allowed_for_scope(
kind,
profile.model(),
scope,
) {
continue;
}
for profile in client.snapshot.profiles_for_provider_scope(kind, scope) {
if seen.insert(profile.model().clone()) {
result.push(PublicModelDescriptor::Adapted(profile));
}
@@ -1629,6 +1603,12 @@ impl ExecutionService for DefaultExecutionService {
result.push(source.content.for_public_model(model));
}
}
// 原生完整对象优先;只补入当前授权账号实际提供的独立适配画像
for profile in client.snapshot.contributed_profiles_for_scope(kind, scope) {
if seen.insert(profile.model().clone()) {
result.push(PublicModelDescriptor::Adapted(profile));
}
}
}
Ok(result)
})
@@ -1724,23 +1704,57 @@ impl ExecutionService for DefaultExecutionService {
}
}
impl ClientIdentityPort for DefaultExecutionService {
fn resolve_client(
&self,
client_key_id: ClientApiKeyId,
settings: Option<RequestSettings>,
) -> BoxFuture<'_, Result<AuthenticatedClient, GatewayError>> {
Box::pin(async move { self.client_for_key_id(&client_key_id, settings).await })
}
fn verify_client(
&self,
request: ClientAuthenticationRequest,
) -> BoxFuture<'_, Result<AuthenticatedClient, GatewayError>> {
Box::pin(async move {
let cancellation = request
.invocation()
.map(|origin| origin.cancellation.clone());
let verification = self.verify_request(request).fuse();
pin_mut!(verification);
match cancellation {
Some(cancellation) => {
let cancelled = cancellation.cancelled().fuse();
pin_mut!(cancelled);
select_biased! {
() = cancelled => Err(GatewayError::new(GatewayErrorKind::Cancelled, "client verification was cancelled")),
result = verification => result.map_err(authentication_gateway_error),
}
}
None => verification.await.map_err(authentication_gateway_error),
}
})
}
}
impl NestedModelExecutionPort for DefaultExecutionService {
fn models(
&self,
context: BoundModelExecutionContext,
client: AuthenticatedClient,
cancellation: CancellationToken,
protocol: String,
client_version: String,
) -> BoxFuture<'_, Result<Vec<PublicModelId>, GatewayError>> {
Box::pin(async move {
let authority = &context.authority;
let cancellation = authority.cancellation.cancelled().fuse();
let cancelled = cancellation.cancelled().fuse();
let timeout = Delay::new(DIAGNOSTIC_TIMEOUT).fuse();
let catalog = self
.client_model_catalog(&authority.client, &protocol, &client_version)
.client_model_catalog(&client, &protocol, &client_version)
.fuse();
pin_mut!(cancellation, timeout, catalog);
pin_mut!(cancelled, timeout, catalog);
let models = select_biased! {
() = cancellation => return Err(GatewayError::new(GatewayErrorKind::Cancelled, "model catalog call was cancelled")),
() = cancelled => return Err(GatewayError::new(GatewayErrorKind::Cancelled, "model catalog call was cancelled")),
() = timeout => return Err(GatewayError::new(GatewayErrorKind::Timeout, "model catalog deadline elapsed")),
result = catalog => result.map_err(|_| GatewayError::new(GatewayErrorKind::Internal, "model catalog is unavailable"))?,
};
@@ -1754,13 +1768,6 @@ impl NestedModelExecutionPort for DefaultExecutionService {
})
}
fn bind(
&self,
binding: BoundModelExecutionBinding,
) -> BoxFuture<'_, Result<BoundModelExecutionContext, GatewayError>> {
Box::pin(async move { self.bind_bound_model_inner(binding).await })
}
fn start(
&self,
request: NestedModelExecutionRequest,
@@ -62,6 +62,17 @@ impl ExtensionCallScope {
}
}
/// 宿主签发的父调用来源;身份裁决、执行链和实际资源仍由各自 owner 持有
#[derive(Clone, Debug)]
pub struct InvocationOrigin {
pub request_id: String,
pub call_id: String,
pub cancellation: crate::lifecycle::CancellationToken,
pub extensions: ExtensionCallScope,
pub plan: Option<super::middleware::FrozenMiddlewarePlan>,
pub effects: Option<Arc<super::nested::ExecutionEffects>>,
}
/// 同一代次的具体执行计划;不拥有发布状态,运行资源由对应发布集合保活
#[derive(Debug, Default)]
pub struct ExecutionExtensionPlans {
@@ -167,7 +167,7 @@ impl AccountAttemptContext {
}
}
/// 附着普通请求认证时冻结的账号范围
/// 附着本次尝试的冻结账号范围,包含路由候选的能力限制
#[must_use]
pub fn with_account_scope(
mut self,
@@ -895,7 +895,7 @@ impl AttemptContext {
self.account.is_diagnostic_required_account()
}
/// 普通请求认证时冻结的账号范围;管理端诊断为 `None`
/// 当前路由候选的冻结账号范围;管理端诊断为 `None`
#[must_use]
pub const fn account_scope(&self) -> Option<&Arc<crate::account::scope::FrozenAccountScope>> {
self.account.account_scope()
@@ -13,13 +13,13 @@ use futures::future::BoxFuture;
use crate::{
account::ProviderAccountId, error::GatewayError, identity::ProviderKind,
lifecycle::CancellationToken, operation::Operation, policy::ClientApiKeyId,
provider_ports::ProviderSessionAffinityKey, routing::PublicModelId,
lifecycle::CancellationToken, operation::Operation, provider_ports::ProviderSessionAffinityKey,
routing::PublicModelId,
};
use super::{
ModelRequestId,
execution::{BoundModelExecutionContext, ExecutionRequestMetadata, StartedExecution},
execution::{AuthenticatedClient, ExecutionRequestMetadata, StartedExecution},
};
/// 同一请求图内不可逆外部副作用的单调水位
@@ -59,37 +59,14 @@ pub struct NestedModelExecutionRequest {
pub parent_account: Option<ProviderAccountId>,
}
/// 插件为当前调用或连接显式绑定的模型执行身份
///
/// Key ID 由插件选择,发起实例由宿主确定
/// Core 在绑定时冻结当前
/// Key 策略;每个实际模型请求仍独立经过准入、预算、账本与计费
pub struct BoundModelExecutionBinding {
pub settings: Option<crate::routing::request_settings::RequestSettings>,
pub client_key_id: ClientApiKeyId,
pub initiating_plugin_instance_id: String,
/// 身份随父调用或连接回收;每个模型请求使用独立的执行期限
pub cancellation: CancellationToken,
/// 宿主保留的扩展调用链,异步观察不能通过新建模型请求清空防递归事实
pub extension_scope: super::extensions::ExtensionCallScope,
}
impl fmt::Debug for BoundModelExecutionBinding {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("BoundModelExecutionBinding")
.field("client_key_id", &self.client_key_id)
.field(
"initiating_plugin_instance_id",
&self.initiating_plugin_instance_id,
)
.finish_non_exhaustive()
}
}
/// 显式身份作用域内的一次模型请求
///
/// 身份与权限可先用于查询;执行链只在真正启动模型请求时建立
pub struct BoundModelExecutionRequest {
pub context: BoundModelExecutionContext,
pub client: AuthenticatedClient,
pub initiating_plugin_instance_id: String,
pub cancellation: CancellationToken,
pub extension_scope: super::extensions::ExtensionCallScope,
pub public_model: PublicModelId,
pub operation: Operation,
pub metadata: ExecutionRequestMetadata,
@@ -101,7 +78,11 @@ impl fmt::Debug for BoundModelExecutionRequest {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter
.debug_struct("BoundModelExecutionRequest")
.field("context", &self.context)
.field("client", &self.client)
.field(
"initiating_plugin_instance_id",
&self.initiating_plugin_instance_id,
)
.field("public_model", &self.public_model)
.field("operation", &self.operation.kind())
.field("provider", &self.provider)
@@ -132,17 +113,12 @@ impl fmt::Debug for NestedModelExecutionRequest {
pub trait NestedModelExecutionPort: Send + Sync {
fn models(
&self,
context: BoundModelExecutionContext,
client: AuthenticatedClient,
cancellation: CancellationToken,
protocol: String,
client_version: String,
) -> BoxFuture<'_, Result<Vec<PublicModelId>, GatewayError>>;
/// 为插件调用冻结显式执行身份;取消父调用会取消其所有模型执行
fn bind(
&self,
binding: BoundModelExecutionBinding,
) -> BoxFuture<'_, Result<BoundModelExecutionContext, GatewayError>>;
fn start(
&self,
request: NestedModelExecutionRequest,
+17
View File
@@ -34,6 +34,7 @@ use engine::ExecutionStore;
use engine::admission::{
ClientAdmissionPort, ClientAdmissionRecoveryPort, restore_client_admission_startup,
};
use engine::authentication::ClientIdentityPort;
use engine::continuation::NativeContinuationPort;
use engine::execution::{
ClientApiKeyUsageSink, ClientKeyVerifier, DefaultExecutionService, ExecutionService,
@@ -127,6 +128,7 @@ pub struct CoreBundle {
client_key_verifier: Arc<dyn ClientKeyVerifier>,
snapshot_control: Arc<dyn SnapshotControl>,
account_probe: Arc<dyn AccountProbe>,
client_identity: Arc<dyn ClientIdentityPort>,
nested_model_execution: Arc<dyn NestedModelExecutionPort>,
affinity_lookup: Arc<dyn AffinityLookupPort>,
health_probes: Vec<Arc<dyn HealthProbe>>,
@@ -181,6 +183,7 @@ impl CoreStartup {
let execution: Arc<dyn ExecutionService> = service.clone();
let client_key_verifier: Arc<dyn ClientKeyVerifier> = service.clone();
let account_probe: Arc<dyn AccountProbe> = service.clone();
let client_identity: Arc<dyn ClientIdentityPort> = service.clone();
let nested_model_execution: Arc<dyn NestedModelExecutionPort> = service.clone();
let affinity_lookup: Arc<dyn AffinityLookupPort> = service;
let snapshot_control: Arc<dyn SnapshotControl> = publisher;
@@ -191,6 +194,7 @@ impl CoreStartup {
client_key_verifier,
snapshot_control,
account_probe,
client_identity,
nested_model_execution,
affinity_lookup,
health_probes,
@@ -266,6 +270,11 @@ impl CoreBundle {
Arc::clone(&self.account_probe)
}
#[must_use]
pub fn client_identity_port(&self) -> Arc<dyn ClientIdentityPort> {
Arc::clone(&self.client_identity)
}
#[must_use]
pub fn nested_model_execution_port(&self) -> Arc<dyn NestedModelExecutionPort> {
Arc::clone(&self.nested_model_execution)
@@ -395,6 +404,7 @@ pub struct CoreCommandPlaneStartup {
pub struct CoreCommandPlaneBundle {
control: CoreControlPlaneBundle,
account_probe: Arc<dyn AccountProbe>,
client_identity: Arc<dyn ClientIdentityPort>,
nested_model_execution: Arc<dyn NestedModelExecutionPort>,
affinity_lookup: Arc<dyn AffinityLookupPort>,
}
@@ -425,11 +435,13 @@ impl CoreCommandPlaneStartup {
.with_snapshot_refresh(Arc::clone(&control.publisher)),
);
let account_probe: Arc<dyn AccountProbe> = service.clone();
let client_identity: Arc<dyn ClientIdentityPort> = service.clone();
let nested_model_execution: Arc<dyn NestedModelExecutionPort> = service.clone();
let affinity_lookup: Arc<dyn AffinityLookupPort> = service;
Ok(CoreCommandPlaneBundle {
control,
account_probe,
client_identity,
nested_model_execution,
affinity_lookup,
})
@@ -452,6 +464,11 @@ impl CoreCommandPlaneBundle {
self.control.snapshot_control()
}
#[must_use]
pub fn client_identity_port(&self) -> Arc<dyn ClientIdentityPort> {
Arc::clone(&self.client_identity)
}
#[must_use]
pub fn nested_model_execution_port(&self) -> Arc<dyn NestedModelExecutionPort> {
Arc::clone(&self.nested_model_execution)
@@ -21,6 +21,17 @@ pub struct ContributedModelAlias {
pub target: UpstreamModelId,
}
/// 适配模型的冻结账号支持事实;没有选择账号或改写请求的能力
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ContributedModel {
pub owner: String,
pub id: PublicModelId,
pub provider: ProviderKind,
pub accounts: BTreeSet<ProviderAccountId>,
pub capabilities: ModelCapabilities,
pub presentation: ModelPresentation,
}
/// Provider 客户端目录条目;缺少原生协议正文时使用已编译的通用画像
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ProviderModelDescriptor {
@@ -4,7 +4,7 @@ use std::{fmt, sync::Arc};
use futures::future::BoxFuture;
use crate::routing::{ConfigRevision, ContributedModelAlias};
use crate::routing::{ConfigRevision, ContributedModel, ContributedModelAlias};
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct ExtensionSetId(String);
@@ -37,6 +37,10 @@ pub trait ExtensionSetLease: Send + Sync {
&[]
}
fn models(&self) -> &[ContributedModel] {
&[]
}
/// 局部故障已有请求级处理计划时,集合仍可提供服务
fn can_serve(&self) -> bool {
self.is_ready()
@@ -71,6 +75,11 @@ impl ExtensionSetReference {
pub fn model_aliases(&self) -> &[ContributedModelAlias] {
self.lease.model_aliases()
}
#[must_use]
pub fn models(&self) -> &[ContributedModel] {
self.lease.models()
}
}
impl fmt::Debug for ExtensionSetReference {
@@ -11,7 +11,7 @@ pub use crate::account::scope::{
};
pub use crate::identity::ProviderKind;
pub use catalog::{
ContributedModelAlias, ProviderCatalogGeneration, ProviderCatalogPort,
ContributedModel, ContributedModelAlias, ProviderCatalogGeneration, ProviderCatalogPort,
ProviderCatalogUnavailable, ProviderModelCapabilities, ProviderModelContent,
ProviderModelDescriptor, PublicModelDescriptor,
};
@@ -652,6 +652,30 @@ impl RuntimeSnapshot {
.map_or(&[], |set| set.model_aliases())
}
fn contributed_models(&self) -> &[super::ContributedModel] {
self.extensions.as_ref().map_or(&[], |set| set.models())
}
pub(crate) fn contributed_profiles_for_scope(
&self,
provider: &ProviderKind,
scope: &FrozenAccountScope,
) -> Vec<super::PublicModelProfile> {
self.contributed_models()
.iter()
.filter(|model| {
&model.provider == provider
&& model.accounts.iter().any(|account| {
scope.account_provider(account) == Some(provider)
&& scope.allows_model(account, &self.mapped_model(model.id.as_str()))
})
})
.map(|model| {
super::PublicModelProfile::new(model.id.clone(), model.presentation.clone())
})
.collect()
}
fn model_alias(&self, model: &str) -> Option<&super::ContributedModelAlias> {
self.model_aliases()
.iter()
@@ -844,6 +868,12 @@ impl RuntimeSnapshot {
.filter(|alias| &alias.provider == provider)
.map(|alias| alias.id.clone()),
);
models.extend(
self.contributed_models()
.iter()
.filter(|model| &model.provider == provider)
.map(|model| model.id.clone()),
);
models.into_iter().collect()
}
@@ -922,10 +952,7 @@ impl RuntimeSnapshot {
) -> Vec<super::PublicModelProfile> {
let mut profiles = BTreeMap::new();
for provider in scope.provider_kinds() {
for profile in self.public_model_profiles_for_provider(provider) {
if !self.catalog_model_allowed_for_scope(provider, profile.model(), scope) {
continue;
}
for profile in self.profiles_for_provider_scope(provider, scope) {
profiles
.entry(profile.model().clone())
.or_insert_with(|| profile.presentation().clone());
@@ -937,6 +964,20 @@ impl RuntimeSnapshot {
.collect()
}
pub(crate) fn profiles_for_provider_scope(
&self,
provider: &ProviderKind,
scope: &FrozenAccountScope,
) -> Vec<super::PublicModelProfile> {
self.public_model_profiles_for_provider(provider)
.into_iter()
.filter(|profile| {
self.native_catalog_model_allowed_for_scope(provider, profile.model(), scope)
})
.chain(self.contributed_profiles_for_scope(provider, scope))
.collect()
}
/// 已知模型只属于明确声明它的 Provider;未知模型仍可由非完整目录透传
#[must_use]
pub fn contains_public_model_for_provider(
@@ -961,7 +1002,15 @@ impl RuntimeSnapshot {
self.provider_models
.get(provider)
.is_some_and(|models| models.contains_key(&upstream_model))
|| self
.contributed_models()
.iter()
.any(|model| &model.provider == provider && &model.id == public_model)
|| (!self.exhaustive_provider_catalogs.contains(provider)
&& !self
.contributed_models()
.iter()
.any(|model| &model.id == public_model)
&& !self.is_known_upstream_model(&upstream_model))
}
@@ -969,6 +1018,10 @@ impl RuntimeSnapshot {
self.provider_models
.values()
.any(|models| models.contains_key(upstream_model))
|| self
.contributed_models()
.iter()
.any(|model| model.id.as_str() == upstream_model.as_str())
}
#[must_use]
@@ -988,6 +1041,24 @@ impl RuntimeSnapshot {
provider: &ProviderKind,
public_model: &PublicModelId,
scope: &FrozenAccountScope,
) -> bool {
self.native_catalog_model_allowed_for_scope(provider, public_model, scope)
|| self.contributed_models().iter().any(|model| {
&model.provider == provider
&& &model.id == public_model
&& model.accounts.iter().any(|account| {
scope.account_provider(account) == Some(provider)
&& scope
.allows_model(account, &self.mapped_model(public_model.as_str()))
})
})
}
fn native_catalog_model_allowed_for_scope(
&self,
provider: &ProviderKind,
public_model: &PublicModelId,
scope: &FrozenAccountScope,
) -> bool {
let upstream_model = self.mapped_model(public_model.as_str());
match self
@@ -999,7 +1070,17 @@ impl RuntimeSnapshot {
scope.account_provider(account) == Some(provider)
&& scope.allows_model(account, &upstream_model)
}),
None => scope.allows_provider_model(provider, &upstream_model),
None => {
// 只有插件声明的模型不能借非完整原生目录扩大到其他账号
let native = self.provider_models.get(provider).is_some_and(|models| {
models.keys().any(|model| model.as_str() == upstream_model)
});
let contributed = self
.contributed_models()
.iter()
.any(|model| &model.provider == provider && &model.id == public_model);
(native || !contributed) && scope.allows_provider_model(provider, &upstream_model)
}
}
}
@@ -1121,7 +1202,11 @@ impl RuntimeSnapshot {
}
.map_err(|_| RoutingError::InvalidIdentifier)?;
// 归属先于账号范围、能力与路由约束判定,原 Provider 被排除不能变成跨平台透传
let known_model = self.is_known_upstream_model(&upstream_model);
let known_model = self.is_known_upstream_model(&upstream_model)
|| self
.contributed_models()
.iter()
.any(|model| &model.id == public_model);
let providers = context.required_provider.as_ref().map_or_else(
|| account_scope.provider_kinds().clone(),
@@ -1155,17 +1240,64 @@ impl RuntimeSnapshot {
.provider_models
.get(provider)
.and_then(|models| models.get(&upstream_model));
let emulated_features = match capabilities {
Some(capabilities) => {
let Some(emulated) = capabilities.match_requirements(&requirements) else {
let contributions: Vec<_> = self
.contributed_models()
.iter()
.filter(|model| &model.provider == provider && &model.id == public_model)
.collect();
let mut candidate_scope = Arc::clone(&account_scope);
let emulated_features = if contributions.is_empty() {
match capabilities {
Some(capabilities) => {
let Some(emulated) = capabilities.match_requirements(&requirements) else {
continue;
};
emulated
}
None if known_model || self.exhaustive_provider_catalogs.contains(provider) => {
continue;
};
emulated
}
None => BTreeSet::new(),
}
None if known_model || self.exhaustive_provider_catalogs.contains(provider) => {
} else {
let native = capabilities.and_then(|value| value.match_requirements(&requirements));
let capable_accounts: BTreeSet<_> = contributions
.iter()
.filter(|model| {
model
.capabilities
.match_requirements(&requirements)
.is_some()
})
.flat_map(|model| model.accounts.iter())
.filter(|account| {
account_scope.account_provider(account) == Some(provider)
&& account_scope.allows_model(account, upstream_model.as_str())
})
.cloned()
.collect();
if native.is_some() {
// 这些账号实际由适配器执行,不能借用同名原生模型的能力
let mut restricted = account_scope.as_ref().clone();
for account in contributions.iter().flat_map(|model| &model.accounts) {
if !capable_accounts.contains(account) {
restricted = restricted.excluding_account(account.clone());
}
}
candidate_scope = Arc::new(restricted);
} else {
// restricted_to 的空集合表示不收窄,因此必须先拒绝空候选
if capable_accounts.is_empty() {
continue;
}
candidate_scope = Arc::new(
account_scope.restricted_to(&Default::default(), &capable_accounts),
);
}
if !candidate_scope.allows_provider_model(provider, upstream_model.as_str()) {
continue;
}
None => BTreeSet::new(),
native.unwrap_or_default()
};
candidates.push(ProviderCandidate {
provider: provider.clone(),
@@ -1173,10 +1305,15 @@ impl RuntimeSnapshot {
.provider_model_presentations
.get(provider)
.and_then(|models| models.get(&upstream_model))
.cloned(),
.cloned()
.or_else(|| {
contributions
.first()
.map(|model| model.presentation.clone())
}),
upstream_model: Some(upstream_model.clone()),
emulated_features,
account_scope: Arc::clone(&account_scope),
account_scope: candidate_scope,
});
}
@@ -0,0 +1,270 @@
//! 身份、冻结权限与查询不建立模型执行图;实际执行才检查调用链并占用执行资源
use super::*;
use gateway_core::engine::{
extensions::{ExtensionCallScope, InvocationOrigin},
nested::{BoundModelExecutionRequest, NestedModelExecutionPort},
};
use gateway_core::lifecycle::CancellationToken;
#[test]
fn identity_queries_keep_frozen_permissions_without_using_execution_resources() {
block_on(async {
let snapshots = RuntimeSnapshotHandle::new(start_snapshot());
let store = Arc::new(TrackingExecutionStore::default());
let usage = Arc::new(RecordingClientApiKeyUsage::default());
let service = DefaultExecutionService::new(
snapshots.clone(),
store.clone(),
ProviderRegistry::new([Arc::new(NativeCatalogProvider {
fail: false,
empty: false,
}) as Arc<dyn Provider>])
.unwrap(),
Arc::new(UnusedAdmissions),
Arc::new(UnusedContinuation),
usage.clone(),
Arc::new(crate::RecordingDiagnostics::default()),
);
let settings = service.request_settings().unwrap();
let key = ClientApiKeyId::new("key_start_test").unwrap();
let client =
ClientIdentityPort::resolve_client(&service, key.clone(), Some(settings.clone()))
.await
.unwrap();
snapshots.publish(start_snapshot_with_policy(
2,
false,
RateLimits::unlimited(),
FastMode::Disabled,
));
assert!(
client
.policy()
.account_scope()
.allows(&ProviderAccountId::new("acct_start").unwrap(),)
);
assert!(
!client
.policy()
.account_scope()
.allows(&ProviderAccountId::new("acct_missing").unwrap(),)
);
let models = NestedModelExecutionPort::models(
&service,
client.clone(),
CancellationToken::default(),
"codex".into(),
"0.154.0".into(),
)
.await
.unwrap();
assert_eq!(models, vec![PublicModelId::new("gpt-start").unwrap()]);
let frozen = ClientIdentityPort::resolve_client(&service, key.clone(), Some(settings))
.await
.unwrap();
assert_eq!(frozen.snapshot().revision().get(), 1);
let current = ClientIdentityPort::resolve_client(&service, key, None).await;
assert!(matches!(current, Err(error) if error.kind() == GatewayErrorKind::Unauthorized));
let cancellation = CancellationToken::default();
cancellation.cancel();
let cancelled = NestedModelExecutionPort::models(
&service,
client,
cancellation,
"codex".into(),
"0.154.0".into(),
)
.await;
assert!(matches!(cancelled, Err(error) if error.kind() == GatewayErrorKind::Cancelled));
assert!(usage.recorded().is_empty());
assert!(!store.touched.load(Ordering::SeqCst));
});
}
#[test]
fn execution_scope_limits_apply_only_when_starting_a_model() {
block_on(async {
let store = Arc::new(TrackingExecutionStore::default());
let usage = Arc::new(RecordingClientApiKeyUsage::default());
let service = DefaultExecutionService::new(
RuntimeSnapshotHandle::new(start_snapshot()),
store.clone(),
ProviderRegistry::new([Arc::new(NativeCatalogProvider {
fail: false,
empty: false,
}) as Arc<dyn Provider>])
.unwrap(),
Arc::new(UnusedAdmissions),
Arc::new(UnusedContinuation),
usage.clone(),
Arc::new(crate::RecordingDiagnostics::default()),
);
let repeated = ExtensionCallScope::default()
.extending("entry".into())
.unwrap();
let exhausted = (0..ExtensionCallScope::MAXIMUM_DEPTH)
.fold(ExtensionCallScope::default(), |scope, index| {
scope.extending(format!("ancestor-{index}")).unwrap()
});
for extension_scope in [repeated, exhausted] {
let client = ClientIdentityPort::resolve_client(
&service,
ClientApiKeyId::new("key_start_test").unwrap(),
None,
)
.await
.unwrap();
let result = NestedModelExecutionPort::start_bound(
&service,
BoundModelExecutionRequest {
client: client.clone(),
initiating_plugin_instance_id: "entry".into(),
cancellation: CancellationToken::default(),
extension_scope,
public_model: PublicModelId::new("gpt-start").unwrap(),
operation: start_operation(),
metadata: ExecutionRequestMetadata {
protocol: "openai".into(),
endpoint: "host.model".into(),
transport: ClientTransport::InternalPlugin,
stream: false,
client_ip: None,
user_agent: None,
previous_response_id: None,
},
provider: None,
account: None,
},
)
.await;
assert!(matches!(result, Err(error) if error.kind() == GatewayErrorKind::PolicyDenied));
let models = NestedModelExecutionPort::models(
&service,
client,
CancellationToken::default(),
"codex".into(),
"0.154.0".into(),
)
.await
.unwrap();
assert_eq!(models, vec![PublicModelId::new("gpt-start").unwrap()]);
}
assert!(usage.recorded().is_empty());
assert!(!store.touched.load(Ordering::SeqCst));
});
}
struct ParentAwareAuthentication {
calls: AtomicUsize,
mapped: bool,
}
impl FrontendAuthenticationPlan for ParentAwareAuthentication {
fn authenticate<'a>(
&'a self,
request: &'a ClientAuthenticationRequest,
) -> BoxFuture<'a, Result<FrontendAuthenticationDecision, FrontendAuthenticationError>> {
Box::pin(async move {
self.calls.fetch_add(1, Ordering::SeqCst);
let origin = request.invocation().expect("parent invocation");
assert_eq!(origin.request_id, "http-request");
assert_eq!(origin.call_id, "http-call");
assert!(origin.extensions.contains("authentication-plugin"));
assert!(origin.effects.is_some());
assert!(!origin.cancellation.is_cancelled());
assert_eq!(request.request_context().unwrap().uri, "/plugin/proxy");
assert!(request.settings().is_some());
Ok(FrontendAuthenticationDecision::Authenticated {
principal: "external-principal".into(),
})
})
}
fn client_key_id(&self, principal: &str) -> Option<ClientApiKeyId> {
assert_eq!(principal, "external-principal");
self.mapped
.then(|| ClientApiKeyId::new("key_start_test").unwrap())
}
fn exclusive(&self) -> bool {
true
}
}
#[test]
fn verification_keeps_parent_origin_and_requires_the_frontend_principal_mapping() {
block_on(async {
for mapped in [true, false] {
let authentication = FrontendAuthenticationExtensionIndex::default();
let plan = Arc::new(ParentAwareAuthentication {
calls: AtomicUsize::new(0),
mapped,
});
let snapshot = start_snapshot().with_extensions(Some(authentication_generation(
&authentication,
"identity-verification",
plan.clone(),
)));
let store = Arc::new(TrackingExecutionStore::default());
let usage = Arc::new(RecordingClientApiKeyUsage::default());
let service = DefaultExecutionService::new(
RuntimeSnapshotHandle::new(snapshot),
store.clone(),
ProviderRegistry::default(),
Arc::new(UnusedAdmissions),
Arc::new(UnusedContinuation),
usage.clone(),
Arc::new(crate::RecordingDiagnostics::default()),
)
.with_frontend_authentication(authentication);
let settings = service.request_settings().unwrap();
let origin = InvocationOrigin {
request_id: "http-request".into(),
call_id: "http-call".into(),
cancellation: CancellationToken::default(),
extensions: plugin_origin_scope()
.extending("authentication-plugin".into())
.unwrap(),
plan: None,
effects: Some(Arc::new(
gateway_core::engine::nested::ExecutionEffects::default(),
)),
};
let mut cancelled_origin = origin.clone();
cancelled_origin.cancellation = CancellationToken::default();
cancelled_origin.cancellation.cancel();
let cancelled = ClientIdentityPort::verify_client(
&service,
ClientAuthenticationRequest::new("External fixture")
.unwrap()
.with_invocation(cancelled_origin)
.with_settings(settings.clone()),
)
.await;
assert!(matches!(cancelled, Err(error) if error.kind() == GatewayErrorKind::Cancelled));
assert_eq!(plan.calls.load(Ordering::SeqCst), 0);
let result = ClientIdentityPort::verify_client(
&service,
ClientAuthenticationRequest::new("External fixture")
.unwrap()
.with_invocation(origin)
.with_settings(settings.clone()),
)
.await;
if mapped {
let client = result.unwrap();
assert_eq!(client.policy().key_id().as_str(), "key_start_test");
assert_eq!(client.request_settings(), Some(&settings));
} else {
assert!(
matches!(result, Err(error) if error.kind() == GatewayErrorKind::Unauthorized)
);
}
assert_eq!(plan.calls.load(Ordering::SeqCst), 1);
assert!(usage.recorded().is_empty());
assert!(!store.touched.load(Ordering::SeqCst));
}
});
}
@@ -1,5 +1,6 @@
//! 执行服务的准入、预算、请求记录与终结行为测试
mod identity;
mod outbound;
use gateway_core::routing::extensions::{ExtensionSetId, ExtensionSetLease, ExtensionSetReference};
@@ -12,6 +13,7 @@ use std::{
time::{Duration, Instant, SystemTime},
};
use gateway_core::engine::authentication::ClientIdentityPort;
use gateway_core::engine::budget::{ClientBudgetCharge, ClientBudgetError, ClientBudgetPort};
use gateway_core::error::GatewayError;
@@ -1215,9 +1217,7 @@ fn plugin_origin_scope() -> gateway_core::engine::extensions::ExtensionCallScope
#[test]
fn explicit_key_model_execution_preserves_http_origin_and_leaves_background_origin_unknown() {
use gateway_core::engine::nested::{
BoundModelExecutionBinding, BoundModelExecutionRequest, NestedModelExecutionPort,
};
use gateway_core::engine::nested::{BoundModelExecutionRequest, NestedModelExecutionPort};
block_on(async {
for has_origin in [true, false] {
let store = Arc::new(TrackingExecutionStore::default());
@@ -1234,11 +1234,17 @@ fn explicit_key_model_execution_preserves_http_origin_and_leaves_background_orig
Arc::new(crate::RecordingDiagnostics::default()),
);
let input = request(&service, ClientTransport::InternalPlugin);
let context = NestedModelExecutionPort::bind(
let client = ClientIdentityPort::resolve_client(
&service,
BoundModelExecutionBinding {
settings: None,
client_key_id: ClientApiKeyId::new("key_start_test").unwrap(),
ClientApiKeyId::new("key_start_test").unwrap(),
None,
)
.await
.unwrap();
let mut started = NestedModelExecutionPort::start_bound(
&service,
BoundModelExecutionRequest {
client,
initiating_plugin_instance_id: "origin-fixture".into(),
cancellation: Default::default(),
extension_scope: if has_origin {
@@ -1246,14 +1252,6 @@ fn explicit_key_model_execution_preserves_http_origin_and_leaves_background_orig
} else {
Default::default()
},
},
)
.await
.unwrap();
let mut started = NestedModelExecutionPort::start_bound(
&service,
BoundModelExecutionRequest {
context: context.clone(),
public_model: input.public_model,
operation: input.operation,
metadata: input.metadata,
@@ -3304,6 +3302,7 @@ struct FailingProvider;
struct NativeCatalogProvider {
fail: bool,
empty: bool,
}
#[async_trait]
@@ -3334,6 +3333,9 @@ impl Provider for NativeCatalogProvider {
if self.fail {
return Err(gateway_core::routing::ProviderCatalogUnavailable);
}
if self.empty {
return Ok(Some(Vec::new()));
}
Ok(Some(
["gpt-new", "gpt-start"]
.into_iter()
@@ -3364,7 +3366,10 @@ impl Provider for NativeCatalogProvider {
#[test]
fn client_catalog_forwards_scope_maps_whole_objects_and_omits_unroutable_models() {
#[derive(Debug)]
struct Aliases(Vec<gateway_core::routing::ContributedModelAlias>);
struct Aliases(
Vec<gateway_core::routing::ContributedModelAlias>,
Vec<gateway_core::routing::ContributedModel>,
);
impl gateway_core::routing::extensions::ExtensionSetLease for Aliases {
fn is_ready(&self) -> bool {
true
@@ -3372,8 +3377,29 @@ fn client_catalog_forwards_scope_maps_whole_objects_and_omits_unroutable_models(
fn model_aliases(&self) -> &[gateway_core::routing::ContributedModelAlias] {
&self.0
}
fn models(&self) -> &[gateway_core::routing::ContributedModel] {
&self.1
}
}
for fail in [false, true] {
for (fail, empty) in [(false, false), (false, true), (true, false)] {
let contributed = [
("gpt-start", "acct_start"),
("adapted", "acct_start"),
("private-model", "acct_other"),
]
.into_iter()
.map(|(id, account)| gateway_core::routing::ContributedModel {
owner: "catalog-plugin".into(),
id: PublicModelId::new(id).unwrap(),
provider: ProviderKind::new("openai").unwrap(),
accounts: [ProviderAccountId::new(account).unwrap()].into(),
capabilities: ModelCapabilities::new([OperationKind::Generate].into(), None),
presentation: gateway_core::routing::ModelPresentation::new(
Some(format!("Plugin {id}")),
None,
),
})
.collect();
let snapshot = start_snapshot();
let settings = snapshot
.settings()
@@ -3388,20 +3414,23 @@ fn client_catalog_forwards_scope_maps_whole_objects_and_omits_unroutable_models(
.with_extensions(Some(ExtensionSetReference::new(
gateway_core::routing::extensions::ExtensionSetId::new("catalog-aliases".into())
.unwrap(),
Arc::new(Aliases(vec![
gateway_core::routing::ContributedModelAlias {
Arc::new(Aliases(
vec![gateway_core::routing::ContributedModelAlias {
owner: "catalog-plugin".into(),
id: PublicModelId::new("plugin-alias").unwrap(),
provider: ProviderKind::new("openai").unwrap(),
target: UpstreamModelId::new("gpt-start").unwrap(),
},
])),
}],
contributed,
)),
)));
let service = DefaultExecutionService::new(
RuntimeSnapshotHandle::new(snapshot),
Arc::new(TrackingExecutionStore::default()),
ProviderRegistry::new([Arc::new(NativeCatalogProvider { fail }) as Arc<dyn Provider>])
.expect("registry"),
ProviderRegistry::new([
Arc::new(NativeCatalogProvider { fail, empty }) as Arc<dyn Provider>
])
.expect("registry"),
Arc::new(UnusedAdmissions),
Arc::new(UnusedContinuation),
Arc::new(RecordingClientApiKeyUsage::default()),
@@ -3417,8 +3446,35 @@ fn client_catalog_forwards_scope_maps_whole_objects_and_omits_unroutable_models(
continue;
}
let models = result.expect("models");
let adapted = models
.iter()
.filter_map(|entry| match entry {
gateway_core::routing::PublicModelDescriptor::Adapted(profile) => {
Some(profile.model().as_str())
}
_ => None,
})
.collect::<Vec<_>>();
assert_eq!(
adapted,
if empty {
vec!["gpt-start", "adapted"]
} else {
vec!["adapted"]
}
);
if empty {
assert_eq!(models.len(), 2);
continue;
}
let pairs = models
.iter()
.filter(|entry| {
matches!(
entry,
gateway_core::routing::PublicModelDescriptor::Native { .. }
)
})
.map(|entry| {
let gateway_core::routing::PublicModelDescriptor::Native { model, payload } = entry
else {
@@ -99,6 +99,247 @@ struct CountingExtensionPreparation {
struct AliasLease(Vec<ContributedModelAlias>);
struct ModelLease(Vec<gateway_core::routing::ContributedModel>);
#[test]
fn same_name_native_and_adapted_sources_keep_account_capabilities_separate() {
use gateway_core::{
account::{AccountModelAccess, AccountModelAccessMode, ProviderAccountId},
operation::{Feature, GenerateRequest, Operation, ProtocolPayload},
routing::{ContributedModel, SupportLevel},
};
struct Catalog;
impl ProviderCatalogPort for Catalog {
fn catalog_generations(&self) -> BTreeMap<ProviderKind, ProviderCatalogGeneration> {
[(
ProviderKind::new("alpha").unwrap(),
ProviderCatalogGeneration::new(1),
)]
.into()
}
fn query_model_capabilities(
&self,
_: &ProviderKind,
) -> BoxFuture<'_, Result<Vec<ProviderModelCapabilities>, ProviderCatalogUnavailable>>
{
Box::pin(async {
Ok(vec![
ProviderModelCapabilities::new(
UpstreamModelId::new("shared-model").unwrap(),
super::capabilities().with_feature(Feature::Tools, SupportLevel::Native),
)
.with_catalog_accounts([ProviderAccountId::new("acct_a").unwrap()].into())
.with_presentation(ModelPresentation::new(Some("Native".into()), None)),
])
})
}
}
let id = PublicModelId::new("shared-model").unwrap();
let b = ProviderAccountId::new("acct_b").unwrap();
let mut contribution = ContributedModel {
owner: "fixture".into(),
id: id.clone(),
provider: ProviderKind::new("alpha").unwrap(),
accounts: [b.clone()].into(),
capabilities: super::capabilities(),
presentation: ModelPresentation::new(Some("Adapted".into()), None),
};
let facts = SnapshotFacts::new(
revision(1),
revision(1),
super::settings(),
vec![],
vec![],
["acct_a", "acct_b", "acct_c"]
.into_iter()
.map(|id| {
SnapshotProviderAccountFacts::new(ProviderAccountId::new(id).unwrap(), "alpha")
.with_model_access(
AccountModelAccess::new(
AccountModelAccessMode::Allowlist,
vec![
if id == "acct_c" {
"other"
} else {
"shared-model"
}
.into(),
],
)
.unwrap(),
)
})
.collect(),
vec![],
);
let native = block_on(
RuntimeSnapshotCompiler::new(
Arc::new(TestSnapshotStore::new(Ok(facts))),
Arc::new(Catalog),
)
.compile(),
)
.unwrap();
let scope = native.all_account_scope();
let b_only = Arc::new(scope.restricted_to(&Default::default(), &[b.clone()].into()));
assert!(native.public_models_for_scope(&b_only).is_empty());
let snapshot = native.with_extensions(Some(ExtensionSetReference::new(
ExtensionSetId::new("adapted".into()).unwrap(),
Arc::new(ModelLease(vec![contribution.clone()])),
)));
assert_eq!(snapshot.public_models_for_scope(&b_only), vec![id.clone()]);
assert_eq!(
snapshot.public_model_profiles_for_scope(&b_only)[0].presentation(),
&contribution.presentation
);
let plan = snapshot
.plan(
&id,
&super::operation(),
b_only.clone(),
&Default::default(),
)
.unwrap();
assert!(plan.candidates()[0].account_scope().allows(&b));
let tools = Operation::Generate(GenerateRequest::from_protocol_payload(
ProtocolPayload::json_object(
"openai",
serde_json::json!({"model":"shared-model","tools":[{"type":"function","name":"test"}]})
.as_object()
.unwrap()
.clone(),
)
.unwrap(),
));
assert!(
snapshot
.plan(&id, &tools, b_only, &Default::default())
.is_err()
);
let plan = snapshot
.plan(&id, &tools, scope.clone(), &Default::default())
.unwrap();
assert!(
plan.candidates()[0]
.account_scope()
.allows(&ProviderAccountId::new("acct_a").unwrap())
);
assert!(!plan.candidates()[0].account_scope().allows(&b));
contribution.accounts = [ProviderAccountId::new("acct_c").unwrap()].into();
let denied = snapshot.with_extensions(Some(ExtensionSetReference::new(
ExtensionSetId::new("denied".into()).unwrap(),
Arc::new(ModelLease(vec![contribution])),
)));
let c_only = Arc::new(scope.restricted_to(
&Default::default(),
&[ProviderAccountId::new("acct_c").unwrap()].into(),
));
assert!(denied.public_models_for_scope(&c_only).is_empty());
assert!(
denied
.plan(&id, &super::operation(), c_only, &Default::default())
.is_err()
);
}
impl ExtensionSetLease for ModelLease {
fn is_ready(&self) -> bool {
true
}
fn models(&self) -> &[gateway_core::routing::ContributedModel] {
&self.0
}
}
#[test]
fn native_model_without_presentation_cannot_borrow_an_unauthorized_plugin_profile() {
let snapshot = super::snapshot().with_extensions(Some(ExtensionSetReference::new(
ExtensionSetId::new("private-profile".into()).unwrap(),
Arc::new(ModelLease(vec![gateway_core::routing::ContributedModel {
owner: "fixture".into(),
id: PublicModelId::new("gpt-5.5").unwrap(),
provider: ProviderKind::new("openai").unwrap(),
accounts: [gateway_core::account::ProviderAccountId::new("acct_private").unwrap()]
.into(),
capabilities: super::capabilities(),
presentation: ModelPresentation::new(Some("Private model".into()), None),
}])),
)));
assert!(
snapshot
.public_model_profiles_for_scope(&super::account_scope())
.is_empty()
);
}
#[test]
fn adapted_models_limit_visibility_and_routing_to_capable_authorized_accounts() {
use gateway_core::{
account::ProviderAccountId,
operation::{Feature, GenerateRequest, Operation, ProtocolPayload},
routing::{ContributedModel, SupportLevel},
};
let account = ProviderAccountId::new("acct_openai").unwrap();
let provider = ProviderKind::new("openai").unwrap();
let id = PublicModelId::new("adapted-model").unwrap();
let contribution = ContributedModel {
owner: "fixture".into(),
id: id.clone(),
provider,
accounts: [account.clone()].into(),
capabilities: super::capabilities(),
presentation: ModelPresentation::new(Some("Adapted".into()), None),
};
let snapshot = super::snapshot().with_extensions(Some(ExtensionSetReference::new(
ExtensionSetId::new("models".into()).unwrap(),
Arc::new(ModelLease(vec![contribution.clone()])),
)));
let scope = super::account_scope();
assert!(snapshot.public_models_for_scope(&scope).contains(&id));
assert!(
snapshot
.public_model_profiles_for_scope(&scope)
.iter()
.any(|p| p.model() == &id)
);
let plan = snapshot
.plan(&id, &super::operation(), scope.clone(), &Default::default())
.unwrap();
assert_eq!(plan.candidates().len(), 1);
assert!(plan.candidates()[0].account_scope().allows(&account));
let other = Arc::new(scope.restricted_to(
&Default::default(),
&[ProviderAccountId::new("acct_xai").unwrap()].into(),
));
assert!(!snapshot.public_models_for_scope(&other).contains(&id));
assert!(
snapshot
.plan(&id, &super::operation(), other, &Default::default())
.is_err()
);
let tools = Operation::Generate(GenerateRequest::from_protocol_payload(
ProtocolPayload::json_object("openai", serde_json::json!({"model":"adapted-model","input":[],"tools":[{"type":"function","name":"test","parameters":{}}]}).as_object().unwrap().clone()).unwrap(),
));
assert!(
snapshot
.plan(&id, &tools, scope.clone(), &Default::default())
.is_err()
);
let mut supported = contribution;
supported.capabilities = supported
.capabilities
.with_feature(Feature::Tools, SupportLevel::Native);
let capable = snapshot.with_extensions(Some(ExtensionSetReference::new(
ExtensionSetId::new("capable".into()).unwrap(),
Arc::new(ModelLease(vec![supported])),
)));
assert!(
capable
.plan(&id, &tools, scope, &Default::default())
.is_ok()
);
}
impl ExtensionSetLease for AliasLease {
fn is_ready(&self) -> bool {
true
+1 -1
View File
@@ -29,7 +29,7 @@ percent-encoding = "2.3.2"
reqwest = { version = "=0.12.28", default-features = false, features = ["json", "stream", "rustls-tls-native-roots", "socks"] }
rustls = { version = "=0.23.46", default-features = false, features = ["aws_lc_rs", "std", "tls12"] }
rustls-native-certs = "=0.8.3"
scraper = "0.27.0"
scraper = "0.29.0"
semver = "1.0.27"
secrecy = "0.10.3"
serde = { version = "1.0.228", features = ["derive"] }
@@ -24,7 +24,7 @@ gateway-plugin-sdk = { path = "../sdk", features = ["io"] }
gateway-protocol = { path = "../../gateway-protocol" }
hex = "0.4.3"
image = { version = "0.25.8", default-features = false, features = ["bmp", "gif", "ico", "jpeg", "png", "webp"] }
jsonschema = { version = "0.58.4", default-features = false }
jsonschema = { version = "0.60.0", default-features = false }
roxmltree = "0.21.1"
semver = "1.0.27"
secrecy = "0.10.3"
@@ -1,12 +1,12 @@
{
"schema_version": 2,
"manifest_schema_versions": [2],
"protocol_versions": [6],
"protocol_versions": [7],
"capabilities": [
{ "capability": "frontend_authentication", "versions": [2] },
{ "capability": "scheduler", "versions": [3] },
{ "capability": "model_router", "versions": [2] },
{ "capability": "model_catalog", "versions": [1] },
{ "capability": "model_catalog", "versions": [1, 2] },
{ "capability": "retry_policy", "versions": [2] },
{ "capability": "middleware", "versions": [6] },
{ "capability": "upstream_adapter", "versions": [5] },
@@ -4,18 +4,40 @@ use gateway_admin::model::{
AdminError,
plugins::instances::{PluginCapabilityBinding, PluginInstance},
};
use gateway_core::routing::{ContributedModelAlias, ProviderKind, PublicModelId, UpstreamModelId};
use gateway_plugin_sdk::{Capability, Manifest, Stage, call::catalog::ModelCatalogRegistration};
use gateway_core::{
account::ProviderAccountId,
operation::{Feature, OperationKind},
routing::{
ContributedModel, ContributedModelAlias, ModelCapabilities, ModelPresentation,
ProviderKind, PublicModelId, SupportLevel, UpstreamModelId,
},
};
use gateway_plugin_sdk::{
Capability, Manifest, Stage,
call::{
catalog::{ModelCatalogRegistration, ModelCatalogRegistrationV2},
middleware::RequestFeature,
},
};
use crate::RpcSession;
pub(crate) struct PreparedCatalog {
pub(crate) aliases: Vec<ContributedModelAlias>,
pub(crate) models: Vec<ContributedModel>,
}
pub(crate) async fn prepare(
manifest: &Manifest,
instance: &PluginInstance,
session: &RpcSession,
) -> Result<Vec<ContributedModelAlias>, AdminError> {
adapters: &[super::upstream_adapter::AdapterEntry],
) -> Result<PreparedCatalog, AdminError> {
if !manifest.contributes.contains_key(&Capability::ModelCatalog) {
return Ok(Vec::new());
return Ok(PreparedCatalog {
aliases: Vec::new(),
models: Vec::new(),
});
}
validate_bindings(manifest, &instance.bindings)?;
let reply = session
@@ -30,13 +52,19 @@ pub(crate) async fn prepare(
if reply.payload.len() > 64 * 1024 {
return Err(AdminError::invalid("插件模型目录信封无效或过大"));
}
let registration: ModelCatalogRegistration = serde_json::from_slice(&reply.payload)
.map_err(|_| AdminError::invalid("插件模型目录声明无效"))?;
if registration.models.len() > 256 {
return Err(AdminError::invalid("单个插件最多贡献 256 个模型别名"));
let (aliases, models) = if manifest.contributes[&Capability::ModelCatalog].version == 2 {
let registration: ModelCatalogRegistrationV2 = serde_json::from_slice(&reply.payload)
.map_err(|_| AdminError::invalid("插件模型目录声明无效"))?;
(registration.aliases, registration.models)
} else {
let registration: ModelCatalogRegistration = serde_json::from_slice(&reply.payload)
.map_err(|_| AdminError::invalid("插件模型目录声明无效"))?;
(registration.models, Vec::new())
};
if aliases.len().saturating_add(models.len()) > 256 {
return Err(AdminError::invalid("单个插件最多贡献 256 个模型条目"));
}
registration
.models
let aliases = aliases
.into_iter()
.map(|model| {
let id = PublicModelId::new(model.id)
@@ -50,7 +78,79 @@ pub(crate) async fn prepare(
.map_err(|_| AdminError::invalid("模型别名目标无效"))?,
})
})
.collect()
.collect::<Result<_, AdminError>>()?;
let models = models
.into_iter()
.map(|model| {
let mut targets = adapters
.iter()
.filter_map(|entry| entry.catalog_target(&model.adapter_id, &model.id));
let target = targets
.next()
.filter(|_| targets.next().is_none())
.ok_or_else(|| AdminError::invalid("适配模型必须关联同实例的静态账号适配器"))?;
let mut capabilities =
ModelCapabilities::new([OperationKind::Generate].into(), model.max_output_tokens);
for feature in model.features {
let feature = match feature {
RequestFeature::Tools => Feature::Tools,
RequestFeature::Vision => Feature::Vision,
RequestFeature::Reasoning => Feature::Reasoning,
RequestFeature::JsonSchema => Feature::JsonSchema,
};
capabilities = capabilities.with_feature(feature, SupportLevel::Native);
}
let p = model.presentation;
Ok(ContributedModel {
owner: instance.id.clone(),
id: PublicModelId::new(model.id)
.map_err(|_| AdminError::invalid("适配模型 ID 无效"))?,
provider: ProviderKind::new(target.provider.as_str())
.map_err(|_| AdminError::invalid("适配模型 Provider 无效"))?,
accounts: target
.account_ids
.iter()
.map(|id| {
ProviderAccountId::new(id.clone())
.map_err(|_| AdminError::invalid("适配模型账号无效"))
})
.collect::<Result<_, _>>()?,
capabilities,
presentation: ModelPresentation::new(p.display_name, p.description)
.with_reasoning(p.default_reasoning_effort, p.supported_reasoning_efforts)
.with_context_window_tokens(p.context_window_tokens)
.with_max_context_window_tokens(p.max_context_window_tokens)
.with_image_input(p.image_input)
.with_agent_tools(p.agent_tools, p.parallel_tool_calls)
.with_search_tool(p.search_tool)
.with_image_detail_original(p.image_detail_original)
.with_verbosity(p.verbosity)
.with_hidden(p.hidden),
})
})
.collect::<Result<_, AdminError>>()?;
Ok(PreparedCatalog { aliases, models })
}
/// 新目录的同名画像必须一致,不由注册顺序拼接能力或改变别名目标
pub(crate) fn validate_models(
aliases: &[ContributedModelAlias],
models: &[ContributedModel],
) -> Result<(), AdminError> {
for (index, model) in models.iter().enumerate() {
if aliases.iter().any(|alias| alias.id == model.id)
|| models[..index].iter().any(|other| {
other.id == model.id
&& (other.provider != model.provider
|| other.presentation != model.presentation
|| other.capabilities != model.capabilities
|| !other.accounts.is_disjoint(&model.accounts))
})
{
return Err(AdminError::invalid("适配模型目录声明冲突"));
}
}
Ok(())
}
pub(crate) fn validate_bindings(
@@ -195,21 +195,29 @@ impl FrontendAuthenticationPlan for PluginFrontendAuthenticationPlan {
let mut context =
session.context(Stage::Authentication, session.budget(Stage::Authentication));
context.request = crate::adapter::request_context(request.request_context());
context.request_id = request.invocation().map(|origin| origin.request_id.clone());
let _scope = callbacks
.prepare_frontend_authentication(&context, request.shared_request_context())
.prepare_frontend_authentication(&context, request)
.map_err(|_| FrontendAuthenticationError)?;
let input = FrontendAuthenticationRequest {
authorization: request.authorization().to_owned(),
};
let reply = session
.call(
"frontend_auth.authenticate",
context,
serde_json::json!({}),
serde_json::to_vec(&input).map_err(|_| FrontendAuthenticationError)?,
)
.await
.map_err(|_| FrontendAuthenticationError)?;
let invocation = session.call(
"frontend_auth.authenticate",
context,
serde_json::json!({}),
serde_json::to_vec(&input).map_err(|_| FrontendAuthenticationError)?,
);
let reply = if let Some(origin) = request.invocation() {
tokio::select! {
biased;
() = origin.cancellation.cancelled() => return Err(FrontendAuthenticationError),
reply = invocation => reply,
}
} else {
invocation.await
}
.map_err(|_| FrontendAuthenticationError)?;
let result: FrontendAuthenticationResult =
serde_json::from_slice(&reply.payload).map_err(|_| FrontendAuthenticationError)?;
match result {
@@ -51,7 +51,9 @@ pub(super) async fn invoke(
request_id: context.request_id.clone(),
call_id: context.call_id.clone(),
cancellation: context.cancellation.clone(),
plan,
plan: Some(plan),
extensions: context.extensions.clone(),
effects: None,
}),
context.extensions.clone(),
invocation.clone(),
@@ -38,7 +38,9 @@ pub(super) async fn invoke(
request_id: context.request_id.clone(),
call_id: context.call_id.clone(),
cancellation: context.cancellation.clone(),
plan: context.plan.clone(),
plan: Some(context.plan.clone()),
extensions: context.extensions.clone(),
effects: None,
}),
context.extensions.clone(),
invocation,
@@ -56,7 +56,9 @@ pub(super) async fn invoke(
request_id: context.connection_id.clone(),
call_id: call.resource_scope_id.clone(),
cancellation: context.cancellation.clone(),
plan,
plan: Some(plan),
extensions: context.extensions.clone(),
effects: None,
}),
context.extensions,
invocation.clone(),
@@ -86,6 +86,14 @@ impl BindingScope {
!self.providers.is_empty()
}
/// 静态目录只能继承账号范围,不能丢失请求级 Key 或组条件
pub(crate) fn matches_catalog(&self, provider: &ProviderKind, model: &str) -> bool {
self.client_keys.is_empty()
&& self.account_groups.is_empty()
&& self.matches_provider(Some(provider))
&& self.matches_model(Some(model))
}
pub(crate) fn matches_observation(&self, observation: &RequestObservation) -> bool {
self.matches_identity(observation.client_key_id(), observation.account_group_ids())
&& self.matches_provider(observation.provider())
@@ -28,6 +28,7 @@ pub(super) struct EventDecoder {
usage: Usage,
contents: usize,
pub(super) completed: bool,
pub(super) replay_blocked: bool,
}
pub(super) struct DecodedEvent {
@@ -50,6 +51,10 @@ impl EventDecoder {
if self.completed || message.event.facts.len() > 64 {
return Err(invalid(sent));
}
// 即使与失败同批携带,也不能把 wire、工具或其他业务事实丢弃后重放
self.replay_blocked |= !message.event.facts.is_empty()
|| message.event.wire.is_some()
|| message.continuation.is_some();
let wire = message
.event
.wire
@@ -75,12 +75,34 @@ pub(super) fn execute(
let result = state.next().await;
match result {
Ok(event) => Ok(Some((event, state))),
Err(error) => Err(state.invocation.account.record_failure(error).await),
Err(error) => {
let error = state.finish_error(error);
Err(state.invocation.account.record_failure(error).await)
}
}
}))
}
impl Execution {
fn finish_error(&mut self, mut error: ProviderError) -> ProviderError {
if let Some(active) = self.active.take() {
let managed = Arc::clone(&active.managed);
// RpcStream 析构同步撤销回调并封存未完成发送,证明不能早于这一步
drop(active);
if error.kind() == ProviderErrorKind::RateLimited
&& error.upstream_status() == Some(429)
&& error.send_state() == UpstreamSendState::Sent
&& managed.send_state.get() == UpstreamSendState::Sent
&& managed.replay.rejected_once()
&& !self.decoder.replay_blocked
&& !self.invocation.context.cancellation().is_cancelled()
{
error = error.with_replay_safe();
}
}
error
}
async fn next(&mut self) -> Result<gateway_core::event::ProviderEvent, ProviderError> {
if self.active.is_none() {
self.active = Some(self.begin().await?);
@@ -37,6 +37,24 @@ pub(crate) struct AdapterEntry {
adapter: Option<Arc<PluginUpstreamAdapter>>,
}
impl AdapterEntry {
pub(crate) fn catalog_target(
&self,
id: &str,
model: &str,
) -> Option<&UpstreamAdapterDeclaration> {
let adapter = self.adapter.as_ref()?;
let declaration = &adapter.declaration;
let provider = ProviderKind::new(declaration.provider.as_str()).ok()?;
(declaration.id == id
&& !declaration.account_ids.is_empty()
&& (declaration.models.is_empty()
|| declaration.models.iter().any(|value| value == model))
&& self.scope.matches_catalog(&provider, model))
.then_some(declaration)
}
}
pub(crate) struct PluginUpstreamAdapter {
instance_id: String,
contribution_id: String,
@@ -20,8 +20,8 @@ use gateway_core::{
use gateway_plugin_sdk::{
CallContext, ErrorCode, PluginFault,
call::host::{
AuthCredential, AuthGetRequest, AuthListRequest, AuthListResult, AuthRuntimeAccount,
AuthSaveRequest, AuthSaveResult, CredentialFacts,
self as wire, AuthCredential, AuthGetRequest, AuthListRequest, AuthListResult,
AuthRuntimeAccount, AuthSaveRequest, AuthSaveResult, CredentialFacts,
},
};
@@ -66,6 +66,32 @@ impl PluginAccounts {
Self { slot }
}
pub(super) async fn http_proxy(
&self,
account_id: String,
credential_revision: u64,
) -> Result<Option<gateway_core::account::OutboundProxy>, PluginFault> {
let id = ProviderAccountId::new(account_id).map_err(|_| invalid())?;
let expected = Revision::new(credential_revision).map_err(|_| invalid())?;
let account = self
.slot
.upgrade()
.map_err(map_admin_error)?
.get_runtime(&id)
.await
.map_err(map_admin_error)?;
if !account.enabled {
return Err(denied());
}
if account.credential_revision != expected {
return Err(PluginFault::new(
ErrorCode::Conflict,
"account credential revision has changed",
));
}
Ok(account.effective_outbound_proxy().cloned())
}
pub(super) async fn call(
&self,
context: &CallContext,
@@ -79,7 +105,7 @@ impl PluginAccounts {
}
let access = self.slot.upgrade().map_err(map_admin_error)?;
match method {
"host.auth.list" => {
wire::AUTH_LIST => {
let request: AuthListRequest = decode(payload)?;
let limit = PageSize::new(request.limit).map_err(|_| invalid())?;
let cursor = request
@@ -105,7 +131,7 @@ impl PluginAccounts {
next_cursor: page.next_cursor.map(|cursor| cursor.as_str().to_owned()),
})
}
"host.auth.get_runtime" => {
wire::AUTH_GET_RUNTIME => {
let request: AuthGetRequest = decode(payload)?;
let account_id =
ProviderAccountId::new(request.account_id).map_err(|_| invalid())?;
@@ -115,7 +141,7 @@ impl PluginAccounts {
.map_err(map_admin_error)?;
encode(&runtime_account(&account))
}
"host.auth.get" => {
wire::AUTH_GET => {
let request: AuthGetRequest = decode(payload)?;
let account_id =
ProviderAccountId::new(request.account_id).map_err(|_| invalid())?;
@@ -131,7 +157,7 @@ impl PluginAccounts {
facts: credential_facts(account, credential.provider_material),
})
}
"host.auth.save" => {
wire::AUTH_SAVE => {
let request: AuthSaveRequest = decode(payload)?;
encode(&self.save(context, scope, request).await?)
}
@@ -16,9 +16,8 @@ pub(super) struct CallResources {
pub(super) cancellation: CancellationToken,
pub(super) scope: Arc<CallbackScope>,
pub(super) http_resources: Arc<http_middleware::resources::Resources>,
pub(super) model_bindings: tokio::sync::Mutex<
BTreeMap<String, gateway_core::engine::execution::BoundModelExecutionContext>,
>,
pub(super) client_identities:
tokio::sync::Mutex<BTreeMap<String, gateway_core::engine::execution::AuthenticatedClient>>,
pub(super) state: Mutex<CallState>,
}
@@ -32,13 +31,18 @@ impl CallResources {
Self {
deadline: deadline.clone(),
operation_timeout: Duration::from_millis(context.timeout_ms),
cancellation: CancellationToken::new(),
cancellation: scope
.origin
.as_ref()
.map_or_else(CancellationToken::new, |origin| {
origin.cancellation.child_token()
}),
scope,
http_resources: middleware
.as_ref()
.and_then(|middleware| middleware.http_resources())
.unwrap_or_default(),
model_bindings: tokio::sync::Mutex::new(BTreeMap::new()),
client_identities: tokio::sync::Mutex::new(BTreeMap::new()),
state: Mutex::new(CallState {
middleware,
..CallState::default()
@@ -46,6 +50,40 @@ impl CallResources {
}
}
/// 子调用共享受信来源;身份解析与执行准入仍留在各能力 owner
pub(super) fn child_origin(
&self,
context: &CallContext,
) -> Result<gateway_core::engine::extensions::InvocationOrigin, PluginFault> {
let parent = self.scope.origin.as_ref();
Ok(gateway_core::engine::extensions::InvocationOrigin {
request_id: parent.map_or_else(
|| {
context
.request_id
.clone()
.unwrap_or_else(|| context.resource_scope_id.clone())
},
|origin| origin.request_id.clone(),
),
call_id: parent.map_or_else(
|| format!("{}:{}", context.resource_scope_id, context.call_id),
|origin| origin.call_id.clone(),
),
cancellation: self.cancellation.child_token(),
extensions: if self.scope.extension_scope.contains(&context.instance_id) {
self.scope.extension_scope.clone()
} else {
self.scope
.extension_scope
.extending(context.instance_id.clone())
.ok_or_else(super::invalid)?
},
plan: parent.and_then(|origin| origin.plan.clone()),
effects: self.scope.execution_effects.clone(),
})
}
pub(super) fn close(&self) {
self.cancellation.cancel();
let mut state = self
@@ -114,7 +152,9 @@ impl CallResources {
.unwrap_or_else(std::sync::PoisonError::into_inner)
.middleware
.clone();
middleware.and_then(|middleware| middleware.request_settings())
middleware
.and_then(|middleware| middleware.request_settings())
.or_else(|| self.scope.settings.clone())
}
/// 建立受管流后,网络操作各自计时;连接空闲时间不消耗后续操作的预算
@@ -0,0 +1,154 @@
//! 客户端认证、冻结权限查询与调用内身份复用
use std::sync::{Arc, OnceLock, Weak};
use gateway_admin::model::AdminError;
use gateway_core::{
account::ProviderAccountId,
engine::{
authentication::{ClientAuthenticationRequest, ClientIdentityPort},
execution::AuthenticatedClient,
},
policy::ClientApiKeyId,
routing::UpstreamModelId,
};
use gateway_plugin_sdk::{CallContext, ErrorCode, PluginFault, call::data};
use super::{CallResources, denied, invalid};
use crate::RpcReply;
const MAX_CLIENT_IDENTITIES_PER_CALL: usize = 16;
pub(crate) struct PluginClientIdentityPortSlot {
port: OnceLock<Weak<dyn ClientIdentityPort>>,
}
impl PluginClientIdentityPortSlot {
pub(crate) const fn new() -> Self {
Self {
port: OnceLock::new(),
}
}
pub(crate) fn bind(&self, port: &Arc<dyn ClientIdentityPort>) -> Result<(), AdminError> {
self.port
.set(Arc::downgrade(port))
.map_err(|_| AdminError::conflict("插件身份端口已经绑定"))
}
fn upgrade(&self) -> Result<Arc<dyn ClientIdentityPort>, PluginFault> {
self.port.get().and_then(Weak::upgrade).ok_or_else(denied)
}
}
pub(super) struct PluginClients {
slot: Arc<PluginClientIdentityPortSlot>,
}
impl PluginClients {
pub(super) fn new(slot: Arc<PluginClientIdentityPortSlot>) -> Self {
Self { slot }
}
pub(super) async fn call(
&self,
context: &CallContext,
call: &Arc<CallResources>,
method: &str,
params: serde_json::Value,
payload: Vec<u8>,
) -> Result<RpcReply, PluginFault> {
match method {
data::CLIENT_VERIFY => {
if params != serde_json::json!({}) {
return Err(invalid());
}
let request: data::VerifyClientRequest =
serde_json::from_slice(&payload).map_err(|_| invalid())?;
let mut authentication = ClientAuthenticationRequest::new(request.authorization)
.map_err(|_| invalid())?
.with_invocation(call.child_origin(context)?);
if let Some(settings) = call.request_settings() {
authentication = authentication.with_settings(settings);
}
let port = self.slot.upgrade()?;
call.observe_dispatch()?;
let client = port
.verify_client(authentication)
.await
.map_err(super::error::gateway)?;
super::admin::encode(&data::VerifiedClient {
client_key_id: client.policy().key_id().as_str().to_owned(),
})
}
data::ACCESS_CHECK => {
if params != serde_json::json!({}) {
return Err(invalid());
}
let request: data::AccessQuery =
serde_json::from_slice(&payload).map_err(|_| invalid())?;
let client = self.identity(call, request.client_key_id).await?;
let scope = client.policy().account_scope();
let model = request
.upstream_model
.map(UpstreamModelId::new)
.transpose()
.map_err(|_| invalid())?;
let accounts = request
.account_ids
.into_iter()
.map(|account_id| {
let account =
ProviderAccountId::new(account_id.clone()).map_err(|_| invalid())?;
let allowed = model.as_ref().map_or_else(
|| scope.allows(&account),
|model| scope.allows_model(&account, model.as_str()),
);
Ok(data::AccountAccessFacts {
account_id,
allowed,
})
})
.collect::<Result<Vec<_>, PluginFault>>()?;
super::admin::encode(&data::AccessFacts {
schema_version: 1,
client_key_id: client.policy().key_id().as_str().to_owned(),
accounts,
})
}
_ => Err(denied()),
}
}
pub(super) async fn identity(
&self,
call: &CallResources,
client_key_id: String,
) -> Result<AuthenticatedClient, PluginFault> {
let port = self.slot.upgrade()?;
let key = ClientApiKeyId::new(client_key_id.clone()).map_err(|_| invalid())?;
let settings = call.request_settings();
let mut identities = call.client_identities.lock().await;
if let Some(client) = identities
.get(&client_key_id)
.filter(|client| client.request_settings() == settings.as_ref())
{
return Ok(client.clone());
}
if !identities.contains_key(&client_key_id)
&& identities.len() >= MAX_CLIENT_IDENTITIES_PER_CALL
{
return Err(PluginFault::new(
ErrorCode::Capacity,
"client identity capacity is exhausted",
));
}
let client = port
.resolve_client(key, settings)
.await
.map_err(super::error::gateway)?;
// 查询与执行共用调用内快照,执行链和子执行取消只在真正启动时建立
identities.insert(client_key_id, client.clone());
Ok(client)
}
}
@@ -98,7 +98,7 @@ impl PluginData {
next_cursor: page.next_cursor.map(|id| id.as_str().to_owned()),
})
}
data::QUOTA_GET | data::QUOTA_REFRESH => {
data::QUOTA_GET | data::QUOTA_REFRESH | data::QUOTA_DETAILS => {
let accounts = self.accounts.upgrade().map_err(map_admin_error)?;
let query: data::QuotaFactsQuery =
serde_json::from_slice(payload).map_err(|_| invalid())?;
@@ -109,6 +109,35 @@ impl PluginData {
accounts.get_quota(&account_id).await
}
.map_err(map_admin_error)?;
if method == data::QUOTA_DETAILS {
return super::admin::encode(&data::QuotaDetails {
schema_version: 1,
account_id: account_id.as_str().to_owned(),
observed_at_ms: quota.observed_at.map(|time| time.timestamp_millis()),
limit_reached: quota.limit_reached,
windows: quota
.windows
.into_iter()
.map(|window| data::QuotaWindowDetails {
key: window.key,
group: window.group,
label: window.label,
limit_id: window.limit_id,
limit_name: window.limit_name,
role: window.role.map(|role| role.as_str().to_owned()),
window_seconds: window.window_seconds,
used_percent: window.used_percent,
reset_at_ms: window.reset_at.map(|time| time.timestamp_millis()),
limit_reached: window.limit_reached,
})
.collect(),
credits: quota.credits.map(|credits| data::QuotaCredits {
has_credits: credits.has_credits,
unlimited: credits.unlimited,
balance: credits.balance,
}),
});
}
serde_json::to_vec(&data::QuotaFacts {
schema_version: 1,
account_id: account_id.as_str().to_owned(),
@@ -0,0 +1,263 @@
//! 将有效父调用的宿主回调分派到所属原生能力,并随调用撤销资源
use std::sync::{Arc, Weak};
use futures::future::BoxFuture;
use gateway_plugin_sdk::{
CallContext, ErrorCode, PluginFault,
call::{host, middleware as middleware_wire, upstream_adapter as upstream_wire},
client::CallDeadline,
};
use super::{CallResources, CallbackScope, PluginCallbacks, denied, http, http_dispatch, upstream};
use crate::{CallbackHandler, RpcReply};
impl CallbackHandler for PluginCallbacks {
fn begin(&self, context: &CallContext, deadline: &CallDeadline) {
let scope = self
.scopes
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.get(&context.resource_scope_id)
.and_then(Weak::upgrade)
.unwrap_or_else(|| {
let mut scope = CallbackScope::for_call(context);
if context.stage == gateway_plugin_sdk::Stage::Registration {
scope.request_body_limit = self.registration_body_limit;
}
Arc::new(scope)
});
let middleware = self
.pending_middleware
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.remove(&context.resource_scope_id);
self.calls
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.insert(
context.call_id,
Arc::new(CallResources::new(context, deadline, scope, middleware)),
);
}
fn finish(&self, context: &CallContext) {
if let Some(call) = self
.calls
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.remove(&context.call_id)
{
call.close();
}
}
fn call(
&self,
context: CallContext,
method: String,
params: serde_json::Value,
payload: Vec<u8>,
) -> BoxFuture<'static, Result<RpcReply, PluginFault>> {
let call = self
.calls
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.get(&context.call_id)
.cloned();
let http = self.http.clone();
let network = self.network.clone();
let maximum_payload = self.maximum_payload;
let log = self.log.clone();
let private_state = self.private_state.clone();
let accounts = self.accounts.clone();
let resources = self.resources.clone();
let data = self.data.clone();
let keys = self.keys.clone();
let clients = self.clients.clone();
let models = self.models.clone();
let request_limits = self.request_limits.clone();
let affinity = self.affinity.clone();
let services = self.services.clone();
let service_actor = self.service_actor.clone();
Box::pin(async move {
let call = call.ok_or_else(denied)?;
let scope = &call.scope;
let (middleware, is_private_state) = {
let state = call
.state
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if state.closed || call.deadline.is_elapsed() {
return Err(denied());
}
if method == host::LOG {
return log.record(&context, params, &payload);
}
// 只证明受管 HTTP 拒绝;其他能力即使只读,也不推测其业务副作用
if let Some(managed) = &scope.upstream
&& !matches!(
method.as_str(),
upstream_wire::HTTP_DO
| upstream_wire::HTTP_DO_STREAM
| upstream_wire::HTTP_STREAM_READ
| upstream_wire::HTTP_STREAM_CLOSE
)
{
managed.replay.invalidate();
}
let middleware = if matches!(
method.as_str(),
middleware_wire::NEXT_METHOD
| middleware_wire::BODY_READ_METHOD
| middleware_wire::BODY_FACTS_METHOD
| middleware_wire::BODY_CLOSE_METHOD
| middleware_wire::http::UPGRADE_METHOD
| middleware_wire::http::RECEIVE_METHOD
| middleware_wire::websocket::SEND_METHOD
) {
Some(state.middleware.clone().ok_or_else(denied)?)
} else {
None
};
let is_private_state = matches!(
method.as_str(),
host::STATE_GET | host::STATE_PUT | host::STATE_DELETE
);
(middleware, is_private_state)
};
if matches!(
method.as_str(),
middleware_wire::http::DISPATCH_METHOD
| middleware_wire::http::BODY_READ_METHOD
| middleware_wire::http::BODY_CLOSE_METHOD
| middleware_wire::http::BODY_CREATE_METHOD
| middleware_wire::http::BODY_WRITE_METHOD
) {
return tokio::select! {
biased;
() = call.cancellation.cancelled() => Err(PluginFault::new(ErrorCode::Cancelled, "HTTP callback cancelled")),
result = http_dispatch::call(&services, &context, &call, &method, params, payload, maximum_payload) => result,
};
}
if let Some(middleware) = middleware {
return middleware
.invoke(method, params, payload, maximum_payload)
.await;
}
if method == gateway_plugin_sdk::call::services::CALL_METHOD {
return services
.call(&context, &call, service_actor, params, &payload)
.await;
}
if is_private_state {
return private_state.call(&method, params, &payload).await;
}
if matches!(
method.as_str(),
gateway_plugin_sdk::call::resources::GROUP_ENSURE
| gateway_plugin_sdk::call::resources::GROUP_MEMBERS
| gateway_plugin_sdk::call::resources::KEY_ENSURE
) {
return resources.call(&context, &method, params, &payload).await;
}
if matches!(
method.as_str(),
host::KEYS_LIST
| gateway_plugin_sdk::call::key_budgets::RESET
| gateway_plugin_sdk::call::key_budgets::GET
| gateway_plugin_sdk::call::key_budgets::UPDATE_LIMITS
) {
return keys.call(&context, &method, params, &payload).await;
}
if matches!(
method.as_str(),
gateway_plugin_sdk::call::data::CLIENT_VERIFY
| gateway_plugin_sdk::call::data::ACCESS_CHECK
) {
return clients
.call(&context, &call, &method, params, payload)
.await;
}
if method == host::MODELS_LIST {
return models.call(&context, &call, &method, params, payload).await;
}
if matches!(
method.as_str(),
host::MODEL_EXECUTE
| host::MODEL_EXECUTE_STREAM
| host::MODEL_STREAM_READ
| host::MODEL_STREAM_CLOSE
) {
return models.call(&context, &call, &method, params, payload).await;
}
if method == host::AFFINITY_LOOKUP {
return affinity.call(&context, params, &payload).await;
}
let scope = Some(scope)
.filter(|scope| scope.authorizes(&context))
.ok_or_else(denied)?;
if matches!(
method.as_str(),
gateway_plugin_sdk::call::data::ACCOUNTS_LIST
| gateway_plugin_sdk::call::data::KEYS_GET
| gateway_plugin_sdk::call::data::QUOTA_GET
| gateway_plugin_sdk::call::data::QUOTA_REFRESH
| gateway_plugin_sdk::call::data::QUOTA_DETAILS
) {
return data.call(&method, params, &payload).await;
}
if matches!(
method.as_str(),
host::AUTH_LIST | host::AUTH_GET | host::AUTH_GET_RUNTIME | host::AUTH_SAVE
) {
return accounts
.call(&context, scope, &method, params, &payload)
.await;
}
let http = http::HttpCallbacks {
client: &http,
network: &network,
scope,
call: &call,
maximum_payload,
request_limits: &request_limits,
};
if method == host::ACCOUNT_HTTP {
return http.account(&accounts, params, payload).await;
}
if matches!(
method.as_str(),
upstream_wire::HTTP_DO
| upstream_wire::HTTP_DO_STREAM
| upstream_wire::HTTP_STREAM_READ
| upstream_wire::HTTP_STREAM_CLOSE
| upstream_wire::WEBSOCKET_OPEN
| upstream_wire::WEBSOCKET_SEND
| upstream_wire::WEBSOCKET_READ
| upstream_wire::WEBSOCKET_CLOSE
) {
return upstream::dispatch(&http, &method, params, payload)
.await
.inspect_err(|_| {
if let Some(managed) = &scope.upstream {
managed.replay.invalidate();
}
});
}
if matches!(
method.as_str(),
host::HTTP_DO
| host::HTTP_DO_STREAM
| host::HTTP_STREAM_READ
| host::HTTP_STREAM_CLOSE
) {
return http.dispatch(&method, params, payload).await;
}
Err(PluginFault::new(
ErrorCode::Unsupported,
"host callback method is not supported",
))
})
}
}
@@ -29,16 +29,42 @@ impl HttpCallbacks<'_> {
self.request_limits.body_bytes(self.call)
}
pub(super) async fn account(
&self,
accounts: &super::accounts::PluginAccounts,
params: serde_json::Value,
payload: Vec<u8>,
) -> Result<RpcReply, PluginFault> {
let request: wire::AccountHttpRequest = decode(params)?;
let maximum_body_bytes = self.body_limit(&payload)?;
let proxy = accounts
.http_proxy(request.account_id, request.credential_revision)
.await?;
self.send(
HttpRequest {
method: request.request.method,
url: request.request.url,
headers: request.request.headers,
body: payload,
maximum_body_bytes,
},
true,
None,
proxy.as_ref(),
)
.await
}
pub(super) async fn send(
&self,
request: HttpRequest,
streaming: bool,
upstream_purpose: Option<gateway_plugin_sdk::call::upstream_adapter::UpstreamPathPurpose>,
proxy: Option<&gateway_core::account::OutboundProxy>,
) -> Result<RpcReply, PluginFault> {
let Self {
client,
network,
scope,
call,
maximum_payload,
..
@@ -48,13 +74,12 @@ impl HttpCallbacks<'_> {
}
let timeout = call.timeout()?;
let attempt = call.start_upstream(upstream_purpose)?;
let response = client
.open(request, scope.proxy.as_ref(), network, timeout)
.await;
attempt.finish(
let response = client.open(request, proxy, network, timeout).await;
attempt.finish_http(
response
.as_ref()
.map_or_else(|error| error.send_state, |_| UpstreamSendState::Sent),
response.as_ref().ok().map(|response| response.status),
);
let response = response.map_err(http_error)?;
let mut body = response.body;
@@ -115,7 +140,7 @@ impl HttpCallbacks<'_> {
) -> Result<RpcReply, PluginFault> {
let call = self.call;
match method {
"host.http.do" | "host.http.do_stream" => {
wire::HTTP_DO | wire::HTTP_DO_STREAM => {
let request: wire::HttpRequest = decode(params)?;
self.send(
HttpRequest {
@@ -125,12 +150,13 @@ impl HttpCallbacks<'_> {
maximum_body_bytes: self.body_limit(&payload)?,
body: payload,
},
method == "host.http.do_stream",
method == wire::HTTP_DO_STREAM,
None,
self.scope.proxy.as_ref(),
)
.await
}
"host.http.stream_read" => {
wire::HTTP_STREAM_READ => {
if !payload.is_empty() {
return Err(invalid());
}
@@ -174,7 +200,7 @@ impl HttpCallbacks<'_> {
payload: chunk.map_or_else(Vec::new, |chunk| chunk.to_vec()),
})
}
"host.http.stream_close" => {
wire::HTTP_STREAM_CLOSE => {
if !payload.is_empty() {
return Err(invalid());
}
@@ -36,25 +36,23 @@ pub(super) async fn call(
request.timeout_ms,
&context.instance_id,
)?;
let parent = call.scope.origin.as_ref();
call.scope.check_child_depth(&context.instance_id)?;
let origin = call.child_origin(context)?;
let child = core::Context {
plugin_instance_id: Some(context.instance_id.clone()),
request_id: context
.request_id
.clone()
.unwrap_or_else(|| context.resource_scope_id.clone()),
request_id: origin.request_id,
call_id: uuid::Uuid::now_v7().to_string(),
parent_call_id: Some(parent.map_or_else(
|| format!("{}:{}", context.resource_scope_id, context.call_id),
|parent| parent.call_id.clone(),
)),
extensions: call.scope.child_extensions(&context.instance_id)?,
// 句柄可以在父 RPC End 后交给 HTTP 发送端;生命周期属于父调用和响应正文,不能在 RPC End 提前取消
cancellation: parent.map_or_else(
|| call.cancellation.child_token(),
|parent| parent.cancellation.child_token(),
),
plan: parent.map(|parent| parent.plan.clone()),
parent_call_id: Some(origin.call_id),
extensions: origin.extensions,
// 转交正文后由消费端持有原请求寿命,父 RPC End 只撤销新业务分派
cancellation: call
.scope
.origin
.as_ref()
.map_or(origin.cancellation, |parent| {
parent.cancellation.child_token()
}),
plan: origin.plan,
};
let body = call.http_resources.body(request.body, payload).await?;
let mut http = core::Request::new(body);
@@ -0,0 +1,243 @@
//! 为具体入口登记回调作用域和一次性中间件绑定,不持有 RPC 调度状态
use std::sync::{Arc, Weak};
use gateway_core::{account::OutboundProxy, lifecycle::CancellationToken};
use gateway_plugin_sdk::{CallContext, PluginFault};
use super::{
CallbackScope, InvocationContext, MiddlewareBinding, MiddlewareCallback, PluginCallbacks,
denied, upstream,
};
impl PluginCallbacks {
pub(crate) fn bind_middleware(
self: &Arc<Self>,
resource_scope_id: String,
invocation: Arc<dyn MiddlewareCallback>,
scope: Arc<CallbackScope>,
) -> Result<MiddlewareBinding, gateway_core::engine::middleware::MiddlewareError> {
let mut pending = self
.pending_middleware
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if pending
.insert(resource_scope_id.clone(), Arc::clone(&invocation))
.is_some()
{
return Err(gateway_core::engine::middleware::MiddlewareError::InvalidState);
}
Ok(MiddlewareBinding::new(
Arc::clone(self),
resource_scope_id,
invocation,
scope,
))
}
pub(super) fn unbind_middleware(
&self,
resource_scope_id: &str,
invocation: &Arc<dyn MiddlewareCallback>,
) {
let mut pending = self
.pending_middleware
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if pending
.get(resource_scope_id)
.is_some_and(|pending| Arc::ptr_eq(pending, invocation))
{
pending.remove(resource_scope_id);
}
}
pub(crate) fn prepare_data_plane(
&self,
context: &CallContext,
effects: Option<Arc<gateway_core::engine::nested::ExecutionEffects>>,
extensions: gateway_core::engine::extensions::ExtensionCallScope,
plan: Option<gateway_core::engine::middleware::FrozenMiddlewarePlan>,
cancellation: CancellationToken,
request_body_limit: Option<usize>,
) -> Result<Arc<CallbackScope>, PluginFault> {
if !matches!(
context.stage,
gateway_plugin_sdk::Stage::Request
| gateway_plugin_sdk::Stage::Attempt
| gateway_plugin_sdk::Stage::Outbound
| gateway_plugin_sdk::Stage::Routing
| gateway_plugin_sdk::Stage::Scheduling
| gateway_plugin_sdk::Stage::Retry
) {
return Err(denied());
}
let mut scope = CallbackScope::for_call(context);
if let Some(effects) = effects {
scope = scope.with_execution_effects(effects);
}
scope.extension_scope = extensions;
scope.request_body_limit = request_body_limit;
scope.origin = plan.map(|plan| InvocationContext {
request_id: context
.request_id
.clone()
.unwrap_or_else(|| context.resource_scope_id.clone()),
call_id: context
.request_id
.clone()
.unwrap_or_else(|| context.resource_scope_id.clone()),
cancellation,
extensions: scope.extension_scope.clone(),
effects: scope.execution_effects.clone(),
plan: Some(plan),
});
self.register_scope(context, scope)
}
pub(crate) fn request_body_limit(&self) -> Result<usize, PluginFault> {
self.request_limits.current_body_bytes()
}
pub(crate) fn prepare_upstream(
&self,
context: &CallContext,
managed: Arc<upstream::ManagedUpstream>,
extension_scope: gateway_core::engine::extensions::ExtensionCallScope,
request_body_limit: usize,
) -> Result<Arc<CallbackScope>, PluginFault> {
if context.stage != gateway_plugin_sdk::Stage::Upstream
|| context.account_id.as_deref() != Some(managed.account.account_id().as_str())
|| context.credential_revision != Some(managed.account.credential_revision().get())
{
return Err(denied());
}
let mut scope = CallbackScope::new(context, managed.account.outbound_proxy().cloned())
.with_upstream(managed);
scope.extension_scope = extension_scope;
scope.request_body_limit = Some(request_body_limit);
self.register_scope(context, scope)
}
pub(crate) fn prepare_management(
&self,
context: &CallContext,
proxy: Option<OutboundProxy>,
request: Option<Arc<gateway_core::engine::extensions::RequestContext>>,
) -> Result<Arc<CallbackScope>, PluginFault> {
if !matches!(
context.stage,
gateway_plugin_sdk::Stage::Management | gateway_plugin_sdk::Stage::PublicManagement
) {
return Err(denied());
}
self.prepare_scope(context, proxy, request)
}
pub(crate) fn prepare_command_line(
&self,
context: &CallContext,
) -> Result<Arc<CallbackScope>, PluginFault> {
if context.stage != gateway_plugin_sdk::Stage::CommandLine {
return Err(denied());
}
self.prepare_scope(context, None, None)
}
pub(crate) fn prepare_frontend_authentication(
&self,
context: &CallContext,
request: &gateway_core::engine::authentication::ClientAuthenticationRequest,
) -> Result<Arc<CallbackScope>, PluginFault> {
if context.stage != gateway_plugin_sdk::Stage::Authentication
|| context.account_id.is_some()
|| context.credential_revision.is_some()
|| context.attempt_id.is_some()
{
return Err(denied());
}
let mut scope = CallbackScope::for_call(context);
scope.settings = request.settings().cloned();
if let Some(origin) = request.invocation() {
scope.extension_scope = origin.extensions.clone();
scope.execution_effects = origin.effects.clone();
scope.origin = Some(origin.clone());
} else if let Some(request) = request.shared_request_context() {
scope.extension_scope = scope.extension_scope.with_request(request);
}
self.register_scope(context, scope)
}
pub(crate) async fn save_command_account(
&self,
context: &CallContext,
scope: &CallbackScope,
request: gateway_plugin_sdk::call::host::AuthSaveRequest,
) -> Result<gateway_plugin_sdk::call::host::AuthSaveResult, PluginFault> {
if context.stage != gateway_plugin_sdk::Stage::CommandLine || !scope.authorizes(context) {
return Err(denied());
}
self.accounts.save(context, scope, request).await
}
fn prepare_scope(
&self,
context: &CallContext,
proxy: Option<OutboundProxy>,
request: Option<Arc<gateway_core::engine::extensions::RequestContext>>,
) -> Result<Arc<CallbackScope>, PluginFault> {
let mut scope = CallbackScope::new(context, proxy);
if let Some(request) = request {
scope.extension_scope = scope.extension_scope.with_request(request);
}
self.register_scope(context, scope)
}
pub(crate) fn prepare_extensions(
&self,
context: &CallContext,
extension_scope: gateway_core::engine::extensions::ExtensionCallScope,
) -> Result<Arc<CallbackScope>, PluginFault> {
let mut scope = CallbackScope::for_call(context);
scope.extension_scope = extension_scope;
self.register_scope(context, scope)
}
pub(crate) fn bind_invocation(
self: &Arc<Self>,
call: &CallContext,
origin: Option<InvocationContext>,
extensions: gateway_core::engine::extensions::ExtensionCallScope,
invocation: Arc<dyn MiddlewareCallback>,
) -> Result<MiddlewareBinding, gateway_core::engine::middleware::MiddlewareError> {
let mut scope = CallbackScope::for_call(call);
scope.extension_scope = extensions;
scope.origin = origin;
let scope = self
.register_scope(call, scope)
.map_err(|_| gateway_core::engine::middleware::MiddlewareError::Fault)?;
self.bind_middleware(call.resource_scope_id.clone(), invocation, scope)
}
fn register_scope(
&self,
context: &CallContext,
scope: CallbackScope,
) -> Result<Arc<CallbackScope>, PluginFault> {
let scope = Arc::new(scope);
let mut scopes = self
.scopes
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
scopes.retain(|_, scope| scope.strong_count() != 0);
if scopes
.get(&context.resource_scope_id)
.and_then(Weak::upgrade)
.is_some()
{
return Err(denied());
}
scopes.insert(context.resource_scope_id.clone(), Arc::downgrade(&scope));
Ok(scope)
}
}
@@ -52,7 +52,7 @@ impl PluginClientKeys {
params: serde_json::Value,
payload: &[u8],
) -> Result<RpcReply, PluginFault> {
if method == "host.keys.list" {
if method == gateway_plugin_sdk::call::host::KEYS_LIST {
return self.ports.list(params, payload).await;
}
if params != serde_json::json!({}) {
@@ -1,21 +1,26 @@
//! 插件宿主回调端口的组合、授权检查与方法分派
//! 插件宿主回调端口、调用作用域与分派的模块入口
mod accounts;
mod admin;
mod affinity;
mod call_resources;
mod client;
mod data;
mod dispatch;
pub(crate) mod error;
mod facts;
mod http;
mod http_dispatch;
pub(crate) mod http_middleware;
mod invocation;
mod keys;
mod log;
mod middleware;
mod model;
pub(crate) mod outbound;
mod ports;
pub(crate) mod private_state;
mod replay;
pub(crate) mod request_limits;
mod resources;
mod scope;
@@ -23,21 +28,11 @@ pub(crate) mod services;
pub(crate) mod upstream;
pub(crate) mod websocket_middleware;
use std::{
collections::BTreeMap,
sync::{Arc, Mutex, Weak},
};
use futures::future::BoxFuture;
use gateway_admin::model::AdminError;
use gateway_core::{account::OutboundProxy, lifecycle::CancellationToken};
use gateway_host::outbound::{HttpClient, NetworkPolicy};
use gateway_plugin_sdk::{CallContext, ErrorCode, PluginFault, client::CallDeadline};
use crate::{CallbackHandler, RpcReply};
pub(crate) use accounts::PluginAccountPortSlot;
pub(crate) use affinity::PluginAffinityPortSlot;
use call_resources::{CallResources, HttpStream};
pub(crate) use client::PluginClientIdentityPortSlot;
use gateway_plugin_sdk::{ErrorCode, PluginFault};
pub(crate) use keys::PluginClientKeyPortSlot;
pub(crate) use middleware::{
MiddlewareBinding, MiddlewareBodyAuthority, MiddlewareCallback, MiddlewareCompletionBody,
@@ -47,509 +42,7 @@ pub(crate) use model::PluginModelPortSlot;
pub(crate) use resources::PluginResourcePorts;
pub(crate) use scope::{CallbackScope, InvocationContext};
pub(crate) struct PluginCallbackPorts {
services: Arc<services::ServicePorts>,
http: Arc<HttpClient>,
network: NetworkPolicy,
accounts: Arc<PluginAccountPortSlot>,
keys: Arc<PluginClientKeyPortSlot>,
models: Arc<PluginModelPortSlot>,
affinity: Arc<PluginAffinityPortSlot>,
resources: Arc<PluginResourcePorts>,
}
impl PluginCallbackPorts {
pub(crate) fn new(
services: Arc<services::ServicePorts>,
http: Arc<HttpClient>,
accounts: Arc<PluginAccountPortSlot>,
keys: Arc<PluginClientKeyPortSlot>,
models: Arc<PluginModelPortSlot>,
affinity: Arc<PluginAffinityPortSlot>,
resources: Arc<PluginResourcePorts>,
) -> Self {
Self {
services,
http,
network: NetworkPolicy::unrestricted(),
accounts,
keys,
models,
affinity,
resources,
}
}
}
pub(crate) struct PluginCallbacks {
pub(crate) registration_body_limit: Option<usize>,
services: Arc<services::ServicePorts>,
service_actor: gateway_admin::model::usage_data::UsageActor,
accounts: Arc<accounts::PluginAccounts>,
resources: Arc<resources::PluginResources>,
data: Arc<data::PluginData>,
keys: Arc<keys::PluginClientKeys>,
affinity: Arc<affinity::PluginAffinity>,
log: Arc<log::PluginLog>,
models: Arc<model::PluginModels>,
private_state: Arc<private_state::PluginPrivateState>,
http: Arc<HttpClient>,
network: Arc<NetworkPolicy>,
scopes: Mutex<BTreeMap<String, Weak<CallbackScope>>>,
pending_middleware: Mutex<BTreeMap<String, Arc<dyn MiddlewareCallback>>>,
calls: Mutex<BTreeMap<u64, Arc<CallResources>>>,
maximum_payload: usize,
request_limits: Arc<request_limits::RequestLimits>,
}
impl PluginCallbacks {
pub(crate) fn new(
instance: &gateway_admin::model::plugins::instances::PluginInstance,
maximum_payload: usize,
manifest: &gateway_plugin_sdk::Manifest,
log_slots: Arc<tokio::sync::Semaphore>,
private_state: Arc<private_state::PluginPrivateState>,
ports: PluginCallbackPorts,
request_limits: Arc<request_limits::RequestLimits>,
) -> Result<Self, AdminError> {
Ok(Self {
registration_body_limit: None,
services: ports.services,
service_actor: gateway_admin::model::usage_data::UsageActor::Plugin {
publisher: manifest.publisher.clone(),
name: manifest.name.clone(),
instance_id: instance.id.clone(),
},
resources: Arc::new(resources::PluginResources::new(instance, ports.resources)),
data: Arc::new(data::PluginData::new(
ports.accounts.clone(),
ports.keys.clone(),
)),
accounts: Arc::new(accounts::PluginAccounts::new(ports.accounts)),
keys: Arc::new(keys::PluginClientKeys::new(instance, ports.keys)),
affinity: Arc::new(affinity::PluginAffinity::new(ports.affinity)),
log: Arc::new(
log::PluginLog::new(manifest, log_slots)
.map_err(|_| AdminError::invalid("插件清单身份不合法"))?,
),
models: Arc::new(model::PluginModels::new(
ports.models,
request_limits.clone(),
maximum_payload,
)),
private_state,
http: ports.http,
network: Arc::new(ports.network),
scopes: Mutex::new(BTreeMap::new()),
pending_middleware: Mutex::new(BTreeMap::new()),
calls: Mutex::new(BTreeMap::new()),
maximum_payload,
request_limits,
})
}
pub(crate) fn bind_middleware(
self: &Arc<Self>,
resource_scope_id: String,
invocation: Arc<dyn MiddlewareCallback>,
scope: Arc<CallbackScope>,
) -> Result<MiddlewareBinding, gateway_core::engine::middleware::MiddlewareError> {
let mut pending = self
.pending_middleware
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if pending
.insert(resource_scope_id.clone(), Arc::clone(&invocation))
.is_some()
{
return Err(gateway_core::engine::middleware::MiddlewareError::InvalidState);
}
Ok(MiddlewareBinding::new(
Arc::clone(self),
resource_scope_id,
invocation,
scope,
))
}
fn unbind_middleware(&self, resource_scope_id: &str, invocation: &Arc<dyn MiddlewareCallback>) {
let mut pending = self
.pending_middleware
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if pending
.get(resource_scope_id)
.is_some_and(|pending| Arc::ptr_eq(pending, invocation))
{
pending.remove(resource_scope_id);
}
}
pub(crate) fn prepare_data_plane(
&self,
context: &CallContext,
effects: Option<Arc<gateway_core::engine::nested::ExecutionEffects>>,
extensions: gateway_core::engine::extensions::ExtensionCallScope,
plan: Option<gateway_core::engine::middleware::FrozenMiddlewarePlan>,
cancellation: CancellationToken,
request_body_limit: Option<usize>,
) -> Result<Arc<CallbackScope>, PluginFault> {
if !matches!(
context.stage,
gateway_plugin_sdk::Stage::Request
| gateway_plugin_sdk::Stage::Attempt
| gateway_plugin_sdk::Stage::Outbound
| gateway_plugin_sdk::Stage::Routing
| gateway_plugin_sdk::Stage::Scheduling
| gateway_plugin_sdk::Stage::Retry
) {
return Err(denied());
}
let mut scope = CallbackScope::for_call(context);
if let Some(effects) = effects {
scope = scope.with_execution_effects(effects);
}
scope.extension_scope = extensions;
scope.request_body_limit = request_body_limit;
scope.origin = plan.map(|plan| InvocationContext {
request_id: context
.request_id
.clone()
.unwrap_or_else(|| context.resource_scope_id.clone()),
call_id: context
.request_id
.clone()
.unwrap_or_else(|| context.resource_scope_id.clone()),
cancellation,
plan,
});
self.register_scope(context, scope)
}
pub(crate) fn request_body_limit(&self) -> Result<usize, PluginFault> {
self.request_limits.current_body_bytes()
}
pub(crate) fn prepare_upstream(
&self,
context: &CallContext,
managed: Arc<upstream::ManagedUpstream>,
extension_scope: gateway_core::engine::extensions::ExtensionCallScope,
request_body_limit: usize,
) -> Result<Arc<CallbackScope>, PluginFault> {
if context.stage != gateway_plugin_sdk::Stage::Upstream
|| context.account_id.as_deref() != Some(managed.account.account_id().as_str())
|| context.credential_revision != Some(managed.account.credential_revision().get())
{
return Err(denied());
}
let mut scope = CallbackScope::new(context, managed.account.outbound_proxy().cloned())
.with_upstream(managed);
scope.extension_scope = extension_scope;
scope.request_body_limit = Some(request_body_limit);
self.register_scope(context, scope)
}
pub(crate) fn prepare_management(
&self,
context: &CallContext,
proxy: Option<OutboundProxy>,
request: Option<Arc<gateway_core::engine::extensions::RequestContext>>,
) -> Result<Arc<CallbackScope>, PluginFault> {
if !matches!(
context.stage,
gateway_plugin_sdk::Stage::Management | gateway_plugin_sdk::Stage::PublicManagement
) {
return Err(denied());
}
self.prepare_scope(context, proxy, request)
}
pub(crate) fn prepare_command_line(
&self,
context: &CallContext,
) -> Result<Arc<CallbackScope>, PluginFault> {
if context.stage != gateway_plugin_sdk::Stage::CommandLine {
return Err(denied());
}
self.prepare_scope(context, None, None)
}
pub(crate) fn prepare_frontend_authentication(
&self,
context: &CallContext,
request: Option<Arc<gateway_core::engine::extensions::RequestContext>>,
) -> Result<Arc<CallbackScope>, PluginFault> {
if context.stage != gateway_plugin_sdk::Stage::Authentication
|| context.account_id.is_some()
|| context.credential_revision.is_some()
|| context.attempt_id.is_some()
{
return Err(denied());
}
self.prepare_scope(context, None, request)
}
pub(crate) async fn save_command_account(
&self,
context: &CallContext,
scope: &CallbackScope,
request: gateway_plugin_sdk::call::host::AuthSaveRequest,
) -> Result<gateway_plugin_sdk::call::host::AuthSaveResult, PluginFault> {
if context.stage != gateway_plugin_sdk::Stage::CommandLine || !scope.authorizes(context) {
return Err(denied());
}
self.accounts.save(context, scope, request).await
}
fn prepare_scope(
&self,
context: &CallContext,
proxy: Option<OutboundProxy>,
request: Option<Arc<gateway_core::engine::extensions::RequestContext>>,
) -> Result<Arc<CallbackScope>, PluginFault> {
let mut scope = CallbackScope::new(context, proxy);
if let Some(request) = request {
scope.extension_scope = scope.extension_scope.with_request(request);
}
self.register_scope(context, scope)
}
pub(crate) fn prepare_extensions(
&self,
context: &CallContext,
extension_scope: gateway_core::engine::extensions::ExtensionCallScope,
) -> Result<Arc<CallbackScope>, PluginFault> {
let mut scope = CallbackScope::for_call(context);
scope.extension_scope = extension_scope;
self.register_scope(context, scope)
}
pub(crate) fn bind_invocation(
self: &Arc<Self>,
call: &CallContext,
origin: Option<InvocationContext>,
extensions: gateway_core::engine::extensions::ExtensionCallScope,
invocation: Arc<dyn MiddlewareCallback>,
) -> Result<MiddlewareBinding, gateway_core::engine::middleware::MiddlewareError> {
let mut scope = CallbackScope::for_call(call);
scope.extension_scope = extensions;
scope.origin = origin;
let scope = self
.register_scope(call, scope)
.map_err(|_| gateway_core::engine::middleware::MiddlewareError::Fault)?;
self.bind_middleware(call.resource_scope_id.clone(), invocation, scope)
}
fn register_scope(
&self,
context: &CallContext,
scope: CallbackScope,
) -> Result<Arc<CallbackScope>, PluginFault> {
let scope = Arc::new(scope);
let mut scopes = self
.scopes
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
scopes.retain(|_, scope| scope.strong_count() != 0);
if scopes
.get(&context.resource_scope_id)
.and_then(Weak::upgrade)
.is_some()
{
return Err(denied());
}
scopes.insert(context.resource_scope_id.clone(), Arc::downgrade(&scope));
Ok(scope)
}
}
impl CallbackHandler for PluginCallbacks {
fn begin(&self, context: &CallContext, deadline: &CallDeadline) {
let scope = self
.scopes
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.get(&context.resource_scope_id)
.and_then(Weak::upgrade)
.unwrap_or_else(|| {
let mut scope = CallbackScope::for_call(context);
if context.stage == gateway_plugin_sdk::Stage::Registration {
scope.request_body_limit = self.registration_body_limit;
}
Arc::new(scope)
});
let middleware = self
.pending_middleware
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.remove(&context.resource_scope_id);
self.calls
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.insert(
context.call_id,
Arc::new(CallResources::new(context, deadline, scope, middleware)),
);
}
fn finish(&self, context: &CallContext) {
if let Some(call) = self
.calls
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.remove(&context.call_id)
{
call.close();
}
}
fn call(
&self,
context: CallContext,
method: String,
params: serde_json::Value,
payload: Vec<u8>,
) -> BoxFuture<'static, Result<RpcReply, PluginFault>> {
let call = self
.calls
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.get(&context.call_id)
.cloned();
let http = self.http.clone();
let network = self.network.clone();
let maximum_payload = self.maximum_payload;
let log = self.log.clone();
let private_state = self.private_state.clone();
let accounts = self.accounts.clone();
let resources = self.resources.clone();
let data = self.data.clone();
let keys = self.keys.clone();
let models = self.models.clone();
let request_limits = self.request_limits.clone();
let affinity = self.affinity.clone();
let services = self.services.clone();
let service_actor = self.service_actor.clone();
Box::pin(async move {
let call = call.ok_or_else(denied)?;
let scope = &call.scope;
let (middleware, is_private_state) = {
let state = call
.state
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if state.closed || call.deadline.is_elapsed() {
return Err(denied());
}
if method == "host.log" {
return log.record(&context, params, &payload);
}
let middleware = if method.starts_with("host.middleware.") {
Some(state.middleware.clone().ok_or_else(denied)?)
} else {
None
};
let is_private_state = matches!(
method.as_str(),
"host.state.get" | "host.state.put" | "host.state.delete"
);
(middleware, is_private_state)
};
if matches!(
method.as_str(),
gateway_plugin_sdk::call::middleware::http::DISPATCH_METHOD
| gateway_plugin_sdk::call::middleware::http::BODY_READ_METHOD
| gateway_plugin_sdk::call::middleware::http::BODY_CLOSE_METHOD
| gateway_plugin_sdk::call::middleware::http::BODY_CREATE_METHOD
| gateway_plugin_sdk::call::middleware::http::BODY_WRITE_METHOD
) {
return tokio::select! {
biased;
() = call.cancellation.cancelled() => Err(PluginFault::new(ErrorCode::Cancelled, "HTTP callback cancelled")),
result = http_dispatch::call(&services, &context, &call, &method, params, payload, maximum_payload) => result,
};
}
if let Some(middleware) = middleware {
return middleware
.invoke(method, params, payload, maximum_payload)
.await;
}
if method == gateway_plugin_sdk::call::services::CALL_METHOD {
return services
.call(&context, &call, service_actor, params, &payload)
.await;
}
if is_private_state {
return private_state.call(&method, params, &payload).await;
}
if matches!(
method.as_str(),
gateway_plugin_sdk::call::resources::GROUP_ENSURE
| gateway_plugin_sdk::call::resources::GROUP_MEMBERS
| gateway_plugin_sdk::call::resources::KEY_ENSURE
) {
return resources.call(&context, &method, params, &payload).await;
}
if matches!(
method.as_str(),
"host.keys.list"
| gateway_plugin_sdk::call::key_budgets::RESET
| gateway_plugin_sdk::call::key_budgets::GET
| gateway_plugin_sdk::call::key_budgets::UPDATE_LIMITS
) {
return keys.call(&context, &method, params, &payload).await;
}
if method == "host.models.list" {
return models.call(&context, &call, &method, params, payload).await;
}
if matches!(
method.as_str(),
"host.model.execute"
| "host.model.execute_stream"
| "host.model.stream_read"
| "host.model.stream_close"
) {
return models.call(&context, &call, &method, params, payload).await;
}
if method == "host.affinity.lookup" {
return affinity.call(&context, params, &payload).await;
}
let scope = Some(scope)
.filter(|scope| scope.authorizes(&context))
.ok_or_else(denied)?;
if matches!(
method.as_str(),
gateway_plugin_sdk::call::data::ACCOUNTS_LIST
| gateway_plugin_sdk::call::data::KEYS_GET
| gateway_plugin_sdk::call::data::QUOTA_GET
| gateway_plugin_sdk::call::data::QUOTA_REFRESH
) {
return data.call(&method, params, &payload).await;
}
if matches!(
method.as_str(),
"host.auth.list" | "host.auth.get" | "host.auth.get_runtime" | "host.auth.save"
) {
return accounts
.call(&context, scope, &method, params, &payload)
.await;
}
let http = http::HttpCallbacks {
client: &http,
network: &network,
scope,
call: &call,
maximum_payload,
request_limits: &request_limits,
};
if method.starts_with("host.upstream.") {
return upstream::dispatch(&http, &method, params, payload).await;
}
http.dispatch(&method, params, payload).await
})
}
}
pub(crate) use ports::{PluginCallbackPorts, PluginCallbacks};
fn denied() -> PluginFault {
PluginFault::new(
@@ -1,4 +1,4 @@
//! 嵌套模型执行;只驱动 Core 会话,不拥有路由、重试或计费规则
//! 模型目录与执行适配;身份解析交给客户端端口,路由与计费沿用 Core
use std::sync::{Arc, OnceLock, Weak};
@@ -9,11 +9,10 @@ use gateway_core::{
engine::{
CommitRequirement, ModelRequestId,
execution::{
BoundModelExecutionContext, ClientTransport, ExecutionRequestMetadata, ExecutionSession,
AuthenticatedClient, ClientTransport, ExecutionRequestMetadata, ExecutionSession,
},
nested::{
BoundModelExecutionBinding, BoundModelExecutionRequest, NestedModelExecutionPort,
NestedModelExecutionRequest,
BoundModelExecutionRequest, NestedModelExecutionPort, NestedModelExecutionRequest,
},
},
event::ProviderEvent,
@@ -22,13 +21,12 @@ use gateway_core::{
GenerateRequest, ImageRequest, ImageRequestKind, Operation, ProtocolPayload,
RawJsonPayload, StandaloneSearchRequest,
},
policy::ClientApiKeyId,
routing::PublicModelId,
};
use gateway_plugin_sdk::{
CallContext, ErrorCode, PluginFault,
call::host::{
ModelEventBatch, ModelExecuteRequest, ModelExecuteResult, ModelListRequest,
self as wire, ModelEventBatch, ModelExecuteRequest, ModelExecuteResult, ModelListRequest,
ModelListResult, ModelOperation, ModelStreamCloseRequest, ModelStreamReadRequest,
ModelStreamReadResult, ModelStreamResult,
},
@@ -63,6 +61,7 @@ impl PluginModelPortSlot {
pub(super) struct PluginModels {
slot: Arc<PluginModelPortSlot>,
clients: Arc<super::client::PluginClients>,
request_limits: Arc<super::request_limits::RequestLimits>,
maximum_payload: usize,
}
@@ -70,11 +69,13 @@ pub(super) struct PluginModels {
impl PluginModels {
pub(super) fn new(
slot: Arc<PluginModelPortSlot>,
clients: Arc<super::client::PluginClients>,
request_limits: Arc<super::request_limits::RequestLimits>,
maximum_payload: usize,
) -> Self {
Self {
slot,
clients,
request_limits,
maximum_payload,
}
@@ -89,7 +90,7 @@ impl PluginModels {
payload: Vec<u8>,
) -> Result<RpcReply, PluginFault> {
match method {
"host.models.list" => {
wire::MODELS_LIST => {
if !payload.is_empty() {
return Err(invalid());
}
@@ -102,19 +103,14 @@ impl PluginModels {
return Err(invalid());
}
let port = self.slot.upgrade()?;
let bound = port
.bind(BoundModelExecutionBinding {
settings: call.request_settings(),
client_key_id: ClientApiKeyId::new(request.client_key_id)
.map_err(|_| invalid())?,
initiating_plugin_instance_id: context.instance_id.clone(),
cancellation: call.cancellation.child_token(),
extension_scope: call.scope.extension_scope.clone(),
})
.await
.map_err(super::error::gateway)?;
let client = self.clients.identity(call, request.client_key_id).await?;
let models = port
.models(bound, request.protocol, request.client_version)
.models(
client,
call.cancellation.clone(),
request.protocol,
request.client_version,
)
.await
.map_err(super::error::gateway)?;
encode_result(ModelListResult {
@@ -124,7 +120,7 @@ impl PluginModels {
.collect(),
})
}
"host.model.execute" | "host.model.execute_stream" => {
wire::MODEL_EXECUTE | wire::MODEL_EXECUTE_STREAM => {
let mut request: ModelExecuteRequest =
serde_json::from_value(params).map_err(|_| invalid())?;
if payload.is_empty() {
@@ -139,11 +135,14 @@ impl PluginModels {
}
};
let port = self.slot.upgrade()?;
let stream = method == "host.model.execute_stream";
let stream = method == wire::MODEL_EXECUTE_STREAM;
let started = if let Some(client_key_id) = request.client_key_id.take() {
let bound = self.binding(&port, context, call, client_key_id).await?;
check_size(bound.request_body_limit())?;
port.start_bound(self.bound_request(request, payload, stream, bound)?)
let client = self.clients.identity(call, client_key_id).await?;
check_size(client.snapshot().responses_max_decompressed_body_bytes())?;
let request = self.request_parts(request, payload, stream)?;
let request = Self::bound_request(context, call, request, client);
call.observe_dispatch()?;
port.start_bound(request)
.await
.map_err(super::error::gateway)?
} else {
@@ -151,7 +150,7 @@ impl PluginModels {
let request = self.request(context, request, payload, stream)?;
port.start(request).await.map_err(super::error::gateway)?
};
if method == "host.model.execute" {
if method == wire::MODEL_EXECUTE {
execute_buffered(started, self.maximum_payload).await
} else {
let stream = Arc::new(ModelStream::new(started.session, self.maximum_payload));
@@ -175,7 +174,7 @@ impl PluginModels {
})
}
}
"host.model.stream_read" => {
wire::MODEL_STREAM_READ => {
if !payload.is_empty() {
return Err(invalid());
}
@@ -214,7 +213,7 @@ impl PluginModels {
payload: read.payload,
})
}
"host.model.stream_close" => {
wire::MODEL_STREAM_CLOSE => {
if !payload.is_empty() {
return Err(invalid());
}
@@ -237,44 +236,6 @@ impl PluginModels {
}
}
async fn binding(
&self,
port: &Arc<dyn NestedModelExecutionPort>,
context: &CallContext,
call: &CallResources,
client_key_id: String,
) -> Result<BoundModelExecutionContext, PluginFault> {
let key = ClientApiKeyId::new(client_key_id.clone()).map_err(|_| invalid())?;
let settings = call.request_settings();
let mut bindings = call.model_bindings.lock().await;
if let Some(bound) = bindings
.get(&client_key_id)
.filter(|bound| bound.request_settings() == settings.as_ref())
{
return Ok(bound.clone());
}
if !bindings.contains_key(&client_key_id) && bindings.len() >= MAX_MODEL_STREAMS_PER_CALL {
return Err(PluginFault::new(
ErrorCode::Capacity,
"model identity capacity is exhausted",
));
}
let bound = port
.bind(BoundModelExecutionBinding {
settings,
client_key_id: key,
initiating_plugin_instance_id: context.instance_id.clone(),
cancellation: call.cancellation.child_token(),
extension_scope: call.scope.extension_scope.clone(),
})
.await
.map_err(super::error::gateway)?;
// 调用持有身份直到 RPC 结束,流式执行不会因单次 callback 返回而提前取消;
// 同一 Key 复用身份快照,每次执行由 Core 创建独立调用图
bindings.insert(client_key_id, bound.clone());
Ok(bound)
}
fn request(
&self,
context: &CallContext,
@@ -310,21 +271,22 @@ impl PluginModels {
}
fn bound_request(
&self,
request: ModelExecuteRequest,
body: Vec<u8>,
stream: bool,
context: BoundModelExecutionContext,
) -> Result<BoundModelExecutionRequest, PluginFault> {
let request = self.request_parts(request, body, stream)?;
Ok(BoundModelExecutionRequest {
context,
context: &CallContext,
call: &CallResources,
request: ModelRequestParts,
client: AuthenticatedClient,
) -> BoundModelExecutionRequest {
BoundModelExecutionRequest {
client,
initiating_plugin_instance_id: context.instance_id.clone(),
cancellation: call.cancellation.clone(),
extension_scope: call.scope.extension_scope.clone(),
public_model: request.public_model,
operation: request.operation,
metadata: request.metadata,
provider: request.provider,
account: request.account,
})
}
}
fn request_parts(
@@ -0,0 +1,102 @@
//! 组装实例宿主端口及调用资源索引,保持资源与插件实例同寿命
use std::{
collections::BTreeMap,
sync::{Arc, Mutex, Weak},
};
use gateway_admin::model::AdminError;
use gateway_host::outbound::{HttpClient, NetworkPolicy};
use super::{
CallResources, CallbackScope, MiddlewareCallback, PluginAccountPortSlot,
PluginAffinityPortSlot, PluginClientIdentityPortSlot, PluginClientKeyPortSlot,
PluginModelPortSlot, PluginResourcePorts,
};
use super::{
accounts, affinity, client, data, keys, log, model, private_state, request_limits, resources,
services,
};
pub(crate) struct PluginCallbackPorts {
pub(crate) services: Arc<services::ServicePorts>,
pub(crate) http: Arc<HttpClient>,
pub(crate) accounts: Arc<PluginAccountPortSlot>,
pub(crate) keys: Arc<PluginClientKeyPortSlot>,
pub(crate) clients: Arc<PluginClientIdentityPortSlot>,
pub(crate) models: Arc<PluginModelPortSlot>,
pub(crate) affinity: Arc<PluginAffinityPortSlot>,
pub(crate) resources: Arc<PluginResourcePorts>,
}
pub(crate) struct PluginCallbacks {
pub(crate) registration_body_limit: Option<usize>,
pub(super) services: Arc<services::ServicePorts>,
pub(super) service_actor: gateway_admin::model::usage_data::UsageActor,
pub(super) accounts: Arc<accounts::PluginAccounts>,
pub(super) resources: Arc<resources::PluginResources>,
pub(super) data: Arc<data::PluginData>,
pub(super) keys: Arc<keys::PluginClientKeys>,
pub(super) affinity: Arc<affinity::PluginAffinity>,
pub(super) log: Arc<log::PluginLog>,
pub(super) clients: Arc<client::PluginClients>,
pub(super) models: Arc<model::PluginModels>,
pub(super) private_state: Arc<private_state::PluginPrivateState>,
pub(super) http: Arc<HttpClient>,
pub(super) network: Arc<NetworkPolicy>,
pub(super) scopes: Mutex<BTreeMap<String, Weak<CallbackScope>>>,
pub(super) pending_middleware: Mutex<BTreeMap<String, Arc<dyn MiddlewareCallback>>>,
pub(super) calls: Mutex<BTreeMap<u64, Arc<CallResources>>>,
pub(super) maximum_payload: usize,
pub(super) request_limits: Arc<request_limits::RequestLimits>,
}
impl PluginCallbacks {
pub(crate) fn new(
instance: &gateway_admin::model::plugins::instances::PluginInstance,
maximum_payload: usize,
manifest: &gateway_plugin_sdk::Manifest,
log_slots: Arc<tokio::sync::Semaphore>,
private_state: Arc<private_state::PluginPrivateState>,
ports: PluginCallbackPorts,
request_limits: Arc<request_limits::RequestLimits>,
) -> Result<Self, AdminError> {
let clients = Arc::new(client::PluginClients::new(ports.clients));
Ok(Self {
registration_body_limit: None,
services: ports.services,
service_actor: gateway_admin::model::usage_data::UsageActor::Plugin {
publisher: manifest.publisher.clone(),
name: manifest.name.clone(),
instance_id: instance.id.clone(),
},
resources: Arc::new(resources::PluginResources::new(instance, ports.resources)),
data: Arc::new(data::PluginData::new(
ports.accounts.clone(),
ports.keys.clone(),
)),
accounts: Arc::new(accounts::PluginAccounts::new(ports.accounts)),
keys: Arc::new(keys::PluginClientKeys::new(instance, ports.keys)),
affinity: Arc::new(affinity::PluginAffinity::new(ports.affinity)),
log: Arc::new(
log::PluginLog::new(manifest, log_slots)
.map_err(|_| AdminError::invalid("插件清单身份不合法"))?,
),
clients: clients.clone(),
models: Arc::new(model::PluginModels::new(
ports.models,
clients,
request_limits.clone(),
maximum_payload,
)),
private_state,
http: ports.http,
network: Arc::new(NetworkPolicy::unrestricted()),
scopes: Mutex::new(BTreeMap::new()),
pending_middleware: Mutex::new(BTreeMap::new()),
calls: Mutex::new(BTreeMap::new()),
maximum_payload,
request_limits,
})
}
}
@@ -21,8 +21,8 @@ use gateway_admin::{
use gateway_plugin_sdk::{
ErrorCode, Manifest, PluginFault,
call::host::{
StateDeleteRequest, StateDeleteResult, StateGetRequest, StateGetResult, StatePutRequest,
StatePutResult, StateRecord,
self as wire, StateDeleteRequest, StateDeleteResult, StateGetRequest, StateGetResult,
StatePutRequest, StatePutResult, StateRecord,
},
};
use sha2::{Digest as _, Sha256};
@@ -99,7 +99,7 @@ impl PluginPrivateState {
.clone()
.ok_or_else(denied_fault)?;
let result = match method {
"host.state.get" => {
wire::STATE_GET => {
let request: StateGetRequest =
serde_json::from_value(params).map_err(|_| invalid_fault())?;
self.authorize(&request.namespace)?;
@@ -115,7 +115,7 @@ impl PluginPrivateState {
});
serde_json::to_value(StateGetResult { record }).map_err(|_| internal_fault())?
}
"host.state.put" => {
wire::STATE_PUT => {
let request: StatePutRequest =
serde_json::from_value(params).map_err(|_| invalid_fault())?;
self.authorize(&request.namespace)?;
@@ -141,7 +141,7 @@ impl PluginPrivateState {
})
.map_err(|_| internal_fault())?
}
"host.state.delete" => {
wire::STATE_DELETE => {
let request: StateDeleteRequest =
serde_json::from_value(params).map_err(|_| invalid_fault())?;
self.authorize(&request.namespace)?;
@@ -0,0 +1,59 @@
//! 记录受管单次 HTTP 拒绝,其他调用或不明结果只会收紧重放资格
use std::sync::Mutex;
#[derive(Default)]
pub(crate) struct HttpReplay(Mutex<State>);
#[derive(Default)]
enum State {
#[default]
Untouched,
Pending,
Rejected,
Ineligible,
}
impl HttpReplay {
pub(super) fn start(&self, inference: bool) {
let mut state = self
.0
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
*state = if inference && matches!(*state, State::Untouched) {
State::Pending
} else {
State::Ineligible
};
}
pub(super) fn finish(&self, status: Option<u16>) {
let mut state = self
.0
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
*state = if status == Some(429) && matches!(*state, State::Pending) {
State::Rejected
} else {
State::Ineligible
};
}
pub(super) fn invalidate(&self) {
*self
.0
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner) = State::Ineligible;
}
/// 调用撤销后读取,防止检查通过后仍有新回调发送
pub(crate) fn rejected_once(&self) -> bool {
matches!(
*self
.0
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner),
State::Rejected
)
}
}
@@ -7,14 +7,7 @@ use gateway_core::{
};
use gateway_plugin_sdk::{CallContext, Stage};
/// 所有中间件共享的父调用身份;不混入协议操作或服务参数
#[derive(Clone)]
pub(crate) struct InvocationContext {
pub request_id: String,
pub call_id: String,
pub cancellation: gateway_core::lifecycle::CancellationToken,
pub plan: gateway_core::engine::middleware::FrozenMiddlewarePlan,
}
pub(crate) use gateway_core::engine::extensions::InvocationOrigin as InvocationContext;
/// 一次回调的调用链、账号与网络事实,由操作持有到完成
pub(crate) struct CallbackScope {
@@ -25,7 +18,8 @@ pub(crate) struct CallbackScope {
credential_revision: Option<u64>,
pub(super) proxy: Option<OutboundProxy>,
pub(super) extension_scope: gateway_core::engine::extensions::ExtensionCallScope,
execution_effects: Option<Arc<ExecutionEffects>>,
pub(super) execution_effects: Option<Arc<ExecutionEffects>>,
pub(super) settings: Option<gateway_core::routing::request_settings::RequestSettings>,
pub(super) upstream: Option<Arc<super::upstream::ManagedUpstream>>,
}
@@ -37,14 +31,13 @@ impl CallbackScope {
}
}
pub(super) fn child_extensions(
pub(super) fn check_child_depth(
&self,
instance_id: &str,
) -> Result<gateway_core::engine::extensions::ExtensionCallScope, gateway_plugin_sdk::PluginFault>
{
) -> Result<(), gateway_plugin_sdk::PluginFault> {
// 上游适配器在进入回调前已登记当前实例;主动子调用沿用该集合
if self.extension_scope.contains(instance_id) {
return Ok(self.extension_scope.clone());
return Ok(());
}
if self.extension_scope.len()
>= gateway_core::engine::extensions::ExtensionCallScope::MAXIMUM_DEPTH
@@ -54,9 +47,7 @@ impl CallbackScope {
"child call depth exceeded",
));
}
self.extension_scope
.extending(instance_id.to_owned())
.ok_or_else(super::invalid)
Ok(())
}
pub(super) fn new(context: &CallContext, proxy: Option<OutboundProxy>) -> Self {
@@ -69,6 +60,7 @@ impl CallbackScope {
proxy,
extension_scope: Default::default(),
execution_effects: None,
settings: None,
upstream: None,
}
}
@@ -106,10 +98,13 @@ impl CallbackScope {
&self,
purpose: Option<gateway_plugin_sdk::call::upstream_adapter::UpstreamPathPurpose>,
) -> HttpAttempt {
let inference = purpose
== Some(gateway_plugin_sdk::call::upstream_adapter::UpstreamPathPurpose::Inference);
if let Some(managed) = &self.upstream {
managed.replay.start(inference);
}
HttpAttempt(Arc::new(HttpAttemptObservation {
effects: if purpose
== Some(gateway_plugin_sdk::call::upstream_adapter::UpstreamPathPurpose::Inference)
{
effects: if inference {
None
} else {
self.execution_effects.clone()
@@ -119,6 +114,10 @@ impl CallbackScope {
.as_ref()
.map(|managed| Arc::clone(&managed.send_state)),
completed: Mutex::new(false),
replay: self
.upstream
.as_ref()
.map(|managed| Arc::clone(&managed.replay)),
}))
}
}
@@ -129,6 +128,7 @@ pub(super) struct HttpAttemptObservation {
effects: Option<Arc<ExecutionEffects>>,
completed: Mutex<bool>,
upstream: Option<Arc<super::upstream::SendWatermark>>,
replay: Option<Arc<super::replay::HttpReplay>>,
}
impl HttpAttempt {
@@ -139,10 +139,18 @@ impl HttpAttempt {
pub(super) fn finish(self, observed: UpstreamSendState) {
self.0.finish(observed);
}
pub(super) fn finish_http(self, observed: UpstreamSendState, status: Option<u16>) {
self.0.finish_http(observed, status);
}
}
impl HttpAttemptObservation {
pub(super) fn finish(&self, observed: UpstreamSendState) {
self.finish_http(observed, None);
}
fn finish_http(&self, observed: UpstreamSendState, status: Option<u16>) {
let mut completed = self
.completed
.lock()
@@ -153,6 +161,9 @@ impl HttpAttemptObservation {
if let Some(upstream) = &self.upstream {
upstream.observe(observed);
}
if let Some(replay) = &self.replay {
replay.finish(status.filter(|_| observed == UpstreamSendState::Sent));
}
if observed != UpstreamSendState::NotSent
&& let Some(effects) = &self.effects
{
@@ -63,27 +63,18 @@ impl ServicePorts {
)
})?;
let call = registry.call(&request.operation, request.input);
let scope = &resources.scope;
let origin = scope.origin.as_ref();
resources.scope.check_child_depth(&context.instance_id)?;
let origin = resources.child_origin(context)?;
let parent = gateway_admin::public_service::Origin {
request_id: origin.map_or_else(
|| {
context
.request_id
.clone()
.unwrap_or_else(|| context.resource_scope_id.clone())
},
|origin| origin.request_id.clone(),
),
call_id: origin.map_or_else(
|| format!("{}:{}", context.resource_scope_id, context.call_id),
|origin| origin.call_id.clone(),
),
cancellation: resources.cancellation.clone(),
extensions: scope.child_extensions(&context.instance_id)?,
plan: origin.map_or(gateway_admin::public_service::Plan::Current, |origin| {
gateway_admin::public_service::Plan::Frozen(Some(origin.plan.clone()))
}),
request_id: origin.request_id,
call_id: origin.call_id,
cancellation: origin.cancellation,
extensions: origin.extensions,
plan: origin
.plan
.map_or(gateway_admin::public_service::Plan::Current, |plan| {
gateway_admin::public_service::Plan::Frozen(Some(plan))
}),
};
// 即使终端只读,服务中间件也可提交写入;不存在的操作不会进入调用链
if registry
@@ -55,6 +55,7 @@ impl SendWatermark {
pub(crate) struct ManagedUpstream {
pub(crate) account: Arc<dyn UpstreamAccountConnection>,
pub(crate) send_state: Arc<SendWatermark>,
pub(crate) replay: Arc<super::replay::HttpReplay>,
pub(crate) effects: Option<Arc<ExecutionEffects>>,
target: Arc<UpstreamTarget>,
socket: tokio::sync::Mutex<Option<(String, Arc<ManagedWebSocket>)>>,
@@ -85,6 +86,7 @@ impl ManagedUpstream {
owner,
websocket_allowed,
send_state: Arc::default(),
replay: Arc::default(),
socket: tokio::sync::Mutex::new(socket),
}))
}
@@ -193,26 +195,31 @@ pub(super) async fn dispatch(
.map(|remaining| remaining.min(Duration::from_secs(120)))
};
match method {
"host.upstream.http.do" | "host.upstream.http.do_stream" => {
adapter::HTTP_DO | adapter::HTTP_DO_STREAM => {
let request = serde_json::from_value(params).map_err(|_| invalid())?;
let maximum_body_bytes = http.body_limit(&payload)?;
let (request, purpose) = managed.request(request, payload, maximum_body_bytes)?;
http.send(request, method.ends_with("do_stream"), Some(purpose))
.await
http.send(
request,
method == adapter::HTTP_DO_STREAM,
Some(purpose),
scope.proxy.as_ref(),
)
.await
}
"host.upstream.http.stream_read" | "host.upstream.http.stream_close" => {
adapter::HTTP_STREAM_READ | adapter::HTTP_STREAM_CLOSE => {
http.dispatch(
if method.ends_with("stream_read") {
"host.http.stream_read"
if method == adapter::HTTP_STREAM_READ {
wire::HTTP_STREAM_READ
} else {
"host.http.stream_close"
wire::HTTP_STREAM_CLOSE
},
params,
payload,
)
.await
}
"host.upstream.websocket.open" => {
adapter::WEBSOCKET_OPEN => {
if !payload.is_empty() || !managed.websocket_allowed {
return Err(denied());
}
@@ -298,7 +305,7 @@ pub(super) async fn dispatch(
payload,
})
}
"host.upstream.websocket.send" => {
adapter::WEBSOCKET_SEND => {
let message: adapter::UpstreamWebSocketMessage =
serde_json::from_value(params).map_err(|_| invalid())?;
let message = match message.kind {
@@ -322,7 +329,7 @@ pub(super) async fn dispatch(
payload: vec![],
})
}
"host.upstream.websocket.read" => {
adapter::WEBSOCKET_READ => {
if params != serde_json::json!({}) || !payload.is_empty() {
return Err(invalid());
}
@@ -353,7 +360,7 @@ pub(super) async fn dispatch(
payload,
})
}
"host.upstream.websocket.close" => {
adapter::WEBSOCKET_CLOSE => {
if params != serde_json::json!({}) || !payload.is_empty() {
return Err(invalid());
}
@@ -10,7 +10,7 @@ use gateway_admin::model::{
AdminError, AdminErrorKind, Revision,
plugins::instances::{
PluginInstance, PluginInstanceRuntime, PluginInstanceRuntimeFailure,
PluginInstanceRuntimeStatus, PluginInstanceSnapshot,
PluginInstanceRuntimeLoad, PluginInstanceRuntimeStatus, PluginInstanceSnapshot,
},
};
use gateway_core::routing::extensions::ExtensionSetReference;
@@ -227,6 +227,17 @@ impl InstanceRuntimeProjection<'_> {
actual_artifact_sha256: active.map(|instance| instance.artifact_sha256.clone()),
failure,
draining_revisions: draining_revisions.into_iter().collect(),
load: active.map(|instance| {
let load = instance.session.load();
PluginInstanceRuntimeLoad {
active_calls: load.active_calls,
executing_calls: load.executing_calls,
queued_calls: load.queued_calls,
resuming_calls: load.resuming_calls,
pending_callbacks: load.pending_callbacks,
active_streams: load.active_streams,
}
}),
}
}
}
@@ -18,6 +18,31 @@ use std::{sync::Arc, time::Duration};
const PROCESS_PREPARATION_TIMEOUT: Duration = Duration::from_secs(30);
#[derive(Debug)]
pub(super) enum InstancePreparationError {
UnloadablePackage(crate::PackageError),
IncompatiblePackage(String),
Failed(AdminError),
}
impl InstancePreparationError {
pub(super) fn into_admin_error(self) -> AdminError {
match self {
Self::UnloadablePackage(error) => crate::package::inspection_error(error),
Self::IncompatiblePackage(reason) => {
AdminError::invalid(format!("插件版本不兼容:{reason}"))
}
Self::Failed(error) => error,
}
}
}
impl From<AdminError> for InstancePreparationError {
fn from(error: AdminError) -> Self {
Self::Failed(error)
}
}
struct StartedInstance {
instance: PluginInstance,
package: Arc<crate::PreparedPackage>,
@@ -31,16 +56,9 @@ impl PluginRuntime {
instance: PluginInstance,
target_revision: Revision,
request_body_limit: Option<usize>,
) -> Result<StartedInstance, AdminError> {
allow_incompatible: bool,
) -> Result<StartedInstance, InstancePreparationError> {
let authorization_binding = instance_fingerprint(&instance)?;
if self.restart_circuit_is_open(&RestartIdentity::new(
instance.id.clone(),
authorization_binding.clone(),
)) {
return Err(AdminError::unavailable(
"插件实例连续异常退出,已暂停自动重启;请检查配置后重新启用",
));
}
let validation_slot = self
.validators
.clone()
@@ -65,46 +83,57 @@ impl PluginRuntime {
Some(&instance.artifact_sha256),
limits,
)
.map_err(|_| AdminError::invalid("插件恢复包校验失败"))?,
.map_err(InstancePreparationError::UnloadablePackage)?,
);
let (configuration, state_configuration) =
super::super::configuration::validate(&instance, package.manifest(), &schemas)?;
crate::adapter::observer::validate_bindings(
package.manifest(),
&instance.bindings,
)?;
crate::adapter::policy::validate_bindings(package.manifest(), &instance.bindings)?;
crate::adapter::middleware::validate_bindings(
package.manifest(),
&instance.bindings,
)?;
crate::adapter::upstream_adapter::validate_bindings(
package.manifest(),
&instance.bindings,
)?;
crate::adapter::frontend_authentication::validate_bindings(
package.manifest(),
&instance.bindings,
)?;
package
.manifest()
.package_for_platform(std::env::consts::OS, std::env::consts::ARCH)
.map_err(|error| InstancePreparationError::UnloadablePackage(error.into()))?;
let requirements = crate::package::compatibility_requirements(package.manifest())?;
if let Some(warning) =
if let Some(reason) =
crate::package::compatibility_warning(&requirements, &host_version)?
{
tracing::warn!(
instance_id = instance.id,
warning,
"插件版本未经宿主保证,尝试启动"
);
if !allow_incompatible {
return Err(InstancePreparationError::IncompatiblePackage(reason));
}
tracing::warn!(instance_id = instance.id, reason, "手动尝试启动不兼容插件");
}
let package = Arc::new(
package
.prepare(&directory)
.map_err(|_| AdminError::unavailable("插件制品准备失败"))?,
);
Ok((package, configuration, instance, state_configuration))
let manifest = package.manifest();
let (configuration, state_configuration) =
super::super::configuration::validate(&instance, manifest, &schemas)?;
crate::adapter::observer::validate_bindings(manifest, &instance.bindings)?;
crate::adapter::policy::validate_bindings(manifest, &instance.bindings)?;
crate::adapter::middleware::validate_bindings(manifest, &instance.bindings)?;
crate::adapter::upstream_adapter::validate_bindings(manifest, &instance.bindings)?;
crate::adapter::frontend_authentication::validate_bindings(
manifest,
&instance.bindings,
)?;
let package =
Arc::new(package.prepare(&directory).map_err(|error| match error {
crate::PackageError::Manifest(_) => {
InstancePreparationError::UnloadablePackage(error)
}
_ => AdminError::unavailable("插件制品准备失败").into(),
})?);
Ok::<_, InstancePreparationError>((
package,
configuration,
instance,
state_configuration,
))
})
.await
.map_err(|_| AdminError::internal("插件校验任务失败"))??;
if self.restart_circuit_is_open(&RestartIdentity::new(
instance.id.clone(),
authorization_binding.clone(),
)) {
return Err(AdminError::unavailable(
"插件实例连续异常退出,已暂停自动重启;请检查配置后重新启用",
)
.into());
}
let manifest = package.package().manifest().clone();
let plugin_id = manifest
.plugin_id()
@@ -161,15 +190,16 @@ impl PluginRuntime {
&manifest,
self.log_slots.clone(),
private_state.clone(),
PluginCallbackPorts::new(
self.service_ports.clone(),
self.http.clone(),
self.account_ports.clone(),
self.client_key_ports.clone(),
self.model_ports.clone(),
self.affinity_ports.clone(),
self.resource_ports.clone(),
),
PluginCallbackPorts {
services: self.service_ports.clone(),
http: self.http.clone(),
accounts: self.account_ports.clone(),
keys: self.client_key_ports.clone(),
clients: self.client_identity_port.clone(),
models: self.model_ports.clone(),
affinity: self.affinity_ports.clone(),
resources: self.resource_ports.clone(),
},
self.request_limits.clone(),
)?;
callbacks.registration_body_limit = request_body_limit;
@@ -218,7 +248,8 @@ impl PluginRuntime {
instance: PluginInstance,
target_revision: Revision,
request_body_limit: Option<usize>,
) -> Result<Arc<PreparedInstance>, AdminError> {
allow_incompatible: bool,
) -> Result<Arc<PreparedInstance>, InstancePreparationError> {
// 启动控制期限只包住制品准备与握手,Ready 后每次注册使用协商预算
let StartedInstance {
instance,
@@ -227,7 +258,12 @@ impl PluginRuntime {
mut prepared,
} = tokio::time::timeout(
PROCESS_PREPARATION_TIMEOUT,
self.start_instance(instance, target_revision, request_body_limit),
self.start_instance(
instance,
target_revision,
request_body_limit,
allow_incompatible,
),
)
.await
.map_err(|_| AdminError::unavailable("插件启动超时"))??;
@@ -255,7 +291,7 @@ impl PluginRuntime {
let descriptor: Registration = serde_json::from_value(registration.result)
.map_err(|_| AdminError::invalid("插件注册结果无效"))?;
if descriptor.contributes != manifest.contributes || !registration.payload.is_empty() {
return Err(AdminError::invalid("插件注册结果与清单不符"));
return Err(AdminError::invalid("插件注册结果与清单不符").into());
}
commands.extend(
crate::adapter::command_line::prepare(
@@ -310,7 +346,6 @@ impl PluginRuntime {
session.clone(),
callbacks.clone(),
)?;
let model_aliases = crate::adapter::catalog::prepare(manifest, &instance, session).await?;
let upstream_entries = crate::adapter::upstream_adapter::prepare(
manifest,
&instance,
@@ -318,6 +353,9 @@ impl PluginRuntime {
Arc::clone(&callbacks),
)
.await?;
let catalog =
crate::adapter::catalog::prepare(manifest, &instance, session, &upstream_entries)
.await?;
prepared.contributions = PreparedContributions {
observer_entries,
commands,
@@ -326,7 +364,8 @@ impl PluginRuntime {
middleware_entries,
upstream_entries,
authentication_entries,
model_aliases,
model_aliases: catalog.aliases,
models: catalog.models,
};
Ok(Arc::new(prepared))
}
@@ -13,7 +13,7 @@ impl PluginRuntime {
.load_instances()
.await
.map_err(|_| AdminError::unavailable("无法读取已安装插件"))?;
let reference = self.prepare_snapshot(snapshot, None, None).await?;
let reference = self.prepare_snapshot(snapshot, None, None, None).await?;
let set = self.prepared_set(&reference).await?;
Ok(crate::PluginCommandSession::new(
reference,
@@ -8,7 +8,8 @@ use super::{
use crate::{
PackageLimits, RpcLimits, RpcSession,
callback::{
PluginAccountPortSlot, PluginAffinityPortSlot, PluginClientKeyPortSlot, PluginModelPortSlot,
PluginAccountPortSlot, PluginAffinityPortSlot, PluginClientIdentityPortSlot,
PluginClientKeyPortSlot, PluginModelPortSlot,
},
};
use gateway_admin::{
@@ -21,7 +22,7 @@ use gateway_admin::{
};
use gateway_core::{
engine::{
authentication::FrontendAuthenticationExtensionIndex,
authentication::{ClientIdentityPort, FrontendAuthenticationExtensionIndex},
extensions::ExecutionExtensionIndex,
nested::{AffinityLookupPort, NestedModelExecutionPort},
observation::RequestObserverExtensionIndex,
@@ -69,6 +70,7 @@ pub struct PluginRuntime {
pub(super) account_ports: Arc<PluginAccountPortSlot>,
pub(super) client_key_ports: Arc<PluginClientKeyPortSlot>,
pub(super) resource_ports: Arc<crate::callback::PluginResourcePorts>,
pub(super) client_identity_port: Arc<PluginClientIdentityPortSlot>,
pub(super) model_ports: Arc<PluginModelPortSlot>,
pub(super) affinity_ports: Arc<PluginAffinityPortSlot>,
pub(super) observers: RequestObserverExtensionIndex,
@@ -113,6 +115,7 @@ impl PluginRuntime {
account_ports: Arc::new(PluginAccountPortSlot::new()),
client_key_ports: Arc::new(PluginClientKeyPortSlot::new()),
resource_ports: Arc::new(crate::callback::PluginResourcePorts::new()),
client_identity_port: Arc::new(PluginClientIdentityPortSlot::new()),
model_ports: Arc::new(PluginModelPortSlot::new()),
affinity_ports: Arc::new(PluginAffinityPortSlot::new()),
observers: RequestObserverExtensionIndex::default(),
@@ -168,6 +171,14 @@ impl PluginRuntime {
self.client_key_ports.bind(access)
}
/// 身份查询独立装配,复用 Core 的认证与冻结权限,不要求绑定模型执行端口
pub fn bind_client_identity_port(
&self,
clients: &Arc<dyn ClientIdentityPort>,
) -> Result<(), AdminError> {
self.client_identity_port.bind(clients)
}
/// Core 激活后一次性绑定嵌套执行与亲和查询端口;Runtime 仅保存 Weak,避免组合根强环
pub fn bind_model_ports(
&self,
@@ -30,6 +30,7 @@ pub(super) struct PreparedSet {
pub(super) commands: Vec<Arc<crate::adapter::command_line::PluginCommand>>,
pub(super) management: Vec<crate::adapter::management::ManagementEntry>,
pub(super) model_aliases: Vec<gateway_core::routing::ContributedModelAlias>,
pub(super) models: Vec<gateway_core::routing::ContributedModel>,
}
#[derive(Clone, Default)]
@@ -43,6 +44,7 @@ pub(super) struct PreparedContributions {
pub(super) authentication_entries:
Vec<crate::adapter::frontend_authentication::FrontendAuthenticationEntry>,
pub(super) model_aliases: Vec<gateway_core::routing::ContributedModelAlias>,
pub(super) models: Vec<gateway_core::routing::ContributedModel>,
}
impl PreparedContributions {
@@ -51,6 +53,7 @@ impl PreparedContributions {
self.commands.extend(other.commands);
self.management.extend(other.management);
self.model_aliases.extend(other.model_aliases);
self.models.extend(other.models);
self.policy_entries.extend(other.policy_entries);
self.middleware_entries.extend(other.middleware_entries);
self.upstream_entries.extend(other.upstream_entries);
@@ -86,6 +89,10 @@ impl ExtensionSetLease for PreparedSet {
&self.model_aliases
}
fn models(&self) -> &[gateway_core::routing::ContributedModel] {
&self.models
}
fn can_serve(&self) -> bool {
// 发布的是可用能力和故障绑定组成的完整计划;单个进程退出不能使原生转发失效
!self.shutting_down.is_cancelled()
@@ -3,6 +3,7 @@
use super::{
PluginRuntime,
diagnostics::{PreparationDiagnostic, runtime_failure_from_admin},
instance::InstancePreparationError,
set::{PreparedContributions, PreparedSet, fingerprint, instance_fingerprint},
};
use crate::ValidatedPackage;
@@ -12,7 +13,10 @@ use gateway_admin::{
model::{
AdminError, Revision,
plugins::{
instances::{PluginInstance, PluginInstanceSnapshot},
instances::{
PluginCompatibilityConfirmation, PluginInstance, PluginInstanceRuntimeFailure,
PluginInstanceSnapshot,
},
state::PluginStateConfiguration,
},
},
@@ -35,13 +39,14 @@ impl PluginRuntime {
snapshot: PluginInstanceSnapshot,
required_revision: Option<Revision>,
request_body_limit: Option<usize>,
confirmation: Option<&PluginCompatibilityConfirmation>,
) -> Result<ExtensionSetReference, AdminError> {
let target_revision = snapshot.config_revision.get();
self.record_preparation(target_revision, PreparationDiagnostic::Preparing);
let result = tokio::select! {
biased;
() = self.shutting_down.cancelled() => Err(AdminError::unavailable("插件运行时正在关闭")),
result = self.prepare_inner(snapshot, required_revision, request_body_limit) => result,
result = self.prepare_inner(snapshot, required_revision, request_body_limit, confirmation) => result,
};
match &result {
Ok(reference) => self.record_preparation(
@@ -63,6 +68,7 @@ impl PluginRuntime {
snapshot: PluginInstanceSnapshot,
required_revision: Option<Revision>,
request_body_limit: Option<usize>,
confirmation: Option<&PluginCompatibilityConfirmation>,
) -> Result<ExtensionSetReference, AdminError> {
if self.shutting_down.is_cancelled() {
return Err(AdminError::unavailable("插件运行时正在关闭"));
@@ -70,6 +76,19 @@ impl PluginRuntime {
if snapshot.instances.len() > 64 {
return Err(AdminError::invalid("最多配置 64 个插件实例"));
}
if let Some(confirmation) = confirmation
&& (confirmation.host_fingerprint
!= crate::package::compatibility_host_fingerprint(&self.config.host_version)?
|| !snapshot.instances.iter().any(|instance| {
instance.enabled
&& instance.id == confirmation.instance_id
&& instance.artifact_sha256 == confirmation.artifact_sha256
}))
{
return Err(AdminError::conflict(
"宿主或插件包已变化,请重新确认启动风险",
));
}
let fingerprint = fingerprint(&snapshot)?;
// 准备串行化与弱索引分锁;诊断和已发布请求不会等待外部 I/O
let _prepare = self.prepare_lock.lock().await;
@@ -83,6 +102,10 @@ impl PluginRuntime {
.get(&fingerprint)
.and_then(Weak::upgrade)
.filter(|set| set.sessions_ready())
.filter(|set| {
confirmation
.is_none_or(|confirmed| !set.failures.contains_key(&confirmed.instance_id))
})
};
if let Some(set) = existing {
if snapshot.instances.iter().any(|instance| {
@@ -130,6 +153,7 @@ impl PluginRuntime {
instance.clone(),
snapshot.config_revision,
request_body_limit,
confirmation.is_some_and(|confirmed| confirmed.instance_id == instance.id),
)
.await
};
@@ -147,8 +171,40 @@ impl PluginRuntime {
contributions.append(candidate.contributions.clone());
sessions.push(candidate);
}
Err(error) if required_revision == Some(instance.revision) => return Err(error),
Err(error) => {
Err(error)
if required_revision == Some(instance.revision)
|| confirmation
.is_some_and(|confirmed| confirmed.instance_id == instance.id) =>
{
return Err(error.into_admin_error());
}
Err(
error @ (InstancePreparationError::UnloadablePackage(_)
| InstancePreparationError::IncompatiblePackage(_)),
) => {
let code = if matches!(&error, InstancePreparationError::IncompatiblePackage(_))
{
"version_incompatible"
} else {
"package_unloadable"
};
let error = error.into_admin_error();
tracing::error!(
event = "plugin.instance.quarantined",
instance_id = instance.id,
reason = error.message(),
"插件未通过启动兼容检查,未挂载任何能力"
);
// 包无法装载时没有可执行能力,不能发布全局拒绝占位阻断宿主
failures.insert(
instance.id.clone(),
PluginInstanceRuntimeFailure {
code: code.to_owned(),
message: error.message().to_owned(),
},
);
}
Err(InstancePreparationError::Failed(error)) => {
// 目录没有拒绝请求用的 binding,不能在恢复失败时悄悄撤销仍启用的别名
// 不复制其他候选的注册结果;让发布事务保留调用方持有的旧有效快照
let has_retained_catalog = self
@@ -161,6 +217,7 @@ impl PluginRuntime {
set.model_aliases
.iter()
.any(|alias| alias.owner == instance.id)
|| set.models.iter().any(|model| model.owner == instance.id)
});
if has_retained_catalog {
return Err(AdminError::unavailable(
@@ -195,7 +252,9 @@ impl PluginRuntime {
upstream_entries,
authentication_entries,
model_aliases,
models,
} = contributions;
crate::adapter::catalog::validate_models(&model_aliases, &models)?;
let id = ExtensionSetId::new(uuid::Uuid::new_v4().to_string())
.map_err(|_| AdminError::internal("插件集合 ID 无效"))?;
let observers = crate::adapter::observer::PluginObserverPlan::compile(
@@ -260,6 +319,7 @@ impl PluginRuntime {
management,
failures,
model_aliases,
models,
shutting_down: self.shutting_down.clone(),
});
self.prepared
@@ -360,7 +420,17 @@ impl PluginPreparation for PluginRuntime {
snapshot: PluginInstanceSnapshot,
) -> Result<ExtensionSetReference, AdminError> {
let required_revision = snapshot.config_revision;
self.prepare_snapshot(snapshot, Some(required_revision), None)
self.prepare_snapshot(snapshot, Some(required_revision), None, None)
.await
}
async fn prepare_confirmed(
&self,
snapshot: PluginInstanceSnapshot,
confirmation: PluginCompatibilityConfirmation,
) -> Result<ExtensionSetReference, AdminError> {
let required_revision = snapshot.config_revision;
self.prepare_snapshot(snapshot, Some(required_revision), None, Some(&confirmation))
.await
}
}
@@ -397,7 +467,7 @@ impl PluginRuntime {
if snapshot.config_revision.get() != revision.get() {
return Err(ExtensionPreparationError);
}
self.prepare_snapshot(snapshot, None, request_body_limit)
self.prepare_snapshot(snapshot, None, request_body_limit, None)
.await
.map_err(|_| ExtensionPreparationError)
})
@@ -69,19 +69,16 @@ pub(crate) fn warning(
requirements: &PluginCompatibilityRequirements,
host: &semver::Version,
) -> Result<Option<String>, AdminError> {
let compatibility = host_compatibility()?;
let mut warnings = Vec::new();
if !semver::VersionReq::parse(&requirements.host_version).is_ok_and(|range| range.matches(host))
{
warnings.push(format!(
"插件声明的宿主范围为 {},当前为 {host}",
requirements.host_version
));
}
for (capability, version) in &requirements.capabilities {
if !compatibility.supports_capability(capability, *version) {
warnings.push(format!("{capability} v{version} 不在宿主支持范围内"));
}
}
let warnings = host_compatibility()?.incompatibilities(requirements, host);
Ok((!warnings.is_empty()).then(|| warnings.join(",")))
}
pub(crate) fn host_fingerprint(host: &semver::Version) -> Result<String, AdminError> {
use sha2::{Digest, Sha256};
host_compatibility()?;
Ok(hex::encode(Sha256::digest(format!(
"{host}\n{}\n{}\n{HOST_COMPATIBILITY_JSON}",
std::env::consts::OS,
std::env::consts::ARCH,
))))
}
@@ -120,6 +120,10 @@ impl PackageInspector {
#[async_trait]
impl PluginPackageInspector for PackageInspector {
fn compatibility_host_fingerprint(&self) -> Result<String, AdminError> {
super::compatibility::host_fingerprint(&self.host_version)
}
async fn compatibility_warning(
&self,
archive: Arc<[u8]>,
@@ -200,6 +204,10 @@ impl PluginPackageInspector for PackageInspector {
let _permit = permit;
let package = ValidatedPackage::read(archive, Some(&expected_sha256), limits)
.map_err(inspection_error)?;
package
.manifest()
.package_for_platform(std::env::consts::OS, std::env::consts::ARCH)
.map_err(|error| inspection_error(error.into()))?;
super::compatibility::requirements(package.manifest())
})
.await
@@ -207,12 +215,22 @@ impl PluginPackageInspector for PackageInspector {
}
}
fn inspection_error(error: PackageError) -> AdminError {
pub(crate) fn inspection_error(error: PackageError) -> AdminError {
match error {
PackageError::Manifest(gateway_plugin_sdk::ManifestError::Incompatible) => {
AdminError::invalid("插件清单或协议版本不兼容,请使用当前 SDK 重新构建")
}
_ => AdminError::invalid("插件包格式、兼容性或摘要校验失败"),
PackageError::Manifest(gateway_plugin_sdk::ManifestError::Platform) => {
AdminError::invalid("插件不支持当前运行平台")
}
PackageError::Manifest(gateway_plugin_sdk::ManifestError::Invalid) => {
AdminError::invalid("插件清单格式不合法")
}
PackageError::Digest => AdminError::invalid("插件包摘要校验失败,请重新安装"),
PackageError::Archive => AdminError::invalid("插件包损坏或归档格式不合法"),
PackageError::Path => AdminError::invalid("插件包包含不安全或重复路径"),
PackageError::Limit => AdminError::invalid("插件包超过大小或文件数量限制"),
PackageError::Cache => AdminError::unavailable("插件制品缓存暂不可用"),
}
}
@@ -8,7 +8,9 @@ mod validation;
pub use cache::PreparedPackage;
pub(crate) use compatibility::{
requirements as compatibility_requirements, warning as compatibility_warning,
host_fingerprint as compatibility_host_fingerprint, requirements as compatibility_requirements,
warning as compatibility_warning,
};
pub use inspection::PackageInspector;
pub(crate) use inspection::inspection_error;
pub use validation::{PackageError, PackageLimits, ValidatedPackage};
@@ -47,4 +47,11 @@ impl ExecutionCapacity {
self.active.close();
self.queued.close();
}
pub(super) fn available(&self) -> (usize, usize) {
(
self.active.available_permits(),
self.queued.available_permits(),
)
}
}
@@ -0,0 +1,671 @@
//! RPC 调用表唯一持有调度、回调、响应流和取消确认,统一完成调用资源回收
use std::{
collections::BTreeMap,
sync::{
Arc, Mutex,
atomic::{AtomicBool, Ordering},
},
time::{Duration, Instant},
};
use gateway_host::process::ProcessControl;
use gateway_plugin_sdk::{
CallContext, Frame, Message, PluginFault,
client::{CallDeadline, CallDeadlinePause, EncodedFrame},
};
use tokio::sync::{Notify, OwnedSemaphorePermit, Semaphore, mpsc, oneshot, watch};
use super::{
CallbackHandler, RpcError, RpcLimits, RpcReply, RpcSessionLoad, admission::ExecutionCapacity,
};
use crate::stream::StreamIngress;
pub(crate) struct Shared {
state: Mutex<State>,
started_at: Instant,
pub(super) expected_stop: AtomicBool,
failure_observer: Option<Arc<dyn Fn(Duration) + Send + Sync>>,
pub control: mpsc::Sender<EncodedFrame>,
pub process: ProcessControl,
pub stopped: watch::Sender<bool>,
callback_slots: Arc<Semaphore>,
pub(super) execution: ExecutionCapacity,
cancellations_changed: Notify,
}
struct State {
failure: Option<RpcError>,
calls: BTreeMap<u64, PendingCall>,
}
struct PendingCall {
_slot: Arc<OwnedSemaphorePermit>,
execution: Option<OwnedSemaphorePermit>,
deadline: CallDeadline,
handler: Arc<dyn CallbackHandler>,
revoked: bool,
transmitted: bool,
cancellation: Option<RpcError>,
context: CallContext,
result: Option<oneshot::Sender<Result<RpcReply, RpcError>>>,
stream: Option<StreamIngress>,
scheduling: Option<Scheduling>,
callbacks: BTreeMap<u64, CallbackTask>,
}
// 调度状态只属于父任务;回调完成与恢复授权分别推进
enum Scheduling {
Running {
epoch: u64,
},
Suspended {
epoch: u64,
pause: Option<CallDeadlinePause>,
},
Resuming {
epoch: u64,
pause: Option<CallDeadlinePause>,
task: tokio::task::AbortHandle,
},
}
struct CallbackTask {
task: tokio::task::AbortHandle,
pauses_budget: bool,
_permit: OwnedSemaphorePermit,
}
impl Drop for CallbackTask {
fn drop(&mut self) {
self.task.abort();
}
}
impl PendingCall {
/// 关闭资源与封存副作用必须先于任何结果通知;取消确认只延续容量租约
fn revoke(&mut self) {
if self.revoked {
return;
}
self.revoked = true;
self.handler.finish(&self.context);
self.callbacks.clear();
self.stop_resumption();
}
fn stop_resumption(&mut self) {
if let Some(Scheduling::Resuming { task, .. }) = self.scheduling.take() {
task.abort();
}
}
fn finish(mut self, result: Result<RpcReply, RpcError>) {
tracing::debug!(
event = "plugin.rpc.finished",
instance_id = self.context.instance_id,
call_id = self.context.call_id,
stage = ?self.context.stage,
outcome = result.as_ref().map_or_else(|error| error.diagnostic().0, |_| "success"),
"插件 RPC 调用结束"
);
let stream = self.stream.take();
let sender = self.result.take();
// 先同步关闭调用资源,Core 才能在收到结果时看到全部发送副作用
drop(self);
if let Some(stream) = stream {
stream.finish(result.as_ref().map(|_| ()).map_err(Clone::clone));
}
if let Some(sender) = sender {
let _ = sender.send(result);
}
}
}
impl Drop for PendingCall {
fn drop(&mut self) {
self.revoke();
}
}
impl Shared {
pub(super) fn new(
limits: RpcLimits,
process: ProcessControl,
control: mpsc::Sender<EncodedFrame>,
failure_observer: Option<Arc<dyn Fn(Duration) + Send + Sync>>,
) -> Self {
let (stopped, _) = watch::channel(false);
Self {
state: Mutex::new(State {
failure: None,
calls: BTreeMap::new(),
}),
started_at: Instant::now(),
expected_stop: AtomicBool::new(false),
failure_observer,
control,
process,
stopped,
callback_slots: Arc::new(Semaphore::new(limits.maximum_callbacks)),
execution: ExecutionCapacity::new(
limits.maximum_executing_calls,
limits.maximum_queued_calls,
limits.maximum_queue_timeout,
),
cancellations_changed: Notify::new(),
}
}
pub(super) fn failure_diagnostic(&self) -> Option<(&'static str, &'static str)> {
self.state().failure.as_ref().map(RpcError::diagnostic)
}
pub(super) fn load(&self, limits: RpcLimits, available_calls: usize) -> RpcSessionLoad {
let state = self.state();
let (available_execution, available_queue) = self.execution.available();
RpcSessionLoad {
active_calls: limits.maximum_calls - available_calls,
executing_calls: limits.maximum_executing_calls - available_execution,
queued_calls: limits.maximum_queued_calls - available_queue,
resuming_calls: state
.calls
.values()
.filter(|call| matches!(call.scheduling, Some(Scheduling::Resuming { .. })))
.count(),
pending_callbacks: limits.maximum_callbacks - self.callback_slots.available_permits(),
active_streams: state
.calls
.values()
.filter(|call| call.stream.is_some() && call.result.is_none())
.count(),
}
}
pub(super) fn begin_call(
&self,
context: CallContext,
deadline: tokio::time::Instant,
stream: Option<StreamIngress>,
slot: Arc<OwnedSemaphorePermit>,
execution: OwnedSemaphorePermit,
handler: Arc<dyn CallbackHandler>,
) -> Result<StartedCall, RpcError> {
let id = context.call_id;
let (result, received) = oneshot::channel();
let mut state = self.state();
if let Some(error) = &state.failure {
return Err(error.clone());
}
let deadline = CallDeadline::new(deadline);
handler.begin(&context, &deadline);
state.calls.insert(
id,
PendingCall {
_slot: slot,
execution: Some(execution),
deadline: deadline.clone(),
handler,
revoked: false,
transmitted: false,
cancellation: None,
context,
result: Some(result),
stream,
scheduling: Some(Scheduling::Running { epoch: 0 }),
callbacks: BTreeMap::new(),
},
);
Ok(StartedCall {
id,
deadline,
received,
})
}
/// 替换维护任务前等待已取消的调用确认;未确认仍沿用会话的有界故障回收
pub(crate) async fn settle_cancellations(&self) {
let ids: Vec<_> = self
.state()
.calls
.iter()
.filter_map(|(id, call)| call.cancellation.is_some().then_some(*id))
.collect();
loop {
let changed = self.cancellations_changed.notified();
tokio::pin!(changed);
changed.as_mut().enable();
if !ids.iter().any(|id| self.is_cancelling(*id)) {
break;
}
changed.await;
}
if self.failure_diagnostic().is_some() {
self.process.exited().await;
}
}
fn state(&self) -> std::sync::MutexGuard<'_, State> {
self.state
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
}
pub fn fail(&self, error: RpcError) {
let uptime = self.started_at.elapsed();
let (calls, failure_observer) = {
let mut state = self.state();
if state.failure.is_some() {
return;
}
state.failure = Some(error.clone());
let observer = (!self.expected_stop.load(Ordering::Acquire))
.then(|| self.failure_observer.clone())
.flatten();
// 消费者可能正在析构已有流;调用对查询者不可见前必须完成同步撤销
for call in state.calls.values_mut() {
call.revoke();
}
(std::mem::take(&mut state.calls), observer)
};
self.execution.close();
for call in calls.into_values() {
call.finish(Err(error.clone()));
}
if let Some(observer) = failure_observer {
observer(uptime);
}
self.stopped.send_replace(true);
self.process.stop();
self.cancellations_changed.notify_waiters();
}
pub(super) fn expect_stop(&self) {
self.expected_stop.store(true, Ordering::Release);
}
pub fn finish(&self, id: u64, result: Result<RpcReply, RpcError>) -> Result<(), RpcError> {
let mut state = self.state();
let call = state.calls.get_mut(&id).ok_or(RpcError::Protocol(None))?;
if call.cancellation.is_some() {
return Ok(());
}
if matches!(&result, Err(RpcError::Remote(fault)) if fault.code == gateway_plugin_sdk::ErrorCode::Timeout)
{
log_timeout(&call.context, "plugin", "initial_reply", "remote_timeout");
}
if result.is_ok() && !matches!(call.scheduling, Some(Scheduling::Running { .. })) {
return Err(RpcError::Protocol(None));
}
if call.stream.is_some() && result.is_ok() {
if call.context.resource_stream {
call.deadline.clear();
}
let initial = call.result.take().ok_or(RpcError::Protocol(None))?;
call.execution.take();
call.scheduling.take();
call.handler.streaming(&call.context);
drop(state);
let _ = initial.send(result);
} else {
let mut pending = state.calls.remove(&id).ok_or(RpcError::Protocol(None))?;
pending.revoke();
drop(state);
pending.finish(result);
}
Ok(())
}
pub fn stream_chunk(&self, id: u64, sequence: u64, payload: Vec<u8>) -> Result<(), RpcError> {
let mut state = self.state();
let call = state.calls.get_mut(&id).ok_or(RpcError::Protocol(None))?;
if call.cancellation.is_some() {
return Ok(());
}
if call.result.is_some() {
return Err(RpcError::Protocol(None));
}
call.stream
.as_mut()
.ok_or(RpcError::Protocol(None))?
.receive(sequence, payload)
}
pub fn stream_end(&self, id: u64, error: Option<PluginFault>) -> Result<(), RpcError> {
let mut state = self.state();
if state
.calls
.get(&id)
.is_some_and(|call| call.cancellation.is_some())
{
return Ok(());
}
let mut call = state.calls.remove(&id).ok_or(RpcError::Protocol(None))?;
call.revoke();
drop(state);
if call.stream.is_none() || call.result.is_some() {
call.finish(Err(RpcError::Protocol(None)));
return Err(RpcError::Protocol(None));
}
let result = error.map_or_else(
|| {
Ok(RpcReply {
result: serde_json::Value::Null,
payload: Vec::new(),
})
},
|error| Err(RpcError::Remote(error)),
);
call.finish(result);
Ok(())
}
pub fn release_stream_credit(&self, id: u64, bytes: u32) -> Result<(), RpcError> {
let mut state = self.state();
// 完成后仍可消费已排队数据,无须再给已关闭的流发信用
let Some(call) = state.calls.get_mut(&id) else {
return Ok(());
};
call.stream
.as_mut()
.ok_or(RpcError::Protocol(None))?
.release(bytes)?;
drop(state);
self.send_control(Frame::control(Message::Credit {
id,
bytes,
frames: 1,
}));
Ok(())
}
pub fn context(&self, id: u64) -> Option<CallContext> {
self.state()
.calls
.get(&id)
.filter(|call| call.cancellation.is_none())
.map(|call| call.context.clone())
}
pub fn is_cancelling(&self, id: u64) -> bool {
self.state()
.calls
.get(&id)
.is_some_and(|call| call.cancellation.is_some())
}
pub fn mark_transmitted(&self, id: u64) -> bool {
let mut state = self.state();
let Some(call) = state.calls.get_mut(&id) else {
return false;
};
if call.cancellation.is_some() {
return false;
}
call.transmitted = true;
true
}
/// 先撤销调用权限,再等待对端确认;只有无响应对端才终止整个 incarnation
pub fn cancel(self: &Arc<Self>, id: u64, error: RpcError) {
let mut state = self.state();
let Some(call) = state.calls.get_mut(&id) else {
return;
};
if call.cancellation.is_some() {
return;
}
if matches!(error, RpcError::Timeout) {
log_timeout(
&call.context,
"runtime",
if call.result.is_some() {
"initial_reply"
} else {
"response_stream"
},
"processing_budget_exhausted",
);
}
if !call.transmitted {
call.revoke();
let call = state.calls.remove(&id);
drop(state);
if let Some(call) = call {
call.finish(Err(error));
}
return;
}
call.cancellation = Some(error.clone());
// 同步撤销不能跨越取消确认:收到确认的并发任务可能立即复用实例
// CallbackHandler::finish 合同要求非阻塞;真正的等待与结果交付放在锁外
call.revoke();
let result = call.result.take();
let stream = call.stream.take();
tracing::debug!(
event = "plugin.rpc.cancelled",
instance_id = call.context.instance_id,
call_id = id,
stage = ?call.context.stage,
reason = error.diagnostic().0,
"插件 RPC 调用已撤销,等待进程确认"
);
drop(state);
if let Some(stream) = stream {
stream.finish(Err(error.clone()));
}
if let Some(result) = result {
let _ = result.send(Err(error.clone()));
}
self.send_control(Frame::control(Message::Cancel { id }));
let shared = Arc::clone(self);
tokio::spawn(async move {
tokio::time::sleep(Duration::from_millis(250)).await;
if shared.is_cancelling(id) {
shared.fail(error);
}
});
}
pub fn acknowledge_cancellation(&self, id: u64) -> Result<(), RpcError> {
let mut state = self.state();
if !state
.calls
.get(&id)
.is_some_and(|call| call.cancellation.is_some())
{
return Err(RpcError::Protocol(None));
}
let call = state.calls.remove(&id);
drop(state);
drop(call);
self.cancellations_changed.notify_waiters();
Ok(())
}
pub fn track_callback(
&self,
parent_id: u64,
id: u64,
method: &str,
task: tokio::task::AbortHandle,
permit: OwnedSemaphorePermit,
) -> bool {
let mut state = self.state();
let Some(call) = state
.calls
.get_mut(&parent_id)
.filter(|call| call.cancellation.is_none())
else {
task.abort();
return false;
};
let pauses_budget = method == gateway_plugin_sdk::call::middleware::NEXT_METHOD;
if !pauses_budget
&& let Some(Scheduling::Suspended { pause, .. } | Scheduling::Resuming { pause, .. }) =
call.scheduling.as_mut()
{
pause.take();
}
call.callbacks.insert(
id,
CallbackTask {
task,
pauses_budget,
_permit: permit,
},
);
true
}
pub(super) fn consume_callback(&self, parent_id: u64, id: u64) -> Result<(), RpcError> {
if id == 0 || !id.is_multiple_of(2) || parent_id == 0 || parent_id.is_multiple_of(2) {
return Err(RpcError::Protocol(None));
}
// 取消确认可以与消费确认交错;确认只能撤销资源,不能重新创建调用
if let Some(call) = self.state().calls.get_mut(&parent_id) {
call.callbacks.remove(&id);
}
Ok(())
}
pub(super) fn suspend(&self, id: u64, epoch: u64) -> Result<(), RpcError> {
if id == 0 || id.is_multiple_of(2) || epoch == 0 {
return Err(RpcError::Protocol(None));
}
let mut state = self.state();
let Some(call) = state.calls.get_mut(&id) else {
return Ok(());
};
if call.cancellation.is_some() {
return Ok(());
}
match call.scheduling.as_ref() {
Some(Scheduling::Running { epoch: previous })
if previous.checked_add(1) == Some(epoch) => {}
_ => return Err(RpcError::Protocol(None)),
}
let pauses = !call.callbacks.is_empty()
&& call
.callbacks
.values()
.all(|callback| callback.pauses_budget);
let pause = pauses.then(|| call.deadline.pause()).flatten();
call.execution.take();
call.scheduling = Some(Scheduling::Suspended { epoch, pause });
Ok(())
}
pub(super) fn request_resume(self: &Arc<Self>, id: u64, epoch: u64) -> Result<(), RpcError> {
if id == 0 || id.is_multiple_of(2) || epoch == 0 {
return Err(RpcError::Protocol(None));
}
let mut state = self.state();
let Some(call) = state.calls.get_mut(&id) else {
return Ok(());
};
if call.cancellation.is_some() {
return Ok(());
}
if !matches!(call.scheduling, Some(Scheduling::Suspended { epoch: current, .. }) if current == epoch)
{
return Err(RpcError::Protocol(None));
}
let Some(Scheduling::Suspended { pause, .. }) = call.scheduling.take() else {
return Err(RpcError::Protocol(None));
};
let shared = Arc::clone(self);
let (ready, started) = oneshot::channel();
let task = tokio::spawn(async move {
if started.await.is_err() {
return;
}
match shared.execution.resume().await {
Ok(permit) => shared.grant_resume(id, epoch, permit),
Err(error) => shared.cancel(id, error),
}
});
call.scheduling = Some(Scheduling::Resuming {
epoch,
pause,
task: task.abort_handle(),
});
let _ = ready.send(());
Ok(())
}
fn grant_resume(&self, id: u64, epoch: u64, permit: OwnedSemaphorePermit) {
let mut state = self.state();
let Some(call) = state
.calls
.get_mut(&id)
.filter(|call| call.cancellation.is_none())
else {
return;
};
if !matches!(call.scheduling, Some(Scheduling::Resuming { epoch: current, .. }) if current == epoch)
{
return;
}
call.execution = Some(permit);
call.scheduling = Some(Scheduling::Running { epoch });
drop(state);
self.send_control(Frame::control(Message::Resumed { id, epoch }));
}
pub fn send_control(&self, frame: Frame) {
let Ok(frame) = EncodedFrame::new(frame) else {
self.fail(RpcError::Protocol(None));
return;
};
self.send_encoded_control(frame);
}
pub(super) fn send_encoded_control(&self, frame: EncodedFrame) {
if self.control.try_send(frame).is_err() {
self.fail(RpcError::Capacity);
}
}
pub(super) fn try_callback_slot(&self) -> Option<OwnedSemaphorePermit> {
self.callback_slots.clone().try_acquire_owned().ok()
}
pub(super) async fn wait_for_callbacks(&self, limits: RpcLimits) {
if let Ok(permits) = self
.callback_slots
.clone()
.acquire_many_owned(limits.maximum_callbacks as u32)
.await
{
drop(permits);
}
}
}
pub(super) struct StartedCall {
pub(super) id: u64,
pub(super) deadline: CallDeadline,
pub(super) received: oneshot::Receiver<Result<RpcReply, RpcError>>,
}
fn log_timeout(
context: &CallContext,
owner: &'static str,
phase: &'static str,
reason: &'static str,
) {
tracing::warn!(
event = "plugin.rpc.timeout",
owner,
phase,
instance_id = context.instance_id,
call_id = context.call_id,
request_id = context.request_id,
attempt_id = context.attempt_id,
stage = ?context.stage,
budget_ms = context.timeout_ms,
reason,
"插件 RPC 调用超时"
);
}
@@ -0,0 +1,192 @@
//! RPC 会话的容量、错误、回调与生命周期合同
use std::{sync::Arc, time::Duration};
use futures::future::BoxFuture;
use gateway_host::process::ProcessStartError;
use gateway_plugin_sdk::{CallBudgetError, CallContext, PluginFault, Stage, client::CallDeadline};
#[derive(Debug, Clone, Copy)]
pub struct RpcLimits {
/// 受管回调与观察队列的缓冲预算,不是请求正文或 IPC 消息总量上限
pub maximum_buffered_body_bytes: usize,
/// 包含初次排队、下游等待、取消确认和未消费流的总在途上限
pub maximum_calls: usize,
pub maximum_executing_calls: usize,
pub maximum_queued_calls: usize,
pub maximum_queue_timeout: Duration,
pub maximum_callbacks: usize,
pub handshake_timeout: Duration,
pub maximum_stderr_bytes: usize,
}
impl Default for RpcLimits {
fn default() -> Self {
Self {
maximum_buffered_body_bytes: 1024 * 1024,
maximum_calls: 256,
maximum_executing_calls: 64,
maximum_queued_calls: 64,
maximum_queue_timeout: Duration::from_secs(30),
maximum_callbacks: 320,
handshake_timeout: Duration::from_secs(5),
maximum_stderr_bytes: 64 * 1024,
}
}
}
#[derive(Debug, Clone, thiserror::Error)]
pub enum RpcError {
#[error("plugin process could not be started: {0}")]
Start(#[source] ProcessStartError),
#[error("plugin handshake does not match the prepared instance")]
Handshake(#[source] Option<gateway_core::error::ErrorSource>),
#[error("plugin call budget negotiation failed: {0}")]
Budget(#[from] CallBudgetError),
#[error("plugin protocol is invalid")]
Protocol(#[source] Option<gateway_core::error::ErrorSource>),
#[error("plugin returned an invalid response at {0:?}")]
InvalidResponse(Stage, #[source] Option<gateway_core::error::ErrorSource>),
#[error("plugin process or transport has stopped")]
Closed(#[source] Option<gateway_core::error::ErrorSource>),
#[error("plugin call exceeded its deadline")]
Timeout,
#[error("plugin session was cancelled")]
Cancelled,
#[error("plugin call capacity is exhausted")]
Capacity,
#[error("plugin admission capacity is exhausted")]
Overloaded,
#[error("plugin call context is invalid")]
Context(#[source] Option<gateway_core::error::ErrorSource>),
#[error("plugin returned an error")]
Remote(PluginFault),
}
impl RpcError {
pub(crate) fn is_overloaded(&self) -> bool {
matches!(self, Self::Overloaded)
|| matches!(self, Self::Remote(fault) if fault.code == gateway_plugin_sdk::ErrorCode::Capacity)
}
/// 会话状态与请求错误共用安全分类,不展开插件返回正文
pub(crate) fn diagnostic(&self) -> (&'static str, &'static str) {
match self {
RpcError::Start(_) => ("process_start", "插件进程启动失败"),
RpcError::Handshake(_) => ("handshake", "插件握手失败"),
RpcError::Budget(_) => ("call_budget", "插件调用预算协商失败"),
RpcError::Protocol(_) => ("protocol", "插件协议错误"),
RpcError::InvalidResponse(stage, _) => (
"invalid_response",
match stage {
Stage::Registration => "注册阶段:插件返回了无效响应",
Stage::Configuration => "配置阶段:插件返回了无效响应",
Stage::Authentication => "认证阶段:插件返回了无效响应",
Stage::Routing => "路由阶段:插件返回了无效响应",
Stage::Scheduling => "调度阶段:插件返回了无效响应",
Stage::Retry => "重试阶段:插件返回了无效响应",
Stage::Http => "HTTP 中间件:插件返回了无效响应",
Stage::Service => "服务中间件:插件返回了无效响应",
Stage::WebSocket => "WebSocket 中间件:插件返回了无效响应",
Stage::Request => "请求中间件:插件返回了无效响应",
Stage::Attempt => "尝试中间件:插件返回了无效响应",
Stage::Outbound => "出站准备中间件:插件返回了无效响应",
Stage::Upstream => "上游适配阶段:插件返回了无效响应",
Stage::Observation => "观察阶段:插件返回了无效响应",
Stage::Management | Stage::PublicManagement => "管理阶段:插件返回了无效响应",
Stage::CommandLine => "命令行阶段:插件返回了无效响应",
Stage::Maintenance => "维护阶段:插件返回了无效响应",
},
),
RpcError::Closed(_) => ("closed", "插件进程或传输已停止"),
RpcError::Timeout => ("timeout", "插件调用超时"),
RpcError::Cancelled => ("cancelled", "插件会话已取消"),
RpcError::Capacity => ("capacity", "插件调用容量已耗尽"),
RpcError::Overloaded => ("overloaded", "插件暂时繁忙"),
RpcError::Context(_) => ("context", "插件调用上下文无效"),
RpcError::Remote(_) => ("remote", "插件返回错误"),
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum RpcSessionDiagnostic {
Ready,
Quiescing,
Failed {
code: &'static str,
message: &'static str,
},
}
pub(crate) struct RpcSessionLifecycle {
pub(super) failure: Arc<dyn Fn(Duration) + Send + Sync>,
end: Option<Box<dyn FnOnce() + Send + Sync>>,
}
impl RpcSessionLifecycle {
pub(crate) fn new(
failure: impl Fn(Duration) + Send + Sync + 'static,
end: impl FnOnce() + Send + Sync + 'static,
) -> Self {
Self {
failure: Arc::new(failure),
end: Some(Box::new(end)),
}
}
pub(super) fn fail(&self, uptime: Duration) {
(self.failure)(uptime);
}
}
/// 从实际租约与调用表采样,各项存在重叠,不构成同一时刻的可加总分区
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) struct RpcSessionLoad {
/// 总在途租约,包含初次排队、取消确认和 End 后仍由消费者持有的流
pub(crate) active_calls: usize,
/// 正在占用执行名额的调用,包含取消确认前尚未归还的名额
pub(crate) executing_calls: usize,
/// 正在占用初次准入排队名额的调用
pub(crate) queued_calls: usize,
/// 调用表中等待恢复授权的调用
pub(crate) resuming_calls: usize,
/// 驻留回调名额,包含已回复但插件尚未消费的结果
pub(crate) pending_callbacks: usize,
/// 已交付首结果且尚未收到 End 或取消的协议响应流
pub(crate) active_streams: usize,
}
impl Drop for RpcSessionLifecycle {
fn drop(&mut self) {
if let Some(end) = self.end.take() {
end();
}
}
}
/// 不派生 Debug,防止原始请求、凭据或模型输出进入宿主诊断
pub struct RpcReply {
pub result: serde_json::Value,
pub payload: Vec<u8>,
}
/// 具体回调在领域适配器校验账号、出站与执行身份;上下文只取自宿主在途调用表
pub trait CallbackHandler: Send + Sync {
/// 在 Call 可见之前建立资源范围;不得执行 I/O 或调用 RPC
fn begin(&self, _context: &CallContext, _deadline: &CallDeadline) {}
/// 调用结束或取消时立即撤销其句柄;实现必须幂等且不能阻塞
fn finish(&self, _context: &CallContext) {}
/// 首个流式结果已到达;资源仍绑定原调用并在终态统一回收
fn streaming(&self, _context: &CallContext) {}
fn call(
&self,
context: CallContext,
method: String,
params: serde_json::Value,
payload: Vec<u8>,
) -> BoxFuture<'static, Result<RpcReply, PluginFault>>;
}
@@ -4,7 +4,6 @@ use std::sync::Arc;
use gateway_plugin_sdk::{
ErrorCode, Frame, Message, PluginFault,
call::middleware::NEXT_METHOD,
client::{EncodedFrame, read_frame},
};
use tokio::{
@@ -12,7 +11,7 @@ use tokio::{
sync::{mpsc, oneshot},
};
use super::session::{CallbackHandler, RpcError, RpcReply, Shared};
use super::{CallbackHandler, RpcError, RpcReply, Shared};
pub(super) fn start<W, R>(
writer: W,
@@ -100,6 +99,13 @@ async fn read_loop<R: AsyncRead + Unpin>(
}
};
let result = match frame.message {
Message::Suspend { id, epoch } if frame.payload.is_empty() => shared.suspend(id, epoch),
Message::Resume { id, epoch } if frame.payload.is_empty() => {
shared.request_resume(id, epoch)
}
Message::CallbackConsumed { id, parent_id } if frame.payload.is_empty() => {
shared.consume_callback(parent_id, id)
}
Message::Cancelled { id } if frame.payload.is_empty() => {
shared.acknowledge_cancellation(id)
}
@@ -125,26 +131,17 @@ async fn read_loop<R: AsyncRead + Unpin>(
Err(RpcError::Protocol(None))
} else if let Some(context) = shared.context(parent_id) {
last_callback = id;
if let Some(permit) = shared.try_callback_slot(&method) {
if let Some(permit) = shared.try_callback_slot() {
let handler = Arc::clone(&callbacks);
let response = Arc::clone(&shared);
let (ready, started) = oneshot::channel();
let pause = (method == NEXT_METHOD)
.then(|| shared.pause_next(parent_id))
.flatten();
let callback_method = method.clone();
let task = tokio::spawn(async move {
// 先把任务归属登记到父调用,再允许执行宿主操作
if started.await.is_err() {
return;
}
let result = handler.call(context, method, params, frame.payload).await;
if pause.is_some()
&& let Err(error) = response.resume_next(parent_id).await
{
response.cancel(parent_id, error);
return;
}
drop(pause);
let reply = match result {
Ok(reply) => Frame {
message: Message::Result {
@@ -166,9 +163,14 @@ async fn read_loop<R: AsyncRead + Unpin>(
),
})),
}
drop(permit);
});
if shared.track_callback(parent_id, task.abort_handle()) {
if shared.track_callback(
parent_id,
id,
&callback_method,
task.abort_handle(),
permit,
) {
let _ = ready.send(());
}
} else {
@@ -1,8 +1,12 @@
//! 插件双向 RPC 会话、调用结果与生命周期的公共入口
mod admission;
mod calls;
mod contract;
mod dispatch;
mod session;
pub use session::{CallbackHandler, RpcError, RpcLimits, RpcReply, RpcSession};
pub(crate) use session::{RpcSessionDiagnostic, RpcSessionLifecycle, Shared};
pub(crate) use calls::Shared;
pub use contract::{CallbackHandler, RpcError, RpcLimits, RpcReply};
pub(crate) use contract::{RpcSessionDiagnostic, RpcSessionLifecycle, RpcSessionLoad};
pub use session::RpcSession;
@@ -1,197 +1,24 @@
//! 插件 RPC 会话的调用登记、超时取消、故障传播与关闭管理
//! 插件 RPC 会话的握手、对外调用、准入和进程关闭协调
use std::{
collections::BTreeMap,
sync::{
Arc, Mutex,
atomic::{AtomicBool, Ordering},
},
sync::{Arc, atomic::Ordering},
time::{Duration, Instant},
};
use futures::future::BoxFuture;
use gateway_host::process::{ProcessControl, ProcessSpec, ProcessStartError, ProcessSupervisor};
use gateway_host::process::{ProcessSpec, ProcessStartError, ProcessSupervisor};
use gateway_plugin_sdk::{
CallBudgetError, CallBudgetSource, CallContext, Frame, Handshake, Message, PROTOCOL_VERSION,
PluginFault, ResolvedCallBudgets, Stage,
client::{CallDeadline, CallDeadlinePause, EncodedFrame, read_frame, write_frame},
};
use tokio::sync::{
Mutex as AsyncMutex, Notify, OwnedSemaphorePermit, Semaphore, mpsc, oneshot, watch,
CallBudgetSource, CallContext, Frame, Handshake, Message, PROTOCOL_VERSION,
ResolvedCallBudgets, Stage,
client::{CallDeadline, EncodedFrame, read_frame, write_frame},
};
use tokio::sync::{Mutex as AsyncMutex, OwnedSemaphorePermit, Semaphore, mpsc};
use super::{
CallbackHandler, RpcError, RpcLimits, RpcReply, RpcSessionDiagnostic, RpcSessionLifecycle,
RpcSessionLoad, Shared, dispatch,
};
use crate::{PreparedPackage, RpcStream, stream::StreamIngress};
use super::{admission::ExecutionCapacity, dispatch};
#[derive(Debug, Clone, Copy)]
pub struct RpcLimits {
/// 受管回调与观察队列的缓冲预算,不是请求正文或 IPC 消息总量上限
pub maximum_buffered_body_bytes: usize,
/// 包含初次排队、下游等待、取消确认和未消费流的总在途上限
pub maximum_calls: usize,
pub maximum_executing_calls: usize,
pub maximum_queued_calls: usize,
pub maximum_queue_timeout: Duration,
pub maximum_callbacks: usize,
pub handshake_timeout: Duration,
pub maximum_stderr_bytes: usize,
}
impl Default for RpcLimits {
fn default() -> Self {
Self {
maximum_buffered_body_bytes: 1024 * 1024,
maximum_calls: 256,
maximum_executing_calls: 64,
maximum_queued_calls: 64,
maximum_queue_timeout: Duration::from_secs(30),
maximum_callbacks: 64,
handshake_timeout: Duration::from_secs(5),
maximum_stderr_bytes: 64 * 1024,
}
}
}
#[derive(Debug, Clone, thiserror::Error)]
pub enum RpcError {
#[error("plugin process could not be started: {0}")]
Start(#[source] ProcessStartError),
#[error("plugin handshake does not match the prepared instance")]
Handshake(#[source] Option<gateway_core::error::ErrorSource>),
#[error("plugin call budget negotiation failed: {0}")]
Budget(#[from] CallBudgetError),
#[error("plugin protocol is invalid")]
Protocol(#[source] Option<gateway_core::error::ErrorSource>),
#[error("plugin returned an invalid response at {0:?}")]
InvalidResponse(Stage, #[source] Option<gateway_core::error::ErrorSource>),
#[error("plugin process or transport has stopped")]
Closed(#[source] Option<gateway_core::error::ErrorSource>),
#[error("plugin call exceeded its deadline")]
Timeout,
#[error("plugin session was cancelled")]
Cancelled,
#[error("plugin call capacity is exhausted")]
Capacity,
#[error("plugin admission capacity is exhausted")]
Overloaded,
#[error("plugin call context is invalid")]
Context(#[source] Option<gateway_core::error::ErrorSource>),
#[error("plugin returned an error")]
Remote(PluginFault),
}
impl RpcError {
pub(crate) fn is_overloaded(&self) -> bool {
matches!(self, Self::Overloaded)
|| matches!(self, Self::Remote(fault) if fault.code == gateway_plugin_sdk::ErrorCode::Capacity)
}
/// 会话状态与请求错误共用安全分类,不展开插件返回正文
pub(crate) fn diagnostic(&self) -> (&'static str, &'static str) {
match self {
RpcError::Start(_) => ("process_start", "插件进程启动失败"),
RpcError::Handshake(_) => ("handshake", "插件握手失败"),
RpcError::Budget(_) => ("call_budget", "插件调用预算协商失败"),
RpcError::Protocol(_) => ("protocol", "插件协议错误"),
RpcError::InvalidResponse(stage, _) => (
"invalid_response",
match stage {
Stage::Registration => "注册阶段:插件返回了无效响应",
Stage::Configuration => "配置阶段:插件返回了无效响应",
Stage::Authentication => "认证阶段:插件返回了无效响应",
Stage::Routing => "路由阶段:插件返回了无效响应",
Stage::Scheduling => "调度阶段:插件返回了无效响应",
Stage::Retry => "重试阶段:插件返回了无效响应",
Stage::Http => "HTTP 中间件:插件返回了无效响应",
Stage::Service => "服务中间件:插件返回了无效响应",
Stage::WebSocket => "WebSocket 中间件:插件返回了无效响应",
Stage::Request => "请求中间件:插件返回了无效响应",
Stage::Attempt => "尝试中间件:插件返回了无效响应",
Stage::Outbound => "出站准备中间件:插件返回了无效响应",
Stage::Upstream => "上游适配阶段:插件返回了无效响应",
Stage::Observation => "观察阶段:插件返回了无效响应",
Stage::Management | Stage::PublicManagement => "管理阶段:插件返回了无效响应",
Stage::CommandLine => "命令行阶段:插件返回了无效响应",
Stage::Maintenance => "维护阶段:插件返回了无效响应",
},
),
RpcError::Closed(_) => ("closed", "插件进程或传输已停止"),
RpcError::Timeout => ("timeout", "插件调用超时"),
RpcError::Cancelled => ("cancelled", "插件会话已取消"),
RpcError::Capacity => ("capacity", "插件调用容量已耗尽"),
RpcError::Overloaded => ("overloaded", "插件暂时繁忙"),
RpcError::Context(_) => ("context", "插件调用上下文无效"),
RpcError::Remote(_) => ("remote", "插件返回错误"),
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(crate) enum RpcSessionDiagnostic {
Ready,
Quiescing,
Failed {
code: &'static str,
message: &'static str,
},
}
pub(crate) struct RpcSessionLifecycle {
failure: Arc<dyn Fn(Duration) + Send + Sync>,
end: Option<Box<dyn FnOnce() + Send + Sync>>,
}
impl RpcSessionLifecycle {
pub(crate) fn new(
failure: impl Fn(Duration) + Send + Sync + 'static,
end: impl FnOnce() + Send + Sync + 'static,
) -> Self {
Self {
failure: Arc::new(failure),
end: Some(Box::new(end)),
}
}
fn fail(&self, uptime: Duration) {
(self.failure)(uptime);
}
}
impl Drop for RpcSessionLifecycle {
fn drop(&mut self) {
if let Some(end) = self.end.take() {
end();
}
}
}
/// 不派生 Debug,防止原始请求、凭据或模型输出进入宿主诊断
pub struct RpcReply {
pub result: serde_json::Value,
pub payload: Vec<u8>,
}
/// 具体回调在领域适配器校验账号、出站与执行身份;上下文只取自宿主在途调用表
pub trait CallbackHandler: Send + Sync {
/// 在 Call 可见之前建立资源范围;不得执行 I/O 或调用 RPC
fn begin(&self, _context: &CallContext, _deadline: &CallDeadline) {}
/// 调用结束或取消时立即撤销其句柄;实现必须幂等且不能阻塞
fn finish(&self, _context: &CallContext) {}
/// 首个流式结果已到达;资源仍绑定原调用并在终态统一回收
fn streaming(&self, _context: &CallContext) {}
fn call(
&self,
context: CallContext,
method: String,
params: serde_json::Value,
payload: Vec<u8>,
) -> BoxFuture<'static, Result<RpcReply, PluginFault>>;
}
pub struct RpcSession {
shared: Arc<Shared>,
handshake: Handshake,
@@ -216,366 +43,6 @@ impl RpcSession {
}
}
pub(crate) struct Shared {
state: Mutex<State>,
started_at: Instant,
expected_stop: AtomicBool,
failure_observer: Option<Arc<dyn Fn(Duration) + Send + Sync>>,
pub control: mpsc::Sender<EncodedFrame>,
pub process: ProcessControl,
pub stopped: watch::Sender<bool>,
callback_slots: Arc<Semaphore>,
next_slots: Arc<Semaphore>,
execution: ExecutionCapacity,
cancellations_changed: Notify,
}
struct State {
failure: Option<RpcError>,
calls: BTreeMap<u64, PendingCall>,
}
struct PendingCall {
_slot: Arc<OwnedSemaphorePermit>,
execution: Option<OwnedSemaphorePermit>,
deadline: CallDeadline,
handler: Arc<dyn CallbackHandler>,
transmitted: bool,
cancellation: Option<RpcError>,
context: CallContext,
result: Option<oneshot::Sender<Result<RpcReply, RpcError>>>,
stream: Option<StreamIngress>,
callbacks: Vec<tokio::task::AbortHandle>,
}
impl PendingCall {
fn finish(mut self, result: Result<RpcReply, RpcError>) {
for callback in self.callbacks.drain(..) {
callback.abort();
}
let stream = self.stream.take();
let sender = self.result.take();
// 先同步关闭调用资源,Core 才能在收到结果时看到全部发送副作用
drop(self);
if let Some(stream) = stream {
stream.finish(result.as_ref().map(|_| ()).map_err(Clone::clone));
}
if let Some(sender) = sender {
let _ = sender.send(result);
}
}
}
impl Drop for PendingCall {
fn drop(&mut self) {
self.handler.finish(&self.context);
}
}
impl Shared {
fn state(&self) -> std::sync::MutexGuard<'_, State> {
self.state
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
}
pub fn fail(&self, error: RpcError) {
let uptime = self.started_at.elapsed();
let (calls, failure_observer) = {
let mut state = self.state();
if state.failure.is_some() {
return;
}
state.failure = Some(error.clone());
let observer = (!self.expected_stop.load(Ordering::Acquire))
.then(|| self.failure_observer.clone())
.flatten();
(std::mem::take(&mut state.calls), observer)
};
self.execution.close();
for call in calls.into_values() {
call.finish(Err(error.clone()));
}
if let Some(observer) = failure_observer {
observer(uptime);
}
self.stopped.send_replace(true);
self.process.stop();
self.cancellations_changed.notify_waiters();
}
fn expect_stop(&self) {
self.expected_stop.store(true, Ordering::Release);
}
pub fn finish(&self, id: u64, result: Result<RpcReply, RpcError>) -> Result<(), RpcError> {
let mut state = self.state();
let call = state.calls.get_mut(&id).ok_or(RpcError::Protocol(None))?;
if call.cancellation.is_some() {
return Ok(());
}
if matches!(&result, Err(RpcError::Remote(fault)) if fault.code == gateway_plugin_sdk::ErrorCode::Timeout)
{
log_timeout(&call.context, "plugin", "initial_reply", "remote_timeout");
}
if call.stream.is_some() && result.is_ok() {
if call.context.resource_stream {
call.deadline.clear();
}
let initial = call.result.take().ok_or(RpcError::Protocol(None))?;
call.execution.take();
call.handler.streaming(&call.context);
let _ = initial.send(result);
} else {
let pending = state.calls.remove(&id).ok_or(RpcError::Protocol(None))?;
drop(state);
pending.finish(result);
}
Ok(())
}
pub fn stream_chunk(&self, id: u64, sequence: u64, payload: Vec<u8>) -> Result<(), RpcError> {
let mut state = self.state();
let call = state.calls.get_mut(&id).ok_or(RpcError::Protocol(None))?;
if call.cancellation.is_some() {
return Ok(());
}
if call.result.is_some() {
return Err(RpcError::Protocol(None));
}
call.stream
.as_mut()
.ok_or(RpcError::Protocol(None))?
.receive(sequence, payload)
}
pub fn stream_end(&self, id: u64, error: Option<PluginFault>) -> Result<(), RpcError> {
let mut state = self.state();
if state
.calls
.get(&id)
.is_some_and(|call| call.cancellation.is_some())
{
return Ok(());
}
let call = state.calls.remove(&id).ok_or(RpcError::Protocol(None))?;
drop(state);
if call.stream.is_none() || call.result.is_some() {
call.finish(Err(RpcError::Protocol(None)));
return Err(RpcError::Protocol(None));
}
let result = error.map_or_else(
|| {
Ok(RpcReply {
result: serde_json::Value::Null,
payload: Vec::new(),
})
},
|error| Err(RpcError::Remote(error)),
);
call.finish(result);
Ok(())
}
pub fn release_stream_credit(&self, id: u64, bytes: u32) -> Result<(), RpcError> {
let mut state = self.state();
// 完成后仍可消费已排队数据,无须再给已关闭的流发信用
let Some(call) = state.calls.get_mut(&id) else {
return Ok(());
};
call.stream
.as_mut()
.ok_or(RpcError::Protocol(None))?
.release(bytes)?;
drop(state);
self.send_control(Frame::control(Message::Credit {
id,
bytes,
frames: 1,
}));
Ok(())
}
pub fn context(&self, id: u64) -> Option<CallContext> {
self.state()
.calls
.get(&id)
.filter(|call| call.cancellation.is_none())
.map(|call| call.context.clone())
}
pub fn pause_next(&self, id: u64) -> Option<CallDeadlinePause> {
let mut state = self.state();
let call = state
.calls
.get_mut(&id)
.filter(|call| call.cancellation.is_none())?;
let pause = call.deadline.pause()?;
call.execution.take();
Some(pause)
}
pub async fn resume_next(&self, id: u64) -> Result<(), RpcError> {
{
let state = self.state();
let call = state
.calls
.get(&id)
.filter(|call| call.cancellation.is_none())
.ok_or(RpcError::Cancelled)?;
// 已交付响应头的流只占在途资源,不重新占用首结果执行名额
if call.result.is_none() {
return Ok(());
}
}
let permit = self.execution.resume().await?;
let mut state = self.state();
let call = state
.calls
.get_mut(&id)
.filter(|call| call.cancellation.is_none())
.ok_or(RpcError::Cancelled)?;
if call.result.is_some() {
call.execution = Some(permit);
}
Ok(())
}
pub fn is_cancelling(&self, id: u64) -> bool {
self.state()
.calls
.get(&id)
.is_some_and(|call| call.cancellation.is_some())
}
pub fn mark_transmitted(&self, id: u64) -> bool {
let mut state = self.state();
let Some(call) = state.calls.get_mut(&id) else {
return false;
};
if call.cancellation.is_some() {
return false;
}
call.transmitted = true;
true
}
/// 先撤销调用权限,再等待对端确认;只有无响应对端才终止整个 incarnation
pub fn cancel(self: &Arc<Self>, id: u64, error: RpcError) {
let mut state = self.state();
let Some(call) = state.calls.get_mut(&id) else {
return;
};
if call.cancellation.is_some() {
return;
}
if matches!(error, RpcError::Timeout) {
log_timeout(
&call.context,
"runtime",
if call.result.is_some() {
"initial_reply"
} else {
"response_stream"
},
"processing_budget_exhausted",
);
}
if !call.transmitted {
if let Some(call) = state.calls.remove(&id) {
call.finish(Err(error));
}
return;
}
call.cancellation = Some(error.clone());
call.handler.finish(&call.context);
for callback in call.callbacks.drain(..) {
callback.abort();
}
if let Some(result) = call.result.take() {
let _ = result.send(Err(error.clone()));
}
if let Some(stream) = call.stream.take() {
stream.finish(Err(error.clone()));
}
drop(state);
self.send_control(Frame::control(Message::Cancel { id }));
let shared = Arc::clone(self);
tokio::spawn(async move {
tokio::time::sleep(Duration::from_millis(250)).await;
if shared.is_cancelling(id) {
shared.fail(error);
}
});
}
pub fn acknowledge_cancellation(&self, id: u64) -> Result<(), RpcError> {
let mut state = self.state();
if !state
.calls
.get(&id)
.is_some_and(|call| call.cancellation.is_some())
{
return Err(RpcError::Protocol(None));
}
state.calls.remove(&id);
drop(state);
self.cancellations_changed.notify_waiters();
Ok(())
}
pub fn track_callback(&self, parent_id: u64, callback: tokio::task::AbortHandle) -> bool {
if let Some(call) = self
.state()
.calls
.get_mut(&parent_id)
.filter(|call| call.cancellation.is_none())
{
// 已完成回调的句柄不继续占用请求预算
call.callbacks.retain(|handle| !handle.is_finished());
call.callbacks.push(callback);
true
} else {
callback.abort();
false
}
}
pub fn send_control(&self, frame: Frame) {
let Ok(frame) = EncodedFrame::new(frame) else {
self.fail(RpcError::Protocol(None));
return;
};
self.send_encoded_control(frame);
}
pub(super) fn send_encoded_control(&self, frame: EncodedFrame) {
if self.control.try_send(frame).is_err() {
self.fail(RpcError::Capacity);
}
}
pub(super) fn try_callback_slot(&self, method: &str) -> Option<OwnedSemaphorePermit> {
let slots = if method == gateway_plugin_sdk::call::middleware::NEXT_METHOD {
&self.next_slots
} else {
&self.callback_slots
};
slots.clone().try_acquire_owned().ok()
}
async fn wait_for_callbacks(&self, limits: RpcLimits) {
for (slots, maximum) in [
(&self.callback_slots, limits.maximum_callbacks),
(&self.next_slots, limits.maximum_calls),
] {
if let Ok(permits) = slots.clone().acquire_many_owned(maximum as u32).await {
drop(permits);
}
}
}
}
impl RpcSession {
pub fn budget(&self, stage: Stage) -> Duration {
self.budgets.budget(stage)
@@ -586,27 +53,9 @@ impl RpcSession {
self.budgets.source(stage)
}
/// 替换维护任务前等待已取消的调用确认;未确认仍沿用会话的有界故障回收
/// 替换维护任务前确认旧调用已取消或所属进程退出
pub(crate) async fn settle_cancellations(&self) {
let ids: Vec<_> = self
.shared
.state()
.calls
.iter()
.filter_map(|(id, call)| call.cancellation.is_some().then_some(*id))
.collect();
loop {
let changed = self.shared.cancellations_changed.notified();
tokio::pin!(changed);
changed.as_mut().enable();
if !ids.iter().any(|id| self.shared.is_cancelling(*id)) {
break;
}
changed.await;
}
if !self.is_ready() {
self.shared.process.exited().await;
}
self.shared.settle_cancellations().await;
}
pub async fn start(
@@ -653,7 +102,7 @@ impl RpcSession {
|| limits.maximum_queued_calls > 256
|| limits.maximum_queue_timeout.is_zero()
|| limits.maximum_callbacks == 0
|| limits.maximum_callbacks > 256
|| limits.maximum_callbacks > 512
|| limits.maximum_buffered_body_bytes < 1024
|| limits.maximum_buffered_body_bytes > 16 * 1024 * 1024
|| limits.handshake_timeout.is_zero()
@@ -755,30 +204,14 @@ impl RpcSession {
let (data, data_receiver) = mpsc::channel(limits.maximum_calls);
let (control, control_receiver) =
mpsc::channel(limits.maximum_callbacks + 2 * limits.maximum_calls + 8);
let (stopped, _) = watch::channel(false);
let callback_slots = Arc::new(Semaphore::new(limits.maximum_callbacks));
let shared = Arc::new(Shared {
state: Mutex::new(State {
failure: None,
calls: BTreeMap::new(),
}),
started_at: Instant::now(),
expected_stop: AtomicBool::new(false),
failure_observer: lifecycle
let shared = Arc::new(Shared::new(
limits,
process,
control,
lifecycle
.as_ref()
.map(|lifecycle| Arc::clone(&lifecycle.failure)),
control,
process,
stopped,
callback_slots,
next_slots: Arc::new(Semaphore::new(limits.maximum_calls)),
execution: ExecutionCapacity::new(
limits.maximum_executing_calls,
limits.maximum_queued_calls,
limits.maximum_queue_timeout,
),
cancellations_changed: Notify::new(),
});
));
dispatch::start(
input,
output,
@@ -813,23 +246,17 @@ impl RpcSession {
#[must_use]
pub fn is_ready(&self) -> bool {
self.shared.failure_diagnostic().is_none()
}
pub(crate) fn load(&self) -> RpcSessionLoad {
self.shared
.state
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.failure
.is_none()
.load(self.limits, self.slots.available_permits())
}
/// 仅返回稳定分类与固定文案,绝不展开插件 fault、stderr 或请求内容
pub(crate) fn diagnostic(&self) -> RpcSessionDiagnostic {
let state = self
.shared
.state
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if let Some(error) = &state.failure {
let (code, message) = error.diagnostic();
if let Some((code, message)) = self.shared.failure_diagnostic() {
return RpcSessionDiagnostic::Failed { code, message };
}
if self.slots.is_closed() {
@@ -874,44 +301,6 @@ impl RpcSession {
Ok(Duration::from_millis(context.timeout_ms))
}
fn begin_call(
&self,
context: CallContext,
deadline: tokio::time::Instant,
stream: Option<StreamIngress>,
slot: Arc<OwnedSemaphorePermit>,
execution: OwnedSemaphorePermit,
) -> Result<StartedCall, RpcError> {
let id = context.call_id;
let (result, received) = oneshot::channel();
let mut state = self.shared.state();
if let Some(error) = &state.failure {
return Err(error.clone());
}
let deadline = CallDeadline::new(deadline);
self.callbacks.begin(&context, &deadline);
state.calls.insert(
id,
PendingCall {
_slot: slot,
execution: Some(execution),
deadline: deadline.clone(),
handler: self.callbacks.clone(),
transmitted: false,
cancellation: None,
context,
result: Some(result),
stream,
callbacks: Vec::new(),
},
);
Ok(StartedCall {
id,
deadline,
received,
})
}
async fn send_call(
&self,
method: &str,
@@ -929,7 +318,19 @@ impl RpcSession {
RpcError::Overloaded
}
})?);
let execution = self.shared.execution.enter().await?;
let queued_at = Instant::now();
let execution = self.shared.execution.enter().await;
tracing::debug!(
event = "plugin.rpc.admission",
instance_id = context.instance_id,
stage = ?context.stage,
request_id = context.request_id,
attempt_id = context.attempt_id,
wait_ms = queued_at.elapsed().as_millis() as u64,
outcome = execution.as_ref().map_or_else(|error| error.diagnostic().0, |_| "admitted"),
"插件 RPC 执行准入完成"
);
let execution = execution?;
let deadline = tokio::time::Instant::now() + timeout;
// FIFO 锁同时拥有 ID 与 data 入队顺序;Call/初始 Credit 入队后必须立即释放,
// 不能把插件回复或重入回调纳入串行区,也不能为等待锁重置调用期限
@@ -953,7 +354,14 @@ impl RpcSession {
let frame = EncodedFrame::new(frame).map_err(|source| {
RpcError::Context(Some(gateway_core::error::ErrorSource::new(source)))
})?;
let started = self.begin_call(context, deadline, stream, slot.clone(), execution)?;
let started = self.shared.begin_call(
context,
deadline,
stream,
slot.clone(),
execution,
self.callbacks.clone(),
)?;
let mut guard = CallGuard {
shared: Arc::clone(&self.shared),
id: started.id,
@@ -1127,36 +535,9 @@ impl Drop for CallGuard {
}
}
struct StartedCall {
id: u64,
deadline: CallDeadline,
received: oneshot::Receiver<Result<RpcReply, RpcError>>,
}
struct CompletedCall {
slot: Arc<OwnedSemaphorePermit>,
id: u64,
deadline: CallDeadline,
reply: RpcReply,
}
fn log_timeout(
context: &CallContext,
owner: &'static str,
phase: &'static str,
reason: &'static str,
) {
tracing::warn!(
event = "plugin.rpc.timeout",
owner,
phase,
instance_id = context.instance_id,
call_id = context.call_id,
request_id = context.request_id,
attempt_id = context.attempt_id,
stage = ?context.stage,
budget_ms = context.timeout_ms,
reason,
"插件 RPC 调用超时"
);
}
@@ -0,0 +1,72 @@
//! 验证回调方法精确分派,未知方法不进入其他能力的授权或资源处理
use gateway_admin::{
model::plugins::management::PluginManagementRequest, ports::plugin_management::PluginManagement,
};
use serde_json::{Value, json};
#[tokio::test]
async fn unknown_callback_methods_are_unsupported_without_affecting_known_methods() {
let Some(environment) = crate::support::environment::Environment::create().await else {
eprintln!("SKIP: plugin integration environment absent");
return;
};
let (runtime, core) = environment.plugin(json!({
"management_registration":{"routes":[{"method":"GET","path":"dispatch","request_content_types":[],"response_content_types":["application/json"]}]},
"data_queries":[
{"method":"host.unknown","query":{}},
{"method":"host.middleware.unknown","query":{}},
{"method":"host.upstream.unknown","query":{}},
{"method":"host.http.unknown","query":{}},
{"method":gateway_plugin_sdk::call::host::HTTP_DO,"query":{}},
{"method":gateway_plugin_sdk::call::middleware::NEXT_METHOD,"query":{}}
]
})).await;
let generation = core
.snapshots()
.acquire()
.unwrap()
.extensions()
.unwrap()
.clone();
let view = runtime.views(&generation).await.unwrap().remove(0);
let response = runtime
.handle(
&generation,
&view.target,
PluginManagementRequest {
request_context: None,
headers: Vec::new(),
method: "GET".into(),
path: "dispatch".into(),
query: String::new(),
content_type: None,
body: Vec::new(),
request_id: "callback-dispatch".into(),
},
)
.await
.unwrap();
let results: Vec<Value> = serde_json::from_slice(&response.body).unwrap();
let codes: Vec<_> = results
.iter()
.map(|result| result["error"].as_str().unwrap())
.collect();
assert_eq!(
codes,
[
"unsupported",
"unsupported",
"unsupported",
"unsupported",
"invalid_input",
"permission_denied"
]
);
drop(generation);
environment.release_plugin_accounts(&runtime);
runtime.shutdown().await;
drop(core);
drop(runtime);
environment.close().await;
}
@@ -1,5 +1,6 @@
//! 插件宿主回调的测试入口
mod dispatch;
mod key_facts;
mod keys;
mod log;
@@ -273,3 +273,117 @@ async fn startup_failure_of_an_unrelated_plugin_preserves_native_authentication_
}
close(environment, runtime, core).await;
}
#[tokio::test]
async fn callback_verification_reenters_the_authentication_instance_without_model_binding() {
assert_callback_identity_queries(true).await;
}
#[tokio::test]
async fn client_queries_do_not_require_model_execution_ports() {
assert_callback_identity_queries(false).await;
}
async fn assert_callback_identity_queries(include_models: bool) {
use gateway_admin::{
model::plugins::management::PluginManagementRequest,
ports::plugin_management::PluginManagement,
};
use gateway_plugin_runtime::RpcLimits;
let Some(environment) = Environment::create().await else {
return;
};
environment
.client_key("key_frontend", "sk-native-fixture")
.await;
let account = environment.account(None).await;
let marker = environment
.directory
.path()
.join("authenticated-facts.json");
let mut facts = vec![json!(
{"method":"host.data.access.check", "query":{"client_key_id":"key_frontend", "account_ids":[account.as_str(), "acct_missing"]}}
)];
if include_models {
facts.push(json!(
{"method":"host.models.list", "params":{"client_key_id":"key_frontend", "protocol":"codex", "client_version":"0.154.0"}}
));
}
let mut config = configuration(
json!({"outcome":"authenticated","principal":"external-user"}),
true,
"External controlled-fixture",
);
config["sdk_session"] = json!(true);
config["frontend_authentication_marker"] = json!(marker);
config["auth_queries"] = json!(facts);
config["management_registration"] = json!({"routes":[{"method":"GET","path":"facts","request_content_types":[],"response_content_types":["application/json"]}]});
let mut queries = vec![json!(
{"method":"host.client.verify", "query":{"authorization":"External controlled-fixture"}}
)];
queries.extend(facts);
config["data_queries"] = json!(queries);
environment.install_plugin(config).await;
let (runtime, core) = environment
.runtime_with_identity_port(
RpcLimits {
maximum_executing_calls: 1,
maximum_queue_timeout: std::time::Duration::from_millis(500),
..RpcLimits::default()
},
crate::support::native::provider_registry(),
)
.await;
if include_models {
runtime
.bind_model_ports(
&core.nested_model_execution_port(),
&core.affinity_lookup_port(),
)
.unwrap();
}
let snapshot = core.snapshots().acquire().unwrap();
let generation = snapshot.extensions().unwrap();
let view = runtime.views(generation).await.unwrap().remove(0);
let response = runtime
.handle(
generation,
&view.target,
PluginManagementRequest {
request_context: None,
headers: Vec::new(),
method: "GET".into(),
path: "facts".into(),
query: String::new(),
content_type: None,
body: Vec::new(),
request_id: "verify-composition".into(),
},
)
.await
.unwrap();
let results: Value = serde_json::from_slice(&response.body).unwrap();
assert_eq!(results[0], json!({"client_key_id":"key_frontend"}));
let authenticated_facts: Value =
serde_json::from_slice(&std::fs::read(marker).unwrap()).unwrap();
assert_eq!(authenticated_facts[0], results[1]);
assert_eq!(
authenticated_facts[0]["accounts"],
json!([
{"account_id":account.as_str(),"allowed":true}, {"account_id":"acct_missing","allowed":false}
])
);
if include_models {
assert_eq!(authenticated_facts[1], results[2]);
// 此替身只登记执行能力,没有原生目录;查询应成功返回空目录而不是执行绑定错误
assert_eq!(results[2], json!({"models":[]}));
}
assert!(
environment
.bound_model_requests("key_frontend")
.await
.is_empty()
);
drop(snapshot);
close(environment, runtime, core).await;
}
@@ -22,7 +22,12 @@ use gateway_plugin_sdk::{Capability, Contributions, Stage};
use serde_json::{Value, json};
#[derive(Default)]
struct Facts(AtomicUsize, AtomicUsize, Option<String>);
struct Facts {
reads: AtomicUsize,
refreshes: AtomicUsize,
email: Option<String>,
runtime_account: Option<AccountRecord>,
}
fn account() -> AccountRecord {
let at = "2026-01-01T00:00:00Z".parse().unwrap();
@@ -71,7 +76,7 @@ impl PluginAccountAccess for Facts {
&self,
account: &ProviderAccountId,
) -> Result<ProviderQuota, AdminError> {
self.1.fetch_add(1, Ordering::SeqCst);
self.refreshes.fetch_add(1, Ordering::SeqCst);
if account.as_str() == "acct_missing" {
return Err(AdminError::not_found("missing account"));
}
@@ -88,19 +93,26 @@ impl PluginAccountAccess for Facts {
}
async fn list(&self, query: PluginAccountListQuery) -> Result<PluginAccountPage, AdminError> {
self.0.fetch_add(1, Ordering::SeqCst);
self.reads.fetch_add(1, Ordering::SeqCst);
assert_eq!(query.limit.get(), 1);
assert_eq!(query.provider_kind.unwrap().as_str(), "openai");
assert_eq!(query.cursor.unwrap().as_str(), "acct_before");
let mut account = account();
account.email.clone_from(&self.2);
account.email.clone_from(&self.email);
Ok(PluginAccountPage {
accounts: vec![account],
next_cursor: Some(ProviderAccountId::new("acct_facts").unwrap()),
})
}
async fn get_runtime(&self, _: &ProviderAccountId) -> Result<AccountRecord, AdminError> {
panic!("事实回调只使用对应的窄查询")
async fn get_runtime(&self, id: &ProviderAccountId) -> Result<AccountRecord, AdminError> {
let account = self
.runtime_account
.as_ref()
.expect("事实回调只使用对应的窄查询");
if account.id != id.as_str() {
return Err(AdminError::not_found("fixture account is unavailable"));
}
Ok(account.clone())
}
async fn get_credential(
&self,
@@ -116,10 +128,14 @@ impl PluginAccountAccess for Facts {
panic!("不允许修改账号")
}
async fn get_quota(&self, account: &ProviderAccountId) -> Result<ProviderQuota, AdminError> {
self.0.fetch_add(1, Ordering::SeqCst);
self.reads.fetch_add(1, Ordering::SeqCst);
assert_eq!(account.as_str(), "acct_facts");
Ok(ProviderQuota {
credits: None,
credits: Some(ProviderQuotaCredits {
has_credits: true,
unlimited: false,
balance: Some("123456789.123456789".into()),
}),
plan_type: Some("private plan".into()),
observed_at: None,
refresh_token_expires_at: None,
@@ -129,17 +145,24 @@ impl PluginAccountAccess for Facts {
label: "private label".into(),
limit_id: None,
limit_name: None,
role: None,
role: Some(ProviderQuotaWindowRole::Primary),
local_usage_attribution: QuotaLocalUsageAttribution::Unavailable,
window_seconds: Some(604800),
used_percent: None,
reset_at: None,
limit_reached: false,
limit_reached: true,
local_usage: None,
provider_data: None,
}],
limit_reached: false,
provider_data: None,
limit_reached: true,
provider_data: Some(ProviderDocument::new(
gateway_core::account::OpaqueProviderData::new(
json!({"private":"provider-internal-fixture"})
.as_object()
.unwrap()
.clone(),
),
)),
})
}
}
@@ -157,11 +180,10 @@ async fn management_facts_validate_queries_without_permission_declarations() {
Default::default(),
)
.await;
let facts = Arc::new(Facts(
AtomicUsize::new(0),
AtomicUsize::new(0),
email.map(str::to_owned),
));
let facts = Arc::new(Facts {
email: email.map(str::to_owned),
..Default::default()
});
let port: Arc<dyn PluginAccountAccess> = facts.clone();
runtime.bind_account_ports(&port).unwrap();
{
@@ -172,6 +194,7 @@ async fn management_facts_validate_queries_without_permission_declarations() {
"data_queries":[
{"method":"host.data.accounts.list","query":{"provider_id":"openai","cursor":"acct_before","limit":1}},
{"method":"host.data.quota.get","query":{"account_id":"acct_facts"}},
{"method":"host.data.quota.details","query":{"account_id":"acct_facts"}},
{"method":"host.data.accounts.list","query":{"limit":0}},
{"method":"host.data.accounts.list","query":{"limit":65535}},
{"method":"host.data.quota.get","query":{"account_id":"acct_facts","sql":"SELECT secret"}}
@@ -209,7 +232,8 @@ async fn management_facts_validate_queries_without_permission_declarations() {
.unwrap();
let result: Value = serde_json::from_slice(&response.body).unwrap();
assert_eq!(facts.0.load(Ordering::SeqCst), 2, "{result}");
assert_eq!(facts.reads.load(Ordering::SeqCst), 3, "{result}");
assert_eq!(facts.refreshes.load(Ordering::SeqCst), 0);
assert_eq!(
result[0],
json!({"schema_version":1,"accounts":[{
@@ -227,7 +251,19 @@ async fn management_facts_validate_queries_without_permission_declarations() {
json!({"schema_version":1,"account_id":"acct_facts","observed_at_ms":null,
"windows":[{"key":"weekly","window_seconds":604800,"used_percent":null,"reset_at_ms":null}]})
);
for invalid in &result.as_array().unwrap()[2..] {
let details: gateway_plugin_sdk::call::data::QuotaDetails =
serde_json::from_value(result[2].clone()).unwrap();
assert_eq!(details.observed_at_ms, None);
assert_eq!(details.windows[0].used_percent, None);
assert_eq!(details.windows[0].role.as_deref(), Some("primary"));
assert_eq!(details.windows[0].label, "private label");
assert!(details.limit_reached && details.windows[0].limit_reached);
assert_eq!(
details.credits.unwrap().balance.as_deref(),
Some("123456789.123456789")
);
assert!(!result[2].to_string().contains("provider-internal-fixture"));
for invalid in &result.as_array().unwrap()[3..] {
assert_eq!(invalid["error"], "invalid_input");
}
@@ -301,7 +337,7 @@ async fn quota_refresh_is_available_without_permission_declarations() {
.unwrap();
let result: Value = serde_json::from_slice(&response.body).unwrap();
assert_eq!(facts.1.load(Ordering::SeqCst), 4);
assert_eq!(facts.refreshes.load(Ordering::SeqCst), 4);
assert_eq!(result[0]["observed_at_ms"], Value::Null);
assert_eq!(
result[1],
@@ -323,3 +359,149 @@ async fn quota_refresh_is_available_without_permission_declarations() {
super::wait_until_empty(cache.path()).await;
}
}
async fn management_queries(accounts: Arc<dyn PluginAccountAccess>, queries: Value) -> Value {
let (cache, store, runtime) =
super::setup_with_contributions(Contributions::from([crate::support::contribution(
Capability::Management,
vec![Stage::Management],
vec![],
vec![],
)]))
.await;
runtime.bind_account_ports(&accounts).unwrap();
store.snapshot.lock().unwrap().instances[0].configuration = json!({
"management_registration":{"routes":[{"method":"GET","path":"facts","request_content_types":[],"response_content_types":["application/json"]}]},
"data_queries":queries,
"sdk_session":true,
});
let generation = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
.await
.unwrap();
let view = runtime.views(&generation).await.unwrap().remove(0);
let response = runtime
.handle(
&generation,
&view.target,
PluginManagementRequest {
request_context: None,
headers: vec![],
method: "GET".into(),
path: "facts".into(),
query: String::new(),
content_type: None,
body: vec![],
request_id: "account-http-fixture".into(),
},
)
.await
.unwrap();
let result = serde_json::from_slice(&response.body).unwrap();
drop(generation);
runtime.shutdown().await;
super::wait_until_empty(cache.path()).await;
result
}
async fn request_head(stream: &mut tokio::net::TcpStream) -> String {
use tokio::io::AsyncReadExt as _;
let mut bytes = Vec::new();
while !bytes.ends_with(b"\r\n\r\n") {
assert!(bytes.len() < 16 * 1024);
bytes.push(stream.read_u8().await.unwrap());
}
String::from_utf8(bytes).unwrap()
}
#[tokio::test]
async fn account_http_uses_the_selected_proxy_and_rejects_unavailable_account_versions() {
use tokio::io::AsyncWriteExt as _;
for scenario in [
"plain",
"explicit-auth",
"rotated",
"disabled",
"proxy-failed",
"missing",
] {
let target = wiremock::MockServer::start().await;
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let proxy = gateway_core::account::OutboundProxy::parse(&format!(
"http://{}",
listener.local_addr().unwrap()
))
.unwrap();
let successful = matches!(scenario, "plain" | "explicit-auth");
let proxy_task = if successful {
Some(tokio::spawn(async move {
let (mut socket, _) = listener.accept().await.unwrap();
let request = request_head(&mut socket).await;
assert!(request.starts_with("GET http://127.0.0.1:"));
socket
.write_all(
b"HTTP/1.1 201 Created\r\nContent-Length: 0\r\nConnection: close\r\n\r\n",
)
.await
.unwrap();
request
}))
} else {
drop(listener);
None
};
let mut selected = account();
selected.outbound_proxy = Some(proxy);
selected.enabled = scenario != "disabled";
if scenario == "rotated" {
selected.credential_revision = Revision::new(2).unwrap();
}
let accounts: Arc<dyn PluginAccountAccess> = Arc::new(Facts {
runtime_account: Some(selected),
..Default::default()
});
let headers = if scenario == "explicit-auth" {
vec![("authorization", b"Bearer plugin-fixture".to_vec())]
} else {
vec![]
};
let results = management_queries(accounts, json!([{
"method":gateway_plugin_sdk::call::host::ACCOUNT_HTTP,
"params":{
"account_id":if scenario == "missing" {"acct_missing"} else {"acct_facts"},
"credential_revision":1,
"request":{"method":"GET","url":format!("{}/facts", target.uri()),"headers":headers},
},
}])).await;
if let Some(task) = proxy_task {
assert_eq!(results[0]["status"], 201, "{scenario}: {results}");
let request = tokio::time::timeout(std::time::Duration::from_secs(3), task)
.await
.unwrap()
.unwrap()
.to_ascii_lowercase();
assert!(
request.starts_with("get http://127.0.0.1:") && request.contains("/facts http/1.1"),
"{request}"
);
assert_eq!(
request.contains("authorization:"),
scenario == "explicit-auth"
);
if scenario == "explicit-auth" {
assert!(request.contains("authorization: bearer plugin-fixture\r\n"));
}
} else {
let expected = match scenario {
"rotated" => "conflict",
"disabled" => "permission_denied",
"missing" => "rejected",
_ => "upstream",
};
assert_eq!(results[0]["error"], expected, "{scenario}: {results}");
}
assert!(
target.received_requests().await.unwrap().is_empty(),
"{scenario} must not connect directly"
);
}
}
@@ -881,6 +881,74 @@ async fn invalid_client_key_and_group_scopes_are_rejected_before_publish() {
}
}
#[tokio::test]
async fn observer_call_load_tracks_the_active_lease_until_completion() {
use gateway_admin::ports::plugins::PluginRuntimeDiagnostics;
let markers = tempfile::tempdir().unwrap();
let started = markers.path().join("started.jsonl");
let completed = markers.path().join("completed.jsonl");
let (cache, store, runtime) = setup(
vec![(
"tracked",
serde_json::json!({
"observation_started_marker": started,
"observation_marker": completed,
"observation_delay_ms": 200,
}),
vec![binding(EventKind::RequestCompleted, 0)],
)],
RpcLimits::default(),
)
.await;
let snapshot = store.snapshot.lock().unwrap().clone();
let generation = gateway_core::routing::extensions::ExtensionPreparationPort::prepare(
&runtime,
ConfigRevision::new(1).unwrap(),
)
.await
.unwrap();
let plan = runtime.observer_registry().resolve(&generation).unwrap();
plan.dispatch(
generation.clone(),
observation("req_tracked", RequestObservationOutcome::Succeeded)
.with_provider(ProviderKind::new("openai").unwrap()),
);
wait_for_lines(&started, 1).await;
let read_load = || async {
runtime
.runtime_diagnostics(&snapshot, Some(1), Some(&generation))
.await
.unwrap()
.remove("tracked")
.unwrap()
.load
.expect("published session has a load snapshot")
};
let load = read_load().await;
assert_eq!(load.active_calls, 1);
assert_eq!(load.executing_calls, 1);
assert_eq!(load.queued_calls, 0);
assert_eq!(load.pending_callbacks, 0);
assert_eq!(load.active_streams, 0);
wait_for_lines(&completed, 1).await;
tokio::time::timeout(Duration::from_secs(1), async {
loop {
let load = read_load().await;
if load.active_calls == 0 && load.executing_calls == 0 {
break;
}
tokio::task::yield_now().await;
}
})
.await
.expect("completed calls release their observed resource occupancy");
drop(plan);
drop(generation);
super::wait_until_empty(cache.path()).await;
}
#[tokio::test]
async fn observer_backpressure_drops_excess_and_inflight_dispatch_keeps_generation_alive() {
let marker_directory = tempfile::tempdir().unwrap();
@@ -1,4 +1,4 @@
//! 验证中间件 Fast 三态设置与不兼容插件的实例故障隔离
//! 验证中间件 Fast 三态设置与运行时故障隔离
use super::*;
use gateway_core::{policy::ClientPolicy, routing::request_settings::RequestSettings};
@@ -138,19 +138,18 @@ async fn middleware_preserves_three_state_fast_and_sources() {
}
#[tokio::test]
async fn unsupported_contract_starts_and_response_errors_do_not_stop_the_instance() {
async fn middleware_failures_preserve_healthy_instances_and_host_availability() {
use gateway_admin::{
model::plugins::instances::PluginInstanceRuntimeStatus,
ports::plugins::PluginRuntimeDiagnostics as _,
};
for failure in ["invalid_head", "crash", "business_error"] {
let mut declaration = crate::support::contribution(
let declaration = crate::support::contribution(
Capability::Middleware,
vec![Stage::Request],
vec!["openai".into()],
vec!["openai".into()],
);
declaration.1.version = 2;
let (cache, store, runtime) =
super::super::setup_with_contributions(Contributions::from([declaration])).await;
{
@@ -173,7 +172,7 @@ async fn unsupported_contract_starts_and_response_errors_do_not_stop_the_instanc
snapshot.instances.push(healthy);
}
let generation = prepare(&runtime).await;
assert!(generation.is_ready(), "版本警告允许随宿主启动");
assert!(generation.is_ready(), "兼容插件应完成启动");
let plan = runtime
.execution_registry()
.middleware(&generation)
@@ -2,7 +2,9 @@
use super::wait_until_unready;
use gateway_admin::model::Revision;
use gateway_admin::model::plugins::instances::PluginInstanceRuntimeStatus;
use gateway_admin::model::plugins::instances::{
PluginInstanceRuntimeLoad, PluginInstanceRuntimeStatus,
};
use gateway_admin::ports::plugins::PluginPreparation;
use gateway_core::routing::ConfigRevision;
use gateway_core::routing::extensions::ExtensionPreparationPort;
@@ -31,6 +33,17 @@ async fn diagnostics_distinguish_running_failed_preparation_and_draining_generat
Some(snapshot.instances[0].artifact_sha256.as_str())
);
assert!(running.failure.is_none());
assert_eq!(
running.load,
Some(PluginInstanceRuntimeLoad {
active_calls: 0,
executing_calls: 0,
queued_calls: 0,
resuming_calls: 0,
pending_callbacks: 0,
active_streams: 0,
})
);
let mut failed = snapshot.clone();
failed.config_revision = Revision::new(2).unwrap();
@@ -56,6 +69,7 @@ async fn diagnostics_distinguish_running_failed_preparation_and_draining_generat
);
assert!(failed_runtime.failure.is_some());
assert_eq!(failed_runtime.actual_revision, Some(1));
assert_eq!(failed_runtime.load, running.load);
let mut disabled = failed;
disabled.config_revision = Revision::new(3).unwrap();
@@ -105,6 +119,7 @@ async fn preparing_diagnostics_never_wait_for_the_serialized_prepare_io() {
diagnostics["instance-one"].status,
PluginInstanceRuntimeStatus::Preparing
);
assert!(diagnostics["instance-one"].load.is_none());
let candidate = preparation.await.expect("delayed candidate");
drop(candidate);
@@ -512,45 +512,304 @@ async fn abandoned_candidate_keeps_shared_process_failure_in_restart_budget() {
}
#[tokio::test]
async fn a_host_version_mismatch_still_restores_the_enabled_plugin() {
use std::sync::Arc;
let (cache, store, _) = super::super::setup().await;
let runtime = gateway_plugin_runtime::PluginRuntime::new(
store.clone(),
async fn incompatible_start_requires_scoped_confirmation_and_rechecks_after_restart() {
use gateway_admin::model::plugins::instances::{
PluginCapabilityBinding, PluginCompatibilityConfirmation, PluginFailurePolicy,
};
use gateway_admin::ports::plugins::{
PluginPackageInspector as _, PluginRuntimeDiagnostics as _,
};
use gateway_plugin_runtime::{PackageInspector, PackageLimits};
for mismatch in ["host", "capability"] {
let mut contribution =
crate::support::contribution(Capability::Middleware, vec![Stage::Http], vec![], vec![]);
if mismatch == "capability" {
contribution.1.version = 3;
}
let (cache, store, initial) =
super::super::setup_with_contributions(Contributions::from([contribution])).await;
drop(initial);
let host = if mismatch == "host" { "3.0.0" } else { "1.0.0" };
let runtime = compatibility_runtime(&cache, store.clone(), host);
let markers = tempfile::tempdir().unwrap();
let marker = markers.path().join("started");
let other_marker = markers.path().join("other-started");
let mut snapshot = store.snapshot.lock().unwrap().clone();
snapshot.instances[0].configuration = serde_json::json!({"startup_marker":marker});
snapshot.instances[0].bindings = vec![PluginCapabilityBinding {
contribution: "test.example.middleware".into(),
stage: "http".into(),
order: 0,
failure_policy: PluginFailurePolicy::Reject,
event: None,
client_key_ids: vec![],
account_group_ids: vec![],
provider_ids: vec![],
models: vec![],
identity_bindings: vec![],
}];
let mut other = snapshot.instances[0].clone();
other.id = "other-instance".into();
other.configuration = serde_json::json!({"startup_marker":other_marker});
snapshot.instances.push(other);
*store.snapshot.lock().unwrap() = snapshot.clone();
let restored = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
.await
.unwrap();
assert!(restored.can_serve());
assert!(runtime.execution_registry().middleware(&restored).is_none());
let diagnostics = runtime
.runtime_diagnostics(&snapshot, Some(1), Some(&restored))
.await
.unwrap();
assert_eq!(
diagnostics["instance-one"].failure.as_ref().unwrap().code,
"version_incompatible"
);
assert_eq!(startup_count(&marker), 0);
assert_eq!(startup_count(&other_marker), 0);
assert!(snapshot.instances.iter().all(|instance| instance.enabled));
snapshot.config_revision = Revision::new(2).unwrap();
snapshot.instances[0].revision = snapshot.config_revision;
assert!(
PluginPreparation::prepare(&runtime, snapshot.clone())
.await
.is_err()
);
let inspector = PackageInspector::new(PackageLimits::default(), host.parse().unwrap());
let mut confirmation = PluginCompatibilityConfirmation {
host_fingerprint: "stale-host".into(),
instance_id: "instance-one".into(),
instance_revision: 1,
artifact_sha256: snapshot.instances[0].artifact_sha256.clone(),
};
assert!(
PluginPreparation::prepare_confirmed(&runtime, snapshot.clone(), confirmation.clone())
.await
.is_err()
);
assert_eq!(startup_count(&marker), 0);
confirmation.host_fingerprint = inspector.compatibility_host_fingerprint().unwrap();
let confirmed =
PluginPreparation::prepare_confirmed(&runtime, snapshot.clone(), confirmation)
.await
.unwrap();
assert!(
runtime
.execution_registry()
.middleware(&confirmed)
.is_some()
);
let diagnostics = runtime
.runtime_diagnostics(&snapshot, Some(2), Some(&confirmed))
.await
.unwrap();
assert_eq!(
diagnostics["instance-one"].status,
PluginInstanceRuntimeStatus::Running
);
assert_eq!(
diagnostics["other-instance"].failure.as_ref().unwrap().code,
"version_incompatible"
);
assert_eq!(startup_count(&marker), 1);
assert_eq!(startup_count(&other_marker), 0);
*store.snapshot.lock().unwrap() = snapshot.clone();
let reused = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(2).unwrap())
.await
.unwrap();
assert_eq!(reused.id(), confirmed.id());
drop(reused);
drop(confirmed);
drop(restored);
runtime.shutdown().await;
super::super::wait_until_empty(cache.path()).await;
let restarted = compatibility_runtime(&cache, store.clone(), host);
let restored =
ExtensionPreparationPort::prepare(&restarted, ConfigRevision::new(2).unwrap())
.await
.unwrap();
assert!(
restarted
.execution_registry()
.middleware(&restored)
.is_none()
);
assert_eq!(
startup_count(&marker),
1,
"confirmation must not survive host restart"
);
assert_eq!(startup_count(&other_marker), 0);
drop(restored);
restarted.shutdown().await;
super::super::wait_until_empty(cache.path()).await;
}
}
fn compatibility_runtime(
cache: &tempfile::TempDir,
store: std::sync::Arc<crate::support::store::Store>,
host: &str,
) -> gateway_plugin_runtime::PluginRuntime {
gateway_plugin_runtime::PluginRuntime::new(
store.clone(),
store,
gateway_plugin_runtime::PluginRuntimeConfig {
cache_directory: cache.path().to_owned(),
host_version: "3.0.0".parse().unwrap(),
host_version: host.parse().unwrap(),
package_limits: Default::default(),
rpc_limits: Default::default(),
restart_circuit: Default::default(),
},
Arc::new(gateway_host::outbound::HttpClient::new().unwrap()),
Arc::new(gateway_host::process::ProcessSupervisor::default()),
);
let restored = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
.await
.unwrap();
assert!(restored.can_serve());
let snapshot = store.snapshot.lock().unwrap().clone();
let diagnostics = gateway_admin::ports::plugins::PluginRuntimeDiagnostics::runtime_diagnostics(
&runtime,
&snapshot,
Some(1),
Some(&restored),
std::sync::Arc::new(gateway_host::outbound::HttpClient::new().unwrap()),
std::sync::Arc::new(gateway_host::process::ProcessSupervisor::default()),
)
.await
.unwrap();
assert!(restored.is_ready());
assert!(snapshot.instances[0].enabled);
assert_eq!(
diagnostics["instance-one"].status,
PluginInstanceRuntimeStatus::Running
);
assert!(diagnostics["instance-one"].failure.is_none());
drop(restored);
runtime.shutdown().await;
super::super::wait_until_empty(cache.path()).await;
}
#[tokio::test]
async fn unloadable_packages_have_no_guards_and_cannot_be_forced_to_start() {
use gateway_admin::model::plugins::instances::{
PluginCapabilityBinding, PluginCompatibilityConfirmation, PluginFailurePolicy,
};
use gateway_admin::ports::plugins::{
PluginPackageInspector as _, PluginRuntimeDiagnostics as _,
};
use gateway_plugin_runtime::{PackageInspector, PackageLimits, ValidatedPackage};
use sha2::{Digest as _, Sha256};
use std::{collections::BTreeMap, sync::Arc};
for defect in ["rpc", "manifest", "platform", "archive", "digest"] {
let (cache, store, initial) = super::super::setup().await;
drop(initial);
let mut store = Arc::try_unwrap(store).ok().unwrap();
let mut broken = store.artifacts.values().next().unwrap().clone();
let package =
ValidatedPackage::read(broken.archive.clone(), None, PackageLimits::default()).unwrap();
let mut manifest = package.manifest().clone();
let bytes = b"must never execute";
manifest
.package
.as_mut()
.unwrap()
.files
.insert("bin/worker".into(), hex::encode(Sha256::digest(bytes)));
match defect {
"rpc" => manifest.package.as_mut().unwrap().protocol_version += 1,
"manifest" => manifest.manifest_version += 1,
"platform" => {
manifest.package.as_mut().unwrap().target.os = if std::env::consts::OS == "windows"
{
"linux"
} else {
"windows"
}
.into()
}
_ => {}
}
broken.archive = if defect == "archive" {
Arc::from(&b"broken archive"[..])
} else {
crate::support::archive(BTreeMap::from([
("plugin.json".into(), serde_json::to_vec(&manifest).unwrap()),
("bin/worker".into(), bytes.to_vec()),
]))
};
broken.metadata.sha256 = if defect == "digest" {
"0".repeat(64)
} else {
hex::encode(Sha256::digest(&broken.archive))
};
let mut failed = store.snapshot.lock().unwrap().instances[0].clone();
failed.id = "unloadable".into();
failed.artifact_sha256 = broken.metadata.sha256.clone();
failed.bindings = [
"http",
"request",
"routing",
"scheduling",
"retry",
"upstream",
"authentication",
]
.into_iter()
.map(|stage| PluginCapabilityBinding {
contribution: "test.example.saved".into(),
stage: stage.into(),
order: 0,
failure_policy: PluginFailurePolicy::Reject,
event: None,
client_key_ids: vec![],
account_group_ids: vec![],
provider_ids: vec![],
models: vec![],
identity_bindings: vec![],
})
.collect();
store
.artifacts
.insert(broken.metadata.sha256.clone(), broken);
store.snapshot.lock().unwrap().instances.push(failed);
let store = Arc::new(store);
let runtime = compatibility_runtime(&cache, store.clone(), "1.0.0");
let restored = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
.await
.unwrap();
assert!(
runtime.execution_registry().middleware(&restored).is_none(),
"{defect}"
);
assert!(
runtime.policy_registry().resolve(&restored).is_none(),
"{defect}"
);
assert!(
runtime
.frontend_authentication_registry()
.resolve(&restored)
.is_none(),
"{defect}"
);
let mut snapshot = store.snapshot.lock().unwrap().clone();
let diagnostics = runtime
.runtime_diagnostics(&snapshot, Some(1), Some(&restored))
.await
.unwrap();
assert_eq!(
diagnostics["unloadable"].failure.as_ref().unwrap().code,
"package_unloadable",
"{defect}"
);
assert_eq!(
diagnostics["instance-one"].status,
PluginInstanceRuntimeStatus::Running
);
snapshot.config_revision = Revision::new(2).unwrap();
snapshot.instances[1].revision = snapshot.config_revision;
let confirmation = PluginCompatibilityConfirmation {
host_fingerprint: PackageInspector::new(
PackageLimits::default(),
"1.0.0".parse().unwrap(),
)
.compatibility_host_fingerprint()
.unwrap(),
instance_id: "unloadable".into(),
instance_revision: 1,
artifact_sha256: snapshot.instances[1].artifact_sha256.clone(),
};
assert!(
PluginPreparation::prepare_confirmed(&runtime, snapshot, confirmation)
.await
.is_err(),
"{defect}"
);
assert!(restored.is_ready());
drop(restored);
runtime.shutdown().await;
super::super::wait_until_empty(cache.path()).await;
}
}
#[tokio::test]
@@ -9,6 +9,189 @@ use gateway_plugin_sdk::Capability;
use gateway_plugin_sdk::Contributions;
use gateway_plugin_sdk::Stage;
#[tokio::test]
async fn adapted_catalog_freezes_adapter_accounts_and_preserves_the_active_generation() {
use gateway_admin::model::plugins::instances::{PluginCapabilityBinding, PluginFailurePolicy};
let mut catalog = crate::support::contribution(
Capability::ModelCatalog,
vec![Stage::Registration],
vec![],
vec![],
);
catalog.1.version = 2;
let adapter = crate::support::contribution(
Capability::UpstreamAdapter,
vec![Stage::Upstream],
vec!["openai".into()],
vec!["openai".into()],
);
let (cache, store, runtime) =
super::super::setup_with_contributions(Contributions::from([catalog, adapter])).await;
let mut snapshot = store.snapshot.lock().unwrap().clone();
snapshot.instances[0].bindings = vec![PluginCapabilityBinding {
contribution: "test.example.upstreamAdapter".into(),
stage: "upstream".into(),
order: 0,
failure_policy: PluginFailurePolicy::Reject,
client_key_ids: vec![],
account_group_ids: vec![],
provider_ids: vec!["openai".into()],
models: vec!["shared-model".into()],
event: None,
identity_bindings: vec![],
}];
snapshot.instances[0].configuration = serde_json::json!({
"upstream_registration":{"adapters":[{
"id":"example","provider":"openai","base_url":"https://example.invalid/",
"paths":[{"path":"responses","purpose":"inference"}], "authentication_kinds":["oauth"],
"account_ids":["acct_b"], "models":["shared-model"], "transport":"http_sse","protocol":"openai"
}]},
"model_catalog":{"aliases":[],"models":[{
"id":"shared-model","adapter_id":"example","features":["tools"],"presentation":{"display_name":"Shared"}
}]}
});
*store.snapshot.lock().unwrap() = snapshot.clone();
let active = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
.await
.unwrap();
assert_eq!(active.models().len(), 1);
assert_eq!(
active.models()[0].accounts.iter().next().unwrap().as_str(),
"acct_b"
);
assert!(active.model_aliases().is_empty());
for scenario in ["accounts", "model", "binding", "key", "group"] {
let mut invalid = snapshot.clone();
invalid.config_revision = Revision::new(2).unwrap();
invalid.instances[0].revision = invalid.config_revision;
let instance = &mut invalid.instances[0];
match scenario {
"accounts" => {
instance.configuration["upstream_registration"]["adapters"][0]["account_ids"] =
serde_json::json!([])
}
"model" => {
instance.configuration["upstream_registration"]["adapters"][0]["models"] =
serde_json::json!(["other"])
}
"binding" => instance.bindings[0].models = vec!["other".into()],
"key" => instance.bindings[0].client_key_ids = vec!["key_one".into()],
"group" => {
instance.bindings[0].account_group_ids =
vec!["grp_0123456789abcdef0123456789abcdef".into()]
}
_ => unreachable!(),
}
let error = PluginPreparation::prepare(&runtime, invalid)
.await
.unwrap_err();
assert_eq!(
error.message(),
"适配模型必须关联同实例的静态账号适配器",
"{scenario}"
);
assert!(active.is_ready());
}
for conflict in [
None,
Some("accounts"),
Some("presentation"),
Some("features"),
Some("alias"),
] {
let mut candidate = snapshot.clone();
candidate.config_revision = Revision::new(2).unwrap();
candidate.instances[0].revision = candidate.config_revision;
let config = &mut candidate.instances[0].configuration;
let mut adapter = config["upstream_registration"]["adapters"][0].clone();
adapter["id"] = "second".into();
adapter["account_ids"] = serde_json::json!([if conflict == Some("accounts") {
"acct_b"
} else {
"acct_a"
}]);
config["upstream_registration"]["adapters"]
.as_array_mut()
.unwrap()
.push(adapter);
let mut model = config["model_catalog"]["models"][0].clone();
model["adapter_id"] = "second".into();
if conflict == Some("presentation") {
model["presentation"]["display_name"] = "Other".into();
}
if conflict == Some("features") {
model["features"] = serde_json::json!([]);
}
config["model_catalog"]["models"]
.as_array_mut()
.unwrap()
.push(model);
if conflict == Some("alias") {
config["model_catalog"]["aliases"] =
serde_json::json!([{"id":"shared-model","provider":"openai","model":"target"}]);
}
let result = PluginPreparation::prepare(&runtime, candidate).await;
if conflict.is_some() {
assert_eq!(result.unwrap_err().message(), "适配模型目录声明冲突");
} else {
let prepared = result.unwrap();
assert_eq!(prepared.models().len(), 2);
assert!(
prepared.models()[0]
.accounts
.is_disjoint(&prepared.models()[1].accounts)
);
}
assert!(active.is_ready());
}
snapshot.config_revision = Revision::new(2).unwrap();
snapshot.instances[0].revision = snapshot.config_revision;
snapshot.instances[0].configuration = serde_json::json!({"startup":"fail"});
*store.snapshot.lock().unwrap() = snapshot.clone();
assert!(
ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(2).unwrap())
.await
.is_err()
);
snapshot.instances[0].enabled = false;
*store.snapshot.lock().unwrap() = snapshot;
let disabled = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(2).unwrap())
.await
.unwrap();
assert!(disabled.models().is_empty());
assert_eq!(active.models().len(), 1);
drop(disabled);
drop(active);
runtime.shutdown().await;
super::super::wait_until_empty(cache.path()).await;
}
#[tokio::test]
async fn adapted_catalog_requires_its_registered_adapter() {
let mut catalog = crate::support::contribution(
Capability::ModelCatalog,
vec![Stage::Registration],
vec![],
vec![],
);
catalog.1.version = 2;
let (cache, store, runtime) =
super::super::setup_with_contributions(Contributions::from([catalog])).await;
let mut snapshot = store.snapshot.lock().unwrap().clone();
snapshot.instances[0].configuration = serde_json::json!({
"model_catalog": {"aliases":[], "models":[{
"id":"shared-model", "adapter_id":"missing", "features":[],
"max_output_tokens":null, "presentation":{}
}]}
});
let error = PluginPreparation::prepare(&runtime, snapshot)
.await
.unwrap_err();
assert_eq!(error.message(), "适配模型必须关联同实例的静态账号适配器");
runtime.shutdown().await;
super::super::wait_until_empty(cache.path()).await;
}
#[tokio::test]
async fn initial_registration_http_uses_the_prepared_settings_without_a_publication() {
use wiremock::{Mock, MockServer, ResponseTemplate, matchers::method};
@@ -848,6 +848,10 @@ async fn adapter_failure_preserves_upstream_diagnostics_and_native_feedback() {
assert_eq!(error.kind(), ProviderErrorKind::RateLimited);
assert_eq!(error.send_state(), UpstreamSendState::Sent);
assert_eq!(error.upstream_status(), Some(429));
assert!(
error.replay_is_safe(),
"宿主观测的单次 HTTP429 应允许 Core 判断重试"
);
assert_eq!(
error.upstream_code().unwrap().as_str(),
"rate_limit_exceeded"
@@ -876,6 +880,197 @@ async fn adapter_failure_preserves_upstream_diagnostics_and_native_feedback() {
super::wait_until_empty(cache.path()).await;
}
#[tokio::test]
async fn adapter_replay_requires_a_single_managed_http_rejection_before_output() {
for scenario in [
"json",
"stream",
"no_request",
"http_200",
"http_503",
"wrong_status",
"wrong_kind",
"sequential",
"concurrent",
"auxiliary",
"unmanaged",
"started",
"text",
"tool",
"failure_wire",
] {
let server = MockServer::start().await;
let status = match scenario {
"http_200" => 200,
"http_503" => 503,
_ => 429,
};
let requests = match scenario {
"no_request" => 0,
"sequential" | "concurrent" => 2,
_ => 1,
};
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(status).set_body_string("{}"))
.expect(requests)
.mount(&server)
.await;
let mut config = configuration(&server.uri(), scenario == "stream");
let callback = config["upstream_callbacks"][0].clone();
match scenario {
"no_request" => config["upstream_callbacks"] = json!([]),
"sequential" => config["upstream_callbacks"] = json!([callback, callback]),
"concurrent" => config["upstream_callbacks"] = json!([[callback, callback]]),
"auxiliary" => {
config["upstream_registration"]["adapters"][0]["paths"][0]["purpose"] =
json!("auxiliary");
}
"unmanaged" => {
config["upstream_callbacks"] = json!([{
"method":"host.http.do", "params":{"method":"POST","url":server.uri()},
"body":"request"
}]);
}
_ => {}
}
let mut events = Vec::new();
if matches!(scenario, "started" | "text" | "tool") {
events.push(json!({"event":{"facts":[{"type":"started","id":"response-one"}]}}));
}
if matches!(scenario, "text" | "tool") {
let kind = if scenario == "text" {
"text"
} else {
"tool_call"
};
events
.push(json!({"event":{"facts":[{"type":"content_added","index":0,"kind":kind}]}}));
let fact = if scenario == "text" {
json!({"type":"text_delta","index":0,"text":"partial"})
} else {
json!({"type":"tool_call_delta","index":0,"id":"call-one","name":"write_file","arguments":"{}"})
};
events.push(json!({"event":{"facts":[fact]}}));
}
let mut failure = json!({
"event":{"facts":[]},
"failure":{"kind":"rate_limited","status":429,"message":"fixture rejection"}
});
if scenario == "wrong_status" {
failure["failure"]["status"] = json!(503);
}
if scenario == "wrong_kind" {
failure["failure"]["kind"] = json!("unavailable");
}
if scenario == "failure_wire" {
failure["wire"] = json!("http");
}
events.push(failure);
config["upstream_events"] = json!(events);
let (cache, _, runtime) = setup(config).await;
let generation =
ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
.await
.unwrap();
let account = Account::new();
let mut stream = execute(&runtime, &generation, Arc::clone(&account), "key-one", None);
let error = loop {
match stream.next().await.expect("fixture must report a failure") {
Ok(_) => {}
Err(error) => break error,
}
};
assert_eq!(
error.kind(),
if scenario == "wrong_kind" {
ProviderErrorKind::Unavailable
} else {
ProviderErrorKind::RateLimited
},
"{scenario}"
);
assert_eq!(
error.replay_is_safe(),
matches!(scenario, "json" | "stream"),
"{scenario}"
);
assert_eq!(
error.send_state(),
if requests == 0 {
UpstreamSendState::NotSent
} else {
UpstreamSendState::Sent
},
"{scenario}"
);
drop(stream);
assert_eq!(Arc::strong_count(&account), 1, "{scenario}");
drop(generation);
runtime.shutdown().await;
super::wait_until_empty(cache.path()).await;
}
}
#[tokio::test]
async fn adapter_rejection_with_pending_http_revokes_the_call_without_replay() {
use tokio::io::{AsyncReadExt as _, AsyncWriteExt as _};
let server = MockServer::start().await;
Mock::given(method("POST"))
.respond_with(ResponseTemplate::new(429).set_body_string("{}"))
.expect(1)
.mount(&server)
.await;
let directory = tempfile::tempdir().unwrap();
let marker = directory.path().join("pending-http-received");
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let pending_url = format!("http://{}/pending", listener.local_addr().unwrap());
let received = marker.clone();
let pending = tokio::spawn(async move {
let (mut socket, _) = listener.accept().await.unwrap();
let mut bytes = [0; 4096];
assert_ne!(socket.read(&mut bytes).await.unwrap(), 0);
std::fs::write(received, b"received").unwrap();
// 不返回响应头;父调用结束必须释放尚未确定结果的 HTTP 请求
while socket.read(&mut bytes).await.unwrap() != 0 {}
let _ = socket.shutdown().await;
});
let mut config = configuration(&server.uri(), false);
let first = config["upstream_callbacks"][0].clone();
config["upstream_callbacks"] = json!([first, {
"method":"host.upstream.http.do",
"params":{"method":"POST","path":pending_url,"headers":[]},
"pending_marker":marker
}]);
config["upstream_events"] = json!([{
"event":{"facts":[]},
"failure":{"kind":"rate_limited","status":429,"message":"fixture rejection"}
}]);
let (cache, _, runtime) = setup(config).await;
let generation = ExtensionPreparationPort::prepare(&runtime, ConfigRevision::new(1).unwrap())
.await
.unwrap();
let account = Account::new();
let mut stream = execute(&runtime, &generation, Arc::clone(&account), "key-one", None);
let error = tokio::time::timeout(Duration::from_secs(10), stream.next())
.await
.unwrap()
.unwrap()
.unwrap_err();
assert_eq!(error.kind(), ProviderErrorKind::RateLimited);
assert!(!error.replay_is_safe());
assert_ne!(error.send_state(), UpstreamSendState::NotSent);
tokio::time::timeout(Duration::from_secs(5), pending)
.await
.unwrap()
.unwrap();
drop(stream);
drop(generation);
runtime.shutdown().await;
assert_eq!(Arc::strong_count(&account), 1);
super::wait_until_empty(cache.path()).await;
}
#[tokio::test]
async fn adapter_overrides_auth_headers_and_rejects_stale_continuation() {
let server = MockServer::start().await;
@@ -1381,6 +1576,168 @@ async fn upstream_adapter_usage_uses_native_account_pricing() {
runtime.shutdown().await;
}
struct RotatingAdapterProvider {
accounts: Vec<Arc<Account>>,
selected: std::sync::Mutex<Vec<ProviderAccountId>>,
}
#[async_trait::async_trait]
impl gateway_core::engine::provider::Provider for RotatingAdapterProvider {
fn name(&self) -> &'static str {
"openai"
}
fn catalog_generation(&self) -> gateway_core::routing::ProviderCatalogGeneration {
Default::default()
}
async fn query_model_capabilities(
&self,
) -> Result<Vec<gateway_core::routing::ProviderModelCapabilities>, ProviderError> {
Ok(vec![gateway_core::routing::ProviderModelCapabilities::new(
UpstreamModelId::new("native-model").unwrap(),
gateway_core::routing::ModelCapabilities::new(
std::collections::BTreeSet::from([
gateway_core::operation::OperationKind::Generate,
]),
None,
),
)])
}
async fn execute(
self: Arc<Self>,
request: gateway_core::engine::provider::ProviderRequest,
context: AttemptContext,
) -> Result<gateway_core::engine::provider::ProviderStream, ProviderError> {
let account = self
.accounts
.iter()
.find(|account| !context.excluded_accounts().contains(&account.id))
.cloned()
.ok_or_else(|| {
ProviderError::new(
ProviderErrorKind::NoEligibleAccount,
UpstreamSendState::NotSent,
)
})?;
self.selected.lock().unwrap().push(account.id.clone());
let candidate = request.candidate();
let model = candidate.upstream_model().unwrap();
let adapter = context
.upstream_adapter_for_account(candidate.provider(), model, &account.id)?
.unwrap();
let metadata = ProviderCallMetadata::new(
candidate.provider().clone(),
model.clone(),
account.id.clone(),
UpstreamTransport::new(adapter.transport()).unwrap(),
);
let events = adapter.execute(UpstreamAdapterInvocation {
operation: request.operation().clone(),
headers: vec![],
context,
metadata: metadata.clone(),
account,
});
Ok(gateway_core::engine::provider::ProviderStream::new(
metadata,
events,
(),
))
}
}
#[tokio::test]
async fn core_rotates_after_a_managed_adapter_http_rejection() {
use gateway_core::engine::{
execution::{ClientTransport, ExecutionRequestMetadata, StartExecution},
provider::{Provider, ProviderRegistry},
};
let Some(environment) = crate::support::environment::Environment::create().await else {
eprintln!("SKIP: plugin integration environment absent");
return;
};
let server = MockServer::start().await;
let calls = Arc::new(AtomicUsize::new(0));
let observed = Arc::clone(&calls);
Mock::given(method("POST"))
.respond_with(move |_: &wiremock::Request| {
let status = if observed.fetch_add(1, Ordering::SeqCst) == 0 {
429
} else {
200
};
ResponseTemplate::new(status).set_body_string("{}")
})
.expect(2)
.mount(&server)
.await;
let first = environment.account(None).await;
let second = environment.account(None).await;
let provider = Arc::new(RotatingAdapterProvider {
accounts: vec![
Account::with_identity(first.as_str(), 1),
Account::with_identity(second.as_str(), 1),
],
selected: Default::default(),
});
let mut config = configuration(&server.uri(), false);
config["upstream_registration"]["adapters"][0]["account_ids"] =
json!([first.as_str(), second.as_str()]);
config["upstream_account_events"] = json!({first.as_str(): [{
"event":{"facts":[]},
"failure":{"kind":"rate_limited","status":429,"message":"fixture rejection"}
}]});
environment.install_plugin(config).await;
let key = "sk-adapter-rotation-fixture";
environment.client_key("key_adapter_rotation", key).await;
let (runtime, core) = environment
.runtime_with_providers(
Default::default(),
ProviderRegistry::new([provider.clone() as Arc<dyn Provider>]).unwrap(),
)
.await;
let execution = core.execution_service();
let operation = Operation::Generate(GenerateRequest::from_protocol_payload(
ProtocolPayload::json_object(
"openai",
json!({"model":"native-model","input":"hello"})
.as_object()
.unwrap()
.clone(),
)
.unwrap(),
));
let mut started = execution
.start(StartExecution {
client: execution.authenticate(key).unwrap(),
public_model: gateway_core::routing::PublicModelId::new("native-model").unwrap(),
operation,
metadata: ExecutionRequestMetadata {
protocol: "openai".into(),
endpoint: "/v1/responses".into(),
transport: ClientTransport::HttpJson,
stream: false,
client_ip: None,
user_agent: None,
previous_response_id: None,
},
})
.await
.unwrap();
started.session.collect_uncommitted().await.unwrap();
started.session.commit_downstream(Some(200)).await.unwrap();
started.session.detach_finalize().await;
assert_eq!(*provider.selected.lock().unwrap(), vec![first, second]);
assert_eq!(calls.load(Ordering::SeqCst), 2);
environment.release_plugin_accounts(&runtime);
drop(execution);
drop(core);
runtime.shutdown().await;
environment.close().await;
}
#[tokio::test]
async fn core_published_adapter_uses_normal_attempt_usage_and_cost_ledger() {
use crate::support::environment::Environment;
@@ -147,3 +147,46 @@ async fn compatibility_warning_rejects_invalid_packages_and_digest_mismatches()
.unwrap_err();
assert_eq!(error.kind(), gateway_admin::model::AdminErrorKind::Invalid);
}
#[tokio::test]
async fn runtime_and_update_preflight_compare_only_declared_contracts() {
let host: PluginHostCompatibility =
serde_json::from_str(include_str!("../../plugin-host-compatibility.json")).unwrap();
for capability_version in [6, 3] {
let mut contribution =
crate::support::contribution(Capability::Middleware, vec![Stage::Http], vec![], vec![]);
contribution.1.version = capability_version;
let archive = crate::support::package_with_contributions(
b"not-an-executable",
Contributions::from([contribution]),
);
let digest = hex::encode(Sha256::digest(&archive));
for version in ["1.0.0", "3.0.0"] {
let version = semver::Version::parse(version).unwrap();
let inspector = PackageInspector::new(PackageLimits::default(), version.clone());
let requirements = inspector
.compatibility(archive.clone(), digest.clone())
.await
.unwrap();
let reasons = host.incompatibilities(&requirements, &version);
assert_eq!(
inspector
.compatibility_warning(archive.clone(), digest.clone())
.await
.unwrap(),
(!reasons.is_empty()).then(|| reasons.join(","))
);
assert_eq!(
reasons.len(),
usize::from(capability_version != 6) + usize::from(version.major != 1)
);
let mut unsupported = requirements;
unsupported.manifest_schema_version = 999;
unsupported.protocol_version = 999;
assert_eq!(
host.incompatibilities(&unsupported, &version).len(),
reasons.len() + 2
);
}
}
}
@@ -29,7 +29,7 @@ fn limits() -> RpcLimits {
maximum_executing_calls: 1,
maximum_queued_calls: 1,
maximum_queue_timeout: Duration::from_millis(200),
maximum_callbacks: 1,
maximum_callbacks: 4,
..Default::default()
}
}
@@ -145,12 +145,13 @@ async fn next_resumption_does_not_need_a_vacant_initial_queue_slot() {
.await
}
});
started.recv().await.unwrap();
// 等待 next 不挤占唯一的普通回调额度
let mut occupying = Box::pin(call(&session, "callback"));
assert!(futures::poll!(&mut occupying).is_pending());
started.recv().await.unwrap();
occupying.await.unwrap();
tokio::time::timeout(Duration::from_secs(2), started.recv())
.await
.expect("next callback should start")
.unwrap();
// 回调通知可能先于 Suspend 到达,持续驱动普通调用才能在名额释放后完成
call(&session, "callback").await.unwrap();
started.try_recv().unwrap();
let mut active = Box::pin(call(&session, "hang"));
assert!(futures::poll!(&mut active).is_pending());
let mut queued = Box::pin(call(&session, "echo"));
@@ -165,7 +166,13 @@ async fn next_resumption_does_not_need_a_vacant_initial_queue_slot() {
assert!(!waiting.is_finished());
drop(active);
assert!(queued.await.is_ok());
assert!(waiting.await.unwrap().is_ok());
assert!(
tokio::time::timeout(Duration::from_secs(2), waiting)
.await
.expect("next call should resume after execution capacity is released")
.unwrap()
.is_ok()
);
assert!(session.is_ready());
session.shutdown(Duration::from_secs(1)).await;
}
@@ -326,3 +333,246 @@ async fn resumption_queue_wait_preserves_the_remaining_plugin_processing_budget(
assert!(session.is_ready());
session.shutdown(Duration::from_secs(1)).await;
}
async fn scheduled<C: CallbackHandler + 'static>(
callbacks: Arc<C>,
mut limits: RpcLimits,
) -> (tempfile::TempDir, Arc<RpcSession>) {
let (cache, prepared, mut handshake, processes) = prepared_session();
handshake.configuration = json!({"sdk_session": true});
limits.maximum_queue_timeout = Duration::from_secs(2);
let session = RpcSession::start(prepared, handshake, limits, &processes, callbacks)
.await
.unwrap();
(cache, Arc::new(session))
}
struct CrossCallbacks {
other: std::sync::Mutex<std::sync::Weak<RpcSession>>,
ready: Arc<tokio::sync::Barrier>,
}
impl CallbackHandler for CrossCallbacks {
fn call(
&self,
_context: CallContext,
_method: String,
_params: serde_json::Value,
_payload: Vec<u8>,
) -> BoxFuture<'static, Result<RpcReply, PluginFault>> {
let other = self.other.lock().unwrap().upgrade().unwrap();
let ready = self.ready.clone();
Box::pin(async move {
ready.wait().await;
other
.call(
"echo",
other.context(Stage::Request, Duration::from_secs(2)),
json!("child"),
Vec::new(),
)
.await
.map_err(|_| PluginFault::new(gateway_plugin_sdk::ErrorCode::Fault, "child failed"))
})
}
}
#[tokio::test]
async fn independent_sessions_release_execution_for_every_callback_method() {
let barrier = Arc::new(tokio::sync::Barrier::new(2));
let left = Arc::new(CrossCallbacks {
other: Default::default(),
ready: barrier.clone(),
});
let right = Arc::new(CrossCallbacks {
other: Default::default(),
ready: barrier,
});
let (_left_cache, a) = scheduled(left.clone(), limits()).await;
let (_right_cache, b) = scheduled(right.clone(), limits()).await;
*left.other.lock().unwrap() = Arc::downgrade(&b);
*right.other.lock().unwrap() = Arc::downgrade(&a);
for method in [
"host.service.call",
"host.http.dispatch",
"host.model.execute",
"host.client.verify",
NEXT_METHOD,
"host.unclassified",
] {
let invoke = |session: Arc<RpcSession>| async move {
session
.call(
"callback",
session.context(Stage::Request, Duration::from_secs(2)),
json!({"method":method}),
Vec::new(),
)
.await
};
let (a_result, b_result) = tokio::time::timeout(Duration::from_secs(3), async {
tokio::join!(invoke(a.clone()), invoke(b.clone()))
})
.await
.unwrap();
assert_eq!(a_result.unwrap().result, json!("child"), "{method}");
assert_eq!(b_result.unwrap().result, json!("child"), "{method}");
}
a.shutdown(Duration::from_secs(1)).await;
b.shutdown(Duration::from_secs(1)).await;
}
struct PartialCallbacks {
entered: mpsc::UnboundedSender<String>,
fast: Arc<Semaphore>,
slow: Arc<Semaphore>,
}
impl CallbackHandler for PartialCallbacks {
fn call(
&self,
_context: CallContext,
method: String,
_params: serde_json::Value,
_payload: Vec<u8>,
) -> BoxFuture<'static, Result<RpcReply, PluginFault>> {
self.entered.send(method.clone()).unwrap();
let gate = match method.as_str() {
"fast" => Some(self.fast.clone()),
"slow" => Some(self.slow.clone()),
_ => None,
};
Box::pin(async move {
if let Some(gate) = gate {
gate.acquire().await.unwrap().forget();
}
Ok(RpcReply {
result: json!(method),
payload: Vec::new(),
})
})
}
}
#[tokio::test]
async fn a_partial_callback_result_requires_a_permit_and_can_wait_again() {
let (entered, mut events) = mpsc::unbounded_channel();
let fast = Arc::new(Semaphore::new(0));
let slow = Arc::new(Semaphore::new(0));
let (cache, session) = scheduled(
Arc::new(PartialCallbacks {
entered,
fast: fast.clone(),
slow: slow.clone(),
}),
limits(),
)
.await;
let parent = tokio::spawn({
let session = session.clone();
async move { call(&session, "partial").await }
});
for _ in 0..2 {
tokio::time::timeout(Duration::from_secs(2), events.recv())
.await
.unwrap()
.unwrap();
}
let marker = cache.path().join("processing");
let busy = tokio::spawn({
let session = session.clone();
let marker = marker.clone();
async move {
session
.call(
"busy",
session.context(Stage::Request, Duration::from_secs(2)),
json!({"marker":marker, "delay_ms":300}),
Vec::new(),
)
.await
}
});
tokio::time::timeout(Duration::from_secs(2), async {
while !marker.exists() {
tokio::task::yield_now().await;
}
})
.await
.unwrap();
fast.add_permits(1);
assert!(
tokio::time::timeout(Duration::from_millis(50), events.recv())
.await
.is_err()
);
busy.await.unwrap().unwrap();
assert_eq!(
tokio::time::timeout(Duration::from_secs(2), events.recv())
.await
.unwrap()
.as_deref(),
Some("processed")
);
assert!(!parent.is_finished());
// 再次等待慢回调后,独立调用可取得同一个执行名额
assert!(call(&session, "echo").await.is_ok());
slow.add_permits(1);
assert_eq!(parent.await.unwrap().unwrap().result, json!("complete"));
session.shutdown(Duration::from_secs(1)).await;
}
#[tokio::test]
async fn polling_wakeups_and_cancelled_suspensions_leave_the_session_usable() {
let (entered, mut events) = mpsc::unbounded_channel();
let slow = Arc::new(Semaphore::new(0));
let (_cache, session) = scheduled(
Arc::new(PartialCallbacks {
entered,
fast: Arc::new(Semaphore::new(0)),
slow,
}),
RpcLimits {
maximum_calls: 1,
..limits()
},
)
.await;
assert_eq!(
call(&session, "yield").await.unwrap().result,
json!("yielded")
);
let parent = tokio::spawn({
let session = session.clone();
async move {
session
.call(
"callback",
session.context(Stage::Request, Duration::from_secs(2)),
json!({"method":"slow"}),
Vec::new(),
)
.await
}
});
assert_eq!(events.recv().await.as_deref(), Some("slow"));
assert!(matches!(
call(&session, "echo").await,
Err(RpcError::Overloaded)
));
parent.abort();
assert!(matches!(parent.await, Err(error) if error.is_cancelled()));
tokio::time::timeout(Duration::from_secs(2), async {
loop {
match call(&session, "echo").await {
Ok(_) => break,
Err(RpcError::Overloaded) => tokio::task::yield_now().await,
Err(error) => panic!("{error}"),
}
}
})
.await
.unwrap();
assert!(session.is_ready());
session.shutdown(Duration::from_secs(1)).await;
}
@@ -0,0 +1,167 @@
//! 验证调用表在成功、远端错误和超时的结果交付前同步撤销回调资源
use std::{
sync::{
Arc, Condvar, Mutex,
atomic::{AtomicBool, Ordering},
},
time::Duration,
};
use gateway_plugin_runtime::{RpcError, RpcLimits, RpcSession};
use gateway_plugin_sdk::Stage;
use serde_json::json;
use super::session::{LifecycleCallbacks, prepared_session};
struct GatedFinish {
ready: Arc<tokio::sync::Notify>,
entered: std::sync::mpsc::Sender<()>,
started: AtomicBool,
finished: AtomicBool,
released: Mutex<bool>,
release: Condvar,
}
impl gateway_plugin_runtime::CallbackHandler for GatedFinish {
fn finish(&self, _: &gateway_plugin_sdk::CallContext) {
if self.started.swap(true, Ordering::AcqRel) {
return;
}
self.entered.send(()).unwrap();
// 测试屏障模拟撤销处理中被调度器暂停,不依赖恰好命中的线程交错
let released = self.released.lock().unwrap();
let (released, timeout) = self
.release
.wait_timeout_while(released, Duration::from_secs(5), |released| !*released)
.unwrap();
assert!(!timeout.timed_out() && *released);
self.finished.store(true, Ordering::Release);
}
fn call(
&self,
_: gateway_plugin_sdk::CallContext,
method: String,
_: serde_json::Value,
_: Vec<u8>,
) -> futures::future::BoxFuture<
'static,
Result<gateway_plugin_runtime::RpcReply, gateway_plugin_sdk::PluginFault>,
> {
let ready = Arc::clone(&self.ready);
Box::pin(async move {
assert_eq!(method, "test.stream_ready");
ready.notified().await;
Ok(gateway_plugin_runtime::RpcReply {
result: json!({}),
payload: vec![],
})
})
}
}
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn dropping_stream_waits_for_resource_revocation_even_after_end_or_process_failure() {
for method in ["stream_single", "stream_single_crash"] {
let (_cache, prepared, handshake, processes) = prepared_session();
let (entered, entry) = std::sync::mpsc::channel();
let callbacks = Arc::new(GatedFinish {
ready: Arc::new(tokio::sync::Notify::new()),
entered,
started: AtomicBool::new(false),
finished: AtomicBool::new(false),
released: Mutex::new(false),
release: Condvar::new(),
});
let session = Arc::new(
RpcSession::start(
prepared,
handshake,
RpcLimits::default(),
&processes,
callbacks.clone(),
)
.await
.unwrap(),
);
let stream = session
.call_stream(
method,
session.context(Stage::Request, Duration::from_secs(5)),
json!({}),
vec![],
)
.await
.unwrap();
// 拿到流后才允许对端结束;整个屏障控制不依赖被暂停的 I/O 调度器
callbacks.ready.notify_one();
entry.recv_timeout(Duration::from_secs(5)).unwrap();
let observed = Arc::clone(&callbacks);
let (started, received) = std::sync::mpsc::channel();
let (completed, completion) = std::sync::mpsc::channel();
let dropped = std::thread::spawn(move || {
started.send(()).unwrap();
drop(stream);
completed
.send(observed.finished.load(Ordering::Acquire))
.unwrap();
});
received.recv_timeout(Duration::from_secs(1)).unwrap();
// 撤销屏障会暂停 I/O 调度线程,不能用同一 runtime 的计时器释放它
let early = completion.recv_timeout(Duration::from_millis(100));
*callbacks.released.lock().unwrap() = true;
callbacks.release.notify_one();
let was_early = early.is_ok();
let revoked =
early.unwrap_or_else(|_| completion.recv_timeout(Duration::from_secs(1)).unwrap());
dropped.join().unwrap();
session.shutdown(Duration::from_secs(1)).await;
assert!(
!was_early && revoked,
"{method}: stream drop must not outrun callback revocation"
);
}
}
#[tokio::test]
async fn terminal_results_revoke_callback_resources_before_waking_callers() {
for method in ["echo", "deny", "hang"] {
let (_cache, prepared, handshake, processes) = prepared_session();
let callbacks = Arc::new(LifecycleCallbacks::default());
let session = RpcSession::start(
prepared,
handshake,
RpcLimits::default(),
&processes,
callbacks.clone(),
)
.await
.unwrap();
let timeout = if method == "hang" {
Duration::from_millis(50)
} else {
Duration::from_secs(2)
};
let result = session
.call(
method,
session.context(Stage::Request, timeout),
json!({}),
Vec::new(),
)
.await;
match method {
"echo" => assert!(result.is_ok()),
"deny" => assert!(matches!(result, Err(RpcError::Remote(_)))),
"hang" => assert!(matches!(result, Err(RpcError::Timeout))),
_ => unreachable!(),
}
assert_eq!(callbacks.begun.load(Ordering::Relaxed), 1);
assert!(
callbacks.finished.load(Ordering::Relaxed) >= 1,
"调用结果已经交付,资源权限必须先同步撤销:{method}"
);
session.shutdown(Duration::from_secs(1)).await;
}
}

Some files were not shown because too many files have changed in this diff Show More