Commit Graph
145 Commits
Author SHA1 Message Date
Zyy 4a8b839780 fix(core): release account leases at upstream stream termination (#742) 2026-11-18 22:34:10 +08:00
ZyyandCodex 4d4c1e48ce feat(plugins)!: expose account binding and selection roles to schedulers
Scheduler v4 reports session ownership separately from native selection preferences and pins. Only eligible candidates are annotated, and cache preferences do not imply a session binding.

BREAKING CHANGE: scheduler plugins must declare capability version 4 and accept the new candidate fields.

Co-authored-by: Codex <noreply@openai.com>
2026-11-17 16:48:06 +08:00
ZyyandCodex f9e364860a fix(openai): rotate native continuations after excessive retry advice
Migrate the session binding before requesting a full client replay when safe native continuation recovery exceeds the configured wait limit. Remove stale capacity retry headers from client responses while preserving upstream diagnostics and account rotation limits.

Co-authored-by: Codex <noreply@openai.com>
2026-11-16 18:40:17 +08:00
d13a4a2138 feat(scheduling): add independent quota reset window weights (#713)
Rank quota scheduling strictly by configured reset windows and persist its settings independently from smart scheduling.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-14 18:13:33 +08:00
d99ecf9aaf fix(openai): align model operations with global client profiles (#708)
Apply the global request profile to account catalog discovery, warmup,
and diagnostic routing. Freeze each operation's identity through the
existing provider profile resolver.

Refresh verified offline releases and centralize their version, artifact,
and environment defaults. Cover ZIP64 MSIX metadata and keep release
update tests independent of the current bundled versions.

Fixes #703

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-14 10:40:12 +08:00
ZyyandZyy 4888681a04 feat(scheduling): weight quota resets by window in smart scheduling (#694)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-11-12 18:56:30 +08:00
ZyyandCodex e2f194875f refactor: clarify protocol ownership and request observations
Separate protocol identifiers from Provider identities and centralize shared
request facts and observations in gateway-protocol. Keep endpoint models
and private continuation state owned by Providers.

Reorganize the architecture documentation and clarify the shutdown guarantees
for buffered charges and observations.

Validated with workspace formatting, strict Clippy, 3013 affected-module
tests and 16 architecture checks. One reference-binary TLS test is ignored.

Co-authored-by: Codex <noreply@openai.com>
2026-11-11 16:04:16 +08:00
koeriaandCursor Agent 914c00785c fix(openai): skip same-account retry on NO_MORE_RETRY (#675)
Treat x-retry-metadata: NO_MORE_RETRY as a hard signal to skip
same-account backoff and transport recovery, even when Retry-After
is within maxRetryAfterSeconds. Capacity failures still rotate under
the existing policy; server_error still returns the upstream error.

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-11-11 14:08:24 +08:00
ZyyandZyy 2fc347508a fix(peer): allow grouped API accounts and reject forwarding loops (#671)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-11-11 01:04:20 +08:00
ZyyandCodex dd65665128 feat(protocol): support Messages ingress and API upstreams
Support Responses, Chat Completions and Messages ingress with Responses
or Messages API upstreams in the existing OpenAI provider.

Share generation admission and delivery, preserve native protocol payloads,
and translate cross-protocol tools, reasoning, streams and usage. Add
Messages token counting and editable upstream protocol settings.

Co-authored-by: Codex <noreply@openai.com>
2026-11-10 22:07:46 +08:00
Zyy 16b3671733 fix(accounts): show proxy failures in connection tests 2026-11-09 12:05:11 +08:00
ZyyandZyy e97b1faf0d feat(openai): derive installation identities from key seeds (#629)
* feat(openai): derive installation identities from key seeds

* docs: explain installation seed migration effects

* docs: fix the key configuration reference

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-11-07 00:33:17 +08:00
ZyyandZyy ee7e837d1f feat(scheduling): split queue timeouts and allow account fallback (#625)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-11-06 23:09:29 +08:00
ZyyandZyy 6de1beb1df perf(openai): reduce scheduling reads and stabilize session affinity (#618)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-11-06 14:44:42 +08:00
6859520969 fix(api): broaden Chat compatibility and capture ingress failures (#610)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-05 11:53:21 +08:00
6b362a5029 fix(accounts): keep failed recovery probes from thawing accounts (#605)
Carry the freeze recovery target into provider account selection so probe responses cannot update credentials or clear the active protection. Let the recovery worker postpone or release the same freeze after observing the upstream result.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-04 19:30:22 +08:00
ZyyandCodex 19ef484969 feat(peer-link)!: share upstream key concurrency and RPM limits
Use a fresh upstream key concurrency limit as the account default for scheduling, leases, and capacity reporting. Show disclosed limits alongside the upstream pool summary.

BREAKING CHANGE: Both peers must support CPR Link schema v3.

Refs: #587

Co-authored-by: Codex <noreply@openai.com>
2026-11-04 18:32:42 +08:00
39aa20a13a fix(plugins): prefer account-scoped profiles for adapted models (#588)
* fix(plugins): prefer account-scoped model profiles in catalogs

Co-authored-by: Codex <noreply@openai.com>

* fix(models): honor scoped plugin profiles in fallback catalogs

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-11-04 13:20:08 +08:00
0ff1a72919 fix(retry): limit upstream Retry-After waits (#578)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-11-02 13:07:20 +08:00
ZyyandCodex a5d3d9f12d fix(peer): recover quota from explicit upstream budget state
Export authoritative budget access flags and keep monetary fields display-only
for peer consumers. Use the official usage_limit_reached error type for budget
exhaustion while preserving ordinary rate-limit cooldowns.

Apply fresh quota facts atomically with peer snapshots through the shared quota
writer, retaining connection, claim, and observation-time guards. Reuse the
existing sync worker and document the recovery contract.

Validated with 234 selected tests, strict workspace Clippy, and Rustfmt.

Co-authored-by: Codex <noreply@openai.com>
2026-10-31 15:58:02 +08:00
ZyyandCodex 958e13c370 fix(errors): standardize client messages and correct error mappings
Return non-retryable client version errors and correct provider error types.
Keep safe user messages separate from operational diagnostics.
Unify error wording and document the conventions in the development skill.

Co-authored-by: Codex <noreply@openai.com>
2026-10-29 01:57:31 +08:00
736eca6593 feat(accounts): support configurable error protection rules (#547)
* feat(accounts): support configurable error protection rules

Replace capacity-specific freezing with bounded, configurable account error rules and shared attempt feedback.

Drop legacy settings without conversion and expose rule editing and freeze reasons in the admin UI.

Co-authored-by: Codex <noreply@openai.com>

* fix(accounts): preserve error protection scheduling boundaries

Keep xAI accounts excluded while recovery probes are pending, and cover
selection before and after recovery.

Separate scheduling cooldown reads from admin display details, share the
atomic Redis migration and selection logic, and use one policy revision.
Remove the unused adaptive concurrency reduction path and its tests.

Validation: 393 targeted backend tests, workspace Clippy with -D warnings,
Rustfmt, frontend type checking and build, and local service health checks.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-28 21:50:51 +08:00
19fbb5da75 feat(plugin): support account-scoped adapted model catalogs (#533)
* feat(plugin): publish account-scoped adapted model catalogs

* fix(core): preserve candidate account scope during execution

Apply the narrowed candidate scope before provider account selection, including affinity and account rotation. Cover adapted-only models and same-name models with incompatible tool capabilities through the real OpenAI selector and local HTTP fixtures.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-27 13:37:57 +08:00
e38cf31323 refactor(plugins): separate client identity from model execution ports (#527)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-26 13:21:31 +08:00
614c81c375 refactor(plugins)!: separate identity queries and cooperative scheduling (#525)
Resolve identity and frozen access facts without constructing model execution
contexts. Coordinate parent task suspension, resumption and retained callbacks
at the SDK and Runtime boundary, with bounded admission and cancellation.
Expose account HTTP, client verification, access facts and quota details through
the existing host owners.

BREAKING CHANGE: Plugin sessions require RPC v7; rebuild and repackage plugins
with the matching SDK. No legacy protocol negotiation or fallback is provided.

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-26 13:05:08 +08:00
ZyyandCodex 3f6607209b refactor(usage): remove reported model tracking and polish UI
Remove upstream response model tracking across providers, storage, APIs,
and plugin facts without migrating historical records.

Refine sidebar hints, locale loading indicators, model columns, and usage
page labels and filters.

Co-authored-by: Codex <noreply@openai.com>
2026-10-25 04:03:27 +08:00
ZyyandCodex 3c05bc8f07 refactor(plugins): remove provider-specific request parameters
Use request, attempt, and outbound middleware for plugin-owned identity
selection and complete request rewriting. Preserve native account identity
and partition WebSocket connections through existing outbound opening changes.

Remove the construction parameter channel and its obsolete tests and docs.
Keep middleware at v6 without a compatibility shim.

Co-authored-by: Codex <noreply@openai.com>
2026-10-25 03:30:52 +08:00
2e7a39e1e8 feat(plugins): add provider request composition middleware (#511)
* feat(plugins): add provider request composition middleware

Expose per-attempt provider parameters and a final outbound preparation chain using middleware v6. Preserve native transport behavior when no outbound scope matches.

Co-authored-by: Codex <noreply@openai.com>

* fix(plugins): sync workbench with middleware v6

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-24 22:58:06 +08:00
16aa08091c feat(plugins): select upstream adapters by account (#509)
* feat(plugins): select upstream adapters within account scopes

* fix(plugins): defer adapter decisions until account selection

Keep scoped adapters pending until an account is known so unavailable
fallbacks cannot mask a healthy earlier match. Filter native xAI accounts
when only an adapter can accept the request, preserving normal selection
and cold-stream behavior.

Cover fallback precedence and mixed-pool prewarm eligibility regressions.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-24 20:20:47 +08:00
ZyyandCodex b4a5210e44 test: streamline behavior coverage and parallel execution
Remove redundant contract cases and express fixtures in terms of current behavior.
Reduce isolated database migration contention, run independent scenarios concurrently,
and replace fixed waits with clock and I/O synchronization. Check absolute Redis
session expiry and wait for PostgreSQL migration session locks to be released.
Document nextest for local test execution.

Validated 4060 tests, 70 Redis ACL tests, and five repeat runs of 26 focused tests.
The 10 plugin state tests also pass 50 repeat runs with four workers.
Rustfmt and strict Clippy pass. Local full-suite execution with 16 workers falls
from 162.760s to 89.072s, excluding compilation.

Co-authored-by: Codex <noreply@openai.com>
2026-10-22 23:30:38 +08:00
Zyy f7f59103a3 fix: harden request lifecycles and align repository documentation
Preserve request evidence and cold middleware boundaries across cancellation. Make backup auditing atomic, retain committed account-group results, and serialize restart operations.

Unify frontend stream authentication and session recovery, clear stale secrets, and remove redundant runtime paths. Correct CI permissions and database test URL handling.

Align documentation with current module responsibilities, APIs, and validation workflows, including both submodules.
2026-10-22 22:06:01 +08:00
ZyyandCodex 3f95f8b0c4 feat(diagnostics): preserve evidence and export ZIP bundles
Capture bounded request fingerprints and terminal usage without rewriting payloads.
Reserve critical trace events and report capture gaps centrally.
Export sanitized ZIP bundles with summaries through native browser downloads.
Simplify account export filenames.

Co-authored-by: Codex <noreply@openai.com>
2026-10-22 19:35:30 +08:00
b8479b59db fix(plugins): preserve request origin and raw upstream observations (#497)
* fix(plugins): preserve request origin and raw upstream observations

Co-authored-by: Codex <noreply@openai.com>

* perf(plugins): share immutable response observations

Avoid copying response metadata and header collections for every stream event. Update Workbench byte headers and virtual-time timeout coverage, and align the large-request fixture with RPC v5.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-21 01:09:58 +08:00
5c115fb38f fix(core): reserve rotation budget during connection recovery (#487)
* fix(retry): rotate accounts after connection failures

Reliable NotSent HTTP connection failures stay on the same account and
egress until the shared 30-second opening window is exhausted. When an
account's proxy does not respond, each opening waits for the full
15-second connect timeout on the same route, so requests that selected
that account fail after about 30 seconds while other accounts stay idle.

After a connection failure, exclude the account and reselect through
the existing rotation path, consuming routing and rotation budget and
sharing the same opening window. Rotation happens after the first
failure because a second same-account opening on an unresponsive
egress would leave too little of the window to rotate. If no other
account is eligible, restore the exclusion and continue the original
same-account HTTP recovery, so single-account deployments keep their
current behavior. Required accounts and continuation attempts are
unchanged.

Refs #485

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(retry): retain same-account connection recovery

Keep existing account rotation rules and cover rotation after a recovered
connection receives a replay-safe rate-limit rejection.

Co-authored-by: Codex <noreply@openai.com>

* fix(core): reserve connection budget for account rotation

Retry the original account before rotating on unsent connection failures.
Split the retry window and retain an opening for HTTP recovery on another
account, with bounded fallback when no alternative can be selected.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-20 17:42:42 +08:00
ZyyandCodex 99ba5b1120 fix(openai): align request ordering and simplify timing metrics
Preserve response.create field order and align OAuth query ordering with
current official Codex sources.

Stop adding the timing-metrics request header by default and remove the
four unused engine TTFT and token interval fields throughout the backend,
plugin observation contracts, and frontend. Keep rate calculations unchanged.

Co-authored-by: Codex <noreply@openai.com>
2026-10-18 16:32:15 +08:00
a82bcdb8f8 fix(providers): preserve upstream failures across account selection (#462)
* fix(openai): distinguish session policy exclusions from unavailable accounts

Co-authored-by: Codex <noreply@openai.com>

* fix(providers): preserve upstream failures after selection exhaustion

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: 2233224230 <111681546+2233224230@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
2026-10-17 21:19:00 +08:00
ZyyandCodex a0e708071a perf(plugins)!: optimize RPC and simplify host boundaries
Move business JSON out of control metadata, encode request snapshot headers
compactly, and reuse encoded frames. Cache compiled configuration schemas
and prepare plugin instances with bounded concurrency.

Ignore unused plugin result fields and remove redundant business validation
while preserving host authorization, resource ownership, and capacity limits.
Keep response decoding failures local to their calls.

Seal callback side effects before completing or cancelling RPC calls, and
recover abandoned state migrations without allowing old owners to promote.

Validation: SDK 112 tests and its doctest, Runtime 203 tests, Core 542 tests,
40 PostgreSQL plugin tests, and 16 architecture checks passed. Strict Clippy,
formatting, and diff checks passed.

BREAKING CHANGE: RPC protocol v4 changes frame payload layout and request
snapshot header encoding. Plugins must update their SDK and protocol
manifest version and be rebuilt.

Co-authored-by: Codex <noreply@openai.com>
2026-10-17 20:54:41 +08:00
338cde1505 feat(scheduling): add plugin filtering and remove automatic switchback (#446)
* feat(scheduling): add plugin filtering and remove automatic switchback

Add scheduler v2 filtered delegation across native selection, affinity,
capacity waits, and lease retries. Keep plugin configuration and business
rules owned by plugins without legacy scheduler compatibility.

Remove automatic switchback from the runtime, SDK, and settings UI.
Migrate the existing settings JSON without creating new tables.

Co-authored-by: Codex <noreply@openai.com>

* fix(dev): align plugin source checks with scheduler v2

Update the workbench submodule to the v2-capable SDK and scheduler,
and document the explicit scheduler manifest version.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-16 15:13:56 +08:00
b25f96559a feat(groups): control image generation by account group (#445)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-16 14:00:14 +08:00
ZyyandCodex a8a96d8919 feat(diagnostics): generate support bundles on the backend
Collect request, retry, error and recovery evidence with bounded reads and
credential redaction. Keep ordinary detail responses small and expose a
shared download action next to the close button.

Preserve export metadata when limits are reached and deduplicate selected
errors without discarding business context based on generic field names.

Co-authored-by: Codex <noreply@openai.com>
2026-10-16 12:49:18 +08:00
ZyyandCodex e0a022518c refactor(backend): simplify shared rules and redundant tests
Reuse Core error classification and token presentation, remove unused
operation APIs and redundant tests, and avoid unnecessary context cloning.
Fix expired reset timestamp underflow and SSE metadata frame boundaries.
Consolidate architecture checks and document the backend audit results.

Validation: backend tests, architecture tests, rustfmt and strict Clippy.
PostgreSQL/Redis integration was not verified without dedicated services.

Co-authored-by: Codex <noreply@openai.com>
2026-10-15 13:46:11 +08:00
ZyyandCodex f6085206eb fix(retry): suppress repeated return-error retries for failed turns
Track short-lived failures by client key, thread, and turn. Skip RetryThenReturnError retries on subsequent requests, clear markers on success with version checks, and preserve other retry policies.

Co-authored-by: Codex <noreply@openai.com>
2026-10-15 12:11:29 +08:00
ZyyandCodex d700df8a4f fix(openai): separate transient retry exhaustion policies
Retry ordinary 503 server errors up to five times on the same account with capped backoff and positive jitter. Preserve upstream Retry-After and return the final error without rotating accounts; retain rotation for explicit capacity rejections.

Co-authored-by: Codex <noreply@openai.com>
2026-10-14 18:12:18 +08:00
ZyyandCodex d814582c09 fix(openai): forward response preludes without buffering
Deliver HTTP/SSE and WebSocket events immediately while preserving bounded
recovery after replaceable preludes. Share prelude classification so response
metadata does not hide WebSocket connection-limit recovery.

Keep irreversible delivery, cancellation, middleware, and retry-budget guards.
Document the distinction between transport keepalive and event idle timeouts.

Refs #405

Co-authored-by: Codex <noreply@openai.com>
2026-10-14 01:51:53 +08:00
ZyyandCodex fce543d4ce feat(settings): remove account request interval limits
Co-authored-by: Codex <noreply@openai.com>
2026-10-13 21:50:53 +08:00
a9a86071bf fix: preserve terminal failure and cleanup causes (#417)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-13 11:31:10 +08:00
ZyyandCodex 9e02e52fec fix(responses): keep SSE alive before first output
Start downstream keepalives while admitted streaming requests await their first output, including middleware processing. Separate transport commitment from the replay barrier so heartbeat frames preserve safe retries and execution accounting.

Co-authored-by: Codex <noreply@openai.com>
2026-10-12 16:57:28 +08:00
45b473c7e2 fix(openai): preserve Lite interrupts across prelude retries (#390)
* fix(openai): preserve Lite interrupts across prelude retries

Deliver known Responses Lite WebSocket preludes before semantic output while retaining existing recovery guards. Keep interrupt intent across retries and retarget it only after the replacement response is delivered.

Co-authored-by: Codex <noreply@openai.com>

* fix(openai): forward Lite events without waiting for output

Deliver unclassified Lite WebSocket events in order and reuse the existing commit boundary to stop replay after their delivery. Preserve error classification, retry budgets, account recovery, and interrupt handling.

Co-authored-by: Codex <noreply@openai.com>

---------

Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-11 22:53:59 +08:00
ZyyandCodex cc69368f77 perf(store): scale request analytics and isolate request history (#389)
Build durable request facts and bounded aggregate queries, freeze historical
identities, and move recovery lease updates out of request history rows.
Align token fields across storage, APIs and pages.

Invalidate analytics only after actual request deletions and track embedded
migration inputs during builds.

Co-authored-by: Codex <noreply@openai.com>
2026-10-11 20:57:36 +08:00
57e284d7d1 fix(openai): preserve stream recovery before semantic output (#380)
Co-authored-by: Zyy <zyycn@users.noreply.github.com>
Co-authored-by: Codex <noreply@openai.com>
2026-10-10 16:30:26 +08:00