docs: record openai parity progress

Co-authored-by: Codex <noreply@openai.com>
This commit is contained in:
Zyy
2026-06-12 11:33:45 +08:00
co-authored by Codex
parent 8c72eec5d8
commit 37f19f7fb7
3 changed files with 164 additions and 4 deletions
+4 -4
View File
@@ -15,12 +15,12 @@ Update this file before each feature commit.
| Cookie persistence | Completed | `4710926` + `5d3cc6e` | `cargo test cookie_jar_captures_and_replays_account_scoped_cookies`; `cargo test cookie_repository_encrypts_and_replays_account_scoped_cookies`; full `fmt/test/clippy` before commit | Adds account-scoped in-memory Cookie jar and encrypted SQLite Cookie repository; Cookies are replayed only for the same account/domain. |
| Account pool and refresh | Partial | `0f97c5c` + `2f25db8` + `4dc0c9a` + `c2cf7a6` + `636fa79` + `fdb9374` | `cargo test account_pool_skips_expired_disabled_banned_and_quota_exhausted_accounts`; `cargo test --test account_pool_scheduling_test`; `cargo test --test account_refresh_test`; `cargo test --test oauth_refresh_test`; `cargo test --test v1_upstream_route_test`; `cargo test --test account_pool_startup_test`; full `fmt/test/clippy` before commit | Implements base account status filtering, refresh scheduler semantics, real OpenAI OAuth refresh-token exchange, 401-triggered refresh and single retry for `/v1/responses`, max concurrent slots, stale slot cleanup, least-used/round-robin/sticky strategies, tier priority, quota-cache filtering, Cloudflare cooldown skip, request staggering metadata, preferred account selection, exclude IDs, capacity summary, settings-backed pool construction, same-process import-to-pool sync, and startup pool snapshots from SQLite with usage-based last-used metadata. Still missing durable release-time slot state, broader upstream retry/fallback wiring, and admin account CRUD. |
| Account repository | Completed | `14fe706` | `cargo test --test account_repository_test`; `cargo fmt --check`; `cargo clippy --all-targets --all-features --locked -- -D warnings`; `cargo test` | Adds encrypted SQLite account token storage, decrypting reads, cursor pagination, status/label updates, release-time usage counter persistence, and refresh-token-preserving token updates. |
| Translation | Completed | `eff182e` | `cargo test chat_completion_translates_to_codex_response_request`; full `fmt/test/clippy` before commit | Adds OpenAI Chat request types, Codex Responses request type, translation, and OpenAI-compatible error body helper. |
| HTTP routes | Partial | `ab0ad8f` + `3830b21` + `6179d44` + `cfec8ba` + `6341336` + `228febd` + `c2cf7a6` + `e9fdea0` + `ee0ed90` + `fdb9374` + `9ca5c7d` | `cargo test v1_requires_client_api_key_not_admin_cookie`; `cargo test models_route_returns_openai_compatible_codex_model_list`; `cargo test --test admin_settings_route_test`; `cargo test --test model_catalog_test`; `cargo test --test admin_accounts_route_test`; `cargo test --test admin_api_keys_route_test`; `cargo test --test admin_usage_stats_route_test`; `cargo test --test v1_upstream_route_test`; full `fmt/test/clippy` before commit | Adds Axum router, health route, `/v1/models`, `/v1/models/catalog`, `/v1/models/{id}`, `/v1/models/{id}/info`, `/debug/models`, `/admin/accounts`, `/admin/accounts/import`, `/admin/api-keys`, `/admin/usage-stats`, `/admin/usage-stats/summary`, `/v1/responses` HTTP SSE collection with output item/delta reconstruction plus `output_text` sync for non-streaming clients and `stream:true` passthrough through imported Codex accounts, `/v1` unauthorized OpenAI error body, global request-id response header, binary listener, `gpt-5.5` as the default/test model baseline, and read-only `/admin/settings` with admin-session auth and lower camelCase envelope. `/admin/accounts` reads metadata only and does not decrypt tokens for list views. Still missing real `/v1/chat/completions` route wiring, OAuth/device/CLI account login flows, admin account mutation/health/quota/Cookie routes, full admin API-key mutation/export/import routes, settings mutation/persistence, diagnostics, backend model refresh, model cache persistence, and WebSocket Responses routes. |
| Translation | Completed | `eff182e` + `b9c30ba` | `cargo test --test routes_chat_test`; `cargo test --test chat_completions_route_test`; full `cargo fmt --check && cargo test && cargo clippy --all-targets --all-features --locked -- -D warnings` before `b9c30ba` | Adds OpenAI Chat request types, expanded Codex Responses request type, OpenAI Chat to Codex translation for system/developer instructions, multimodal content, tool/function calls, response formats, reasoning effort, service tier, and parallel tool calls. Codex SSE now converts back to OpenAI chat completions and chat chunks with text deltas, reasoning content, tool calls, usage details, and OpenAI-compatible error bodies. |
| HTTP routes | Partial | `ab0ad8f` + `3830b21` + `6179d44` + `cfec8ba` + `6341336` + `228febd` + `c2cf7a6` + `e9fdea0` + `ee0ed90` + `fdb9374` + `9ca5c7d` + `b9c30ba` | `cargo test v1_requires_client_api_key_not_admin_cookie`; `cargo test models_route_returns_openai_compatible_codex_model_list`; `cargo test --test admin_settings_route_test`; `cargo test --test model_catalog_test`; `cargo test --test admin_accounts_route_test`; `cargo test --test admin_api_keys_route_test`; `cargo test --test admin_usage_stats_route_test`; `cargo test --test v1_upstream_route_test`; `cargo test --test chat_completions_route_test`; full `cargo fmt --check && cargo test && cargo clippy --all-targets --all-features --locked -- -D warnings` before `b9c30ba` | Adds Axum router, health route, `/v1/models`, `/v1/models/catalog`, `/v1/models/{id}`, `/v1/models/{id}/info`, `/debug/models`, `/admin/accounts`, `/admin/accounts/import`, `/admin/api-keys`, `/admin/usage-stats`, `/admin/usage-stats/summary`, `/v1/responses` HTTP SSE collection with output item/delta reconstruction plus `output_text` sync for non-streaming clients, default streaming Responses passthrough when `stream` is omitted, Responses field parity for service tier, tool choice, parallel tool calls, text format, prompt cache key, include, client metadata, reasoning include default, and Codex context headers, plus a dedicated `/v1/chat/completions` route backed by imported Codex accounts. `/v1` unauthorized and model errors remain OpenAI-compatible, global request-id response headers are preserved, `gpt-5.5` remains the default/test model baseline, and `/admin/settings` is read-only with admin-session auth and lower camelCase envelope. `/admin/accounts` reads metadata only and does not decrypt tokens for list views. Still missing OAuth/device/CLI account login flows, admin account mutation/health/quota/Cookie routes, full admin API-key mutation/export/import routes, settings mutation/persistence, diagnostics, backend model refresh, model cache persistence, live Chat streaming transform, and real WebSocket Responses transport. |
| Admin/frontend response contract | Completed | `1c34ea7` + `b0c10f2` | `cargo test admin_envelope_serializes_lower_camel_case_request_id`; `cargo test admin_logs_reject_missing_admin_session_cookie`; `cargo test v1_response_has_request_id_header_without_admin_body_field`; full `fmt/test/clippy` before commit | Adds reusable admin envelopes with body codes, lower camelCase JSON, paginated list shape, requestId bodies, admin session cookie enforcement, and keeps `/v1/*` OpenAI-compatible. |
| HTTP auth boundary | Completed | `cec4607` + `e9fdea0` | `cargo test --test http_auth_test`; `cargo test --test routes_responses_test`; `cargo test --test admin_api_keys_route_test`; `cargo test --test admin_logs_route_test`; `cargo fmt --check`; `cargo clippy --all-targets --all-features --locked -- -D warnings`; `cargo test` | Adds a real `src/http/auth.rs` boundary for client API key and admin session cookie parsing. `/v1/*` accepts only enabled local `Bearer cpr_...` client API keys verified from SQLite, admin session parsing ignores client API keys, and route handlers no longer own ad hoc credential parsing. |
| Model catalog | Partial | `6341336` | `cargo test --test model_catalog_test`; `cargo test model_catalog_route_returns_codex_metadata_without_alias_entries`; full `fmt/test/clippy` before commit | Adds a native static Codex model catalog with `gpt-5.5` default, configured aliases, reasoning/service-tier suffix parsing, OpenAI-compatible `/v1/models`, extended catalog/detail/info routes, and debug summary. Still missing backend Codex model fetching per plan, cache persistence, `/admin/refresh-models`, plan allowlist integration from fetched models, and full model metadata parity. |
| Upstream lifecycle | Partial | `88cad87` + `225fbff` + `c2cf7a6` + `636fa79` + `ee0ed90` + `fdb9374` + `5bf012c` + `9ca5c7d` | `cargo test responses_route_rejects_non_codex_provider_models`; `cargo test v1_requires_client_api_key_not_admin_cookie`; `cargo test --test codex_sse_usage_test`; `cargo test --test codex_client_test`; `cargo test --test oauth_refresh_test`; `cargo test --test v1_upstream_route_test`; `cargo test --test account_pool_startup_test`; full `fmt/test/clippy` before commit | `/v1` enforces client API key shape, rejects non-Codex provider models through the model catalog, returns OpenAI-compatible errors, includes reusable SSE/usage helpers, sends Codex Desktop HTTP SSE requests through imported accounts, replays/captures encrypted Cookies, collects `response.completed`, reconstructs missing assistant output from `response.output_item.done` or text deltas, synchronizes `output_text` for non-streaming clients, passes through client `stream:true` SSE, refreshes and retries once after upstream 401 when a refresh token is stored, records token usage, writes v1 lifecycle event logs, records refresh failure classification for upstream 401, and restores the account pool from SQLite on startup. Still missing Codex request context fields, SSE error/failed mapping, rate-limit header capture, retry/fallback across accounts beyond refresh retry, 429 quota cache, 402/403 classification, and previous-response/session affinity. |
| Codex WebSocket boundary | Completed | `cc0516d` | `cargo test --test codex_websocket_test`; `cargo fmt --check`; `cargo clippy --all-targets --all-features --locked -- -D warnings`; `cargo test` | Adds `src/codex/websocket.rs` with tested transport classification for HTTP SSE versus WebSocket-only requests. `previous_response_id` and explicit WebSocket requests now fail through a typed boundary instead of being silently downgraded; real WebSocket transport remains an upstream lifecycle follow-up. |
| Upstream lifecycle | Partial | `88cad87` + `225fbff` + `c2cf7a6` + `636fa79` + `ee0ed90` + `fdb9374` + `5bf012c` + `9ca5c7d` + `b9c30ba` | `cargo test responses_route_rejects_non_codex_provider_models`; `cargo test v1_requires_client_api_key_not_admin_cookie`; `cargo test --test codex_sse_usage_test`; `cargo test --test codex_client_test`; `cargo test --test oauth_refresh_test`; `cargo test --test v1_upstream_route_test`; `cargo test --test account_pool_startup_test`; full `cargo fmt --check && cargo test && cargo clippy --all-targets --all-features --locked -- -D warnings` before `b9c30ba` | `/v1` enforces client API key shape, rejects non-Codex provider models through the model catalog, returns OpenAI-compatible errors, includes reusable SSE/usage helpers, sends Codex Desktop HTTP SSE requests through imported accounts, replays/captures encrypted Cookies, forwards Codex request context headers, normalizes `service_tier: fast` to upstream `priority`, collects `response.completed`, reconstructs missing assistant output from `response.output_item.done` or text deltas, synchronizes `output_text` for non-streaming clients, passes through Responses streaming SSE by default, adapts Chat Completions responses/chunks, refreshes and retries once after upstream 401 when a refresh token is stored, records token usage, writes v1 lifecycle event logs, records refresh failure classification for upstream 401, and restores the account pool from SQLite on startup. Still missing SSE error/failed mapping, rate-limit header capture, retry/fallback across accounts beyond refresh retry, 429 quota cache, 402/403 classification, previous-response/session affinity, and live Chat streaming transform. |
| Codex WebSocket boundary | Completed | `cc0516d` + `b9c30ba` | `cargo test --test codex_websocket_test`; full `cargo fmt --check && cargo test && cargo clippy --all-targets --all-features --locked -- -D warnings` before `b9c30ba` | Adds `src/codex/websocket.rs` with tested transport classification for HTTP SSE versus WebSocket-only requests. `previous_response_id` and explicit WebSocket requests now fail through a typed boundary instead of being silently downgraded, and the local-only `use_websocket` transport flag is not serialized into upstream JSON. Real WebSocket transport remains an upstream lifecycle follow-up. |
| Fingerprint updates | Partial | `dd26d31` + `e32220a` | `cargo test update_manifest_updates_app_version_and_build_number`; `cargo test fingerprint_updater_fetches_manifest_and_persists_history`; full `fmt/test/clippy` before commit | Parses a JSON update manifest and persists app version/build number history to SQLite. Still missing original appcast XML polling, update-state persistence, runtime header fingerprint selection from the latest stored fingerprint, and optional local Codex.app Chromium extraction. |
| Runtime docs and packaging | Completed | `33b63f3` | `cargo fmt --check`; `cargo clippy --all-targets --all-features --locked -- -D warnings`; `cargo test`; `cargo run`; `curl -s -i http://127.0.0.1:8080/health` | README scope, `.env.example`, Dockerfile, compose file, and health smoke test are complete. |
@@ -0,0 +1,116 @@
# OpenAI GPT Codex Parity Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Port the seven OpenAI GPT/Codex parity gaps from the TypeScript reference into the Rust service without adding outbound proxy pools or non-GPT providers.
**Architecture:** Add real Chat Completions support beside Responses, expand Codex request types and transport, then harden the shared upstream lifecycle. Keep route parsing, translation, transport, account state, model catalog, and admin operations in focused modules that match the existing Rust layout.
**Tech Stack:** Rust, Axum, Reqwest/rustls, serde, sqlx/SQLite, wiremock, tokio, futures, optional WebSocket transport dependency.
**Checkpoint 2026-06-12:** Commit `b9c30ba` completed Tasks 1-3 for the in-scope OpenAI GPT/Codex path: Chat Completions route/translation/output, Responses request field parity, default Responses streaming, Codex context headers, and local-only WebSocket flag serialization cleanup. Remaining major work starts at Task 4: real WebSocket transport for `previous_response_id`, then account fallback/rate limits, model refresh/cache, and scoped admin operations.
---
### Task 1: Chat Completions Translation
**Files:**
- Modify: `src/translation/openai_to_codex.rs`
- Modify: `src/codex/types.rs`
- Test: `tests/routes_chat_test.rs`
- [x] Write failing tests for system/developer instructions, tool calls, function outputs, image parts, response_format, reasoning effort, and service tier.
- [x] Run: `cargo test --test routes_chat_test`
- [x] Expand Chat request structs and translation helpers.
- [x] Run: `cargo test --test routes_chat_test`
### Task 2: Chat Completions Route And Output
**Files:**
- Modify: `src/app.rs`
- Modify: `src/http/v1.rs`
- Modify: `src/translation/codex_to_openai.rs`
- Test: `tests/chat_completions_route_test.rs`
- [x] Write failing route tests proving `/v1/chat/completions` sends translated Codex payloads and returns OpenAI chat JSON/SSE.
- [x] Run: `cargo test --test chat_completions_route_test`
- [x] Add a dedicated `chat_completions` handler and Codex-to-OpenAI collectors/streamers.
- [x] Run: `cargo test --test chat_completions_route_test`
### Task 3: Responses Field Parity
**Files:**
- Modify: `src/codex/types.rs`
- Modify: `src/http/v1.rs`
- Modify: `src/codex/client.rs`
- Test: `tests/responses_field_parity_test.rs`
- [x] Write failing tests for `service_tier`, `tool_choice`, `parallel_tool_calls`, `text.format`, `prompt_cache_key`, `include`, `client_metadata`, and Codex context headers.
- [x] Run: `cargo test --test v1_upstream_route_test`
- [x] Expand request parsing, serde output, and header/body forwarding.
- [x] Run: `cargo test --test v1_upstream_route_test`
### Task 4: WebSocket Previous Response Support
**Files:**
- Modify: `Cargo.toml`
- Modify: `src/codex/websocket.rs`
- Modify: `src/codex/client.rs`
- Test: `tests/codex_websocket_test.rs`
- [ ] Write failing tests proving `previous_response_id` uses WebSocket and HTTP fallback is not used when server-side history is required.
- [ ] Run: `cargo test --test codex_websocket_test`
- [ ] Add WebSocket response.create transport that emits SSE-compatible output.
- [ ] Run: `cargo test --test codex_websocket_test`
### Task 5: Upstream Retry, Fallback, And Rate Limits
**Files:**
- Modify: `src/http/v1.rs`
- Modify: `src/accounts/pool.rs`
- Modify: `src/accounts/repository.rs`
- Test: `tests/upstream_fallback_test.rs`
- Test: `tests/account_pool_scheduling_test.rs`
- [ ] Write failing tests for 429 quota marking, retry-after handling, fallback account retry, refresh retry preservation, and exhausted no-account responses.
- [ ] Run: `cargo test --test upstream_fallback_test --test account_pool_scheduling_test`
- [ ] Add error classification and fallback acquire/release paths without adding proxy-pool support.
- [ ] Run: `cargo test --test upstream_fallback_test --test account_pool_scheduling_test`
### Task 6: Model Catalog Refresh
**Files:**
- Modify: `src/models/catalog.rs`
- Create: `src/models/repository.rs`
- Modify: `src/http/v1.rs`
- Test: `tests/model_catalog_test.rs`
- [ ] Write failing tests for backend model snapshots, suffix parsing including `none` and `minimal`, plan allowlist generation, and `/admin/refresh-models`.
- [ ] Run: `cargo test --test model_catalog_test`
- [ ] Add SQLite-backed model cache and refresh route using existing Codex accounts.
- [ ] Run: `cargo test --test model_catalog_test`
### Task 7: Scoped Admin Account Operations
**Files:**
- Modify: `src/http/admin.rs`
- Modify: `src/accounts/repository.rs`
- Modify: `src/cookies/repository.rs`
- Test: `tests/admin_accounts_route_test.rs`
- Test: `tests/admin_api_keys_route_test.rs`
- [ ] Write failing tests for manual add, delete, batch status, label, quota fetch, health check, cookie get/set/delete, import/export, API key disable/delete.
- [ ] Run: `cargo test --test admin_accounts_route_test --test admin_api_keys_route_test`
- [ ] Implement scoped admin operations needed to operate Codex-backed OpenAI GPT routes.
- [ ] Run: `cargo test --test admin_accounts_route_test --test admin_api_keys_route_test`
### Task 8: Final Verification And Docs
**Files:**
- Modify: `docs/implementation-status.md`
- Modify: `README.md` if route behavior changes
- [ ] Run: `cargo fmt --check`
- [ ] Run: `cargo test`
- [ ] Run: `cargo clippy --all-targets --all-features --locked -- -D warnings`
- [ ] Update implementation status with completed parity items and any intentionally omitted non-goals.
@@ -0,0 +1,44 @@
# OpenAI GPT Codex Parity Design
## Goal
Port the OpenAI GPT/Codex compatibility logic from `/home/zyy/桌面/Codes/codex-proxy` into this Rust service while keeping the Rust project intentionally scoped to Codex-backed OpenAI-compatible APIs.
## Explicit Non-Goals
- Do not port outbound proxy pools, proxy assignment, or proxy health checks.
- Do not port Anthropic, Gemini, Ollama, custom provider routing, Electron, frontend UI, or update flows.
- Do not add direct third-party OpenAI API-key provider routing. The `/v1` API remains backed by imported ChatGPT/Codex accounts.
## Architecture
The Rust service will use a shared Codex upstream lifecycle for `/v1/responses` and `/v1/chat/completions`. Route-specific code parses and translates client API shapes, then delegates account acquisition, Codex transport, retry/fallback, usage recording, cookie capture, and event logging to common helpers.
OpenAI Chat compatibility is implemented as a real route, not an alias to Responses. Chat requests translate to Codex Responses requests, and Codex SSE output translates back to OpenAI chat completion JSON or chat completion chunks.
Responses compatibility keeps native Responses passthrough semantics but adds the TS implementation's in-scope fields: reasoning/service tier, tool choice, parallel tool calls, text formats, prompt cache key, include, client metadata, and Codex context headers. `previous_response_id` uses WebSocket transport rather than being rejected.
## Components
- `src/translation/openai_to_codex.rs`: OpenAI Chat request schema and Chat-to-Codex translation, including messages, tools/functions, multimodal input, response formats, reasoning effort, and service tier.
- `src/translation/codex_to_openai.rs`: Codex SSE to OpenAI Chat JSON/SSE conversion, including text, reasoning, tool calls, usage, and errors.
- `src/codex/types.rs`: Expanded Codex request/response types for Responses and WebSocket.
- `src/codex/client.rs`: HTTP SSE and WebSocket Codex transport, context headers, cookies, and rate-limit header capture.
- `src/http/v1.rs`: Separate `/v1/responses` and `/v1/chat/completions` handlers sharing upstream lifecycle helpers.
- `src/accounts/pool.rs` and repositories: fallback selection, quota/rate-limit updates, durable usage release, and account state mutation hooks.
- `src/models/catalog.rs`: static catalog plus persisted backend snapshots and admin refresh trigger.
- `src/http/admin.rs`: scoped account/API-key/account-health/quota/cookie/admin model operations needed to operate Codex-backed GPT compatibility.
## Data Flow
For Chat Completions, the route validates the local client API key, parses the OpenAI request, validates the model against the catalog, translates to a Codex request, acquires an account, sends the request, and converts the Codex stream into OpenAI Chat output. Streaming clients receive OpenAI chat completion chunks and `[DONE]`; non-streaming clients receive a `chat.completion` object.
For Responses, the route validates the local client API key, parses native Responses fields, builds a Codex request, acquires an account, sends via WebSocket when required, and otherwise streams raw Codex SSE or collects `response.completed` for non-streaming clients.
## Error Handling
Client authentication errors stay OpenAI-compatible. Model misses return `model_not_found`. Upstream 401 triggers refresh and one retry for the same account. 429 and quota exhaustion update account state and retry another account when available. Retryable transport/empty-response failures can retry another account. Terminal upstream errors are formatted as OpenAI-compatible errors for Chat and Responses-compatible failed events for streaming Responses.
## Testing
Development follows test-first changes. Tests cover translation units, route behavior with `wiremock`, WebSocket transport boundaries, fallback account retry, rate-limit/quota state changes, model refresh cache behavior, and admin account operations. Existing full verification remains `cargo fmt --check`, `cargo test`, and `cargo clippy --all-targets --all-features --locked -- -D warnings`.