The AdminLoginSource extension was never injected, so every login request
fell back to the shared "unknown" failure bucket: five wrong passwords from
any unauthenticated visitor locked the real administrator out for fifteen
minutes, renewable indefinitely. Attach a login-route middleware that keys
the bucket on the socket peer IP taken from ConnectInfo; forwarded headers
are client-forgeable and stay excluded, so reverse-proxy deployments retain
the previous shared-bucket behavior. Add a router-level regression test
proving throttling of one source leaves other sources unaffected.
Co-authored-by: Claude <noreply@anthropic.com>
Connection tests run through a transient execution store, so a failed probe
left no trace at all: nothing in model_requests, nothing in ops_events, and
nothing in the log at info level. The reason a probe failed — upstream status,
provider error code, send state — was only reachable by reproducing it.
Add a ProbeFailure port on ExecutionStore, defaulting to a no-op so existing
implementations are unaffected, and have the execution service report probe
failures to the persistent store while the probe itself keeps using the
transient one. The Postgres store logs a warning and appends an ops event with
no model_request_id.
Such events already fit the surrounding contracts: the ops error list left
joins model_requests, health metrics inner join it so probes stay out of
business failure rates, and retention already targets rows with a null
model_request_id.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Selection-stage failures all collapsed into a generic "upstream service is
unavailable", so a 30s runtime cooldown was indistinguishable from a real
upstream outage. The connection test pins a single account and has no
fallback, making every retry inside the window look like a dead upstream.
Split the runtime-cooldown case out of NoEligibleSession into a new
AccountCoolingDown variant carrying the remaining wait, and attach a
structured code and error type to each selection failure. The admin
connection test now renders the reason and the remaining cooldown.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Split the dashboard summary into two three-way query batches so one render holds at most three pool connections and cannot starve data-plane writes under the five-second acquire timeout. Restore the fresh account read before the passive-quota availability write-back: the revision fence only covers credential rotation, so deciding transitions from the selection-time snapshot could overwrite availability changes made by other paths and delay exhausted-account recovery. Drop the redundant admin_audit_events retention index (the initial created_at index already serves the delete predicate) and document the rolling-deploy indexing procedure in the migration.
Co-authored-by: Claude <noreply@anthropic.com>
Drop the account re-read from passive rate-limit quota sync: the quota CAS is fenced on the account revision and the follow-up state write carries the same fence, so the availability transition can use the selection-time snapshot and converge on the next response. Stream Grok inference chunks as Bytes end to end instead of copying every chunk into a fresh Vec.
Co-authored-by: Claude <noreply@anthropic.com>
Add partial indexes for the three retention delete predicates and delete in bounded ctid batches with per-batch commits, replacing the hourly full-table scan and unbounded single-transaction delete. Run the six dashboard-summary queries concurrently so page latency tracks the slowest query instead of their sum. Make the PostgreSQL pool budget configurable via an optional store.pool section (max_connections, acquire_timeout_seconds) with a 5-second acquire timeout default so pool exhaustion fails fast instead of queueing for 30 seconds.
Co-authored-by: Claude <noreply@anthropic.com>
Combine the model-request insert with the first attempt into one ExecutionStore call (default implementation keeps the sequential behavior; the Postgres store overrides it with a single INSERT), removing one database round trip from the pre-first-token critical path. Reuse a session-scoped deadline timer in the event pull loop instead of registering a new global timer per event, and attach continuation session state in place so a uniquely held payload is no longer force-deep-copied by clone-before-make_mut.
Co-authored-by: Claude <noreply@anthropic.com>
Track a scanned-prefix cursor in SseEventDecoder so a large frame arriving across many chunks is scanned incrementally instead of rescanned from the start on every push (separator matching backs off three bytes to catch CRLF pairs split across chunks). Decode WebSocket response.create frames through a Map-accepting request-decode core, dropping the serialize-and-reparse round trip on the request start path.
Co-authored-by: Claude <noreply@anthropic.com>
Parse each WebSocket frame once in the exchange reducer and reuse the parsed JSON across rate-limit, metadata, turn-state, and forwarding extraction (previously up to eight full parses per frame). Serialize the response.create frame by borrowing the request body instead of deep-copying it (byte-identical output, including a client-supplied type key), skip audit artifact construction entirely when no audit directory is configured, and take pending turn-state updates instead of cloning them on every chunk.
Co-authored-by: Claude <noreply@anthropic.com>
Add a serde(other) variant to GrokCatalogReasoningEffort and drop unrecognized efforts from the capability projection, mirroring the api_backend handling: a novel effort value in one model previously failed the whole snapshot parse and collapsed /v1/models to the default model once the cache expired.
Co-authored-by: Claude <noreply@anthropic.com>
Scope the completed-usage-fact predicate to per-aggregate FILTER clauses in request_metrics and request_metric_series instead of the whole query: the global filter pinned outcome='succeeded', so failure, cancelled, incomplete, and caller-error counts were structurally zero and the dashboard always reported a 100% success rate. Outcome counts now cover every request in range while token, cache, latency, and cost-coverage aggregates remain restricted to committed usage facts.
Co-authored-by: Claude <noreply@anthropic.com>
Cross-major releases are withheld from has_update with an explanatory warning, and perform_update rejects a cross-major target before fetching the release: config and migration contracts are only compatible within a major, so a one-click cross-major update previously produced a binary that fails to start. Same-major updates keep the existing flow.
Co-authored-by: Claude <noreply@anthropic.com>
Replace the wildcard allow_headers with an explicit whitelist (authorization, content-type, x-api-key, and the configured request-id header): tower-http asserts against wildcard headers combined with allow_credentials, so any non-empty cors_allowed_origins previously panicked inside initialize() before the server could listen. Add a preflight assembly test covering a configured origin.
Co-authored-by: Claude <noreply@anthropic.com>
Surface the upstream error message, code, and type from response.failed/error stream events through the client-visible error pipeline, matching the reference translation baseline and restoring fatal-vs-retryable classification and retry pacing for downstream Codex clients. Messages are scrubbed of account fingerprints (UUIDs) and control characters before exposure; unstructured failures keep the sanitized generic error, and non-2xx HTTP transport errors keep their existing rewrap semantics.
Co-authored-by: Claude <noreply@anthropic.com>
Rewrite the refresh, token-classification, passthrough, and stream-tolerance comments to state the behavior and constraints they guard, dropping change narration and reviewer-facing baseline references. Also apply rustfmt reflow in the openai test files.
Co-authored-by: Claude <noreply@anthropic.com>
Replace the fixed 10-minute refresh retry with the shared Redis-backed exponential backoff (5s x3, capped at 5m): transient and rejected refresh failures record a per-account attempt count inside a 30-minute window and defer the next attempt accordingly, and a successful rotation resets the counter. Aligns the xAI refresh path with the openai provider; invalid_grant/banned stay terminal and ambiguous failures keep their 24-hour cooldown.
Co-authored-by: Claude <noreply@anthropic.com>
Enable raw SSE passthrough on the WebSocket upstream so client bytes are never re-serialized, preserving numeric precision on the transparent /v1/responses path.
Stop invalidating accounts on transient credential-refresh failures: classify 5xx/429 by status, defer identity-unavailable instead of terminating, and retry with a Redis-backed exponential backoff (5s x3, capped at 5m) that resets on success. Only invalid_grant/banned/rejected remain terminal, restoring the pre-refactor and official-client behavior.
Co-authored-by: Claude <noreply@anthropic.com>
Tolerate field-validation failures on non-terminal content events (empty deltas, duplicate indices, missing fields) and skipped transforms by forwarding the wire event without canonical extraction, matching the reference translation baseline. Terminal completed/incomplete events keep failing closed so usage and cost validation are preserved; genuine upstream errors still propagate.
Co-authored-by: Claude <noreply@anthropic.com>
Drop production APIs with no callers: the model-request history module, dead ClientApiKey/ProviderAccount repository methods, the CollectedResponse IR, canonical_facts_mut, the UsageExtraction error variant, unused xAI credential-refresh helpers, and the unused uuid/zeroize/chrono dependencies.
Co-authored-by: Claude <noreply@anthropic.com>
Return the Codex catalog only for client_version requests while preserving the standard OpenAI-compatible model list for other clients. Keep the login action pending until successful route loading completes.
Co-authored-by: Codex <noreply@openai.com>
Read Grok reasoning effort menus from the official catalog, expose xhigh to Codex, and generate command-authenticated client configuration that refreshes remote model metadata without advertising unsupported priority service.
Co-authored-by: Codex <noreply@openai.com>
Include each update event ID in the SSE JSON payload so the frontend appends distinct progress rows instead of replacing one undefined-ID row.
Co-authored-by: Codex <noreply@openai.com>
Record successful and failed official npm checks, retain the last successful version on failure, and publish the checked timestamp through the dashboard wire profile.
Co-authored-by: Codex <noreply@openai.com>
Keep per-account HTTP/2 pools warm, build cold clients outside the shared cache lock, and record cache, DNS, and response-header timings for successful and failed requests.
Co-authored-by: Codex <noreply@openai.com>
Restore the pre-refactor update stage events and human-readable download progress, preserve progressPercent through the admin SSE wire, and wait for the stream subscription before starting replacement.
Co-authored-by: Codex <noreply@openai.com>
Preserve transient credential failures as runtime cooldowns, restore account-aware retry and observability facts, and align Grok model/tool adaptation with Codex clients.
Keep complete SSE events on the live path while retaining only protocol-required tool and continuation state.
Co-authored-by: Codex <noreply@openai.com>
Refresh verified dependencies, harden CI test credentials, and align release builders with the current supported image.
Co-authored-by: Codex <noreply@openai.com>
Serve connection-test model lists from a provider-plan Redis cache, refresh manually through bounded same-plan candidates, and keep periodic catalog refreshes isolated from OpenAI ETag handling.
Co-authored-by: Codex <noreply@openai.com>
Restore provider-native quota and reasoning observations, complete request lifecycle accounting, and align account and usage surfaces with the restored semantics.
Co-authored-by: Codex <noreply@openai.com>
Keep OpenAI Responses bodies and wire events transparent, remove the generic typed compaction and response reconstruction path, and localize Grok compaction adaptation to the xAI provider.
Co-authored-by: Codex <noreply@openai.com>
Collapse routing, health, circuit state, account pools, and admin APIs onto the fixed OpenAI and xAI Provider boundary while keeping credential runtime state account-scoped.
Remove configurable provider-instance and routing-catalog persistence, use fixed official endpoints in provider adapters, align account routes and frontend contracts, and mirror provider tests under their source modules.
Co-authored-by: Codex <noreply@openai.com>
Allow explicitly selected provider accounts to be deleted regardless of scheduling state, matching the pre-refactor behavior.
Upsert verified credential imports by upstream identity, preserve existing account IDs, and return the committed IDs to admin callers.
Co-authored-by: Codex <noreply@openai.com>
Remove unused full-response transports and unconnected domain reservations while keeping test-only setup inside integration tests.
Restore image-generation accounting and OpenAI WebSocket pool observations across the engine, store, and admin boundaries without changing the transparent OpenAI WS/SSE path.
Co-authored-by: Codex <noreply@openai.com>
Move protocol and provider fixtures and observations into integration tests while exercising the production decoder, encoder, transport, DNS, and lease paths.
Remove direct credential seeding and routing test setters while preserving OpenAI transparent proxy and WebSocket/SSE fallback behavior.
Co-authored-by: Codex <noreply@openai.com>
Add typed compaction operations across the gateway, OpenAI, and xAI providers with provider-owned request, response, accounting, and observability contracts.
Align xAI custom apply_patch calls with the patch function argument and re-encode restored custom calls during provider replay.
Co-authored-by: Codex <noreply@openai.com>
Restore provider-owned identity profiles, credential refresh, cache routing, model catalogs, and runtime configuration across OpenAI and xAI without compatibility paths.
Keep deployment configuration, frontend visibility, and architecture documentation aligned with the finalized multi-provider boundary.
Co-authored-by: Codex <noreply@openai.com>
Move control-plane use cases, host lifecycle, provider capabilities, execution orchestration, and worker contracts into their final owners while keeping the application bootstrap as a thin composition root.
Restore verified OpenAI transport, continuation, pooling, fingerprint, accounting, and observability semantics, and retain the validated xAI multi-account integration.
Co-authored-by: Codex <noreply@openai.com>
Rebuild the backend as the frozen gateway workspace with a single PostgreSQL schema, provider-owned OpenAI and xAI boundaries, and mirrored domain tests.
Restore the admin product contracts for accounts, model catalogs, usage observability, billing details, and provider-scoped settings without compatibility shims.
Co-authored-by: Codex <noreply@openai.com>
zyycnandClaude
