- read every multipart backup payload to its requested length
- make remote verification failures diagnosable
- refine status details, loading icons, and attempt-chain presentation
Co-authored-by: Codex <noreply@openai.com>
quota_limit_reached without Redis cooldown maps to quota_exhausted
now that rate_limited comes only from 429 cooldown. Adjust summary
assertions: quota_exhausted 1→2, rate_limited 1→0.
Co-authored-by: Codex <noreply@openai.com>
Unify account status derivation in gateway-admin via AccountStatusSignals,
removing the duplicated rules in store and use_case. AccountStatus now has
five operational classes (normal/quota_exhausted/rate_limited/disabled/error),
with AccountErrorReason carrying the concrete cause.
Add provider_accounts.last_error_message, written by record_failure with the
raw upstream error and cleared on recovery to ready. The admin error badge
shows the cause and upstream text on hover.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
quota_recovery_confirmed had one call site; inlining the evidence match
into quota_success_state makes the recovery rules (successful response
always recovers; refreshed snapshot requires old reset elapsed) visible
at the decision point. Rust best practices §1.8: don't extract helpers
for single-call simple logic.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dashboard reads the materialized column instead of parsing raw quota
JSON (design §3.3/§5.3): Provider computes snapshot-level limit_reached
on quota writes, store persists it, Admin account_status and Dashboard
both consume the same value.
- QuotaObservation gains limit_reached; openai writes fact.exhausted()
(429 always true), xai writes used_percent>=100
- compare_and_swap_provider_quota updates the column; get_quotas and
ACCOUNT_SELECT read it back
- ProviderAccountSummary carries quota_limit_reached; admin_account_status
derives RateLimited from it (Unknown -> Invalid)
- Dashboard provider_account_metrics reads quota_limit_reached instead of
provider_quota_json->'rate_limit'
- Migration 0003 adds the column with backfill (coalesce for NULL when
rate_limit key is absent); 0004 fixes NULL backfill for already-applied
environments
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Split credential/quota.rs into quota/{mod,snapshot,recovery}.rs with
corresponding test modules. Snapshot aggregation is snapshot-level:
top-level limit_reached/allowed and any window used>=100 count as
exhaustion, scheduling signals survive limit-only windows, and window
rolling is O(1) arithmetic.
402 recovery uses the pre-write confirmed reset (old reset elapsed plus
fresh snapshot not exhausted, or explicit allowed=true) instead of the
new snapshot's future reset, fixing the worker recovery deadlock.
Scheduling projection TTL is capped at the nearest active reset so
cache expiry never outlives the limit.
Admin derives seven mutually exclusive statuses (Active/RateLimited/
QuotaExhausted/Expired/Invalid/Disabled/Banned) from availability plus
snapshot-level quota.limitReached; Unknown maps to invalid. Dashboard
metrics are mutually exclusive (rate_limited disjoint from active),
ignore reset-expired windows, and cover additional rate limits.
xAI: bare 402 writes a short account cooldown instead of persisting
QuotaExhausted; ModelQuotaExhausted keeps model scope; runtime cooldown
survives credential rotation; account deletion clears all account and
model-scoped cooldown keys via new ProviderCooldownPort::clear_all.
Migration 0002 is frozen in the sha256 manifest.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Remove the persistent Cooldown state and cooldown_until/availability_reason
columns; rate limiting (429/503) now lives entirely in provider quota data
with projected window rolling, while availability carries only terminal or
intervention-required states (QuotaExhausted for 402, worker auto-recovers
on quota reset). xAI migrated to the same model (runtime cooldown via Redis,
no persistent state). Frontend derives 限流中 from quota window limitReached.
Align quota window parsing with the official codex schema: drop the invented
monthly_limit/spend_control windows, add the top-level code_review_rate_limit
window, prefer upstream limit_name for display labels, and keep duration
guessing only for grouping.
Also fix: account directory degrades one failing quota to an empty projection
instead of failing the whole page; dashboard rate_limited bucket SQL; fake
store Mutex re-entrancy deadlock; remove dead tokenRefreshing field.
Migration 0002 drops cooldown columns and rewrites the availability CHECK.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Default wire profile os_type to macOS to match the audited official
desktop user-agent
- Retry quota usage 5xx rejections with 1s/2s exponential backoff,
leaving 4xx to the existing quota state transitions
- Classify 503 with Retry-After into a resettable cooldown instead of
an unclassified transient warning
- Add account_id and conversation hash to WebSocket pump exit logs so
idle-connection resets can be attributed to an account
Co-authored-by: Codex <noreply@openai.com>
Dispatch releases from the long-lived publishing branch so Cargo caches are shared across version tags. Store prebuilt image caches in per-architecture GHCR refs and include toolchain and workspace manifests in Cargo cache invalidation.
Co-authored-by: Codex <noreply@openai.com>
Fold the completed provider snapshot, S3 backup, and redundant-index changes into the initial PostgreSQL baseline. Keep the settings section tabs aligned with the input surface color.
Co-authored-by: Codex <noreply@openai.com>
When upstream rejects a NewChain request with websocket_connection_limit_reached
before the first deliverable event, retry once on a fresh connection and
downgrade to HTTP/SSE if the new connection is still limited or the retry
preparation fails, matching the official retry semantics. Non-NewChain
requests keep the previous behavior of delivering the error frame as-is,
and the code is persisted in upstream diagnostics. All pre-delivery
fallbacks share one helper that marks the transport decision and wait
metrics.
Co-authored-by: Codex <noreply@openai.com>
The official Codex app-server enables enable_request_compression by
default (Stable, default_enabled=true) and sends /v1/responses bodies as
zstd level 3 with Content-Encoding: zstd. Serialize the upstream request
to JSON, compress with zstd level 3, and attach the encoding header on
the HTTP/SSE path; WebSocket continues to use permessage-deflate.
Adds RequestBodyEncode/RequestCompression error variants, a zstd capture
helper for wiremock tests, and a transport test asserting the compressed
body decodes back to the original JSON.
Co-authored-by: Codex <noreply@openai.com>
The example config moved to codex_version 0.146.0 and desktop_build 6119,
but the rejection tests still replaced the old values, silently no-op'ing
and accepting invalid configs.
Co-authored-by: Codex <noreply@openai.com>
rustls 0.23 compiles both ring and aws-lc-rs in this workspace (aws-sdk-s3
pulls aws-lc-rs), so the process-level CryptoProvider cannot be auto-selected
and any TLS use panics inside spawned tasks - which made the websocket idle
timeout test hang forever. Install the pinned ring provider once before the
first TLS use, with a startup call in the gateway binary as a fallback.
Co-authored-by: Codex <noreply@openai.com>
Official Codex Desktop app-server and Grok CLI build reqwest without
compression features, so upstream requests carry no Accept-Encoding.
Drop the gzip/brotli/zstd/deflate features and the explicit
'accept-encoding: identity' Grok header so the wire matches official
clients. Verified: reqwest only inserts Accept-Encoding through the
tower-http Decompression layer, which is compiled solely when a
compression feature is enabled; the official binary's 'gzip, deflate'
string is the h2 HPACK static table entry, not a request header.
Also stop downstream clients from injecting device attestation/integrity
headers (x-oai-attestation, x-oai-is, x-oai-is-update) at both the API
passthrough boundary and the provider header builder; only the official
client or the gateway itself may supply those.
Co-authored-by: Codex <noreply@openai.com>
Pin rustls =0.23.36 and hyper-rustls =0.27.7 so the upstream ClientHello
matches the official app-server binary (reqwest 0.12.28 + ring unchanged).
Verified against ChatGPT.app Resources/codex from the current 26.727.51351
build: rustls 0.23.36, hyper-rustls 0.27.7, ring 0.17.14, reqwest 0.12.28.
rustls 0.23.37+ reorders ClientHello extensions, which changes JA3 even
though cipher suites, groups, signature algorithms and ALPN stay identical.
Keep the ring provider: reqwest rustls-tls-native-roots is __rustls-ring
and matches the official binary; do not switch the TLS path to aws-lc-rs
without re-capturing the handshake.
Co-authored-by: Codex <noreply@openai.com>
The sidebar and login logo boxes kept a #1c2a3d background around a
smaller favicon image, leaving an inconsistent ring in dark mode. The
image now fills the box and the divergent box background is removed, so
the favicon's own black tile is the visual mark.
Co-authored-by: Codex <noreply@openai.com>
OpenAI: Codex Core 0.146.0, Desktop 26.727.51351 build 6119, and os_type
macOS (official Rust app-server os_info output) so the startup User-Agent
matches the official client before the periodic release check catches up.
xAI: grok-shell client_version 0.2.118, the current @xai-official/grok
npm release.
Co-authored-by: Codex <noreply@openai.com>
Adds a full-field RuntimeSettings wire fixture with request/response field
lockstep, and a cargo-metadata based guard that compares every workspace
internal runtime edge against a frozen allowed-edges table.
Co-authored-by: Codex <noreply@openai.com>
UsageRecord metadata now carries only provider observations; protocol,
latency details, http version, status codes, websocket pool and image
generation facts moved to the top-level record contract. Contract tests
were updated to pin the new wire shape.
Co-authored-by: Codex <noreply@openai.com>
system-update now uses a discriminated phase union so mutually exclusive
operation states cannot be expressed together; connection-test event shapes
and cost chart points are typed instead of any.
Co-authored-by: Codex <noreply@openai.com>
Components now consume `UsageViewModel` produced by `normalizeUsageRecord`;
metadata fallback exists only in the normalizer. Also complete the typed
request params for ops errors so the usage module stays build-clean.
Co-authored-by: Codex <noreply@openai.com>
Block client originator/User-Agent at both the API passthrough boundary
and the provider header builder, so every upstream request always carries
the runtime wire profile instead of leaking downstream client fingerprints.
Generate User-Agent in the official Codex Desktop Rust-core format
(originator/version (os; arch) terminal) and drop the fabricated
'(Codex Desktop; <desktop version>)' suffix. Verified against the current
official client 26.727.51351, which spawns the app-server with
CODEX_INTERNAL_ORIGINATOR_OVERRIDE=Codex Desktop and never appends a
desktop version to the model-request User-Agent.
WebSocket pool fingerprint tests now rotate on the Codex Core release
instead of the desktop release, matching the fields that actually change
the User-Agent.
Co-authored-by: Codex <noreply@openai.com>
Split the four oversized files into mod.rs-style module directories without
changing public ports or behavior:
- gateway-store: observability/{model,mapping,admin_adapter,queries/*},
provider_accounts/{rows,repository,mapping,admin_adapter,core_adapter},
root {config,bundle,workers,admin_adapter,value}
- gateway-api: admin/observability/{query,wire,routes,presenter} with
mirrored contract tests and updated frozen architecture manifest
Co-authored-by: Codex <noreply@openai.com>
Replace any[] updateLogs with SystemUpdateEvent (id, level, message, at,
terminal) so the log pipeline is type-checked.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Core Decimal gains canonical() and checked_div_u64(); gateway-admin
DecimalAmount now delegates arithmetic to core instead of reimplementing
scaled parsing, removing the duplicate scaled_amount logic.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace read-full-control-plane-and-copy-all-fields with a targeted
mutation that updates only admin_api_key plus revision and audit. No
longer implicitly coupled to unrelated runtime settings fields.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ZyyandCodex
ZyyandZyy
dependabot[bot]