Compare commits

...
8 Commits
Author SHA1 Message Date
ZyyandCodex 595cda541d style: apply cargo fmt
Co-authored-by: Codex <noreply@openai.com>
2026-07-06 17:55:54 +08:00
ZyyandCodex 90ed8bf6bc chore: bump version to 1.0.4
Co-authored-by: Codex <noreply@openai.com>
2026-07-06 17:49:48 +08:00
ZyyandCodex 1101eee628 fix: align quota window accounting
Co-authored-by: Codex <noreply@openai.com>
2026-07-06 17:49:21 +08:00
ZyyandCodex 0e952c9d83 fix: enforce websocket first token retries
Co-authored-by: Codex <noreply@openai.com>
2026-07-06 17:49:15 +08:00
ZyyandCodex 875d610812 fix: refine dashboard metrics
Co-authored-by: Codex <noreply@openai.com>
2026-07-06 16:53:31 +08:00
ZyyandCodex 9c37d262d1 fix: retry websocket first token stalls
Co-authored-by: Codex <noreply@openai.com>
2026-07-06 16:53:07 +08:00
ZyyandCodex bd95adadf9 fix: align dashboard daily metrics
Co-authored-by: Codex <noreply@openai.com>
2026-07-06 13:19:02 +08:00
zyycnandCodex 6e6627b0c9 docs: refresh project documentation
Co-authored-by: Codex <noreply@openai.com>
2026-07-05 22:39:26 +08:00
44 changed files with 2113 additions and 719 deletions
+18 -16
View File
@@ -15,11 +15,11 @@
## 概览
Codex Proxy RS 是一个单进程服务:
Codex Proxy RS 是一个单进程网关:
- Rust/Axum 后端提供 OpenAI 兼容 `/v1/*` 代理、管理端 API、SQLite 持久化和静态前端托管。
- Vue 管理端用于账号导入、API Key、用量统计、请求记录、运行参数、模型映射和系统更新。
- 运行数据默认收敛到仓库 `.runtime/`,Docker 也使用同一套宿主机目录做 bind mount。
- Rust/Axum 后端提供 Responses 兼容代理、管理端 API、SQLite 持久化和静态前端托管。
- Vue 管理端负责账号导入、API Key、用量统计、请求记录、运行参数、模型映射和系统更新。
- 运行数据默认写入仓库 `.runtime/`;Docker 部署也使用这套宿主机目录挂载。
## 环境
@@ -55,7 +55,7 @@ mkdir -p .runtime/data .runtime/logs
CPR_CONFIG_FILE=.runtime/config.yaml cargo run --manifest-path backend/Cargo.toml
```
服务默认监听 `http://0.0.0.0:8080`。
服务默认监听 `0.0.0.0:8080`。本机访问通常使用 `http://127.0.0.1:8080`。
## Docker 部署
@@ -106,8 +106,11 @@ docker compose -f deploy/docker-compose.yml up -d
| 端点 | 说明 |
| --- | --- |
| `POST /v1/responses` | OpenAI Responses 兼容接口 |
| `POST /v1/responses/review` | Review 模型请求入口 |
| `POST /v1/responses/compact` | Compact 请求入口 |
| `GET /v1/models` | 模型列表 |
| `GET /v1/models/{id}` | 模型详情 |
| `GET /v1/models/{id}/info` | 模型运行信息 |
| `GET /v1/models/catalog` | 管理端可见模型目录 |
| `/api/admin/*` | 管理端 API |
@@ -136,11 +139,11 @@ pnpm --dir frontend build
CPR_CONFIG_FILE=.runtime/config.yaml cargo run --manifest-path backend/Cargo.toml
```
管理端主要功能:
管理端功能:
- 账号导入、OAuth 授权、连接测试、quota 刷新、token 刷新。
- 客户端 API Key 管理。
- 请求记录、Token 用量、模型用量和账号额度。
- Dashboard、请求记录、Token 用量、模型用量和账号额度。
- 运行参数、模型别名、账号选择策略。
- 版本检查、在线更新、更新日志、重启和回滚。
@@ -151,7 +154,7 @@ CPR_CONFIG_FILE=.runtime/config.yaml cargo run --manifest-path backend/Cargo.tom
`.runtime` 约定:
```text
.runtime/config.yaml # 本地或 Docker 启动配置,按实际运行环境写路径
.runtime/config.yaml # 本地或 Docker 启动配置,路径按运行环境写
.runtime/data/codex-proxy-rs.sqlite # SQLite 数据库
.runtime/data/installation_id # 上游 installation id
.runtime/data/update-state.json # 在线更新状态
@@ -161,22 +164,21 @@ CPR_CONFIG_FILE=.runtime/config.yaml cargo run --manifest-path backend/Cargo.tom
## 发布与更新
版本信息在 `release/version.yaml`,发布平台在 `release/platforms.yaml`。
当前版本记录在 `release/version.yaml`,发布平台记录在 `release/platforms.yaml`。
发布命令:
```bash
release/publish 0.1.5
release/publish 1.0.4
```
该命令会更新版本文件、生成版本提交、创建 `v<version>` tag 并推送,随后 GitHub Actions 构建 Release 归档和 GHCR 多平台镜像。
该命令会更新版本文件、生成版本提交、创建 `v<version>` tag 并推送。GitHub Actions 随后构建 Release 归档和 GHCR 多平台镜像。
在线更新由主服务处理,管理端调用 `/api/admin/system/*`:
- 检查 GitHub Release。
- 下载匹配当前平台的归档和 `checksums.txt`。
- 校验 checksum。
- 替换二进制和 `web/dist`。
- 查询 GitHub Release。
- 下载当前平台对应的归档和 `checksums.txt`。
- 校验 checksum 后替换二进制和 `web/dist`。
- 重启时,Docker 模式依赖 `restart: unless-stopped` 拉起新容器进程;非 Docker 模式会先安排新进程延迟启动,再关闭当前进程。
## 项目结构
@@ -185,7 +187,7 @@ release/publish 0.1.5
backend/ Rust/Axum 后端、SQLite migration、集成测试
frontend/ Vue 3 管理端
deploy/ Dockerfile、Compose、部署配置模板
docs/ 架构、审计和维护文档
docs/ 架构和维护文档
release/ 版本、平台和发布脚本
skills/ 项目本地 Codex skill
```
+13 -1
View File
@@ -43,6 +43,16 @@ pub(crate) struct AdminAccountQuotaUsageWindow {
pub(crate) key: String,
pub(crate) start: DateTime<Utc>,
pub(crate) end: DateTime<Utc>,
pub(crate) window_seconds: u64,
}
impl AdminAccountQuotaUsageWindow {
pub(crate) fn duration_seconds(&self) -> u64 {
self.end
.signed_duration_since(self.start)
.num_seconds()
.max(0) as u64
}
}
#[derive(Debug, Clone, Copy, Default)]
@@ -101,12 +111,14 @@ impl AdminAccountQuotaData {
impl AdminAccountQuotaWindowData {
fn usage_window(&self) -> Option<AdminAccountQuotaUsageWindow> {
let end = self.reset_at?;
let seconds = i64::try_from(self.window_seconds?).ok()?;
let window_seconds = self.window_seconds?;
let seconds = i64::try_from(window_seconds).ok()?;
let start = end.checked_sub_signed(Duration::seconds(seconds))?;
(start <= end).then(|| AdminAccountQuotaUsageWindow {
key: self.key.clone(),
start,
end,
window_seconds,
})
}
}
+120
View File
@@ -303,6 +303,14 @@ struct AccountModelUsageRecord {
last_used_at: Option<DateTime<Utc>>,
}
#[derive(Debug, Clone, Copy, Default)]
struct AccountUsageWindowStats {
request_count: i64,
input_tokens: i64,
output_tokens: i64,
cached_tokens: i64,
}
#[derive(Debug, Clone)]
struct AccountQuotaUsageWindow {
account_id: String,
@@ -873,6 +881,19 @@ async fn account_list_stats(
.unwrap_or_default();
let mut quota_by_account = quota_by_account.clone();
apply_quota_window_token_usage(state, accounts, &mut quota_by_account).await;
let selected_quota_windows = selected_quota_windows_by_account(accounts, &quota_by_account);
let quota_window_usage_stats =
account_usage_stats_by_quota_window(state, &selected_quota_windows).await;
let usage_records = usage_records
.into_iter()
.map(|usage| {
apply_selected_quota_window_usage(
usage,
&selected_quota_windows,
&quota_window_usage_stats,
)
})
.collect::<Vec<_>>();
let models_by_account = list_current_window_model_usage(state, &usage_records).await;
AccountListStats {
@@ -885,6 +906,105 @@ async fn account_list_stats(
}
}
fn selected_quota_windows_by_account(
accounts: &[AdminAccountMetadata],
quota_by_account: &HashMap<String, AdminAccountQuotaData>,
) -> HashMap<String, AdminAccountQuotaUsageWindow> {
accounts
.iter()
.filter_map(|account| {
quota_by_account
.get(&account.id)
.and_then(selected_quota_window)
.map(|window| (account.id.clone(), window))
})
.collect()
}
fn selected_quota_window(quota: &AdminAccountQuotaData) -> Option<AdminAccountQuotaUsageWindow> {
quota.usage_windows().into_iter().max_by(|left, right| {
left.duration_seconds()
.cmp(&right.duration_seconds())
.then_with(|| left.end.cmp(&right.end))
.then_with(|| left.key.cmp(&right.key))
})
}
async fn account_usage_stats_by_quota_window(
state: &AppState,
windows_by_account: &HashMap<String, AdminAccountQuotaUsageWindow>,
) -> HashMap<String, AccountUsageWindowStats> {
if windows_by_account.is_empty() {
return HashMap::new();
}
let mut builder = QueryBuilder::<Sqlite>::new(
"select
account_id,
coalesce(sum(request_count), 0) as request_count,
coalesce(sum(input_tokens), 0) as input_tokens,
coalesce(sum(output_tokens), 0) as output_tokens,
coalesce(sum(cached_tokens), 0) as cached_tokens
from usage_time_buckets
where ",
);
for (index, (account_id, window)) in windows_by_account.iter().enumerate() {
if index > 0 {
builder.push(" or ");
}
let bucket_start = china_quarter_hour_start(window.start);
builder.push("(account_id = ");
builder.push_bind(account_id);
builder.push(" and bucket_start >= ");
builder.push_bind(bucket_start.to_rfc3339());
builder.push(" and bucket_start <= ");
builder.push_bind(window.end.to_rfc3339());
builder.push(")");
}
builder.push(" group by account_id");
let rows = builder
.build()
.fetch_all(state.services.background_tasks.accounts.pool())
.await
.unwrap_or_default();
rows.into_iter()
.map(|row| {
(
row.get::<String, _>("account_id"),
AccountUsageWindowStats {
request_count: row.get("request_count"),
input_tokens: row.get("input_tokens"),
output_tokens: row.get("output_tokens"),
cached_tokens: row.get("cached_tokens"),
},
)
})
.collect()
}
fn apply_selected_quota_window_usage(
mut usage: AdminUsageRecord,
selected_quota_windows: &HashMap<String, AdminAccountQuotaUsageWindow>,
quota_window_usage_stats: &HashMap<String, AccountUsageWindowStats>,
) -> AdminUsageRecord {
let Some(window) = selected_quota_windows.get(&usage.account_id) else {
return usage;
};
let stats = quota_window_usage_stats
.get(&usage.account_id)
.copied()
.unwrap_or_default();
usage.window_request_count = stats.request_count;
usage.window_input_tokens = stats.input_tokens;
usage.window_output_tokens = stats.output_tokens;
usage.window_cached_tokens = stats.cached_tokens;
usage.window_started_at = Some(window.start);
usage.window_reset_at = Some(window.end);
usage.limit_window_seconds = Some(window.window_seconds);
usage
}
async fn apply_account_quota_window_token_usage(
state: &AppState,
account_id: &str,
+254 -170
View File
@@ -1,6 +1,6 @@
//! 管理端 Dashboard 聚合视图。
use std::{cmp::Reverse, collections::HashMap};
use std::collections::HashMap;
use axum::{
extract::{Query, State},
@@ -15,14 +15,19 @@ use crate::{
admin::{
auth::session::AdminAuth,
monitoring::{
account_usage_service::{
AdminUsageRecord, AdminUsageSummary, AdminUsageTimeBucketRecord,
},
account_usage_service::{AdminUsageSummary, AdminUsageTimeBucketRecord},
diagnostics::{
fingerprint_diagnostics, AccountCapacityDiagnostics, AccountPoolDiagnostics,
},
usage_record_routes::{usage_record_items, UsageRecordData},
usage_record_model::{
metadata_nonnegative_i64, metadata_string, UsageRecord, UsageRecordLevel,
},
usage_record_routes::{
usage_cost_details, usage_record_models, usage_token_details,
UsageRecordCostDetailsData, UsageRecordTokenDetailsData,
},
usage_record_service::AdminUsageRecordFilter,
usage_record_store::UsageRecordAccountUsage,
},
response::{AdminEnvelope, AdminError, AdminResponse},
},
@@ -40,11 +45,15 @@ use crate::{
upstream::accounts::model::{Account, AccountStatus},
};
const DASHBOARD_USAGE_RECORD_LIMIT: u32 = 1000;
const DASHBOARD_ACCOUNT_USAGE_LIMIT: u32 = 4;
const DASHBOARD_USAGE_RECORD_LIMIT: u32 = 10;
const HEALTH_TIMELINE_SLOT_MINUTES: i64 = 15;
const DASHBOARD_TIME_BUCKET_SLOTS: i64 = 7 * 24 * 4;
const HEALTH_TIMELINE_SLOTS: i64 = 24 * 4;
const HEALTH_TIMELINE_STABLE_SUCCESS_THRESHOLD: u64 = 3;
const FIVE_HOUR_WINDOW_SECONDS: u64 = 18_000;
const WEEK_WINDOW_SECONDS: u64 = 604_800;
const MONTH_WINDOW_SECONDS: u64 = 2_592_000;
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
@@ -69,7 +78,7 @@ struct DashboardSummaryData {
health_timeline: DashboardHealthTimelineData,
account_usage: Vec<DashboardAccountUsageData>,
service_statuses: Vec<DashboardServiceStatusData>,
usage_records: Vec<UsageRecordData>,
usage_records: Vec<DashboardUsageRecordData>,
pool_summary: AccountPoolDiagnostics,
capacity_info: AccountCapacityDiagnostics,
rotation_strategy: Option<String>,
@@ -80,10 +89,7 @@ struct DashboardSummaryData {
struct DashboardHealthTimelineData {
title: String,
description: String,
range_display: String,
reliability_display: String,
oldest_label: String,
newest_label: String,
points: String,
}
@@ -112,11 +118,8 @@ struct DashboardAccountsCardData {
struct DashboardTrafficCardData {
today_requests: String,
today_requests_value: u64,
yesterday_requests: String,
yesterday_requests_value: u64,
total_requests: String,
rpm: String,
tpm: String,
}
#[derive(Debug, Serialize)]
@@ -124,13 +127,9 @@ struct DashboardTrafficCardData {
struct DashboardTokenCardData {
today_tokens: String,
today_tokens_value: u64,
yesterday_tokens: String,
yesterday_tokens_value: u64,
total_tokens: String,
today_cost_usd: String,
today_cost_usd_value: f64,
total_cost_usd: String,
total_cost_usd_value: f64,
}
#[derive(Debug, Serialize)]
@@ -138,12 +137,10 @@ struct DashboardTokenCardData {
struct DashboardCacheCardData {
today_hit_rate: String,
today_hit_rate_value: Option<f64>,
yesterday_hit_rate: String,
yesterday_hit_rate_value: Option<f64>,
total_hit_rate: String,
total_cached_tokens: String,
first_token_latency_ms: String,
completion_latency_ms: String,
average_first_token_latency_ms: String,
}
#[derive(Debug, Serialize)]
@@ -166,7 +163,7 @@ struct DashboardTrendPointData {
output_tokens_value: u64,
cached_tokens: String,
cached_tokens_value: u64,
tokens: String,
cache_hit_rate_value: f64,
tokens_value: u64,
errors: String,
errors_value: u64,
@@ -191,14 +188,34 @@ struct DashboardTrendSummaryData {
#[derive(Debug, Serialize)]
#[serde(rename_all = "camelCase")]
struct DashboardAccountUsageData {
name: String,
id: String,
email: String,
plan: String,
requests: String,
plan_type: Option<String>,
tokens: String,
quota_used_percent: Option<f64>,
last_used: String,
status: String,
}
#[derive(Debug, Serialize)]
#[serde(rename_all = "camelCase")]
struct DashboardUsageRecordData {
id: String,
level: UsageRecordLevel,
route: Option<String>,
model: Option<String>,
status_code: Option<i64>,
transport: Option<String>,
created_at_display: String,
account_email: Option<String>,
requested_model: Option<String>,
upstream_model: Option<String>,
client_ip: Option<String>,
user_agent: Option<String>,
reasoning_effort: Option<String>,
token_details: UsageRecordTokenDetailsData,
cost_details: Option<UsageRecordCostDetailsData>,
first_token_latency_ms_display: String,
latency_ms_display: String,
}
#[derive(Debug, Serialize)]
@@ -221,8 +238,14 @@ struct UsageWindow {
first_token_latency_count: u64,
max_first_token_bucket_latency: u64,
min_first_token_bucket_latency: u64,
latency_sum: u64,
latency_count: u64,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
enum QuotaWindowPriority {
FiveHour,
Weekly,
Monthly,
Other,
}
impl UsageWindow {
@@ -250,10 +273,6 @@ impl UsageWindow {
fn min_first_token_bucket_latency(self) -> Option<u64> {
(self.min_first_token_bucket_latency > 0).then_some(self.min_first_token_bucket_latency)
}
fn avg_latency(self) -> Option<u64> {
self.latency_sum.checked_div(self.latency_count)
}
}
/// `GET /api/admin/dashboard/summary`
@@ -269,53 +288,56 @@ pub(crate) async fn dashboard_summary(
.await
.unwrap_or_default();
let capacity = state.services.account_pool.capacity_summary_now().await;
let summary = state
let now = Utc::now();
let today_filter = today_usage_record_filter(now);
let usage_summary = state
.services
.usage
.summary()
.await
.unwrap_or_else(|_| empty_usage_summary());
let usage_records = state
let account_usage_records = state
.services
.usage
.list(None, 200)
.usage_records
.account_usage(today_filter.clone(), DASHBOARD_ACCOUNT_USAGE_LIMIT)
.await
.map(|page| page.items)
.unwrap_or_default();
let recent_events = state
.services
.usage_records
.list(
None,
DASHBOARD_USAGE_RECORD_LIMIT,
AdminUsageRecordFilter::default(),
)
.list(None, DASHBOARD_USAGE_RECORD_LIMIT, today_filter)
.await
.map(|page| page.items)
.unwrap_or_default();
let now = Utc::now();
let time_buckets = dashboard_time_buckets(&state, now).await;
let pool_summary = account_pool_summary(&accounts);
let trend = dashboard_trend_data(&time_buckets, query.kind.unwrap_or_default());
let quota_used_by_account = account_quota_used_percent_by_id(&state, &usage_records).await;
let quota_used_by_account =
account_quota_used_percent_by_id(&state, &account_usage_records).await;
let settings = state.services.settings.current();
let recent_usage_records = recent_events.into_iter().take(10).collect::<Vec<_>>();
let recent_usage_records = recent_events;
let account_emails = state
.services
.usage_records
.account_email_map(&recent_usage_records)
.await
.map_err(|error| AdminError::usage_record_accounts_failed(error.to_string()))?;
let dashboard_usage_records = usage_record_items(recent_usage_records, &account_emails);
let dashboard_usage_records =
dashboard_usage_record_items(recent_usage_records, &account_emails);
Ok(AdminResponse::new(
StatusCode::OK,
AdminEnvelope::ok(DashboardSummaryData {
cards: dashboard_cards(&accounts, &summary, &time_buckets),
cards: dashboard_cards(&accounts, &time_buckets, &usage_summary),
trend,
health_timeline: dashboard_health_timeline_data(&time_buckets),
account_usage: account_usage_data(&accounts, &usage_records, &quota_used_by_account),
account_usage: account_usage_data(
&accounts,
&account_usage_records,
&quota_used_by_account,
now,
),
service_statuses: service_status_data(&state),
usage_records: dashboard_usage_records,
pool_summary,
@@ -325,6 +347,14 @@ pub(crate) async fn dashboard_summary(
))
}
fn today_usage_record_filter(now: DateTime<Utc>) -> AdminUsageRecordFilter {
AdminUsageRecordFilter {
start_time: Some(china_day_start(now)),
end_time: Some(now),
..AdminUsageRecordFilter::default()
}
}
/// `GET /api/admin/dashboard/trend?kind=usage|latency|errors`
pub(crate) async fn dashboard_trend(
State(state): State<AppState>,
@@ -344,16 +374,16 @@ pub(crate) async fn dashboard_trend(
fn dashboard_cards(
accounts: &[Account],
summary: &AdminUsageSummary,
buckets: &[AdminUsageTimeBucketRecord],
usage_summary: &AdminUsageSummary,
) -> DashboardCardsData {
dashboard_cards_at(accounts, summary, buckets, Utc::now())
dashboard_cards_at(accounts, buckets, usage_summary, Utc::now())
}
fn dashboard_cards_at(
accounts: &[Account],
summary: &AdminUsageSummary,
buckets: &[AdminUsageTimeBucketRecord],
usage_summary: &AdminUsageSummary,
now: DateTime<Utc>,
) -> DashboardCardsData {
let today_start = china_day_start(now);
@@ -361,10 +391,11 @@ fn dashboard_cards_at(
let today = usage_window(buckets, today_start, now);
let yesterday = usage_window(buckets, yesterday_start, today_start);
let today_cost = cost_window(buckets, today_start, now).unwrap_or(0.0);
let total_cost = total_cost(buckets);
let total_requests = nonnegative_i64_to_u64(usage_summary.request_count);
let total_tokens = nonnegative_i64_to_u64(usage_summary.total_tokens);
let total_cached_tokens = nonnegative_i64_to_u64(usage_summary.cached_tokens);
let total_hit_rate = summary_cache_hit_rate(usage_summary);
let total_input = nonnegative_i64_to_u64(summary.input_tokens);
let total_cached = nonnegative_i64_to_u64(summary.cached_tokens);
let total_accounts = accounts.len() as u64;
let enabled_accounts = accounts
.iter()
@@ -382,13 +413,6 @@ fn dashboard_cards_at(
)
})
.count() as u64;
let total_requests = nonnegative_i64_to_u64(summary.request_count);
let total_tokens = nonnegative_i64_to_u64(summary.input_tokens + summary.output_tokens);
let total_hit_rate = if total_input > 0 {
Some(total_cached as f64 / total_input as f64)
} else {
None
};
DashboardCardsData {
accounts: DashboardAccountsCardData {
@@ -402,38 +426,53 @@ fn dashboard_cards_at(
traffic: DashboardTrafficCardData {
today_requests: format_compact_number(today.requests),
today_requests_value: today.requests,
yesterday_requests: format_compact_number(yesterday.requests),
yesterday_requests_value: yesterday.requests,
total_requests: format_compact_number(total_requests),
rpm: format_compact_number(today.requests),
tpm: format_tokens(today.tokens()),
},
tokens: DashboardTokenCardData {
today_tokens: format_tokens(today.tokens()),
today_tokens_value: today.tokens(),
yesterday_tokens: format_tokens(yesterday.tokens()),
yesterday_tokens_value: yesterday.tokens(),
total_tokens: format_tokens(total_tokens),
today_cost_usd: format_optional_cost(Some(today_cost)),
today_cost_usd_value: today_cost,
total_cost_usd: format_optional_cost(Some(total_cost)),
total_cost_usd_value: total_cost,
today_cost_usd: format_cost(today_cost),
},
cache: DashboardCacheCardData {
today_hit_rate: format_rate(today.cache_hit_rate()),
today_hit_rate_value: today.cache_hit_rate(),
yesterday_hit_rate: format_rate(yesterday.cache_hit_rate()),
yesterday_hit_rate_value: yesterday.cache_hit_rate(),
total_hit_rate: format_rate(total_hit_rate),
total_cached_tokens: format_tokens(total_cached),
first_token_latency_ms: format_optional_duration_ms(today.avg_first_token_latency()),
completion_latency_ms: format_optional_duration_ms(today.avg_latency()),
total_cached_tokens: format_tokens(total_cached_tokens),
average_first_token_latency_ms: format_optional_duration_ms(
today.avg_first_token_latency(),
),
},
}
}
fn format_optional_cost(value: Option<f64>) -> String {
value.map_or_else(|| "-".to_string(), format_cost)
fn empty_usage_summary() -> AdminUsageSummary {
AdminUsageSummary {
account_count: 0,
request_count: 0,
empty_response_count: 0,
input_tokens: 0,
output_tokens: 0,
cached_tokens: 0,
reasoning_tokens: 0,
total_tokens: 0,
image_input_tokens: 0,
image_output_tokens: 0,
image_request_count: 0,
image_request_failed_count: 0,
}
}
fn summary_cache_hit_rate(summary: &AdminUsageSummary) -> Option<f64> {
let input_tokens = nonnegative_i64_to_u64(summary.input_tokens);
if input_tokens > 0 {
Some(nonnegative_i64_to_u64(summary.cached_tokens) as f64 / input_tokens as f64)
} else {
None
}
}
fn format_optional_duration_ms(value: Option<u64>) -> String {
@@ -468,8 +507,9 @@ fn dashboard_trend_data_at(
now: DateTime<Utc>,
) -> DashboardTrendData {
let current_hour = china_hour_start(now);
let start = current_hour - Duration::hours(23);
let mut buckets = (0..24)
let start = china_day_start(now);
let elapsed_hours = current_hour.signed_duration_since(start).num_hours();
let mut buckets = (0..=elapsed_hours)
.map(|index| {
let bucket_start = start + Duration::hours(index);
(bucket_start, UsageWindow::default())
@@ -496,6 +536,7 @@ fn dashboard_trend_data_at(
let max_latency = bucket.max_first_token_bucket_latency().unwrap_or(0);
let min_latency = bucket.min_first_token_bucket_latency().unwrap_or(0);
let tokens = bucket.tokens();
let cache_hit_rate = bucket.cache_hit_rate().unwrap_or(0.0);
let success_rate = if bucket.requests > 0 {
((bucket.requests - bucket.errors) as f64 / bucket.requests as f64 * 1000.0).round()
/ 10.0
@@ -512,7 +553,7 @@ fn dashboard_trend_data_at(
output_tokens_value: bucket.output_tokens,
cached_tokens: format_tokens(bucket.cached_tokens),
cached_tokens_value: bucket.cached_tokens,
tokens: format_tokens(tokens),
cache_hit_rate_value: cache_hit_rate,
tokens_value: tokens,
errors: format_compact_number(bucket.errors),
errors_value: bucket.errors,
@@ -585,15 +626,12 @@ fn dashboard_health_timeline_data_at(
};
DashboardHealthTimelineData {
title: "请求健康时间线".to_string(),
description: "今日请求可靠性".to_string(),
range_display: format!("{} - {}", china_datetime(&start), china_datetime(&now)),
description: "请求可靠性".to_string(),
reliability_display: if requests > 0 {
format!("{reliability:.1}%")
} else {
"-".to_string()
},
oldest_label: "00:00".to_string(),
newest_label: "24:00".to_string(),
points: buckets
.into_iter()
.map(|(bucket_start, bucket)| {
@@ -759,22 +797,6 @@ fn cost_window(
has_usage.then_some(total)
}
fn total_cost(records: &[AdminUsageTimeBucketRecord]) -> f64 {
let mut total = 0.0;
let mut has_usage = false;
for record in records {
if let Some(cost) = record.cost_usd {
total += cost;
has_usage = true;
}
}
if has_usage {
return total;
}
0.0
}
fn apply_bucket(window: &mut UsageWindow, record: &AdminUsageTimeBucketRecord) {
window.requests += nonnegative_i64_to_u64(record.request_count);
window.input_tokens += nonnegative_i64_to_u64(record.input_tokens);
@@ -792,8 +814,6 @@ fn apply_bucket(window: &mut UsageWindow, record: &AdminUsageTimeBucketRecord) {
window.min_first_token_bucket_latency.min(latency)
};
}
window.latency_sum += nonnegative_i64_to_u64(record.latency_sum);
window.latency_count += nonnegative_i64_to_u64(record.latency_count);
}
fn first_token_bucket_latency(record: &AdminUsageTimeBucketRecord) -> Option<u64> {
@@ -822,68 +842,51 @@ fn account_pool_summary(accounts: &[Account]) -> AccountPoolDiagnostics {
fn account_usage_data(
accounts: &[Account],
usage_records: &[AdminUsageRecord],
usage_records: &[UsageRecordAccountUsage],
quota_used_by_account: &HashMap<String, f64>,
now: DateTime<Utc>,
) -> Vec<DashboardAccountUsageData> {
let account_by_id = accounts
.iter()
.map(|account| (account.id.as_str(), account))
.collect::<HashMap<_, _>>();
let mut records = usage_records.iter().collect::<Vec<_>>();
records.sort_by_key(|record| Reverse((record.last_used_at, record.account_id.clone())));
let mut rows = records
usage_records
.iter()
.map(|usage| {
let account = account_by_id.get(usage.account_id.as_str()).copied();
let name = account
.and_then(|account| account.label.clone())
.or_else(|| usage.label.clone())
.or_else(|| {
usage
.email
.as_ref()
.and_then(|email| email.split('@').next().map(ToString::to_string))
})
.unwrap_or_else(|| usage.account_id.chars().take(8).collect());
let status = account.map_or_else(
|| "active".to_string(),
|account| account.status.to_string(),
);
let quota_used_percent = quota_used_by_account
.get(&usage.account_id)
.copied()
.or_else(|| (status == "quota_exhausted").then_some(100.0));
let requests = nonnegative_i64_to_u64(usage.request_count);
let tokens = nonnegative_i64_to_u64(usage.input_tokens + usage.output_tokens);
.or_else(|| {
matches!(
account.map(|account| account.status),
Some(AccountStatus::QuotaExhausted)
)
.then_some(100.0)
});
DashboardAccountUsageData {
name,
email: usage.email.clone().unwrap_or_else(|| "-".to_string()),
plan: usage
.plan_type
.clone()
.unwrap_or_else(|| "free".to_string()),
requests: format_compact_number(requests),
tokens: format_tokens(tokens),
id: usage.account_id.clone(),
email: account
.and_then(|account| account.email.as_ref())
.cloned()
.expect("dashboard account usage requires email"),
plan_type: account
.and_then(|account| account.plan_type.as_ref())
.cloned(),
tokens: format_tokens(usage.total_tokens),
quota_used_percent,
last_used: china_relative_time(usage.last_used_at, Utc::now()),
status,
last_used: china_relative_time(Some(usage.last_used_at), now),
}
})
.collect::<Vec<_>>();
rows.truncate(4);
rows
.collect()
}
async fn account_quota_used_percent_by_id(
state: &AppState,
usage_records: &[AdminUsageRecord],
usage_records: &[UsageRecordAccountUsage],
) -> HashMap<String, f64> {
let mut records = usage_records.iter().collect::<Vec<_>>();
records.sort_by_key(|record| Reverse((record.last_used_at, record.account_id.clone())));
records.truncate(4);
let mut quota_used_by_account = HashMap::with_capacity(records.len());
for usage in records {
let mut quota_used_by_account = HashMap::with_capacity(usage_records.len());
for usage in usage_records {
let Ok(Some(quota_json)) = state
.services
.accounts
@@ -901,27 +904,67 @@ async fn account_quota_used_percent_by_id(
fn quota_used_percent(quota_json: &str) -> Option<f64> {
let quota = serde_json::from_str::<Value>(quota_json).ok()?;
let mut values = Vec::new();
if let Some(used_percent) = quota
.pointer("/monthly_limit/used_percent")
.and_then(percent_value)
{
values.push(used_percent);
let mut selected = None;
if let Some(monthly_limit) = quota.get("monthly_limit") {
select_quota_used_percent(
&mut selected,
quota_window_priority(monthly_limit, QuotaWindowPriority::Monthly),
monthly_limit.get("used_percent").and_then(percent_value),
);
}
if let Some(snapshots) = quota.get("snapshots").and_then(Value::as_array) {
for snapshot in snapshots {
for role in ["primary", "secondary"] {
if let Some(used_percent) = snapshot
.get(role)
.and_then(|window| window.get("used_percent"))
.and_then(percent_value)
{
values.push(used_percent);
if let Some(window) = snapshot.get(role) {
select_quota_used_percent(
&mut selected,
quota_window_priority(window, QuotaWindowPriority::Other),
window.get("used_percent").and_then(percent_value),
);
}
}
}
}
values.into_iter().max_by(f64::total_cmp)
selected.map(|(_, used_percent)| used_percent)
}
fn select_quota_used_percent(
selected: &mut Option<(QuotaWindowPriority, f64)>,
priority: QuotaWindowPriority,
used_percent: Option<f64>,
) {
let Some(used_percent) = used_percent else {
return;
};
match selected {
Some((selected_priority, selected_percent))
if priority > *selected_priority
|| (priority == *selected_priority && used_percent <= *selected_percent) => {}
_ => *selected = Some((priority, used_percent)),
}
}
fn quota_window_priority(window: &Value, fallback: QuotaWindowPriority) -> QuotaWindowPriority {
let window_seconds = window
.get("window_minutes")
.and_then(Value::as_u64)
.and_then(|minutes| minutes.checked_mul(60));
match window_seconds {
Some(seconds) if quota_window_matches(seconds, FIVE_HOUR_WINDOW_SECONDS) => {
QuotaWindowPriority::FiveHour
}
Some(seconds) if quota_window_matches(seconds, WEEK_WINDOW_SECONDS) => {
QuotaWindowPriority::Weekly
}
Some(seconds) if quota_window_matches(seconds, MONTH_WINDOW_SECONDS) => {
QuotaWindowPriority::Monthly
}
_ => fallback,
}
}
fn quota_window_matches(actual: u64, expected: u64) -> bool {
actual > 0 && actual.abs_diff(expected) <= expected / 20
}
fn percent_value(value: &Value) -> Option<f64> {
@@ -931,6 +974,64 @@ fn percent_value(value: &Value) -> Option<f64> {
percent.is_finite().then_some(percent.clamp(0.0, 100.0))
}
fn dashboard_usage_record_items(
items: Vec<UsageRecord>,
account_emails: &HashMap<String, String>,
) -> Vec<DashboardUsageRecordData> {
items
.into_iter()
.map(|record| dashboard_usage_record_data(record, account_emails))
.collect()
}
fn dashboard_usage_record_data(
record: UsageRecord,
account_emails: &HashMap<String, String>,
) -> DashboardUsageRecordData {
let account_email = record
.account_id
.as_deref()
.and_then(|account_id| account_emails.get(account_id))
.cloned()
.or_else(|| metadata_string(&record.metadata, &["accountEmail", "account_email"]));
let (requested_model, upstream_model) = usage_record_models(&record);
let client_ip = metadata_string(&record.metadata, &["clientIp", "ipAddress", "ip_address"]);
let user_agent = metadata_string(&record.metadata, &["userAgent", "user_agent"]);
let reasoning_effort =
metadata_string(&record.metadata, &["reasoningEffort", "reasoning_effort"]);
let token_details = usage_token_details(&record.metadata);
let cost_details = usage_cost_details(&record, upstream_model.as_deref(), &token_details);
let first_token_latency_ms = metadata_nonnegative_i64(
&record.metadata,
&[
"firstTokenMs",
"first_token_ms",
"firstTokenLatencyMs",
"first_token_latency_ms",
],
);
DashboardUsageRecordData {
id: record.id,
level: record.level,
route: record.route,
model: record.model,
status_code: record.status_code,
transport: record.transport,
created_at_display: china_datetime(&record.created_at),
account_email,
requested_model,
upstream_model,
client_ip,
user_agent,
reasoning_effort,
token_details,
cost_details,
first_token_latency_ms_display: format_duration_ms(first_token_latency_ms),
latency_ms_display: format_duration_ms(record.latency_ms),
}
}
fn service_status_data(state: &AppState) -> Vec<DashboardServiceStatusData> {
let fingerprint = fingerprint_diagnostics(&state.services.fingerprint);
vec![
@@ -993,20 +1094,3 @@ fn format_fingerprint_updated_at(value: Option<String>) -> String {
}
china_datetime_rfc3339_str(value)
}
fn empty_usage_summary() -> AdminUsageSummary {
AdminUsageSummary {
account_count: 0,
request_count: 0,
empty_response_count: 0,
input_tokens: 0,
output_tokens: 0,
cached_tokens: 0,
reasoning_tokens: 0,
total_tokens: 0,
image_input_tokens: 0,
image_output_tokens: 0,
image_request_count: 0,
image_request_failed_count: 0,
}
}
@@ -545,7 +545,7 @@ fn usage_record_data(
}
}
fn usage_record_models(record: &UsageRecord) -> (Option<String>, Option<String>) {
pub(crate) fn usage_record_models(record: &UsageRecord) -> (Option<String>, Option<String>) {
let stored_model = record
.model
.as_deref()
@@ -560,7 +560,7 @@ fn usage_record_models(record: &UsageRecord) -> (Option<String>, Option<String>)
(requested_model, upstream_model)
}
fn usage_token_details(metadata: &Value) -> UsageRecordTokenDetailsData {
pub(crate) fn usage_token_details(metadata: &Value) -> UsageRecordTokenDetailsData {
let input_tokens = metadata_usage_u64(metadata, "inputTokens");
let output_tokens = metadata_usage_u64(metadata, "outputTokens");
let cached_tokens = metadata_usage_u64(metadata, "cachedTokens");
@@ -582,7 +582,7 @@ fn usage_token_details(metadata: &Value) -> UsageRecordTokenDetailsData {
}
}
fn usage_cost_details(
pub(crate) fn usage_cost_details(
record: &UsageRecord,
upstream_model: Option<&str>,
tokens: &UsageRecordTokenDetailsData,
@@ -11,8 +11,9 @@ use crate::{
usage_record_model::metadata_service_tier,
usage_record_model::{UsageRecord, UsageRecordLevel},
usage_record_store::{
SqliteUsageRecordStore, UsageRecordBreakdown, UsageRecordEndpointSource,
UsageRecordFilter, UsageRecordModelSource, UsageRecordSummary, UsageRecordTrendPoint,
SqliteUsageRecordStore, UsageRecordAccountUsage, UsageRecordBreakdown,
UsageRecordEndpointSource, UsageRecordFilter, UsageRecordModelSource,
UsageRecordSummary, UsageRecordTrendPoint,
},
},
infra::json::{NumberedPage, Page},
@@ -212,6 +213,18 @@ impl AdminUsageRecordService {
.map_err(|_| AdminUsageRecordError::List)
}
/// 按账号聚合使用记录。
pub async fn account_usage(
&self,
filter: AdminUsageRecordFilter,
limit: u32,
) -> Result<Vec<UsageRecordAccountUsage>, AdminUsageRecordError> {
self.store
.account_usage(filter.into(), limit)
.await
.map_err(|_| AdminUsageRecordError::List)
}
/// 按模型来源聚合使用记录分布。
pub async fn model_distribution(
&self,
@@ -198,6 +198,15 @@ impl SqliteUsageRecordStore {
usage_summary(&self.pool, &filter).await
}
/// 按账号聚合使用记录。
pub async fn account_usage(
&self,
filter: UsageRecordFilter,
limit: u32,
) -> SqliteUsageRecordStoreResult<Vec<UsageRecordAccountUsage>> {
usage_account_usage(&self.pool, &filter, limit).await
}
/// 按模型来源聚合使用记录分布。
pub async fn model_distribution(
&self,
@@ -477,6 +486,45 @@ async fn usage_summary(
})
}
async fn usage_account_usage(
pool: &SqlitePool,
filter: &UsageRecordFilter,
limit: u32,
) -> SqliteUsageRecordStoreResult<Vec<UsageRecordAccountUsage>> {
let mut builder = QueryBuilder::<Sqlite>::new(
"select
account_id,
sum(coalesce(cast(json_extract(metadata_json, '$.usage.inputTokens') as integer), cast(json_extract(metadata_json, '$.inputTokens') as integer), 0)) as input_tokens,
sum(coalesce(cast(json_extract(metadata_json, '$.usage.outputTokens') as integer), cast(json_extract(metadata_json, '$.outputTokens') as integer), 0)) as output_tokens,
max(created_at) as last_used_at
from usage_records",
);
push_filter(&mut builder, filter, None)?;
builder.push(" and account_id is not null and trim(account_id) <> ''");
builder.push(" group by account_id order by last_used_at desc, account_id asc limit ");
builder.push_bind(i64::from(limit.clamp(1, 50)));
builder
.build()
.fetch_all(pool)
.await?
.into_iter()
.map(|row| {
let input_tokens =
optional_nonnegative_i64_to_u64(row.get::<Option<i64>, _>("input_tokens"));
let output_tokens =
optional_nonnegative_i64_to_u64(row.get::<Option<i64>, _>("output_tokens"));
Ok(UsageRecordAccountUsage {
account_id: row.get("account_id"),
input_tokens,
output_tokens,
total_tokens: input_tokens + output_tokens,
last_used_at: parse_rfc3339(&row.get::<String, _>("last_used_at"))?,
})
})
.collect()
}
async fn usage_record_account_email_map(
pool: &SqlitePool,
items: &[UsageRecord],
@@ -978,6 +1026,21 @@ pub struct UsageRecordSummary {
pub average_latency_ms: Option<f64>,
}
/// 账号使用记录聚合。
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct UsageRecordAccountUsage {
/// 账号 ID。
pub account_id: String,
/// 输入 Token。
pub input_tokens: u64,
/// 输出 Token。
pub output_tokens: u64,
/// 总 Token。
pub total_tokens: u64,
/// 最近使用时间。
pub last_used_at: DateTime<Utc>,
}
/// 模型分布来源。
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum UsageRecordModelSource {
+12
View File
@@ -164,6 +164,17 @@ pub struct WebSocketPoolConfig {
pub max_age_ms: u64,
/// 单账号最大连接数。
pub max_per_account: usize,
/// 首个内容帧(首 token)到达前的绝对超时;`0` 表示禁用。
///
/// 从发出 `response.create` 起算,覆盖建连/发送/上游排队/思考的全程,直到收到
/// 首个真实内容帧(`response.created`/`response.in_progress` 等纯生命周期帧不计)。
/// 超时判定为连接落到病态上游后端,丢弃并换新连接重试。
#[serde(default = "default_ws_first_token_timeout_ms")]
pub first_token_timeout_ms: u64,
}
fn default_ws_first_token_timeout_ms() -> u64 {
15_000
}
impl Default for WebSocketPoolConfig {
@@ -172,6 +183,7 @@ impl Default for WebSocketPoolConfig {
enabled: true,
max_age_ms: 3_300_000,
max_per_account: 8,
first_token_timeout_ms: default_ws_first_token_timeout_ms(),
}
}
}
+7 -1
View File
@@ -168,15 +168,21 @@ impl Services {
enabled: true,
max_age: std::time::Duration::from_millis(config.ws_pool.max_age_ms),
max_per_account: config.ws_pool.max_per_account,
first_token_timeout: (config.ws_pool.first_token_timeout_ms > 0).then(|| {
std::time::Duration::from_millis(config.ws_pool.first_token_timeout_ms)
}),
..CodexWebSocketPoolConfig::default()
}))
});
let codex = {
let websocket_first_token_timeout = (config.ws_pool.first_token_timeout_ms > 0)
.then(|| std::time::Duration::from_millis(config.ws_pool.first_token_timeout_ms));
let client = CodexBackendClient::new(
build_reqwest_client(config.tls.force_http11)?,
config.api.base_url.clone(),
fingerprint.clone(),
);
)
.with_websocket_first_token_timeout(websocket_first_token_timeout);
if let Some(pool) = &websocket_pool {
StdArc::new(client.with_websocket_pool(pool.clone()))
} else {
@@ -89,6 +89,7 @@ impl PeriodicTaskRunner for QuotaRefreshTask {
Ok(summary) => {
debug!(
candidates = summary.candidates,
skipped_cooldown = summary.skipped_cooldown,
skipped_recent = summary.skipped_recent,
failed = summary.failed,
"没有需要刷新的 quota 锁定或待验证账号"
+23 -3
View File
@@ -2,6 +2,7 @@
use std::{collections::HashMap, sync::Arc, time::Duration};
use chrono::Utc;
use thiserror::Error;
use tokio::time::{sleep, Instant};
use tracing::warn;
@@ -22,6 +23,7 @@ use crate::upstream::{
const MIN_REFRESH_INTERVAL_SECS: u64 = 30 * 60;
const DEFAULT_REQUEST_SPACING_SECS: u64 = 3;
const COOLDOWN_REFRESH_GRACE_SECS: i64 = 30;
/// 运行时 quota refresh 服务。
pub struct RuntimeQuotaRefreshService {
@@ -109,6 +111,7 @@ impl RuntimeQuotaRefreshService {
};
let min_interval = Duration::from_secs(self.min_refresh_interval_secs);
let now = Instant::now();
let now_utc = Utc::now();
let mut candidates = accounts
.into_iter()
@@ -121,9 +124,18 @@ impl RuntimeQuotaRefreshService {
while let Some(account) = candidates.next() {
summary.candidates += 1;
if last_refreshed
.get(&account.id)
.is_some_and(|last| now.duration_since(*last) < min_interval)
let cooldown_ready = match account.quota_cooldown_until {
Some(cooldown_until) if !quota_cooldown_ready(cooldown_until, now_utc) => {
summary.skipped_cooldown += 1;
continue;
}
Some(_) => true,
None => false,
};
if !cooldown_ready
&& last_refreshed
.get(&account.id)
.is_some_and(|last| now.duration_since(*last) < min_interval)
{
summary.skipped_recent += 1;
continue;
@@ -221,6 +233,12 @@ impl RuntimeQuotaRefreshService {
}
}
fn quota_cooldown_ready(cooldown_until: chrono::DateTime<Utc>, now: chrono::DateTime<Utc>) -> bool {
cooldown_until
.checked_add_signed(chrono::Duration::seconds(COOLDOWN_REFRESH_GRACE_SECS))
.is_some_and(|ready_at| now >= ready_at)
}
/// 单次配额刷新摘要。
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct QuotaRefreshSummary {
@@ -228,6 +246,8 @@ pub struct QuotaRefreshSummary {
pub scanned: usize,
/// 配额锁定或待验证候选账号数。
pub candidates: usize,
/// 配额冷却宽限期内跳过的账号数。
pub skipped_cooldown: usize,
/// 成功刷新账号数。
pub refreshed: usize,
/// 最近刷新过而跳过的账号数。
+14 -8
View File
@@ -43,11 +43,17 @@ pub enum ModelServiceError {
#[error("no active accounts available for model refresh")]
NoAccounts,
/// 快照写入失败。
#[error("failed to store model snapshot")]
StoreSnapshot,
#[error("failed to store model snapshot: {source}")]
StoreSnapshot {
#[source]
source: ModelSnapshotStoreError,
},
/// 刷新后重新读取快照失败。
#[error("failed to load model snapshots")]
LoadSnapshots,
#[error("failed to load model snapshots: {source}")]
LoadSnapshots {
#[source]
source: ModelSnapshotStoreError,
},
/// 所有计划都刷新失败。
#[error("all model refresh plans failed")]
AllPlansFailed(ModelRefreshResult),
@@ -251,10 +257,10 @@ impl ModelService {
}
}
fn map_store_snapshot_error(_: ModelSnapshotStoreError) -> ModelServiceError {
ModelServiceError::StoreSnapshot
fn map_store_snapshot_error(source: ModelSnapshotStoreError) -> ModelServiceError {
ModelServiceError::StoreSnapshot { source }
}
fn map_load_snapshots_error(_: ModelSnapshotStoreError) -> ModelServiceError {
ModelServiceError::LoadSnapshots
fn map_load_snapshots_error(source: ModelSnapshotStoreError) -> ModelServiceError {
ModelServiceError::LoadSnapshots { source }
}
@@ -7,7 +7,7 @@ use thiserror::Error;
use super::{info::CodexModelInfo, snapshot::ModelPlanSnapshot};
/// 模型快照存储错误。
#[derive(Debug, Error)]
#[derive(Debug, Error, PartialEq, Eq)]
pub enum ModelSnapshotStoreError {
/// 底层存储失败。
#[error("model snapshot store operation failed: {message}")]
@@ -640,18 +640,6 @@ impl CodexResponsesRequest {
self.body.get("reasoning")
}
/// 设置 reasoning 配置(模型后缀路由)。
pub fn set_reasoning(&mut self, reasoning: Option<Value>) {
match reasoning {
Some(value) => {
self.body.insert("reasoning".to_string(), value);
}
None => {
self.body.remove("reasoning");
}
}
}
/// 工具定义数组(非数组或空时 None)。
pub fn tools(&self) -> Option<&[Value]> {
self.body
@@ -834,6 +834,17 @@ pub fn is_terminal_websocket_event(event: &str) -> bool {
matches!(event, "response.completed" | "response.failed" | "error")
}
/// 判断是否为「首个内容帧之前的纯生命周期帧」。
///
/// `response.created` / `response.in_progress` 仅表示上游已受理请求,尚未开始产出
/// 真实内容(含 reasoning)。首 token 熔断计时不应被它们解除,否则连接一收到
/// `response.created`(通常 1s 内)就被误判为「已开始产出」,使真正的首内容帧
/// 可以无限拖延而不触发保护。其余任何帧(reasoning/delta/output_item 等)到达即视为
/// 上游已开始真实产出,解除熔断——对慢思考模型友好。
pub fn is_pre_content_lifecycle_event(event: &str) -> bool {
matches!(event, "response.created" | "response.in_progress")
}
/// 将 WebSocket 错误帧映射为上游 HTTP 状态分类。
pub fn classify_websocket_error_frame(raw: &str) -> Option<ClassifiedWebSocketError> {
let value = serde_json::from_str::<Value>(raw).ok()?;
+40 -6
View File
@@ -46,7 +46,9 @@ use super::websocket::{
write_websocket_audit_artifact_from_env, CodexWebSocketConnection, CodexWebSocketExchangeError,
CodexWebSocketRateLimitHeaderUpdates, CodexWebSocketTurnStateUpdate,
};
use super::websocket_pool::{CodexWebSocketPool, CodexWebSocketPoolKey, WebSocketPoolDecision};
use super::websocket_pool::{
CodexWebSocketPool, CodexWebSocketPoolKey, WebSocketPoolDecision, DEFAULT_FIRST_TOKEN_TIMEOUT,
};
// ---------------------------------------------------------------------------
// Constants
@@ -324,6 +326,7 @@ pub struct CodexBackendClient {
pub(super) base_url: String,
fingerprint: Fingerprint,
websocket_pool: Option<Arc<CodexWebSocketPool>>,
websocket_first_token_timeout: Option<Duration>,
}
impl CodexBackendClient {
@@ -334,9 +337,15 @@ impl CodexBackendClient {
base_url: base_url.into().trim_end_matches('/').to_string(),
fingerprint,
websocket_pool: None,
websocket_first_token_timeout: Some(DEFAULT_FIRST_TOKEN_TIMEOUT),
}
}
pub fn with_websocket_first_token_timeout(mut self, timeout: Option<Duration>) -> Self {
self.websocket_first_token_timeout = timeout.filter(|timeout| !timeout.is_zero());
self
}
/// 为 Responses WebSocket 请求启用连接池。
pub fn with_websocket_pool(mut self, pool: Arc<CodexWebSocketPool>) -> Self {
self.websocket_pool = Some(pool);
@@ -590,10 +599,23 @@ impl CodexBackendClient {
let pool_log_context = pool_key.as_ref().map(WebSocketPoolLogContext::from_key);
let exchange = match (self.websocket_pool.as_deref(), pool_key) {
(Some(pool), Some(key)) => {
execute_response_create_request_with_pool(&prepared, Some((pool, key)), started_at)
.await
execute_response_create_request_with_pool(
&prepared,
Some((pool, key)),
started_at,
self.websocket_first_token_timeout,
)
.await
}
_ => {
execute_response_create_request_with_pool(
&prepared,
None,
started_at,
self.websocket_first_token_timeout,
)
.await
}
_ => execute_response_create_request_with_pool(&prepared, None, started_at).await,
}
.map_err(websocket_exchange_error_to_client_error)?;
log_websocket_pool_decision(
@@ -642,9 +664,21 @@ impl CodexBackendClient {
let pool_log_context = pool_key.as_ref().map(WebSocketPoolLogContext::from_key);
let exchange = match (self.websocket_pool.as_deref(), pool_key) {
(Some(pool), Some(key)) => {
execute_response_create_request_stream_with_pool(&prepared, Some((pool, key))).await
execute_response_create_request_stream_with_pool(
&prepared,
Some((pool, key)),
self.websocket_first_token_timeout,
)
.await
}
_ => {
execute_response_create_request_stream_with_pool(
&prepared,
None,
self.websocket_first_token_timeout,
)
.await
}
_ => execute_response_create_request_stream_with_pool(&prepared, None).await,
}
.map_err(websocket_exchange_error_to_client_error)?;
log_websocket_pool_decision(
+397 -96
View File
@@ -1,6 +1,7 @@
//! Codex WebSocket 连接建立(关键函数)。
use std::{
collections::VecDeque,
io,
path::{Path, PathBuf},
pin::Pin,
@@ -29,7 +30,7 @@ use crate::upstream::protocol::events::{self, TokenUsage};
use crate::upstream::protocol::responses::CodexResponsesRequest;
use crate::upstream::protocol::sse::SseError;
use crate::upstream::protocol::websocket::{
classify_websocket_error_frame, is_terminal_websocket_event,
classify_websocket_error_frame, is_pre_content_lifecycle_event, is_terminal_websocket_event,
retry_after_seconds_from_wrapped_error_headers, websocket_event_to_sse_frame,
websocket_event_type, websocket_incomplete_response_reason, websocket_metadata_turn_state,
websocket_response_completed_parse_error, websocket_response_create_payload_text,
@@ -50,6 +51,7 @@ const WEBSOCKET_EXTENSIONS: &str = "permessage-deflate; client_max_window_bits";
const WEBSOCKET_RECEIVE_IDLE_TIMEOUT: Duration = Duration::from_secs(20);
const WEBSOCKET_ACTIVE_STREAM_IDLE_TIMEOUT: Duration = Duration::from_secs(5 * 60);
const WEBSOCKET_STREAM_BUFFER: usize = 16;
const WEBSOCKET_FIRST_TOKEN_FRESH_RETRY_ATTEMPTS: usize = 2;
/// WebSocket audit artifact 输出目录环境变量。
pub const WS_AUDIT_DIR_ENV: &str = "CODEX_PROXY_WS_AUDIT_DIR";
@@ -200,6 +202,12 @@ pub enum CodexWebSocketExchangeError {
/// 底层失效原因。
message: String,
},
/// 首个内容帧在配置的绝对超时内未到达(连接落到病态上游后端)。
#[error("websocket first content frame not received within {timeout:?}")]
FirstTokenTimeout {
/// 首 token 超时时长。
timeout: Duration,
},
}
impl CodexWebSocketRequest {
@@ -412,30 +420,94 @@ fn request_line_for_endpoint(endpoint: &str) -> String {
async fn execute_fresh_response_create_request(
request: &CodexWebSocketRequest,
started_at: Instant,
first_token_timeout: Option<Duration>,
) -> Result<CodexWebSocketExchange, CodexWebSocketExchangeError> {
let (websocket, response) =
connect_pumped_websocket(request.connection(), PumpKeepalive::disabled()).await?;
let first_token_started_at = Instant::now();
websocket
.send(Message::Text(request.payload_text().to_string().into()))
.await?;
let metadata = websocket_connection_metadata(&response);
let (exchange, _websocket, _metadata) =
collect_websocket_response(websocket, metadata, false, started_at).await?;
let (exchange, _websocket, _metadata) = collect_websocket_response(
websocket,
metadata,
false,
started_at,
first_token_started_at,
first_token_timeout,
)
.await?;
Ok(exchange)
}
async fn execute_fresh_response_create_request_stream(
request: &CodexWebSocketRequest,
first_token_timeout: Option<Duration>,
) -> Result<CodexWebSocketStreamingExchange, CodexWebSocketExchangeError> {
let (websocket, response) =
let (mut websocket, response) =
connect_pumped_websocket(request.connection(), PumpKeepalive::disabled()).await?;
let first_token_started_at = Instant::now();
websocket
.send(Message::Text(request.payload_text().to_string().into()))
.await?;
let metadata = websocket_connection_metadata(&response);
Ok(stream_websocket_response(websocket, metadata, None, None))
let mut metadata = websocket_connection_metadata(&response);
let prefetched_frames = match prefetch_first_content_stream_frames(
&mut websocket,
&mut metadata,
first_token_started_at,
first_token_timeout,
)
.await
{
Ok(prefetched_frames) => prefetched_frames,
Err(error) => {
websocket.close().await;
return Err(error);
}
};
Ok(stream_websocket_response(
websocket,
metadata,
None,
prefetched_frames,
))
}
async fn execute_fresh_response_create_request_with_retries(
request: &CodexWebSocketRequest,
started_at: Instant,
first_token_timeout: Option<Duration>,
attempts: usize,
) -> Result<CodexWebSocketExchange, CodexWebSocketExchangeError> {
let attempts = attempts.max(1);
for attempt in 1..=attempts {
match execute_fresh_response_create_request(request, started_at, first_token_timeout).await
{
Ok(exchange) => return Ok(exchange),
Err(error) if attempt < attempts && is_first_token_timeout(&error) => continue,
Err(error) => return Err(error),
}
}
unreachable!("fresh websocket retry loop always returns");
}
async fn execute_fresh_response_create_request_stream_with_retries(
request: &CodexWebSocketRequest,
first_token_timeout: Option<Duration>,
attempts: usize,
) -> Result<CodexWebSocketStreamingExchange, CodexWebSocketExchangeError> {
let attempts = attempts.max(1);
for attempt in 1..=attempts {
match execute_fresh_response_create_request_stream(request, first_token_timeout).await {
Ok(exchange) => return Ok(exchange),
Err(error) if attempt < attempts && is_first_token_timeout(&error) => continue,
Err(error) => return Err(error),
}
}
unreachable!("fresh websocket stream retry loop always returns");
}
// ---------------------------------------------------------------------------
@@ -446,9 +518,16 @@ pub(crate) async fn execute_response_create_request_with_pool(
request: &CodexWebSocketRequest,
pool: Option<(&CodexWebSocketPool, CodexWebSocketPoolKey)>,
started_at: Instant,
fallback_first_token_timeout: Option<Duration>,
) -> Result<CodexWebSocketExchange, CodexWebSocketExchangeError> {
let Some((pool, key)) = pool else {
return execute_fresh_response_create_request(request, started_at).await;
return execute_fresh_response_create_request_with_retries(
request,
started_at,
fallback_first_token_timeout,
WEBSOCKET_FIRST_TOKEN_FRESH_RETRY_ATTEMPTS,
)
.await;
};
match pool.acquire(&key).await {
@@ -459,9 +538,15 @@ pub(crate) async fn execute_response_create_request_with_pool(
match result {
Err(CodexWebSocketExchangeError::ReusedConnectionDiedBeforeFirstFrame {
..
}) => {
let mut exchange =
execute_fresh_response_create_request(request, started_at).await?;
})
| Err(CodexWebSocketExchangeError::FirstTokenTimeout { .. }) => {
let mut exchange = execute_fresh_response_create_request_with_retries(
request,
started_at,
pool.first_token_timeout(),
WEBSOCKET_FIRST_TOKEN_FRESH_RETRY_ATTEMPTS,
)
.await?;
exchange.pool_decision = Some(WebSocketPoolDecision::retry_after_stale_reuse());
Ok(exchange)
}
@@ -480,16 +565,37 @@ pub(crate) async fn execute_response_create_request_with_pool(
started_at,
)
.await;
if result.is_err() {
pool.discard(&key).await;
match result {
Ok(mut exchange) => {
exchange.pool_decision = Some(WebSocketPoolDecision::new());
Ok(exchange)
}
Err(error) if is_first_token_timeout(&error) => {
pool.discard(&key).await;
let mut exchange = execute_fresh_response_create_request_with_retries(
request,
started_at,
pool.first_token_timeout(),
1,
)
.await?;
exchange.pool_decision = Some(WebSocketPoolDecision::new());
Ok(exchange)
}
Err(error) => {
pool.discard(&key).await;
Err(error)
}
}
result.map(|mut exchange| {
exchange.pool_decision = Some(WebSocketPoolDecision::new());
exchange
})
}
WebSocketPoolAcquire::Bypass(reason) => {
let mut exchange = execute_fresh_response_create_request(request, started_at).await?;
let mut exchange = execute_fresh_response_create_request_with_retries(
request,
started_at,
pool.first_token_timeout(),
WEBSOCKET_FIRST_TOKEN_FRESH_RETRY_ATTEMPTS,
)
.await?;
exchange.pool_decision = Some(WebSocketPoolDecision::bypass(reason));
Ok(exchange)
}
@@ -499,9 +605,15 @@ pub(crate) async fn execute_response_create_request_with_pool(
pub(crate) async fn execute_response_create_request_stream_with_pool(
request: &CodexWebSocketRequest,
pool: Option<(&CodexWebSocketPool, CodexWebSocketPoolKey)>,
fallback_first_token_timeout: Option<Duration>,
) -> Result<CodexWebSocketStreamingExchange, CodexWebSocketExchangeError> {
let Some((pool, key)) = pool else {
return execute_fresh_response_create_request_stream(request).await;
return execute_fresh_response_create_request_stream_with_retries(
request,
fallback_first_token_timeout,
WEBSOCKET_FIRST_TOKEN_FRESH_RETRY_ATTEMPTS,
)
.await;
};
match pool.acquire(&key).await {
@@ -516,9 +628,14 @@ pub(crate) async fn execute_response_create_request_stream_with_pool(
match result {
Err(CodexWebSocketExchangeError::ReusedConnectionDiedBeforeFirstFrame {
..
}) => {
let mut exchange =
execute_fresh_response_create_request_stream(request).await?;
})
| Err(CodexWebSocketExchangeError::FirstTokenTimeout { .. }) => {
let mut exchange = execute_fresh_response_create_request_stream_with_retries(
request,
pool.first_token_timeout(),
WEBSOCKET_FIRST_TOKEN_FRESH_RETRY_ATTEMPTS,
)
.await?;
exchange.pool_decision = Some(WebSocketPoolDecision::retry_after_stale_reuse());
Ok(exchange)
}
@@ -536,16 +653,35 @@ pub(crate) async fn execute_response_create_request_stream_with_pool(
key.clone(),
)
.await;
if result.is_err() {
pool.discard(&key).await;
match result {
Ok(mut exchange) => {
exchange.pool_decision = Some(WebSocketPoolDecision::new());
Ok(exchange)
}
Err(error) if is_first_token_timeout(&error) => {
pool.discard(&key).await;
let mut exchange = execute_fresh_response_create_request_stream_with_retries(
request,
pool.first_token_timeout(),
1,
)
.await?;
exchange.pool_decision = Some(WebSocketPoolDecision::new());
Ok(exchange)
}
Err(error) => {
pool.discard(&key).await;
Err(error)
}
}
result.map(|mut exchange| {
exchange.pool_decision = Some(WebSocketPoolDecision::new());
exchange
})
}
WebSocketPoolAcquire::Bypass(reason) => {
let mut exchange = execute_fresh_response_create_request_stream(request).await?;
let mut exchange = execute_fresh_response_create_request_stream_with_retries(
request,
pool.first_token_timeout(),
WEBSOCKET_FIRST_TOKEN_FRESH_RETRY_ATTEMPTS,
)
.await?;
exchange.pool_decision = Some(WebSocketPoolDecision::bypass(reason));
Ok(exchange)
}
@@ -560,14 +696,22 @@ async fn execute_fresh_pooled_response_create_request(
) -> Result<CodexWebSocketExchange, CodexWebSocketExchangeError> {
let (websocket, response) =
connect_pumped_websocket(request.connection(), pool.keepalive()).await?;
let first_token_started_at = Instant::now();
websocket
.send(Message::Text(request.payload_text().to_string().into()))
.await?;
let now = Instant::now();
let metadata = websocket_connection_metadata(&response);
let (exchange, websocket, metadata) =
collect_websocket_response(websocket, metadata, false, started_at).await?;
let (exchange, websocket, metadata) = collect_websocket_response(
websocket,
metadata,
false,
started_at,
first_token_started_at,
pool.first_token_timeout(),
)
.await?;
pool.put(
key,
PooledWebSocketConnection {
@@ -585,14 +729,29 @@ async fn execute_fresh_pooled_response_create_request_stream(
pool: CodexWebSocketPool,
key: CodexWebSocketPoolKey,
) -> Result<CodexWebSocketStreamingExchange, CodexWebSocketExchangeError> {
let (websocket, response) =
let (mut websocket, response) =
connect_pumped_websocket(request.connection(), pool.keepalive()).await?;
let first_token_started_at = Instant::now();
websocket
.send(Message::Text(request.payload_text().to_string().into()))
.await?;
let now = Instant::now();
let metadata = websocket_connection_metadata(&response);
let mut metadata = websocket_connection_metadata(&response);
let prefetched_frames = match prefetch_first_content_stream_frames(
&mut websocket,
&mut metadata,
first_token_started_at,
pool.first_token_timeout(),
)
.await
{
Ok(prefetched_frames) => prefetched_frames,
Err(error) => {
websocket.close().await;
return Err(error);
}
};
Ok(stream_websocket_response(
websocket,
metadata,
@@ -601,7 +760,7 @@ async fn execute_fresh_pooled_response_create_request_stream(
key,
created_at: now,
}),
None,
prefetched_frames,
))
}
@@ -612,6 +771,7 @@ async fn execute_pooled_response_create_request(
connection: PooledWebSocketConnection,
started_at: Instant,
) -> Result<CodexWebSocketExchange, CodexWebSocketExchangeError> {
let first_token_started_at = Instant::now();
if let Err(error) = connection
.websocket
.send(Message::Text(request.payload_text().to_string().into()))
@@ -626,8 +786,15 @@ async fn execute_pooled_response_create_request(
}
let created_at = connection.created_at;
match collect_websocket_response(connection.websocket, connection.metadata, true, started_at)
.await
match collect_websocket_response(
connection.websocket,
connection.metadata,
true,
started_at,
first_token_started_at,
pool.first_token_timeout(),
)
.await
{
Ok((exchange, websocket, metadata)) => {
pool.put(
@@ -652,10 +819,13 @@ async fn execute_pooled_response_create_request_stream(
request: &CodexWebSocketRequest,
pool: CodexWebSocketPool,
key: CodexWebSocketPoolKey,
mut connection: PooledWebSocketConnection,
connection: PooledWebSocketConnection,
) -> Result<CodexWebSocketStreamingExchange, CodexWebSocketExchangeError> {
if let Err(error) = connection
.websocket
let mut websocket = connection.websocket;
let mut metadata = connection.metadata;
let created_at = connection.created_at;
let first_token_started_at = Instant::now();
if let Err(error) = websocket
.send(Message::Text(request.payload_text().to_string().into()))
.await
{
@@ -667,51 +837,34 @@ async fn execute_pooled_response_create_request_stream(
);
}
let prefetched_text = first_reused_stream_text(&mut connection.websocket).await?;
let prefetched_frames = match prefetch_first_content_stream_frames(
&mut websocket,
&mut metadata,
first_token_started_at,
pool.first_token_timeout(),
)
.await
{
Ok(prefetched_frames) => prefetched_frames,
Err(error) => {
pool.discard(&key).await;
websocket.close().await;
return Err(reused_stream_prefetch_error(error));
}
};
Ok(stream_websocket_response(
connection.websocket,
connection.metadata,
websocket,
metadata,
Some(WebSocketStreamPoolReturn {
pool,
key,
created_at: connection.created_at,
created_at,
}),
Some(prefetched_text),
prefetched_frames,
))
}
async fn first_reused_stream_text(
websocket: &mut PumpedWebSocket,
) -> Result<String, CodexWebSocketExchangeError> {
loop {
let message = match next_websocket_message(websocket, WEBSOCKET_RECEIVE_IDLE_TIMEOUT).await
{
Ok(Some(message)) => message,
Ok(None) => {
return Err(
CodexWebSocketExchangeError::ReusedConnectionDiedBeforeFirstFrame {
message: "websocket closed".to_string(),
},
);
}
Err(error) => return Err(reused_connection_died_before_first_frame(&error)),
};
match message {
Message::Text(text) => return Ok(text.to_string()),
Message::Close(_) => {
return Err(
CodexWebSocketExchangeError::ReusedConnectionDiedBeforeFirstFrame {
message: "websocket closed".to_string(),
},
);
}
Message::Binary(_) => return Err(CodexWebSocketExchangeError::UnexpectedBinaryEvent),
_ => {}
}
}
}
// ---------------------------------------------------------------------------
// Public entry points
// ---------------------------------------------------------------------------
@@ -720,7 +873,7 @@ async fn first_reused_stream_text(
pub async fn execute_response_create_request(
request: &CodexWebSocketRequest,
) -> Result<CodexWebSocketExchange, CodexWebSocketExchangeError> {
execute_response_create_request_with_pool(request, None, Instant::now()).await
execute_response_create_request_with_pool(request, None, Instant::now(), None).await
}
// ---------------------------------------------------------------------------
@@ -732,6 +885,8 @@ async fn collect_websocket_response(
mut metadata: CodexWebSocketConnectionMetadata,
reused_connection: bool,
started_at: Instant,
first_token_started_at: Instant,
first_token_timeout: Option<Duration>,
) -> Result<
(
CodexWebSocketExchange,
@@ -741,27 +896,37 @@ async fn collect_websocket_response(
CodexWebSocketExchangeError,
> {
let mut body = String::new();
let mut saw_response_frame = false;
let mut saw_content_frame = false;
let mut first_token_ms = None;
loop {
let message =
match next_websocket_message(&mut websocket, receive_idle_timeout(saw_response_frame))
.await
{
Ok(message) => message,
Err(error) if reused_connection && !saw_response_frame => {
let receive_timeout = receive_idle_timeout_with_first_token_deadline(
saw_content_frame,
first_token_started_at,
first_token_timeout,
)?;
let message = match next_websocket_message(&mut websocket, receive_timeout).await {
Ok(message) => message,
Err(error) if !saw_content_frame => {
if let Some(timeout) =
elapsed_first_token_timeout(first_token_started_at, first_token_timeout)
{
return Err(CodexWebSocketExchangeError::FirstTokenTimeout { timeout });
}
if reused_connection {
return Err(reused_connection_died_before_first_frame(&error));
}
Err(error) => return Err(error),
};
return Err(error);
}
Err(error) => return Err(error),
};
let Some(message) = message else {
break;
};
let text = match message {
Message::Text(text) => text,
Message::Binary(_) => return Err(CodexWebSocketExchangeError::UnexpectedBinaryEvent),
Message::Close(_) if reused_connection && !saw_response_frame => {
Message::Close(_) if reused_connection && !saw_content_frame => {
return Err(
CodexWebSocketExchangeError::ReusedConnectionDiedBeforeFirstFrame {
message: "websocket closed".to_string(),
@@ -799,8 +964,14 @@ async fn collect_websocket_response(
let event = websocket_event_type(&raw);
let forwarded = if let Some(frame) = websocket_event_to_sse_frame(&raw) {
body.push_str(&frame);
saw_response_frame = true;
response_meta::update_first_token_ms(started_at, body.as_bytes(), &mut first_token_ms);
if event_is_first_content(event.as_deref()) {
saw_content_frame = true;
response_meta::update_first_token_ms(
started_at,
body.as_bytes(),
&mut first_token_ms,
);
}
true
} else {
false
@@ -820,7 +991,7 @@ async fn collect_websocket_response(
}
}
if reused_connection && !saw_response_frame {
if reused_connection && !saw_content_frame {
return Err(
CodexWebSocketExchangeError::ReusedConnectionDiedBeforeFirstFrame {
message: "websocket closed before terminal event".to_string(),
@@ -831,6 +1002,84 @@ async fn collect_websocket_response(
Err(CodexWebSocketExchangeError::ClosedBeforeTerminal)
}
async fn prefetch_first_content_stream_frames(
websocket: &mut PumpedWebSocket,
metadata: &mut CodexWebSocketConnectionMetadata,
first_token_started_at: Instant,
first_token_timeout: Option<Duration>,
) -> Result<Vec<String>, CodexWebSocketExchangeError> {
let mut prefetched_frames = Vec::new();
loop {
let receive_timeout =
receive_timeout_before_first_content(first_token_started_at, first_token_timeout)?;
let message = match next_websocket_message(websocket, receive_timeout).await {
Ok(Some(message)) => message,
Ok(None) => return Err(CodexWebSocketExchangeError::ClosedBeforeTerminal),
Err(error) => {
if let Some(timeout) =
elapsed_first_token_timeout(first_token_started_at, first_token_timeout)
{
return Err(CodexWebSocketExchangeError::FirstTokenTimeout { timeout });
}
return Err(error);
}
};
let raw = match message {
Message::Text(text) => text.to_string(),
Message::Binary(_) => return Err(CodexWebSocketExchangeError::UnexpectedBinaryEvent),
Message::Close(_) => return Err(CodexWebSocketExchangeError::ClosedBeforeTerminal),
_ => continue,
};
if let Some(classified) = classify_websocket_error_frame(&raw) {
let retry_after_seconds = retry_after_seconds_from_wrapped_error_headers(&raw)
.or_else(|| events::retry_after_seconds_from_body(&raw));
return Err(CodexWebSocketExchangeError::Upstream {
status_code: classified.status_code,
retry_after_seconds,
body: raw,
set_cookie_headers: Vec::new(),
});
}
if let Some(reason) = websocket_incomplete_response_reason(&raw) {
return Err(CodexWebSocketExchangeError::IncompleteResponse { reason });
}
if let Some(message) = websocket_response_completed_parse_error(&raw) {
return Err(CodexWebSocketExchangeError::InvalidCompletedResponse { message });
}
if let Some(headers) = websocket_rate_limit_event_headers(&raw) {
metadata.rate_limit_headers.extend(headers);
continue;
}
if let Some(metadata_turn_state) = websocket_metadata_turn_state(&raw) {
metadata.turn_state = Some(metadata_turn_state);
continue;
}
let event = websocket_event_type(&raw);
if websocket_event_to_sse_frame(&raw).is_none() {
continue;
}
let is_first_content = event_is_first_content(event.as_deref());
prefetched_frames.push(raw);
if is_first_content {
return Ok(prefetched_frames);
}
}
}
fn reused_stream_prefetch_error(error: CodexWebSocketExchangeError) -> CodexWebSocketExchangeError {
match error {
CodexWebSocketExchangeError::ClosedBeforeTerminal
| CodexWebSocketExchangeError::ReceiveIdleTimeout { .. }
| CodexWebSocketExchangeError::Transport(_) => {
reused_connection_died_before_first_frame(&error)
}
error => error,
}
}
fn reused_connection_died_before_first_frame(
error: &CodexWebSocketExchangeError,
) -> CodexWebSocketExchangeError {
@@ -863,7 +1112,7 @@ fn stream_websocket_response(
websocket: PumpedWebSocket,
metadata: CodexWebSocketConnectionMetadata,
pool_return: Option<WebSocketStreamPoolReturn>,
prefetched_text: Option<String>,
prefetched_frames: Vec<String>,
) -> CodexWebSocketStreamingExchange {
let response_metadata = metadata.clone();
let rate_limit_header_updates = Arc::new(Mutex::new(Vec::new()));
@@ -876,7 +1125,7 @@ fn stream_websocket_response(
websocket,
metadata,
pool_return,
prefetched_text,
prefetched_frames,
rate_limit_header_updates_for_task,
turn_state_update_for_task,
tx,
@@ -903,16 +1152,16 @@ async fn forward_websocket_response_stream(
mut websocket: PumpedWebSocket,
mut metadata: CodexWebSocketConnectionMetadata,
pool_return: Option<WebSocketStreamPoolReturn>,
prefetched_text: Option<String>,
prefetched_frames: Vec<String>,
rate_limit_header_updates: CodexWebSocketRateLimitHeaderUpdates,
turn_state_update: CodexWebSocketTurnStateUpdate,
tx: mpsc::Sender<Result<Bytes, CodexWebSocketExchangeError>>,
) {
let mut pool_return = pool_return;
let mut prefetched_text = prefetched_text;
let mut saw_response_frame = false;
let mut prefetched_frames: VecDeque<String> = prefetched_frames.into();
let mut saw_content_frame = false;
loop {
let raw = if let Some(text) = prefetched_text.take() {
let raw = if let Some(text) = prefetched_frames.pop_front() {
text
} else {
let message = tokio::select! {
@@ -925,7 +1174,7 @@ async fn forward_websocket_response_stream(
}
message = next_websocket_message(
&mut websocket,
receive_idle_timeout(saw_response_frame),
receive_idle_timeout(saw_content_frame),
) => message,
};
let message = match message {
@@ -1009,7 +1258,9 @@ async fn forward_websocket_response_stream(
discard_stream_websocket(websocket, pool_return).await;
return;
}
saw_response_frame = true;
if event_is_first_content(event.as_deref()) {
saw_content_frame = true;
}
if terminal {
finish_stream_websocket(websocket, metadata, pool_return.take()).await;
return;
@@ -1030,6 +1281,56 @@ fn receive_idle_timeout(saw_response_frame: bool) -> Duration {
}
}
fn receive_idle_timeout_with_first_token_deadline(
saw_content_frame: bool,
first_token_started_at: Instant,
first_token_timeout: Option<Duration>,
) -> Result<Duration, CodexWebSocketExchangeError> {
if saw_content_frame {
return Ok(WEBSOCKET_ACTIVE_STREAM_IDLE_TIMEOUT);
}
receive_timeout_before_first_content(first_token_started_at, first_token_timeout)
}
fn receive_timeout_before_first_content(
first_token_started_at: Instant,
first_token_timeout: Option<Duration>,
) -> Result<Duration, CodexWebSocketExchangeError> {
let Some(first_token_timeout) = first_token_timeout.filter(|timeout| !timeout.is_zero()) else {
return Ok(WEBSOCKET_RECEIVE_IDLE_TIMEOUT);
};
let Some(remaining) = first_token_timeout.checked_sub(first_token_started_at.elapsed()) else {
return Err(CodexWebSocketExchangeError::FirstTokenTimeout {
timeout: first_token_timeout,
});
};
if remaining.is_zero() {
return Err(CodexWebSocketExchangeError::FirstTokenTimeout {
timeout: first_token_timeout,
});
}
Ok(WEBSOCKET_RECEIVE_IDLE_TIMEOUT.min(remaining))
}
fn elapsed_first_token_timeout(
first_token_started_at: Instant,
first_token_timeout: Option<Duration>,
) -> Option<Duration> {
let first_token_timeout = first_token_timeout.filter(|timeout| !timeout.is_zero())?;
(first_token_started_at.elapsed() >= first_token_timeout).then_some(first_token_timeout)
}
fn event_is_first_content(event: Option<&str>) -> bool {
match event {
Some(event) => !is_pre_content_lifecycle_event(event),
None => true,
}
}
fn is_first_token_timeout(error: &CodexWebSocketExchangeError) -> bool {
matches!(error, CodexWebSocketExchangeError::FirstTokenTimeout { .. })
}
async fn next_websocket_message(
websocket: &mut PumpedWebSocket,
receive_timeout: Duration,
@@ -17,6 +17,7 @@ const DEFAULT_MAX_AGE: Duration = Duration::from_mins(55);
const DEFAULT_MAINTENANCE_INTERVAL: Duration = Duration::from_secs(25);
const DEFAULT_PING_INTERVAL: Duration = Duration::from_secs(25);
const DEFAULT_PING_TIMEOUT: Duration = Duration::from_secs(5);
pub(crate) const DEFAULT_FIRST_TOKEN_TIMEOUT: Duration = Duration::from_secs(15);
/// WebSocket 连接池 key。
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
@@ -87,6 +88,8 @@ pub struct CodexWebSocketPoolConfig {
pub ping_timeout: Duration,
/// idle socket 无活动多久后视为失活。
pub liveness_timeout: Option<Duration>,
/// 首个内容帧到达前的绝对超时;`None` 表示禁用首 token 熔断。
pub first_token_timeout: Option<Duration>,
}
impl Default for CodexWebSocketPoolConfig {
@@ -101,6 +104,7 @@ impl Default for CodexWebSocketPoolConfig {
// idle 连接不设失活截断:靠 ping/pong 保活,只在 max_age(55 分钟)
// 或 ping 失败时关闭,最大化跨轮复用(对齐 Codex CLI 的长连接策略)。
liveness_timeout: None,
first_token_timeout: Some(DEFAULT_FIRST_TOKEN_TIMEOUT),
}
}
}
@@ -143,6 +147,11 @@ impl CodexWebSocketPool {
self.config.keepalive()
}
/// 首个内容帧到达前的绝对超时;`None` 表示禁用首 token 熔断。
pub(crate) fn first_token_timeout(&self) -> Option<Duration> {
self.config.first_token_timeout
}
pub(crate) async fn acquire(&self, key: &CodexWebSocketPoolKey) -> WebSocketPoolAcquire {
let mut expired_connection = None;
let acquire = {
+127 -11
View File
@@ -259,11 +259,11 @@ async fn admin_accounts_list_should_include_usage_quota_and_model_stats() {
assert_eq!(response.status(), StatusCode::OK);
let body = response_json(response).await;
let item = &body["data"]["items"][0];
assert_eq!(item["usage"]["requestCount"], 7);
assert_eq!(item["usage"]["inputTokens"], 120_000);
assert_eq!(item["usage"]["outputTokens"], 4_000);
assert_eq!(item["usage"]["cachedTokens"], 40_000);
assert_eq!(item["usage"]["totalTokens"], 124_000);
assert_eq!(item["usage"]["requestCount"], 107);
assert_eq!(item["usage"]["inputTokens"], 1_022_000);
assert_eq!(item["usage"]["outputTokens"], 94_500);
assert_eq!(item["usage"]["cachedTokens"], 49_100);
assert_eq!(item["usage"]["totalTokens"], 1_116_500);
assert_eq!(item["quota"]["windows"].as_array().unwrap().len(), 3);
assert_eq!(item["quota"]["windows"][0]["labelDisplay"], "月限额");
assert_eq!(item["quota"]["windows"][0]["group"], "monthly");
@@ -277,12 +277,128 @@ async fn admin_accounts_list_should_include_usage_quota_and_model_stats() {
assert!(item["quota"]["windows"][2]["windowUsedDisplay"]
.as_str()
.is_some_and(|value| value.contains(" / 7.0d")));
assert_eq!(item["usage"]["createdTokens"], 80_000);
assert_eq!(item["usage"]["readTokens"], 40_000);
assert_eq!(item["usage"]["models"].as_array().unwrap().len(), 2);
assert_eq!(item["usage"]["models"][0]["model"], "gpt-5.5");
assert_eq!(item["usage"]["models"][0]["requestCount"], 4);
assert_eq!(item["usage"]["models"][1]["model"], "gpt-5");
assert_eq!(item["usage"]["createdTokens"], 972_900);
assert_eq!(item["usage"]["readTokens"], 49_100);
assert_eq!(item["usage"]["models"].as_array().unwrap().len(), 4);
assert_eq!(item["usage"]["models"][0]["model"], "gpt-outside-window");
assert_eq!(item["usage"]["models"][0]["requestCount"], 99);
assert_eq!(item["usage"]["models"][1]["model"], "gpt-5.5");
assert_eq!(item["usage"]["models"][2]["model"], "gpt-5");
assert_eq!(item["usage"]["models"][3]["model"], "gpt-five-hour-window");
}
#[tokio::test]
async fn admin_accounts_list_should_use_largest_quota_reset_interval_for_usage_stats() {
let (app, _state, pool, _dir) =
admin_accounts_test_app("admin-accounts-largest-quota-window.sqlite", 71).await;
seed_usage_account(
&pool,
UsageAccountSeed {
id: "acct_custom_window",
email: "custom-window@example.com",
label: "custom",
plan_type: "plus",
request_count: 10,
empty_response_count: 0,
input_tokens: 500,
output_tokens: 100,
cached_tokens: 50,
window_request_count: 1,
window_input_tokens: 100,
window_output_tokens: 10,
window_cached_tokens: 5,
window_started_at: "2026-07-13T19:00:00Z",
window_reset_at: "2026-07-14T00:00:00Z",
limit_window_seconds: 18_000,
last_used_at: "2026-06-30T01:10:00Z",
},
)
.await;
sqlx::query("update accounts set quota_json = ?, quota_fetched_at = ? where id = ?")
.bind(
json!({
"plan_type": "plus",
"snapshots": [{
"source": "custom",
"limit_name": "Codex",
"metered_feature": "codex",
"allowed": true,
"limit_reached": false,
"blocked": false,
"primary": {
"used_percent": 50,
"remaining_percent": 50,
"reset_at": 1783987200,
"window_minutes": 20160,
"limit_reached": false
},
"secondary": {
"used_percent": 10,
"remaining_percent": 90,
"reset_at": 1783987200,
"window_minutes": 10080,
"limit_reached": false
}
}]
})
.to_string(),
)
.bind("2026-06-30T01:10:00Z")
.bind("acct_custom_window")
.execute(&pool)
.await
.unwrap();
sqlx::query(
"insert into usage_time_buckets (bucket_start, account_id, model, request_count, input_tokens, output_tokens, cached_tokens, updated_at) values (?, ?, ?, ?, ?, ?, ?, ?)",
)
.bind("2026-06-30T01:00:00+00:00")
.bind("acct_custom_window")
.bind("gpt-custom-14d")
.bind(5)
.bind(10_000)
.bind(2_000)
.bind(1_000)
.bind("2026-06-30T01:10:00Z")
.execute(&pool)
.await
.unwrap();
sqlx::query(
"insert into usage_time_buckets (bucket_start, account_id, model, request_count, input_tokens, output_tokens, cached_tokens, updated_at) values (?, ?, ?, ?, ?, ?, ?, ?)",
)
.bind("2026-06-29T23:45:00+00:00")
.bind("acct_custom_window")
.bind("gpt-before-14d")
.bind(99)
.bind(900_000)
.bind(90_000)
.bind(9_000)
.bind("2026-06-29T23:59:00Z")
.execute(&pool)
.await
.unwrap();
let response = app
.oneshot(
Request::builder()
.method("GET")
.uri("/api/admin/accounts?page=1&pageSize=10")
.header("cookie", "cpr_admin_session=session_1")
.header("x-request-id", "req_accounts_largest_quota_window")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::OK);
let body = response_json(response).await;
let item = &body["data"]["items"][0];
assert_eq!(item["usage"]["requestCount"], 5);
assert_eq!(item["usage"]["inputTokens"], 10_000);
assert_eq!(item["usage"]["outputTokens"], 2_000);
assert_eq!(item["usage"]["cachedTokens"], 1_000);
assert_eq!(item["usage"]["models"].as_array().unwrap().len(), 1);
assert_eq!(item["usage"]["models"][0]["model"], "gpt-custom-14d");
}
#[tokio::test]
+326 -86
View File
@@ -14,7 +14,7 @@ use codex_proxy_rs::{
},
infra::{
database::connect_sqlite,
time::{china_day_start, china_hour, china_hour_start},
time::{china_datetime, china_day_start, china_hour, china_hour_start},
},
proxy::dispatch::session_affinity::SqliteSessionAffinityStore,
runtime::{
@@ -66,8 +66,9 @@ async fn dashboard_summary_should_render_dash_when_fingerprint_updated_at_is_mis
}
#[tokio::test]
async fn dashboard_summary_should_calculate_today_traffic_and_latency_from_time_buckets() {
let (app, store, _pool, _dir) = dashboard_test_app(
async fn dashboard_summary_should_calculate_today_traffic_and_average_first_token_latency_from_time_buckets(
) {
let (app, store, pool, _dir) = dashboard_test_app(
"dashboard-traffic-latency.sqlite",
crate::support::fingerprint::test_fingerprint(),
)
@@ -93,25 +94,32 @@ async fn dashboard_summary_should_calculate_today_traffic_and_latency_from_time_
.append(&usage_record_with_tokens(yesterday_record_at, 30))
.await
.unwrap();
seed_usage_summary_with_cached_tokens(&pool, 120, 30, 24).await;
let body = dashboard_summary(app).await;
assert_eq!(body["data"]["cards"]["traffic"]["rpm"], "2");
assert_eq!(body["data"]["cards"]["traffic"]["tpm"], "30");
assert_eq!(body["data"]["cards"]["traffic"]["todayRequests"], "2");
assert_eq!(body["data"]["cards"]["traffic"]["todayRequestsValue"], 2);
assert_eq!(body["data"]["cards"]["traffic"]["yesterdayRequests"], "1");
assert_eq!(body["data"]["cards"]["traffic"]["totalRequests"], "42");
assert_eq!(
body["data"]["cards"]["traffic"]["yesterdayRequestsValue"],
1
);
assert_eq!(body["data"]["cards"]["tokens"]["todayTokens"], "30");
assert_eq!(body["data"]["cards"]["tokens"]["todayTokensValue"], 30);
assert_eq!(body["data"]["cards"]["tokens"]["yesterdayTokens"], "30");
assert_eq!(body["data"]["cards"]["tokens"]["totalTokens"], "150");
assert_eq!(body["data"]["cards"]["tokens"]["yesterdayTokensValue"], 30);
assert_eq!(body["data"]["cards"]["cache"]["totalHitRate"], "20.0%");
assert_eq!(body["data"]["cards"]["cache"]["totalCachedTokens"], "24");
assert_eq!(
body["data"]["cards"]["cache"]["firstTokenLatencyMs"],
body["data"]["cards"]["cache"]["averageFirstTokenLatencyMs"],
"300 ms"
);
assert_eq!(
body["data"]["cards"]["cache"]["completionLatencyMs"],
"1.50 s"
);
assert!(body["data"]["cards"]["cache"]
.as_object()
.unwrap()
.get("averageLatencyMs")
.is_none());
}
#[tokio::test]
@@ -151,7 +159,12 @@ async fn dashboard_trend_should_bucket_usage_by_china_hour() {
.await;
let now = Utc::now();
store
.append(&usage_record_with_tokens(now - Duration::seconds(1), 10))
.append(&usage_record_with_token_parts(
now - Duration::seconds(1),
40,
60,
10,
))
.await
.unwrap();
@@ -164,7 +177,41 @@ async fn dashboard_trend_should_bucket_usage_by_china_hour() {
.expect("trend should include the inserted usage bucket");
assert_eq!(point["time"], format!("{:02}", china_hour(&now)));
assert_eq!(point["tokensValue"], 10);
assert_eq!(point["tokensValue"], 100);
assert_eq!(point["cachedTokensValue"], 10);
assert_f64_eq(point["cacheHitRateValue"].as_f64().unwrap(), 0.25);
}
#[tokio::test]
async fn dashboard_trend_should_only_include_china_today() {
let (app, store, _pool, _dir) = dashboard_test_app(
"dashboard-trend-china-today.sqlite",
crate::support::fingerprint::test_fingerprint(),
)
.await;
let now = Utc::now();
let today_start = china_day_start(now);
store
.append(&usage_record_with_tokens(
today_start - Duration::hours(1),
99,
))
.await
.unwrap();
store
.append(&usage_record_with_tokens(now, 7))
.await
.unwrap();
let body = dashboard_trend(app, "usage").await;
let points = body["data"]["points"].as_array().unwrap();
let token_total = points
.iter()
.map(|point| point["tokensValue"].as_u64().unwrap())
.sum::<u64>();
assert_eq!(points.first().unwrap()["time"], "00");
assert_eq!(token_total, 7);
}
#[tokio::test]
@@ -241,7 +288,7 @@ async fn dashboard_summary_should_return_requested_trend_kind() {
#[tokio::test]
async fn dashboard_summary_should_calculate_priority_cost_from_event_service_tier() {
let cost = dashboard_summary_total_cost_for_usage_record(
let cost = dashboard_summary_today_cost_for_usage_record(
"dashboard-cost-priority.sqlite",
"gpt-5.5",
json!({
@@ -255,12 +302,12 @@ async fn dashboard_summary_should_calculate_priority_cost_from_event_service_tie
)
.await;
assert_f64_eq(cost, 8.75);
assert_eq!(cost, "$8.75");
}
#[tokio::test]
async fn dashboard_summary_should_charge_cached_input_at_cache_read_price() {
let cost = dashboard_summary_total_cost_for_usage_record(
let cost = dashboard_summary_today_cost_for_usage_record(
"dashboard-cost-cached.sqlite",
"gpt-5.5",
json!({
@@ -273,12 +320,12 @@ async fn dashboard_summary_should_charge_cached_input_at_cache_read_price() {
)
.await;
assert_f64_eq(cost, 0.03275);
assert_eq!(cost, "$0.03");
}
#[tokio::test]
async fn dashboard_summary_should_apply_long_context_prices() {
let cost = dashboard_summary_total_cost_for_usage_record(
let cost = dashboard_summary_today_cost_for_usage_record(
"dashboard-cost-long-context.sqlite",
"gpt-5.5",
json!({
@@ -291,12 +338,12 @@ async fn dashboard_summary_should_apply_long_context_prices() {
)
.await;
assert_f64_eq(cost, 3.045);
assert_eq!(cost, "$3.04");
}
#[tokio::test]
async fn dashboard_summary_should_normalize_codex_model_names_for_cost() {
let cost = dashboard_summary_total_cost_for_usage_record(
let cost = dashboard_summary_today_cost_for_usage_record(
"dashboard-cost-codex-model.sqlite",
"codex-auto-review",
json!({
@@ -309,7 +356,7 @@ async fn dashboard_summary_should_normalize_codex_model_names_for_cost() {
)
.await;
assert_f64_eq(cost, 1.75);
assert_eq!(cost, "$1.75");
}
#[tokio::test]
@@ -323,62 +370,106 @@ async fn dashboard_summary_should_not_price_usage_summary_without_model_dimensio
let body = dashboard_summary(app).await;
assert_f64_eq(
body["data"]["cards"]["tokens"]["totalCostUsdValue"]
.as_f64()
.unwrap(),
0.0,
);
assert_eq!(body["data"]["cards"]["tokens"]["todayCostUsd"], "$0.00");
}
#[tokio::test]
async fn dashboard_summary_should_return_backend_formatted_time_fields() {
let (app, store, pool, _dir) = dashboard_test_app(
let (app, store, _pool, _dir) = dashboard_test_app(
"dashboard-backend-formatted-time.sqlite",
crate::support::fingerprint::test_fingerprint(),
)
.await;
let record_at = Utc::now() - Duration::seconds(1);
store
.append(&usage_record_with_tokens(
"2026-06-18T12:34:56Z".parse().unwrap(),
10,
))
.append(&usage_record_with_tokens(record_at, 10))
.await
.unwrap();
seed_usage_summary_with_last_used(&pool, 1, 1, "2000-01-01T00:00:00Z").await;
let body = dashboard_summary(app).await;
assert_eq!(
body["data"]["usageRecords"][0]["createdAtDisplay"],
"2026-06-18 20:34:56"
china_datetime(&record_at)
);
assert_eq!(body["data"]["accountUsage"][0]["lastUsed"], "2000-01-01");
assert!(body["data"]["usageRecords"][0].get("metadata").is_none());
}
#[tokio::test]
async fn dashboard_summary_should_only_return_today_account_usage_and_usage_records() {
let (app, store, pool, _dir) = dashboard_test_app(
"dashboard-summary-today-records.sqlite",
crate::support::fingerprint::test_fingerprint(),
)
.await;
seed_dashboard_account(&pool, "acct_today", "today@example.com").await;
seed_dashboard_account(&pool, "acct_yesterday", "yesterday@example.com").await;
let now = Utc::now();
let today_start = china_day_start(now);
let today_record_at = if now - today_start > Duration::minutes(1) {
today_start + Duration::minutes(1)
} else {
now - Duration::seconds(1)
};
let yesterday_record_at = today_start - Duration::minutes(1);
store
.append(&usage_record_with_account_tokens(
yesterday_record_at,
"acct_yesterday",
90,
))
.await
.unwrap();
store
.append(&usage_record_with_account_tokens(
today_record_at,
"acct_today",
7,
))
.await
.unwrap();
let body = dashboard_summary(app).await;
assert_eq!(body["data"]["usageRecords"].as_array().unwrap().len(), 1);
assert_eq!(
body["data"]["usageRecords"][0]["accountEmail"],
"today@example.com"
);
assert_eq!(body["data"]["accountUsage"].as_array().unwrap().len(), 1);
assert_eq!(
body["data"]["accountUsage"][0]["email"],
"today@example.com"
);
assert_eq!(body["data"]["accountUsage"][0]["tokens"], "7");
}
#[tokio::test]
async fn dashboard_summary_should_order_account_usage_by_recent_usage() {
let (app, _store, pool, _dir) = dashboard_test_app(
let (app, store, pool, _dir) = dashboard_test_app(
"dashboard-account-usage-recent-order.sqlite",
crate::support::fingerprint::test_fingerprint(),
)
.await;
seed_dashboard_account_usage(
&pool,
"acct_old_heavy",
"old-heavy@example.com",
2_600_000,
"2026-06-29T09:00:00Z",
)
.await;
seed_dashboard_account_usage(
&pool,
"acct_recent_light",
"recent-light@example.com",
660_000,
"2026-06-30T08:00:00Z",
)
.await;
seed_dashboard_account(&pool, "acct_old_heavy", "old-heavy@example.com").await;
seed_dashboard_account(&pool, "acct_recent_light", "recent-light@example.com").await;
let now = Utc::now();
store
.append(&usage_record_with_account_tokens(
now - Duration::minutes(2),
"acct_old_heavy",
2_600_000,
))
.await
.unwrap();
store
.append(&usage_record_with_account_tokens(
now - Duration::minutes(1),
"acct_recent_light",
660_000,
))
.await
.unwrap();
let body = dashboard_summary(app).await;
@@ -386,13 +477,117 @@ async fn dashboard_summary_should_order_account_usage_by_recent_usage() {
body["data"]["accountUsage"][0]["email"],
"recent-light@example.com"
);
assert_eq!(body["data"]["accountUsage"][0]["requests"], "660K");
assert_eq!(body["data"]["accountUsage"][0]["id"], "acct_recent_light");
assert_eq!(body["data"]["accountUsage"][0]["planType"], "K12");
assert_eq!(body["data"]["accountUsage"][0]["tokens"], "660K");
assert_eq!(
body["data"]["accountUsage"][1]["email"],
"old-heavy@example.com"
);
}
#[tokio::test]
async fn dashboard_summary_should_use_five_hour_quota_window_for_account_usage() {
let (app, store, pool, _dir) = dashboard_test_app(
"dashboard-account-usage-five-hour-quota.sqlite",
crate::support::fingerprint::test_fingerprint(),
)
.await;
seed_dashboard_account(&pool, "acct_quota_five_hour", "quota-five-hour@example.com").await;
store
.append(&usage_record_with_account_tokens(
Utc::now() - Duration::seconds(1),
"acct_quota_five_hour",
1,
))
.await
.unwrap();
set_account_quota_json(
&pool,
"acct_quota_five_hour",
json!({
"monthly_limit": {
"used_percent": 92.0,
"window_minutes": 43_200
},
"snapshots": [
{
"source": "core",
"primary": {
"used_percent": 31.0,
"window_minutes": 300
},
"secondary": {
"used_percent": 88.0,
"window_minutes": 10_080
}
}
]
}),
)
.await;
let body = dashboard_summary(app).await;
assert_f64_eq(
body["data"]["accountUsage"][0]["quotaUsedPercent"]
.as_f64()
.unwrap(),
31.0,
);
}
#[tokio::test]
async fn dashboard_summary_should_fall_back_to_weekly_quota_window_when_five_hour_missing() {
let (app, store, pool, _dir) = dashboard_test_app(
"dashboard-account-usage-weekly-quota.sqlite",
crate::support::fingerprint::test_fingerprint(),
)
.await;
seed_dashboard_account(&pool, "acct_quota_weekly", "quota-weekly@example.com").await;
store
.append(&usage_record_with_account_tokens(
Utc::now() - Duration::seconds(1),
"acct_quota_weekly",
1,
))
.await
.unwrap();
set_account_quota_json(
&pool,
"acct_quota_weekly",
json!({
"monthly_limit": {
"used_percent": 92.0,
"window_minutes": 43_200
},
"snapshots": [
{
"source": "experimental",
"primary": {
"used_percent": 99.0,
"window_minutes": 120
},
"secondary": {
"used_percent": 47.0,
"window_minutes": 10_080
}
}
]
}),
)
.await;
let body = dashboard_summary(app).await;
assert_f64_eq(
body["data"]["accountUsage"][0]["quotaUsedPercent"]
.as_f64()
.unwrap(),
47.0,
);
}
#[tokio::test]
async fn dashboard_summary_should_return_today_health_timeline() {
let (app, store, _pool, _dir) = dashboard_test_app(
@@ -416,19 +611,16 @@ async fn dashboard_summary_should_return_today_health_timeline() {
let points = body["data"]["healthTimeline"]["points"].as_str().unwrap();
assert_eq!(body["data"]["healthTimeline"]["title"], "请求健康时间线");
assert_eq!(
body["data"]["healthTimeline"]["description"],
"今日请求可靠性"
);
assert_eq!(body["data"]["healthTimeline"]["description"], "请求可靠性");
assert_eq!(points.len(), 96);
assert_eq!(points.matches('4').count(), 1);
}
async fn dashboard_summary_total_cost_for_usage_record(
async fn dashboard_summary_today_cost_for_usage_record(
db_name: &str,
model: &str,
metadata: Value,
) -> f64 {
) -> String {
let (app, store, _pool, _dir) =
dashboard_test_app(db_name, crate::support::fingerprint::test_fingerprint()).await;
store
@@ -437,9 +629,10 @@ async fn dashboard_summary_total_cost_for_usage_record(
.unwrap();
let body = dashboard_summary(app).await;
body["data"]["cards"]["tokens"]["totalCostUsdValue"]
.as_f64()
body["data"]["cards"]["tokens"]["todayCostUsd"]
.as_str()
.unwrap()
.to_string()
}
async fn dashboard_test_app(
@@ -526,6 +719,39 @@ async fn seed_usage_summary(pool: &SqlitePool, input_tokens: u64, output_tokens:
.await;
}
async fn seed_usage_summary_with_cached_tokens(
pool: &SqlitePool,
input_tokens: u64,
output_tokens: u64,
cached_tokens: u64,
) {
sqlx::query(
"insert into accounts (id, email, access_token, status, added_at, updated_at) values (?, ?, ?, ?, ?, ?)",
)
.bind("acct_usage_cached")
.bind("acct-usage-cached@example.com")
.bind("cipher")
.bind("active")
.bind("2026-06-18T00:00:00Z")
.bind("2026-06-18T00:00:00Z")
.execute(pool)
.await
.unwrap();
sqlx::query(
"insert into account_usage (account_id, request_count, input_tokens, output_tokens, cached_tokens, total_tokens, last_used_at) values (?, ?, ?, ?, ?, ?, ?)",
)
.bind("acct_usage_cached")
.bind(42_i64)
.bind(input_tokens as i64)
.bind(output_tokens as i64)
.bind(cached_tokens as i64)
.bind((input_tokens + output_tokens) as i64)
.bind("2026-06-18T00:00:00Z")
.execute(pool)
.await
.unwrap();
}
async fn seed_usage_summary_with_last_used(
pool: &SqlitePool,
input_tokens: u64,
@@ -533,9 +759,10 @@ async fn seed_usage_summary_with_last_used(
last_used_at: &str,
) {
sqlx::query(
"insert into accounts (id, access_token, status, added_at, updated_at) values (?, ?, ?, ?, ?)",
"insert into accounts (id, email, access_token, status, added_at, updated_at) values (?, ?, ?, ?, ?, ?)",
)
.bind("acct_usage")
.bind("acct-usage@example.com")
.bind("cipher")
.bind("active")
.bind("2026-06-18T00:00:00Z")
@@ -557,18 +784,14 @@ async fn seed_usage_summary_with_last_used(
.unwrap();
}
async fn seed_dashboard_account_usage(
pool: &SqlitePool,
account_id: &str,
email: &str,
request_count: i64,
last_used_at: &str,
) {
async fn seed_dashboard_account(pool: &SqlitePool, account_id: &str, email: &str) {
sqlx::query(
"insert into accounts (id, email, access_token, status, added_at, updated_at) values (?, ?, ?, ?, ?, ?)",
"insert into accounts (id, email, chatgpt_account_id, plan_type, access_token, status, added_at, updated_at) values (?, ?, ?, ?, ?, ?, ?, ?)",
)
.bind(account_id)
.bind(email)
.bind(format!("upstream_{account_id}"))
.bind("K12")
.bind("cipher")
.bind("active")
.bind("2026-06-18T00:00:00Z")
@@ -576,18 +799,16 @@ async fn seed_dashboard_account_usage(
.execute(pool)
.await
.unwrap();
sqlx::query(
"insert into account_usage (account_id, request_count, input_tokens, output_tokens, total_tokens, last_used_at) values (?, ?, ?, ?, ?, ?)",
)
.bind(account_id)
.bind(request_count)
.bind(0_i64)
.bind(0_i64)
.bind(0_i64)
.bind(last_used_at)
.execute(pool)
.await
.unwrap();
}
async fn set_account_quota_json(pool: &SqlitePool, account_id: &str, quota_json: Value) {
sqlx::query("update accounts set quota_json = ?, quota_fetched_at = ? where id = ?")
.bind(quota_json.to_string())
.bind("2026-06-30T08:01:00Z")
.bind(account_id)
.execute(pool)
.await
.unwrap();
}
fn service_status_value<'a>(body: &'a Value, label: &str) -> &'a str {
@@ -601,19 +822,38 @@ fn service_status_value<'a>(body: &'a Value, label: &str) -> &'a str {
}
fn usage_record_with_tokens(created_at: chrono::DateTime<Utc>, total_tokens: u64) -> UsageRecord {
usage_record_with_token_parts(created_at, total_tokens, 0, 0)
}
fn usage_record_with_token_parts(
created_at: chrono::DateTime<Utc>,
input_tokens: u64,
output_tokens: u64,
cached_tokens: u64,
) -> UsageRecord {
usage_record(
created_at,
"gpt-5.5",
json!({
"usage": {
"inputTokens": total_tokens,
"outputTokens": 0u64,
"cachedTokens": 0u64
"inputTokens": input_tokens,
"outputTokens": output_tokens,
"cachedTokens": cached_tokens
}
}),
)
}
fn usage_record_with_account_tokens(
created_at: chrono::DateTime<Utc>,
account_id: &str,
total_tokens: u64,
) -> UsageRecord {
let mut record = usage_record_with_tokens(created_at, total_tokens);
record.account_id = Some(account_id.to_string());
record
}
fn usage_record(created_at: chrono::DateTime<Utc>, model: &str, metadata: Value) -> UsageRecord {
let mut record = UsageRecord::new("v1.response", UsageRecordLevel::Info, "completed");
record.model = Some(model.to_string());
@@ -21,6 +21,7 @@ async fn scheduler_handle_should_shutdown_websocket_pool() {
max_per_account: 1,
enabled: true,
ping_timeout: Duration::ZERO,
first_token_timeout: None,
}));
SchedulerHandle::from_websocket_pool(pool.clone())
@@ -488,6 +488,145 @@ async fn quota_refresh_service_should_refresh_quota_exhausted_accounts() {
assert_eq!(stored, ("active".to_string(), 0));
}
#[tokio::test]
async fn quota_refresh_service_should_skip_recent_locked_account_before_cooldown_expires() {
let server = MockServer::start().await;
let dir = tempfile::tempdir().expect("temp dir");
let db = dir
.path()
.join("quota-refresh-recent-future-cooldown.sqlite");
let pool = connect_sqlite(&format!("sqlite://{}", db.display()))
.await
.expect("sqlite pool");
let store = SqliteAccountStore::new(pool.clone());
insert_quota_locked_account(&store, &pool, "acct-quota", "access-token").await;
set_quota_cooldown_until(&pool, "acct-quota", Utc::now() + Duration::minutes(5)).await;
let codex = CodexBackendClient::new(
reqwest::Client::new(),
server.uri(),
crate::support::fingerprint::test_fingerprint(),
);
let service =
RuntimeQuotaRefreshService::with_min_refresh_interval_secs(store, Arc::new(codex), 1_800);
let mut last_refreshed = HashMap::new();
last_refreshed.insert("acct-quota".to_string(), tokio::time::Instant::now());
let summary = service
.refresh_locked_accounts(&mut last_refreshed)
.await
.expect("quota refresh should succeed");
assert_eq!(
(
summary.candidates,
summary.skipped_cooldown,
summary.skipped_recent,
summary.refreshed,
usage_request_count(&server).await,
),
(1, 1, 0, 0, 0)
);
}
#[tokio::test]
async fn quota_refresh_service_should_skip_recent_locked_account_inside_cooldown_grace() {
let server = MockServer::start().await;
let dir = tempfile::tempdir().expect("temp dir");
let db = dir
.path()
.join("quota-refresh-recent-cooldown-grace.sqlite");
let pool = connect_sqlite(&format!("sqlite://{}", db.display()))
.await
.expect("sqlite pool");
let store = SqliteAccountStore::new(pool.clone());
insert_quota_locked_account(&store, &pool, "acct-quota", "access-token").await;
set_quota_cooldown_until(&pool, "acct-quota", Utc::now() - Duration::seconds(20)).await;
let codex = CodexBackendClient::new(
reqwest::Client::new(),
server.uri(),
crate::support::fingerprint::test_fingerprint(),
);
let service =
RuntimeQuotaRefreshService::with_min_refresh_interval_secs(store, Arc::new(codex), 1_800);
let mut last_refreshed = HashMap::new();
last_refreshed.insert("acct-quota".to_string(), tokio::time::Instant::now());
let summary = service
.refresh_locked_accounts(&mut last_refreshed)
.await
.expect("quota refresh should succeed");
assert_eq!(
(
summary.candidates,
summary.skipped_cooldown,
summary.skipped_recent,
summary.refreshed,
usage_request_count(&server).await,
),
(1, 1, 0, 0, 0)
);
}
#[tokio::test]
async fn quota_refresh_service_should_bypass_recent_skip_after_cooldown_grace() {
let server = MockServer::start().await;
Mock::given(method("GET"))
.and(path("/api/codex/usage"))
.respond_with(ResponseTemplate::new(200).set_body_json(json!({
"plan_type": "plus",
"rate_limit": {
"allowed": true,
"limit_reached": false,
"primary_window": {
"used_percent": 8,
"reset_at": 1_800_000_000,
"limit_window_seconds": 18_000
}
}
})))
.expect(1)
.mount(&server)
.await;
let dir = tempfile::tempdir().expect("temp dir");
let db = dir
.path()
.join("quota-refresh-recent-expired-cooldown.sqlite");
let pool = connect_sqlite(&format!("sqlite://{}", db.display()))
.await
.expect("sqlite pool");
let store = SqliteAccountStore::new(pool.clone());
insert_quota_locked_account(&store, &pool, "acct-quota", "access-token").await;
set_quota_cooldown_until(&pool, "acct-quota", Utc::now() - Duration::seconds(31)).await;
let codex = CodexBackendClient::new(
reqwest::Client::new(),
server.uri(),
crate::support::fingerprint::test_fingerprint(),
);
let service =
RuntimeQuotaRefreshService::with_min_refresh_interval_secs(store, Arc::new(codex), 1_800);
let mut last_refreshed = HashMap::new();
last_refreshed.insert("acct-quota".to_string(), tokio::time::Instant::now());
let summary = service
.refresh_locked_accounts(&mut last_refreshed)
.await
.expect("quota refresh should succeed");
assert_eq!(
(
summary.skipped_recent,
summary.skipped_cooldown,
summary.refreshed,
usage_request_count(&server).await,
),
(0, 0, 1, 1)
);
}
#[tokio::test]
async fn quota_refresh_service_should_stagger_multiple_locked_account_requests() {
let server = MockServer::start().await;
@@ -573,6 +712,19 @@ async fn insert_quota_locked_account(
.expect("quota lock should be set");
}
async fn set_quota_cooldown_until(
pool: &sqlx::SqlitePool,
id: &str,
cooldown_until: chrono::DateTime<Utc>,
) {
sqlx::query("update accounts set quota_cooldown_until = ? where id = ?")
.bind(cooldown_until.to_rfc3339())
.bind(id)
.execute(pool)
.await
.expect("quota cooldown should be set");
}
async fn wait_for_usage_requests(server: &MockServer, expected_count: usize) {
let deadline = tokio::time::Instant::now() + StdDuration::from_secs(2);
loop {
@@ -11,7 +11,6 @@ fn codex_websocket_payload_audit_snapshot_should_redact_user_content() {
})],
);
request.set_previous_response_id(Some("resp_secret".to_string()));
request.set_reasoning(Some(json!({"effort": "medium"})));
request.set_service_tier(Some("flex".to_string()));
request.set_prompt_cache_key(Some("cache-secret".to_string()));
request.set_client_metadata(Some(json!({
@@ -33,7 +32,6 @@ fn codex_websocket_payload_audit_snapshot_should_redact_user_content() {
"stream",
"store",
"previous_response_id",
"reasoning",
"service_tier",
"prompt_cache_key",
"client_metadata",
+1
View File
@@ -246,6 +246,7 @@ pub(crate) fn websocket_pool_config_for_tests(
ping_interval,
ping_timeout: Duration::from_secs(1),
liveness_timeout,
first_token_timeout: None,
}
}
+85 -8
View File
@@ -1024,19 +1024,15 @@ async fn codex_backend_client_stream_should_reject_binary_websocket_event() {
crate::support::fingerprint::test_fingerprint(),
);
let mut response = backend
let Err(error) = backend
.create_response_stream(
&request,
request_context("req_stream_binary", Some("chatgpt-account")),
)
.await
.expect("websocket stream should open");
let error = response
.body
.next()
.await
.expect("stream should yield binary error")
.expect_err("binary event should be rejected");
else {
panic!("binary event before first content should reject stream open");
};
server.await.unwrap();
assert!(matches!(
@@ -1045,6 +1041,87 @@ async fn codex_backend_client_stream_should_reject_binary_websocket_event() {
));
}
#[tokio::test]
async fn codex_backend_client_stream_should_retry_without_pool_when_first_content_times_out() {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
let accepted_connections = Arc::new(AtomicUsize::new(0));
let accepted_connections_for_server = Arc::clone(&accepted_connections);
let server = tokio::spawn(async move {
let (first_stream, _) = listener.accept().await.unwrap();
accepted_connections_for_server.fetch_add(1, Ordering::SeqCst);
let mut first_websocket = accept_codex_test_websocket(first_stream).await;
let _first_message = first_websocket.next().await.unwrap().unwrap();
first_websocket
.send(Message::Text(
json!({
"type": "response.created",
"response": {
"id": "resp_no_pool_first_token_stalled",
"object": "response"
}
})
.to_string()
.into(),
))
.await
.unwrap();
let (second_stream, _) = listener.accept().await.unwrap();
accepted_connections_for_server.fetch_add(1, Ordering::SeqCst);
let mut second_websocket = accept_codex_test_websocket(second_stream).await;
let _second_message = second_websocket.next().await.unwrap().unwrap();
second_websocket
.send(Message::Text(
json!({
"type": "response.output_text.delta",
"delta": "no-pool retry recovered"
})
.to_string()
.into(),
))
.await
.unwrap();
second_websocket
.send(Message::Text(
completed_websocket_response("resp_no_pool_retry_recovered", 3, 1).into(),
))
.await
.unwrap();
second_websocket.close(None).await.unwrap();
drop(first_websocket);
});
let backend = CodexBackendClient::new(
reqwest::Client::builder().no_proxy().build().unwrap(),
format!("http://{addr}"),
crate::support::fingerprint::test_fingerprint(),
)
.with_websocket_first_token_timeout(Some(Duration::from_millis(30)));
let request = pooled_websocket_request("conversation-first-token-no-pool-timeout");
let response = backend
.create_response_stream(
&request,
request_context("req_first_token_no_pool_timeout", Some("chatgpt-account")),
)
.await
.expect("no-pool first-token timeout should retry before returning stream");
let decision = response.websocket_pool_decision;
let mut stream = response.body;
let mut body = String::new();
while let Some(chunk) = stream.next().await {
let chunk = chunk.expect("retried no-pool stream chunk should be valid");
body.push_str(std::str::from_utf8(&chunk).unwrap());
}
server.await.unwrap();
assert!(body.contains("no-pool retry recovered"));
assert!(body.contains("resp_no_pool_retry_recovered"));
assert!(!body.contains("resp_no_pool_first_token_stalled"));
assert!(decision.is_none());
assert_eq!(accepted_connections.load(Ordering::SeqCst), 2);
}
#[tokio::test]
async fn codex_backend_client_stream_should_error_when_websocket_closes_before_terminal() {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
@@ -155,6 +155,280 @@ async fn codex_backend_client_should_open_fresh_socket_when_idle_pooled_websocke
assert_eq!(accepted_connections.load(Ordering::SeqCst), 2);
}
#[tokio::test]
async fn codex_backend_client_stream_should_retry_fresh_socket_when_first_content_times_out() {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
let accepted_connections = Arc::new(AtomicUsize::new(0));
let accepted_connections_for_server = Arc::clone(&accepted_connections);
let server = tokio::spawn(async move {
let (first_stream, _) = listener.accept().await.unwrap();
accepted_connections_for_server.fetch_add(1, Ordering::SeqCst);
let mut first_websocket = accept_codex_test_websocket(first_stream).await;
let _first_message = first_websocket.next().await.unwrap().unwrap();
first_websocket
.send(Message::Text(
json!({
"type": "response.created",
"response": {
"id": "resp_first_token_fresh_stalled",
"object": "response"
}
})
.to_string()
.into(),
))
.await
.unwrap();
let (second_stream, _) = listener.accept().await.unwrap();
accepted_connections_for_server.fetch_add(1, Ordering::SeqCst);
let mut second_websocket = accept_codex_test_websocket(second_stream).await;
let _second_message = second_websocket.next().await.unwrap().unwrap();
second_websocket
.send(Message::Text(
json!({
"type": "response.output_text.delta",
"delta": "fresh retry recovered"
})
.to_string()
.into(),
))
.await
.unwrap();
second_websocket
.send(Message::Text(
completed_websocket_response("resp_first_token_fresh_recovered", 3, 1).into(),
))
.await
.unwrap();
second_websocket.close(None).await.unwrap();
drop(first_websocket);
});
let pool = Arc::new(CodexWebSocketPool::with_config(CodexWebSocketPoolConfig {
first_token_timeout: Some(Duration::from_millis(30)),
..websocket_pool_config_for_tests(None, None, None)
}));
let backend = CodexBackendClient::new(
reqwest::Client::builder().no_proxy().build().unwrap(),
format!("http://{addr}"),
crate::support::fingerprint::test_fingerprint(),
)
.with_websocket_pool(Arc::clone(&pool));
let request = pooled_websocket_request("conversation-first-token-fresh-timeout");
let response = backend
.create_response_stream(
&request,
request_context("req_first_token_fresh_timeout", Some("chatgpt-account")),
)
.await
.expect("fresh first-token timeout should retry before returning stream");
let decision = response.websocket_pool_decision;
let mut stream = response.body;
let mut body = String::new();
while let Some(chunk) = stream.next().await {
let chunk = chunk.expect("retried stream chunk should be valid");
body.push_str(std::str::from_utf8(&chunk).unwrap());
}
server.await.unwrap();
assert!(body.contains("fresh retry recovered"));
assert!(body.contains("resp_first_token_fresh_recovered"));
assert!(!body.contains("resp_first_token_fresh_stalled"));
assert_eq!(decision.unwrap().kind(), "new");
assert_eq!(accepted_connections.load(Ordering::SeqCst), 2);
}
#[tokio::test]
async fn codex_backend_client_stream_should_retry_reused_socket_when_first_content_times_out() {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
let accepted_connections = Arc::new(AtomicUsize::new(0));
let accepted_connections_for_server = Arc::clone(&accepted_connections);
let server = tokio::spawn(async move {
let (first_stream, _) = listener.accept().await.unwrap();
accepted_connections_for_server.fetch_add(1, Ordering::SeqCst);
let mut first_websocket = accept_codex_test_websocket(first_stream).await;
let _seed_message = first_websocket.next().await.unwrap().unwrap();
first_websocket
.send(Message::Text(
completed_websocket_response("resp_first_token_reuse_seed", 2, 1).into(),
))
.await
.unwrap();
let _reused_message = first_websocket.next().await.unwrap().unwrap();
first_websocket
.send(Message::Text(
json!({
"type": "response.in_progress",
"response": {
"id": "resp_first_token_reuse_stalled",
"object": "response"
}
})
.to_string()
.into(),
))
.await
.unwrap();
let (second_stream, _) = listener.accept().await.unwrap();
accepted_connections_for_server.fetch_add(1, Ordering::SeqCst);
let mut second_websocket = accept_codex_test_websocket(second_stream).await;
let _retry_message = second_websocket.next().await.unwrap().unwrap();
second_websocket
.send(Message::Text(
json!({
"type": "response.output_text.delta",
"delta": "reuse retry recovered"
})
.to_string()
.into(),
))
.await
.unwrap();
second_websocket
.send(Message::Text(
completed_websocket_response("resp_first_token_reuse_recovered", 3, 1).into(),
))
.await
.unwrap();
second_websocket.close(None).await.unwrap();
drop(first_websocket);
});
let pool = Arc::new(CodexWebSocketPool::with_config(CodexWebSocketPoolConfig {
first_token_timeout: Some(Duration::from_millis(30)),
..websocket_pool_config_for_tests(None, None, None)
}));
let backend = CodexBackendClient::new(
reqwest::Client::builder().no_proxy().build().unwrap(),
format!("http://{addr}"),
crate::support::fingerprint::test_fingerprint(),
)
.with_websocket_pool(Arc::clone(&pool));
let request = pooled_websocket_request("conversation-first-token-reuse-timeout");
let seed = backend
.create_response(
&request,
request_context("req_first_token_reuse_seed", Some("chatgpt-account")),
)
.await
.expect("seed response should populate pool");
let response = backend
.create_response_stream(
&request,
request_context("req_first_token_reuse_timeout", Some("chatgpt-account")),
)
.await
.expect("reused first-token timeout should retry before returning stream");
let decision = response.websocket_pool_decision;
let mut stream = response.body;
let mut body = String::new();
while let Some(chunk) = stream.next().await {
let chunk = chunk.expect("retried reused stream chunk should be valid");
body.push_str(std::str::from_utf8(&chunk).unwrap());
}
server.await.unwrap();
assert!(seed.body.contains("resp_first_token_reuse_seed"));
assert!(body.contains("reuse retry recovered"));
assert!(body.contains("resp_first_token_reuse_recovered"));
assert!(!body.contains("resp_first_token_reuse_stalled"));
assert_eq!(decision.unwrap().kind(), "retry_after_stale_reuse");
assert_eq!(accepted_connections.load(Ordering::SeqCst), 2);
}
#[tokio::test]
async fn codex_backend_client_stream_should_retry_bypass_socket_when_first_content_times_out() {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
let addr = listener.local_addr().unwrap();
let accepted_connections = Arc::new(AtomicUsize::new(0));
let accepted_connections_for_server = Arc::clone(&accepted_connections);
let server = tokio::spawn(async move {
let (first_stream, _) = listener.accept().await.unwrap();
accepted_connections_for_server.fetch_add(1, Ordering::SeqCst);
let mut first_websocket = accept_codex_test_websocket(first_stream).await;
let _first_message = first_websocket.next().await.unwrap().unwrap();
first_websocket
.send(Message::Text(
json!({
"type": "response.created",
"response": {
"id": "resp_bypass_first_token_stalled",
"object": "response"
}
})
.to_string()
.into(),
))
.await
.unwrap();
let (second_stream, _) = listener.accept().await.unwrap();
accepted_connections_for_server.fetch_add(1, Ordering::SeqCst);
let mut second_websocket = accept_codex_test_websocket(second_stream).await;
let _second_message = second_websocket.next().await.unwrap().unwrap();
second_websocket
.send(Message::Text(
json!({
"type": "response.output_text.delta",
"delta": "bypass retry recovered"
})
.to_string()
.into(),
))
.await
.unwrap();
second_websocket
.send(Message::Text(
completed_websocket_response("resp_bypass_retry_recovered", 3, 1).into(),
))
.await
.unwrap();
second_websocket.close(None).await.unwrap();
drop(first_websocket);
});
let pool = Arc::new(CodexWebSocketPool::with_config(CodexWebSocketPoolConfig {
max_per_account: 0,
first_token_timeout: Some(Duration::from_millis(30)),
..websocket_pool_config_for_tests(None, None, None)
}));
let backend = CodexBackendClient::new(
reqwest::Client::builder().no_proxy().build().unwrap(),
format!("http://{addr}"),
crate::support::fingerprint::test_fingerprint(),
)
.with_websocket_pool(Arc::clone(&pool));
let request = pooled_websocket_request("conversation-first-token-bypass-timeout");
let response = backend
.create_response_stream(
&request,
request_context("req_first_token_bypass_timeout", Some("chatgpt-account")),
)
.await
.expect("bypass first-token timeout should retry before returning stream");
let decision = response.websocket_pool_decision;
let mut stream = response.body;
let mut body = String::new();
while let Some(chunk) = stream.next().await {
let chunk = chunk.expect("retried bypass stream chunk should be valid");
body.push_str(std::str::from_utf8(&chunk).unwrap());
}
server.await.unwrap();
assert!(body.contains("bypass retry recovered"));
assert!(body.contains("resp_bypass_retry_recovered"));
assert!(!body.contains("resp_bypass_first_token_stalled"));
let decision = decision.expect("bypass decision");
assert_eq!(decision.kind(), "bypass");
assert_eq!(decision.reason(), Some("cap"));
assert_eq!(accepted_connections.load(Ordering::SeqCst), 2);
}
#[tokio::test]
async fn codex_backend_client_should_not_reuse_pooled_websocket_across_local_accounts() {
let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
+11 -10
View File
@@ -1,6 +1,6 @@
# Deploy
# 部署
Docker 部署入口集中在本目录。
本目录包含 Dockerfile、Compose 文件和 Docker 配置模板。
## Compose
@@ -11,16 +11,17 @@ mkdir -p .runtime/data .runtime/logs
cp deploy/config.example.yaml .runtime/config.yaml
```
`deploy/config.example.yaml` 是 Docker 部署模板。Compose 默认把配置、数据和日志都放在仓库 `.runtime/` 下:
`deploy/config.example.yaml` 是 Docker 部署模板。Compose 默认使用仓库根目录的 `.runtime/`:
- `.runtime/config.yaml` -> `/app/config.yaml`
- `.runtime/data` -> `/app/data`
- `.runtime/logs` -> `/app/logs`
复制后必须设置 `admin.default_password`,空值或常见弱口令会被后端拒绝启动。
如果前面还有 Nginx、Caddy、Cloudflare Tunnel 等反向代理,建议把后端实际看到的代理 peer IP 或 CIDR 写入 `server.trusted_proxies`。Docker 端口映射经宿主反代访问时,这个 peer 通常是 Compose 网络网关,例如 `172.18.0.1/32`,不是宿主 `127.0.0.1/32`。
复制后必须设置 `admin.default_password`。空值、长度不足 12 位或常见弱口令会被后端拒绝。
`server.trusted_proxies` 非空时,只有这些 peer 的 `CF-Connecting-IP` / `X-Real-IP` / `X-Forwarded-For` 会被采信;为空时进入兼容自动模式,按同一组头解析真实 IP。
如果前面还有 Nginx、Caddy、Cloudflare Tunnel 等反向代理,把后端实际看到的代理 peer IP 或 CIDR 写入 `server.trusted_proxies`。经宿主机反代访问 Docker 端口时,这个 peer 通常是 Compose 网络网关,例如 `172.18.0.1/32`,不是宿主 `127.0.0.1/32`。
`server.trusted_proxies` 非空时,只有这些 peer 的 `CF-Connecting-IP` / `X-Real-IP` / `X-Forwarded-For` 会被采信;为空时进入自动解析模式,按同一组头解析真实 IP。
构建并启动:
@@ -29,7 +30,7 @@ docker compose -f deploy/docker-compose.yml build
docker compose -f deploy/docker-compose.yml up -d
```
可以通过环境变量覆盖宿主机路径;容器内会通过 `CPR_CONFIG_FILE=/app/config.yaml` 显式读取挂载后的配置:
可以通过环境变量覆盖宿主机路径。容器内始终通过 `CPR_CONFIG_FILE=/app/config.yaml` 读取挂载后的配置:
```bash
CPR_CONFIG_FILE=/path/to/config.yaml \
@@ -38,7 +39,7 @@ CPR_LOG_DIR=/path/to/logs \
docker compose -f deploy/docker-compose.yml up -d
```
## Build
## 构建
Docker 构建上下文保持仓库根目录:
@@ -46,13 +47,13 @@ Docker 构建上下文保持仓库根目录:
docker build -f deploy/Dockerfile -t codex-proxy-rs:latest .
```
也可以通过 Compose 构建:
也可以用 Compose 构建:
```bash
docker compose -f deploy/docker-compose.yml build
```
需要注入版本信息时:
需要写入版本元数据时:
```bash
CPR_VERSION="$(ruby -ryaml -e 'puts YAML.load_file("release/version.yaml").fetch("version").delete_prefix("v")')" \
+1 -1
View File
@@ -6,7 +6,7 @@ server:
# 可信反向代理 IP 或 CIDR。非空时只有这些 peer 的转发 IP 头会被采信。
# Docker 端口映射经宿主反代访问时,后端看到的 peer 通常是 Compose 网络网关
# (例如 172.18.0.1),不是宿主 127.0.0.1。
# 为空时进入兼容自动模式,按 CF-Connecting-IP / X-Real-IP / X-Forwarded-For 解析。
# 为空时进入自动解析模式,按 CF-Connecting-IP / X-Real-IP / X-Forwarded-For 解析。
trusted_proxies: []
api:
+12 -12
View File
@@ -1,17 +1,17 @@
# Codex Proxy RS 架构说明
# Codex Proxy RS 架构
## 总览
Codex Proxy RS 是一个单主进程应用。Rust 后端同时承担 OpenAI 兼容代理、管理端 API、SQLite 持久化、后台任务、静态前端托管和在线更新;Vue 管理端作为 SPA 由同一个后端进程服务。
Codex Proxy RS 是单进程应用。Rust 后端同时处理 Responses 兼容代理、管理端 API、SQLite 持久化、后台任务、静态前端托管和在线更新;Vue 管理端作为 SPA 由同一个进程托管。
```mermaid
flowchart LR
Client["客户端 / OpenAI SDK"] -->|"Authorization: Bearer<br/>/v1/*"| Http["Axum Router"]
Client["客户端 / OpenAI SDK"] -->|"Authorization: Bearer<br/>/v1/responses"| Http["Axum Router"]
Browser["浏览器"] -->|"管理端 SPA"| Web["web/dist"]
Browser -->|"/api/admin/*"| Http
subgraph App["codex-proxy-rs"]
Http --> Proxy["proxy<br/>OpenAI 兼容入口<br/>客户端 API Key 鉴权"]
Http --> Proxy["proxy<br/>Responses / Models 入口<br/>客户端 API Key 鉴权"]
Http --> Admin["admin<br/>账号 / Key / 设置 / 监控 / 系统更新"]
Http --> Web
@@ -36,11 +36,11 @@ flowchart LR
- `backend/`:Rust/Axum 后端、SQLite migration、构建脚本和集成测试。
- `frontend/`:Vue 3 管理端,使用 Vite 8、Tailwind v4、Pinia、Vue Router、ECharts 和 lucide 图标。
- `deploy/`:Dockerfile、Compose 和 Docker 部署配置模板。
- `docs/`:长期维护文档和审计记录,只记录当前有效结构与决策。
- `docs/`:长期维护文档,记录当前有效结构与决策。
- `release/`:版本号、平台矩阵和发布脚本。
- `skills/`:项目本地 Codex skill。
当前后端是 `backend/` 单 crate。不要在文档里保留旧迁移布局或兼容性重导出约束,除非代码重新引入对应结构。
当前后端是 `backend/` 单 crate。历史迁移布局不再作为当前架构的一部分记录。
## 后端结构
@@ -54,7 +54,7 @@ flowchart LR
主要模块:
- `admin/`:管理端 API,包括登录 session、账号、API Key、监控、运行设置和系统更新。
- `proxy/`:OpenAI 兼容 HTTP 入口、客户端 API Key 鉴权、请求分派。
- `proxy/`:Responses/Models HTTP 入口、客户端 API Key 鉴权、请求分派。
- `proxy/openai/`:Responses 和 Models 的协议入口。
- `proxy/dispatch/`:账号选择、quota/限流处理、错误映射、session affinity 和失败回退。
- `upstream/`:ChatGPT/Codex 上游账号、模型、协议、传输、token 刷新和 fingerprint。
@@ -103,14 +103,14 @@ Docker 也使用宿主机 `.runtime`,但容器内配置路径仍然写 `/app/d
## 代理请求链路
客户端请求进入 `/v1/*`:
客户端请求进入 `/v1/responses`、`/v1/responses/review`、`/v1/responses/compact` 或 `/v1/models*`:
1. `proxy/auth` 校验客户端 API Key,并记录调用上下文。
2. `proxy/openai` 解析 OpenAI 兼容请求。
2. `proxy/openai` 解析 Responses 或 Models 请求。
3. `proxy/dispatch` 按模型、quota、账号状态和选择策略挑选账号。
4. `upstream/protocol` 转换为 ChatGPT/Codex 上游协议。
5. `upstream/transport` 使用 HTTP、SSE 或 WebSocket 访问上游。
6. 响应和错误被转换回 OpenAI 兼容格式。
6. 响应和错误被转换回 OpenAI Responses/Models 兼容格式。
7. 用量、模型维度统计、请求记录、账号窗口和失败状态写回 SQLite/账号池。
账号状态由 SQLite 恢复到内存账号池,并在请求、token 刷新、quota 刷新、Cloudflare 处理和失败回退时同步更新。
@@ -132,7 +132,7 @@ Docker 也使用宿主机 `.runtime`,但容器内配置路径仍然写 `/app/d
- 账号管理:导入、OAuth、连接测试、额度、模型、调度状态、刷新能力。
- API Key 管理:客户端 Key 和管理员 API Key。
- Dashboard:请求趋势、账号概览、服务状态。
- Dashboard:核心指标、请求趋势、账号概览、服务状态和健康时间线。
- 请求明细:网关调用、上游状态、Token 消耗、错误和详情。
- 设置:运行参数、模型映射、账号选择策略。
- 系统更新:版本、检查更新、SSE 更新日志、更新、重启、回滚。
@@ -148,7 +148,7 @@ Docker 也使用宿主机 `.runtime`,但容器内配置路径仍然写 `/app/d
发布命令:
```bash
release/publish 0.1.5
release/publish 1.0.4
```
发布 workflow 校验触发 tag 必须等于 `v` + `release/version.yaml`。构建时注入:
@@ -4,9 +4,15 @@ import { computed } from 'vue'
import AccountPlanBadge from './AccountPlanBadge.vue'
type AccountIdentity = {
id?: string | number | null
email: string
planType?: string | null
}
const props = withDefaults(
defineProps<{
account: any
account: AccountIdentity
size?: 'md' | 'lg'
showPlan?: boolean
}>(),
@@ -16,24 +22,11 @@ const props = withDefaults(
},
)
const displayTitle = computed(
() =>
props.account.label?.trim() ||
props.account.email ||
props.account.accountId ||
props.account.id,
)
const emailText = computed(() => props.account.email.trim())
const secondaryText = computed(() => {
const title = displayTitle.value
const secondary = [
props.account.email,
props.account.accountId,
props.account.userId,
props.account.id,
].find((value) => value && value !== title)
return secondary || props.account.id
})
const displayTitle = computed(() => emailText.value.split('@')[0])
const secondaryText = emailText
const initial = computed(() => displayTitle.value.slice(0, 1).toUpperCase())
+2 -2
View File
@@ -156,13 +156,13 @@ bindAccountLoader(loadAccounts)
<div class="flex shrink-0 items-center gap-2">
<BaseButton
v-if="selectedIds.size > 0"
variant="default"
:loading="exportingAccounts"
:disabled="accounts.length === 0"
@click="handleExportAccounts"
>
<Download class="size-4" />
{{ selectedIds.size > 0 ? '导出选中' : '导出' }}
导出选中 ({{ selectedIds.size }})
</BaseButton>
<BaseButton variant="primary" @click="openCreateAccount">
<Plus class="size-4" />
@@ -1,24 +1,7 @@
<script setup lang="ts">
import { computed } from 'vue'
const props = defineProps<{
defineProps<{
apiKey: any
editing: boolean
editingValue: string
saving: boolean
}>()
const emit = defineEmits<{
startEdit: [apiKey: any]
updateEdit: [value: string]
submitEdit: [apiKeyId: string]
cancelEdit: []
}>()
const editValue = computed({
get: () => props.editingValue,
set: (value: string) => emit('updateEdit', value),
})
</script>
<template>
@@ -26,36 +9,8 @@ const editValue = computed({
<span class="text-[13px] font-bold text-(--cp-text-primary)">
{{ apiKey.name }}
</span>
<div v-if="editing" class="flex items-center gap-2">
<input
v-model="editValue"
type="text"
class="h-(--cp-input-height-inline) min-w-34 rounded-(--cp-input-radius-small) border-0 bg-(--cp-input-soft-bg) px-2.5 text-[13px] font-[650] text-(--cp-text-primary) shadow-(--cp-shadow-input) outline-none transition placeholder:text-(--cp-text-muted) focus:bg-(--cp-input-soft-bg-focus) focus:shadow-(--cp-shadow-input-focus)"
:disabled="saving"
@keyup.enter="emit('submitEdit', apiKey.id)"
@keyup.escape="emit('cancelEdit')"
/>
<button
class="text-[12px] text-(--cp-accent-primary) hover:underline disabled:cursor-not-allowed disabled:text-(--cp-disabled-text) disabled:no-underline"
:disabled="saving"
@click="emit('submitEdit', apiKey.id)"
>
{{ saving ? '保存中' : '保存' }}
</button>
<button
class="text-[12px] text-(--cp-text-tertiary) hover:underline disabled:cursor-not-allowed disabled:text-(--cp-disabled-text) disabled:no-underline"
:disabled="saving"
@click="emit('cancelEdit')"
>
取消
</button>
</div>
<button
v-else-if="apiKey.label"
class="text-left text-[12px] font-[650] text-(--cp-text-tertiary) hover:text-(--cp-info-text)"
@click="emit('startEdit', apiKey)"
>
<span v-if="apiKey.label" class="text-[12px] font-[650] text-(--cp-text-tertiary)">
{{ apiKey.label }}
</button>
</span>
</div>
</template>
@@ -16,18 +16,15 @@ export function useApiKeyMutations(selectedIds: Ref<Set<string>>) {
const showSingleDeleteModal = ref(false)
const showKeyModal = ref(false)
const createdKey = ref('')
const editingLabel = ref<{ id: string; value: string } | null>(null)
const pendingDeleteKey = ref<any | null>(null)
const creatingKeyAction = useAsyncAction()
const deletingKeyAction = useAsyncAction()
const batchDeletingAction = useAsyncAction()
const updatingStatusKeys = useIdSet<string>()
const savingLabelKeys = useIdSet<string>()
const creatingKey = creatingKeyAction.loading
const deletingKey = deletingKeyAction.loading
const batchDeleting = batchDeletingAction.loading
const updatingStatusKeyIds = updatingStatusKeys.ids
const savingLabelKeyIds = savingLabelKeys.ids
const createForm = ref({
name: '',
@@ -124,40 +121,6 @@ export function useApiKeyMutations(selectedIds: Ref<Set<string>>) {
})
}
async function handleUpdateLabel(keyId: string, label: string) {
await savingLabelKeys.run(keyId, async () => {
try {
await updateApiKey({ id: keyId, label: label || null })
editingLabel.value = null
await loadApiKeys()
toast.success('标签已更新')
} catch (error: any) {
toast.error(error.message || '标签更新失败')
}
})
}
function startEditLabel(key: any) {
editingLabel.value = { id: key.id, value: key.label || '' }
}
function cancelEditLabel() {
editingLabel.value = null
}
function currentEditingLabelValue() {
return editingLabel.value?.value ?? ''
}
function submitEditingLabel(keyId: string) {
void handleUpdateLabel(keyId, currentEditingLabelValue())
}
function updateEditingLabelValue(value: string) {
if (!editingLabel.value) return
editingLabel.value.value = value
}
async function copyToClipboard(text: string) {
if (!text) {
toast.error('复制失败')
@@ -188,24 +151,17 @@ export function useApiKeyMutations(selectedIds: Ref<Set<string>>) {
showSingleDeleteModal,
showKeyModal,
createdKey,
editingLabel,
pendingDeleteKey,
creatingKey,
deletingKey,
batchDeleting,
updatingStatusKeyIds,
savingLabelKeyIds,
createForm,
handleCreate,
requestDeleteKey,
handleDelete,
handleBatchDelete,
handleToggleStatus,
startEditLabel,
cancelEditLabel,
currentEditingLabelValue,
submitEditingLabel,
updateEditingLabelValue,
copyToClipboard,
maskKey,
}
+1 -17
View File
@@ -30,24 +30,17 @@ const {
showSingleDeleteModal,
showKeyModal,
createdKey,
editingLabel,
pendingDeleteKey,
creatingKey,
deletingKey,
batchDeleting,
updatingStatusKeyIds,
savingLabelKeyIds,
createForm,
handleCreate,
requestDeleteKey,
handleDelete,
handleBatchDelete,
handleToggleStatus,
startEditLabel,
cancelEditLabel,
currentEditingLabelValue,
submitEditingLabel,
updateEditingLabelValue,
copyToClipboard,
maskKey,
} = useApiKeyMutations(selectedIds)
@@ -158,16 +151,7 @@ function importToCcs(apiKey: any) {
</template>
<template #identity="{ row }">
<ApiKeyIdentityCell
:api-key="row"
:editing="editingLabel?.id === row.id"
:editing-value="currentEditingLabelValue()"
:saving="savingLabelKeyIds.has(row.id)"
@cancel-edit="cancelEditLabel"
@start-edit="startEditLabel"
@submit-edit="submitEditingLabel"
@update-edit="updateEditingLabelValue"
/>
<ApiKeyIdentityCell :api-key="row" />
</template>
<template #prefix="{ row }">
@@ -5,7 +5,7 @@ import { CircleCheck, RefreshCw, ShieldAlert, TriangleAlert } from '@lucide/vue'
import BaseCard from '../../../components/base/BaseCard.vue'
import BaseEmpty from '../../../components/base/BaseEmpty.vue'
import AccountPlanBadge from '../../accounts/components/AccountPlanBadge.vue'
import AccountIdentityCell from '../../accounts/components/AccountIdentityCell.vue'
const props = defineProps<{
accounts: any[]
@@ -145,46 +145,52 @@ const quotaToneClasses: Record<string, string> = {
容量、并发与分配策略
</p>
<div class="mt-7.75 grid h-30.5 content-between rounded-[14px] bg-(--cp-bg-subtle) p-4">
<span class="block h-3.5 text-xs leading-[1.15] font-[650] text-(--cp-text-secondary)"
>槽位占用</span
>
<div>
<div class="grid h-8.5 grid-cols-[minmax(0,1fr)_auto] items-start gap-4">
<strong
class="font-mono text-[32px] leading-[1.05] font-[760] tabular-nums text-(--cp-text-primary)"
>{{ usedRatio }}</strong
>
<span class="mt-3.5 text-xs leading-[1.15] font-[650] text-(--cp-text-secondary)"
>{{ usedPercent }}% 已占用</span
>
</div>
<div class="mt-4 h-2.5 w-full overflow-hidden rounded-full bg-(--cp-progress-track)">
<i
class="block h-2.5 rounded-full bg-(--cp-success)"
:style="{ width: `${usedPercent}%` }"
/>
</div>
</div>
</div>
<div class="mt-4 grid h-22.5 grid-cols-3 gap-4 rounded-[14px] bg-(--cp-bg-subtle) p-4">
<div v-for="stat in scheduleStats" :key="stat.label" class="grid content-between">
<span class="text-xs leading-[1.15] font-[650] text-(--cp-text-secondary)">{{
stat.label
}}</span>
<strong
class="block font-mono text-[21px] leading-[1.1] font-[760] tabular-nums text-(--cp-text-primary)"
>{{ stat.value }}</strong
<div class="mt-6.75 grid gap-4 xl:h-82.5 xl:grid-rows-[122px_90px_minmax(0,1fr)]">
<div class="grid h-30.5 content-between rounded-[14px] bg-(--cp-bg-subtle) p-4 xl:h-auto">
<span class="block h-3.5 text-xs leading-[1.15] font-[650] text-(--cp-text-secondary)"
>槽位占用</span
>
<div>
<div class="grid h-8.5 grid-cols-[minmax(0,1fr)_auto] items-start gap-4">
<strong
class="font-mono text-[32px] leading-[1.05] font-[760] tabular-nums text-(--cp-text-primary)"
>{{ usedRatio }}</strong
>
<span class="mt-3.5 text-xs leading-[1.15] font-[650] text-(--cp-text-secondary)"
>{{ usedPercent }}% 已占用</span
>
</div>
<div class="mt-4 h-2.5 w-full overflow-hidden rounded-full bg-(--cp-progress-track)">
<i
class="block h-2.5 rounded-full bg-(--cp-success)"
:style="{ width: `${usedPercent}%` }"
/>
</div>
</div>
</div>
</div>
<div class="mt-4 grid h-20.5 content-between rounded-[14px] bg-(--cp-bg-subtle) p-4">
<span class="text-xs leading-[1.15] font-[650] text-(--cp-text-secondary)">分配策略</span>
<strong class="block text-[17px] leading-[1.15] font-[650] text-(--cp-text-primary)">{{
strategyLabel
}}</strong>
<div
class="grid h-22.5 grid-cols-3 gap-4 rounded-[14px] bg-(--cp-bg-subtle) p-4 xl:h-auto"
>
<div v-for="stat in scheduleStats" :key="stat.label" class="grid content-between">
<span class="text-xs leading-[1.15] font-[650] text-(--cp-text-secondary)">{{
stat.label
}}</span>
<strong
class="block font-mono text-[21px] leading-[1.1] font-[760] tabular-nums text-(--cp-text-primary)"
>{{ stat.value }}</strong
>
</div>
</div>
<div class="grid h-20.5 content-between rounded-[14px] bg-(--cp-bg-subtle) p-4 xl:h-auto">
<span class="text-xs leading-[1.15] font-[650] text-(--cp-text-secondary)"
>分配策略</span
>
<strong class="block text-[17px] leading-[1.15] font-[650] text-(--cp-text-primary)">{{
strategyLabel
}}</strong>
</div>
</div>
</section>
@@ -201,72 +207,52 @@ const quotaToneClasses: Record<string, string> = {
v-if="accounts.length === 0"
compact
title="暂无账号用量"
description="账号产生请求后会显示 24h 排序和 Token 负载。"
description="账号产生请求后会显示最近使用排序和 Token 负载。"
class="min-h-40 place-content-center xl:h-full"
/>
<template v-else>
<article
v-for="account in accounts"
:key="account.name"
class="grid w-full shrink-0 grid-cols-[34px_minmax(0,1fr)] gap-x-3 rounded-[14px] bg-(--cp-bg-subtle) px-3.5 py-3.5 hover:bg-(--cp-bg-muted) xl:h-19.5 xl:grid-cols-[34px_minmax(0,1.28fr)_minmax(132px,0.86fr)] xl:items-center xl:gap-x-3 xl:py-0"
:key="account.id"
class="grid w-full shrink-0 grid-cols-1 gap-3 rounded-xl bg-(--cp-bg-subtle) px-3.5 py-3.5 transition-colors hover:bg-(--cp-bg-muted) xl:h-19.5 xl:grid-cols-[minmax(0,1fr)_minmax(70px,0.36fr)_minmax(74px,0.38fr)_minmax(82px,0.46fr)] xl:items-center xl:gap-4 xl:py-0"
>
<AccountIdentityCell :account="account" show-plan class="min-w-0" />
<span
class="inline-flex size-8.5 items-center justify-center rounded-full"
:class="rowToneClasses[account.tone]"
class="grid min-w-0 grid-cols-[minmax(0,0.82fr)_minmax(0,0.64fr)_minmax(104px,1fr)] items-end gap-3 xl:contents"
>
{{ account.name[0] }}
</span>
<span class="flex min-w-0 max-w-full flex-col gap-1.5 xl:gap-4.25">
<span class="flex min-w-0 max-w-full items-center gap-2">
<strong
class="min-w-0 truncate text-sm leading-[1.15] font-[650] text-(--cp-text-primary)"
:title="account.name"
>{{ account.name }}</strong
>
<AccountPlanBadge class="shrink-0" :plan-type="account.planType" />
</span>
<span
class="block min-w-0 max-w-full truncate text-xs leading-[1.15] font-semibold text-(--cp-text-secondary)"
:title="account.email"
>{{ account.email }}</span
>
</span>
<span
class="col-span-2 mt-3 grid grid-cols-3 gap-3 pl-11 xl:col-span-1 xl:mt-0 xl:grid-cols-[minmax(0,56px)_minmax(72px,1fr)] xl:gap-6 xl:pl-0"
>
<span class="grid min-w-0 gap-1.5 xl:gap-1">
<span
class="text-[11px] leading-[1.15] font-[650] text-(--cp-text-muted) xl:hidden"
>最近</span
<span class="grid min-w-0 gap-1">
<span class="text-[11px] leading-none font-[720] text-(--cp-text-muted)"
>Token</span
>
<strong
class="hidden shrink-0 font-mono text-sm leading-[1.15] font-bold tabular-nums text-(--cp-text-primary) xl:block"
class="font-mono text-sm leading-[1.15] font-[780] tabular-nums text-(--cp-text-primary)"
>{{ account.tokens }}</strong
>
</span>
<span class="grid min-w-0 gap-1">
<span class="text-[11px] leading-none font-[720] text-(--cp-text-muted)"
>最近</span
>
<span
class="min-w-0 truncate text-xs leading-[1.15] font-semibold text-(--cp-text-secondary) xl:whitespace-nowrap"
class="min-w-0 truncate text-xs leading-[1.15] font-[680] text-(--cp-text-secondary) xl:whitespace-nowrap"
>
{{ account.lastUsed }}
</span>
</span>
<span class="grid min-w-0 gap-1.5 xl:hidden">
<span
class="text-[11px] leading-[1.15] font-[650] text-(--cp-text-muted) xl:hidden"
>Token</span
>
<strong
class="w-full font-mono text-sm leading-[1.15] font-bold tabular-nums text-(--cp-text-primary)"
>{{ account.tokens }}</strong
>
</span>
<span class="grid min-w-0 gap-2">
<span
class="text-[11px] leading-[1.15] font-[650] text-(--cp-text-muted) xl:hidden"
>额度</span
>
<span
class="mt-1 block h-1.5 w-full overflow-hidden rounded-full bg-(--cp-bg-muted)"
>
<span class="flex items-center justify-between gap-2">
<span class="text-[11px] leading-none font-[720] text-(--cp-text-muted)"
>额度</span
>
<span
class="font-mono text-[10px] leading-none font-[760] text-(--cp-text-muted)"
>{{ account.quotaPercent }}%</span
>
</span>
<span class="block h-1.5 w-full overflow-hidden rounded-full bg-(--cp-bg-muted)">
<i
class="block h-1.5 rounded-full"
:class="quotaToneClasses[account.quotaTone]"
@@ -186,7 +186,7 @@ function getSeries() {
'成功率',
props.points.map((p) => p.successRateValue),
themeColor('--cp-success', '#10B981'),
false,
true,
1,
),
lineSeries(
@@ -197,7 +197,7 @@ function getSeries() {
]
}
function lineSeries(name: string, data: number[], color: string, area = false, yAxisIndex = 0) {
function lineSeries(name: string, data: number[], color: string, area = true, yAxisIndex = 0) {
return {
name,
type: 'line' as const,
@@ -255,7 +255,7 @@ function handleTrendChange(value: string) {
v-if="!hasSamples"
compact
title="暂无趋势数据"
description="最近 24 小时暂无请求日志。"
description="当日暂无请求日志。"
class="h-full place-content-center bg-transparent"
/>
</div>
@@ -38,7 +38,7 @@ const valueToneClasses: Record<string, string> = {
<div
v-for="item in items"
:key="item.label"
class="grid h-12.5 w-full grid-cols-[22px_12px_1fr_1fr_1fr] items-center rounded-[14px] bg-(--cp-bg-subtle) px-3.5"
class="grid h-12.5 w-full grid-cols-[22px_12px_minmax(128px,0.82fr)_minmax(0,1.08fr)_minmax(72px,0.92fr)] items-center rounded-[14px] bg-(--cp-bg-subtle) px-3.5"
>
<span
class="inline-flex size-6 items-center justify-center rounded-lg"
@@ -131,32 +131,23 @@ export function useDashboard(): any {
traffic: {
todayRequests: '0',
todayRequestsValue: 0,
yesterdayRequests: '0',
yesterdayRequestsValue: 0,
totalRequests: '0',
rpm: '0',
tpm: '0',
},
tokens: {
todayTokens: '0',
todayTokensValue: 0,
yesterdayTokens: '0',
yesterdayTokensValue: 0,
totalTokens: '0',
todayCostUsd: '—',
todayCostUsdValue: 0,
totalCostUsd: '—',
totalCostUsdValue: 0,
},
cache: {
todayHitRate: '—',
todayHitRateValue: null,
yesterdayHitRate: '—',
yesterdayHitRateValue: null,
totalHitRate: '—',
totalCachedTokens: '0',
firstTokenLatencyMs: '—',
completionLatencyMs: '—',
averageFirstTokenLatencyMs: '—',
},
},
trend: {
@@ -166,11 +157,8 @@ export function useDashboard(): any {
},
healthTimeline: {
title: '请求健康时间线',
description: '今日请求可靠性',
rangeDisplay: '-',
description: '请求可靠性',
reliabilityDisplay: '-',
oldestLabel: '最早',
newestLabel: '最新',
points: '',
},
accountUsage: [],
@@ -220,7 +208,7 @@ export function useDashboard(): any {
],
},
{
title: '今日请求',
title: '请求次数',
value: traffic.todayRequests,
valueRaw: traffic.todayRequestsValue,
valueFormatter: formatDashboardCompactNumber,
@@ -234,14 +222,14 @@ export function useDashboard(): any {
details: [
{ label: '总请求', value: traffic.totalRequests, tone: 'info' },
{
label: '首 Token',
value: cache.firstTokenLatencyMs,
label: '首字延迟',
value: cache.averageFirstTokenLatencyMs,
tone: 'info',
},
],
},
{
title: '今日 Token',
title: 'Token',
value: tokens.todayTokens,
valueRaw: tokens.todayTokensValue,
valueFormatter: formatDashboardCompactNumber,
@@ -262,14 +250,14 @@ export function useDashboard(): any {
],
},
{
title: '今日缓存命中',
title: '缓存命中',
value: cache.todayHitRate,
valueRaw: cache.todayHitRateValue,
valueFormatter: formatDashboardRate,
icon: Timer,
tone: cache.todayHitRateValue && cache.todayHitRateValue > 0 ? 'warning' : 'normal',
sparkline: sparkline(
points.map((point) => point.cachedTokensValue),
points.map((point) => point.cacheHitRateValue),
'warning',
),
trend: trendState(
@@ -278,9 +266,9 @@ export function useDashboard(): any {
'warning',
),
details: [
{ label: '总缓存命中', value: cache.totalHitRate, tone: 'warning' },
{ label: '平均缓存命中', value: cache.totalHitRate, tone: 'warning' },
{
label: '缓存 Token',
label: '缓存',
value: cache.totalCachedTokens,
tone: 'warning',
},
@@ -353,16 +341,13 @@ export function useDashboard(): any {
}
function accountUsageItem(item: any) {
const tone = accountTone(item.status)
const quotaPercent = quotaUsedPercent(item.quotaUsedPercent)
return {
name: item.name,
email: item.email || '-',
planType: item.plan || 'free',
requests: item.requests,
id: item.id,
email: item.email,
planType: item.planType,
tokens: item.tokens,
lastUsed: item.lastUsed || '-',
tone,
lastUsed: item.lastUsed,
quotaPercent,
quotaTone: quotaTone(quotaPercent),
}
@@ -435,14 +420,6 @@ export function useDashboard(): any {
return '--cp-info'
}
function accountTone(status: string) {
if (status === 'active') return 'success'
if (status === 'quota_exhausted' || status === 'expired') return 'warning'
if (status === 'banned' || status === 'disabled') return 'danger'
if (status === 'refreshing') return 'info'
return 'normal'
}
onMounted(() => {
void loadDashboardData()
startAutoRefresh()
+1 -1
View File
@@ -39,7 +39,7 @@ const {
系统概览
</h1>
<p class="mt-2.5 text-[15px] leading-[1.15] font-semibold mb-0 text-(--cp-text-secondary)">
今日统计 · 自动刷新 30s
当日统计 · 自动刷新 30s
</p>
</div>
@@ -49,7 +49,7 @@ function togglePasswordVisible(): void {
as="form"
:padded="false"
variant="elevated"
class="login-form relative grid min-h-129.25 w-[min(450px,100%)] gap-3 rounded-lg px-7.5 pt-6.5 pb-6 max-[560px]:min-h-auto max-[560px]:gap-3 max-[560px]:p-5.5"
class="login-form relative grid min-h-129.25 w-[min(440px,100%)] gap-3 rounded-lg px-7.5 pt-6.5 pb-6 max-[560px]:min-h-auto max-[560px]:gap-3 max-[560px]:p-5.5"
@submit.prevent="emit('submit')"
>
<div class="login-form-line" aria-hidden="true" />
+2 -2
View File
@@ -22,7 +22,7 @@ export const usageRecordColumns = [
{
key: 'route',
label: '端点',
width: '172px',
width: '185px',
ellipsis: false,
cellClass: 'font-mono text-[12px] font-[650]',
},
@@ -74,7 +74,7 @@ export const usageRecordColumns = [
{
key: 'clientIp',
label: 'IP',
width: '146px',
width: '200px',
ellipsis: false,
},
{
+1 -1
View File
@@ -1 +1 @@
version: "1.0.3"
version: "1.0.4"
+2 -2
View File
@@ -5,7 +5,7 @@ description: Codex Proxy RS 项目专用指南。Use when working in the codex-p
# Codex Proxy RS
使用这个 skill 来开发 Codex Proxy RS,避免每次重新摸索项目结构、开发流程和约束。
这个 skill 记录 Codex Proxy RS 的仓库结构、开发命令和约束。
## 开始前
@@ -20,7 +20,7 @@ description: Codex Proxy RS 项目专用指南。Use when working in the codex-p
- 后端:遵循 `backend/src` 里的 Rust/Axum 模式;集成测试放 `backend/tests`,禁止把测试代码放进 `backend/src`。
- 前端:使用 Vue 3 `<script setup>`、TypeScript、Tailwind v4 utilities 和已有 CSS 主题 token。
- Docker / 发布:保持 runtime 镜像最小化,并维护现有 GitHub Release、GHCR、在线更新契约。
- 文档:文档要短、准、和真实行为一致;除非用户要求,不新增冗余文档文件。
- 文档:短、准、和真实行为一致;除非用户要求,不新增冗余文档文件。
## 验证
@@ -18,7 +18,7 @@
- `backend/tests/`:集成测试和 fixtures。测试代码只放这里,禁止放进 `backend/src`。
- `frontend/`:Vue 3 管理端 SPA,使用 Vite 8、Tailwind v4、Pinia、Vue Router、Axios、lucide icons、ECharts。
- `deploy/`:Dockerfile、Dockerfile 专用 ignore 文件、Compose 文件、`config.example.yaml`。
- `docs/`:只放长期维护文档,例如架构说明;不要保留过程型迁移文档作为开发约束。
- `docs/`:长期维护文档;不要保留过程型迁移文档作为开发约束。
- `skills/`:项目本地 Codex skills。
## 后端规范
@@ -75,15 +75,16 @@ cargo run --manifest-path backend/Cargo.toml --bin codex-proxy-rs
- 本地 Docker 首次部署命令:
```bash
cp deploy/config.example.yaml deploy/config.yaml
mkdir -p .runtime/data .runtime/logs
cp deploy/config.example.yaml .runtime/config.yaml
docker compose -f deploy/docker-compose.yml build
docker compose -f deploy/docker-compose.yml up -d
```
- 运行时挂载关系:
- `deploy/config.yaml` -> `/app/config.yaml`
- `deploy/data` -> `/app/data`
- `deploy/logs` -> `/app/logs`
- `.runtime/config.yaml` -> `/app/config.yaml`
- `.runtime/data` -> `/app/data`
- `.runtime/logs` -> `/app/logs`
## 发布和版本契约
@@ -99,7 +100,6 @@ docker compose -f deploy/docker-compose.yml up -d
```text
codex-proxy-rs_<version>_linux_amd64.tar.gz
codex-proxy-rs_<version>_linux_arm64.tar.gz
codex-proxy-rs_<version>_darwin_amd64.tar.gz
codex-proxy-rs_<version>_darwin_arm64.tar.gz
codex-proxy-rs_<version>_windows_amd64.zip
checksums.txt
@@ -114,10 +114,11 @@ checksums.txt
- 后端接口:
- `GET /api/admin/system/version`
- `GET /api/admin/system/check-updates`
- `GET /api/admin/system/update-detail`
- `GET /api/admin/system/update-events`
- `GET /api/admin/system/update-status`
- `POST /api/admin/system/update`
- `POST /api/admin/system/rollback`
- `POST /api/admin/system/restart`
- 前端更新 UI:`frontend/src/layout/components/SystemUpdateModal.vue`。
- SSE 日志由后端生成中文消息;前端只负责展示,不做状态文案映射。