Compare commits

...

114 Commits

Author SHA1 Message Date
tuna2134@コマリン親衛隊
cc664fae2d Update CI.yml 2025-01-20 08:43:05 +09:00
tuna2134@コマリン親衛隊
71ec658772 CI.yml を更新 2025-01-20 07:55:03 +09:00
コマリン親衛隊
dff939091c Merge pull request #165 from tuna2134/renovate/log-0.x-lockfile
fix(deps): update rust crate log to v0.4.24
2025-01-11 11:55:02 +09:00
renovate[bot]
8a28a4e7a5 fix(deps): update rust crate log to v0.4.24 2025-01-11 02:50:24 +00:00
コマリン親衛隊
21f845a799 Merge pull request #162 from tuna2134/renovate/serde_json-1.x-lockfile
fix(deps): update rust crate serde_json to v1.0.135
2025-01-09 09:40:07 +09:00
renovate[bot]
4ff9a38a80 fix(deps): update rust crate serde_json to v1.0.135 2025-01-08 15:24:08 +00:00
コマリン親衛隊
20cc0573b5 Merge pull request #161 from tuna2134/renovate/utoipa-monorepo
fix(deps): update utoipa monorepo
2025-01-06 13:28:46 +09:00
renovate[bot]
4b932d568d fix(deps): update utoipa monorepo 2025-01-06 00:39:56 +00:00
コマリン親衛隊
6237cd0fec Merge pull request #159 from tuna2134/renovate/serde-monorepo
fix(deps): update rust crate serde to v1.0.217
2024-12-28 13:09:10 +09:00
renovate[bot]
35fabdf681 fix(deps): update rust crate serde to v1.0.217 2024-12-27 22:05:25 +00:00
コマリン親衛隊
f09343c97f Merge pull request #155 from tuna2134/renovate/env_logger-0.x-lockfile
chore(deps): update rust crate env_logger to v0.11.6
2024-12-23 23:38:00 +09:00
コマリン親衛隊
f2570d89d0 Merge pull request #158 from tuna2134/renovate/anyhow-1.x-lockfile
chore(deps): update rust crate anyhow to v1.0.95
2024-12-22 21:52:38 +09:00
renovate[bot]
ac2a09d6af chore(deps): update rust crate anyhow to v1.0.95 2024-12-22 12:39:36 +00:00
コマリン親衛隊
c6eaf9cb9f Merge pull request #156 from tuna2134/renovate/serde_json-1.x-lockfile
fix(deps): update rust crate serde_json to v1.0.134
2024-12-22 10:30:46 +09:00
renovate[bot]
f2395096ca fix(deps): update rust crate serde_json to v1.0.134 2024-12-21 21:03:32 +00:00
renovate[bot]
3f6f4ccb6f chore(deps): update rust crate env_logger to v0.11.6 2024-12-20 20:48:36 +00:00
コマリン親衛隊
67eba8ee6c Merge pull request #152 from tuna2134/renovate/utoipa-monorepo
fix(deps): update rust crate utoipa to v5.3.0
2024-12-20 09:03:53 +09:00
renovate[bot]
0aa1bc8733 fix(deps): update rust crate utoipa to v5.3.0 2024-12-19 18:06:44 +00:00
コマリン親衛隊
d1970d99be Merge pull request #151 from tuna2134/renovate/pyo3-0.x-lockfile
fix(deps): update rust crate pyo3 to v0.23.3
2024-12-19 08:30:37 +09:00
コマリン親衛隊
fddb35e592 Update CI.yml 2024-12-19 08:28:53 +09:00
renovate[bot]
e26715c809 fix(deps): update rust crate pyo3 to v0.23.3 2024-12-18 23:20:53 +00:00
コマリン親衛隊
26aa4b7df0 Merge pull request #150 from tuna2134/next
Next
2024-12-19 08:19:48 +09:00
コマリン親衛隊
de18846280 Update sbv2.rs 2024-12-19 07:50:13 +09:00
コマリン親衛隊
38c2e69648 Merge pull request #133 from tuna2134/renovate/pyo3-0.x
fix(deps): update rust crate pyo3 to 0.23.0
2024-12-19 07:47:03 +09:00
コマリン親衛隊
593dbaf19d Merge pull request #144 from tuna2134/renovate/tokenizers-0.x
fix(deps): update rust crate tokenizers to 0.21.0
2024-12-19 07:44:45 +09:00
コマリン親衛隊
bf44b07be1 Merge pull request #146 from tuna2134/renovate/tokio-1.x-lockfile
fix(deps): update rust crate tokio to v1.42.0
2024-12-19 07:44:30 +09:00
コマリン親衛隊
102a8eb065 Merge pull request #149 from tuna2134/renovate/serde-monorepo
fix(deps): update rust crate serde to v1.0.216
2024-12-19 07:44:21 +09:00
renovate[bot]
68edb3187f fix(deps): update rust crate serde to v1.0.216 2024-12-11 03:08:44 +00:00
コマリン親衛隊
4a81a06faf Merge pull request #147 from tuna2134/renovate/anyhow-1.x-lockfile
chore(deps): update rust crate anyhow to v1.0.94
2024-12-07 14:27:55 +09:00
renovate[bot]
caf541ef65 chore(deps): update rust crate anyhow to v1.0.94 2024-12-03 23:23:13 +00:00
renovate[bot]
05c3846b7b fix(deps): update rust crate tokio to v1.42.0 2024-12-03 18:00:26 +00:00
renovate[bot]
1b2054c4b8 fix(deps): update rust crate tokenizers to 0.21.0 2024-11-27 14:26:20 +00:00
コマリン親衛隊
a7fbfa2017 Merge pull request #138 from tuna2134/aivmx
support aivmx
2024-11-20 16:10:06 +09:00
tuna2134
db09b73b32 support aivmx 2024-11-20 07:01:43 +00:00
tuna2134
843ef36148 Merge branch 'main' of https://github.com/tuna2134/sbv2-api into aivmx 2024-11-20 04:15:44 +00:00
コマリン親衛隊
aa7fc2e3b0 Delete convert/LICENSE 2024-11-20 13:13:17 +09:00
コマリン親衛隊
fc4a79c111 Create LICENSE 2024-11-20 13:12:40 +09:00
コマリン親衛隊
4db7f49fa5 Update and rename LICENSE to convert/LICENSE 2024-11-20 13:11:58 +09:00
tuna2134
edee0710aa support noise_scale 2024-11-20 02:53:14 +00:00
tuna2134
9bcbd496e5 fix 2024-11-20 02:42:33 +00:00
tuna2134
90b3ba2e40 fix bug 2024-11-20 02:42:19 +00:00
tuna2134
9ceec03bd0 fix bug 2024-11-20 02:39:38 +00:00
tuna2134
5e9df65656 add aivmx test 2024-11-20 02:36:42 +00:00
tuna2134
2eda2fe9ca fix 2024-11-20 02:14:59 +00:00
tuna2134
9c9119a107 support aivmx 2024-11-20 01:42:04 +00:00
コマリン親衛隊
2c1a1dffc0 Merge pull request #135 from tuna2134/renovate/serde_json-1.x-lockfile
fix(deps): update rust crate serde_json to v1.0.133
2024-11-17 14:50:23 +09:00
renovate[bot]
ed7bf53b89 fix(deps): update rust crate serde_json to v1.0.133 2024-11-17 03:44:30 +00:00
コマリン親衛隊
4375df2689 Merge pull request #134 from tuna2134/renovate/axum-0.x-lockfile
fix(deps): update rust crate axum to v0.7.9
2024-11-17 12:43:38 +09:00
renovate[bot]
789cef74ce fix(deps): update rust crate axum to v0.7.9 2024-11-16 22:43:20 +00:00
コマリン親衛隊
5b403a2255 Merge pull request #132 from tuna2134/renovate/axum-0.x-lockfile
fix(deps): update rust crate axum to v0.7.8
2024-11-16 09:22:17 +09:00
renovate[bot]
62653ec1c3 fix(deps): update rust crate pyo3 to 0.23.0 2024-11-15 23:06:13 +00:00
renovate[bot]
83076227e7 fix(deps): update rust crate axum to v0.7.8 2024-11-15 18:22:03 +00:00
tuna2134
f90904a337 fix version 2024-11-13 12:02:36 +00:00
tuna2134
4e0c8591cd fix 2024-11-13 12:00:59 +00:00
コマリン親衛隊
997b562682 Merge pull request #131 from tuna2134/add-spealer
話者指定を追加
2024-11-13 20:58:11 +09:00
tuna2134
fbd62315d0 clippy 2024-11-13 11:46:47 +00:00
tuna2134
060af0c187 format 2024-11-13 11:43:52 +00:00
tuna2134
b76738f467 add speaker id code 2024-11-13 11:39:05 +00:00
コマリン親衛隊
8598167114 Merge pull request #130 from tuna2134/tuna2134-patch-1
Style ID指定できるようにした
2024-11-13 11:46:17 +09:00
tuna2134
001f61bb6a fix types 2024-11-13 02:24:09 +00:00
コマリン親衛隊
9b9962ed29 Style ID指定できるようにした 2024-11-13 11:16:24 +09:00
コマリン親衛隊
b414d22a3b Merge pull request #129 from tuna2134/renovate/serde-monorepo
fix(deps): update rust crate serde to v1.0.215
2024-11-13 11:09:25 +09:00
renovate[bot]
248363ae4a fix(deps): update rust crate serde to v1.0.215 2024-11-12 00:53:00 +00:00
コマリン親衛隊
c4b61a36db Merge pull request #128 from tuna2134/renovate/thiserror-1.x-lockfile
fix(deps): update rust crate thiserror to v1.0.69
2024-11-10 20:38:21 +09:00
renovate[bot]
35d16d88a8 fix(deps): update rust crate thiserror to v1.0.69 2024-11-10 07:11:07 +00:00
コマリン親衛隊
fe48d6a034 Merge pull request #127 from tuna2134/renovate/tokio-1.x-lockfile
fix(deps): update rust crate tokio to v1.41.1
2024-11-08 09:55:35 +09:00
renovate[bot]
bca4b2053f fix(deps): update rust crate tokio to v1.41.1 2024-11-07 13:40:20 +00:00
コマリン親衛隊
3330242cd8 Merge pull request #120 from tuna2134/renovate/tokenizers-0.x-lockfile 2024-11-07 00:17:47 +08:00
コマリン親衛隊
f10f71f29b Merge pull request #124 from tuna2134/renovate/anyhow-1.x-lockfile 2024-11-06 21:12:26 +08:00
renovate[bot]
7bd39b7182 chore(deps): update rust crate anyhow to v1.0.93 2024-11-06 13:01:27 +00:00
コマリン親衛隊
2d557fb0ee Merge pull request #123 from Googlefan256/main 2024-11-06 21:00:38 +08:00
Googlefan
14d631eeaa wip: max loaded models 2024-11-06 10:43:41 +00:00
コマリン親衛隊
380daf479c Merge pull request #122 from tuna2134/renovate/pyo3-0.x-lockfile 2024-11-06 09:57:35 +08:00
renovate[bot]
cb814a9952 fix(deps): update rust crate pyo3 to v0.22.6 2024-11-06 01:24:53 +00:00
renovate[bot]
795caf626c fix(deps): update rust crate tokenizers to v0.20.3 2024-11-05 18:02:17 +00:00
コマリン親衛隊
fb32357f31 Merge pull request #119 from tuna2134/renovate/thiserror-1.x-lockfile 2024-11-05 09:49:04 +08:00
renovate[bot]
e4010b3b83 fix(deps): update rust crate thiserror to v1.0.68 2024-11-04 19:39:16 +00:00
コマリン親衛隊
17244a9ede Merge pull request #118 from tuna2134/renovate/thiserror-1.x-lockfile
fix(deps): update rust crate thiserror to v1.0.67
2024-11-04 01:23:40 +09:00
renovate[bot]
61b04fd3d7 fix(deps): update rust crate thiserror to v1.0.67 2024-11-03 16:01:48 +00:00
コマリン親衛隊
4e57a22a40 Merge pull request #117 from tuna2134/renovate/utoipa-5.x-lockfile
fix(deps): update rust crate utoipa to v5.2.0
2024-11-03 08:01:41 +09:00
renovate[bot]
8e10057882 fix(deps): update rust crate utoipa to v5.2.0 2024-11-02 15:44:49 +00:00
コマリン親衛隊
0222b9a189 Merge pull request #116 from tuna2134/renovate/tar-0.x-lockfile
fix(deps): update rust crate tar to v0.4.43
2024-11-02 16:14:41 +09:00
renovate[bot]
5e96d5aef7 fix(deps): update rust crate tar to v0.4.43 2024-11-02 06:41:37 +00:00
コマリン親衛隊
234120f510 Merge pull request #115 from tuna2134/renovate/thiserror-1.x-lockfile 2024-11-02 07:08:57 +09:00
コマリン親衛隊
08f7ab88ec Merge pull request #114 from tuna2134/renovate/anyhow-1.x-lockfile 2024-11-02 07:08:40 +09:00
renovate[bot]
005c67c9b6 fix(deps): update rust crate thiserror to v1.0.66 2024-11-01 17:30:59 +00:00
renovate[bot]
cb08b5b582 chore(deps): update rust crate anyhow to v1.0.92 2024-11-01 17:30:55 +00:00
コマリン親衛隊
105b3ce8de Merge pull request #113 from tuna2134/renovate/onnxruntime-web-1.x-lockfile
fix(deps): update dependency onnxruntime-web to v1.20.0
2024-10-31 12:55:53 +09:00
renovate[bot]
78a5016abc fix(deps): update dependency onnxruntime-web to v1.20.0 2024-10-31 01:30:18 +00:00
コマリン親衛隊
7e6bd4ad0a Merge pull request #112 from tuna2134/renovate/serde-monorepo 2024-10-29 07:47:34 +09:00
renovate[bot]
e1c6cd04b7 fix(deps): update rust crate serde to v1.0.214 2024-10-28 19:40:13 +00:00
コマリン親衛隊
a15efdff09 Merge pull request #110 from tuna2134/renovate/node-22.x-lockfile
chore(deps): update dependency @types/node to v22.8.1
2024-10-28 15:33:34 +09:00
コマリン親衛隊
21823721d0 Merge pull request #111 from tuna2134/renovate/utoipa-5.x-lockfile
fix(deps): update rust crate utoipa to v5.1.3
2024-10-28 15:33:24 +09:00
renovate[bot]
aad978be4b fix(deps): update rust crate utoipa to v5.1.3 2024-10-27 15:20:25 +00:00
renovate[bot]
6dd2cbd991 chore(deps): update dependency @types/node to v22.8.0 2024-10-25 13:50:26 +00:00
コマリン親衛隊
d7b76cc207 Merge pull request #109 from tuna2134/renovate/regex-1.x-lockfile
fix(deps): update rust crate regex to v1.11.1
2024-10-25 02:00:29 +09:00
renovate[bot]
ae0ccb29d2 fix(deps): update rust crate regex to v1.11.1 2024-10-24 16:27:25 +00:00
tuna2134
4bcde2e4b4 bump library version 2024-10-24 08:01:09 +00:00
コマリン親衛隊
2356c896f6 Merge pull request #108 from tuna2134/renovate/utoipa-5.x-lockfile
fix(deps): update rust crate utoipa to v5.1.2
2024-10-24 00:55:42 +09:00
renovate[bot]
d5445abeee fix(deps): update rust crate utoipa to v5.1.2 2024-10-23 15:50:59 +00:00
コマリン親衛隊
673ec0067d Merge pull request #107 from tuna2134/renovate/node-22.x-lockfile
chore(deps): update dependency @types/node to v22.7.9
2024-10-23 19:05:43 +09:00
renovate[bot]
74f657cb33 chore(deps): update dependency @types/node to v22.7.9 2024-10-23 04:51:07 +00:00
コマリン親衛隊
08be778cc5 Merge pull request #105 from tuna2134/renovate/thiserror-1.x-lockfile
fix(deps): update rust crate thiserror to v1.0.65
2024-10-23 08:08:57 +09:00
コマリン親衛隊
6da2f5a0bb Merge pull request #104 from tuna2134/renovate/serde-monorepo
fix(deps): update rust crate serde to v1.0.213
2024-10-23 08:08:50 +09:00
コマリン親衛隊
107190765f Merge pull request #106 from tuna2134/renovate/anyhow-1.x-lockfile
chore(deps): update rust crate anyhow to v1.0.91
2024-10-23 08:08:41 +09:00
renovate[bot]
df726e6f7b fix(deps): update rust crate serde to v1.0.213 2024-10-22 22:07:24 +00:00
renovate[bot]
e5b1ccc36b chore(deps): update rust crate anyhow to v1.0.91 2024-10-22 22:07:14 +00:00
renovate[bot]
40cb604c57 fix(deps): update rust crate thiserror to v1.0.65 2024-10-22 18:02:37 +00:00
コマリン親衛隊
9152c80c76 Merge pull request #102 from tuna2134/renovate/serde-monorepo
fix(deps): update rust crate serde to v1.0.211
2024-10-22 19:30:19 +09:00
コマリン親衛隊
574092562e Merge pull request #103 from tuna2134/renovate/tokio-1.x-lockfile
fix(deps): update rust crate tokio to v1.41.0
2024-10-22 19:30:09 +09:00
renovate[bot]
2e931adce7 fix(deps): update rust crate tokio to v1.41.0 2024-10-22 10:09:19 +00:00
renovate[bot]
e36c395db1 fix(deps): update rust crate serde to v1.0.211 2024-10-22 10:09:13 +00:00
コマリン親衛隊
cfe88629ab Merge pull request #101 from tuna2134/renovate/node-22.x-lockfile
chore(deps): update dependency @types/node to v22.7.8
2024-10-22 13:18:01 +09:00
renovate[bot]
30a98f0968 chore(deps): update dependency @types/node to v22.7.8 2024-10-22 03:41:35 +00:00
13 changed files with 664 additions and 256 deletions

View File

@@ -79,8 +79,6 @@ jobs:
strategy:
matrix:
platform:
- runner: macos-12
target: x86_64
- runner: macos-14
target: aarch64
steps:
@@ -132,7 +130,7 @@ jobs:
args: --non-interactive --skip-existing wheels-*/*
push-docker:
runs-on: ubuntu-latest
runs-on: ${{ matrix.platform }}
if: "startsWith(github.ref, 'refs/tags/')"
permissions:
contents: read
@@ -140,9 +138,7 @@ jobs:
strategy:
matrix:
tag: [cpu, cuda]
platform:
- linux/amd64
- linux/arm64
platform: [ubuntu-latest, ubuntu-24.04-arm]
steps:
- uses: actions/checkout@v4
- name: Set up QEMU
@@ -163,4 +159,3 @@ jobs:
tags: |
ghcr.io/${{ github.repository }}:${{ matrix.tag }}
file: docker/${{ matrix.tag }}.Dockerfile
platforms: ${{ matrix.platform }}

598
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -94,7 +94,7 @@ model = get_net_g(
)
def forward(x, x_len, sid, tone, lang, bert, style, length_scale, sdp_ratio):
def forward(x, x_len, sid, tone, lang, bert, style, length_scale, sdp_ratio, noise_scale, noise_scale_w):
return model.infer(
x,
x_len,
@@ -105,6 +105,8 @@ def forward(x, x_len, sid, tone, lang, bert, style, length_scale, sdp_ratio):
style,
sdp_ratio=sdp_ratio,
length_scale=length_scale,
noise_scale=noise_scale,
noise_scale_w=noise_scale_w,
)
@@ -122,6 +124,8 @@ torch.onnx.export(
style_vec_tensor,
torch.tensor(1.0),
torch.tensor(0.0),
torch.tensor(0.6777),
torch.tensor(0.8),
),
f"../models/model_{out_name}.onnx",
verbose=True,
@@ -144,6 +148,8 @@ torch.onnx.export(
"style_vec",
"length_scale",
"sdp_ratio",
"noise_scale",
"noise_scale_w"
],
output_names=["output"],
)

View File

@@ -1,6 +1,6 @@
[package]
name = "sbv2_api"
version = "0.2.0-alpha"
version = "0.2.0-alpha4"
edition = "2021"
[dependencies]
@@ -9,7 +9,7 @@ axum = "0.7.5"
dotenvy.workspace = true
env_logger.workspace = true
log = "0.4.22"
sbv2_core = { version = "0.2.0-alpha2", path = "../sbv2_core" }
sbv2_core = { version = "0.2.0-alpha2", path = "../sbv2_core", features = ["aivmx"] }
serde = { version = "1.0.210", features = ["derive"] }
tokio = { version = "1.40.0", features = ["full"] }
utoipa = { version = "5.0.0", features = ["axum_extras"] }

View File

@@ -40,6 +40,14 @@ fn length_default() -> f32 {
1.0
}
fn style_id_default() -> i32 {
0
}
fn speaker_id_default() -> i64 {
0
}
#[derive(Deserialize, ToSchema)]
struct SynthesizeRequest {
text: String,
@@ -48,6 +56,10 @@ struct SynthesizeRequest {
sdp_ratio: f32,
#[serde(default = "length_default")]
length_scale: f32,
#[serde(default = "style_id_default")]
style_id: i32,
#[serde(default = "speaker_id_default")]
speaker_id: i64,
}
#[utoipa::path(
@@ -65,15 +77,18 @@ async fn synthesize(
ident,
sdp_ratio,
length_scale,
style_id,
speaker_id,
}): Json<SynthesizeRequest>,
) -> AppResult<impl IntoResponse> {
log::debug!("processing request: text={text}, ident={ident}, sdp_ratio={sdp_ratio}, length_scale={length_scale}");
let buffer = {
let tts_model = state.tts_model.lock().await;
let mut tts_model = state.tts_model.lock().await;
tts_model.easy_synthesize(
&ident,
&text,
0,
style_id,
speaker_id,
SynthesizeOptions {
sdp_ratio,
length_scale,
@@ -94,6 +109,9 @@ impl AppState {
let mut tts_model = TTSModelHolder::new(
&fs::read(env::var("BERT_MODEL_PATH")?).await?,
&fs::read(env::var("TOKENIZER_PATH")?).await?,
env::var("HOLDER_MAX_LOADED_MODElS")
.ok()
.and_then(|x| x.parse().ok()),
)?;
let models = env::var("MODELS_PATH").unwrap_or("models".to_string());
let mut f = fs::read_dir(&models).await?;
@@ -122,6 +140,20 @@ impl AppState {
log::warn!("Error loading {entry}: {e}");
};
log::info!("Loaded: {entry}");
} else if name.ends_with(".aivmx") {
let entry = &name[..name.len() - 6];
log::info!("Try loading: {entry}");
let aivmx_bytes = match fs::read(format!("{models}/{entry}.aivmx")).await {
Ok(b) => b,
Err(e) => {
log::warn!("Error loading aivmx bytes from file {entry}: {e}");
continue;
}
};
if let Err(e) = tts_model.load_aivmx(entry, aivmx_bytes) {
log::error!("Error loading {entry}: {e}");
}
log::info!("Loaded: {entry}");
}
}
for entry in entries {

View File

@@ -1,6 +1,6 @@
[package]
name = "sbv2_bindings"
version = "0.2.0-alpha2"
version = "0.2.0-alpha4"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
@@ -11,5 +11,5 @@ crate-type = ["cdylib"]
[dependencies]
anyhow.workspace = true
ndarray.workspace = true
pyo3 = { version = "0.22.0", features = ["anyhow"] }
pyo3 = { version = "0.23.0", features = ["anyhow"] }
sbv2_core = { version = "0.2.0-alpha2", path = "../sbv2_core" }

View File

@@ -23,10 +23,15 @@ pub struct TTSModel {
#[pymethods]
impl TTSModel {
#[pyo3(signature = (bert_model_bytes, tokenizer_bytes, max_loaded_models=None))]
#[new]
fn new(bert_model_bytes: Vec<u8>, tokenizer_bytes: Vec<u8>) -> anyhow::Result<Self> {
fn new(
bert_model_bytes: Vec<u8>,
tokenizer_bytes: Vec<u8>,
max_loaded_models: Option<usize>,
) -> anyhow::Result<Self> {
Ok(Self {
model: TTSModelHolder::new(bert_model_bytes, tokenizer_bytes)?,
model: TTSModelHolder::new(bert_model_bytes, tokenizer_bytes, max_loaded_models)?,
})
}
@@ -38,10 +43,21 @@ impl TTSModel {
/// BERTモデルのパス
/// tokenizer_path : str
/// トークナイザーのパス
/// max_loaded_models: int | None
/// 同時にVRAMに存在するモデルの数
#[pyo3(signature = (bert_model_path, tokenizer_path, max_loaded_models=None))]
#[staticmethod]
fn from_path(bert_model_path: String, tokenizer_path: String) -> anyhow::Result<Self> {
fn from_path(
bert_model_path: String,
tokenizer_path: String,
max_loaded_models: Option<usize>,
) -> anyhow::Result<Self> {
Ok(Self {
model: TTSModelHolder::new(fs::read(bert_model_path)?, fs::read(tokenizer_path)?)?,
model: TTSModelHolder::new(
fs::read(bert_model_path)?,
fs::read(tokenizer_path)?,
max_loaded_models,
)?,
})
}
@@ -121,11 +137,12 @@ impl TTSModel {
/// voice_data : bytes
/// 音声データ
fn synthesize<'p>(
&'p self,
&'p mut self,
py: Python<'p>,
text: String,
ident: String,
style_id: i32,
speaker_id: i64,
sdp_ratio: f32,
length_scale: f32,
) -> anyhow::Result<Bound<PyBytes>> {
@@ -133,13 +150,14 @@ impl TTSModel {
ident.as_str(),
&text,
style_id,
speaker_id,
SynthesizeOptions {
sdp_ratio,
length_scale,
..Default::default()
},
)?;
Ok(PyBytes::new_bound(py, &data))
Ok(PyBytes::new(py, &data))
}
fn unload(&mut self, ident: String) -> bool {

View File

@@ -1,7 +1,7 @@
[package]
name = "sbv2_core"
description = "Style-Bert-VITSの推論ライブラリ"
version = "0.2.0-alpha2"
version = "0.2.0-alpha4"
edition = "2021"
license = "MIT"
readme = "../README.md"
@@ -10,20 +10,22 @@ documentation = "https://docs.rs/sbv2_core"
[dependencies]
anyhow.workspace = true
base64 = { version = "0.22.1", optional = true }
dotenvy.workspace = true
env_logger.workspace = true
hound = "3.5.1"
jpreprocess = { version = "0.10.0", features = ["naist-jdic"] }
ndarray.workspace = true
npyz = { version = "0.8.3", optional = true }
num_cpus = "1.16.0"
once_cell.workspace = true
ort = { git = "https://github.com/pykeio/ort.git", version = "2.0.0-rc.6", optional = true }
ort = { git = "https://github.com/pykeio/ort.git", version = "2.0.0-rc.8", optional = true }
regex = "1.10.6"
serde = { version = "1.0.210", features = ["derive"] }
serde_json = "1.0.128"
tar = "0.4.41"
thiserror = "1.0.63"
tokenizers = { version = "0.20.0", default-features = false }
tokenizers = { version = "0.21.0", default-features = false }
zstd = "0.13.2"
[features]
@@ -35,4 +37,6 @@ directml = ["ort/directml", "std"]
tensorrt = ["ort/tensorrt", "std"]
coreml = ["ort/coreml", "std"]
default = ["std"]
no_std = ["tokenizers/unstable_wasm"]
no_std = ["tokenizers/unstable_wasm"]
aivmx = ["npyz", "base64"]
base64 = ["dep:base64"]

View File

@@ -21,6 +21,9 @@ pub enum Error {
HoundError(#[from] hound::Error),
#[error("model not found error")]
ModelNotFoundError(String),
#[cfg(feature = "base64")]
#[error("base64 error")]
Base64Error(#[from] base64::DecodeError),
#[error("other")]
OtherError(String),
}

View File

@@ -11,14 +11,26 @@ fn main_inner() -> anyhow::Result<()> {
let mut tts_holder = tts::TTSModelHolder::new(
&fs::read(env::var("BERT_MODEL_PATH")?)?,
&fs::read(env::var("TOKENIZER_PATH")?)?,
env::var("HOLDER_MAX_LOADED_MODElS")
.ok()
.and_then(|x| x.parse().ok()),
)?;
tts_holder.load_sbv2file(ident, fs::read(env::var("MODEL_PATH")?)?)?;
#[cfg(not(feature = "aivmx"))]
{
tts_holder.load_sbv2file(ident, fs::read(env::var("MODEL_PATH")?)?)?;
}
#[cfg(feature = "aivmx")]
{
tts_holder.load_aivmx(ident, fs::read(env::var("MODEL_PATH")?)?)?;
}
let audio = tts_holder.easy_synthesize(ident, &text, 0, tts::SynthesizeOptions::default())?;
let audio =
tts_holder.easy_synthesize(ident, &text, 0, 0, tts::SynthesizeOptions::default())?;
fs::write("output.wav", audio)?;
Ok(())
}
#[cfg(not(feature = "std"))]
fn main_inner() -> anyhow::Result<()> {
Ok(())

View File

@@ -52,11 +52,14 @@ pub fn synthesize(
session: &Session,
bert_ori: Array2<f32>,
x_tst: Array1<i64>,
sid: Array1<i64>,
tones: Array1<i64>,
lang_ids: Array1<i64>,
style_vector: Array1<f32>,
sdp_ratio: f32,
length_scale: f32,
noise_scale: f32,
noise_scale_w: f32,
) -> Result<Array3<f32>> {
let bert = bert_ori.insert_axis(Axis(0));
let x_tst_lengths: Array1<i64> = array![x_tst.shape()[0] as i64];
@@ -67,13 +70,15 @@ pub fn synthesize(
let outputs = session.run(ort::inputs! {
"x_tst" => x_tst,
"x_tst_lengths" => x_tst_lengths,
"sid" => array![0_i64],
"sid" => sid,
"tones" => tones,
"language" => lang_ids,
"bert" => bert,
"style_vec" => style_vector,
"sdp_ratio" => array![sdp_ratio],
"length_scale" => array![length_scale],
"noise_scale" => array![noise_scale],
"noise_scale_w" => array![noise_scale_w]
}?)?;
let audio_array = outputs["output"]

View File

@@ -1,7 +1,13 @@
use crate::error::{Error, Result};
use crate::{jtalk, model, style, tokenizer, tts_util};
#[cfg(feature = "aivmx")]
use base64::prelude::{Engine as _, BASE64_STANDARD};
#[cfg(feature = "aivmx")]
use ndarray::ShapeBuilder;
use ndarray::{concatenate, Array1, Array2, Array3, Axis};
use ort::Session;
#[cfg(feature = "aivmx")]
use std::io::Cursor;
use tokenizers::Tokenizer;
#[derive(PartialEq, Eq, Clone)]
@@ -24,9 +30,10 @@ where
}
pub struct TTSModel {
vits2: Session,
vits2: Option<Session>,
style_vectors: Array2<f32>,
ident: TTSIdent,
bytes: Option<Vec<u8>>,
}
/// High-level Style-Bert-VITS2's API
@@ -35,6 +42,7 @@ pub struct TTSModelHolder {
bert: Session,
models: Vec<TTSModel>,
jtalk: jtalk::JTalk,
max_loaded_models: Option<usize>,
}
impl TTSModelHolder {
@@ -43,9 +51,13 @@ impl TTSModelHolder {
/// # Examples
///
/// ```rs
/// let mut tts_holder = TTSModelHolder::new(std::fs::read("deberta.onnx")?, std::fs::read("tokenizer.json")?)?;
/// let mut tts_holder = TTSModelHolder::new(std::fs::read("deberta.onnx")?, std::fs::read("tokenizer.json")?, None)?;
/// ```
pub fn new<P: AsRef<[u8]>>(bert_model_bytes: P, tokenizer_bytes: P) -> Result<Self> {
pub fn new<P: AsRef<[u8]>>(
bert_model_bytes: P,
tokenizer_bytes: P,
max_loaded_models: Option<usize>,
) -> Result<Self> {
let bert = model::load_model(bert_model_bytes, true)?;
let jtalk = jtalk::JTalk::new()?;
let tokenizer = tokenizer::get_tokenizer(tokenizer_bytes)?;
@@ -54,6 +66,7 @@ impl TTSModelHolder {
models: vec![],
jtalk,
tokenizer,
max_loaded_models,
})
}
@@ -62,6 +75,53 @@ impl TTSModelHolder {
self.models.iter().map(|m| m.ident.to_string()).collect()
}
#[cfg(feature = "aivmx")]
pub fn load_aivmx<I: Into<TTSIdent>, P: AsRef<[u8]>>(
&mut self,
ident: I,
aivmx_bytes: P,
) -> Result<()> {
let ident = ident.into();
if self.find_model(ident.clone()).is_err() {
let mut load = true;
if let Some(max) = self.max_loaded_models {
if self.models.iter().filter(|x| x.vits2.is_some()).count() >= max {
load = false;
}
}
let model = model::load_model(&aivmx_bytes, false)?;
let metadata = model.metadata()?;
if let Some(aivm_style_vectors) = metadata.custom("aivm_style_vectors")? {
let aivm_style_vectors = BASE64_STANDARD.decode(aivm_style_vectors)?;
let style_vectors = Cursor::new(&aivm_style_vectors);
let reader = npyz::NpyFile::new(style_vectors)?;
let style_vectors = {
let shape = reader.shape().to_vec();
let order = reader.order();
let data = reader.into_vec::<f32>()?;
let shape = match shape[..] {
[i1, i2] => [i1 as usize, i2 as usize],
_ => panic!("expected 2D array"),
};
let true_shape = shape.set_f(order == npyz::Order::Fortran);
ndarray::Array2::from_shape_vec(true_shape, data)?
};
drop(metadata);
self.models.push(TTSModel {
vits2: if load { Some(model) } else { None },
bytes: if self.max_loaded_models.is_some() {
Some(aivmx_bytes.as_ref().to_vec())
} else {
None
},
ident,
style_vectors,
})
}
}
Ok(())
}
/// Load a .sbv2 file binary
///
/// # Examples
@@ -94,10 +154,25 @@ impl TTSModelHolder {
) -> Result<()> {
let ident = ident.into();
if self.find_model(ident.clone()).is_err() {
let mut load = true;
if let Some(max) = self.max_loaded_models {
if self.models.iter().filter(|x| x.vits2.is_some()).count() >= max {
load = false;
}
}
self.models.push(TTSModel {
vits2: model::load_model(vits2_bytes, false)?,
vits2: if load {
Some(model::load_model(&vits2_bytes, false)?)
} else {
None
},
style_vectors: style::load_style(style_vectors_bytes)?,
ident,
bytes: if self.max_loaded_models.is_some() {
Some(vits2_bytes.as_ref().to_vec())
} else {
None
},
})
}
Ok(())
@@ -145,6 +220,42 @@ impl TTSModelHolder {
.find(|m| m.ident == ident)
.ok_or(Error::ModelNotFoundError(ident.to_string()))
}
fn find_and_load_model<I: Into<TTSIdent>>(&mut self, ident: I) -> Result<bool> {
let ident = ident.into();
let (bytes, style_vectors) = {
let model = self
.models
.iter()
.find(|m| m.ident == ident)
.ok_or(Error::ModelNotFoundError(ident.to_string()))?;
if model.vits2.is_some() {
return Ok(true);
}
(model.bytes.clone().unwrap(), model.style_vectors.clone())
};
self.unload(ident.clone());
let s = model::load_model(&bytes, false)?;
if let Some(max) = self.max_loaded_models {
if self.models.iter().filter(|x| x.vits2.is_some()).count() >= max {
self.unload(self.models.first().unwrap().ident.clone());
}
}
self.models.push(TTSModel {
bytes: Some(bytes.to_vec()),
vits2: Some(s),
style_vectors,
ident: ident.clone(),
});
let model = self
.models
.iter()
.find(|m| m.ident == ident)
.ok_or(Error::ModelNotFoundError(ident.to_string()))?;
if model.vits2.is_some() {
return Ok(true);
}
Err(Error::ModelNotFoundError(ident.to_string()))
}
/// Get style vector by style id and weight
///
@@ -167,12 +278,19 @@ impl TTSModelHolder {
/// let audio = tts_holder.easy_synthesize("tsukuyomi", "こんにちは", 0, SynthesizeOptions::default())?;
/// ```
pub fn easy_synthesize<I: Into<TTSIdent> + Copy>(
&self,
&mut self,
ident: I,
text: &str,
style_id: i32,
speaker_id: i64,
options: SynthesizeOptions,
) -> Result<Vec<u8>> {
self.find_and_load_model(ident)?;
let vits2 = &self
.find_model(ident)?
.vits2
.as_ref()
.ok_or(Error::ModelNotFoundError(ident.into().to_string()))?;
let style_vector = self.get_style_vector(ident, style_id, options.style_weight)?;
let audio_array = if options.split_sentences {
let texts: Vec<&str> = text.split('\n').collect();
@@ -183,14 +301,17 @@ impl TTSModelHolder {
}
let (bert_ori, phones, tones, lang_ids) = self.parse_text(t)?;
let audio = model::synthesize(
&self.find_model(ident)?.vits2,
vits2,
bert_ori.to_owned(),
phones,
Array1::from_vec(vec![speaker_id]),
tones,
lang_ids,
style_vector.clone(),
options.sdp_ratio,
options.length_scale,
0.677,
0.8,
)?;
audios.push(audio.clone());
if i != texts.len() - 1 {
@@ -204,47 +325,21 @@ impl TTSModelHolder {
} else {
let (bert_ori, phones, tones, lang_ids) = self.parse_text(text)?;
model::synthesize(
&self.find_model(ident)?.vits2,
vits2,
bert_ori.to_owned(),
phones,
Array1::from_vec(vec![speaker_id]),
tones,
lang_ids,
style_vector,
options.sdp_ratio,
options.length_scale,
0.677,
0.8,
)?
};
tts_util::array_to_vec(audio_array)
}
/// Synthesize text to audio
///
/// # Note
/// This function is for low-level usage, use `easy_synthesize` for high-level usage.
#[allow(clippy::too_many_arguments)]
pub fn synthesize<I: Into<TTSIdent>>(
&self,
ident: I,
bert_ori: Array2<f32>,
phones: Array1<i64>,
tones: Array1<i64>,
lang_ids: Array1<i64>,
style_vector: Array1<f32>,
sdp_ratio: f32,
length_scale: f32,
) -> Result<Vec<u8>> {
let audio_array = model::synthesize(
&self.find_model(ident)?.vits2,
bert_ori.to_owned(),
phones,
tones,
lang_ids,
style_vector,
sdp_ratio,
length_scale,
)?;
tts_util::array_to_vec(audio_array)
}
}
/// Synthesize options

View File

@@ -10,14 +10,14 @@ importers:
dependencies:
onnxruntime-web:
specifier: ^1.19.2
version: 1.19.2
version: 1.20.0
devDependencies:
'@biomejs/biome':
specifier: ^1.9.2
version: 1.9.4
'@types/node':
specifier: ^22.7.4
version: 22.7.7
version: 22.8.0
esbuild:
specifier: ^0.24.0
version: 0.24.0
@@ -254,8 +254,8 @@ packages:
'@protobufjs/utf8@1.1.0':
resolution: {integrity: sha512-Vvn3zZrhQZkkBE8LSuW3em98c0FwgO4nxzv6OdSxPKJIEKY2bGbHn+mhGIPerzI4twdxaP8/0+06HBpwf345Lw==}
'@types/node@22.7.7':
resolution: {integrity: sha512-SRxCrrg9CL/y54aiMCG3edPKdprgMVGDXjA3gB8UmmBW5TcXzRUYAh8EWzTnSJFAd1rgImPELza+A3bJ+qxz8Q==}
'@types/node@22.8.0':
resolution: {integrity: sha512-84rafSBHC/z1i1E3p0cJwKA+CfYDNSXX9WSZBRopjIzLET8oNt6ht2tei4C7izwDeEiLLfdeSVBv1egOH916hg==}
esbuild@0.24.0:
resolution: {integrity: sha512-FuLPevChGDshgSicjisSooU0cemp/sGXR841D5LHMB7mTVOmsEHcAxaH3irL53+8YDIeVNQEySh4DaYU/iuPqQ==}
@@ -271,11 +271,11 @@ packages:
long@5.2.3:
resolution: {integrity: sha512-lcHwpNoggQTObv5apGNCTdJrO69eHOZMi4BNC+rTLER8iHAqGrUVeLh/irVIM7zTw2bOXA8T6uNPeujwOLg/2Q==}
onnxruntime-common@1.19.2:
resolution: {integrity: sha512-a4R7wYEVFbZBlp0BfhpbFWqe4opCor3KM+5Wm22Az3NGDcQMiU2hfG/0MfnBs+1ZrlSGmlgWeMcXQkDk1UFb8Q==}
onnxruntime-common@1.20.0:
resolution: {integrity: sha512-9ehS4ul5fBszIcHhfxuDgk45lO+Fqrxmrgwk1Pxb1JRvbQiCB/v9Royv95SRCWHktLMviqNjBsEd/biJhd39cg==}
onnxruntime-web@1.19.2:
resolution: {integrity: sha512-r0ok6KpTUXR4WA+rHvUiZn7JoH02e8iS7XE1p5bXk7q3E0UaRFfYvpMNUHqEPiTBMuIssfBxDCQjUihV8dDFPg==}
onnxruntime-web@1.20.0:
resolution: {integrity: sha512-IoUf8dqHFJLV4DUSz+Ok+xxyN6cQk57gb20m6PZE5gag3QXuvegYMq9dG8t/QF4JjTKIwvfvnr16ouzCCB9IMA==}
platform@1.3.6:
resolution: {integrity: sha512-fnWVljUchTro6RiCFvCXBbNhJc2NijN7oIQxbwsyL0buWJPG85v81ehlHI9fXrJsMNgTofEoWIQeClKpgxFLrg==}
@@ -424,7 +424,7 @@ snapshots:
'@protobufjs/utf8@1.1.0': {}
'@types/node@22.7.7':
'@types/node@22.8.0':
dependencies:
undici-types: 6.19.8
@@ -461,14 +461,14 @@ snapshots:
long@5.2.3: {}
onnxruntime-common@1.19.2: {}
onnxruntime-common@1.20.0: {}
onnxruntime-web@1.19.2:
onnxruntime-web@1.20.0:
dependencies:
flatbuffers: 1.12.0
guid-typescript: 1.0.9
long: 5.2.3
onnxruntime-common: 1.19.2
onnxruntime-common: 1.20.0
platform: 1.3.6
protobufjs: 7.4.0
@@ -486,7 +486,7 @@ snapshots:
'@protobufjs/path': 1.1.2
'@protobufjs/pool': 1.1.0
'@protobufjs/utf8': 1.1.0
'@types/node': 22.7.7
'@types/node': 22.8.0
long: 5.2.3
typescript@5.6.3: {}