Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -257,6 +257,12 @@ add_library(engine_runtime STATIC
src/models/omnivoice/assets.cpp
src/models/omnivoice/tokenizer_text.cpp
src/models/omnivoice/audio_tokenizer.cpp
src/models/outetts/assets.cpp
src/models/outetts/dac.cpp
src/models/outetts/llama.cpp
src/models/outetts/loader.cpp
src/models/outetts/session.cpp
src/models/outetts/tokenizer.cpp
src/models/omnivoice/prompt_builder.cpp
src/models/omnivoice/generator.cpp
src/models/omnivoice/postprocess.cpp
Expand Down Expand Up @@ -716,6 +722,7 @@ if (ENGINE_BUILD_WARMBENCH)
add_engine_warmbench(moss_tts_local_warm_bench tests/moss_tts_local/moss_tts_local_warm_bench.cpp)
add_engine_warmbench(nemotron_asr_warm_bench tests/nemotron_asr/nemotron_asr_warm_bench.cpp)
add_engine_warmbench(omnivoice_warm_bench tests/omnivoice/omnivoice_warm_bench.cpp)
add_engine_warmbench(outetts_warm_bench tests/outetts/outetts_warm_bench.cpp)
add_engine_warmbench(pocket_tts_warm_bench tests/pocket_tts/pocket_tts_warm_bench.cpp)
add_engine_warmbench(qwen3_asr_warm_bench tests/qwen3_asr/qwen3_asr_warm_bench.cpp)
add_engine_warmbench(qwen3_forced_aligner_warm_bench tests/qwen3_forced_aligner/qwen3_forced_aligner_warm_bench.cpp)
Expand Down Expand Up @@ -785,6 +792,14 @@ if (ENGINE_BUILD_TESTS)
COMMAND audio_chunking_test
)

add_engine_unittest(outetts_generation_budget_test tests/unittests/test_outetts_generation_budget.cpp)
target_include_directories(outetts_generation_budget_test PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/tests/unittests)

add_test(
NAME outetts_generation_budget_test
COMMAND outetts_generation_budget_test
)

add_engine_unittest(subtitle_formatter_test tests/unittests/test_subtitle_formatter.cpp)

add_test(
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ audio.cpp would not be moving this quickly without generous contributors bringin
| **miocodec** | audio codec, voice conversion backend | lang agnostic | MioCodec v2, 25 Hz, 44.1 kHz |
| **miotts** | TTS, voice cloning | en, ja | MioTTS-1.7B |
| **omnivoice** | TTS, voice cloning, voice design | 646+ langs | OmniVoice, Qwen3-0.6B based |
| **outetts** | TTS, voice cloning | en, ar, zh, nl, fr, de, it, ja, ko, lt, ru, es, pt, be, bn, ka, hu, lv, fa, pl, sw, ta, uk | Llama-OuteTTS-1.0-1B |
| **pocket_tts** | TTS, voice cloning | en, de, it, pt, es | PocketTTS-100M |
| **nemotron_asr** | ASR | 100+ ASR prompt codes incl. auto | Nemotron 3.5 ASR Streaming 0.6B |
| **qwen3_asr** | ASR | zh, en, yue, ar, de, fr, es, pt, id, it, ko, ru, th, vi, ja, tr, hi, ms, nl, sv, da, fi, pl, cs, fil, fa, el, ro, hu, mk | Qwen3-ASR-0.6B, Qwen3-ASR-1.7B-hf |
Expand Down Expand Up @@ -386,6 +387,7 @@ Recommended top-level install packages:
| `moss_tts_local_v1_5` | MOSS-TTS-Local Transformer v1.5 | No |
| `nemotron_asr` | Nemotron ASR | **Yes** |
| `omnivoice` | OmniVoice | **Yes** |
| `outetts_1_0_1b` | OuteTTS 1.0 1B with IBM DAC codec and Qwen3-aligned voice cloning | No |
| `parakeet_tdt_0_6b_v3` | Parakeet TDT 0.6B v3 | **Yes** |
| `pocket_tts` | PocketTTS | **Yes** |
| `qwen3_asr_0_6b` | Qwen3 ASR 0.6B | **Yes** |
Expand Down Expand Up @@ -639,10 +641,10 @@ The framework also has a reusable GGUF tensor source and a streaming converter.
container reader is shared by all model families; a family still has to list a `.gguf`
checkpoint as one of its accepted assets because model configuration and tensor naming
remain architecture-specific. Qwen3 ASR, Qwen3 Forced Aligner, Qwen3 TTS, Nemotron
3.5 ASR, VibeVoice-ASR, Higgs Audio STT, Hviske ASR, and Citrinet ASR currently accept
3.5 ASR, VibeVoice-ASR, Higgs Audio STT, Hviske ASR, Citrinet ASR, and OuteTTS currently accept
`model.gguf` (including `speech_tokenizer/model.gguf` for TTS). The converter recursively embeds sidecar files
up to 64 MiB by default using binary-safe metadata, including nested tokenizer models,
and Qwen3 ASR, Nemotron ASR, VibeVoice-ASR, Higgs Audio STT, Hviske ASR, and Citrinet ASR
and Qwen3 ASR, Nemotron ASR, VibeVoice-ASR, Higgs Audio STT, Hviske ASR, Citrinet ASR, and OuteTTS
can load the resulting `model.gguf` as a standalone file. The converter embeds the selected
package spec in new GGUF files. Standalone conversion with embedded sidecars is the default
and fails if required package resources are missing. Pass `--no-sidecars` only to explicitly
Expand Down
1 change: 1 addition & 0 deletions docs/gguf.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,7 @@ Status labels:
| `moss_tts_nano` | Done | Pass | --- | Pass | No (similarity drift, frame drift, text large drift) |
| `nemotron_asr` | Done | Pass | --- | Pass | Pass (minor filler drift) |
| `omnivoice` | Done | Pass | --- | No (runtime assert, no audio) | No (runtime assert, no audio) |
| `outetts` | Done | Pass (TTS + clone) | --- | --- | Pass (TTS + clone) |
| `pocket_tts` | No | --- | --- | --- | --- |
| `qwen3_asr` | Done | Pass | --- | Pass | Pass |
| `qwen3_forced_aligner` | Done | Pass | --- | Pass | Pass |
Expand Down
3 changes: 3 additions & 0 deletions docs/memory_saver.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ Native/default weights were used for all rows.
- VoxCPM2 used the OpenAI-compatible offline speech endpoint with a 2048-character voice-design request, `seed=1234`, `max_tokens=512`, `num_inference_steps=10`, and `guidance_scale=2.0`. The default and `mem_saver` WAV outputs were byte-identical.
- IndexTTS2 used a five-request server sequence with the same seeds and references for default and `mem_saver`: normal text, longer text, longer emotion-text request with a different reference, shorter text, then longer text.
- Irodori TTS 500M used a five-request server sequence with the same seeds and options for default and `mem_saver`: reference text, longer reference text, longer no-reference emoji/style text, shorter reference text, then longer reference text.
- OuteTTS 1.0 1B Q8 used a five-request long-lived-session sequence on an RTX 3090: repeated fixed-seed TTS, four-chunk long-form TTS, then repeated fixed-seed cloning with the same reference. CUDA timing is the mean of three fresh processes per mode, alternated to reduce ordering bias; memory saver was 0.35-0.53% slower per request and 0.44% slower over the mean sequence total. This is close to run-to-run variance and is not evidence of a speed benefit. The default and `mem_saver` WAV outputs were byte-identical. VRAM is total-device usage with no other CUDA workload; resident VRAM was sampled during a five-second post-sequence hold. Per-request timing and RTF are recorded in `docs/outetts_validation.md`.

| Model | Mode | Peak VRAM | Resident VRAM | Server wall | Audio | RTF |
|---|---|---:|---:|---:|---:|---:|
Expand Down Expand Up @@ -58,3 +59,5 @@ Native/default weights were used for all rows.
| Irodori TTS 500M | mem_saver | 11222 MiB | 3570 MiB | 3276.203 ms | 95.4s | 0.0343418 |
| Irodori TTS 500M 6000-char | default | 18693 MiB | 13367 MiB | 27185.7 ms | 777.92s | 0.0349466 |
| Irodori TTS 500M 6000-char | mem_saver | 11588 MiB | 3609 MiB | 27828 ms | 777.92s | 0.0357724 |
| OuteTTS 1.0 1B Q8 | default | 17653 MiB | 294 MiB | 29911.38 ms | 11.237s | 2.662 |
| OuteTTS 1.0 1B Q8 | mem_saver | 5780 MiB | 294 MiB | 30043.25 ms | 11.237s | 2.674 |
Loading
Loading