Bench for local text-to-speech (TTS) models. Three lenses, on whatever hardware you put it on:
- Speed — cold + warm TTFA (time to first audio), RTFx (realtime speed; higher = faster than realtime), memory, on CPU / CUDA / Apple Silicon
- Listen — every model on every prompt, default voice + voice cloning, with inline audio players, so you can pick a model by ear
- Scores — objective metrics per model: UTMOS (naturalness), WER (intelligibility), SIM (cloning fidelity), scored over the bench prompts via seed-tts-eval-style ASR + speaker-verification. Sortable, with a Default/Cloning toggle.
An objective quality score (NAQ) was prototyped but isn't part of the bench — the v2 features didn't track subjective ranking closely enough to publish, so it was pulled and is being redesigned separately. The bench measures speed; quality is by-ear via the Listen lens.
5uck1ess.github.io/tts-bench — listen to every model, no install. Three lenses:
- Listen — one consolidated gallery with an inline
<audio>player for every model on every prompt, in default voice and voice cloning (each clone sits next to the reference it's imitating). Browse by prompt (compare all models on one sentence) or by model (audition one model across prompts); only one clip plays at a time. Audio is rig-independent, so each sample is sourced once from the highest-fidelity rig and tagged with where it came from. Quality, prosody, and artifacts are obvious in 5 seconds — benchmark tables can't show that. - Speed — per-rig leaderboards (Ryzen 9 9950X3D + RTX 5090, Apple M4, Ryzen + RTX 3090) with cold/warm TTFA, RTFx, and memory, sortable. Pick the box you actually own.
- Scores — objective metrics per model (UTMOS naturalness, WER intelligibility, SIM cloning fidelity), sortable, with a Default/Cloning toggle. Human votes remain the preference ground truth; these are objective backstops.
Full per-rig reports (every model × prompt × device, plus by-prompt samples) are linked from the Archive.
Quality is subjective, so the ground truth is your ears. The companion TTS Voting Arena is a public, blind A/B listening test — two clips, no model names shown, pick the one that sounds better. No login, ~5 seconds a vote.
- Default voice — which model sounds more natural?
- Cloning — which clone better matches the reference voice?
Votes feed a live human-preference Elo leaderboard right there on the arena. This is where the "best sounding" and cloning calls above come from — every vote sharpens the ranking.
Requires uv and Python 3.11. ~10-15 min install. Disk for the full set is large: ~39 GB of per-model venvs in the repo, plus ~125 GB of model weights downloaded to your Hugging Face cache (~/.cache/huggingface, not the repo) — ~165 GB all-in. Individual models are far smaller, so installing a subset costs a fraction of that.
# Windows — everything, or just the models you want
.\install.ps1
.\install.ps1 kokoro,piper,miso
python bench.py# macOS / Linux — everything, or just the models you want
./install.sh
./install.sh kokoro piper miso
python bench.pyPass model names to install only those (names = the venvs/<name> slugs, which match the tables below — lowercase, e.g. kokoro, f5tts, chatterbox, miso). A few share one install: neutts covers NeuTTS Air + Nano, chatterbox both ChatterBox variants, vibevoice the 0.5B/1.5B, moss_tts both MOSS checkpoints, fish is Fish Speech 1.5. Add scoring (plus scoring_sim on Linux) for the objective-metrics venv. bench.py only runs models whose venv exists, so a partial install benches cleanly — install more models later by re-running with new names.
Interactive feel-test: python speak.py kokoro. One-shot A/B comparison: python compare.py "your phrase". See docs/architecture.md for the runner protocol and how to add a model.
Fastest:
- CPU (Ryzen 9 9950X3D, Windows): Piper — 107ms warm TTFA, 59× RTFx
- CUDA (RTX 5090): Kokoro — 67ms warm TTFA, 104× RTFx
- CPU + MPS (Apple M4, 16 GB): Piper — 208ms warm TTFA, 32× RTFx
Best sounding: No objective ranking right now — the NAQ score is paused pending redesign. Open the Demos site and use the Listen lens.
Best cloning — blind A/B votes (these measure voice-match preference, not intelligibility):
-
- OmniVoice — top on voice/accent match (24-1-3), but it can garble or drop words; a timbre-focused A/B vote doesn't penalize that, so read this as "best voice match," not "best overall clone." Audition it first — objective WER (the new Scores lens) is meant to catch exactly this gap.
-
- Echo-TTS — near-tied #1 (21-1-6), clean 44.1 kHz
-
- IndexTTS-2 — third (16-2-5), accent held
→ full per-rig results · → full cloning ranking
| Model | Params | Released | Predefined | Cloning | Multilingual | SR | Expressive | License |
|---|---|---|---|---|---|---|---|---|
| KittenTTS Nano 0.1 | <100M | Aug 2025 | ✓ | — | — | 24k | — | Apache 2.0 |
| Kokoro | 82M | Dec 2024 | ✓ | — | ✓ | 24k | — | Apache 2.0 |
| LFM2.5-Audio 1.5B | 1.5B | Dec 2025 | ✓ (4) | — | — (en) | 24k | — | LFM Open v1.0 |
| LuxTTS | 123M | Jan 2026 | ✓ | — | — | 22.05k | — | MIT |
| Magpie-TTS | 357M | Dec 2025 | ✓ | — | ✓ (9) | 22.05k | emotion voices* | NVIDIA OML |
| Maya1 | 3B | Oct 2025 | ✓ (voice desc) | — | — | 24k | tags + desc | Apache 2.0 |
| MeloTTS | ~52M | Feb 2024 | ✓ | — | — (en) | 44.1k | — | MIT |
| Orpheus TTS | 3B | Mar 2025 | ✓ (8) | — | — (en) | 24k | tags | Apache 2.0 |
| OuteTTS 1.0 1B | ~1B | Apr 2025 | ✓ | ✓ | ✓ (12) | 44.1k | — | CC-BY-NC-SA 4.0 + Llama 3.2 |
| Parler-TTS Mini v1 | 878M | Jun 2024 | ✓ (voice desc) | — | — | 44.1k | desc* | Apache 2.0 |
| Piper | ~15M | Jan 2023 | ✓ | — | ✓ | 22.05k | — | GPL-3.0 |
| Soprano 1.1 80M | 80M | Jan 2026 | ✓ | — | — | 32k | — | Apache 2.0 |
| Supertonic 3 | 99M | May 2026 | ✓ | — | ✓ (31) | 24k | tags | MIT + OpenRAIL-M |
| VibeVoice Realtime 0.5B | 0.5B | Dec 2025 | ✓ | — | — | 24k | — | MIT |
| Voxtral 4B TTS | 4B | Nov 2025 | ✓ (20) | ✓ | ✓ | 24k | — | CC-BY-NC 4.0 |
| Model | Params | Released | Predefined | Cloning | Multilingual | SR | Expressive | License |
|---|---|---|---|---|---|---|---|---|
| ChatterBox | 1.2B | Apr 2025 | — | ✓ | — | 24k | knob | MIT |
| ChatterBox Turbo | 744M | Dec 2025 | — | ✓ | — | 24k | tags* | MIT |
| Coqui XTTS-v2 | 750M | Oct 2023 | — | ✓ | ✓ (17) | 24k | — | CPML (non-commercial) |
| CosyVoice 3 0.5B | 0.5B | Dec 2025 | — | ✓ | ✓ | 24k | desc | Apache 2.0 |
| Dia 1.6B-0626 | 1.6B | Jun 2025 | — | ✓ | — | 44.1k | tags | Apache 2.0 |
| dots.tts (soar) | 2B | Jun 2026 | — | ✓ | ✓ (24) | 48k | — | Apache 2.0 |
| DramaBox | 3.3B | Apr 2026 | — | ✓ | — (en) | 48k | desc | LTX-2 Community (NC) |
| Echo-TTS | ~2.8B | Dec 2025 | — | ✓ | — | 44.1k | tags | CC-BY-NC-SA 4.0 |
| F5-TTS v1 | 330M | Oct 2024 | — | ✓ | ✓ | 24k | — | CC-BY-NC |
| Fish Speech 1.5 | ~500M | Nov 2024 | — | ✓ | ✓ | 44.1k | — | CC-BY-NC-SA 4.0 |
| Fish Speech S2-Pro | 4B | Mar 2026 | — | ✓ | — | 44.1k | tags | Research (non-commercial) |
| Higgs Audio v3 TTS | 4B | Jun 2026 | — | ✓ | ✓ (100) | 24k | tags | Research (NC) |
| IndexTTS-2 | 1.5B | Jun 2025 | — | ✓ | ✓ | 24k | emo-ref + desc + knob | Apache 2.0 |
| LongCat-AudioDiT 1B | 1.42B | Mar 2026 | — | ✓ | ✓ (zh+en) | 24k | — | MIT |
| LongCat-AudioDiT 3.5B | 3.83B | Mar 2026 | — | ✓ | ✓ (zh+en) | 24k | — | MIT |
| Mars5-TTS | 1.2B | Jun 2024 | — | ✓ | — | 24k | — | AGPL-3.0 |
| MetaVoice-1B | 1.2B | Feb 2024 | — | ✓ | — | 48k | — | Apache 2.0 |
| MioTTS 0.1B | 0.1B | Feb 2026 | — | ✓ | ✓ (en+ja) | 44.1k | — | Falcon-LLM |
| MioTTS 0.6B | 0.6B | Feb 2026 | — | ✓ | ✓ (en+ja) | 44.1k | — | Apache 2.0 |
| MiraTTS | 0.5B | Dec 2025 | — | ✓ | — | 48k | knob | MIT |
| Miso TTS 8B | 8.2B | May 2026 | — | ✓ | — (en) | 24k | — | Modified MIT |
| MOSS-TTS v1.0 | 8B (Qwen3) | Feb 2026 | — | ✓ | ✓ (20) | 24k | — | Apache 2.0 |
| MOSS-TTS v1.5 | 8B (Qwen3) | May 2026 | — | ✓ | ✓ (31) | 24k | tags (pause) | Apache 2.0 |
| MOSS-TTS-Nano | 100M | Apr 2026 | — | ✓ | ✓ (zh+en) | 48k | — | Apache 2.0 |
| NeuTTS Air | 748M | Sep 2025 | — | ✓ | — | 24k | — | Apache 2.0 |
| NeuTTS Nano | 229M | Dec 2025 | — | ✓ | — | 24k | — | Apache 2.0 |
| OmniVoice | ~1B | Mar 2026 | — | ✓ | ✓ (600+) | 24k | tags* | Apache 2.0 code / CC-BY-NC weights |
| OpenVoice v2 | ~100M | Apr 2024 | — | ✓ | ✓ | 22.05k | knob | MIT |
| Pocket-TTS | 100M | Jan 2026 | — | ✓ | — | 24k | — | Apache 2.0 |
| Qwen3-TTS 1.7B Base | 1.7B | Jan 2026 | — | ✓ | ✓ | 24k | — | Apache 2.0 |
| Qwen3-TTS 1.7B (CUDA-graph) | 1.7B | Jan 2026 | — | ✓ | ✓ | 24k | — | MIT |
| Sesame CSM-1B | 1B | Mar 2025 | — | ✓ | — | 24k | — | Apache 2.0 |
| Step-Audio-EditX | 3B | Oct 2025 | — | ✓ | — | 24k | tags + desc | Apache 2.0 |
| StyleTTS 2 | ~148M | Jun 2023 | — | ✓ | — | 24k | knob | MIT |
| VibeVoice 1.5B | 1.5B | Aug 2025 | — | ✓ | — | 24k | — | MIT |
| VibeVoice 7B | 7B | Sep 2025 | — | ✓ | — | 24k | — | MIT |
| VoxCPM2 | 2B | Apr 2026 | — | ✓ | ✓ (30) | 48k | desc | Apache 2.0 |
| WavTTS | 0.67B | May 2026 | — | ✓ | ✓ (zh+en) | 16k | — | MIT code / CC-BY-NC 4.0 weights |
| ZipVoice | 123M | Jun 2025 | — | ✓ | ✓ (zh+en) | 24k | — | Apache 2.0 |
| Zonos v0.1 | 1.6B | Feb 2025 | — | ✓ | ✓ | 44.1k | emo-ref + knob | Apache 2.0 |
Expressive column — what explicit emotion/delivery control the model offers:
tags= inline cues in the text itself ((laughs),[sigh],<laugh>);desc= natural-language style/emotion instructions;knob= numeric or preset parameter (exaggeration, style enum, pitch/speed);emo-ref= emotion conditioned on a separate reference clip or emotion vector;—= none (for cloning models, expression simply follows the reference clip).*= caveat applies. Exact syntax, sources, and caveats per model: docs/expressive-control.md. Note the bench feeds every model the same plain prompts for fairness, so these features are not exercised in any score.
Full per-model gotchas + license details: docs/known-issues.md. Models considered but excluded: docs/considered.md.
Predefined vs Cloning. Predefined models have fixed/selectable speaker voices baked into the weights — they speak with no reference needed. Cloning (zero-shot) models have no voice of their own: they synthesize whatever voice you hand them as a reference clip at inference. Given no reference, a pure zero-shot model falls back to a bundled sample (this bench uses
chris_hemsworth_15s.wav), so its "default voice" is just a clone of that clip. A few models do both (e.g. Voxtral has 20 presets and cloning).
Rig availability: Voxtral is Mac (MLX, preset-voice only) + Linux (vLLM, cloning); Fish S2-Pro / MetaVoice / Step-Audio-EditX / Higgs Audio v3 / dots.tts / Orpheus / CosyVoice 3 are Linux-only (CUDA) — Higgs v3 is the one server-backed model (it runs via a Docker
sgl-omniHTTP server, not an in-process load), dots.tts is Linux-only because itsWeTextProcessing→pyninidependency won't build under Windows MSVC, and Orpheus (vLLM) + CosyVoice 3 (cu121 / torch 2.3.1) have no Blackwell-compatible Windows path; Echo-TTS and DramaBox are Windows + Linux (CUDA-only, no CPU/MPS; DramaBox needs ~18 GB VRAM). The rest run on Windows + Linux CUDA, most on CPU/MPS too. Per-rig speed + samples on the Demos site.
41 of the 55 tracked models can clone a voice from a reference clip. Three reference formats supported (wav only / wav + transcript / HF-gated wav). Drop a reference into reference/, then python bench.py --reference reference/myvoice.wav.
Reference-format docs + the blind-vote cloning ranking (28 of 32 cloning models, 397 votes, human-preference A/B): docs/cloning.md.
| Machine | Used for |
|---|---|
| Windows desktop (Ryzen 9 9950X3D / 128 GB / RTX 5090 32 GB) | Windows CPU + CUDA bench rows |
| Linux workstation (Ryzen 9 5900XT / 64 GB / RTX 3090 24 GB, Ubuntu Server 24.04) | Linux CPU + CUDA; the only rig that runs Fish-Speech S2 natively |
| Mac (Apple M4 / 16 GB / M4 GPU) | Mac CPU + MPS bench rows |
If you reproduce on different hardware, file an issue or PR with your results and we'll add a column.
- Full results tables — per-rig, per-prompt, per-model
- Cloning ranking — reference formats + blind-vote ranking (28 of 32 cloning models, human-preference A/B)
- Architecture — bench design, runner protocol, adding a model
- Expressive control — which models take emotion tags / style prompts / knobs, with exact syntax
- Known issues — per-model gotchas + per-license table
- Considered but skipped — models evaluated and excluded
- Tasks & pending work — open issues, planned features
- Methodology — what's measured, why cold + warm, why reproducible
MIT for the bench code in this repo. Each TTS model has its own license — see docs/known-issues.md for the full per-model table.
If this bench saved you a weekend of writing your own:
