TTS and Voice Cloning: What Makes a Speaker Corpus Fit for Purpose
20
TTS and Voice Cloning: What Makes a Speaker Corpus Fit for Purpose
Text-to-speech and voice cloning training corpora have fundamentally different requirements than ASR training data. ASR wants breadth: many speakers, many environments, many accents. TTS wants depth: acoustic fidelity, verbatim transcript exactness, prosodic range, and explicit contributor consent for synthesis of an identifiable voice. This post covers what separates a fit-for-purpose TTS corpus from a repurposed ASR dataset, and the specific dimensions to demand from a vendor in 2026.
TL;DR
TTS training requires acoustic fidelity, verbatim transcripts, prosodic range, and consent for identifiable-voice synthesis; ASR corpora meet none of these by default.
Production TTS training data needs 44.1 kHz sample rate minimum, 24-bit depth, controlled acoustic environment, noise floor -50 dB or better.
Single-speaker TTS typically needs 10 to 30 hours; foundation TTS pretraining needs 10,000 to 100,000+ hours multi-speaker.
Voice cloning consent is a distinct scope from general AI training consent; voice actors have won claims where this was conflated.
Silencio operates both real-world and controlled-capture programs, with active single-speaker and multi-speaker TTS corpus commissioning across low-resource languages.
Why can't you use an ASR corpus to train TTS?
Because the property requirements diverge in every dimension.
Property | ASR corpus | TTS corpus |
|---|---|---|
Acoustic fidelity | Robust to noise, phone-quality acceptable | 44.1 kHz+, controlled noise floor |
Transcript exactness | Naturalistic, disfluencies optional | Verbatim, every word matches audio |
Prosodic range | Not required | Statements, questions, emphasis, emotion |
Consent scope | AI training | Identifiable-voice synthesis |
Acoustic fidelity. ASR is robust to noise, low bit rate, and lossy compression. TTS is not. A model trained on 16 kHz phone-captured audio produces synthetic speech that sounds like phone-captured audio. For premium TTS, training data at 44.1 kHz or higher, 24-bit or 32-bit depth, with a controlled noise floor is required.
Transcript exactness. ASR training benefits from transcripts that reflect what was actually said, including disfluencies, in a naturalistic form. TTS training requires verbatim transcripts that exactly match the audio: every word, every filler, every hesitation, tagged where appropriate. Sentence-level non-verbatim transcripts are essentially unusable for TTS.
Prosodic range. For expressive TTS, the training data has to include the prosodic range the model needs to reproduce: statements, questions, exclamations, contrastive emphasis, emotional coloring. General-purpose ASR corpora rarely span this range.
Consent for synthesis of an identifiable voice. ASR training uses voice recordings as input to a model that transcribes speech, without reproducing the speaker's voice. TTS and voice cloning specifically reproduce the speaker's voice. Contributor consent for voice-training use must specifically cover synthesis and identifiable-voice reproduction.
What are the acoustic fidelity requirements for TTS?
The floor for production-grade TTS training data.
Property | Minimum for production TTS |
|---|---|
Sample rate | 44.1 kHz (48 kHz preferred) |
Bit depth | 24-bit (16-bit acceptable for some tiers) |
Noise floor | -50 dB or better |
Recording environment | Controlled (studio or well-treated home studio) |
Microphone | Large-diaphragm condenser or equivalent broadcast-grade |
Post-processing | Minimal (no compression, no aggressive EQ) |
Voice cloning has similar or stricter requirements, plus a controlled speaker-recording protocol (consistent mic distance, consistent room acoustic across sessions).
Note: this is the opposite of the real-world capture requirement for ASR. ASR wants real-world; TTS wants controlled. Buyers doing both need separate corpora for each.
What are the transcript requirements?
Verbatim transcripts that match the audio word-for-word, including:
All spoken words in their exact form
All disfluencies (uh, um, ah) where present
Sentence-level punctuation that reflects speaker intonation
Optionally: tone or prosody tags for expressive TTS
Machine-generated transcripts are inadequate as a starting point for TTS training. Human native-speaker transcription with verification is the production standard.
What speaker corpus size is required?
Depends on TTS type.
TTS type | Volume required |
|---|---|
Single-speaker TTS | 10 to 30 hours from a single speaker, controlled environment, verbatim transcripts, prosodic range coverage |
Multi-speaker TTS (voice selection) | 5 to 20 hours per speaker, across 10 to 100 speakers, with per-speaker metadata |
Voice cloning (few-shot) | 30 seconds to 5 minutes per target speaker + base model trained on 500 to 5,000 hours multi-speaker |
Foundation TTS model pretraining | 10,000 to 100,000+ hours multi-speaker for the base model, then downstream fine-tuning |
What are the consent-for-synthesis requirements?
Explicit consent for identifiable-voice synthesis is a distinct scope from general voice AI training. The consent flow should specifically disclose:
The contributor's voice will be used to train a model that reproduces their voice or a synthetic voice
Whether the synthetic voice will be identifiable as the contributor
Downstream uses of the synthetic voice (commercial products, entertainment, accessibility, dubbing)
Contributor's ability to withdraw consent and require deletion or de-training
Contributor compensation model (per-hour, royalty, or hybrid)
Voice actors and professional speakers have brought successful claims against AI providers that trained voice cloning models on their commercial recordings without this specific consent scope. Enterprise buyers of voice cloning training data need indemnification against this exposure.
Common misconceptions about TTS and voice cloning data
Three assumptions surface repeatedly in buyer conversations and are wrong.
Misconception 1: "A large ASR corpus can be filtered for the high-quality subset and reused for TTS." The high-quality subset of an ASR corpus is still typically 16 kHz phone audio, with naturalistic (non-verbatim) transcripts, and consent scoped to ASR training. Filtering does not upgrade sample rate, retroactively verify transcripts, or expand the consent scope. TTS needs its own corpus, collected against TTS requirements.
Misconception 2: "Few-shot voice cloning bypasses the training-data problem." Only for the reference audio. The base model behind the few-shot cloner still needs a large, controlled, consent-cleared multi-speaker corpus. The few-shot approach reduces the enrollment burden per new voice, not the base training burden.
Misconception 3: "Voice cloning consent is covered by our general AI terms of service." Only if the terms explicitly disclose identifiable-voice synthesis as a purpose. Where they do not, voice actors and professional speakers have won claims. General AI-training consent language rarely covers this scope, and courts are increasingly treating identifiable-voice synthesis as a separate disclosure requirement.
What is the low-resource-language TTS opportunity?
TTS quality in languages below the top ten is severely bottlenecked by data. High-quality expressive TTS in Bengali, Vietnamese, Swahili, Amharic, and dozens of comparable languages does not exist at production quality because the training data does not exist at the volume and controlled-acoustic conditions required.
The demand is growing sharply: accessibility applications (voice output for the visually impaired, alternative and augmentative communication for the speech-impaired), dubbing and localization, voice agents in emerging markets, and educational applications. Buyers commissioning single-speaker or multi-speaker TTS corpora in low-resource languages are the current frontier.
Silencio's contributor network can source native speakers across most low-resource languages for scoped TTS collection, in either home-studio controlled environments or full studio depending on the buyer's fidelity requirements.
What should buyers demand from a TTS/voice cloning data vendor?
At minimum:
Acoustic fidelity documentation (sample rate, bit depth, noise floor) per speaker corpus
Verbatim transcripts with prosody tags where required
Per-speaker consent artifact covering identifiable-voice synthesis
DPA covering the specific downstream use (product, dubbing, accessibility, etc.)
Warranty/indemnification for the voice-synthesis consent scope
Sample per-speaker recordings before commitment
For voice cloning: documented protocol for reference-audio collection
Silencio operates both real-world and controlled-capture programs, with active single-speaker and multi-speaker TTS corpus commissioning across low-resource languages and explicit synthesis-scope consent documentation.