Friday 31 January 2025
Artificial intelligence has made tremendous progress in recent years, and one of the most impressive achievements is the development of text-to-speech (TTS) systems that can generate natural-sounding speech from written text. The latest breakthrough comes from a team of researchers who have created a system that can produce high-quality spoken language without requiring extensive training data.
The system, which was developed for the CoVoC Challenge 2024, uses a combination of cutting-edge technologies to achieve impressive results. At its core is a LLaMA-based codec language model, which is capable of generating natural-sounding speech from text inputs. This model is trained on large-scale datasets and fine-tuned using high-quality audio data.
To improve the system’s ability to generate spontaneous speech, the researchers introduced a delay pattern into the model’s architecture. This allows it to better capture the nuances of spoken language, including pauses, hesitations, and other natural variations.
The team also used a technique called Classifier-Free Guidance (CFG) to enhance the model’s conditional guidance on token prediction. This involves masking the text encoding sequence or semantic token sequence during training, which helps the model learn to generate more coherent and natural-sounding speech.
To test the system’s capabilities, the researchers generated 12,800 hours of Mandarin TTS data using a dataset called WenetSpeech4TTS. They then fine-tuned the model on high-quality spontaneous speech data from the HQ-Conversations dataset.
The results were impressive: the system achieved an average MOS score of 3.61, ranking third in the constrained track of the CoVoC Challenge 2024. The system also excelled in terms of speech quality and similarity, with a MOS score of 3.84 and a rank of second.
One notable aspect of this research is its ability to generate high-quality spoken language without requiring extensive training data. This could have significant implications for TTS systems, which are increasingly being used in applications such as voice assistants and automated customer service agents.
The researchers’ approach also demonstrates the potential of using pre-trained language models as a foundation for TTS systems. By leveraging these models and fine-tuning them on high-quality audio data, it may be possible to develop more accurate and natural-sounding TTS systems without requiring large amounts of training data.
Overall, this research represents an important milestone in the development of TTS systems and highlights the potential of using cutting-edge AI technologies to generate high-quality spoken language.
Cite this article: “Advances in Text-to-Speech Technology: A Breakthrough in High-Quality Spoken Language Generation”, The Science Archive, 2025.
Tts, Covoc Challenge 2024, Llama-Based Codec, Natural-Sounding Speech, Classifier-Free Guidance, Wenetspeech4Tts, Hq-Conversations, Mos Score, Mandarin, Language Models
