Naturalness of Synthesized Speech: A Subjective-Comparative Study Utilizing ACR Listening-Opinion Tests between Siri and Google Translate Using News Content - Daengsi, Therdpong; Pornpongtechavanich, Phisit (Burapha Science Journal, 2026)
Text-To-Speech synthesis (TTS) is one of the most important technologies based on human language processing at present. However, one of interesting open issues about synthesized speech is naturalness of speech. This article presents the application of a speech quality assessment method to assess the naturalness of Thai synthesized speech from two popular TTS systems, Siri and Google Translate. For methodology, Thai synthesized speech, associated with two royal news and two general news (COVID-19), provided by Siri and Google Translate have been assessed by sixteen Thai male volunteers and sixteen Thai female volunteers. It has been found the overall result that the value of the naturalness - Mean Opinion Score (MOS) of Thai synthesized speech provided by Google Translate is 3.53 ± 0.67, which is higher than the value of 3.16 ± 0.77 provided by Siri. Furthermore, after the statistical analysis using t-test, it is been found that the p-value is 0.037. In conclusion, the speech synthesis engine in Google Translate provides better naturalness than the one in Siri significantly. Therefore, the methodology in this article can be applied to assess naturalness level of other applications/services/systems in order to improve synthesized speech quality.