Neural network models for whisper to normal speech conversion

• Type: Article in International Scientific Journal
• Date: 2025

Authors

Yamamura, F (Author) Other; Scalassara, R (Author) Other; Oliveira, A (Author) Other; Aníbal Ferreira (Author) FEUP

Journal

Title: U.Porto Journal of Engineering

Vol. 11

Pages: 116-129

Publisher: Universidade do Porto. Faculdade de Engenharia

Indexing

Scopus - 0 Citations

Other information

Authenticus ID: P-018-HWJ

DOI: 10.24840/2183-6493_0011-001_002739

Abstract (EN): Whispers are common and essential for secondary communication. Nonetheless, individuals with aphonia, including laryngectomees, rely on whispers as their primary means of communication. Due to the distinct features between whispered and regular speech, debates have emerged in the field of speech recognition, highlighting the challenge of effectively converting between them. This study investigates the characteristics of whispered speech and proposes a system for converting whispered vowels into normal ones. The system is developed using multilayer perceptron networks and two types of generative adversarial networks. Three metrics are analyzed to evaluate the performance of the system: mel-cepstral distortion, root mean square error of the fundamental frequency, and accuracy with f1-score of a vowel classifier. Overall, the perceptron networks demonstrated better results, with no significant differences observed between male and female voices or the presence/absence of speech silence, except for improved accuracy in estimating the fundamental frequency during the conversion process. © 2025, Universidade do Porto - Faculdade de Engenharia. All rights reserved.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 13