Audiovisual Classification of Group Emotion Valence Using Activity Recognition Networks

• Type: Article in International Conference Proceedings Book
• Date: 2020

Authors

Pinto, JR (Author) Other; Gonçalves, T (Author) Other; Pinto, C (Author) Other; Sanhudo, L (Author) Other; Fonseca, J (Author) Other; Gonçalves, F (Author) Other; Carvalho, P (Author) Other; Jaime S Cardoso (Author) FEUP

Conference proceedings International

Title: 4th IEEE International Conference on Image Processing, Applications and Systems, IPAS 2020, Virtual Event, Italy, December 9-11, 2020

Pages: 114-119

4th IEEE International Conference on Image Processing, Applications and Systems, IPAS 2020

9 December 2020 through 11 December 2020

Indexing

Scopus - 2 Citations

Other information

Authenticus ID: P-00T-BWN

DOI: 10.1109/ipas50080.2020.9334943

Abstract (EN): Despite recent efforts, accuracy in group emotion recognition is still generally low. One of the reasons for these underwhelming performance levels is the scarcity of available labeled data which, like the literature approaches, is mainly focused on still images. In this work, we address this problem by adapting an inflated ResNet-50 pretrained for a similar task, activity recognition, where large labeled video datasets are available. Audio information is processed using a Bidirectional Long Short-Term Memory (Bi-LSTM) network receiving extracted features. A multimodal approach fuses audio and video information at the score level using a support vector machine classifier. Evaluation with data from the EmotiW 2020 AV Group-Level Emotion sub-challenge shows a final test accuracy of 65.74% for the multimodal approach, approximately 18% higher than the official baseline. The results show that using activity recognition pretraining offers performance advantages for group-emotion recognition and that audio is essential to improve the accuracy and robustness of video-based recognition. © 2020 IEEE.

Language: English

Type (Professor's evaluation): Scientific