Multimodal Learning for Sign Language Recognition

• Type: Article in International Conference Proceedings Book
• Date: 2017

Authors

Ferreira, PM (Author) Other; Jaime S Cardoso (Author) FEUP; Rebelo, A (Author) Other

Conference proceedings International

Title: PATTERN RECOGNITION AND IMAGE ANALYSIS (IBPRIA 2017)

Pages: 313-321

8th Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA)

Univ Algarve, Faro, PORTUGAL, JUN 20-23, 2017

Indexing

ISI Web of Knowledge - 3 Citations

Scopus - 4 Citations

Other information

Authenticus ID: P-00N-B26

DOI: 10.1007/978-3-319-58838-4_35

Abstract (EN): Sign Language Recognition (SLR) has becoming one of the most important research areas in the field of human computer interaction. SLR systems are meant to automatically translate sign language into text or speech, in order to reduce the communicational gap between deaf and hearing people. The aim of this paper is to exploit multimodal learning techniques for an accurate SLR, making use of data provided by Kinect and Leap Motion. In this regard, single-modality approaches as well as different multimodal methods, mainly based on convolutional neural networks, are proposed. Experimental results demonstrate that multimodal learning yields an overall improvement in the sign recognition performance.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 9