Assessing the Influence of Multimodal Feedback in Mobile-Based Musical Task Performance

• Type: Article in International Scientific Journal
• Date: 2022-08-08

Authors

Alexandre Clément (Author) FEUP; Gilberto Bernardes (Author) FEUP

Journal

The Journal is awaiting validation by the Administrative Services.

Title: MULTIMODAL TECHNOLOGIES AND INTERACTION

Vol. 6 No. 8

Final page: 68

ISSN: 2414-4088

Indexing

ISI Web of Knowledge - 0 Citations

Scopus - 0 Citations

Other information

Authenticus ID: P-00X-0SQ

DOI: 10.3390/mti6080068

Abstract (EN): Digital musical instruments have become increasingly prevalent in musical creation and production. Optimizing their usability and, particularly, their expressiveness, has become essential to their study and practice. The absence of multimodal feedback, present in traditional acoustic instruments, has been identified as an obstacle to complete performer-instrument interaction in particular due to the lack of embodied control. Mobile-based digital musical instruments present a particular case by natively providing the possibility of enriching basic auditory feedback with additional multimodal feedback. In the experiment presented in this article, we focused on using visual and haptic feedback to support and enrich auditory content to evaluate the impact on basic musical tasks (i.e., note pitch tuning accuracy and time). The experiment implemented a protocol based on presenting several musical note examples to participants and asking them to reproduce them, with their performance being compared between different multimodal feedback combinations. Collected results show that additional visual feedback was found to reduce user hesitation in pitch tuning, allowing users to reach the proximity of desired notes in less time. Nonetheless, neither visual nor haptic feedback was found to significantly impact pitch tuning time and accuracy compared to auditory-only feedback.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 19