Evaluation of Lyrics Extraction from Folk Music Sheets Using Vision Language Models (VLMs)

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2025

Autores

Mendes, AS (Autor) Outra; Murciego, AL (Autor) Outra; Silva, LA (Autor) Outra; Jiménez-Bravo, DM (Autor) Outra; Navarro-Cáceres, M (Autor) Outra; Bernardes, G (Autor) FEUP

Ata de Conferência Internacional

Título: PROGRESS IN ARTIFICIAL INTELLIGENCE, EPIA 2024, PT I

Páginas: 91-102

23rd EPIA Conference on Artificial Intelligence-EPIA

Viana do Castelo, PORTUGAL, SEP 03-06, 2024

Indexação

ISI Web of Knowledge - 0 Citações

Scopus - 0 Citações

Outras Informações

ID Authenticus: P-017-C9T

DOI: 10.1007/978-3-031-73497-7_8

Abstract (EN): Monodic folk music has traditionally been preserved in physical documents. It constitutes a vast archive that needs to be digitized to facilitate comprehensive analysis using AI techniques. A critical component of music score digitization is the transcription of lyrics, an extensively researched process in Optical Character Recognition (OCR) and document layout analysis. These fields typically require the development of specific models that operate in several stages: first, to detect the bounding boxes of specific texts, then to identify the language, and finally, to recognize the characters. Recent advances in vision language models (VLMs) have introduced multimodal capabilities, such as processing images and text, which are competitive with traditional OCR methods. This paper proposes an end-to-end system for extracting lyrics from images of handwritten musical scores. We aim to evaluate the performance of two state-of-the-art VLMs to determine whether they can eliminate the need to develop specialized text recognition and OCR models for this task. The results of the study, obtained from a dataset in a real-world application environment, are presented along with promising new research directions in the field. This progress contributes to preserving cultural heritage and opens up new possibilities for global analysis and research in folk music.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 12