Can Llama 3 Accurately Assess Readability? A Comparative Study Using Lead Sections from Wikipedia

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2025

Autores

Rodrigues, JF (Autor) Outra; Cardoso, HL (Autor) Outra; Carla Teixeira Lopes (Autor) FEUP

Ata de Conferência Internacional

Título: RESEARCH CHALLENGES IN INFORMATION SCIENCE, RCIS 2025, PT II

Páginas: 89-97

19th International Conference on Research Challenges in Information Science-RCIS-Annual

Sevilla, SPAIN, MAY 20-23, 2025

Indexação

ISI Web of Knowledge - 0 Citações

Scopus - 0 Citações

Outras Informações

ID Authenticus: P-018-RZ8

DOI: 10.1007/978-3-031-92471-2_6

Abstract (EN): Text readability is vital for effective communication and learning, especially for those with lower information literacy. This research aims to assess Llama 3's ability to grade readability and compare its alignment with established metrics. For that purpose, we create a new dataset of article lead sections from English and Simple English Wikipedia, covering nine categories. The model is prompted to rate the readability of the texts on a grade-level scale, and an in-depth analysis of the results is conducted. While Llama 3 correlates strongly with most metrics, it may underestimate text grade levels.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 9