Evaluating Llama 3 for Text Simplification: A Study on Wikipedia Lead Sections

• Type: Article in International Conference Proceedings Book
• Date: 2025

Authors

Rodrigues, JF (Author) Other; Cardoso, HL (Author) Other; Carla Teixeira Lopes (Author) FEUP

Conference proceedings International

Title: Companion Proceedings of the ACM on Web Conference 2025, WWW 2025, Sydney, NSW, Australia, 28 April 2025 - 2 May 2025

Pages: 1273-1277

34th ACM Web Conference, WWW Companion 2025

Sydney, 28 April 2025 through 2 May 2025

Indexing

ISI Web of Knowledge - 0 Citations

Scopus - 1 Citation

Other information

Authenticus ID: P-018-Z2Z

DOI: 10.1145/3701716.3715467

Abstract (EN): Text simplification converts complex text into simpler language, improving readability and comprehension. This study evaluates the effectiveness of open-source large language models for text simplification across various categories. We created a dataset of 66,620 lead section pairs from English and Simple English Wikipedia, spanning nine categories, and tested Llama 3 for text simplification. We assessed its output for readability, simplicity, and meaning preservation. Results show improved readability, with simplification varying by category. Texts on Time were the most shortened, while Leisurerelated texts had the greatest reduction of words/characters and syllables per sentence. Meaning preservation was most effective for the Objects and Education categories.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 5