Multilingual characterization and extraction of narratives from online news: annotation guidelines

• Type: Other Publications
• Date: 2025

Authors

Stefanovitch, Nicolas (Author) Other; Mahmoud, Tarek (Author) Other; Nikolaidis, Nikolaos (Author) Other; Campos, Jorge Alípio (Author) Other; Dimitrov, Dimitar (Author) Other; Silvano, Maria da Purificação (Author) FLUP

Scientific classification

CORDIS: Humanities > language sciences > Linguistics > Computational linguistics

Other information

Abstract (EN): This document provides the detailed annotation guidelines and annotation campaign design used for the creation of the datasets for the SemEval 2025 Task 10 on Multilingual Characterization and Extraction of Narratives from Online News. During this campaign, a total of 2419 documents were annotated by 35 individuals in 5 languages, covering over 96 fine-grained and 22 coarse-grained labels spanning 3 taxonomies. The annotation task is subdivided into three subtasks: Entity Framing, Narrative Classification, and Explanation of Narrative Classification. This document provides a detailed description of each task and the related taxonomies used, i.e., Named Entity Role taxonomy, and Fine-grained Narrative taxonomies covering two domains: Climate Change and Ukraine-Russia war. We also provide the annotation guidelines for all three tasks and the description of the overall annotation procedure management. The taxonomies are accompanied by examples for each label in all the 5 languages of the shared task. Additionally, full-fledged annotated examples for all tasks are provided. Furthermore, we describe Inception, the annotation platform used for the annotation process, and provide step-by-step instructions for the annotators on how to use it. Finally, based on the lessons learned we provide suggestions on how to run similar-in-nature large multilingual annotation campaigns.

Language: English

Type (Professor's evaluation): Scientific

Notes: Também, participaram neste artigo: Shivam Sharma Roman Yangarber, Nuno Guimarães, Elisa Sartori, Ana Filipa Pacheco Cecília Ortiz, Cláudia Couto, Glória Reis de Oliveira, Ari Gonçalves, Ivan Koychev, Ivo Moravski, Nicolo Faggiani, Sopho Kharazi, Bonka Kotseva, Ion Androutsopoulos, John Pavlopoulos, Gayatri Oke, Kanupriya Pathak, Dhairya Suman, Sohini Mazumdar, Tanmoy Chakraborty, Zhuohan Xie, Denis Kvachev, Irina Gatsuk, Ksenia Semenova, Matilda Villanen, Aamos Waher, Daria Lyakhnovich, Giovanni Da San Martino, Preslav Nakov e Jakub Piskorsk.