Predicting argument density from multiple annotations

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2022

Autores

Rocha, Gil (Autor) Outra; Leite, Bernardo (Autor) Outra; Trigo, Luís (Autor) FLUP; Cardoso, Henrique Lopes (Autor) FEUP; Sousa-Silva, Rui (Autor) FLUP; Carvalho, Paula (Autor) Outra; Martins, Bruno (Autor) Outra; Won, Miguel (Autor) Outra

Ata de Conferência Internacional

Título: Natural Language Processing and Information Systems (NLDB 2022) : Proceedings / 27th International Conference on Applications of Natural Language to Information Systems

Páginas: 227-239

27th International Conference on Applications of Natural Language to Information Systems (NLDB 2022)

Valencia, 2022

Indexação

ISI Web of Knowledge - 1 Citação

Scopus

Outras Informações

ID Authenticus: P-00W-RH0

DOI: 10.1007/978-3-031-08473-7_21

Abstract (EN): Annotating a corpus with argument structures is a complex task, and it is even more challenging when addressing text genres where argumentative discourse markers do not abound. We explore a corpus of opinion articles annotated by multiple annotators, providing diverse perspectives of the argumentative content therein. New annotation aggregation methods are explored, diverging from the traditional ones that try to minimize presumed errors from annotator disagreement. The impact of our methods is assessed for the task of argument density prediction, seen as an initial step in the argument mining pipeline. We evaluate and compare models trained for this regression task in different generated datasets, considering their prediction error and also from a ranking perspective. Results confirm the expectation that addressing argument density from a ranking perspective is more promising than looking at the problem as a mere regression task. We also show that probabilistic aggregation, which weighs tokens by considering all annotators, is a more interesting approach, achieving encouraging results as it accommodates different annotator perspectives. The code and models are publicly available at https://github.com/DARGMINTS/argument-density.

Idioma: Inglês

Tipo (Avaliação Docente): Científica