Leveraging LLMs to improve human annotation efficiency with INCEpTION

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2025

Autores

Cunha, Luís Filipe (Autor) Outra; Yu, Nana (Autor) FLUP; Silvano, Purificação (Autor) FLUP; Campos, Ricardo (Autor) Outra; Jorge, Alípio (Autor) FCUP

Ata de Conferência Internacional

Título: Advances in Information Retrieval: 47th European Conference on Information Retrieval, ECIR 2025: Conference proceedings

Páginas: 53-58

47th European Conference on Information Retrieval, ECIR 2025

Lucca, 2025

Indexação

Crossref

Outras Informações

DOI: 10.1007/978-3-031-88720-8_10

Abstract (EN): Manual text annotation is a complex and time-consuming task. However, recent advancements demonstrate that such a task can be accelerated with automated pre-annotation. In this paper, we present a methodology to improve the efficiency of manual text annotation by leveraging LLMs for text pre-annotation. For this purpose, we train a BERT model for a token classification task and integrate it into the INCEpTION annotation tool to generate span-level suggestions for human annotators. To assess the usefulness of our approach, we con- ducted an experiment where an experienced linguist annotated plain text both with and without our model’s pre-annotations. Our results show that the model-assisted approach reduces annotation time by nearly 23%.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 5