Leveraging LLMs to improve human annotation efficiency with INCEpTION

• Type: Article in International Conference Proceedings Book
• Date: 2025

Authors

Cunha, Luís Filipe (Author) Other; Yu, Nana (Author) FLUP; Silvano, Purificação (Author) FLUP; Campos, Ricardo (Author) Other; Jorge, Alípio (Author) FCUP

Conference proceedings International

Title: Advances in Information Retrieval: 47th European Conference on Information Retrieval, ECIR 2025: Conference proceedings

Pages: 53-58

47th European Conference on Information Retrieval, ECIR 2025

Lucca, 2025

Indexing

Crossref

Other information

DOI: 10.1007/978-3-031-88720-8_10

Abstract (EN): Manual text annotation is a complex and time-consuming task. However, recent advancements demonstrate that such a task can be accelerated with automated pre-annotation. In this paper, we present a methodology to improve the efficiency of manual text annotation by leveraging LLMs for text pre-annotation. For this purpose, we train a BERT model for a token classification task and integrate it into the INCEpTION annotation tool to generate span-level suggestions for human annotators. To assess the usefulness of our approach, we con- ducted an experiment where an experienced linguist annotated plain text both with and without our model’s pre-annotations. Our results show that the model-assisted approach reduces annotation time by nearly 23%.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 5