The incremental process of building an annotation scheme for clinical narratives in portuguese: the contribution of human variation analysis

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2025

Autores

Fernandes, Ana Luísa (Autor) Outra; Silvano, Purificação (Autor) FLUP; Leal, António (Autor) FLUP; Guimarães, Nuno (Autor) FEUP; Silva, Rita Ribeiro da (Autor) Outra; Cunha, Luís Filipe (Autor) Outra; Jorge, Alípio (Autor) FCUP

Ata de Conferência Internacional

Título: Research Challenges in Information Science - 19th International Conference, RCIS 2025, Seville, Spain, May 20-23, 2025, Proceedings, Part II

Páginas: 332-343

19th International Conference on Research Challenges in Information Science, RCIS 2025

Seville, 20 May 2025 through 23 May 2025

Outras Informações

DOI: 10.18653/v1/2025.law-1.28

Abstract (EN): The development of a robust annotation scheme and corresponding guidelines is crucial for pro- ducing annotated datasets that advance both lin- guistic and computational research. This paper presents a case study that outlines a method- ology for designing an annotation scheme and its guidelines, specifically aimed at represent- ing morphosyntactic and semantic information regarding temporal features, as well as medi- cal information in medical reports written in Portuguese. We detail a multi-step process that includes reviewing existing frameworks, con- ducting an annotation experiment to determine the optimal approach, and designing a model based on these findings. We validated the ap- proach through a pilot experiment where we assessed the reliability and applicability of the annotation scheme and guidelines. In this ex- periment, two annotators independently anno- tated a patient’s medical report consisting of six documents using the proposed model, while a curator established the ground truth. The analy- sis of inter-annotator agreement and the annota- tion results enabled the identification of sources of human variation and provided insights for further refinement of the annotation scheme and guidelines.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 11