BATS-PT: assessing portuguese masked language models in lexico-semantic analogy solving and relation completion

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2024

Autores

Oliveira, Hugo Gonçalo (Autor) Outra; Rodrigues, Ricardo (Autor) Outra; Ferreira, Bruno (Autor) Outra; Silvano, Maria da Purificação (Autor) FLUP; Carvalho, Sara (Autor) Outra

Ata de Conferência Internacional

Título: Proceedings of the 16th International Conference on Computational Processing of Portuguese, PROPOR 2024, Santiago de Compostela, Galicia/Spain, 12-15 March, 2024

16th International Conference on Computational Processing of Portuguese, PROPOR 2024

Santiago de Compostela, 2024

Outras Informações

Abstract (EN): This paper presents BATS-PT, the manual translation of the lexicographic portion of the Bigger Analogy Test Set (BATS) to European Portuguese. BATS-PT covers ten types of lexicosemantic analogies and can be used for assessing word embeddings and language models. Following this, the dataset is showcased while assessing two pretrained language models for Portuguese, BERTimbau and Albertina, in two tasks: analogy solving and relation completion, both in zero- and few-shot mask-prediction approaches. Experiments reveal different performance across relations and, in both tasks, the best overall performance was achieved with BERTimbau, in a five-shot scenario. We further discuss the limitations of the reported experiments and directions towards future improvements in these tasks

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 11