Advancing Neural Encoding of Portuguese with Transformer Albertina PT-*

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2023

Autores

Rodrigues, J (Autor) Outra; Gomes, L (Autor) Outra; Silva, J (Autor) Outra; Branco, A (Autor) Outra; Santos, R (Autor) Outra; Henrique Lopes Cardoso (Autor) FEUP; Osório, T (Autor) Outra

Ata de Conferência Internacional

Título: PROGRESS IN ARTIFICIAL INTELLIGENCE, EPIA 2023, PT I

Páginas: 441-453

22nd EPIA Conference on Artificial Intelligence (EPIA)

Azores, PORTUGAL, SEP 05-08, 2023

Indexação

ISI Web of Knowledge - 0 Citações

Scopus - 0 Citações

Outras Informações

ID Authenticus: P-00Z-KWK

DOI: 10.1007/978-3-031-49008-8_35

Abstract (EN): To advance the neural encoding of Portuguese (PT), and a fortiori the technological preparation of this language for the digital age, we developed a Transformer-based foundation model that sets a new state of the art in this respect for two of its variants, namely European Portuguese from Portugal (PT-PT) and American Portuguese from Brazil (PT-BR). To develop this encoder, which we named Albertina PT-*, a strong model was used as a starting point, DeBERTa, and its pre-training was done over data sets of Portuguese, namely over a data set we gathered for PT-PT and over the brWaC corpus for PT-BR. The performance of Albertina and competing models was assessed by evaluating them on prominent downstream language processing tasks adapted for Portuguese. Both Albertina versions are distributed free of charge and under a most permissive license possible and can be run on consumer-grade hardware, thus seeking to contribute to the advancement of research and innovation in language technology for Portuguese.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 13