Learning Word Embeddings from the Portuguese Twitter Stream: A Study of Some Practical Aspects

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2017

Autores

Eugénio Oliveira (Autor) FEUP; Pedro Saleiro (Autor) FEUP; Luís António Diniz Fernandes de Morais Sarmento (Autor) Outra; Eduarda Mendes Rodrigues (Autor) FEUP; Carlos Soares (Autor) FEUP

Ata de Conferência Internacional

Título: EPIA

Páginas: 880-891

18th EPIA Conference on Artificial Intelligence, EPIA 2017

5 September 2017 through 8 September 2017

Indexação

ISI Web of Knowledge - 0 Citações

Scopus - 2 Citações

Classificação Científica

CORDIS: Ciências Físicas > Ciência de computadores > Informática ; Ciências Físicas > Ciência de computadores

FOS: Ciências da engenharia e tecnologias > Engenharia electrotécnica, electrónica e informática

Outras Informações

ID Authenticus: P-00M-YFF

DOI: 10.1007/978-3-319-65340-2_71

Abstract (EN): This paper describes a preliminary study for producing and distributing a large-scale database of embeddings from the Portuguese Twitter stream. We start by experimenting with a relatively small sample and focusing on three challenges: volume of training data, vocabulary size and intrinsic evaluation metrics. Using a single GPU, we were able to scale up vocabulary size from 2048 words embedded and 500K training examples to 32768 words over 10M training examples while keeping a stable validation loss and approximately linear trend on training time per epoch. We also observed that using less than 50% of the available training examples for each vocabulary size might result in overfitting. Results on intrinsic evaluation show promising performance for a vocabulary size of 32768 words. Nevertheless, intrinsic evaluation metrics suffer from over-sensitivity to their corresponding cosine similarity thresholds, indicating that a wider range of metrics need to be developed to track progress. © Springer International Publishing AG 2017.

Idioma: Inglês

Tipo (Avaliação Docente): Científica