Improving Transfer Learning in Unsupervised Language Adaptation

• Type: Article in International Conference Proceedings Book
• Date: 2021

Authors

Gil Rocha (Author) Other; Henrique Lopes Cardoso (Author) FEUP

Conference proceedings International

Title: ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING, ICANN 2021, PT V

Pages: 588-599

30th International Conference on Artificial Neural Networks (ICANN)

ELECTR NETWORK, SEP 14-17, 2021

Indexing

ISI Web of Knowledge - 1 Citation

Scopus - 1 Citation

Other information

Authenticus ID: P-00V-CPH

DOI: 10.1007/978-3-030-86383-8_47

Abstract (EN): Unsupervised language adaptation aims to improve the cross-lingual ability of models that are fine-tuned on a specific task and source language, without requiring labeled data on the target language. On the other hand, recent multilingual language models (such as mBERT) have achieved new state-of-the-art results on a variety of tasks and languages, when employed in a direct transfer approach. In this work, we explore recently proposed unsupervised language adaptation methods - Adversarial Training and Encoder Alignment - to fine-tune language models on a specific task and language pairs, showing that the cross-lingual ability of the models can be further improved. We focus on two conceptually different tasks, Natural Language Inference and Sentiment Analysis, and analyze the performance of the explored models. In particular, Encoder Alignment is the best approach for most of the settings explored in this work, only underperforming in the presence of domain-shift between source and target languages.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 12