Validation of language agnostic models for discourse marker detection

• Type: Article in International Conference Proceedings Book
• Date: 2023

Authors

Damova, Mariana (Author) Other; Mishev, Kostadin (Author) Other; Oleskeviciene, Giedre Valunaite (Author) Other; Liebeskind, Chaya (Author) Other; Silvano, Maria da Purificação (Author) FLUP; Trajanov, Dimitar (Author) Other; Truica, Ciprian-Octavian (Author) Other; Apostol, Elena-Simona (Author) Other; Chiarcos, Christian (Author) Other; Baczkowska, Anna (Author) Other

Conference proceedings International

Title: Language, Data and Knowledge 2023 (LDK 2023): Proceedings of the 4th Conference on Language, Data and Knowledge

Pages: 434-439

4th Conference on Language, Data and Knowledge

Vienna, 2023

Other information

Abstract (EN): Using language models to detect or predict the presence of language phenomena in the text has become a mainstream research topic. With the rise of generative models, experiments using deep learning and transformer models trigger intense interest. Aspects like precision of predictions, portability to other languages or phenomena, scale have been central to the research community. Discourse markers, as language phenomena, perform important functions, such as signposting, signalling, and rephrasing, by facilitating discourse organization. Our paper is about discourse markers detection, a complex task as it pertains to a language phenomenon manifested by expressions that can occur as content words in some contexts and as discourse markers in others. We have adopted language agnostic model trained in English to predict the discourse marker presence in texts in 8 other unseen by the model languages with the goal to evaluate how well the model performs in different structure and lexical properties languages. We report on the process of evaluation and validation of the model’s performance across European Portuguese, Hebrew, German, Polish, Romanian, Bulgarian, Macedonian, and Lithuanian and about the results of this validation. This research is a key step towards multilingual language processing.

Language: English

Type (Professor's evaluation): Scientific