Towards a Human-AI Hybrid Framework for Inter-Researcher Similarity Detection

• Type: Article in International Conference Proceedings Book
• Date: 2021

Authors

Guimarães, D (Author) Other; Paulino, D (Author) Other; Correia, A (Author) Other; Trigo, Luís (Author) FEUP; Pavel Brazdil (Author) FEP; Paredes, H (Author) Other

Conference proceedings International

Title: 2nd IEEE International Conference on Human-Machine Systems, ICHMS 2021, Magdeburg, Germany, September 8-10, 2021

Pages: 1-4

2021 IEEE International Conference on Human-Machine Systems, ICHMS 2021

8 September 2021 through 10 September 2021

Indexing

ISI Web of Knowledge - 0 Citations

Scopus - 0 Citations

Other information

Authenticus ID: P-00V-N56

DOI: 10.1109/ichms53169.2021.9582633

Abstract (EN): Understanding the intellectual landscape of scientific communities and their collaborations has become an indispensable part of research per se. In this regard, measuring similarities among scientific documents can help researchers to identify groups with similar interests as a basis for strengthening collaboration and university-industry linkages. To this end, we intend to evaluate the performance of hybrid crowd-computing methods in measuring the similarity between document pairs by comparing the results achieved by crowds and artificial intelligence (AI) algorithms. That said, in this paper we designed two types of experiments to illustrate some issues in calculating how similar an automatic solution is to a given ground truth. In the first type of experiments, we created a crowdsourcing campaign consisting of four human intelligence tasks (HITs) in which the participants had to indicate whether or not a set of papers belonged to the same author. The second type involves a set of natural language processing (NLP) processes in which we used the TF-IDF measure and the Bidirectional Encoder Representation from Transformers (BERT) model. The results of the two types of experiments carried out in this study provide preliminary insight into detecting major contributions from human-AI cooperation at similarity calculation in order to achieve better decision support. We believe that in this case decision makers can be better informed about potential collaborators based on content-based insights enhanced by hybrid human-AI mechanisms. © 2021 IEEE.

Language: English

Type (Professor's evaluation): Scientific