Human vs. Automatic Annotation Regarding the Task of Relevance Detection in Social Networks

• Type: Article in International Conference Proceedings Book
• Date: 2018

Authors

Guimaraes, N (Author) Other; Miranda, F (Author) Other; Figueira, A (Author) FCUP

Conference proceedings International

Title: ADVANCES IN INTERNET, DATA & WEB TECHNOLOGIES

Pages: 922-933

6th International Conference on Emerging Internet, Data and Web Technologies (EIDWT)

Polytechn Univ Tirana, Tirana, ALBANIA, MAR 15-17, 2018

Indexing

ISI Web of Knowledge - 3 Citations

Scopus - 3 Citations

Other information

Authenticus ID: P-00N-KK6

DOI: 10.1007/978-3-319-75928-9_85

Abstract (EN): The burst of social networks and the possibility of being continuously connected has provided a fast way for information diffusion. More specifically, real-time posting allowed news and events to be reported quicker through social networks than traditional news media. However, the massive data that is daily available makes newsworthy information a needle in a haystack. Therefore, our goal is to build models that can detect journalistic relevance automatically in social networks. In order to do it, it is essential to establish a ground truth with a large number of entries that can provide a suitable basis for the learning algorithms due to the difficulty inherent to the ambiguity and wide scope associated with the concept of relevance. In this paper, we propose and compare two different methodologies to annotate posts regarding their relevance: automatic and human annotation. Preliminary results show that supervised models trained with the automatic annotation methodology tend to perform better than using human annotation in a test dataset labeled by experts.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 12