Comparative evaluation of web search engines in health information retrieval

• Type: Article in International Scientific Journal
• Date: 2011

Authors

Carla Teixeira Lopes (Author) FEUP; Cristina Ribeiro (Author) FEUP

Journal

Title: Online Information Review

Vol. 35

Pages: 869-892

ISSN: 1468-4527

Publisher: Emerald

Indexing

ISI Web of Knowledge - 9 Citations

Scopus - 13 Citations

Other information

Authenticus ID: P-002-Z8Z

DOI: 10.1108/14684521111193175

Abstract (EN): Purpose - The intent of this work is to evaluate several generalist and health-specific search engines for retrieval of health information by consumers: to compare the retrieval effectiveness of these engines for different types of clinical queries, medical specialties and condition severity; and to compare the use of evaluation metrics for binary relevance scales and for graded ones. Design/methodology/approach - The authors conducted a study in which users evaluated the relevance of documents retrieved by four search engines for two different health information needs. Users could choose between generalist (Bing, Google, Sapo and Yahoo!) and health-specific (MedlinePlus, SapoSande and WebMD) search engines. The authors then analysed the differences between search engines and groups of information needs with six different measures: graded average precision (gap), average precision (ap), gap@5, gap@10, ap@5 and ap@10. Findings The results show that generalist web search engines surpass the precision of health-specific engines. Google has the best performance, mainly in the top ten results. It was found that information needs associated with severe conditions are associated with higher precision, as are overview and psychiatry questions. Originality/value - The study is one of the first to use a recently proposed measure to evaluate the effectiveness of retrieval systems with graded relevance scales. It includes tasks from several medical specialties, types of clinical questions and different levels of severity which, to the best of the authors' knowledge, has not been clone before. Moreover, users have considerable involvement in the experiment. The results help in understanding how search engines differ in their responses to health information needs, what types of online health information are more common on the web and how to improve this type of search.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 24