A COMPARISON OF SCORING METRICS FOR PREDICTING THE NEXT NAVIGATION STEP WITH MARKOV MODEL-BASED SYSTEMS

• Type: Article in International Scientific Journal
• Date: 2010

Authors

José Borges (Author) FEUP; Mark Levene (Adapter) Other

Journal

Title: INTERNATIONAL JOURNAL OF INFORMATION TECHNOLOGY & DECISION MAKING

Vol. 9

Pages: 547-573

ISSN: 0219-6220

Indexing

ISI Web of Knowledge - 3 Citations

ISI Web of Science

Scopus - 6 Citations

INSPEC

Scientific classification

FOS: Natural sciences > Computer and information sciences

Other information

Authenticus ID: P-003-59S

DOI: 10.1142/s0219622010003956

Abstract (EN): The problem of predicting the next request during a user's navigation session has been extensively studied. In this context, higher-order Markov models have been widely used to model navigation sessions and to predict the next navigation step, while prediction accuracy has been mainly evaluated with the hit and miss score. We claim that this score, although useful, is not sufficient for evaluating next link prediction models with the aim of finding a sufficient order of the model, the size of a recommendation set, and assessing the impact of unexpected events on the prediction accuracy. Herein, we make use of a variable length Markov model to compare the usefulness of three alternatives to the hit and miss score: the Mean Absolute Error, the Ignorance Score, and the Brier score. We present an extensive evaluation of the methods on real data sets and a comprehensive comparison of the scoring methods.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 27