A comparison of ranking methods for classification algorithm selection

• Type: Article in International Scientific Journal
• Date: 2000

Authors

Brazdil, PB (Author) FEP; Soares, C (Author) FEP

Journal

Title: Lecture Notes in Computer Science

Vol. 1810

Pages: 63-74

ISSN: 0302-9743

Publisher: Springer Nature

Indexing

ISI Web of Knowledge - 109 Citations

Scopus - 138 Citations

Scientific classification

FOS: Natural sciences > Computer and information sciences

Other information

Authenticus ID: P-001-1VS

DOI: 10.1007/3-540-45164-1_8

Abstract (EN): We investigate the problem of using past performance information to select an algorithm for a given classification problem. We present three ranking methods for that purpose: average ranks, success rate ratios and significant wins. We also analyze the problem of evaluating and comparing these methods. The evaluation technique used is based on a leave-one-out procedure. On each iteration, the method generates a ranking using the results obtained by the algorithms on the training datasets. This ranking is then evaluated by calculating its distance from the ideal ranking built using the performance information on the test dataset. The distance measure adopted here, average correlation, is based on Spearman's rank correlation coefficient. To compare ranking methods, a combination of Friedman's test and Dunn's multiple comparison procedure is adopted. When applied to the methods presented here, these tests indicate that the success rate ratios and average ranks methods perform better than significant wins.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 12