Ranking learning algorithms: Using IBL and meta-learning on accuracy and time results

• Type: Article in International Scientific Journal
• Date: 2003

Authors

Brazdil, PB (Author) FEP; Soares, C (Author) FEP; Da Costa, JP (Author) FCUP

Journal

Title: Machine Learning

Vol. 50

Pages: 251-277

ISSN: 0885-6125

Publisher: Springer Nature

Indexing

ISI Web of Knowledge - 246 Citations

ISI Web of Science

Scopus - 314 Citations

Scientific classification

FOS: Natural sciences > Computer and information sciences

Other information

Authenticus ID: P-000-HMB

DOI: 10.1023/a:1021713901879

Abstract (EN): We present a meta-learning method to support selection of candidate learning algorithms. It uses a k-Nearest Neighbor algorithm to identify the datasets that are most similar to the one at hand. The distance between datasets is assessed using a relatively small set of data characteristics, which was selected to represent properties that affect algorithm performance. The performance of the candidate algorithms on those datasets is used to generate a recommendation to the user in the form of a ranking. The performance is assessed using a multicriteria evaluation measure that takes not only accuracy, but also time into account. As it is not common in Machine Learning to work with rankings, we had to identify and adapt existing statistical techniques to devise an appropriate evaluation methodology. Using that methodology, we show that the meta-learning method presented leads to significantly better rankings than the baseline ranking method. The evaluation methodology is general and can be adapted to other ranking problems. Although here we have concentrated on ranking classification algorithms, the meta-learning framework presented can provide assistance in the selection of combinations of methods or more complex problem solving strategies.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 27