Fast Algorithm Selection Using Learning Curves

• Type: Article in International Conference Proceedings Book
• Date: 2015

Authors

van Rijn, JN (Author) Other; Abdulrahman, SM (Author) Other; Pavel Brazdil (Author) FEP; Vanschoren, J (Author) Other

Conference proceedings International

Title: Advances in Intelligent Data Analysis XIV

Pages: 298-309

14th International Symposium on Intelligent Data Analysis (IDA)

Saint Etienne, FRANCE, OCT 22-24, 2015

Indexing

ISI Web of Knowledge - 28 Citations

Scopus - 36 Citations

Other information

Authenticus ID: P-00G-T0K

DOI: 10.1007/978-3-319-24465-5_26

Abstract (EN): One of the challenges in Machine Learning to find a classifier and parameter settings that work well on a given dataset. Evaluating all possible combinations typically takes too much time, hence many solutions have been proposed that attempt to predict which classifiers are most promising to try. As the first recommended classifier is not always the correct choice, multiple recommendations should be made, making this a ranking problem rather than a classification problem. Even though this is a well studied problem, there is currently no good way of evaluating such rankings. We advocate the use of Loss Time Curves, as used in the optimization literature. These visualize the amount of budget (time) needed to converge to a acceptable solution. We also investigate a method that utilizes the measured performances of classifiers on small samples of data to make such recommendation, and adapt it so that it works well in Loss Time space. Experimental results show that this method converges extremely fast to an acceptable solution.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 12