Combining Feature and Algorithm Hyperparameter Selection using some Metalearning Methods

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2017

Autores

Cachada, M (Autor) Outra; Abdulrahman, SM (Autor) Outra; Pavel Brazdil (Autor) FEP

Ata de Conferência Internacional

Título: AutoML@PKDD/ECML

Páginas: 69-83

2017 International Workshop on Automatic Selection, Configuration and Composition of Machine Learning Algorithms, AutoML 2017

22 September 2017

Indexação

Scopus - 4 Citações

Outras Informações

ID Authenticus: P-00N-AQQ

Abstract (EN): Machine learning users need methods that can help them identify algorithms or even workflows (combination of algorithms with preprocessing tasks, using or not hyperparameter configurations that are different from the defaults), that achieve the potentially best performance. Our study was oriented towards average ranking (AR), an algorithm selection method that exploits meta-data obtained on prior datasets. We focused on extending the use of a variant of AR¿ that takes A3R as the relevant metric (combining accuracy and run time). The extension is made at the level of diversity of the portfolio of workflows that is made available to AR. Our aim was to establish whether feature selection and different hyperparameter configurations improve the process of identifying a good solution. To evaluate our proposal we have carried out extensive experiments in a leave-one-out mode. The results show that AR¿ was able to select workflows that are likely to lead to good results, especially when the portfolio is diverse. We additionally performed a comparison of AR¿ with Auto-WEKA, running with different time budgets. Our proposed method shows some advantage over Auto-WEKA, particularly when the time budgets are small.

Idioma: Inglês

Tipo (Avaliação Docente): Científica