Contextual Relative Entropy Policy Search with Covariance Matrix Adaptation

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2016

Autores

Abdolmaleki, A (Autor) Outra; Simoes, D (Autor) Outra; lau, n (Autor) FCUP; reis, lp (Autor) REIT; Neumann, G (Autor) Outra

Ata de Conferência Internacional

Título: 2016 International Conference on Autonomous Robot Systems and Competitions, ICARSC 2016, Bragança, Portugal, May 4-6, 2016

Páginas: 94-99

Indexação

ISI Web of Knowledge - 8 Citações

Scopus - 10 Citações

Outras Informações

ID Authenticus: P-00M-9DZ

DOI: 10.1109/icarsc.2016.31

Abstract (EN): Stochastic search algorithms are black-box optimizers of an objective function. They have recently gained a lot of attention in operations research, machine learning and policy search of robot motor skills due to their ease of use and their generality. However, with slightly different tasks or objective functions, many stochastic search algorithms require complete re-learning in order to adapt the solution to the new objective function or the new context. As such, we consider the contextual stochastic search paradigm. Here, we want to find good parameter vectors for multiple related tasks, where each task is described by a continuous context vector. Hence, the objective function might change slightly for each parameter vector evaluation. Contextual algorithms have been investigated in the field of policy search. However, contextual policy search algorithms typically suffer from premature convergence and perform unfavourably in comparison with state of the art stochastic search methods. In this paper, we investigate a contextual stochastic search algorithm known as Contextual Relative Entropy Policy Search (CREPS), an information-theoretic algorithm that can learn for multiple tasks simultaneously. We extend that algorithm with a covariance matrix adaptation technique that alleviates the premature convergence problem. We call the new algorithm Contextual Relative Entropy Policy Search with Covariance Matrix Adaptation (CREPS-CMA). We will show that CREPS-CMA outperforms the original CREPS by orders of magnitude. We illustrate the performance of CREPS-CMA on several contextual tasks, including a complex simulated robot kick task.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 6