Contextual Direct Policy Search

• Tipo: Artigo em Revista Científica Internacional
• Data: 2019

Autores

Abbas Abdolmaleki (Autor) Outra; David Simoes (Autor) Outra; Nuno Lau (Autor) Outra; Luís Paulo Reis (Autor) FEUP; Gerhard Neumann (Autor) Outra

Revista

Título: Journal of Intelligent and Robotic Systems: Theory and Applications

Vol. 96 Nº 2

Páginas: 141-157

ISSN: 0921-0296

Editora: Springer Nature

Indexação

ISI Web of Knowledge - 0 Citações

Scopus - 0 Citações

Outras Informações

ID Authenticus: P-00R-ZZ5

DOI: 10.1007/s10846-018-0968-4

Abstract (EN): Stochastic search and optimization techniques are used in a vast number of areas, ranging from refining the design of vehicles, determining the effectiveness of new drugs, developing efficient strategies in games, or learning proper behaviors in robotics. However, they specialize for the specific problem they are solving, and if the problem's context slightly changes, they cannot adapt properly. In fact, they require complete re-leaning in order to perform correctly in new unseen scenarios, regardless of how similar they are to previous learned environments. Contextual algorithms have recently emerged as solutions to this problem. They learn the policy for a task that depends on a given context, such that widely different contexts belonging to the same task are learned simultaneously. That being said, the state-of-the-art proposals of this class of algorithms prematurely converge, and simply cannot compete with algorithms that learn a policy for a single context. We describe the Contextual Relative Entropy Policy Search (CREPS) algorithm, which belongs to the before-mentioned class of contextual algorithms. We extend it with a technique that allows the algorithm to severely increase its performance, and we call it Contextual Relative Entropy Policy Search with Covariance Matrix Adaptation (CREPS-CMA). We propose two variants, and demonstrate their behavior in a set of classic contextual optimization problems, and on complex simulator robot tasks.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 17