Contextual Direct Policy Search

• Type: Article in International Scientific Journal
• Date: 2019

Authors

Abbas Abdolmaleki (Author) Other; David Simoes (Author) Other; Nuno Lau (Author) Other; Luís Paulo Reis (Author) FEUP; Gerhard Neumann (Author) Other

Journal

Title: Journal of Intelligent and Robotic Systems: Theory and Applications

Vol. 96 No. 2

Pages: 141-157

ISSN: 0921-0296

Publisher: Springer Nature

Indexing

ISI Web of Knowledge - 0 Citations

Scopus - 0 Citations

Other information

Authenticus ID: P-00R-ZZ5

DOI: 10.1007/s10846-018-0968-4

Abstract (EN): Stochastic search and optimization techniques are used in a vast number of areas, ranging from refining the design of vehicles, determining the effectiveness of new drugs, developing efficient strategies in games, or learning proper behaviors in robotics. However, they specialize for the specific problem they are solving, and if the problem's context slightly changes, they cannot adapt properly. In fact, they require complete re-leaning in order to perform correctly in new unseen scenarios, regardless of how similar they are to previous learned environments. Contextual algorithms have recently emerged as solutions to this problem. They learn the policy for a task that depends on a given context, such that widely different contexts belonging to the same task are learned simultaneously. That being said, the state-of-the-art proposals of this class of algorithms prematurely converge, and simply cannot compete with algorithms that learn a policy for a single context. We describe the Contextual Relative Entropy Policy Search (CREPS) algorithm, which belongs to the before-mentioned class of contextual algorithms. We extend it with a technique that allows the algorithm to severely increase its performance, and we call it Contextual Relative Entropy Policy Search with Covariance Matrix Adaptation (CREPS-CMA). We propose two variants, and demonstrate their behavior in a set of classic contextual optimization problems, and on complex simulator robot tasks.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 17