Regularized Covariance Estimation for Weighted Maximum Likelihood Policy Search Methods

• Tipo: Artigo em Livro de Atas de Conferência Internacional
• Data: 2015

Autores

Abdolmaleki, A (Autor) Outra; lau, n (Autor) FCUP; reis, lp (Autor) REIT; Neumann, G (Autor) Outra

Ata de Conferência Internacional

Título: IEEE-RAS International Conference on Humanoid Robots

Páginas: 154-159

15th IEEE RAS International Conference on Humanoid Robots, Humanoids 2015

3 November 2015 through 5 November 2015

Indexação

ISI Web of Knowledge - 3 Citações

Scopus - 12 Citações

Outras Informações

ID Authenticus: P-00K-AP3

DOI: 10.1109/HUMANOIDS.2015.7363529

Abstract (EN): Many episode-based (or direct) policy search algorithms, maintain a multivariate Gaussian distribution as search distribution over the parameter space of some objective function. One class of algorithms, such as episodic REPS, PoWER or PI2 uses, a weighted maximum likelihood estimate (WMLE) to update the mean and covariance matrix of this distribution in each iteration. However, due to high dimensionality of covariance matrices and limited number of samples, the WMLE is an unreliable estimator. The use of WMLE leads to overfitted covariance estimates, and, hence the variance/entropy of the search distribution decreases too quickly, which may cause premature convergence. In order to alleviate this problem, the estimated covariance matrix can be regularized in different ways, for example by using a convex combination of the diagonal covariance estimate and the sample covariance estimate. In this paper, we propose a new covariance matrix regularization technique for policy search methods that uses the convex combination of the sample covariance matrix and the old covariance matrix used in last iteration. The combination weighting is determined by specifying the desired entropy of the new search distribution. With this mechanism, the entropy of the search distribution can be gradually decreased without damage from the maximum likelihood estimate.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 6