LSTM, ConvLSTM, MDN-RNN and GridLSTM Memory-based Deep Reinforcement Learning

• Type: Article in International Conference Proceedings Book
• Date: 2023

Authors

Duarte, FF (Author) Other; Lau, N (Author) Other; Pereira, A (Author) Other; reis, lp (Author) FEUP

Conference proceedings International

Title: Proceedings of the 15th International Conference on Agents and Artificial Intelligence, ICAART 2023, Volume 2, Lisbon, Portugal, February 22-24, 2023

Pages: 169-179

Indexing

Scopus - 0 Citations

Other information

Authenticus ID: P-00Y-FJN

DOI: 10.5220/0011664900003393

Abstract (EN): Memory-based Deep Reinforcement Learning has been shown to be a viable solution to successfully learn control policies directly from high-dimensional sensory data in complex vision-based control tasks. At the core of this success lies the Long Short-Term Memory or LSTM, a well-known type of Recurrent Neural Network. More recent developments have introduced the ConvLSTM, a convolutional variant of the LSTM and the MDN-RNN, a Mixture Density Network combined with an LSTM, as memory modules in the context of Deep Reinforcement Learning. The defining characteristic of the ConvLSTM is its ability to preserve spatial information, which may prove to be a crucial factor when dealing with vision-based control tasks while the MDN-RNN can act as a predictive memory eschewing the need to explicitly plan ahead. Also of interest to this work is the GridLSTM, a network of LSTM cells arranged in a multidimensional grid. The objective of this paper is therefore to perform a comparative study of several memory modules, based on the LSTM, ConvLSTM, MDN-RNN and GridLSTM in the scope of Deep Reinforcement Learning, and more specifically as the memory modules of the agent. All experiments were validated using the Atari 2600 videogame benchmark. © 2021 by SCITEPRESS ¿ Science and Technology Publications, Lda. All rights reserved.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 10