Study on LSTM and ConvLSTM Memory-Based Deep Reinforcement Learning

• Type: Article in International Conference Proceedings Book
• Date: 2024

Authors

Duarte, FF (Author) Other; Lau, N (Author) Other; Pereira, A (Author) Other; reis, lp (Author) FEUP

Conference proceedings International

Title: AGENTS AND ARTIFICIAL INTELLIGENCE, ICAART 2023

Pages: 223-243

15th International Conference on Agents and Artificial Intelligence (ICAART)

Lisbon, PORTUGAL, FEB 22-24, 2023

Indexing

ISI Web of Knowledge - 0 Citations

Scopus - 0 Citations

Other information

Authenticus ID: P-010-9GE

DOI: 10.1007/978-3-031-55326-4_11

Abstract (EN): Memory-based Deep Reinforcement Learning (DRL) has been successfully applied to solve vision-based control tasks from high-dimensional sensory data. While most of this work leverages the Long Short-Term Memory (LSTM) as the memory module of the agent, recent developments have revisited and extended the original formulation of the LSTM. Some of these developments include the ConvLSTM, a convolutional-based implementation of the LSTM, the MDN-RNN, the combination of a Mixture Density Network with an LSTM and the GridLSTM, a multidimensional grid of LSTM cells. It seems however unclear how these different memory modules compare to each other in terms of agent performance, when applied in the context of DRL. This work aims to perform a comparative study of several memory-based DRL agents, based on the LSTM, ConvLSTM, MDN-RNN and GridLSTM memory modules. The results obtained seem to support the claim that in some cases these more recent memory modules can improve the performance of the agent, to varying degrees, when compared to a baseline agent based on an LSTM. The experimental results were validated in the Atari 2600 videogame platform.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 21