Dynamically Choosing the Number of Heads in Multi-Head Attention

• Type: Article in International Conference Proceedings Book
• Date: 2024

Authors

Duarte, FF (Author) Other; Lau, N (Author) Other; Pereira, A (Author) Other; reis, lp (Author) FEUP

Conference proceedings International

Title: Proceedings of the 16th International Conference on Agents and Artificial Intelligence, ICAART 2024, Volume 2, Rome, Italy, February 24-26, 2024

Pages: 358-367

Indexing

Scopus - 0 Citations

Other information

Authenticus ID: P-010-A06

DOI: 10.5220/0012384500003636

Abstract (EN): Deep Learning agents are known to be very sensitive to their parameterization values. Attention-based Deep Reinforcement Learning agents further complicate this issue due to the additional parameterization associated to the computation of their attention function. One example of this concerns the number of attention heads to use when dealing with multi-head attention-based agents. Usually, these hyperparameters are set manually, which may be neither optimal nor efficient. This work addresses the issue of choosing the appropriate number of attention heads dynamically, by endowing the agent with a policy h trained with policy gradient. At each timestep of agent-environment interaction, h is responsible for choosing the most suitable number of attention heads according to the contextual memory of the agent. This dynamic parameterization is compared to a static parameterization in terms of performance. The role of h is further assessed by providing additional analysis concerning the distribution of the number of attention heads throughout the training procedure and the course of the game. The Atari 2600 videogame benchmark was used to perform and validate all the experiments. © 2024 by SCITEPRESS ¿ Science and Technology Publications, Lda.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 9