Exploring communication protocols and centralized critics in multi-agent deep learning

• Type: Article in International Scientific Journal
• Date: 2020

Authors

Simoes, D (Author) Other; lau, n (Author) Other; reis, lp (Author) FEUP

Journal

Title: Integrated Computer-Aided Engineering

Vol. 27

Pages: 333-351

ISSN: 1069-2509

Publisher: IOS PRESS

Indexing

ISI Web of Knowledge - 8 Citations

Scopus - 9 Citations

Other information

Authenticus ID: P-00S-X73

DOI: 10.3233/ica-200631

Abstract (EN): Tackling multi-agent environments where each agent has a local limited observation of the global state is a non-trivial task that often requires hand-tuned solutions. A team of agents coordinating in such scenarios must handle the complex underlying environment, while each agent only has partial knowledge about the environment. Deep reinforcement learning has been shown to achieve super-human performance in single-agent environments, and has since been adapted to the multi-agent paradigm. This paper proposes A3C3, a multi-agent deep learning algorithm, where agents are evaluated by a centralized referee during the learning phase, but remain independent from each other in actual execution. This referee's neural network is augmented with a permutation invariance architecture to increase its scalability to large teams. A3C3 also allows agents to learn communication protocols with which agents share relevant information to their team members, allowing them to overcome their limited knowledge, and achieve coordination. A3C3 and its permutation invariant augmentation is evaluated in multiple multi-agent test-beds, which include partially-observable scenarios, swarm environments, and complex 3D soccer simulations.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 19