Learning low level skills from scratch for humanoid robot soccer using deep reinforcement learning

• Type: Article in International Conference Proceedings Book
• Date: 2019

Authors

Miguel Abreu (Author) Other; Nuno Lau (Author) Other; Armando Jorge Sousa (Author) FEUP; Luís Paulo Reis (Author) FEUP

Conference proceedings International

Title: 2019 19TH IEEE INTERNATIONAL CONFERENCE ON AUTONOMOUS ROBOT SYSTEMS AND COMPETITIONS (ICARSC 2019)

Pages: 256-263

19th IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC)

Gondomar, PORTUGAL, APR 24-26, 2019

Indexing

ISI Web of Knowledge - 9 Citations

Scopus - 19 Citations

Other information

Authenticus ID: P-00Q-TA3

DOI: 10.1109/icarsc.2019.8733632

Abstract (EN): Reinforcement learning algorithms are now more appealing than ever. Recent approaches bring power and tuning simplicity to the everyday work machine. The possibilities are endless, and the idea of automating learning without domain knowledge is quite tempting for many researchers. However, in competitive environments such as the RoboCup 3D Soccer Simulation League, there is a lot to be done regarding humanlike behaviors. Current teams use many mechanical movements to perform basic skills, such as running and dribbling the ball. This paper aims to use the PPO algorithm to optimize those skills, achieving natural gaits without sacrificing performance. We use Simspark to simulate a NAO humanoid robot, using visual and body sensors to control its actuators. Based on our results, we propose an indirect control approach and detailed parameter setups to obtain natural running and dribbling behaviors. The obtained performance is in some cases comparable or better than the top RoboCup teams. However, some skills are not ready to be applied in competitive environments yet, due to instability. This work contributes towards the improvement of RoboCup and some related technical challenges.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 8