Deep learning-based human action recognition to leverage context awareness in collaborative assembly

• Tipo: Artigo em Revista Científica Internacional
• Data: 2023

Autores

Moutinho, D (Autor) Outra; Rocha, LF (Autor) FEUP; Costa, CM (Autor) Outra; Luis F Teixeira (Autor) FEUP; Germano Veiga (Autor) Outra

Revista

Título: Robotics and Computer-Integrated Manufacturing

Vol. 80

ISSN: 0736-5845

Editora: Elsevier

Indexação

ISI Web of Knowledge - 1 Citação

Scopus

Outras Informações

ID Authenticus: P-00X-A8M

DOI: 10.1016/j.rcim.2022.102449

Abstract (EN): Human-Robot Collaboration is a critical component of Industry 4.0, contributing to a transition towards more flexible production systems that are quickly adjustable to changing production requirements. This paper aims to increase the natural collaboration level of a robotic engine assembly station by proposing a cognitive system powered by computer vision and deep learning to interpret implicit communication cues of the operator. The proposed system, which is based on a residual convolutional neural network with 34 layers and a long -short term memory recurrent neural network (ResNet-34 + LSTM), obtains assembly context through action recognition of the tasks performed by the operator. The assembly context was then integrated in a collaborative assembly plan capable of autonomously commanding the robot tasks. The proposed model showed a great performance, achieving an accuracy of 96.65% and a temporal mean intersection over union (mIoU) of 94.11% for the action recognition of the considered assembly. Moreover, a task-oriented evaluation showed that the proposed cognitive system was able to leverage the performed human action recognition to command the adequate robot actions with near-perfect accuracy. As such, the proposed system was considered as successful at increasing the natural collaboration level of the considered assembly station.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 15