Synthesizing Human Activity for Data Generation

• Type: Article in International Scientific Journal
• Date: 2023

Authors

Romero, A (Author) Other; Pedro Carvalho (Author) Other; Luís Corte-Real (Author) FEUP; Pereira, A (Author) Other

Journal

Title: Journal of Imaging

Vol. 9

Publisher: MDPI

Indexing

ISI Web of Knowledge - 0 Citations

Scopus - 0 Citations

Other information

Authenticus ID: P-00Z-944

DOI: 10.3390/jimaging9100204

Abstract (EN): The problem of gathering sufficiently representative data, such as those about human actions, shapes, and facial expressions, is costly and time-consuming and also requires training robust models. This has led to the creation of techniques such as transfer learning or data augmentation. However, these are often insufficient. To address this, we propose a semi-automated mechanism that allows the generation and editing of visual scenes with synthetic humans performing various actions, with features such as background modification and manual adjustments of the 3D avatars to allow users to create data with greater variability. We also propose an evaluation methodology for assessing the results obtained using our method, which is two-fold: (i) the usage of an action classifier on the output data resulting from the mechanism and (ii) the generation of masks of the avatars and the actors to compare them through segmentation. The avatars were robust to occlusion, and their actions were recognizable and accurate to their respective input actors. The results also showed that even though the action classifier concentrates on the pose and movement of the synthetic humans, it strongly depends on contextual information to precisely recognize the actions. Generating the avatars for complex activities also proved problematic for action recognition and the clean and precise formation of the masks.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 16