Hybrid time-spatial video saliency detection method to enhance human action recognition systems

• Tipo: Artigo em Revista Científica Internacional
• Data: 2024

Autores

Gharahbagh, AA (Autor) Outra; Hajihashemi, V (Autor) Outra; Marta Campos Ferreira (Autor) FEUP; Machado, JJM (Autor) FEUP; Tavares, JMRS (Autor) Outra

Revista

Título: Multimedia Tools and Applications

Vol. 83

ISSN: 1380-7501

Editora: Springer Nature

Indexação

ISI Web of Knowledge - 0 Citações

Scopus

Outras Informações

ID Authenticus: P-00Z-Z8H

DOI: 10.1007/s11042-024-18126-x

Abstract (EN): Since digital media has become increasingly popular, video processing has expanded in recent years. Video processing systems require high levels of processing, which is one of the challenges in this field. Various approaches, such as hardware upgrades, algorithmic optimizations, and removing unnecessary information, have been suggested to solve this problem. This study proposes a video saliency map based method that identifies the critical parts of the video and improves the system's overall performance. Using an image registration algorithm, the proposed method first removes the camera's motion. Subsequently, each video frame's color, edge, and gradient information are used to obtain a spatial saliency map. Combining spatial saliency with motion information derived from optical flow and color-based segmentation can produce a saliency map containing both motion and spatial data. A nonlinear function is suggested to properly combine the temporal and spatial saliency maps, which was optimized using a multi-objective genetic algorithm. The proposed saliency map method was added as a preprocessing step in several Human Action Recognition (HAR) systems based on deep learning, and its performance was evaluated. Furthermore, the proposed method was compared with similar methods based on saliency maps, and the superiority of the proposed method was confirmed. The results show that the proposed method can improve HAR efficiency by up to 6.5% relative to HAR methods with no preprocessing step and 3.9% compared to the HAR method containing a temporal saliency map.

Idioma: Inglês

Tipo (Avaliação Docente): Científica

Nº de páginas: 21