Multimodal PointPillars for Efficient Object Detection in Autonomous Vehicles

• Tipo: Artigo em Revista Científica Internacional
• Data: 2024

Autores

Oliveira M. (Autor) Outra; Cerqueira R. (Autor) Outra; Pinto J.R. (Autor) Outra; Fonseca J. (Autor) Outra; Luis F Teixeira (Autor) FEUP

Revista

Título: IEEE Transactions on Intelligent Vehicles

Páginas: 1-11

Editora: IEEE

Indexação

Scopus - 2 Citações

Outras Informações

ID Authenticus: P-010-JQT

DOI: 10.1109/tiv.2024.3409409

Abstract (EN): Autonomous Vehicles aim to understand their surrounding environment by detecting relevant objects in the scene, which can be performed using a combination of sensors. The accurate prediction of pedestrians is a particularly challenging task, since the existing algorithms have more difficulty detecting small objects. This work studies and addresses this often overlooked problem by proposing Multimodal PointPillars (M-PP), a fast and effective novel fusion architecture for 3D object detection. Inspired by both MVX-Net and PointPillars, image features from a 2D CNN-based feature map are fused with the 3D point cloud in an early fusion architecture. By changing the heavy 3D convolutions of MVX-Net to a set of convolutional layers in 2D space, along with combining LiDAR and image information at an early stage, M-PP considerably improves inference time over the baseline, running at 28.49 Hz. It achieves inference speeds suitable for real-world applications while keeping the high performance of multimodal approaches. Extensive experiments show that our proposed architecture outperforms both MVX-Net and PointPillars for the pedestrian class in the KITTI 3D object detection dataset, with 62.78% in <inline-formula><tex-math notation="LaTeX">$AP_{BEV}$</tex-math></inline-formula> (moderate difficulty), while also outperforming MVX-Net in the nuScenes dataset. Moreover, experiments were conducted to measure the detection performance based on object distance. The performance of M-PP surpassed other methods in pedestrian detection at any distance, particularly for faraway objects (more than 30 meters). Qualitative analysis shows that M-PP visibly outperformed MVX-Net for pedestrians and cyclists, while simultaneously making accurate predictions of cars.

Idioma: Inglês

Tipo (Avaliação Docente): Científica