Key Indicators to Assess the Performance of LiDAR-Based Perception Algorithms: A Literature Review

• Type: Another Publication in an International Scientific Journal
• Date: 2023-10-06

Authors

José Machado da Silva (Author) FEUP; K. Chiranjeevi (Author) FEUP; Correia, M. V. (Author) FEUP

Journal

Title: IEEE Access

Vol. 11

ISSN: 2169-3536

Publisher: IEEE

Indexing

ISI Web of Knowledge - 0 Citations

Scopus - 0 Citations

Google Scholar

INSPEC

Other information

Authenticus ID: P-00Z-3DT

DOI: 10.1109/access.2023.3321912

Abstract (EN): Perception algorithms are essential for autonomous or semi-autonomous vehicles to perceive the semantics of their surroundings, including object detection, panoptic segmentation, and tracking. Decision-making in case of safety-critical situations, like autonomous emergency braking and collision avoidance, relies on the outputs of these algorithms. This makes it essential to correctly assess such perception systems before their deployment and to monitor their performance when in use. It is difficult to test and validate these systems, particularly at runtime, due to the high-level and complex representations of their outputs. This paper presents an overview of different existing metrics used for the evaluation of LiDAR-based perception systems, emphasizing particularly object detection and tracking algorithms due to their importance in the final perception outcome. Along with generally used metrics, we also discuss the impact of Planning KL-Divergence (PKL), Timed Quality Temporal Logic (TQTL), and Spatio-temporal Quality Logic (STQL) metrics on object detection algorithms. In the case of panoptic segmentation, Panoptic Quality (PQ) and Parsing Covering (PC) metrics are analysed resorting to some pretrained models. Finally, it addresses the application of diverse metrics to evaluate different pretrained models with the respective perception algorithms on publicly available datasets. Besides the identification of the various metrics being proposed, their performance and influence on models are also assessed after conducting new tests or reproducing the experimental results of the reference under consideration.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 27