A citywide TD-learning based intelligent traffic signal control for autonomous vehicles: Performance evaluation using SUMO

• Type: Article in International Scientific Journal
• Date: 2025

Authors

Selim Reza (Author) Other; Marta Campos Ferreira (Author) FEUP; J. J. M. Machado (Author) FEUP; João Manuel R. S. Tavares (Author) FEUP

Journal

Title: Expert Systems

Vol. 42 No. 1

Pages: e13301-1-18

ISSN: 0266-4720

Publisher: Wiley-Blackwell

Indexing

ISI Web of Knowledge - 3 Citations

ISI Web of Science

Scopus - 5 Citations

Clarivate’s Books Citation Index

Scientific classification

CORDIS: Technological sciences

FOS: Engineering and technology

Other information

Authenticus ID: P-00Y-70N

DOI: 10.1111/exsy.13301

Abstract (EN): An autonomous vehicle can sense its environment and operate without human involvement. Its adequate management in an intelligent transportation system could significantly reduce traffic congestion and overall travel time in a network. Adaptive traffic signal controller (ATSC) based on multi-agent systems using state-action-reward-state-action (SARSA (lambda)) are well-known state-of-the-art models to manage autonomous vehicles within urban areas. However, this study found inefficient weights updating mechanisms of the conventional SARSA (lambda) models. Therefore, it proposes a Gaussian function to regulate the eligibility trace vector's decay mechanism effectively. On the other hand, an efficient understanding of the state of the traffic environment is crucial for an agent to take optimal actions. The conventional models feed the state values to the agents through the MinMax normalization technique, which sometimes shows less efficiency and robustness. So, this study suggests the MaxAbs scaled state values instead of MinMax to address the problem. Furthermore, the combination of the A-star routing algorithm and proposed model demonstrated a good increase in performance relatively to the conventional SARSA (lambda)-based routing algorithms. The proposed model and the baselines were implemented in a microscopic traffic simulation environment using the SUMO package over a complex real-world-like 21-intersections network to evaluate their performance. The results showed a reduction of the vehicle's average total waiting time and total stops by a mean value of 59.9% and 17.55% compared to the considered baselines. Also, the A-star combined with the proposed controller outperformed the conventional approaches by increasing the vehicle's average trip speed by 3.4%.

Language: English

Type (Professor's evaluation): Scientific

No. of pages: 18