This article explores the impact of distribution shift in traffic signal control using reinforcement learning agents. Distribution shift occurs when the data distribution during deployment (i.e., test data) differs from the distribution of the training data, affecting model performance. Through simulations based on real-world traffic data, we analyze two key components of this shift: Kullback–Leibler (KL) divergence in traffic patterns and total traffic volume. Our results show that when the volume of vehicles increases by 1000 from the training volume, the average delay increases by 103%, while a 0.1 increase in KL divergence increases the delay by 58%. This work shows the measurable impact of distribution shift on artificial intelligence agent-based traffic signal system performance.

The use of Machine Learning (ML) and Artificial Intelligence (AI) in smart transportation networks has increased significantly in the last few years. Among these ML and AI approaches, Reinforcement Learning (RL) has been shown to be a very promising approach by several authors. However, a problem with using Reinforcement Learning in Traffic Signal Control is the reliability of the trained RL agents due to the dynamically changing distribution of the input data with respect to the distribution of the data used for training. This presents a major challenge and a reliability problem for the trained network of AI agents and could have very undesirable and even detrimental consequences if a suitable solution is not found. Several researchers have tried to address this problem using different approaches. In particular, Meta Reinforcement Learning (Meta RL) promises to be an effective solution. In this paper, we evaluate and analyze a state-of-the-art Meta RL approach called MetaLight and show that, while under certain conditions MetaLight can indeed lead to reasonably good results, under some other conditions it might not perform well (with errors of up to 22%), suggesting that Meta RL schemes are often not robust enough and can even pose major reliability problems.