Nowadays, traffic congestion poses critical problems including the undermined mobility and sustainability efficiencies. Mitigating traffic congestions in urban areas is a crucial task for both research and in practice. With decades of experience in road traffic controls, there is still room for improving traffic control measures; especially with the emerging technologies, such as artificial intelligence (AI), the Internet of Things (IoT), and Big Data. The focus of this thesis lies in the development and implementation of enhanced traffic signal control systems, one of the most ubiquitous and challenging traffic control measures.

This thesis makes the following major contributions. Firstly, a simulation-based optimization framework is proposed, which is inherently general in which various signal control types, and different simulation models and optimization methods can be integrated. Requiring heavy computing resources is a common issue of simulation-based optimization approaches, which is addressed by an advanced genetic algorithm and parallel traffic simulation in this study.

The second contribution is an investigation of an intelligent local control system. The local signal control operation is formulated as a sequential decision-making process where each controller or control component is modeled as an intelligent agent. The agents make decisions based on traffic conditions and the deployed road infrastructure, as well as the implemented control scheme. A non-parametric state estimation method and an adaptive control scheme by reinforcement learning (RL) are introduced to facilitate such an intelligent system.

The local intelligence is expanded to an arterial road using a decentralized design, which is enabled by a hierarchical framework. Then, a network of signalized intersections is operated under the cooperation of agents at different levels of hierarchy. An agent at a lower level is instructed by the agent at the next higher level toward a common operational goal. Agents at the same level can communicate with their neighbors and perform collective behaviors.

Additionally, a multi-objective RL approach is in use to handle the potential conflict between agents at different hierarchical levels. Simulation experiments have been carried out, and the results verify the capabilities of the proposed methodologies in traffic signal control applications. Furthermore, this thesis demonstrates an opportunity to employ the systems in practice when the system is programmed on an intermediate hardware device. Such a device can receive streaming detection data from signal controller hardware or the simulation environment and override the controlled traffic lights in real time.