Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Hierarchical multi-agent control of traffic lights based on collective learning
KTH, School of Architecture and the Built Environment (ABE), Civil and Architectural Engineering, Transport Planning, Economics and Engineering. (System Simulation & Control (S2CLab))ORCID iD: 0000-0002-1375-9054
KTH, School of Architecture and the Built Environment (ABE), Civil and Architectural Engineering, Transport Planning, Economics and Engineering. iTekn Solutions, Stockholm, Sweden. (System Simulation & Control (S2CLab))
2018 (English)In: Engineering applications of artificial intelligence, ISSN 0952-1976, E-ISSN 1873-6769, Vol. 68, p. 236-248Article in journal (Refereed) Published
Abstract [en]

Increasing traffic congestion poses significant challenges for urban planning and management in metropolitan areas around the world. One way to tackle the problem is to resort to the emerging technologies in artificial intelligence. Traffic light control is one of the most traditional and important instruments for urban traffic management. The present study proposes a traffic light control system enabled by a hierarchical multi-agent modeling framework in a decentralized manner. In the framework, a traffic network is decomposed into regions represented by region agents. Each region consists of intersections, modeled by intersection agents who coordinate with neighboring intersection agents through communication. For each intersection, a collection of turning movement agents operate individually and implement optimal actions according to local control policies. By employing a reinforcement learning algorithm for each turning movement agent, the intersection controllers are enabled with the capability to make their timing decisions in a complex and dynamic environment. In addition, the traffic light control operates with an advanced phase composition process dynamically combining compatible turning movements. Moreover, the collective operations performed by the agents in a road network are further coordinated by varying priority settings for relevant turning movements. A case study was carried out by simulations to evaluate the performance of the proposed control system while comparing it with an optimized vehicle-actuated control system. The results show that the proposed traffic light system, after a collective machine learning process, not only improves the local signal operations at individual intersections but also enhances the traffic performance at the regional level through coordination of specific turning movements.

Place, publisher, year, edition, pages
Elsevier, 2018. Vol. 68, p. 236-248
Keywords [en]
Hierarchical model of traffic system, Multi-agent traffic light control, Decentralized system, Learning-based control, Collective machine learning
National Category
Other Engineering and Technologies Transport Systems and Logistics Computer and Information Sciences
Research subject
Transport Science
Identifiers
URN: urn:nbn:se:kth:diva-223272DOI: 10.1016/j.engappai.2017.10.013ISI: 000423894400021Scopus ID: 2-s2.0-85034445822OAI: oai:DiVA.org:kth-223272DiVA, id: diva2:1183324
Funder
J. Gust. Richert stiftelse, 2015-00205
Note

QC 20180514

Available from: 2018-02-16 Created: 2018-02-16 Last updated: 2018-05-25Bibliographically approved
In thesis
1. Advance Traffic Signal Control Systems with Emerging Technologies
Open this publication in new window or tab >>Advance Traffic Signal Control Systems with Emerging Technologies
2018 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Nowadays, traffic congestion poses critical problems including the undermined mobility and sustainability efficiencies. Mitigating traffic congestions in urban areas is a crucial task for both research and in practice. With decades of experience in road traffic controls, there is still room for improving traffic control measures; especially with the emerging technologies, such as artificial intelligence (AI), the Internet of Things (IoT), and Big Data. The focus of this thesis lies in the development and implementation of enhanced traffic signal control systems, one of the most ubiquitous and challenging traffic control measures.

This thesis makes the following major contributions. Firstly, a simulation-based optimization framework is proposed, which is inherently general in which various signal control types, and different simulation models and optimization methods can be integrated. Requiring heavy computing resources is a common issue of simulation-based optimization approaches, which is addressed by an advanced genetic algorithm and parallel traffic simulation in this study.

The second contribution is an investigation of an intelligent local control system. The local signal control operation is formulated as a sequential decision-making process where each controller or control component is modeled as an intelligent agent. The agents make decisions based on traffic conditions and the deployed road infrastructure, as well as the implemented control scheme. A non-parametric state estimation method and an adaptive control scheme by reinforcement learning (RL) are introduced to facilitate such an intelligent system.

The local intelligence is expanded to an arterial road using a decentralized design, which is enabled by a hierarchical framework. Then, a network of signalized intersections is operated under the cooperation of agents at different levels of hierarchy. An agent at a lower level is instructed by the agent at the next higher level toward a common operational goal. Agents at the same level can communicate with their neighbors and perform collective behaviors.

Additionally, a multi-objective RL approach is in use to handle the potential conflict between agents at different hierarchical levels. Simulation experiments have been carried out, and the results verify the capabilities of the proposed methodologies in traffic signal control applications. Furthermore, this thesis demonstrates an opportunity to employ the systems in practice when the system is programmed on an intermediate hardware device. Such a device can receive streaming detection data from signal controller hardware or the simulation environment and override the controlled traffic lights in real time.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2018
Series
TRITA-ABE-DLT ; 189
National Category
Transport Systems and Logistics Computer Sciences
Research subject
Transport Science
Identifiers
urn:nbn:se:kth:diva-227713 (URN)978-91-7729-758-1 (ISBN)
Public defence
2018-06-12, F3, Lindstedtsvägen 26, Stockholm, 14:00 (English)
Opponent
Supervisors
Available from: 2018-05-14 Created: 2018-05-11 Last updated: 2018-05-14Bibliographically approved

Open Access in DiVA

fulltext(1947 kB)12 downloads
File information
File name FULLTEXT01.pdfFile size 1947 kBChecksum SHA-512
c6a8f3c5d86f0bdbc95ba9e974f3f4bef339f49935b9d376d5879606adaee78a68fa96c028c20c79e7ca7a94cb8f0eb48383ebca893cb96d1bcfee19da8acfd3
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records BETA

Jin, JunchenMa, Xiaoliang

Search in DiVA

By author/editor
Jin, JunchenMa, Xiaoliang
By organisation
Transport Planning, Economics and Engineering
In the same journal
Engineering applications of artificial intelligence
Other Engineering and TechnologiesTransport Systems and LogisticsComputer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 12 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 243 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf