kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Joint Trajectory and Handover Management for UAVs Co-existing with Terrestrial Users: A Multi-Agent DRL Approach
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Communication Systems, CoS.ORCID iD: 0000-0002-3968-1179
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Communication Systems, CoS.
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Communication Systems, CoS.ORCID iD: 0000-0001-5298-7490
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Communication Systems, CoS, Radio Systems Laboratory (RS Lab).ORCID iD: 0000-0001-8517-7996
Show others and affiliations
2025 (English)In: Article in journal (Other academic) Submitted
Abstract [en]

Despite increasing interest in cellular-connected unmanned aerial vehicles (UAVs), their integration into existing cellular networks poses substantial challenges, including intense interference from UAVs to terrestrial user equipments (UEs) and numerous redundant handovers. To jointly reduce the generated interference and redundant handovers of cellular-connected UAVs while keeping their low transmission delay, we define an optimization problem with total available bandwidth and quality of service (QoS) constraints. Then, we formulate the optimization problem as a partially observable Markov decision process (POMDP) within a cooperative game. We have further developed a collaborative trajectory and handover management scheme using a Multi-Agent Deep Reinforcement Learning (MADRL) algorithm, specifically the Q-learning with a MIXer network (QMIX) algorithm, to optimize the aforementioned three metrics jointly. To demonstrate the superiority of our proposed scheme, we compare it with two benchmarks, namely the conventional handover management (CHM) scheme and the independent dueling double deep recurrent Q-network (ID3RQN) scheme. Simulation results show that QMIX outperforms the other schemes. Compared with the CHM scheme, QMIX reduces the delay, interference, and number of handovers for UAVs by an average of 46.9%, 70.0% and 90.5%, respectively. Compared with the ID3RQN scheme, QMIX reduces the three metrics by an average of 90.0%, 43.0% and 41.7%, respectively. 

Place, publisher, year, edition, pages
2025.
Keywords [en]
Cellular-connected UAVs, Trajectory design, Handover management, Multi-agent deep reinforcement learning, Multi-objective optimization.
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Computer Science; Telecommunication
Identifiers
URN: urn:nbn:se:kth:diva-358035OAI: oai:DiVA.org:kth-358035DiVA, id: diva2:1923986
Funder
Vinnova
Note

Submitted to: IEEE Transactions on Cognitive Communications and Networking 

Available from: 2025-01-02 Created: 2025-01-02 Last updated: 2025-01-03Bibliographically approved
In thesis
1. AI Assisted Mobility Management for Cellular Connected UAVs
Open this publication in new window or tab >>AI Assisted Mobility Management for Cellular Connected UAVs
2025 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Unmanned Aerial Vehicles (UAVs) connected to cellular networks, i.e., cellular-connected UAVs, introduce unique challenges and opportunities in mobility management that distinguish them from terrestrial users. This thesis presents a comprehensive approach for optimizing UAV integration into cellular networks.

We first investigate the distinct mobility management needs for cellular-connected UAVs. Unlike terrestrial mobility management, which primarily focuses on preventing radio link failures at cell edges, UAVs experience fragmented and overlapping coverage, often with line-of-sight visibility to multiple ground base stations (BSs). Consequently, UAV mobility management must address not only link stability but also the minimization of unnecessary handovers with sustained service availability, particularly in uplink scenarios.To tackle these challenges, we propose two solutions, a model-based handover parameter optimization algorithm and a model-free deep reinforcement learning (DRL) based handover algorithm, both designed specifically for UAV mobility management.We extend the problem by integrating UAV path planning with wireless objectives, including interference management, delay reduction, and minimized handovers. This results in a joint optimization framework for UAV trajectory planning, handover management, and radio resource allocation. To solve this multi-objective problem, we develop a multi-agent DRL algorithm that combines mission-specific trajectory planning with network-driven adjustments, optimizing resource allocation and handover transitions.

Furthermore, we address mobility management in multi-connectivity scenarios where UAVs are served by clusters of distributed BSs. As UAVs move, the serving BS clusters must be dynamically reconfigured, necessitating coordinated resource allocation under stringent and time-sensitive reliability constraints. We propose a centralized, fully distributed, and hierarchical DRL-based approaches to achieve reliable connectivity, reduce power consumption, and minimize cluster reconfiguration frequency.

Lastly, to evaluate a network’s capability to support range-based localization for cellular-connected UAVs, we introduce an analytical framework. This framework defines B-localizability as the probability of a UAV receiving sufficient localization signals from at least B ground BSs, meeting a specific Signal-to-Interference-plus-Noise Ratio (SINR) threshold. By incorporating UAV parameters within a three-dimensional environment, we provide insights into localizability factors such as distance distributions, path loss, interference, and SINR. 

Abstract [sv]

Obemannade luftfarkoster (UAV:er) som är anslutna till mobilnätverk medför unika utmaningar och möjligheter inom mobilitetshantering som skiljer sig från dem för markbundna användare. Denna avhandling presenterar ett omfattande tillvägagångssätt för att optimera UAV-integration med mobilnätverk.

Vi undersöker först de särskilda behoven av mobilitetshantering för cellulärt anslutna UAV:er. Till skillnad från mobilitetshantering för markanvändare, som främst fokuserar på att förhindra radiolänkfel vid cellkanter, upplever UAV:er fragmenterad och överlappande täckning med siktlinje till flera markbasstationer (BS:er). Därför måste mobilitetshanteringen för UAV:er inte bara hantera länkstabilitet utan även minimera onödiga överlämningar och säkerställa bibehållen tjänstetillgänglighet, särskilt i uppströmskommunikation.

För att hantera dessa utmaningar föreslår vi både modellbaserade och modellfria algoritmer specifikt utformade för UAV-mobilitetshantering. Vi utökar problemet genom att integrera UAV-ruttplanering med trådlösa mål, inklusive störningshantering, minskad fördröjning och minimerade överlämningar. Detta resulterar i en gemensam optimeringsram för UAV-ruttplanering, överlämningshantering och radioresurstilldelning. För att lösa detta multiobjektivproblem utvecklar vi en algoritm baserad på djup förstärkningsinlärning (DRL) som kombinerar uppdragsbaserad ruttplanering med nätverksdrivna justeringar, vilket optimerar resursallokering och överlämningshantering.

Vidare behandlar vi mobilitetshantering i multikonnektivitetsscenarier där UAV:er betjänas av kluster av distribuerade basstationer. När UAV:er rör sig måste de servande BS-klustren dynamiskt omkonfigureras, vilket kräver samordnad resursallokering under strikta och tidskänsliga tillförlitlighetskrav. Vi föreslår ett centraliserat, fullt distribuerat och hierarkiskt DRL-baserat tillvägagångssätt för att uppnå tillförlitlig anslutning, minska strömförbrukningen och minimera frekvensen av klusteromkonfigureringar.

Slutligen, för att utvärdera nätverkets förmåga att stödja positionsbaserad lokalisering för cellulärt anslutna UAV:er, introducerar vi en analytisk ram. Denna ram definierar B-lokaliserbarhet som sannolikheten för att en UAV tar emot tillräckliga lokaliseringssignaler från minst B markbasstationer, som uppfyller en specifik signal-till-interferens-plus-brusförhållande (SINR) tröskel. Genom att inkludera UAV-parametrar i en tredimensionell miljö tillhandahåller vi insikter om lokaliserbarhetsfaktorer som distansfördelningar, dämpning, interferens och SINR. 

Place, publisher, year, edition, pages
Sweden: KTH Royal Institute of Technology, 2025. p. 224
Series
TRITA-EECS-AVL ; 2025:10
Keywords
Unmanned aerial vehicles, Reinforcement Learning, Wireless networks, Reliability, Air-to-ground channel, Mobility management, Han dover, Unmanned aerial vehicles, Reinforcement Learning, Wireless networks, Reliability, Air-to-ground channel, Mobility management, Handover
National Category
Communication Systems Engineering and Technology
Research subject
Telecommunication
Identifiers
urn:nbn:se:kth:diva-358036 (URN)978-91-8106-164-2 (ISBN)
Public defence
2025-01-31, https://kth-se.zoom.us/j/61632995144, Ka-Sal C, Kistangangen 16, Kista, 10:00 (English)
Opponent
Supervisors
Funder
Vinnova
Note

QC 20250102

Available from: 2025-01-02 Created: 2025-01-02 Last updated: 2025-01-27Bibliographically approved

Open Access in DiVA

fulltext(660 kB)105 downloads
File information
File name FULLTEXT01.pdfFile size 660 kBChecksum SHA-512
9bf80ac74c2b4e22982c301baee3442235e97a20332fead6f5dbcc1003d31a2334789ed236d475dab55ae1bd2b7620542cddd4ede80261f122a52d98070b3529
Type fulltextMimetype application/pdf

Authority records

Deng, YuhangZhang, ShuaiMeer, Irshad AhmadÖzger, MustafaCavdar, Cicek

Search in DiVA

By author/editor
Deng, YuhangZhang, ShuaiMeer, Irshad AhmadÖzger, MustafaCavdar, Cicek
By organisation
Communication Systems, CoSRadio Systems Laboratory (RS Lab)
Electrical Engineering, Electronic Engineering, Information Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 105 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 304 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf