Despite increasing interest in cellular-connected unmanned aerial vehicles (UAVs), their integration into existing cellular networks poses substantial challenges, including intense interference from UAVs to terrestrial user equipments (UEs) and numerous redundant handovers. To jointly reduce the generated interference and redundant handovers of cellular-connected UAVs while keeping their low transmission delay, we define an optimization problem subject to constraints on total available bandwidth and quality of service (QoS). Then, we formulate the optimization problem as a decentralized partially observable Markov decision process (Dec-POMDP) in the context of a cooperative game. We further develope a collaborative trajectory and handover management scheme using a multi-agent deep reinforcement learning algorithm, specifically the Q-learning with a MIXer network (QMIX) algorithm, to jointly optimize the aforementioned three metrics. Simulation results demonstrate that QMIX significantly outperforms two benchmark schemes: the conventional handover management (CHM) scheme and the independent dueling double deep recurrent Q-network (ID3RQN) scheme. Compared with the CHM scheme, QMIX reduces the delay, interference, and number of handovers for UAVs by an average of 46.9%, 70.0% and 90.5%, respectively. Compared with the ID3RQN scheme, QMIX reduces the three metrics by an average of 90.0%, 43.0% and 41.7%, respectively.

Unmanned Aerial Vehicles (UAVs) connected to cellular networks, i.e., cellular-connected UAVs, introduce unique challenges and opportunities in mobility management that distinguish them from terrestrial users. This thesis presents a comprehensive approach for optimizing UAV integration into cellular networks.

We first investigate the distinct mobility management needs for cellular-connected UAVs. Unlike terrestrial mobility management, which primarily focuses on preventing radio link failures at cell edges, UAVs experience fragmented and overlapping coverage, often with line-of-sight visibility to multiple ground base stations (BSs). Consequently, UAV mobility management must address not only link stability but also the minimization of unnecessary handovers with sustained service availability, particularly in uplink scenarios.To tackle these challenges, we propose two solutions, a model-based handover parameter optimization algorithm and a model-free deep reinforcement learning (DRL) based handover algorithm, both designed specifically for UAV mobility management.We extend the problem by integrating UAV path planning with wireless objectives, including interference management, delay reduction, and minimized handovers. This results in a joint optimization framework for UAV trajectory planning, handover management, and radio resource allocation. To solve this multi-objective problem, we develop a multi-agent DRL algorithm that combines mission-specific trajectory planning with network-driven adjustments, optimizing resource allocation and handover transitions.

Furthermore, we address mobility management in multi-connectivity scenarios where UAVs are served by clusters of distributed BSs. As UAVs move, the serving BS clusters must be dynamically reconfigured, necessitating coordinated resource allocation under stringent and time-sensitive reliability constraints. We propose a centralized, fully distributed, and hierarchical DRL-based approaches to achieve reliable connectivity, reduce power consumption, and minimize cluster reconfiguration frequency.

Lastly, to evaluate a network’s capability to support range-based localization for cellular-connected UAVs, we introduce an analytical framework. This framework defines B-localizability as the probability of a UAV receiving sufficient localization signals from at least B ground BSs, meeting a specific Signal-to-Interference-plus-Noise Ratio (SINR) threshold. By incorporating UAV parameters within a three-dimensional environment, we provide insights into localizability factors such as distance distributions, path loss, interference, and SINR. 

Obemannade luftfarkoster (UAV:er) som är anslutna till mobilnätverk medför unika utmaningar och möjligheter inom mobilitetshantering som skiljer sig från dem för markbundna användare. Denna avhandling presenterar ett omfattande tillvägagångssätt för att optimera UAV-integration med mobilnätverk.

Vi undersöker först de särskilda behoven av mobilitetshantering för cellulärt anslutna UAV:er. Till skillnad från mobilitetshantering för markanvändare, som främst fokuserar på att förhindra radiolänkfel vid cellkanter, upplever UAV:er fragmenterad och överlappande täckning med siktlinje till flera markbasstationer (BS:er). Därför måste mobilitetshanteringen för UAV:er inte bara hantera länkstabilitet utan även minimera onödiga överlämningar och säkerställa bibehållen tjänstetillgänglighet, särskilt i uppströmskommunikation.

För att hantera dessa utmaningar föreslår vi både modellbaserade och modellfria algoritmer specifikt utformade för UAV-mobilitetshantering. Vi utökar problemet genom att integrera UAV-ruttplanering med trådlösa mål, inklusive störningshantering, minskad fördröjning och minimerade överlämningar. Detta resulterar i en gemensam optimeringsram för UAV-ruttplanering, överlämningshantering och radioresurstilldelning. För att lösa detta multiobjektivproblem utvecklar vi en algoritm baserad på djup förstärkningsinlärning (DRL) som kombinerar uppdragsbaserad ruttplanering med nätverksdrivna justeringar, vilket optimerar resursallokering och överlämningshantering.

Vidare behandlar vi mobilitetshantering i multikonnektivitetsscenarier där UAV:er betjänas av kluster av distribuerade basstationer. När UAV:er rör sig måste de servande BS-klustren dynamiskt omkonfigureras, vilket kräver samordnad resursallokering under strikta och tidskänsliga tillförlitlighetskrav. Vi föreslår ett centraliserat, fullt distribuerat och hierarkiskt DRL-baserat tillvägagångssätt för att uppnå tillförlitlig anslutning, minska strömförbrukningen och minimera frekvensen av klusteromkonfigureringar.

Slutligen, för att utvärdera nätverkets förmåga att stödja positionsbaserad lokalisering för cellulärt anslutna UAV:er, introducerar vi en analytisk ram. Denna ram definierar B-lokaliserbarhet som sannolikheten för att en UAV tar emot tillräckliga lokaliseringssignaler från minst B markbasstationer, som uppfyller en specifik signal-till-interferens-plus-brusförhållande (SINR) tröskel. Genom att inkludera UAV-parametrar i en tredimensionell miljö tillhandahåller vi insikter om lokaliserbarhetsfaktorer som distansfördelningar, dämpning, interferens och SINR. 

Multi-connectivity (MC) for aerial users via a set of ground access points offers the potential for highly reliable communication. Within an open radio access network (O-RAN) architecture, edge clouds (ECs) enable MC with low latency for users within their coverage area. However, ensuring seamless service continuity for transitional users - those moving between the coverage areas of neighboring ECs - poses challenges due to centralized processing demands. To address this, we formulate a problem facilitating soft handovers between ECs, ensuring seamless transitions while maintaining service continuity for all users. We propose a hierarchical multi-agent reinforcement learning (HMARL) algorithm to dynamically determine the optimal functional split configuration for transitional and non-transitional users. Simulation results show that the proposed approach outperforms the conventional functional split in terms of the percentage of users maintaining service continuity, with at most 4% optimality gap. Additionally, HMARL achieves better scalability compared to the static baselines.

Despite increasing interest in cellular-connected unmanned aerial vehicles (UAVs), their integration into existing cellular networks poses substantial challenges, including intense interference from UAVs to terrestrial user equipments (UEs) and numerous redundant handovers. To jointly reduce the generated interference and redundant handovers of cellular-connected UAVs while keeping their low transmission delay, we define an optimization problem with total available bandwidth and quality of service (QoS) constraints. Then, we formulate the optimization problem as a partially observable Markov decision process (POMDP) within a cooperative game. We have further developed a collaborative trajectory and handover management scheme using a Multi-Agent Deep Reinforcement Learning (MADRL) algorithm, specifically the Q-learning with a MIXer network (QMIX) algorithm, to optimize the aforementioned three metrics jointly. To demonstrate the superiority of our proposed scheme, we compare it with two benchmarks, namely the conventional handover management (CHM) scheme and the independent dueling double deep recurrent Q-network (ID3RQN) scheme. Simulation results show that QMIX outperforms the other schemes. Compared with the CHM scheme, QMIX reduces the delay, interference, and number of handovers for UAVs by an average of 46.9%, 70.0% and 90.5%, respectively. Compared with the ID3RQN scheme, QMIX reduces the three metrics by an average of 90.0%, 43.0% and 41.7%, respectively. 

In modern cell-less wireless networks, mobility management is undergoing a significant transformation, transitioning from single-link handover management to a more adaptable multi-connectivity cluster reconfiguration approach, including often conflicting objectives like energy-efficient power allocation and satisfying varying reliability requirements. In this work, we address the challenge of dynamic clustering and power allocation for unmanned aerial vehicle (UAV) communication in wireless interference networks. Our objective encompasses meeting varying reliability demands, minimizing power consumption, and reducing the frequency of cluster reconfiguration. To achieve these objectives, we introduce a novel approach based on reinforcement learning using a masked soft actor-critic algorithm, specifically tailored for dynamic clustering and power allocation.

Unmanned Aerial Vehicles (UAVs) connected to cellular networks present novel challenges and opportunities in mobility management and localization, distinct from those faced by terrestrial users. This thesis presents an integrated approach, combining two key aspects essential for the integration of UAVs with cellular networks.

Firstly, it introduces the mobility management challenges for cellular-connected UAVs, which differ significantly from terrestrial users. While terrestrial mobility management primarily aims to prevent radio link failures near cell boundaries, aerial users experience fragmented and overlapping coverage with line-of-sight conditions involving multiple ground base stations (BSs). Thus, mobility management for UAVs extends beyond link failure avoidance, aiming to minimize unnecessary handovers while ensuring extended service availability, particularly in up-link communication. Line-of-sight conditions from a UAV to multiple BSs increase the likelihood of frequent handovers, resulting in control packet overheads and communication delays. This thesis proposes two approaches to address these challenges: 1) A model-based service availability-aware Mobility Robustness Optimization (MRO) adapting handover parameters to maintain high service availability with minimal handovers, and 2) A model-free approach using Deep Q-networks to decrease unnecessary handovers while preserving high service availability. Simulation results demonstrate that both the proposed algorithms converge promptly and increase the service availability by more than 40 %  while the number of handovers is reduced by more than 50%  as compared to traditional approaches.

Secondly, to assess the ability of a network to support the range-based localization for cellular-connected UAVs, an analytical framework is introduced. The metric B-localizability is defined as the probability of successfully receiving localization signals above a specified Signal-to-Interference plus Noise Ratio (SINR) threshold from at least B ground BSs. The framework, accounting for UAV-related parameters in a three-dimensional environment, provides comprehensive insights into factors influencing localizability, such as distance distributions, path loss, interference, and received SINR. Simulation studies explore the correlation between localizability and the number of participating BSs, SINR requirements, air-to-ground channel characteristics, and network coordination. Additionally, an optimization problem is formulated to maximize localizability, investigating the impact of UAV altitude across different scenarios. Our study reveals that in an urban macro environment, the effectiveness of cellular network-based localization increases with altitude, with localizability reaching 100% above 60 meters. This finding indicates that utilizing cellular networks for UAV localization is a viable option.

Unmanned Aerial Vehicles (UAV) anslutna till cellulära nätverk presenterar nya utmaningar och möjligheter inom mobilitetshantering och lokalisering, skilda från dem som markanvändare står inför. Denna avhandling presenterar ett integrerat tillvägagångssätt, som kombinerar två nyckelaspekter som är väsentliga för integrationen av UAV:er med cellulära nätverk.

För det första introducerar den mobilitetshanteringsutmaningarna för mobilanslutna UAV:er, som skiljer sig avsevärt från markbundna användare. Medan markbunden mobilitetshantering i första hand syftar till att förhindra radiolänkfel nära cellgränser, upplever antennanvändare fragmenterad och överlappande täckning med siktlinjeförhållanden som involverar flera markbasstationer (BS). Mobilitetshantering för UAV sträcker sig sålunda bortom att undvika länkfel, och syftar till att minimera onödiga överlämningar samtidigt som man säkerställer utökad servicetillgänglighet, särskilt i upplänkskommunikation. Synlinjeförhållanden från en UAV till flera BS:er ökar sannolikheten för frekventa överlämningar, vilket resulterar i kontrollpaketkostnader och kommunikationsförseningar. Denna avhandling föreslår två tillvägagångssätt för att möta dessa utmaningar: 1) En modellbaserad tjänsttillgänglighetsmedveten Mobility Robustness Optimization (MRO) som anpassar parametrar för överlämning för att bibehålla hög servicetillgänglighet med minimal överlämning, och 2) Ett modellfritt tillvägagångssätt med Deep Q- nätverk för att minska onödiga överlämningar samtidigt som hög servicetillgänglighet bibehålls. Simuleringsresultat visar att båda de föreslagna algoritmerna konvergerar snabbt och ökar tjänstens tillgänglighet med mer än 40% medan antalet överlämningar minskas med mer än 50% jämfört med traditionella metoder.

För det andra, för att bedöma förmågan hos ett nätverk att stödja den räckviddsbaserade lokaliseringen för de cellulärt anslutna UAV:erna, introduceras ett analytiskt ramverk.Metriska B-lokaliseringsförmågan definieras som sannolikheten för att framgångsrikt ta emot lokaliseringssignaler över en specificerad signal-till-interferens plus brusförhållande (SINR) tröskel från minst B jord BSs.Ramverket, som tar hänsyn till UAV-relaterade parametrar i en tredimensionell miljö, ger omfattande insikter i faktorer som påverkar lokaliserbarhet, såsom avståndsfördelningar, vägförlust, störningar och mottagen SINR. Simuleringsstudier undersöker korrelationen mellan lokaliserbarhet och antalet deltagande BS:er, SINR-krav, luft-till-mark-kanalegenskaper och nätverkskoordination. Dessutom har ett optimeringsproblem formulerats för att maximera lokaliseringsförmågan, undersöka effekten av UAV-höjd över olika scenarier. Vår studie avslöjar att i en urban makromiljö ökar effektiviteten av mobilnätsbaserad lokalisering med höjden, med lokaliserbarhet som når 100% över $60$ meter. Detta fynd indikerar att användning av mobilnät för UAV-lokalisering är ett gångbart alternativ.

Mobility management for terrestrial users is mostlyconcerned with avoiding radio link failure for the edge users wherethe cell boundaries are defined. The problem becomes interestingfor an aerial user experiencing fragmented coverage in the sky andline-of-sight conditions with multiple ground base stations (BSs).For aerial users, mobility management is not only concerned withavoiding link failures but also avoiding unnecessary handoverswhile maintaining extended service availability, especially inup-link communication. The line of sight conditions from anUnmanned Aerial Vehicle (UAV) to multiple neighboring BSs makeit more prone to frequent handovers, leading to control packetoverheads and delays in the communication service. Depending onthe use cases, UAVs require a certain level of service availability,which makes their mobility management a critical task. Thecurrent mobility robustness optimization (MRO) procedure thatadaptively manages handover parameters to avoid unnecessaryhandovers is optimized only for terrestrial users. It needs tobe updated to capture the unique mobility challenges of aerialusers. In this work, we propose two approaches to accomplishthis: 1) A model based service availability-aware MRO wherehandover control parameters, such as handover margin and timeto trigger are tuned to maintain high service availability witha minimum number of handovers, and, 2) A deep Q-networkbased model free approach for decreasing unnecessary handoverswhile maintaining high service availability. Simulation resultsdemonstrate that both the proposed algorithms converge promptlyand increase the service availability by more than 40% while thenumber of handovers is reduced by more than 50% as comparedto traditional approaches.

To enable pervasive applications of cellular-connected unmanned aerial vehicles (UAVs), localization plays a key role. The successful reception of localization signals from multiple base stations (BSs) is the first step to localize targets, which is called cellular localizability. In this paper, we propose an analytical framework to characterize the B-localizability of UAVs, which is defined as the probability of successfully receiving localization signals above a certain signal-to-interference plus noise ratio (SINR) level from at least B ground BSs. Our framework considers UAV-related system parameters in a three-dimensional environment and provides a comprehensive insight into factors affecting localizability such as distance distributions, path loss, interference, and received SINR. We perform simulation studies to explore the relationship between localizability and the number of participating BSs, SINR requirements of the received localization signals, air-to-ground channel characteristics, and network coordination. We also formulate an optimization problem to maximize localizability and investigate the effects of UAV altitude in different scenarios. Our study reveals that in an urban macro environment, the effectiveness of cellular network-based localization increases with altitude, with localizability reaching 100% above 60 meters. This finding indicates that utilizing cellular networks for UAV localization is a viable option.

The ubiquitous cellular network is a strong candidate for providing UAVs’ wireless connectivity. Due to the maneuverability advantage and higher altitude, UAVs could have line-of-sight (LoS) connectivity with more base station (BS) candidates than terrestrial users. However, the LoS connectivity could also enhance the propagation of up-link interference caused by UAVs over co-existing terrestrial users. In addition, UAVs would perform more handovers than terrestrial users when moving due to the extensive overlap in the coverage areas of many BS candidates. The solution is to bypass the overlapping coverage areas by designing the UAVs’ trajectory and to reduce interference by optimizing radio resource allocation through handover management. This paper studies the joint optimization of a UAV’s trajectory design and handover management to minimize the weighted sum of three key performance indicators (KPIs): delay, up-link interference, and handover numbers. A dueling double deep Q-network (D3QN) based reinforcement learning algorithm is proposed to solve the optimization problem. Results show that the proposed approach can reduce the handover numbers by 90% and the interference by 18% at the cost of a small increment in transmission delay when compared with the benchmark scheme, which controls the UAV to move along the shortest path and perform handover based on received signal strength. Finally, we verify the advantage of introducing trajectory design, which can reduce the interference by 29% and eliminate the handover numbers by 33% when compared to the D3QN-based policy without trajectory design.

Modern communication systems need to fulfill multiple and often conflicting objectives at the same time. In particular, new applications require high reliability while operating at low transmit powers. Moreover, reliability constraints may vary over time depending on the current state of the system. One solution to address this problem is to use joint transmissions from a number of base stations (BSs) to meet the reliability requirements. However, this approach is inefficient when considering the overall total transmit power. In this work, we propose a reinforcement learning-based power allocation scheme for an unmanned aerial vehicle (UAV) communication system with varying communication reliability requirements. In particular, the proposed scheme aims to minimize the total transmit power of all BSs while achieving an outage probability that is less than a tolerated threshold. This threshold varies over time, e.g., when the UAV enters a critical zone with high-reliability requirements. Our results show that the proposed learning scheme uses dynamic power allocation to meet varying reliability requirements, thus effectively conserving energy.