Mobility management for terrestrial users is mostlyconcerned with avoiding radio link failure for the edge users wherethe cell boundaries are defined. The problem becomes interestingfor an aerial user experiencing fragmented coverage in the sky andline-of-sight conditions with multiple ground base stations (BSs).For aerial users, mobility management is not only concerned withavoiding link failures but also avoiding unnecessary handoverswhile maintaining extended service availability, especially inup-link communication. The line of sight conditions from anUnmanned Aerial Vehicle (UAV) to multiple neighboring BSs makeit more prone to frequent handovers, leading to control packetoverheads and delays in the communication service. Depending onthe use cases, UAVs require a certain level of service availability,which makes their mobility management a critical task. Thecurrent mobility robustness optimization (MRO) procedure thatadaptively manages handover parameters to avoid unnecessaryhandovers is optimized only for terrestrial users. It needs tobe updated to capture the unique mobility challenges of aerialusers. In this work, we propose two approaches to accomplishthis: 1) A model based service availability-aware MRO wherehandover control parameters, such as handover margin and timeto trigger are tuned to maintain high service availability witha minimum number of handovers, and, 2) A deep Q-networkbased model free approach for decreasing unnecessary handoverswhile maintaining high service availability. Simulation resultsdemonstrate that both the proposed algorithms converge promptlyand increase the service availability by more than 40% while thenumber of handovers is reduced by more than 50% as comparedto traditional approaches.

The ubiquitous cellular network is a strong candidate for providing UAVs’ wireless connectivity. Due to the maneuverability advantage and higher altitude, UAVs could have line-of-sight (LoS) connectivity with more base station (BS) candidates than terrestrial users. However, the LoS connectivity could also enhance the propagation of up-link interference caused by UAVs over co-existing terrestrial users. In addition, UAVs would perform more handovers than terrestrial users when moving due to the extensive overlap in the coverage areas of many BS candidates. The solution is to bypass the overlapping coverage areas by designing the UAVs’ trajectory and to reduce interference by optimizing radio resource allocation through handover management. This paper studies the joint optimization of a UAV’s trajectory design and handover management to minimize the weighted sum of three key performance indicators (KPIs): delay, up-link interference, and handover numbers. A dueling double deep Q-network (D3QN) based reinforcement learning algorithm is proposed to solve the optimization problem. Results show that the proposed approach can reduce the handover numbers by 90% and the interference by 18% at the cost of a small increment in transmission delay when compared with the benchmark scheme, which controls the UAV to move along the shortest path and perform handover based on received signal strength. Finally, we verify the advantage of introducing trajectory design, which can reduce the interference by 29% and eliminate the handover numbers by 33% when compared to the D3QN-based policy without trajectory design.

Despite increasing interest in cellular-connected unmanned aerial vehicles (UAVs), their integration into existing cellular networks poses substantial challenges, including intense interference from UAVs to terrestrial user equipments (UEs) and numerous redundant handovers. To jointly reduce the generated interference and redundant handovers of cellular-connected UAVs while keeping their low transmission delay, we define an optimization problem with total available bandwidth and quality of service (QoS) constraints. Then, we formulate the optimization problem as a partially observable Markov decision process (POMDP) within a cooperative game. We have further developed a collaborative trajectory and handover management scheme using a Multi-Agent Deep Reinforcement Learning (MADRL) algorithm, specifically the Q-learning with a MIXer network (QMIX) algorithm, to optimize the aforementioned three metrics jointly. To demonstrate the superiority of our proposed scheme, we compare it with two benchmarks, namely the conventional handover management (CHM) scheme and the independent dueling double deep recurrent Q-network (ID3RQN) scheme. Simulation results show that QMIX outperforms the other schemes. Compared with the CHM scheme, QMIX reduces the delay, interference, and number of handovers for UAVs by an average of 46.9%, 70.0% and 90.5%, respectively. Compared with the ID3RQN scheme, QMIX reduces the three metrics by an average of 90.0%, 43.0% and 41.7%, respectively. 

Modern communication systems need to fulfill multiple and often conflicting objectives at the same time. In particular, new applications require high reliability while operating at low transmit powers. Moreover, reliability constraints may vary over time depending on the current state of the system. One solution to address this problem is to use joint transmissions from a number of base stations (BSs) to meet the reliability requirements. However, this approach is inefficient when considering the overall total transmit power. In this work, we propose a reinforcement learning-based power allocation scheme for an unmanned aerial vehicle (UAV) communication system with varying communication reliability requirements. In particular, the proposed scheme aims to minimize the total transmit power of all BSs while achieving an outage probability that is less than a tolerated threshold. This threshold varies over time, e.g., when the UAV enters a critical zone with high-reliability requirements. Our results show that the proposed learning scheme uses dynamic power allocation to meet varying reliability requirements, thus effectively conserving energy.

In modern cell-less wireless networks, mobility management is undergoing a significant transformation, transitioning from single-link handover management to a more adaptable multi-connectivity cluster reconfiguration approach, including often conflicting objectives like energy-efficient power allocation and satisfying varying reliability requirements. In this work, we address the challenge of dynamic clustering and power allocation for unmanned aerial vehicle (UAV) communication in wireless interference networks. Our objective encompasses meeting varying reliability demands, minimizing power consumption, and reducing the frequency of cluster reconfiguration. To achieve these objectives, we introduce a novel approach based on reinforcement learning using a masked soft actor-critic algorithm, specifically tailored for dynamic clustering and power allocation.

Multi-connectivity involves dynamic cluster formation among distributed access points (APs) and coordinated resource allocation from these APs, highlighting the need for efficient mobility management strategies for users with multi-connectivity. In this paper, we propose a novel mobility management scheme for unmanned aerial vehicles (UAVs) that uses dynamic cluster reconfiguration with energy-efficient power allocation in a wireless interference network. Our objective encompasses meeting stringent reliability demands, minimizing joint power consumption, and reducing the frequency of cluster reconfiguration. To achieve these objectives, we propose a hierarchical multi-agent deep reinforcement learning (H-MADRL) framework, specifically tailored for dynamic clustering and power allocation. The edge cloud connected with a set of APs through low latency optical back-haul links hosts the high-level agent responsible for the optimal clustering policy, while low-level agents reside in the APs and are responsible for the power allocation policy.  To further improve the learning efficiency, we propose a novel action-observation transition-driven learning algorithm that allows the low-level agents to use the action space from the high-level agent as part of the local observation space.This allows the lower-level agents to share partial information about the clustering policy and allocate the power more efficiently.The simulation results demonstrate that our proposed distributed algorithm achieves comparable performance to the centralized algorithm. Additionally, it offers better scalability, as the decision time for clustering and power allocation increases by only 10% when doubling the number of APs, compared to a 90% increase observed with the centralized approach.

To enable pervasive applications of cellular-connected unmanned aerial vehicles (UAVs), localization plays a key role. The successful reception of localization signals from multiple base stations (BSs) is the first step to localize targets, which is called cellular localizability. In this paper, we propose an analytical framework to characterize the B-localizability of UAVs, which is defined as the probability of successfully receiving localization signals above a certain signal-to-interference plus noise ratio (SINR) level from at least B ground BSs. Our framework considers UAV-related system parameters in a three-dimensional environment and provides a comprehensive insight into factors affecting localizability such as distance distributions, path loss, interference, and received SINR. We perform simulation studies to explore the relationship between localizability and the number of participating BSs, SINR requirements of the received localization signals, air-to-ground channel characteristics, and network coordination. We also formulate an optimization problem to maximize localizability and investigate the effects of UAV altitude in different scenarios. Our study reveals that in an urban macro environment, the effectiveness of cellular network-based localization increases with altitude, with localizability reaching 100% above 60 meters. This finding indicates that utilizing cellular networks for UAV localization is a viable option.