In green massive MIMO networks, reducing power consumption (PC) while ensuring user quality of service (QoS) is critical for sustainable operation. To this end, we propose a robust and scalable reinforcement learning framework based on independent proximal policy optimization (IPPO), enabling intelligent base station (BS) control through a three-dimensional configuration of antenna activation, sleep mode transitions, and user offloading. Compared to a non-learning simple energy-saving policy, our proposed IPPO algorithm achieves approximately a 20.3% reduction in PC and a 49% improvement in energy efficiency (EE). In addition, it demonstrates significantly faster convergence and better scalability than multi-agent PPO (MAPPO), reducing convergence time by approximately 75% with 49 BSs and by around 90% with 81 BSs.

Despite increasing interest in cellular-connected unmanned aerial vehicles (UAVs), their integration into existing cellular networks poses substantial challenges, including intense interference from UAVs to terrestrial user equipments (UEs) and numerous redundant handovers. To jointly reduce the generated interference and redundant handovers of cellular-connected UAVs while keeping their low transmission delay, we define an optimization problem subject to constraints on total available bandwidth and quality of service (QoS). Then, we formulate the optimization problem as a decentralized partially observable Markov decision process (Dec-POMDP) in the context of a cooperative game. We further develope a collaborative trajectory and handover management scheme using a multi-agent deep reinforcement learning algorithm, specifically the Q-learning with a MIXer network (QMIX) algorithm, to jointly optimize the aforementioned three metrics. Simulation results demonstrate that QMIX significantly outperforms two benchmark schemes: the conventional handover management (CHM) scheme and the independent dueling double deep recurrent Q-network (ID3RQN) scheme. Compared with the CHM scheme, QMIX reduces the delay, interference, and number of handovers for UAVs by an average of 46.9%, 70.0% and 90.5%, respectively. Compared with the ID3RQN scheme, QMIX reduces the three metrics by an average of 90.0%, 43.0% and 41.7%, respectively.

The evolution toward 6G networks introduces unprecedented challenges and opportunities, particularly in the realm of serving both aerial and ground users seamlessly. In this article, we propose a holistic adaptive combined airspace and non-terrestrial network (NTN) architecture designed to address the unique requirements of the 6G era. Three principle features - joint sensing, communication, and computation (JSCC) in three dimensions (3D), cloud-native and artificial intelligence (AI) native, and the flexibility of radio access network (RAN) and core functions of the proposed architecture - are presented. Next, two application scenarios are analyzed: one catering to aerial users and the other supporting ground users, each, in particular, supporting communication links. Finally, we look into the network management and control aspects of the proposed architecture and discuss challenges and future research directions.

Given the advancements in next-generation low Earth orbit (LEO) satellites, there is an expected shift from transparent architectures (acting as radio repeaters) to regenerative architectures (hosting a part or all of the gNodeB (gNB) onboard). Such regenerative architectures enable disaggregation and distribution of radio access network (RAN) functions between the ground and space. Open RAN is a promising approach for non-terrestrial networks and offers flexible function placement through open interfaces. The present study examines three open RAN-based regenerative architectures, namely, Split 7.2x (low-layer physical functions onboard), Split 2 (Layers 1 and 2 onboard), and a gNB onboard the satellite. Handover (HO) management becomes increasingly complex in this disaggregated RAN, particularly for LEO satellites, where the part of the gNB is constantly in motion. The choice of regenerative architecture and its dynamic topology influence the additional HO control signals required between the satellite and ground stations. Using a realistic dynamic LEO constellation model, we analyze the interplay among conditional handover (CHO) delay, computational complexity, and control signaling overhead under different network architectures. Our findings reveal that transitioning from a transparent architecture to Split 7.2x does not reduce CHO delay despite the introduction of additional onboard processing. The gNB onboard the satellite minimizes cumulative CHO delay but demands 55%-70% more computational resources than the Split 7.2x architecture. Conversely, although Split 7.2x is computationally more efficient, it increases the cumulative CHO delay by 25%-30%. Additionally, we observed that under limited onboard processing conditions, only the transparent and Split 7.2x architectures supported delay-sensitive services up to 100 ms. In contrast, under ample processing conditions, gNB was suitable for stringent 50 ms requirements, while Split 2 best supported delay-tolerant services with 200 ms requirements.

This paper addresses the challenge of optimizing handover (HO) performance in non-terrestrial networks (NTNs) to enhance user equipment (UE) effective service time, defined as the active service time excluding HO delays and radio link failure (RLF) periods. Availability is defined as the normalized effective service time which is effected by different HO scenarios: Intra-satellite HO is the HO from one beam to another within the same satellite; inter-satellite HO refers to the HO from one satellite to another where satellites can be connected to the same or different GSs. We investigate the impact of open radio access network (O-RAN) functional splits (FSs) between ground station (GS) and LEO satellites on HO delay and assess how beam configurations affect RLF rates and intra- and inter-satellite HO rates. This work focuses on three O-RAN FSs - split 7.2x (low layer 1 functions on the satellite), split 2 (layer 1 and layer 2 functions on the satellite), and gNB onboard the satellite - and two beam configurations (19-beam and 127-beam). In a realistic dynamic LEO satellite constellation where different types of HO scenarios are simulated, we maximize effective service time by tuning the time-to-trigger (TTT) and HO margin (HOM) parameters. Our findings reveal that the gNB onboard the satellite achieves the highest availability, approximately 95.4%, while the split 7.2x exhibits the lowest availability, around 92.8% due to higher intra-satellite HO delays.

Despite increasing interest in cellular-connected unmanned aerial vehicles (UAVs), their integration into existing cellular networks poses substantial challenges, including intense interference from UAVs to terrestrial user equipments (UEs) and numerous redundant handovers. To jointly reduce the generated interference and redundant handovers of cellular-connected UAVs while keeping their low transmission delay, we define an optimization problem with total available bandwidth and quality of service (QoS) constraints. Then, we formulate the optimization problem as a partially observable Markov decision process (POMDP) within a cooperative game. We have further developed a collaborative trajectory and handover management scheme using a Multi-Agent Deep Reinforcement Learning (MADRL) algorithm, specifically the Q-learning with a MIXer network (QMIX) algorithm, to optimize the aforementioned three metrics jointly. To demonstrate the superiority of our proposed scheme, we compare it with two benchmarks, namely the conventional handover management (CHM) scheme and the independent dueling double deep recurrent Q-network (ID3RQN) scheme. Simulation results show that QMIX outperforms the other schemes. Compared with the CHM scheme, QMIX reduces the delay, interference, and number of handovers for UAVs by an average of 46.9%, 70.0% and 90.5%, respectively. Compared with the ID3RQN scheme, QMIX reduces the three metrics by an average of 90.0%, 43.0% and 41.7%, respectively. 

In this paper, we focus on minimizing the total energy consumption of multi-cell massive multiple-input multiple-output (MIMO) networks while simultaneously guaranteeing user quality of service (QoS). This is achieved by optimizing the multi-level advanced sleep modes (ASM), antenna switching, and user association of the base stations (BSs). Due to the interdependence of user association and inter-cell interference in the network, collaborative efforts among individual BSs become imperative. The problem is modeled as a decentralized partially observable Markov decision process (DEC-POMDP) and a multi-agent proximal policy optimization (MAPPO) algorithm is proposed to obtain a collaborative BS control policy. Simulation results demonstrate that the obtained policy can significantly improve network energy efficiency, adaptively switch the BSs into different depths of sleep, reduce inter-cell interference, and maintain good QoS compared to the two benchmark algorithms. The results also validate that enabling user offloading among BSs can improve both user QoS and system performance. The superiority of MAPPO is further affirmed by comparing it with the single-agent deep Q network (DQN) algorithm.

In this paper, we study the task-offloading problem for Internet of Remote Things (IoRT) devices. In the considered scenario, the Low Earth Orbit (LEO) satellite nearest to the devices collects their generated tasks and makes offloading decisions for each task, including selecting the offloading destination and allocating computational resources. Moreover, we consider the motion of satellites within a constellation and assume that a random fraction of computing resources on each edge server is occupied during each time slot. Our objective is to minimize the total longterm latency for all IoRT devices. To address this problem, we propose a Proximal Policy Optimization (PPO)-based algorithm to learn the near-optimal policy. The simulation results demonstrate that our proposed algorithm reduces latency by an average of 21.49% and 50.79% compared to the two benchmark algorithms, perspectively.

This paper explores the evolution of Radio Access Network (RAN) architectures and their integration into Non-Terrestrial Networks (NTN) to address escalating mobile traffic demands. Focusing on Low Earth Orbit (LEO) satellites as key components of NTN, we examine the feasibility of RAN function splits (FSs) in terms of fronthaul (FH) latency, elevation angle, and bandwidth (BW) across LEO satellites and ground stations (GS), alongside evaluating performance of Conditional Handover (CHO) procedures under diverse scenarios. By assessing performance metrics such as handover duration, disconnection time, and control traffic volume, we provide insights on several aspects such as stringent constraints for Low Layer Splits (LLSs), leading to longer delays during mobility procedures and increased control traffic across the feeder link in comparison with the case when gNodeB is onboard satellite. Despite challenges, LLSs demonstrate minimal onboard satellite computational requirements, promising reduced power consumption and payload weight. These findings underscore the architectural possibilities and challenges within the telecommunications industry, paving the way for future advancements in NTN RAN design and operation.

Space air ground integrated network (SAGIN), leveraging low earth orbit (LEO) satellites and Unmanned Aerial Vehicles (UAVs), is expected to play a key role in providing services to Internet of Remote Things (IoRT) in the sixth generation (6G) communications. Our considered SAGIN incorporates a cache node on the UAV to cope with the data rate fluctuation in the backhaul link (UAV to satellite), allowing temporary storage of collected data during low data rate periods. In this paper, we aim to minimize the completion time of data collection in SAGIN by optimizing the UAV trajectory, IoRT device association scheme, and data caching policy (whether to store data temporarily or not in the UAV). Since the formulated problem is challenging to solve by using traditional optimization methods due to the unknown number of decision variables and the changing environment, we propose a deep reinforcement learning (DRL)-based algorithm to efficiently solve it. Simulation results demonstrate that our proposed algorithm requires less time to complete data collection compared to both the circular trajectory scheme and the no-cache node scheme under various setups. Moreover, our proposed algorithm can adapt to uneven data distribution by approaching closer to the IoRT nodes with large data sizes, and it can also mitigate the influence of backhaul link fluctuations with the aid of the cache node.