In cell-free massive MIMO networks, scalability is one of the fundamental problems since a significant number of access points (APs) are widely distributed throughout the network area to cater to the needs of multiple user equipments (UEs). One approach to addressing this issue is through network-centric clustering, which involves dividing the network area into isolated clusters of APs, each connected to its cloud unit (CU). To address these challenges, this paper proposes a deep reinforcement learning (DRL) algorithm that jointly optimizes the network-centric cluster boundaries and decides AP deployment in each cluster to improve long-term energy efficiency. The DRL agent also aims to minimize the average UE drop rate by considering the delay requirements of each UE's requested service. The results show that at least 16% improvement in energy efficiency is obtained compared to the heuristically developed benchmarks.

Aiming at the implementation of energy-efficient cell-free networks, several approaches have been proposed in the literature, which consider different access point (AP) switch ON/OFF (ASO) strategies for power minimization. Different from prior works, this paper focuses on additional factors that have an adverse effect not only on total power consumption but also on implementation complexity and operation cost. For instance, too frequent ON/OFF switching in an AP can lead to tapering off the potential power saving of ASO by incurring extra power consumption due to excessive switching. Indeed, frequent switching of APs might also result in thermal fatigue and severe lifetime degeneration. Moreover, time variations in the AP-UE (user equipment) clusters in favor of energy saving in a dynamic network bring additional signaling and implementation complexity. Thus, we propose a multi-objective optimization problem that aims to minimize the total power consumption together with AP switching and AP- UE clustering variations in comparison to the previous state of the network. The proposed problem is cast in mixed integer quadratic programming form and solved optimally. Our simulation results show that by limiting AP switching (node switching) and AP- UE cluster reformation switching (link switching), the total power consumption at the radio site only slightly increases, but the number of average switching drops significantly regardless of node or link switching. It achieves a good balance on the trade-off between radio power consumption and the side effects excessive switching will bring.