This paper studies the global energy efficiency (GEE) optimization problem in a typical heterogeneous network (HetNet), where a macro base station (BS) and multiple micro BSs share the same spectrum band. Conventional optimization methods typically involve collecting global instantaneous channel state information (CSI) and utilizing centralized optimization algorithms to obtain the optimal transmit power and enhance the GEE. However, it is expensive to obtain the global instantaneous CSI in practical scenarios, and the optimization algorithms tend to be time-consuming. In this paper, we develop a lightweight cloud-edge collaboration framework based on the deep reinforcement learning (DRL) technique, such that the BSs in the edge do not need to exchange local instantaneous information with each other, and the core network in the cloud can collect only historical data rates and energy consumption information from each edge BS and feeds the calculated global reward back to them. Within the frame-work, we establish an independent actor-critic structure for each BS, and design a multi-agent independent actor-critic (MAIAC) power control algorithm, which enables each BS to determine its transmit power locally and enhance the GEE based on only local information. Simulation results indicate that the proposed MAIAC algorithm can achieve comparable GEE performance with conventional algorithms while requiring significantly less time complexity.