In smart grid (SG), smart meter (SM) is the key component, which collects real-time grid load data to support intelligent applications, such as load prediction, failure detection, and dynamic billing. Despite benefits, the SM readings of grid loads also potentially leak personal information, resulting in smart meter privacy problem. Rechargeable battery (RB) deployed at the user side can be utilized to shape grid loads, thereby enhancing privacy preservation and cost efficiency. The energy management design is formulated as a sequential decision optimization problem to tradeoff the privacy risk, which is measured by the Kullback-Leibler (KL) divergence between grid loads and target loads, and the energy cost, which depends on a time-of-use electricity energy price. A novel adversarial deep reinforcement learning (ADRL) is proposed to efficiently design the privacy-preserving and cost-efficient energy management policy. The effectiveness of the ADRL-aided energy management policy design is verified through experiments and the superiority of the proposed approach is shown by comparing with state-of-the-art privacy-preserving load shaping method.