The rapid advancement of intelligent devices and smart meters has positioned smart buildings as key components in modern living. To ensure users’ comfort, regulate the indoor temperature, and minimize the overall cost of a building, an effective energy management system (EMS) is necessary. This paper proposes a deep deterministic policy gradient (DDPG)based energy management system (EMS) for smart buildings, incorporating photovoltaic (PV) panels, energy storage systems (ESS), heat pump (HP), and EV chargers. Unlike traditional methods that linearize temperature variations, our approach leverages DDPG to handle nonlinear models. The proposed EMS automatically addresses uncertainties in temperature and solar irradiation and provides a real-time scheduling commands without the demand for predicted data. Simulation results demonstrate the effectiveness of the DDPG-based EMS in minimizing costs while ensuring user comfort, outperforming traditional methods in both cost savings and temperature control precision.