The proliferation of electric vehicles (EVs) has resulted in new charging infrastructure at all levels, including domestically. These new domestic EVs can potentially provide vehicle to home (V2H) services where EVs are used as energy storage systems (ESSs) for the home when they are not in use. Energy management systems (EMSs) can control these EVs to minimize the electricity cost to the owner but must satisfy constraints. Uncertainty in EV availability and the microgrid environment is also a challenge and can be addressed through real-time operation. Hence this paper formulates the EV charge/discharge scheduling problem as a Markov Decision Process (MDP). A safe implementation of Proximal Policy Optimization (PPO) is proposed for real-time optimization and compared to a day-ahead Mixed Integer Linear Programming (MILP) benchmark. The resulting PPO agent is able to minimize RA and SD costs for a typical EV user 3% better than the MILP solution. It obtains a 39% higher electricity cost than MILP, but unlike MILP does not require accurate forecasting data and operates in real-time.
QC 20230227