We address the challenge of estimating rare events associated with stochastic differential equations using importance sampling. The importance sampling zero variance measure in these settings can be inferred from a solution to the Hamilton-Jacobi-Bellman partial differential equation (HJB-PDE) associated with a value function for the underlying process. Guided by this equation, we use a neural network to learn the zero variance change of measure. To improve performance of our estimation, we pursue two new ideas. First, we adopt a loss function that combines three objectives which collectively contribute to improving the performance of our estimator. Second, we embed our rare event problem into a sequence of problems with increasing rarity. We find that a well chosen schedule of rarity increase substantially speeds up rare event simulation. Our approach is illustrated on Brownian motion, Orstein-Uhlenbeck (OU) process, Cox–Ingersoll–Ross (CIR) process as well as Langevin double-well diffusion.