Despite ongoing efforts to decarbonize society, climate change continues to result in more extreme events that can reduce the electrical grid’s ability to reliably supply power to end users. At the same time, across the distribution grid, Distributed Energy Resources (DERs), such as renewable generation, energy storage systems, and flexible resources, offer the possibility of operating the grid in novel ways. Distribution System Operator (DSOs) could employ these DERs to improve their ability to mitigate extreme events. This work therefore demonstrates how DERs, in the form of flexible loads, can be controlled by a Deep Reinforcement Learning (DRL) agent to minimize Energy Not Supplied (ENS) in the immediate aftermath of an extreme event. To obtain near-optimal performance on unseen scenarios, an enhanced Implicit Q Network (IQN+) architecture is proposed, trained, and evaluated on a modified CIGRE MV benchmark grid. The resulting IQN+ agent can outperform a passive baseline policy, a trained Rainbow DQN policy, and a single-timestep Optimal Power Flow (OPF) based policy on the test set. Sensitivity analysis reveals that the location and quantity of available DERs also impact the efficacy of the IQN+ agent, with the agent preferentially using actions on loads that result in greater average episode durations. These results highlight the potential for RL to rapidly provide decision support to operators by suggesting potential remedial actions to mitigate the impact of extreme events.