Reliable cellular connectivity is critical during disasters, yet base stations must often operate on limited battery backup during large-scale outages. This paper develops a disaster recovery automation (DRA) module for open radio access network (O-RAN) systems that uses reinforcement learning (RL) to manage multi-layer radio access technologies autonomously. The RAT-layer control problem is formulated as a Markov decision process whose state captures battery level, service mode, neighbor signal quality, and site context, while actions activate or deactivate selected RAT layers under emergency conditions. The module is evaluated in an earthquake-calibrated simulator built from real key performance indicator (KPI) traces by comparing Monte Carlo, Q-learning, state-action-reward-state-action (SARSA), Expected SARSA, and a neural-network-based machine learning (ML) agent against All-On and All-Off baselines. Results show that Expected SARSA achieves the longest operational lifetime, 331.5min, extending the All-On baseline of 201.0min by 65%. The findings indicate that adaptive RL-based RAT management can preserve essential service availability while extending battery lifetime, outperforming both no-control operation and naive shutdown without relying on hand-crafted rules.