Railway timetables have an important role in efficient and punctual railway operations. In particular, the robustness of the timetable has a direct impact on the traffic's punctuality. To evaluate the robustness of a timetable, simulation is commonly used. A simulation study may indicate that some trains are too sensitive against minor delays, which may lead to that they fall out of their planned channel of operations (defined by their surrounding trains). We define this as robustness vulnerabilities of the timetable. The work explores reinforcement learning (RL) as a method to resolve timetable robustness vulnerabilities. We formulate a RL-based model for the robust railway timetabling problem and will explore different RL algorithms and compare with timetables generated using optimization-based methods from our previous work [1, 2]. The models are evaluated using microscopic RailSys simulation for the traffic in the westbound direction of the Swedish Western Main Line. The results are expected to provide better support for robust railway timetabling in practice.