Frictional sleepers have been shown to enhance lateral resistance effectively. However, the current design of the bottom grooves in frictional sleepers lacks standardized specifications, and the fundamental mechanisms responsible for enhancing lateral resistance remain unclear. To explore the underlying mechanisms behind the influence of different bottom groove patterns on lateral resistance, five distinct groove shapes for frictional sleepers were designed, and a series of hybrid discrete element method-finite difference method (DEM-FDM) numerical simulations were conducted by using the single sleeper pull-out test (SSPT) as the basis. The results show that in comparison with the standard sleepers, there exists a considerable variation in the degree of increase in lateral resistance among different types of frictional sleepers, ranging from 35.1% to 80.2%. Specifically, the lateral resistance increases logarithmically as the groove side area increases. The analyses show that the effective contact area (i.e., the contact area between ballast particles and the sleeper's bottom surface) was increased due to the constraint effect of the bottom grooves, simultaneously leading to increasing strength of the horizontal force chains near the grooves. Using discontinuous grooves can diminish the initial interference with the ballasts near the sleepers. Frictional sleepers primarily demonstrate anisotropy in the horizontal plane, which constitutes one of the mesoscale factors contributing to the enhancement of lateral resistance. In addition, the displacement and rotation of particles around the sleepers are constrained, whereas the most significant constraint was observed on the ballast shoulders. It is suggested to apply the frictional sleepers with discontinuous grooves and consider patterns that aim to maximize the groove side area in the design and maintenance of ballasted railway tracks.