Understanding the influence of surface roughness on drag forces remains a significant challenge in fluid dynamics. This paper presents a convolutional neural network (CNN) that predicts drag solely by the topography of rough surfaces and is capable of discovering spatial patterns linked to drag-inducing structures. A CNN model was developed to analyze spatial information from the topography of a rough surface and predict the roughness function, Δ U + , obtained from direct numerical simulation. This model enables the prediction of drag from rough surface data alone, which was not possible with previous methods owing to the large number of surface-derived parameters. Additionally, the retention of spatial information by the model enables the creation of a feature map that accentuates critical areas for drag prediction on rough surfaces. By interpreting the feature maps, we show that the developed CNN model is able to discover spatial patterns associated with drag distributions across rough surfaces, even without a direct training on drag distribution data. The analysis of the feature map indicates that, even without flow field information, the CNN model extracts the importance of the flow-directional slope and height of roughness elements as key factors in inducing pressure drag. This study demonstrates that CNN-based drag prediction is grounded in physical principles of fluid dynamics, underscoring the utility of CNNs in both predicting and understanding drag on rough surfaces.