The martensite start temperature is a critical parameter for steels with metastable austenite. Although numerous models have been developed to predict the martensite start (M-s) temperature, the complexity of the martensitic transformation greatly limits their performance and extensibility. In this work, we apply deep data mining of thermodynamic calculations and deep learning to develop a generic model for M-s prediction. Deep data mining was used to establish a hierarchical database with three levels of information. Then, a convolutional neural network model, which can accurately treat the hierarchical data structure, was used to obtain the final model. By integrating thermodynamic calculations, traditional machine learning and deep learning modeling, the final predictor model shows excellent generalizability and extensibility, i.e. model performance both within and beyond the composition range of the original database. The effects of 15 alloying elements were considered successfully using the proposed methodology. The work suggests that, with the help of deep data mining considering the physical mechanisms, deep learning methods can partially mitigate the challenge with limited data in materials science and provide a means for solving complex problems with small databases.