In modern manufacturing, vision-based defect recognition is an essential technology to guarantee product quality, and it plays an important role in industrial intelligence. With the developments of industrial big data, defect images can be captured by ubiquitous sensors. And, how to realize accuracy recognition has become a research hotspot. In the past several years, many vision-based defect recognition methods have been proposed, and some newly-emerged techniques, such as deep learning, have become increasingly popular and have addressed many challenging problems effectively. Hence, a comprehensive review is urgently needed, and it can promote the development and bring some insights in this area. This paper surveys the recent advances in vision-based defect recognition and presents a systematical review from a feature perspective. This review divides the recent methods into designed-feature based methods and learned-feature based methods, and summarizes the advantages, disadvantages and application scenarios. Furthermore, this paper also summarizes the performance metrics for vision-based defect recognition methods. And some challenges and development trends are also discussed.