In recent years, the installation of renewable energy generation systems based on photovoltaic (PV) panels has experienced massive increments and PV parks with thousands of panels are now becoming commonplace. Yet, there are some challenges, like inspection and fault detection. Lately, these operations have been approached using drones. This project adds the use of deep learning, more specifically proposes the convolutional neural network (CNN) algorithm, the YOLOv5 model and Real-Time Messaging Protocol (RTMP) protocol to achieve real-time detection of PV panels failures. The YOLOv5 model was trained by sets sorted into 9 different categories including fault and abnormal objects' coverage. This multi-class classification system was investigated by a variety of evaluation indexes to show effectiveness and accuracy. The system was also examined with its different fault classes. The performance results demonstrate that the mean average precision could reach up to 98% with a good training set, confirming the feasibility of proposed approaches.