In the milling process, chatter is easy to occur and has a very adverse impact on the quality of the workpiece and the production efficiency. A chatter feature extraction method based on optimized variational mode decomposition (OVMD) and multi-scale permutation entropy (MPE) was proposed to solve the problem that it is difficult to detect the machining chatter state during milling. The methodology presented in this article allows the occurrence of machining chatter to be effectively identified through real-time digital signal processing and analysis. First, in order to solve the problem of variational mode decomposition (VMD) parameter selection, an automatic selection method based on particle swarm optimization (PSO) and the maximum crest factor of the envelope spectrum (CE) was proposed. Then, the decomposed signal was reconstructed based on the energy ratio. In order to solve the problem that the single-scale permutation entropy (PE) cannot detect milling chatter well, the MPE was introduced to detect milling chatter. Finally, experimental verification was carried out, and the MPE of the reconstructed signals at different scales was extracted and analyzed. The results show that using the OVMD algorithm to process the signals can significantly improve the discrimination of MPE. With the increase of the scale factor, the MPE of the milling signals tends to decrease. At the same time, MPE is better than single-scale PE in chatter detection, and the MPE at scale factor of 4 is more conducive to chatter detection.