The manufacturing landscape has witnessed a paradigm shift towards multi-variety and small-batch production for the customized and personalized product (CPP). But this paradigm poses significant challenges for the cloud manufacturing system: 1) wired production machines cannot support the ultra-flexible resource allocation for the CPP job; 2) the scheduling model largely neglects the reconfiguration time of machines; 3) the intelligent scheduling method is difficult to learn the policy in the high-dimensional CPP solution space. To address these issues, we propose an edge-computing and wireless-connection based CPP manufacturing system framework which allows for the dynamic and ultra-flexible allocation of operations and resources. Then reconfiguration time is modelled in the optimization problem and a knowledge-guided deep reinforcement learning algorithm is proposed to effectively explore optimal CPP scheduling policy in the high dimensional solution space. The experimental results demonstrated that the proposed algorithm obtained better scheduling results than traditional scheduling rules, effectively balancing processing time and reconfiguration time, thereby minimizing the overall jobshop makespan.