Cloud manufacturing is a service-oriented networked manufacturing model that embraces the concept of 'Everything-as-a-Service'. In cloud manufacturing, distributed manufacturing resources encompassed in the product lifecycle are transformed into manufacturing services. Industrial robots are an important category of manufacturing resources in cloud manufacturing. During the past years, robots have been demonstrated to be able to learn various dexterous manipulation skills through training with deep reinforcement learning (DRL). In cloud manufacturing, there are many complex industrial application scenarios that require dexterous robots. Hence, robot training, which enables robots to learn various manipulation skills, becomes an important requirement for cloud manufacturing in the future, leading to the concept of 'Robot Training-as-a-Service'. This paper focuses on industrial robot training in the context of cloud manufacturing. First, related work on cloud manufacturing, DRL, DRL-based robot training, and cloud-edge collaboration is briefly reviewed and analyzed. Then, a framework for industrial robot training in cloud manufacturing with DRL is proposed, and a simplified case study is presented to demonstrate the basic principle of the framework. Finally, possible future research issues are discussed.