In order to address the problems with current robotic automated assembly such as limitations of model-based methods in unstructured assembly scenarios, low training efficiency of learning-based methods, and limited performance of policy generalization, this paper proposes two modeling methodologies based on deep reinforcement learning under the overall framework of phased assembly for complex robotic assembly tasks, i.e., separated-phased policy modeling (SPM) and integrated policy modeling (IPM). Regarding policy learning with deep reinforcement learning, we present a refined SAC algorithm that merges a policy-lead mechanism and an LSTM network (i.e., PL-LSTM-SAC). A comprehensive testbed based on the assembly of a triple-task planetary gear train is designed to validate the framework and the proposed approach. Experimental results indicate that the trained assembly policies for each task are effective under both policy modeling methodologies, but SPM has higher stability and policy convergence efficiency than IPM. Physical tests indicate the sim-to-real transferability of the trained policies with both SPM and IPM and an average success rate of 92.0 % is achieved. The PL-LSTM-SAC algorithm proposed can significantly accelerate training speed and enhance compliance and overall performance of assembly actions by a 13.9 % reduction in the average contact force during assembly processes.