The management of assembly tasks within manufacturing, which traditionally relies on using stopwatches and video review, is both labour-intensive and prone to errors. This paper explores an approach utilizing machine learning (ML) and human pose estimation technologies to automate and enhance the classification and management of time blocks for manual assembly tasks in manufacturing environments. We developed and tested ML models capable of classifying manual assembly actions by converting video clips into a time series coordinate dataset via a human pose estimation library. The research highlights the potential of these technologies to significantly reduce the reliance on manual methods by providing a more adaptable, efficient, and scalable system for time data management. Our findings demonstrate accuracy variances across different actions, underscoring the challenges and potential of integrating ML in real-world manufacturing settings. This study provides a promising direction towards revolutionizing traditional practices and enhancing operational efficiencies in manufacturing.