Adaptive real-time similar repetitive manual procedure prediction and robotic procedure generation for human-robot collaborationShow others and affiliations
2023 (English)In: Advanced Engineering Informatics, ISSN 1474-0346, E-ISSN 1873-5320, Vol. 58, article id 102129Article in journal (Refereed) Published
Abstract [en]
Manual procedure recognition and prediction are essential for practical human-robot collaboration in industrial tasks, such as collaborative assembly. However, current research mostly focuses on diverse human motions, while the similar repetitive manual procedures that are prevalent in real production tasks are often overlooked. Furthermore, the dynamic uncertainty caused by human-robot interferences and the generalisation of individuals, scenarios, and multiple sensor deployments pose challenges for implementing manual procedure prediction and robotic procedure generation. To address these issues, this paper proposes a real-time, similar repetitive procedure-oriented human skeleton processing system that employs the human skeleton as a robust modality. It utilises an improved deep spatial-temporal graph convolutional network and a FIFO queue-based discriminator for real-time data processing, procedure prediction, and generation. The proposed method is validated on multiple datasets with tens of individuals engaged in a real dynamic and uncertain human-robot collaborative assembly cell and able to run on entry-level hardware. The results demonstrate competitive performance of handcraft feature-free, early prediction and generalisation on individual variance, environment background, camera position, lighting conditions, and stochastic interference in human-robot collaboration.
Place, publisher, year, edition, pages
Elsevier BV , 2023. Vol. 58, article id 102129
Keywords [en]
Collaborative assembly, Human-robot collaboration, Manual procedure prediction, Robotic procedure generation, Similar repetitive manual procedure
National Category
Computer graphics and computer vision Robotics and automation
Identifiers
URN: urn:nbn:se:kth:diva-336296DOI: 10.1016/j.aei.2023.102129ISI: 001106726400001Scopus ID: 2-s2.0-85169785935OAI: oai:DiVA.org:kth-336296DiVA, id: diva2:1796649
Note
QC 20230913
2023-09-132023-09-132025-02-05Bibliographically approved