kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Adaptive real-time similar repetitive manual procedure prediction and robotic procedure generation for human-robot collaboration
KTH, School of Industrial Engineering and Management (ITM), Production engineering. School of Information Engineering, Wuhan University of Technology, Wuhan 430070, China; Hubei Key Laboratory of Broadband Wireless Communication and Sensor Networks, Wuhan University of Technology, Wuhan 430070, China.ORCID iD: 0000-0002-0222-912x
School of Information Engineering, Wuhan University of Technology, Wuhan 430070, China; Hubei Key Laboratory of Broadband Wireless Communication and Sensor Networks (Wuhan University of Technology), Wuhan 430070, China.
School of Information Engineering, Wuhan University of Technology, Wuhan 430070, China; Hubei Key Laboratory of Broadband Wireless Communication and Sensor Networks (Wuhan University of Technology), Wuhan 430070, China.
KTH, School of Industrial Engineering and Management (ITM), Production engineering.ORCID iD: 0000-0001-8679-8049
Show others and affiliations
2023 (English)In: Advanced Engineering Informatics, ISSN 1474-0346, E-ISSN 1873-5320, Vol. 58, article id 102129Article in journal (Refereed) Published
Abstract [en]

Manual procedure recognition and prediction are essential for practical human-robot collaboration in industrial tasks, such as collaborative assembly. However, current research mostly focuses on diverse human motions, while the similar repetitive manual procedures that are prevalent in real production tasks are often overlooked. Furthermore, the dynamic uncertainty caused by human-robot interferences and the generalisation of individuals, scenarios, and multiple sensor deployments pose challenges for implementing manual procedure prediction and robotic procedure generation. To address these issues, this paper proposes a real-time, similar repetitive procedure-oriented human skeleton processing system that employs the human skeleton as a robust modality. It utilises an improved deep spatial-temporal graph convolutional network and a FIFO queue-based discriminator for real-time data processing, procedure prediction, and generation. The proposed method is validated on multiple datasets with tens of individuals engaged in a real dynamic and uncertain human-robot collaborative assembly cell and able to run on entry-level hardware. The results demonstrate competitive performance of handcraft feature-free, early prediction and generalisation on individual variance, environment background, camera position, lighting conditions, and stochastic interference in human-robot collaboration.

Place, publisher, year, edition, pages
Elsevier BV , 2023. Vol. 58, article id 102129
Keywords [en]
Collaborative assembly, Human-robot collaboration, Manual procedure prediction, Robotic procedure generation, Similar repetitive manual procedure
National Category
Computer graphics and computer vision Robotics and automation
Identifiers
URN: urn:nbn:se:kth:diva-336296DOI: 10.1016/j.aei.2023.102129ISI: 001106726400001Scopus ID: 2-s2.0-85169785935OAI: oai:DiVA.org:kth-336296DiVA, id: diva2:1796649
Note

QC 20230913

Available from: 2023-09-13 Created: 2023-09-13 Last updated: 2025-02-05Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Liu, ZhihaoWang, Lihui

Search in DiVA

By author/editor
Liu, ZhihaoWang, Lihui
By organisation
Production engineering
In the same journal
Advanced Engineering Informatics
Computer graphics and computer visionRobotics and automation

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 27 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf