kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Integrating Large Language Model for Natural Language-Based Instruction toward Robust Human-Robot Collaboration
Department of Mechanical and Aerospace Engineering, Case Western Reserve University, Cleveland, OH, 44106, USA.
Department of Mechanical and Aerospace Engineering, Case Western Reserve University, Cleveland, OH, 44106, USA.
Department of Mechanical and Aerospace Engineering, Case Western Reserve University, Cleveland, OH, 44106, USA.
KTH, School of Industrial Engineering and Management (ITM), Production engineering, Industrial Production Systems.ORCID iD: 0000-0002-1909-0507
Show others and affiliations
2024 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Human-Robot Collaboration (HRC) aims to create environments where robots can understand workspace dynamics and actively assist humans in operations, with the human intention recognition being fundamental to efficient and safe task fulfillment. Language-based control and communication is a natural and convenient way to convey human intentions. However, traditional language models require instructions to be articulated following a rigid, predefined syntax, which can be unnatural, inefficient, and prone to errors. This paper investigates the reasoning abilities that emerged from the recent advancement of Large Language Models (LLMs) to overcome these limitations, allowing for human instructions to be used to enhance human-robot communication. For this purpose, a generic GPT 3.5 model has been fine-tuned to interpret and translate varied human instructions into essential attributes, such as task relevancy and tools and/or parts required for the task. These attributes are then fused with perceived on-going robot action to generate a sequence of relevant actions. The developed technique is evaluated in a case study where robots initially misinterpreted human actions and picked up wrong tools and parts for assembly. It is shown that the fine-tuned LLM can effectively identify corrective actions across a diverse range of instructional human inputs, thereby enhancing the robustness of human-robot collaborative assembly for smart manufacturing.

Place, publisher, year, edition, pages
Elsevier BV , 2024. p. 313-318
Keywords [en]
Error correction, Human-robot collaboration, Large language model, Natural language processing
National Category
Production Engineering, Human Work Science and Ergonomics Robotics and automation
Identifiers
URN: urn:nbn:se:kth:diva-358215DOI: 10.1016/j.procir.2024.10.093Scopus ID: 2-s2.0-85214970201OAI: oai:DiVA.org:kth-358215DiVA, id: diva2:1924849
Conference
18th IFAC Workshop on Time Delay Systems, TDS 2024, Udine, Italy, October 2-5, 2023
Note

QC 20250114

Available from: 2025-01-07 Created: 2025-01-07 Last updated: 2025-02-05Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Liu, SichaoWang, Lihui

Search in DiVA

By author/editor
Liu, SichaoWang, Lihui
By organisation
Industrial Production Systems
Production Engineering, Human Work Science and ErgonomicsRobotics and automation

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 43 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf