kth.sePublications KTH
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Addressing Failures in Robotics Using Vision-Based Language Models (VLMs) and Behavior Trees (BT)
Lund Univ, Lund, Sweden.
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.ORCID iD: 0000-0003-0312-8811
Lund Univ, Lund, Sweden.
2025 (English)In: EUROPEAN ROBOTICS FORUM 2025 / [ed] Huber, M Verl, A Kraus, W, Springer Nature , 2025, Vol. 36, p. 281-287Conference paper, Published paper (Refereed)
Abstract [en]

In this paper, we propose an approach that combines Vision Language Models (VLMs) and Behavior Trees (BTs) to address failures in robotics. Current robotic systems can handle known failures with preexisting recovery strategies, but they are often ill-equipped to manage unknown failures or anomalies. We introduce VLMs as a monitoring tool to detect and identify failures during task execution. Additionally, VLMs generate missing conditions or skill templates that are then incorporated into the BT, ensuring the system can autonomously address similar failures in future tasks. We validate our approach through simulations in several failure scenarios.

Place, publisher, year, edition, pages
Springer Nature , 2025. Vol. 36, p. 281-287
Series
Springer Proceedings in Advanced Robotics, ISSN 2511-1256
Keywords [en]
Robotics, Failure Detection, Behavior Trees, Vision Language Models, Recovery Behaviors
National Category
Robotics and automation
Identifiers
URN: urn:nbn:se:kth:diva-374068DOI: 10.1007/978-3-031-89471-8_43ISI: 001553155000043Scopus ID: 2-s2.0-105006603744ISBN: 978-3-031-89574-6 (print)ISBN: 978-3-031-89471-8 (print)ISBN: 978-3-031-89470-1 (print)OAI: oai:DiVA.org:kth-374068DiVA, id: diva2:2021146
Conference
16th European Robotics Forum-ERF-Annual, MAR 25-27, 2025, Stuttgart, GERMANY
Note

QC 20251212

Available from: 2025-12-12 Created: 2025-12-12 Last updated: 2025-12-12Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Styrud, Jonathan

Search in DiVA

By author/editor
Styrud, Jonathan
By organisation
Robotics, Perception and Learning, RPL
Robotics and automation

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 43 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf