kth.sePublications KTH
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Detecting the Intention of Object Handover in Human-Robot Collaborations: An EEG Study
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.ORCID iD: 0000-0003-2533-7868
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.ORCID iD: 0000-0003-1932-1595
Ericsson Res, Stockholm, Sweden..
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.ORCID iD: 0000-0001-7091-0104
Show others and affiliations
2023 (English)In: 2023 32ND IEEE INTERNATIONAL CONFERENCE ON ROBOT AND HUMAN INTERACTIVE COMMUNICATION, RO-MAN, Institute of Electrical and Electronics Engineers (IEEE) , 2023, p. 549-555Conference paper, Published paper (Refereed)
Abstract [en]

Human-robot collaboration (HRC) relies on smooth and safe interactions. In this paper, we focus on the human-to-robot handover scenario, where the robot acts as a taker. We investigate the feasibility of detecting the intention of a human-to-robot handover action through the analysis of electroencephalogram (EEG) signals. Our study confirms that temporal patterns in EEG signals provide information about motor planning and can be leveraged to predict the likelihood of an individual executing a motor task with an average accuracy of 94.7%. We also suggest the effectiveness of the time-frequency features of EEG signals in the final second prior to the movement for distinguishing between handover action and other actions. Furthermore, we classify human intentions for different tasks based on time-frequency representations of pre-movement EEG signals and achieve an average accuracy of 63.5% for contrasting every two tasks against each other. The result encourages the possibility of using EEG signals to detect human handover intention in HRC tasks.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE) , 2023. p. 549-555
Series
IEEE RO-MAN, ISSN 1944-9445
National Category
Robotics and automation
Identifiers
URN: urn:nbn:se:kth:diva-342040DOI: 10.1109/RO-MAN57019.2023.10309426ISI: 001108678600078Scopus ID: 2-s2.0-85186991854OAI: oai:DiVA.org:kth-342040DiVA, id: diva2:1826025
Conference
32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), AUG 28-31, 2023, Busan, SOUTH KOREA
Note

Part of proceedings ISBN 979-8-3503-3670-2

QC 20240110

Available from: 2024-01-10 Created: 2024-01-10 Last updated: 2026-04-16Bibliographically approved
In thesis
1. Informing Machines about Human Mental States via EEG Decoding
Open this publication in new window or tab >>Informing Machines about Human Mental States via EEG Decoding
2026 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Brain activity is a rich source of information about human mental states and can potentially inform interactive intelligent systems about human intentions and perception of the environment. Among non-invasive neuroimaging techniques, electroencephalography (EEG) is particularly suited for interactive applications due to its portability, relatively low cost, and real-time measurements. Advances in artificial intelligence (AI) have enabled decoding models to extract complex task-related patterns from EEG signals and reveal information related to intention and perception. However, inherent properties of EEG signals, such as low signal-to-noise ratio, low spatial resolution, and high inter-trial variability, make decoding challenging. These limitations become even more pronounced when decoding targets are high-dimensional, such as natural images. At the same time, existing research is strongly biased toward well-established and highly separable conditions, while more nuanced scenarios remain underexplored. In this thesis, we move beyond conventional EEG applications by investigating settings in which decoding targets are high-dimensional and/or elicited by nuanced conditions. We assess decoding feasibility in these demanding scenarios and propose methods that leverage pretrained models of the stimulus modality for complex targets. We begin by examining the EEG decoding pipeline and discuss how EEG and task constraints shape the architecture and representation choices of decoding models. We then integrate pretrained stimulus models as priors for predicting high-dimensional outputs and propose two complementary approaches to align stimulus representations with EEG activity. The first uses EEG responses as feedback in an online closed-loop framework to guide a pretrained generative model toward a user’s intended mental image. The second aligns EEG representations with perceptually informed embedding spaces from pretrained vision models, improving the retrieval of perceived images from EEG.To go beyond well-established paradigms, we investigate EEG-based intention decoding in demanding same-limb human–robot collaboration scenarios. We also examine how informative EEG signals are about olfactory perception. Finally, we discuss methodological and evaluation challenges in small-scale, task-based EEG datasets, including risks of performance overestimation and limited generalization.

Abstract [sv]

Hjärnaktivitet utgör en rik informationskälla som kan ge interaktiva intelligenta system insikt i mänskliga intentioner och hur omgivningen uppfattas. Bland icke-invasiva tekniker är elektroencefalografi (EEG) särskilt lämpad för interaktiva tillämpningar tack vare portabilitet, relativt låg kostnad och realtidskapacitet. Framsteg inom artificiell intelligens har möjliggjort avkodningsmodeller som kan extrahera komplexa mönster från EEG-signaler och därigenom avslöja kognitiv information. Samtidigt medför EEG-signalens inneboende egenskaper, såsom låg signal-till-brus-kvot, låg spatial upplösning och hög variabilitet, betydande avkodningsutmaningar. Dessa begränsningar förvärras när avkodningsmålen är högdimensionella, exempelvis naturliga bilder. Dessutom är befintlig forskning i hög grad inriktad på väletablerade, tydligt separerbara experimentella förhållanden, medan nyanserade scenarier ofta förblir outredda.I denna avhandling går vi bortom konventionella EEG-tillämpningar genom att undersöka situationer där avkodningsmålen är högdimensionella och/eller framkallas av mer subtila betingelser. Vi utvärderar möjligheten till avkodning i dessa krävande scenarier och föreslår metoder som utnyttjar förtränade stimulusmodeller.Vi analyserar inledningsvis EEG-avkodningspipelinen och diskuterar hur datans egenskaper och uppgiftskrav påverkar arkitektur- och representationsval. Därefter integrerar vi förtränade stimulusmodeller som priorer för att förutsäga högdimensionella utdata, och föreslår två kompletterande metoder för att anpassa stimulusrepresentationer till EEG-aktivitet. Den första metoden använder EEG-responser som återkoppling i ett slutet system för att styra en förtränad generativ modell mot användarens avsedda mentala bild. Den andra anpassar EEG-representationer till perceptuellt informerade inbäddningsrum från förtränade visionsmodeller, vilket förbättrar återskapandet av uppfattade bilder.För att gå bortom etablerade paradigm undersöker vi även EEG-baserad intentionsavkodning i krävande scenarier för människa–robot-samarbete där samma kroppsdel används. Vidare analyserar vi i vilken utsträckning EEG-signaler innehåller information om luktperception. Slutligen diskuteras metodologiska och utvärderingsrelaterade utmaningar i småskaliga EEG-dataset, inklusive risker för prestandaöverskattning och begränsad generaliserbarhet.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2026. p. 70
Series
TRITA-EECS-AVL ; 2026:32
Keywords
Electroencephalography (EEG), Brain-Computer Interface (BCI), Machine Learning, Elektroencefalografi (EEG), Hjärn-datorgränssnitt, Maskininlärning
National Category
Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-379337 (URN)978-91-8106-577-0 (ISBN)
Public defence
2026-05-12, F3 (Flodis), Lindstedtsvägen 26 & 28, Stockholm, 13:00 (English)
Opponent
Supervisors
Note

QC 20260416

Available from: 2026-04-16 Created: 2026-04-16 Last updated: 2026-04-20Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Rajabi, NonaKhanna, ParagYadollahi, ElmiraBjörkman, MårtenSmith, ChristianKragic, Danica

Search in DiVA

By author/editor
Rajabi, NonaKhanna, ParagYadollahi, ElmiraBjörkman, MårtenSmith, ChristianKragic, Danica
By organisation
Robotics, Perception and Learning, RPL
Robotics and automation

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 187 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf