kth.sePublications
Change search
Link to record
Permanent link

Direct link
Alternative names
Publications (10 of 177) Show all publications
Mohamed, Y., Lemaignan, S., Güneysu, A., Jensfelt, P. & Smith, C. (2025). Context Matters: Understanding Socially Appropriate Affective Responses Via Sentence Embeddings. In: Social Robotics - 16th International Conference, ICSR + AI 2024, Proceedings: . Paper presented at 16th International Conference on Social Robotics, ICSR + AI 2024, Odense, Denmark, October 23-26, 2024 (pp. 78-91). Springer Nature
Open this publication in new window or tab >>Context Matters: Understanding Socially Appropriate Affective Responses Via Sentence Embeddings
Show others...
2025 (English)In: Social Robotics - 16th International Conference, ICSR + AI 2024, Proceedings, Springer Nature , 2025, p. 78-91Conference paper, Published paper (Refereed)
Abstract [en]

As AI systems increasingly engage in social interactions, comprehending human social dynamics is crucial. Affect recognition enables systems to respond appropriately to emotional nuances in social situations. However, existing multimodal approaches lack accounting for the social appropriateness of detected emotions within their contexts. This paper presents a novel methodology leveraging sentence embeddings to distinguish socially appropriate and inappropriate interactions for more context-aware AI systems. Our approach measures the semantic distance between facial expression descriptions and predefined reference points. We evaluate our method using a benchmark dataset and a real-world robot deployment in a library, combining GPT-4(V) for expression descriptions and ada-2 for sentence embeddings to detect socially inappropriate interactions. Our results underscore the importance of considering contextual factors for effective social interaction understanding through context-aware affect recognition, contributing to the development of socially intelligent AI capable of interpreting and responding to human affect appropriately.

Place, publisher, year, edition, pages
Springer Nature, 2025
Keywords
embeddings, human-robot interaction, machine learning, Social representation
National Category
Sociology (Excluding Social Work, Social Anthropology, Demography and Criminology) Robotics and automation
Identifiers
urn:nbn:se:kth:diva-362501 (URN)10.1007/978-981-96-3522-1_9 (DOI)2-s2.0-105002016733 (Scopus ID)
Conference
16th International Conference on Social Robotics, ICSR + AI 2024, Odense, Denmark, October 23-26, 2024
Note

Part of ISBN 9789819635214

QC 20250428

Available from: 2025-04-16 Created: 2025-04-16 Last updated: 2025-04-28Bibliographically approved
Zhang, Q., Yang, Y., Li, P., Andersson, O. & Jensfelt, P. (2025). SeFlow: A Self-supervised Scene Flow Method in Autonomous Driving. In: Roth, S Russakovsky, O Sattler, T Varol, G Leonardis, A Ricci, E (Ed.), COMPUTER VISION-ECCV 2024, PT I: . Paper presented at 18th European Conference on Computer Vision (ECCV), SEP 29-OCT 04, 2024, Milan, ITALY (pp. 353-369). Springer Nature, 15059
Open this publication in new window or tab >>SeFlow: A Self-supervised Scene Flow Method in Autonomous Driving
Show others...
2025 (English)In: COMPUTER VISION-ECCV 2024, PT I / [ed] Roth, S Russakovsky, O Sattler, T Varol, G Leonardis, A Ricci, E, Springer Nature , 2025, Vol. 15059, p. 353-369Conference paper, Published paper (Refereed)
Abstract [en]

Scene flow estimation predicts the 3D motion at each point in successive LiDAR scans. This detailed, point-level, information can help autonomous vehicles to accurately predict and understand dynamic changes in their surroundings. Current state-of-the-art methods require annotated data to train scene flow networks and the expense of labeling inherently limits their scalability. Self-supervised approaches can overcome the above limitations, yet face two principal challenges that hinder optimal performance: point distribution imbalance and disregard for object-level motion constraints. In this paper, we propose SeFlow, a self-supervised method that integrates efficient dynamic classification into a learning-based scene flow pipeline. We demonstrate that classifying static and dynamic points helps design targeted objective functions for different motion patterns. We also emphasize the importance of internal cluster consistency and correct object point association to refine the scene flow estimation, in particular on object details. Our real-time capable method achieves state-of-the-art performance on the self-supervised scene flow task on Argoverse 2 and Waymo datasets. The code is open-sourced at https://github.com/KTH-RPL/SeFlow.

Place, publisher, year, edition, pages
Springer Nature, 2025
Series
Lecture Notes in Computer Science, ISSN 0302-9743 ; 15059
Keywords
3D scene flow, self-supervised, autonomous driving, large-scale point cloud
National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-357529 (URN)10.1007/978-3-031-73232-4_20 (DOI)001346378300020 ()2-s2.0-85206389477 (Scopus ID)
Conference
18th European Conference on Computer Vision (ECCV), SEP 29-OCT 04, 2024, Milan, ITALY
Note

Part of ISBN 978-3-031-73231-7; 978-3-031-73232-4

QC 20241209

Available from: 2024-12-09 Created: 2024-12-09 Last updated: 2025-02-07Bibliographically approved
Stower, R., Gautier, A., Wozniak, M. K., Jensfelt, P., Tumova, J. & Leite, I. (2025). Take a Chance on Me: How Robot Performance and Risk Behaviour Affects Trust and Risk-Taking. In: HRI 2025 - Proceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interaction: . Paper presented at 20th Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2025, Melbourne, Australia, Mar 4 2025 - Mar 6 2025 (pp. 391-399). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Take a Chance on Me: How Robot Performance and Risk Behaviour Affects Trust and Risk-Taking
Show others...
2025 (English)In: HRI 2025 - Proceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interaction, Institute of Electrical and Electronics Engineers (IEEE) , 2025, p. 391-399Conference paper, Published paper (Refereed)
Abstract [en]

Real-world human-robot interactions often encompass uncertainty. This uncertainty can be handled in different ways, for example by designing robot planners to be more or less risk-tolerant. However, how users actually perceive different risk-taking behaviours in robots has yet to be described. Additionally, in the absence of guarantees on optimal robot performance, the interaction between risk and performance on user perceptions is also unclear. To address this gap, we conducted a user study with 84 participants investigating how robot performance and risk behaviour affects users' trust and risk-taking decisions. Participants collaborated with a Franka robot arm to perform a block-stacking task. We compared a robot which displays consistent but sub-optimal behaviours to a robot displaying risky but occasionally optimal behaviour. Risky robot behaviour led to higher trust than consistent behaviour when the robot was on average good at stacking blocks (high expectation), but lower trust when the robot was on average bad at stacking blocks (low expectation). Individual risk-willingness also predicted likelihood of selecting the risky robot over the consistent robot for future interactions, but only when the average expectation was low. These findings have implications for risk-aware planning and decision-making in mixed human-robot systems.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2025
Keywords
collaborative robot, failure, risk-taking, trust, user study
National Category
Robotics and automation Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-363768 (URN)10.1109/HRI61500.2025.10973966 (DOI)2-s2.0-105004879443 (Scopus ID)
Conference
20th Annual ACM/IEEE International Conference on Human-Robot Interaction, HRI 2025, Melbourne, Australia, Mar 4 2025 - Mar 6 2025
Note

Part of ISBN 9798350378931

QC 20250527

Available from: 2025-05-21 Created: 2025-05-21 Last updated: 2025-05-27Bibliographically approved
Gaspar Sánchez, J. M., Bruns, L., Tumova, J., Jensfelt, P. & Törngren, M. (2025). Transitional Grid Maps: Joint Modeling of Static and Dynamic Occupancy. IEEE Open Journal of Intelligent Transportation Systems, 6, 1-10
Open this publication in new window or tab >>Transitional Grid Maps: Joint Modeling of Static and Dynamic Occupancy
Show others...
2025 (English)In: IEEE Open Journal of Intelligent Transportation Systems, E-ISSN 2687-7813, Vol. 6, p. 1-10Article in journal (Refereed) Published
Abstract [en]

Autonomous agents rely on sensor data to construct representations of their environments, essential for predicting future events and planning their actions. However, sensor measurements suffer from limited range, occlusions, and sensor noise. These challenges become more evident in highly dynamic environments. This work proposes a probabilistic framework to jointly infer which parts of an environment are statically and which parts are dynamically occupied. We formulate the problem as a Bayesian network and introduce minimal assumptions that significantly reduce the complexity of the problem. Based on those, we derive Transitional Grid Maps (TGMs), an efficient analytical solution. Using real data, we demonstrate how this approach produces better maps than the state-of-the-art by keeping track of both static and dynamic elements and, as a side effect, can help improve existing SLAM algorithms.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2025
National Category
Computer Sciences Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-359349 (URN)10.1109/ojits.2024.3521449 (DOI)2-s2.0-85210909052 (Scopus ID)
Note

QC 20250130

Available from: 2025-01-30 Created: 2025-01-30 Last updated: 2025-05-27Bibliographically approved
Khoche, A., Asefaw, A., González, A., Timus, B., Mansouri, S. S. & Jensfelt, P. (2024). Addressing Data Annotation Challenges in Multiple Sensors: A Solution for Scania Collected Datasets. In: 2024 European Control Conference, ECC 2024: . Paper presented at 2024 European Control Conference, ECC 2024, Stockholm, Sweden, Jun 25 2024 - Jun 28 2024 (pp. 1032-1038). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Addressing Data Annotation Challenges in Multiple Sensors: A Solution for Scania Collected Datasets
Show others...
2024 (English)In: 2024 European Control Conference, ECC 2024, Institute of Electrical and Electronics Engineers (IEEE) , 2024, p. 1032-1038Conference paper, Published paper (Refereed)
Abstract [en]

Data annotation in autonomous vehicles is a critical step in the development of Deep Neural Network (DNN) based models or the performance evaluation of the perception system. This often takes the form of adding 3D bounding boxes on time-sequential and registered series of point-sets captured from active sensors like Light Detection and Ranging (LiDAR) and Radio Detection and Ranging (RADAR). When annotating multiple active sensors, there is a need to motion compensate and translate the points to a consistent coordinate frame and timestamp respectively. However, highly dynamic objects pose a unique challenge, as they can appear at different timestamps in each sensor's data. Without knowing the speed of the objects, their position appears to be different in different sensor outputs. Thus, even after motion compensation, highly dynamic objects are not matched from multiple sensors in the same frame, and human annotators struggle to add unique bounding boxes that capture all objects. This article focuses on addressing this challenge, primarily within the context of Scania-collected datasets. The proposed solution takes a track of an annotated object as input and uses the Moving Horizon Estimation (MHE) to robustly estimate its speed. The estimated speed profile is utilized to correct the position of the annotated box and add boxes to object clusters missed by the original annotation.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:kth:diva-351940 (URN)10.23919/ECC64448.2024.10590958 (DOI)001290216500156 ()2-s2.0-85200563187 (Scopus ID)
Conference
2024 European Control Conference, ECC 2024, Stockholm, Sweden, Jun 25 2024 - Jun 28 2024
Note

Part of ISBN [9783907144107]

QC 20240830

Available from: 2024-08-19 Created: 2024-08-19 Last updated: 2025-04-28Bibliographically approved
Ericson, L. & Jensfelt, P. (2024). Beyond the Frontier: Predicting Unseen Walls From Occupancy Grids by Learning From Floor Plans. IEEE Robotics and Automation Letters, 9(8), 6832-6839
Open this publication in new window or tab >>Beyond the Frontier: Predicting Unseen Walls From Occupancy Grids by Learning From Floor Plans
2024 (English)In: IEEE Robotics and Automation Letters, E-ISSN 2377-3766, Vol. 9, no 8, p. 6832-6839Article in journal (Refereed) Published
Abstract [en]

In this letter, we tackle the challenge of predicting the unseen walls of a partially observed environment as a set of 2D line segments, conditioned on occupancy grids integrated along the trajectory of a 360(degrees) LIDAR sensor. A dataset of such occupancy grids and their corresponding target wall segments is collected by navigating a virtual robot between a set of randomly sampled waypoints in a collection of office-scale floor plans from a university campus. The line segment prediction task is formulated as an autoregressive sequence prediction task, and an attention-based deep network is trained on the dataset. The sequence-based autoregressive formulation is evaluated through predicted information gain, as in frontier-based autonomous exploration, demonstrating significant improvements over both non-predictive estimation and convolution-based image prediction found in the literature. Ablations on key components are evaluated, as well as sensor range and the occupancy grid's metric area. Finally, model generality is validated by predicting walls in a novel floor plan reconstructed on-the-fly in a real-world office environment.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
Keywords
Deep learning methods, planning under uncertainty, autonomous agents, learning from experience, map-predictive exploration
National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:kth:diva-350044 (URN)10.1109/LRA.2024.3410164 (DOI)001251164900004 ()2-s2.0-85195423165 (Scopus ID)
Note

QC 20240705

Available from: 2024-07-05 Created: 2024-07-05 Last updated: 2025-05-06Bibliographically approved
Zangeneh, F., Bruns, L., Dekel, A., Pieropan, A. & Jensfelt, P. (2024). Conditional Variational Autoencoders for Probabilistic Pose Regression. In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024: . Paper presented at 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024, Abu Dhabi, United Arab Emirates, Oct 14 2024 - Oct 18 2024 (pp. 2794-2800). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Conditional Variational Autoencoders for Probabilistic Pose Regression
Show others...
2024 (English)In: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024, Institute of Electrical and Electronics Engineers (IEEE) , 2024, p. 2794-2800Conference paper, Published paper (Refereed)
Abstract [en]

Robots rely on visual relocalization to estimate their pose from camera images when they lose track. One of the challenges in visual relocalization is repetitive structures in the operation environment of the robot. This calls for probabilistic methods that support multiple hypotheses for robot's pose. We propose such a probabilistic method to predict the posterior distribution of camera poses given an observed image. Our proposed training strategy results in a generative model of camera poses given an image, which can be used to draw samples from the pose posterior distribution. Our method is streamlined and well-founded in theory and outperforms existing methods on localization in presence of ambiguities.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
National Category
Computer graphics and computer vision Robotics and automation
Identifiers
urn:nbn:se:kth:diva-359873 (URN)10.1109/IROS58592.2024.10802091 (DOI)001411890000287 ()2-s2.0-85216445787 (Scopus ID)
Conference
2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2024, Abu Dhabi, United Arab Emirates, Oct 14 2024 - Oct 18 2024
Note

Part of ISBN 9798350377705

QC 20250213

Available from: 2025-02-12 Created: 2025-02-12 Last updated: 2025-04-25Bibliographically approved
Khoche, A., Sánchez, L. P., Batool, N., Mansouri, S. S. & Jensfelt, P. (2024). Towards Long-Range 3D Object Detection for Autonomous Vehicles. In: 35th IEEE Intelligent Vehicles Symposium, IV 2024: . Paper presented at 35th IEEE Intelligent Vehicles Symposium, IV 2024, Jeju Island, Korea, Jun 2 2024 - Jun 5 2024 (pp. 2206-2212). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Towards Long-Range 3D Object Detection for Autonomous Vehicles
Show others...
2024 (English)In: 35th IEEE Intelligent Vehicles Symposium, IV 2024, Institute of Electrical and Electronics Engineers (IEEE) , 2024, p. 2206-2212Conference paper, Published paper (Refereed)
Abstract [en]

3D object detection at long-range is crucial for ensuring the safety and efficiency of self-driving vehicles, allowing them to accurately perceive and react to objects, obstacles, and potential hazards from a distance. But most current state-of-the-art LiDAR based methods are range limited due to sparsity at long-range, which generates a form of domain gap between points closer to and farther away from the ego vehicle. Another related problem is the label imbalance for faraway objects, which inhibits the performance of Deep Neural Networks at long-range. To address the above limitations, we investigate two ways to improve long-range performance of current LiDAR-based 3D detectors. First, we combine two 3D detection networks, referred to as range experts, one specializing at near to mid-range objects, and one at long-range 3D detection. To train a detector at long-range under a scarce label regime, we further weigh the loss according to the labelled point's distance from ego vehicle. Second, we augment LiDAR scans with virtual points generated using Multimodal Virtual Points (MVP), a readily available image-based depth completion algorithm. Our experiments on the long-range Argoverse2 (AV2) dataset indicate that MVP is more effective in improving long range performance, while maintaining a straightforward implementation. On the other hand, the range experts offer a computationally efficient and simpler alternative, avoiding dependency on image-based segmentation networks and perfect camera-LiDAR calibration.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2024
National Category
Computer graphics and computer vision Robotics and automation
Identifiers
urn:nbn:se:kth:diva-351752 (URN)10.1109/IV55156.2024.10588513 (DOI)001275100902040 ()2-s2.0-85199779839 (Scopus ID)
Conference
35th IEEE Intelligent Vehicles Symposium, IV 2024, Jeju Island, Korea, Jun 2 2024 - Jun 5 2024
Note

Part of ISBN [9798350348811]

QC 20240823

Available from: 2024-08-13 Created: 2024-08-13 Last updated: 2025-02-05Bibliographically approved
Zhang, Q., Duberg, D., Geng, R., Jia, M., Wang, L. & Jensfelt, P. (2023). A Dynamic Points Removal Benchmark in Point Cloud Maps. In: 2023 IEEE 26th International Conference on Intelligent Transportation Systems, ITSC 2023: . Paper presented at 26th IEEE International Conference on Intelligent Transportation Systems, ITSC 2023, Bilbao, Spain, Sep 24 2023 - Sep 28 2023 (pp. 608-614). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>A Dynamic Points Removal Benchmark in Point Cloud Maps
Show others...
2023 (English)In: 2023 IEEE 26th International Conference on Intelligent Transportation Systems, ITSC 2023, Institute of Electrical and Electronics Engineers (IEEE) , 2023, p. 608-614Conference paper, Published paper (Refereed)
Abstract [en]

In the field of robotics, the point cloud has become an essential map representation. From the perspective of downstream tasks like localization and global path planning, points corresponding to dynamic objects will adversely affect their performance. Existing methods for removing dynamic points in point clouds often lack clarity in comparative evaluations and comprehensive analysis. Therefore, we propose an easy-to-extend unified benchmarking framework for evaluating techniques for removing dynamic points in maps. It includes refactored state-of-art methods and novel metrics to analyze the limitations of these approaches. This enables researchers to dive deep into the underlying reasons behind these limitations. The benchmark makes use of several datasets with different sensor types. All the code and datasets related to our study are publicly available for further development and utilization.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
National Category
Robotics and automation
Identifiers
urn:nbn:se:kth:diva-344365 (URN)10.1109/ITSC57777.2023.10422094 (DOI)2-s2.0-85186537890 (Scopus ID)
Conference
26th IEEE International Conference on Intelligent Transportation Systems, ITSC 2023, Bilbao, Spain, Sep 24 2023 - Sep 28 2023
Note

Part of ISBN 9798350399462

QC 20240315

Available from: 2024-03-13 Created: 2024-03-13 Last updated: 2025-02-09Bibliographically approved
Zangeneh, F., Bruns, L., Dekel, A., Pieropan, A. & Jensfelt, P. (2023). A Probabilistic Framework for Visual Localization in Ambiguous Scenes. In: Proceedings - ICRA 2023: IEEE International Conference on Robotics and Automation. Paper presented at 2023 IEEE International Conference on Robotics and Automation, ICRA 2023, London, United Kingdom of Great Britain and Northern Ireland, May 29 2023 - Jun 2 2023 (pp. 3969-3975). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>A Probabilistic Framework for Visual Localization in Ambiguous Scenes
Show others...
2023 (English)In: Proceedings - ICRA 2023: IEEE International Conference on Robotics and Automation, Institute of Electrical and Electronics Engineers (IEEE) , 2023, p. 3969-3975Conference paper, Published paper (Refereed)
Abstract [en]

Visual localization allows autonomous robots to relocalize when losing track of their pose by matching their current observation with past ones. However, ambiguous scenes pose a challenge for such systems, as repetitive structures can be viewed from many distinct, equally likely camera poses, which means it is not sufficient to produce a single best pose hypothesis. In this work, we propose a probabilistic framework that for a given image predicts the arbitrarily shaped posterior distribution of its camera pose. We do this via a novel formulation of camera pose regression using variational inference, which allows sampling from the predicted distribution. Our method outperforms existing methods on localization in ambiguous scenes. We open-source our approach and share our recorded data sequence at github.com/efreidun/vapor.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2023
National Category
Computer graphics and computer vision Robotics and automation Signal Processing
Identifiers
urn:nbn:se:kth:diva-336775 (URN)10.1109/ICRA48891.2023.10160466 (DOI)001036713003052 ()2-s2.0-85168671933 (Scopus ID)
Conference
2023 IEEE International Conference on Robotics and Automation, ICRA 2023, London, United Kingdom of Great Britain and Northern Ireland, May 29 2023 - Jun 2 2023
Note

Part of ISBN 9798350323658

QC 20230920

Available from: 2023-09-20 Created: 2023-09-20 Last updated: 2025-02-05Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-1170-7162

Search in DiVA

Show all publications