Change search
Link to record
Permanent link

Direct link
BETA
Alternative names
Publications (10 of 282) Show all publications
Mitsioni, I., Karayiannidis, Y., Stork, J. A. & Kragic, D. (2019). Data-Driven Model Predictive Control for the Contact-Rich Task of Food Cutting. In: : . Paper presented at The 2019 IEEE-RAS International Conference on Humanoid Robots, Toronto, Canada, October 15-17, 2019..
Open this publication in new window or tab >>Data-Driven Model Predictive Control for the Contact-Rich Task of Food Cutting
2019 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Modelling of contact-rich tasks is challenging and cannot be entirely solved using classical control approaches due to the difficulty of constructing an analytic description of the contact dynamics. Additionally, in a manipulation task like food-cutting, purely learning-based methods such as Reinforcement Learning, require either a vast amount of data that is expensive to collect on a real robot, or a highly realistic simulation environment, which is currently not available. This paper presents a data-driven control approach that employs a recurrent neural network to model the dynamics for a Model Predictive Controller. We build upon earlier work limited to torque-controlled robots and redefine it for velocity controlled ones. We incorporate force/torque sensor measurements, reformulate and further extend the control problem formulation. We evaluate the performance on objects used for training, as well as on unknown objects, by means of the cutting rates achieved and demonstrate that the method can efficiently treat different cases with only one dynamic model. Finally we investigate the behavior of the system during force-critical instances of cutting and illustrate its adaptive behavior in difficult cases.

National Category
Engineering and Technology Robotics
Identifiers
urn:nbn:se:kth:diva-258796 (URN)
Conference
The 2019 IEEE-RAS International Conference on Humanoid Robots, Toronto, Canada, October 15-17, 2019.
Note

QC 20191021

Available from: 2019-09-16 Created: 2019-09-16 Last updated: 2019-10-21Bibliographically approved
Cruciani, S., Hang, K., Smith, C. & Kragic, D. (2019). Dual-Arm In-Hand Manipulation Using Visual Feedback. In: : . Paper presented at IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids) October 15-17, 2019 Toronto, Canada (pp. 411-418).
Open this publication in new window or tab >>Dual-Arm In-Hand Manipulation Using Visual Feedback
2019 (English)Conference paper, Published paper (Refereed)
Abstract [en]

In this work, we address the problem of executing in-hand manipulation based on visual input. Given an initial grasp, the robot has to change its grasp configuration without releasing the object. We propose a method for in-hand manipulation planning and execution based on information on the object’s shape using a dual-arm robot. From the available information on the object, which can be a complete point cloud but also partial data, our method plans a sequence of rotations and translations to reconfigure the object’s pose. This sequence is executed using non-prehensile pushes defined as relative motions between the two robot arms.

National Category
Robotics
Identifiers
urn:nbn:se:kth:diva-262881 (URN)
Conference
IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids) October 15-17, 2019 Toronto, Canada
Note

QC 20191129

Available from: 2019-10-22 Created: 2019-10-22 Last updated: 2019-11-29Bibliographically approved
Yuan, W., Hang, K., Kragic, D., Wang, M. Y. & Stork, J. A. (2019). End-to-end nonprehensile rearrangement with deep reinforcement learning and simulation-to-reality transfer. Robotics and Autonomous Systems, 119, 119-134
Open this publication in new window or tab >>End-to-end nonprehensile rearrangement with deep reinforcement learning and simulation-to-reality transfer
Show others...
2019 (English)In: Robotics and Autonomous Systems, ISSN 0921-8890, E-ISSN 1872-793X, Vol. 119, p. 119-134Article in journal (Refereed) Published
Abstract [en]

Nonprehensile rearrangement is the problem of controlling a robot to interact with objects through pushing actions in order to reconfigure the objects into a predefined goal pose. In this work, we rearrange one object at a time in an environment with obstacles using an end-to-end policy that maps raw pixels as visual input to control actions without any form of engineered feature extraction. To reduce the amount of training data that needs to be collected using a real robot, we propose a simulation-to-reality transfer approach. In the first step, we model the nonprehensile rearrangement task in simulation and use deep reinforcement learning to learn a suitable rearrangement policy, which requires in the order of hundreds of thousands of example actions for training. Thereafter, we collect a small dataset of only 70 episodes of real-world actions as supervised examples for adapting the learned rearrangement policy to real-world input data. In this process, we make use of newly proposed strategies for improving the reinforcement learning process, such as heuristic exploration and the curation of a balanced set of experiences. We evaluate our method in both simulation and real setting using a Baxter robot to show that the proposed approach can effectively improve the training process in simulation, as well as efficiently adapt the learned policy to the real world application, even when the camera pose is different from simulation. Additionally, we show that the learned system not only can provide adaptive behavior to handle unforeseen events during executions, such as distraction objects, sudden changes in positions of the objects, and obstacles, but also can deal with obstacle shapes that were not present in the training process.

Place, publisher, year, edition, pages
ELSEVIER, 2019
Keywords
Nonprehensile rearrangement, Deep reinforcement learning, Transfer learning
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:kth:diva-259430 (URN)10.1016/j.robot.2019.06.007 (DOI)000482250400009 ()2-s2.0-85068467713 (Scopus ID)
Note

QC 20190924

Available from: 2019-09-24 Created: 2019-09-24 Last updated: 2019-09-24Bibliographically approved
Sibirtseva, E., Ghadirzadeh, A., Leite, I., Björkman, M. & Kragic, D. (2019). Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality. In: Virtual, Augmented and Mixed Reality. Multimodal Interaction 11th International Conference, VAMR 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26–31, 2019, Proceedings: . Paper presented at 11th International Conference on Virtual, Augmented and Mixed Reality, VAMR 2019, held as part of the 21st International Conference on Human-Computer Interaction, HCI International 2019; Orlando; United States; 26 July 2019 through 31 July 2019 (pp. 108-123). Springer Verlag
Open this publication in new window or tab >>Exploring Temporal Dependencies in Multimodal Referring Expressions with Mixed Reality
Show others...
2019 (English)In: Virtual, Augmented and Mixed Reality. Multimodal Interaction 11th International Conference, VAMR 2019, Held as Part of the 21st HCI International Conference, HCII 2019, Orlando, FL, USA, July 26–31, 2019, Proceedings, Springer Verlag , 2019, p. 108-123Conference paper, Published paper (Refereed)
Abstract [en]

In collaborative tasks, people rely both on verbal and non-verbal cues simultaneously to communicate with each other. For human-robot interaction to run smoothly and naturally, a robot should be equipped with the ability to robustly disambiguate referring expressions. In this work, we propose a model that can disambiguate multimodal fetching requests using modalities such as head movements, hand gestures, and speech. We analysed the acquired data from mixed reality experiments and formulated a hypothesis that modelling temporal dependencies of events in these three modalities increases the model’s predictive power. We evaluated our model on a Bayesian framework to interpret referring expressions with and without exploiting the temporal prior.

Place, publisher, year, edition, pages
Springer Verlag, 2019
Series
Lecture Notes in Artificial Intelligence, ISSN 0302-9743 ; 11575
Keywords
Human-robot interaction, Mixed reality, Multimodal interaction, Referring expressions, Human computer interaction, Human robot interaction, Bayesian frameworks, Collaborative tasks, Hand gesture, Head movements, Multi-modal, Multi-Modal Interactions, Predictive power
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:kth:diva-262467 (URN)10.1007/978-3-030-21565-1_8 (DOI)2-s2.0-85069730416 (Scopus ID)9783030215644 (ISBN)
Conference
11th International Conference on Virtual, Augmented and Mixed Reality, VAMR 2019, held as part of the 21st International Conference on Human-Computer Interaction, HCI International 2019; Orlando; United States; 26 July 2019 through 31 July 2019
Note

QC 20191017

Available from: 2019-10-17 Created: 2019-10-17 Last updated: 2019-10-17Bibliographically approved
Haustein, J. A., Hang, K., Stork, J. A. & Kragic, D. (2019). Object Placement Planning and Optimization for Robot Manipulators. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019): . Paper presented at International Conference on Intelligent Robots and Systems (IROS), Macau, China, November 4-8, 2019.
Open this publication in new window or tab >>Object Placement Planning and Optimization for Robot Manipulators
2019 (English)In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019), 2019Conference paper, Published paper (Refereed)
Abstract [en]

We address the problem of planning the placement of a rigid object with a dual-arm robot in a cluttered environment. In this task, we need to locate a collision-free pose for the object that a) facilitates the stable placement of the object, b) is reachable by the robot and c) optimizes a user-given placement objective. In addition, we need to select which robot arm to perform the placement with. To solve this task, we propose an anytime algorithm that integrates sampling-based motion planning with a novel hierarchical search for suitable placement poses. Our algorithm incrementally produces approach motions to stable placement poses, reaching placements with better objective as runtime progresses. We evaluate our approach for two different placement objectives, and observe its effectiveness even in challenging scenarios.

Keywords
Motion planning, Object placing
National Category
Computer Vision and Robotics (Autonomous Systems)
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-264015 (URN)
Conference
International Conference on Intelligent Robots and Systems (IROS), Macau, China, November 4-8, 2019
Funder
Swedish Foundation for Strategic Research Knut and Alice Wallenberg Foundation
Note

QC 20191210

Available from: 2019-11-20 Created: 2019-11-20 Last updated: 2019-12-10Bibliographically approved
Hang, K., Lyu, X., Song, H., Stork, J. A., Dollar, A. M., Kragic, D. & Zhang, F. (2019). Perching and resting-A paradigm for UAV maneuvering with modularized landing gears. SCIENCE ROBOTICS, 4(28), Article ID eaau6637.
Open this publication in new window or tab >>Perching and resting-A paradigm for UAV maneuvering with modularized landing gears
Show others...
2019 (English)In: SCIENCE ROBOTICS, ISSN 2470-9476, Vol. 4, no 28, article id eaau6637Article in journal (Refereed) Published
Abstract [en]

Perching helps small unmanned aerial vehicles (UAVs) extend their time of operation by saving battery power. However, most strategies for UAV perching require complex maneuvering and rely on specific structures, such as rough walls for attaching or tree branches for grasping. Many strategies to perching neglect the UAV's mission such that saving battery power interrupts the mission. We suggest enabling UAVs with the capability of making and stabilizing contacts with the environment, which will allow the UAV to consume less energy while retaining its altitude, in addition to the perching capability that has been proposed before. This new capability is termed "resting." For this, we propose a modularized and actuated landing gear framework that allows stabilizing the UAV on a wide range of different structures by perching and resting. Modularization allows our framework to adapt to specific structures for resting through rapid prototyping with additive manufacturing. Actuation allows switching between different modes of perching and resting during flight and additionally enables perching by grasping. Our results show that this framework can be used to perform UAV perching and resting on a set of common structures, such as street lights and edges or corners of buildings. We show that the design is effective in reducing power consumption, promotes increased pose stability, and preserves large vision ranges while perching or resting at heights. In addition, we discuss the potential applications facilitated by our design, as well as the potential issues to be addressed for deployment in practice.

Place, publisher, year, edition, pages
AMER ASSOC ADVANCEMENT SCIENCE, 2019
National Category
Robotics
Identifiers
urn:nbn:se:kth:diva-251220 (URN)10.1126/scirobotics.aau6637 (DOI)000464024300001 ()2-s2.0-85063677452 (Scopus ID)
Note

QC 20190523

Available from: 2019-05-23 Created: 2019-05-23 Last updated: 2019-05-23Bibliographically approved
Haustein, J. A., Cruciani, S., Asif, R., Hang, K. & Kragic, D. (2019). Placing Objects with prior In-Hand Manipulation using Dexterous Manipulation Graphs. In: : . Paper presented at IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids), Toronto, Canada, October 15-17, 2019. (pp. 477-484).
Open this publication in new window or tab >>Placing Objects with prior In-Hand Manipulation using Dexterous Manipulation Graphs
Show others...
2019 (English)Conference paper, Published paper (Refereed)
Abstract [en]

We address the problem of planning the placement of a grasped object with a robot manipulator. More specifically, the robot is tasked to place the grasped object such that a placement preference function is maximized. For this, we present an approach that uses in-hand manipulation to adjust the robot’s initial grasp to extend the set of reachable placements. Given an initial grasp, the algorithm computes a set of grasps that can be reached by pushing and rotating the object in-hand. With this set of reachable grasps, it then searches for a stable placement that maximizes the preference function. If successful it returns a sequence of in-hand pushes to adjust the initial grasp to a more advantageous grasp together with a transport motion that carries the object to the placement. We evaluate our algorithm’s performance on various placing scenarios, and observe its effectiveness also in challenging scenes containing many obstacles. Our experiments demonstrate that re-grasping with in-hand manipulation increases the quality of placements the robot can reach. In particular, it enables the algorithm to find solutions in situations where safe placing with the initial grasp wouldn’t be possible.

National Category
Robotics
Identifiers
urn:nbn:se:kth:diva-262882 (URN)
Conference
IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids), Toronto, Canada, October 15-17, 2019.
Note

QC 20191115

Available from: 2019-10-22 Created: 2019-10-22 Last updated: 2019-11-15Bibliographically approved
Billard, A. & Kragic, D. (2019). Trends and challenges in robot manipulation. Science, 364(6446), 1149-+
Open this publication in new window or tab >>Trends and challenges in robot manipulation
2019 (English)In: Science, ISSN 0036-8075, E-ISSN 1095-9203, Vol. 364, no 6446, p. 1149-+Article, review/survey (Refereed) Published
Abstract [en]

Dexterous manipulation is one of the primary goals in robotics. Robots with this capability could sort and package objects, chop vegetables, and fold clothes. As robots come to work side by side with humans, they must also become human-aware. Over the past decade, research has made strides toward these goals. Progress has come from advances in visual and haptic perception and in mechanics in the form of soft actuators that offer a natural compliance. Most notably, immense progress in machine learning has been leveraged to encapsulate models of uncertainty and to support improvements in adaptive and robust control. Open questions remain in terms of how to enable robots to deal with the most unpredictable agent of all, the human.

Place, publisher, year, edition, pages
American Association for the Advancement of Science, 2019
National Category
Robotics
Identifiers
urn:nbn:se:kth:diva-255314 (URN)10.1126/science.aat8414 (DOI)000472175100030 ()31221831 (PubMedID)2-s2.0-85068153256 (Scopus ID)
Note

QC 20190807

Available from: 2019-08-07 Created: 2019-08-07 Last updated: 2019-08-07Bibliographically approved
Arnekvist, I., Kragic, D. & Stork, J. A. (2019). Vpe: Variational policy embedding for transfer reinforcement learning. In: 2019 International Conference on Robotics And Automation (ICRA): . Paper presented at 2019 International Conference on Robotics and Automation, ICRA 2019; Palais des Congres de Montreal, Montreal; Canada; 20 May 2019 through 24 May 2019 (pp. 36-42). Institute of Electrical and Electronics Engineers (IEEE)
Open this publication in new window or tab >>Vpe: Variational policy embedding for transfer reinforcement learning
2019 (English)In: 2019 International Conference on Robotics And Automation (ICRA), Institute of Electrical and Electronics Engineers (IEEE), 2019, p. 36-42Conference paper, Published paper (Refereed)
Abstract [en]

Reinforcement Learning methods are capable of solving complex problems, but resulting policies might perform poorly in environments that are even slightly different. In robotics especially, training and deployment conditions often vary and data collection is expensive, making retraining undesirable. Simulation training allows for feasible training times, but on the other hand suffer from a reality-gap when applied in real-world settings. This raises the need of efficient adaptation of policies acting in new environments. We consider the problem of transferring knowledge within a family of similar Markov decision processes. We assume that Q-functions are generated by some low-dimensional latent variable. Given such a Q-function, we can find a master policy that can adapt given different values of this latent variable. Our method learns both the generative mapping and an approximate posterior of the latent variables, enabling identification of policies for new tasks by searching only in the latent space, rather than the space of all policies. The low-dimensional space, and master policy found by our method enables policies to quickly adapt to new environments. We demonstrate the method on both a pendulum swing-up task in simulation, and for simulation-to-real transfer on a pushing task.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2019
Series
IEEE International Conference on Robotics and Automation ICRA, ISSN 1050-4729
National Category
Computer Vision and Robotics (Autonomous Systems) Computer Sciences
Identifiers
urn:nbn:se:kth:diva-258072 (URN)10.1109/ICRA.2019.8793556 (DOI)000494942300006 ()2-s2.0-85071508761 (Scopus ID)9781538660263 (ISBN)
Conference
2019 International Conference on Robotics and Automation, ICRA 2019; Palais des Congres de Montreal, Montreal; Canada; 20 May 2019 through 24 May 2019
Projects
Factories of the Future (FACT)
Note

QC 20190916

Available from: 2019-09-09 Created: 2019-09-09 Last updated: 2019-12-12Bibliographically approved
Sibirtseva, E., Kontogiorgos, D., Nykvist, O., Karaoguz, H., Leite, I., Gustafson, J. & Kragic, D. (2018). A Comparison of Visualisation Methods for Disambiguating Verbal Requests in Human-Robot Interaction. In: 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN): . Paper presented at ROMAN 2018.
Open this publication in new window or tab >>A Comparison of Visualisation Methods for Disambiguating Verbal Requests in Human-Robot Interaction
Show others...
2018 (English)In: 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2018Conference paper, Published paper (Refereed)
Abstract [en]

Picking up objects requested by a human user is a common task in human-robot interaction. When multiple objects match the user's verbal description, the robot needs to clarify which object the user is referring to before executing the action. Previous research has focused on perceiving user's multimodal behaviour to complement verbal commands or minimising the number of follow up questions to reduce task time. In this paper, we propose a system for reference disambiguation based on visualisation and compare three methods to disambiguate natural language instructions. In a controlled experiment with a YuMi robot, we investigated realtime augmentations of the workspace in three conditions - head-mounted display, projector, and a monitor as the baseline - using objective measures such as time and accuracy, and subjective measures like engagement, immersion, and display interference. Significant differences were found in accuracy and engagement between the conditions, but no differences were found in task time. Despite the higher error rates in the head-mounted display condition, participants found that modality more engaging than the other two, but overall showed preference for the projector condition over the monitor and head-mounted display conditions.

National Category
Human Computer Interaction
Identifiers
urn:nbn:se:kth:diva-235548 (URN)10.1109/ROMAN.2018.8525554 (DOI)978-1-5386-7981-4 (ISBN)
Conference
ROMAN 2018
Note

QC 20181207

Available from: 2018-09-29 Created: 2018-09-29 Last updated: 2018-12-07Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0003-2965-2953

Search in DiVA

Show all publications