kth.sePublications
Change search
Refine search result
1234567 1 - 50 of 708
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Abdul Khader, Shahbaz
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Data-Driven Methods for Contact-Rich Manipulation: Control Stability and Data-Efficiency2021Doctoral thesis, comprehensive summary (Other academic)
    Abstract [en]

    Autonomous robots are expected to make a greater presence in the homes and workplaces of human beings. Unlike their industrial counterparts, autonomous robots have to deal with a great deal of uncertainty and lack of structure in their environment. A remarkable aspect of performing manipulation in such a scenario is the possibility of physical contact between the robot and the environment. Therefore, not unlike human manipulation, robotic manipulation has to manage contacts, both expected and unexpected, that are often characterized by complex interaction dynamics.

    Skill learning has emerged as a promising approach for robots to acquire rich motion generation capabilities. In skill learning, data driven methods are used to learn reactive control policies that map states to actions. Such an approach is appealing because a sufficiently expressive policy can almost instantaneously generate appropriate control actions without the need for computationally expensive search operations. Although reinforcement learning (RL) is a natural framework for skill learning, its practical application is limited for a number of reasons. Arguably, the two main reasons are the lack of guaranteed control stability and poor data-efficiency. While control stability is necessary for ensuring safety and predictability, data-efficiency is required for achieving realistic training times. In this thesis, solutions are sought for these two issues in the context of contact-rich manipulation.

    First, this thesis addresses the problem of control stability. Despite unknown interaction dynamics during contact, skill learning with stability guarantee is formulated as a model-free RL problem. The thesis proposes multiple solutions for parameterizing stability-aware policies. Some policy parameterizations are partly or almost wholly deep neural networks. This is followed by policy search solutions that preserve stability during random exploration, if required. In one case, a novel evolution strategies-based policy search method is introduced. It is shown, with the help of real robot experiments, that Lyapunov stability is both possible and beneficial for RL-based skill learning.

    Second, this thesis addresses the issue of data-efficiency. Although data-efficiency is targeted by formulating skill learning as a model-based RL problem, only the model learning part is addressed. In addition to benefiting from the data-efficiency and uncertainty representation of the Gaussian process, this thesis further investigates the benefits of adopting the structure of hybrid automata for learning forward dynamics models. The method also includes an algorithm for predicting long-term trajectory distributions that can represent discontinuities and multiple modes. The proposed method is shown to be more data-efficient than some state-of-the-art methods. 

    Download full text (pdf)
    fulltext
  • 2.
    Abdul Khader, Shahbaz
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. ABB Future Labs, CH-5405 Baden, Switzerland..
    Yin, Hang
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Falco, Pietro
    ABB Corp Res, S-72178 Västerås, Sweden..
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Data-Efficient Model Learning and Prediction for Contact-Rich Manipulation Tasks2020In: IEEE Robotics and Automation Letters, E-ISSN 2377-3766, Vol. 5, no 3, p. 4321-4328Article in journal (Refereed)
    Abstract [en]

    In this letter, we investigate learning forward dynamics models and multi-step prediction of state variables (long-term prediction) for contact-rich manipulation. The problems are formulated in the context of model-based reinforcement learning (MBRL). We focus on two aspects-discontinuous dynamics and data-efficiency-both of which are important in the identified scope and pose significant challenges to State-of-the-Art methods. We contribute to closing this gap by proposing a method that explicitly adopts a specific hybrid structure for the model while leveraging the uncertainty representation and data-efficiency of Gaussian process. Our experiments on an illustrative moving block task and a 7-DOF robot demonstrate a clear advantage when compared to popular baselines in low data regimes.

  • 3.
    Abdul Khader, Shahbaz
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Yin, Hang
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Falco, Pietro
    ABB Corporate Research, Vasteras, 72178, Sweden.
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. ABB Corporate Research, Vasteras, 72178, Sweden.
    Learning deep energy shaping policies for stability-guaranteed manipulation2021In: IEEE Robotics and Automation Letters, E-ISSN 2377-3766, Vol. 6, no 4, p. 8583-8590Article in journal (Refereed)
    Abstract [en]

    Deep reinforcement learning (DRL) has been successfully used to solve various robotic manipulation tasks. However, most of the existing works do not address the issue of control stability. This is in sharp contrast to the control theory community where the well-established norm is to prove stability whenever a control law is synthesized. What makes traditional stability analysis difficult for DRL are the uninterpretable nature of the neural network policies and unknown system dynamics. In this work, stability is obtained by deriving an interpretable deep policy structure based on the energy shaping control of Lagrangian systems. Then, stability during physical interaction with an unknown environment is established based on passivity. The result is a stability guaranteeing DRL in a model-free framework that is general enough for contact-rich manipulation tasks. With an experiment on a peg-in-hole task, we demonstrate, to the best of our knowledge, the first DRL with stability guarantee on a real robotic manipulator.

  • 4.
    Abdul Khader, Shahbaz
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Yin, Hang
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Falco, Pietro
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. KTH, School of Electrical Engineering and Computer Science (EECS), Centres, Centre for Autonomous Systems, CAS.
    Learning Deep Neural Policies with Stability GuaranteesManuscript (preprint) (Other academic)
    Abstract [en]

    Deep reinforcement learning (DRL) has been successfully used to solve various robotic manipulation tasks. However, most of the existing works do not address the issue of control stability. This is in sharp contrast to the control theory community where the well-established norm is to prove stability whenever a control law is synthesized. What makes traditional stability analysis difficult for DRL are the uninterpretable nature of the neural network policies and unknown system dynamics. In this work, unconditional stability is obtained by deriving an interpretable deep policy structure based on the energy shaping control of Lagrangian systems. Then, stability during physical interaction with an unknown environment is established based on passivity. The result is a stability guaranteeing DRL in a model-free framework that is general enough for contact-rich manipulation tasks. With an experiment on a peg-in-hole task, we demonstrate, to the best of our knowledge, the first DRL with stability guarantee on a real robotic manipulator.

    Download full text (pdf)
    fulltext
  • 5.
    Abdul Khader, Shahbaz
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. ABB Corp Res, Västerås, Sweden..
    Yin, Hang
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Falco, Pietro
    ABB Corp Res, Västerås, Sweden..
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Learning Stable Normalizing-Flow Control for Robotic Manipulation2021In: 2021 IEEE International Conference On Robotics And Automation (ICRA 2021), Institute of Electrical and Electronics Engineers (IEEE) , 2021, p. 1644-1650Conference paper (Refereed)
    Abstract [en]

    Reinforcement Learning (RL) of robotic manipulation skills, despite its impressive successes, stands to benefit from incorporating domain knowledge from control theory. One of the most important properties that is of interest is control stability. Ideally, one would like to achieve stability guarantees while staying within the framework of state-of-the-art deep RL algorithms. Such a solution does not exist in general, especially one that scales to complex manipulation tasks. We contribute towards closing this gap by introducing normalizing-flow control structure, that can be deployed in any latest deep RL algorithms. While stable exploration is not guaranteed, our method is designed to ultimately produce deterministic controllers with provable stability. In addition to demonstrating our method on challenging contact-rich manipulation tasks, we also show that it is possible to achieve considerable exploration efficiency-reduced state space coverage and actuation efforts- without losing learning efficiency.

  • 6.
    Abdul Khader, Shahbaz
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Yin, Hang
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Pietro, Falco
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. KTH, School of Electrical Engineering and Computer Science (EECS), Centres, Centre for Autonomous Systems, CAS.
    Learning Stable Normalizing-Flow Control for Robotic ManipulationManuscript (preprint) (Other academic)
    Abstract [en]

    Reinforcement Learning (RL) of robotic manipu-lation skills, despite its impressive successes, stands to benefitfrom incorporating domain knowledge from control theory. Oneof the most important properties that is of interest is controlstability. Ideally, one would like to achieve stability guaranteeswhile staying within the framework of state-of-the-art deepRL algorithms. Such a solution does not exist in general,especially one that scales to complex manipulation tasks. Wecontribute towards closing this gap by introducing normalizing-flow control structure, that can be deployed in any latest deepRL algorithms. While stable exploration is not guaranteed,our method is designed to ultimately produce deterministiccontrollers with provable stability. In addition to demonstratingour method on challenging contact-rich manipulation tasks, wealso show that it is possible to achieve considerable explorationefficiency–reduced state space coverage and actuation efforts–without losing learning efficiency.

    Download full text (pdf)
    fulltext
  • 7.
    Agrawal, Alekh
    et al.
    Microsoft Research.
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Wu, Cathy
    Massachusetts Institute of Technology.
    et al.,
    The Second Annual Conference on Learning for Dynamics and Control: Editorial2020In: Proceedings of Machine Learning Research, ML Research Press , 2020, Vol. 120Conference paper (Refereed)
  • 8.
    Ahlberg, Sofie
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Axelsson, Agnes
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Speech, Music and Hearing, TMH.
    Yu, Pian
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Shaw Cortez, Wenceslao E.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Gao, Yuan
    Uppsala Univ, Dept Informat Technol, Uppsala, Sweden.;Shenzhen Inst Artificial Intelligence & Robot Soc, Ctr Intelligent Robots, Shenzhen, Peoples R China..
    Ghadirzadeh, Ali
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Castellano, Ginevra
    Uppsala Univ, Dept Informat Technol, Uppsala, Sweden..
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Skantze, Gabriel
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Speech, Music and Hearing, TMH.
    Dimarogonas, Dimos V.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Co-adaptive Human-Robot Cooperation: Summary and Challenges2022In: Unmanned Systems, ISSN 2301-3850, E-ISSN 2301-3869, Vol. 10, no 02, p. 187-203Article in journal (Refereed)
    Abstract [en]

    The work presented here is a culmination of developments within the Swedish project COIN: Co-adaptive human-robot interactive systems, funded by the Swedish Foundation for Strategic Research (SSF), which addresses a unified framework for co-adaptive methodologies in human-robot co-existence. We investigate co-adaptation in the context of safe planning/control, trust, and multi-modal human-robot interactions, and present novel methods that allow humans and robots to adapt to one another and discuss directions for future work.

  • 9.
    Alexanderson, Simon
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Speech, Music and Hearing, TMH.
    Henter, Gustav Eje
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Speech, Music and Hearing, TMH.
    Kucherenko, Taras
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Beskow, Jonas
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Speech, Music and Hearing, TMH.
    Style-Controllable Speech-Driven Gesture Synthesis Using Normalising Flows2020In: Computer graphics forum (Print), ISSN 0167-7055, E-ISSN 1467-8659, Vol. 39, no 2, p. 487-496Article in journal (Refereed)
    Abstract [en]

    Automatic synthesis of realistic gestures promises to transform the fields of animation, avatars and communicative agents. In off-line applications, novel tools can alter the role of an animator to that of a director, who provides only high-level input for the desired animation; a learned network then translates these instructions into an appropriate sequence of body poses. In interactive scenarios, systems for generating natural animations on the fly are key to achieving believable and relatable characters. In this paper we address some of the core issues towards these ends. By adapting a deep learning-based motion synthesis method called MoGlow, we propose a new generative model for generating state-of-the-art realistic speech-driven gesticulation. Owing to the probabilistic nature of the approach, our model can produce a battery of different, yet plausible, gestures given the same input speech signal. Just like humans, this gives a rich natural variation of motion. We additionally demonstrate the ability to exert directorial control over the output style, such as gesture level, speed, symmetry and spacial extent. Such control can be leveraged to convey a desired character personality or mood. We achieve all this without any manual annotation of the data. User studies evaluating upper-body gesticulation confirm that the generated motions are natural and well match the input speech. Our method scores above all prior systems and baselines on these measures, and comes close to the ratings of the original recorded motions. We furthermore find that we can accurately control gesticulation styles without unnecessarily compromising perceived naturalness. Finally, we also demonstrate an application of the same method to full-body gesticulation, including the synthesis of stepping motion and stance.

    Download full text (pdf)
    fulltext
    Download full text (pdf)
    erratum
  • 10.
    Alexanderson, Simon
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Speech, Music and Hearing, TMH.
    Székely, Éva
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Speech, Music and Hearing, TMH.
    Henter, Gustav Eje
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Speech, Music and Hearing, TMH. KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Kucherenko, Taras
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Beskow, Jonas
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Speech, Music and Hearing, TMH.
    Generating coherent spontaneous speech and gesture from text2020In: Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents, IVA 2020, Association for Computing Machinery (ACM) , 2020Conference paper (Refereed)
    Abstract [en]

    Embodied human communication encompasses both verbal (speech) and non-verbal information (e.g., gesture and head movements). Recent advances in machine learning have substantially improved the technologies for generating synthetic versions of both of these types of data: On the speech side, text-to-speech systems are now able to generate highly convincing, spontaneous-sounding speech using unscripted speech audio as the source material. On the motion side, probabilistic motion-generation methods can now synthesise vivid and lifelike speech-driven 3D gesticulation. In this paper, we put these two state-of-the-art technologies together in a coherent fashion for the first time. Concretely, we demonstrate a proof-of-concept system trained on a single-speaker audio and motion-capture dataset, that is able to generate both speech and full-body gestures together from text input. In contrast to previous approaches for joint speech-and-gesture generation, we generate full-body gestures from speech synthesis trained on recordings of spontaneous speech from the same person as the motion-capture data. We illustrate our results by visualising gesture spaces and textspeech-gesture alignments, and through a demonstration video.

  • 11.
    Almeida, Diogo
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Dual-Arm Robotic Manipulation under Uncertainties and Task-Based Redundancy2019Doctoral thesis, comprehensive summary (Other academic)
    Abstract [en]

    Robotic manipulators are mostly employed in industrial environments, where their tasks can be prescribed with little to no uncertainty. This is possible in scenarios where the deployment time of robot workcells is not prohibitive, such as in the automotive industry. In other contexts, however, the time cost of setting up a classical robotic automation workcell is often prohibitive. This is the case with cellphone manufacturing, for example, which is currently mostly executed by human workers. Robotic automation is nevertheless desirable in these human-centric environments, as a robot can automate the most tedious parts of an assembly. To deploy robots in these environments, however, requires an ability to deal with uncertainties and to robustly execute any given task. In this thesis, we discuss two topics related to autonomous robotic manipulation. First, we address parametric uncertainties in manipulation tasks, such as the location of contacts during the execution of an assembly. We propose and experimentally evaluate two methods that rely on force and torque measurements to produce estimates of task related uncertainties: a method for dexterous manipulation under uncertainties which relies on a compliant rotational degree of freedom at the robot's gripper grasp point and exploits contact  with an external surface, and a cooperative manipulation system which is able to identify the kinematics of a two degrees of freedom mechanism. Then, we consider redundancies in dual-arm robotic manipulation. Dual-armed robots offer a large degree of redundancy which can be exploited to ensure a more robust task execution. When executing an assembly task, for instance, robots can freely change the location of the assembly in their workspace without affecting the task execution. We discuss methods that explore these types of redundancies in relative motion tasks in the form of asymmetries in their execution. Finally, we approach the converse problem by presenting a system which is able to balance measured forces and torques at its end-effectors by leveraging relative motion between them, while grasping a rigid tray. This is achieved through discrete sliding of the grasp points, which constitutes a novel application of bimanual dexterous manipulation.

    Download full text (pdf)
    fulltext
  • 12.
    Almeida, Diogo
    et al.
    KTH, School of Computer Science and Communication (CSC), Robotics, perception and learning, RPL.
    Ambrus, Rares
    KTH, School of Computer Science and Communication (CSC), Computer Vision and Active Perception, CVAP.
    Caccamo, Sergio
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Chen, Xi
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Cruciani, Silvia
    Pinto Basto De Carvalho, Joao F
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Haustein, Joshua
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Marzinotto, Alejandro
    KTH, School of Computer Science and Communication (CSC), Computer Vision and Active Perception, CVAP.
    Vina, Francisco
    KTH.
    Karayiannidis, Yiannis
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Ögren, Petter
    KTH, School of Engineering Sciences (SCI), Mathematics (Dept.), Optimization and Systems Theory.
    Jensfelt, Patric
    KTH, School of Computer Science and Communication (CSC), Robotics, perception and learning, RPL.
    Kragic, Danica
    KTH, School of Computer Science and Communication (CSC), Robotics, perception and learning, RPL.
    Team KTH’s Picking Solution for the Amazon Picking Challenge 20162017In: Warehouse Picking Automation Workshop 2017: Solutions, Experience, Learnings and Outlook of the Amazon Robotics Challenge, 2017Conference paper (Other (popular science, discussion, etc.))
    Abstract [en]

    In this work we summarize the solution developed by Team KTH for the Amazon Picking Challenge 2016 in Leipzig, Germany. The competition simulated a warehouse automation scenario and it was divided in two tasks: a picking task where a robot picks items from a shelf and places them in a tote and a stowing task which is the inverse task where the robot picks items from a tote and places them in a shelf. We describe our approach to the problem starting from a high level overview of our system and later delving into details of our perception pipeline and our strategy for manipulation and grasping. The solution was implemented using a Baxter robot equipped with additional sensors.

  • 13.
    Almeida, Diogo
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Ambrus, Rares
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Caccamo, Sergio
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Chen, Xi
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Cruciani, Silvia
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Pinto Basto de Carvalho, Joao Frederico
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Haustein, Joshua
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Marzinotto, Alejandro
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Viña, Francisco
    Karayiannidis, Yiannis
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Ögren, Petter
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Jensfelt, Patric
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Team KTH’s Picking Solution for the Amazon Picking Challenge 20162020In: Advances on Robotic Item Picking: Applications in Warehousing and E-Commerce Fulfillment, Springer Nature , 2020, p. 53-62Chapter in book (Other academic)
    Abstract [en]

    In this chapter we summarize the solution developed by team KTH for the Amazon Picking Challenge 2016 in Leipzig, Germany. The competition, which simulated a warehouse automation scenario, was divided into two parts: a picking task, where the robot picks items from a shelf and places them into a tote, and a stowing task, where the robot picks items from a tote and places them in a shelf. We describe our approach to the problem starting with a high-level overview of the system, delving later into the details of our perception pipeline and strategy for manipulation and grasping. The hardware platform used in our solution consists of a Baxter robot equipped with multiple vision sensors.

  • 14.
    Almeida, Diogo
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Ataer-Cansizoglu, Esra
    Wayfair, Boston, MA 02116, USA.
    Corcodel, Radu
    Mitsubishi Electric Research Labs (MERL), Cambridge, MA 02139, USA.
    Detection, Tracking and 3D Modeling of Objects with Sparse RGB-D SLAM and Interactive Perception2019In: IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2019Conference paper (Refereed)
    Abstract [en]

    We present an interactive perception system that enables an autonomous agent to deliberately interact with its environment and produce 3D object models. Our system verifies object hypotheses through interaction and simultaneously maintains 3D SLAM maps for each rigidly moving object hypothesis in the scene. We rely on depth-based segmentation and a multigroup registration scheme to classify features into various object maps. Our main contribution lies in the employment of a novel segment classification scheme that allows the system to handle incorrect object hypotheses, common in cluttered environments due to touching objects or occlusion. We start with a single map and initiate further object maps based on the outcome of depth segment classification. For each existing map, we select a segment to interact with and execute a manipulation primitive with the goal of disturbing it. If the resulting set of depth segments has at least one segment that did not follow the dominant motion pattern of its respective map, we split the map, thus yielding updated object hypotheses. We show qualitative results with a Fetch manipulator and objects of various shapes, which showcase the viability of the method for identifying and modelling multiple objects through repeated interactions.

    Download full text (pdf)
    fulltext
  • 15.
    Almeida, Diogo
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Karayiannidis, Yiannis
    A Lyapunov-Based Approach to Exploit Asymmetries in Robotic Dual-Arm Task Resolution2019In: 58th IEEE Conference on Decision and Control (CDC), 2019Conference paper (Refereed)
    Abstract [en]

    Dual-arm manipulation tasks can be prescribed to a robotic system in terms of desired absolute and relative motion of the robot’s end-effectors. These can represent, e.g., jointly carrying a rigid object or performing an assembly task. When both types of motion are to be executed concurrently, the symmetric distribution of the relative motion between arms prevents task conflicts. Conversely, an asymmetric solution to the relative motion task will result in conflicts with the absolute task. In this work, we address the problem of designing a control law for the absolute motion task together with updating the distribution of the relative task among arms. Through a set of numerical results, we contrast our approach with the classical symmetric distribution of the relative motion task to illustrate the advantages of our method.

    Download full text (pdf)
    fulltext
  • 16.
    Almeida, Diogo
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Karayiannidis, Yiannis
    Chalmers Univ Technol, Dept Elect Engn, S-41296 Gothenburg, Sweden..
    Asymmetric Dual-Arm Task Execution Using an Extended Relative Jacobian2022In: Robotics Research: 19th International Symposium  ISRR / [ed] Asfour, T Yoshida, E Park, J Christensen, H Khatib, O, Springer Nature , 2022, Vol. 20, p. 18-34Conference paper (Refereed)
    Abstract [en]

    Coordinated dual-arm manipulation tasks can be broadly characterized as possessing absolute and relative motion components. Relative motion tasks, in particular, are inherently redundant in the way they can be distributed between end-effectors. In this work, we analyse cooperative manipulation in terms of the asymmetric resolution of relative motion tasks. We discuss how existing approaches enable the asymmetric execution of a relative motion task, and show how an asymmetric relative motion space can be defined. We leverage this result to propose an extended relative Jacobian to model the cooperative system, which allows a user to set a concrete degree of asymmetry in the task execution. This is achieved without the need for prescribing an absolute motion target. Instead, the absolute motion remains available as a functional redundancy to the system. We illustrate the properties of our proposed Jacobian through numerical simulations of a novel differential Inverse Kinematics algorithm.

  • 17.
    Almeida, Diogo
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Karayiannidis, Yiannis
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Asymmetric Dual-Arm Task Execution using an Extended Relative Jacobian2019In: The International Symposium on Robotics Research, 2019Conference paper (Refereed)
    Abstract [en]

    Coordinated dual-arm manipulation tasks can be broadly characterized as possessing absolute and relative motion components. Relative motion tasks, in particular, are inherently redundant in the way they can be distributed between end-effectors. In this work, we analyse cooperative manipulation in terms of the asymmetric resolution of relative motion tasks. We discuss how existing approaches enable the asymmetric execution of a relative motion task, and show how an asymmetric relative motion space can be defined. We leverage this result to propose an extended relative Jacobian to model the cooperative system, which allows a user to set a concrete degree of asymmetry in the task execution. This is achieved without the need for prescribing an absolute motion target. Instead, the absolute motion remains available as a functional redundancy to the system. We illustrate the properties of our proposed Jacobian through numerical simulations of a novel differential Inverse Kinematics algorithm.

    Download full text (pdf)
    fulltext
  • 18.
    Almeida, Diogo
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Karayiannidis, Yiannis
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. Dept. of Electrical Eng., Chalmers University of Technology.
    Cooperative Manipulation and Identification of a 2-DOF Articulated Object by a Dual-Arm Robot2018In: 2018 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA) / [ed] IEEE, 2018, p. 5445-5451Conference paper (Refereed)
    Abstract [en]

    In this work, we address the dual-arm manipula-tion of a two degrees-of-freedom articulated object that consistsof two rigid links. This can include a linkage constrainedalong two motion directions, or two objects in contact, wherethe contact imposes motion constraints. We formulate theproblem as a cooperative task, which allows the employment ofcoordinated task space frameworks, thus enabling redundancyexploitation by adjusting how the task is shared by the robotarms. In addition, we propose a method that can estimate thejoint location and the direction of the degrees-of-freedom, basedon the contact forces and the motion constraints imposed bythe object. Experimental results demonstrate the performanceof the system in its ability to estimate the two degrees of freedomindependently or simultaneously.

    Download full text (pdf)
    fulltext
  • 19.
    Almeida, Diogo
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Karayiannidis, Yiannis
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. Chalmers University of Technology.
    Folding Assembly by Means of Dual-Arm Robotic Manipulation2016In: 2016 IEEE International Conference on Robotics and Automation, IEEE conference proceedings, 2016, p. 3987-3993Conference paper (Refereed)
    Abstract [en]

    In this paper, we consider folding assembly as an assembly primitive suitable for dual-arm robotic assembly, that can be integrated in a higher level assembly strategy. The system composed by two pieces in contact is modelled as an articulated object, connected by a prismatic-revolute joint. Different grasping scenarios were considered in order to model the system, and a simple controller based on feedback linearisation is proposed, using force torque measurements to compute the contact point kinematics. The folding assembly controller has been experimentally tested with two sample parts, in order to showcase folding assembly as a viable assembly primitive.

    Download full text (pdf)
    fulltext
  • 20.
    Almeida, João Tiago
    et al.
    KTH.
    Leite, Iolanda
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Yadollahi, Elmira
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Would you help me?: Linking robot's perspective-taking to human prosocial behavior2023In: HRI 2023: Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction, Association for Computing Machinery (ACM) , 2023, p. 388-397Conference paper (Refereed)
    Abstract [en]

    Despite the growing literature on human attitudes toward robots, particularly prosocial behavior, little is known about how robots' perspective-taking, the capacity to perceive and understand the world from other viewpoints, could infuence such attitudes and perceptions of the robot. To make robots and AI more autonomous and self-aware, more researchers have focused on developing cognitive skills such as perspective-taking and theory of mind in robots and AI. The present study investigated whether a robot's perspectivetaking choices could infuence the occurrence and extent of exhibiting prosocial behavior toward the robot.We designed an interaction consisting of a perspective-taking task, where we manipulated how the robot instructs the human to fnd objects by changing its frame of reference and measured the human's exhibition of prosocial behavior toward the robot. In a between-subject study (N=70), we compared the robot's egocentric and addressee-centric instructions against a control condition, where the robot's instructions were object-centric. Participants' prosocial behavior toward the robot was measured using a voluntary data collection session. Our results imply that the occurrence and extent of prosocial behavior toward the robot were signifcantly infuenced by the robot's visuospatial perspective-taking behavior. Furthermore, we observed, through questionnaire responses, that the robot's choice of perspectivetaking could potentially infuence the humans' perspective choices, were they to reciprocate the instructions to the robot.

  • 21.
    Andersen, Pia Haubro
    et al.
    Swedish Univ Agr Sci, Dept Anat Physiol & Biochem, SE-75007 Uppsala, Sweden..
    Broomé, Sofia
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Rashid, Maheen
    Univ Calif Davis, Dept Comp Sci, Davis, CA 95616 USA..
    Lundblad, Johan
    Swedish Univ Agr Sci, Dept Anat Physiol & Biochem, SE-75007 Uppsala, Sweden..
    Ask, Katrina
    Swedish Univ Agr Sci, Dept Anat Physiol & Biochem, SE-75007 Uppsala, Sweden..
    Li, Zhenghong
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. SUNY Stony Brook, Dept Comp Sci, Stony Brook, NY 11794 USA..
    Hernlund, Elin
    Swedish Univ Agr Sci, Dept Anat Physiol & Biochem, SE-75007 Uppsala, Sweden..
    Rhodin, Marie
    Swedish Univ Agr Sci, Dept Anat Physiol & Biochem, SE-75007 Uppsala, Sweden..
    Kjellström, Hedvig
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Towards Machine Recognition of Facial Expressions of Pain in Horses2021In: Animals, E-ISSN 2076-2615, Vol. 11, no 6, article id 1643Article, review/survey (Refereed)
    Abstract [en]

    Simple Summary Facial activity can convey valid information about the experience of pain in a horse. However, scoring of pain in horses based on facial activity is still in its infancy and accurate scoring can only be performed by trained assessors. Pain in humans can now be recognized reliably from video footage of faces, using computer vision and machine learning. We examine the hurdles in applying these technologies to horses and suggest two general approaches to automatic horse pain recognition. The first approach involves automatically detecting objectively defined facial expression aspects that do not involve any human judgment of what the expression "means". Automated classification of pain expressions can then be done according to a rule-based system since the facial expression aspects are defined with this information in mind. The other involves training very flexible machine learning methods with raw videos of horses with known true pain status. The upside of this approach is that the system has access to all the information in the video without engineered intermediate methods that have filtered out most of the variation. However, a large challenge is that large datasets with reliable pain annotation are required. We have obtained promising results from both approaches. Automated recognition of human facial expressions of pain and emotions is to a certain degree a solved problem, using approaches based on computer vision and machine learning. However, the application of such methods to horses has proven difficult. Major barriers are the lack of sufficiently large, annotated databases for horses and difficulties in obtaining correct classifications of pain because horses are non-verbal. This review describes our work to overcome these barriers, using two different approaches. One involves the use of a manual, but relatively objective, classification system for facial activity (Facial Action Coding System), where data are analyzed for pain expressions after coding using machine learning principles. We have devised tools that can aid manual labeling by identifying the faces and facial keypoints of horses. This approach provides promising results in the automated recognition of facial action units from images. The second approach, recurrent neural network end-to-end learning, requires less extraction of features and representations from the video but instead depends on large volumes of video data with ground truth. Our preliminary results suggest clearly that dynamics are important for pain recognition and show that combinations of recurrent neural networks can classify experimental pain in a small number of horses better than human raters.

  • 22.
    Andersson, Klas
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Improving Fixed Wing UAV Endurance, by Cooperative Autonomous Soaring2021Doctoral thesis, comprehensive summary (Other academic)
    Abstract [en]

    The ever-expanding use and development of smaller UAVs (Unmanned Aerial Vehicles) has highlighted an increasing demand for extended range and endurance for this type of vehicles. 

    In this thesis, the development of a concept and system for autonomous soaring of cooperating unmanned aerial vehicles is presented. The purpose of the developed system is to extend endurance by harvesting energy available in the atmosphere in the form of thermal updrafts, in a similar way that some birds and manned gliders do. By using this “free” energy, considerable improvements in maximum achievable endurance can be realized under a wide variety of atmospherical and weather conditions. 

    The work included theoretical analysis, simulations, and finally flight test- ing of the soaring controller and the system. The system was initially devel- oped as a single-vehicle concept and thereafter extended into a system consist- ing of two cooperating gliders. The purpose of the extension to cooperation, was to further improve the performance of the system by increasing the ability to locate the rising air of thermal updrafts. 

    The theoretical analysis proved the soaring algorithm’s thermal centering controller to be stable. The trials showed the concept of autonomous soaring to function as expected from the simulations. Further it revealed that, by applying the idea, extensive performance gains can be achieved under a fairly wide variety of conditions. 

    The cooperative soaring, likewise, functioned as anticipated and the glid- ers found, cooperated, and climbed together in updrafts. This represents the first and presumably only time cooperative autonomous soaring in this way, has been successfully demonstrated in flight. To draw further conclusions on the advantages of cooperative soaring additional flight trials would, however, be beneficial. 

    Possible issues and limitations were highlighted during the trials and a number of potential improvements were identified. 

    As a part of the work, trials were conducted to verify the viability to implement the system into “real world” operational scenarios. As a proof of concept this was done by tasking the autonomous gliders to perform data/communications relay missions for other UAV systems sending imagery to the ground-station from beyond line of sight (BLOS). The outcome of the trials was positive and the concept appeared to be well suited for these types of missions. The comms relay system was further developed into a hybrid system where the optimal location concerning relay performance was autonomously sought out, after-which the attentiveness then switched to autonomous thermal soaring in the vicinity of this ideal relay position. The hybrid system was tested in simulation and partially flight tested. 

    Download full text (pdf)
    fulltext
  • 23.
    Antonova, Rika
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. KTH, School of Electrical Engineering and Computer Science (EECS), Centres, Centre for Autonomous Systems, CAS.
    Transfer-Aware Kernels, Priors and Latent Spaces from Simulation to Real Robots2020Doctoral thesis, comprehensive summary (Other academic)
    Abstract [en]

    Consider challenging sim-to-real cases lacking high-fidelity simulators and allowing only 10-20 hardware trials. This work shows that even imprecise simulation can be beneficial if used to build transfer-aware representations.

    First, the thesis introduces an informed kernel that embeds the space of simulated trajectories into a lower-dimensional space of latent paths. It uses a sequential variational autoencoder (sVAE) to handle large-scale training from simulated data. Its modular design enables quick adaptation when used for Bayesian optimization (BO) on hardware. The thesis and the included publications demonstrate that this approach works for different areas of robotics: locomotion and manipulation. Furthermore, a variant of BO that ensures recovery from negative transfer when using corrupted kernels is introduced. An application to task-oriented grasping validates its performance on hardware.

    For the case of parametric learning, simulators can serve as priors or regularizers. This work describes how to use simulation to regularize a VAE's decoder to bind the VAE's latent space to simulator parameter posterior. With that, training on a small number of real trajectories can quickly shift the posterior to reflect reality. The included publication demonstrates that this approach can also help reinforcement learning (RL) quickly overcome the sim-to-real gap on a manipulation task on hardware.

    A longer-term vision is to shape latent spaces without needing to mandate a particular simulation scenario. A first step is to learn general relations that hold on sequences of states from a set of related domains. This work introduces a unifying mathematical formulation for learning independent analytic relations. Relations are learned from source domains, then used to help structure the latent space when learning on target domains. This formulation enables a more general, flexible and principled way of shaping the latent space. It formalizes the notion of learning independent relations, without imposing restrictive simplifying assumptions or requiring domain-specific information. This work presents mathematical properties, concrete algorithms and experimental validation of successful learning and transfer of latent relations.

    Download full text (pdf)
    phd_thesis_rika_antonova
  • 24.
    Antonova, Rika
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Centres, Centre for Autonomous Systems, CAS. KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Kokic, Mia
    KTH, School of Electrical Engineering and Computer Science (EECS), Centres, Centre for Autonomous Systems, CAS. KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Stork, Johannes A.
    KTH, School of Electrical Engineering and Computer Science (EECS), Centres, Centre for Autonomous Systems, CAS. KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Centres, Centre for Autonomous Systems, CAS. KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation2018In: Proceedings of The 2nd Conference on Robot Learning, PMLR 87, 2018, p. 641-650Conference paper (Refereed)
    Abstract [en]

    We develop an approach that benefits from large simulated datasets and takes full advantage of the limited online data that is most relevant. We propose a variant of Bayesian optimization that alternates between using informed and uninformed kernels. With this Bernoulli Alternation Kernel we ensure that discrepancies between simulation and reality do not hinder adapting robot control policies online. The proposed approach is applied to a challenging real-world problem of task-oriented grasping with novel objects. Our further contribution is a neural network architecture and training pipeline that use experience from grasping objects in simulation to learn grasp stability scores. We learn task scores from a labeled dataset with a convolutional network, which is used to construct an informed kernel for our variant of Bayesian optimization. Experiments on an ABB Yumi robot with real sensor data demonstrate success of our approach, despite the challenge of fulfilling task requirements and high uncertainty over physical properties of objects.

    Download full text (pdf)
    fulltext
  • 25.
    Antonova, Rika
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Maydanskiy, Maksim
    Uppsala University.
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Devlin, Sam
    Microsoft Research.
    Hofmann, Katja
    Microsoft Research.
    Analytic Manifold Learning: Unifying and Evaluating Representations for Continuous ControlManuscript (preprint) (Other academic)
    Abstract [en]

    We address the problem of learning reusable state representations from streaming high-dimensional observations. This is important for areas like Reinforcement Learning (RL), which yields non-stationary data distributions during training. We make two key contributions. First, we propose an evaluation suite that measures alignment between latent and true low-dimensional states. We benchmark several widely used unsupervised learning approaches. This uncovers the strengths and limitations of existing approaches that impose additional constraints/objectives on the latent space. Our second contribution is a unifying mathematical formulation for learning latent relations. We learn analytic relations on source domains, then use these relations to help structure the latent space when learning on target domains. This formulation enables a more general, flexible and principled way of shaping the latent space. It formalizes the notion of learning independent relations, without imposing restrictive simplifying assumptions or requiring domain-specific information. We present mathematical properties, concrete algorithms for implementation and experimental validation of successful learning and transfer of latent relations.

  • 26.
    Antonova, Rika
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. KTH, School of Electrical Engineering and Computer Science (EECS), Centres, Centre for Autonomous Systems, CAS.
    Rai, Akshara
    Facebook AI Research.
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. KTH, School of Electrical Engineering and Computer Science (EECS), Centres, Centre for Autonomous Systems, CAS.
    How to Sim2Real with Gaussian Processes: Prior Mean versus Kernels as Priors2020In: 2nd Workshop on Closing the Reality Gap in Sim2Real Transfer for Robotics. RSS, 2020. https://sim2real.github.io, 2020Conference paper (Other academic)
    Abstract [en]

    Gaussian Processes (GPs) have been widely used in robotics as models, and more recently as key structures in active learning algorithms, such as Bayesian optimization. GPs consist of two main components: the mean function and the kernel. Specifying a prior mean function has been a common way to incorporate prior knowledge. When a prior mean function could not be constructed manually, the next default has been to incorporate prior (simulated) observations into a GP as 'fake' data. Then, this GP would be used to further learn from true data on the target (real) domain. We argue that embedding prior knowledge into GP kernels instead provides a more flexible way to capture simulation-based information. We give examples of recent works that demonstrate the wide applicability of such kernel-centric treatment when using GPs as part of Bayesian optimization. We also provide discussion that helps to build intuition for why such 'kernels as priors' view is beneficial.

    Download full text (pdf)
    fulltext
  • 27.
    Antonova, Rika
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Rai, Akshara
    Facebook AI Research.
    Li, Tianyu
    Facebook AI Research.
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Bayesian optimization in variational latent spaces with dynamic compression2020In: Proceedings of Machine Learning Research: Volume 100: Proceedings of the 3rd Annual Conference on Robot Learning (CoRL), 2020, Vol. 100, p. 456-465Conference paper (Refereed)
    Abstract [en]

    Data-efficiency is crucial for autonomous robots to adapt to new tasks and environments. In this work, we focus on robotics problems with a budget of only 10-20 trials. This is a very challenging setting even for data- efficient approaches like Bayesian optimization (BO), especially when optimizing higher-dimensional controllers. Previous work extracted expert-designed low-dimensional features from simulation trajectories to construct informed kernels and run ultra sample-efficient BO on hardware. We remove the need for expert-designed features by proposing a model and architecture for a sequential variational autoencoder that embeds the space of simulated trajectories into a lower-dimensional space of latent paths in an unsupervised way. We further compress the search space for BO by reducing exploration in parts of the state space that are undesirable, without requiring explicit constraints on controller parameters. We validate our approach with hardware experiments on a Daisy hexapod robot and an ABB Yumi manipulator. We also present simulation experiments with further comparisons to several baselines on Daisy and two manipulators. Our experiments indicate the proposed trajectory-based kernel with dynamic compression can offer ultra data-efficient optimization.

  • 28.
    Antonova, Rika
    et al.
    Stanford University, Stanford, CA, USA.
    Varava, Anastasiia
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Shi, Peiyang
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Pinto Basto de Carvalho, Joao Frederico
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Sequential Topological Representations for Predictive Models of Deformable Objects2021In: Proceedings of the 3rd Conference on Learning for Dynamics and Control, L4DC 2021, ML Research Press , 2021, p. 348-360Conference paper (Refereed)
    Abstract [en]

    Deformable objects present a formidable challenge for robotic manipulation due to the lack of canonical low-dimensional representations and the difficulty of capturing, predicting, and controlling such objects. We construct compact topological representations to capture the state of highly deformable objects that are topologically nontrivial. We develop an approach that tracks the evolution of this topological state through time. Under several mild assumptions, we prove that the topology of the scene and its evolution can be recovered from point clouds representing the scene. Our further contribution is a method to learn predictive models that take a sequence of past point cloud observations as input and predict a sequence of topological states, conditioned on target/future control actions. Our experiments with highly deformable objects in simulation show that the proposed multistep predictive models yield more precise results than those obtained from computational topology libraries. These models can leverage patterns inferred across various objects and offer fast multistep predictions suitable for real-time applications.

  • 29.
    Arndt, Karol
    et al.
    Aalto Univ, Espoo, Finland..
    Hazara, Murtaza
    Aalto Univ, Espoo, Finland.;Katholieke Univ Leuven, Dept Mech Engn, Leuven, Belgium.;Flanders Make, Robot Core Lab, Lommel, Belgium..
    Ghadirzadeh, Ali
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. Aalto Univ, Espoo, Finland.
    Kyrki, Ville
    Aalto Univ, Espoo, Finland..
    Meta Reinforcement Learning for Sim-to-real Domain Adaptation2020In: 2020 IEEE International Conference On Robotics And Automation (ICRA), IEEE , 2020, p. 2725-2731Conference paper (Refereed)
    Abstract [en]

    Modern reinforcement learning methods suffer from low sample efficiency and unsafe exploration, making it infeasible to train robotic policies entirely on real hardware. In this work, we propose to address the problem of sim-to-real domain transfer by using meta learning to train a policy that can adapt to a variety of dynamic conditions, and using a task-specific trajectory generation model to provide an action space that facilitates quick exploration. We evaluate the method by performing domain adaptation in simulation and analyzing the structure of the latent space during adaptation. We then deploy this policy on a KUKA LBR 4+ robot and evaluate its performance on a task of hitting a hockey puck to a target. Our method shows more consistent and stable domain adaptation than the baseline, resulting in better overall performance.

  • 30.
    Arnekvist, Isac
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Transfer Learning using low-dimensional Representations in Reinforcement Learning2020Licentiate thesis, comprehensive summary (Other academic)
    Abstract [en]

    Successful learning of behaviors in Reinforcement Learning (RL) are often learned tabula rasa, requiring many observations and interactions in the environment. Performing this outside of a simulator, in the real world, often becomes infeasible due to the large amount of interactions needed. This has motivated the use of Transfer Learning for Reinforcement Learning, where learning is accelerated by using experiences from previous learning in related tasks. In this thesis, I explore how we can transfer from a simple single-object pushing policy, to a wide array of non-prehensile rearrangement problems. I then explain how we can model task differences using a low-dimensional latent variable representation to make adaption to novel tasks efficient. Lastly, the dependence of accurate function approximation is sometimes problematic, especially in RL, where statistics of target variables are not known a priori. I present observations, along with explanations, that small target variances along with momentum optimization of ReLU-activated neural network parameters leads to dying ReLU.

    Download full text (pdf)
    fulltext
  • 31.
    Arnekvist, Isac
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Stork, Johannes A.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. Center for Applied Autonomous Sensor Systems, Örebro University, Sweden.
    Vpe: Variational policy embedding for transfer reinforcement learning2019In: 2019 International Conference on Robotics And Automation (ICRA), Institute of Electrical and Electronics Engineers (IEEE), 2019, p. 36-42Conference paper (Refereed)
    Abstract [en]

    Reinforcement Learning methods are capable of solving complex problems, but resulting policies might perform poorly in environments that are even slightly different. In robotics especially, training and deployment conditions often vary and data collection is expensive, making retraining undesirable. Simulation training allows for feasible training times, but on the other hand suffer from a reality-gap when applied in real-world settings. This raises the need of efficient adaptation of policies acting in new environments. We consider the problem of transferring knowledge within a family of similar Markov decision processes. We assume that Q-functions are generated by some low-dimensional latent variable. Given such a Q-function, we can find a master policy that can adapt given different values of this latent variable. Our method learns both the generative mapping and an approximate posterior of the latent variables, enabling identification of policies for new tasks by searching only in the latent space, rather than the space of all policies. The low-dimensional space, and master policy found by our method enables policies to quickly adapt to new environments. We demonstrate the method on both a pendulum swing-up task in simulation, and for simulation-to-real transfer on a pushing task.

    Download full text (pdf)
    fulltext
  • 32.
    Arnekvist, Isac
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Pinto Basto de Carvalho, Joao Frederico
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. KTH, School of Electrical Engineering and Computer Science (EECS), Centres, Centre for Autonomous Systems, CAS.
    Stork, Johannes Andreas
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. KTH, School of Electrical Engineering and Computer Science (EECS), Centres, Centre for Autonomous Systems, CAS.
    The effect of target normalization and momentum on dying reluManuscript (preprint) (Other academic)
  • 33.
    Arriola-Rios, Veronica E.
    et al.
    Univ Nacl Autonoma Mexico, UNAM, Fac Sci, Dept Math, Mexico City, DF, Mexico..
    Guler, Puren
    Örebro Univ, Ctr Appl Autonomous Sensor Syst, Autonomous Mobile Manipulat Lab, Örebro, Sweden..
    Ficuciello, Fanny
    Univ Naples Federico II, PRISMA Lab, Dept Elect Engn & Informat Technol, Naples, Italy..
    Kragic, Danica
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL. KTH, School of Electrical Engineering and Computer Science (EECS), Centres, Centre for Autonomous Systems, CAS.
    Siciliano, Bruno
    Univ Naples Federico II, PRISMA Lab, Dept Elect Engn & Informat Technol, Naples, Italy..
    Wyatt, Jeremy L.
    Univ Birmingham, Sch Comp Sci, Birmingham, W Midlands, England..
    Modeling of Deformable Objects for Robotic Manipulation: A Tutorial and Review2020In: Frontiers in Robotics and AI, E-ISSN 2296-9144, Vol. 7, article id 82Article, review/survey (Refereed)
    Abstract [en]

    Manipulation of deformable objects has given rise to an important set of open problems in the field of robotics. Application areas include robotic surgery, household robotics, manufacturing, logistics, and agriculture, to name a few. Related research problems span modeling and estimation of an object's shape, estimation of an object's material properties, such as elasticity and plasticity, object tracking and state estimation during manipulation, and manipulation planning and control. In this survey article, we start by providing a tutorial on foundational aspects of models of shape and shape dynamics. We then use this as the basis for a review of existing work on learning and estimation of these models and on motion planning and control to achieve desired deformations. We also discuss potential future lines of work.

  • 34.
    Athanasiadis, Ioannis
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS).
    Bore, Nils
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Folkesson, John
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Underwater Image Classification via Multiview-based Auxiliary Learning2022In: 2022 OCEANS HAMPTON ROADS, Institute of Electrical and Electronics Engineers (IEEE) , 2022Conference paper (Refereed)
    Abstract [en]

    In this study, we considered the problem of underwater image classification in the context of underwater pipeline inspection. This task is particularly more difficult than many classification tasks due to the subtle differences between classes, the inherent imbalance in the occurrence of the classes and the variability caused by water clarity. We experimented with both transformer and CNN architectures while for the latter, we also employed auxiliary learning as well as capitalizing on the multiview aspect on the underwater pipeline capturing setup. Additionally, we also adopted a DNN interpretability approach to locate the regions relevant to the predictions using solely image-level annotations. Finally, the extracted explainability cues were integrated into the training process with the purpose of producing more robust predictions and more complete explanations.

  • 35.
    Baldassarre, Federico
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Structured Representations for Explainable Deep Learning2023Doctoral thesis, comprehensive summary (Other academic)
    Abstract [en]

    Deep learning has revolutionized scientific research and is being used to take decisions in increasingly complex scenarios. With growing power comes a growing demand for transparency and interpretability. The field of Explainable AI aims to provide explanations for the predictions of AI systems. The state of the art of AI explainability, however, is far from satisfactory. For example, in Computer Vision, the most prominent post-hoc explanation methods produce pixel-wise heatmaps over the input domain, which are meant to visualize the importance of individual pixels of an image or video. We argue that such dense attribution maps are poorly interpretable to non-expert users because of the domain in which explanations are formed - we may recognize shapes in a heatmap but they are just blobs of pixels. In fact, the input domain is closer to the raw data of digital cameras than to the interpretable structures that humans use to communicate, e.g. objects or concepts. In this thesis, we propose to move beyond dense feature attributions by adopting structured internal representations as a more interpretable explanation domain. Conceptually, our approach splits a Deep Learning model in two: the perception step that takes as input dense representations and the reasoning step that learns to perform the task at hand. At the interface between the two are structured representations that correspond to well-defined objects, entities, and concepts. These representations serve as the interpretable domain for explaining the predictions of the model, allowing us to move towards more meaningful and informative explanations. The proposed approach introduces several challenges, such as how to obtain structured representations, how to use them for downstream tasks, and how to evaluate the resulting explanations. The works included in this thesis address these questions, validating the approach and providing concrete contributions to the field. For the perception step, we investigate how to obtain structured representations from dense representations, whether by manually designing them using domain knowledge or by learning them from data without supervision. For the reasoning step, we investigate how to use structured representations for downstream tasks, from Biology to Computer Vision, and how to evaluate the learned representations. For the explanation step, we investigate how to explain the predictions of models that operate in a structured domain, and how to evaluate the resulting explanations. Overall, we hope that this work inspires further research in Explainable AI and helps bridge the gap between high-performing Deep Learning models and the need for transparency and interpretability in real-world applications.

    Download full text (pdf)
    kappa
  • 36.
    Baldassarre, Federico
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Azizpour, Hossein
    KTH, School of Computer Science and Communication (CSC), Computer Vision and Active Perception, CVAP.
    Explainability Techniques for Graph Convolutional Networks2019Conference paper (Refereed)
    Abstract [en]

    Graph Networks are used to make decisions in potentially complex scenarios but it is usually not obvious how or why they made them. In this work, we study the explainability of Graph Network decisions using two main classes of techniques, gradient-based and decomposition-based, on a toy dataset and a chemistry task. Our study sets the ground for future development as well as application to real-world problems.

    Download full text (pdf)
    fulltext
  • 37.
    Baldassarre, Federico
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Azizpour, Hossein
    Towards Self-Supervised Learning of Global and Object-Centric Representations2022Conference paper (Refereed)
    Abstract [en]

    Self-supervision allows learning meaningful representations of natural images, which usually contain one central object. How well does it transfer to multi-entity scenes? We discuss key aspects of learning structured object-centric representations with self-supervision and validate our insights through several experiments on the CLEVR dataset. Regarding the architecture, we confirm the importance of competition for attention-based object discovery, where each image patch is exclusively attended by one object. For training, we show that contrastive losses equipped with matching can be applied directly in a latent space, avoiding pixel-based reconstruction. However, such an optimization objective is sensitive to false negatives (recurring objects) and false positives (matching errors). Careful consideration is thus required around data augmentation and negative sample selection.

  • 38.
    Baldassarre, Federico
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Debard, Quentin
    Pontiveros, Gonzalo Fiz
    Wijaya, Tri Kurniawan
    Quantitative Metrics for Evaluating Explanations of Video DeepFake Detectors2022Conference paper (Refereed)
    Abstract [en]

    The proliferation of DeepFake technology is a rising challenge in today’s society, owing to more powerful and accessible generation methods. To counter this, the research community has developed detectors of ever-increasing accuracy. However, the ability to explain the decisions of such models to users lags behind performance and is considered an accessory in large-scale benchmarks, despite being a crucial requirement for the correct deployment of automated tools for moderation and censorship. We attribute the issue to the reliance on qualitative comparisons and the lack of established metrics. We describe a simple set of metrics to evaluate the visual quality and informativeness of explanations of video DeepFake classifiers from a human-centric perspective. With these metrics, we compare common approaches to improve explanation quality and discuss their effect on both classification and explanation performance on the recent DFDC and DFD datasets.

  • 39.
    Baldassarre, Federico
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    El-Nouby, Alaaeldin
    Jégou, Hervé
    Variable Rate Allocation for Vector-Quantized Autoencoders2023In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Institute of Electrical and Electronics Engineers (IEEE) , 2023Conference paper (Refereed)
    Abstract [en]

    Vector-quantized autoencoders have recently gained interest in image compression, generation and self-supervised learning. However, as a neural compression method, they lack the possibility to allocate a variable number of bits to each image location, e.g. according to the semantic content or local saliency. In this paper, we address this limitation in a simple yet effective way. We adopt a product quantizer (PQ) that produces a set of discrete codes for each image patch rather than a single index. This PQ-autoencoder is trained end-to-end with a structured dropout that selectively masks a variable number of codes at each location. These mechanisms force the decoder to reconstruct the original image based on partial information and allow us to control the local rate. The resulting model can compress images on a wide range of operating points of the rate-distortion curve and can be paired with any external method for saliency estimation to control the compression rate at a local level. We demonstrate the effectiveness of our approach on the popular Kodak and ImageNet datasets by measuring both distortion and perceptual quality metrics.

  • 40.
    Baldassarre, Federico
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Menéndez Hurtado, David
    KTH, School of Engineering Sciences (SCI), Applied Physics, Biophysics. KTH, Centres, Science for Life Laboratory, SciLifeLab.
    Elofsson, Arne
    Azizpour, Hossein
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    GraphQA: Protein Model Quality Assessment using Graph Convolutional Networks2020In: Bioinformatics, ISSN 1367-4803, E-ISSN 1367-4811, Vol. 37, no 3, p. 360-366Article in journal (Refereed)
    Abstract [en]

    Motivation

    Proteins are ubiquitous molecules whose function in biological processes is determined by their 3D structure. Experimental identification of a protein’s structure can be time-consuming, prohibitively expensive, and not always possible. Alternatively, protein folding can be modeled using computational methods, which however are not guaranteed to always produce optimal results.

    GraphQA is a graph-based method to estimate the quality of protein models, that possesses favorable properties such as representation learning, explicit modeling of both sequential and 3D structure, geometric invariance, and computational efficiency.

    Results

    GraphQA performs similarly to state-of-the-art methods despite using a relatively low number of input features. In addition, the graph network structure provides an improvement over the architecture used in ProQ4 operating on the same input features. Finally, the individual contributions of GraphQA components are carefully evaluated.

    Availability and implementation

    PyTorch implementation, datasets, experiments, and link to an evaluation server are available through this GitHub repository: github.com/baldassarreFe/graphqa

    Supplementary information

    Supplementary material is available at Bioinformatics online.

  • 41.
    Baldassarre, Federico
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Smith, Kevin
    KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Computational Science and Technology (CST).
    Sullivan, Josephine
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Azizpour, Hossein
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Explanation-Based Weakly-Supervised Learning of Visual Relations with Graph Networks2020In: Proceedings, Part XXVIII Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Springer Nature , 2020, p. 612-630Conference paper (Refereed)
    Abstract [en]

    Visual relationship detection is fundamental for holistic image understanding. However, the localization and classification of (subject, predicate, object) triplets remain challenging tasks, due to the combinatorial explosion of possible relationships, their long-tailed distribution in natural images, and an expensive annotation process. This paper introduces a novel weakly-supervised method for visual relationship detection that relies on minimal image-level predicate labels. A graph neural network is trained to classify predicates in images from a graph representation of detected objects, implicitly encoding an inductive bias for pairwise relations. We then frame relationship detection as the explanation of such a predicate classifier, i.e. we obtain a complete relation by recovering the subject and object of a predicted predicate. We present results comparable to recent fully- and weakly-supervised methods on three diverse and challenging datasets: HICO-DET for human-object interaction, Visual Relationship Detection for generic object-to-object relations, and UnRel for unusual triplets; demonstrating robustness to non-comprehensive annotations and good few-shot generalization.

    Download full text (pdf)
    fulltext
  • 42.
    Baldvinsson, Jon R.
    et al.
    KTH. Ericsson Res, Stockholm, Sweden..
    Ganjalizadeh, Milad
    KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Communication Systems, CoS. Ericsson Res, Stockholm, Sweden..
    AlAbbasi, Abdulrahman
    Ericsson Res, Stockholm, Sweden..
    Björkman, Mårten
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Payberah, Amir H.
    IL-GAN: Rare Sample Generation via Incremental Learning in GANs2022In: 2022 IEEE GLOBAL COMMUNICATIONS CONFERENCE (GLOBECOM 2022), Institute of Electrical and Electronics Engineers (IEEE) , 2022, p. 621-626Conference paper (Refereed)
    Abstract [en]

    Industry 4.0 imposes strict requirements on the fifth generation of wireless systems (5G), such as high reliability, high availability, and low latency. Guaranteeing such requirements implies that system failures should occur with an extremely low probability. However, some applications (e.g., training a reinforcement learning algorithm to operate in highly reliable systems or rare event simulations) require access to a broad range of observed failures and extreme values, preferably in a short time. In this paper, we propose IL-GAN, an alternative training framework for generative adversarial networks (GANs), which leverages incremental learning (IL) to enable the generation to learn the tail behavior of the distribution using only a few samples. We validate the proposed IL-GAN with data from 5G simulations on a factory automation scenario and real measurements gathered from various video streaming platforms. Our evaluations show that, compared to the state-of-the-art, our solution can significantly improve the learning and generation performance, not only for the tail distribution but also for the rest of the distribution.

  • 43.
    Barbosa, Fernando S.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Towards Safer and Risk-aware Motion Planning and Control for Robotic Systems2022Doctoral thesis, comprehensive summary (Other academic)
    Abstract [en]

    Safety and risk-awareness are important properties for robotic systems, be it for protecting them from potentially dangerous internal states, or for avoiding collisions with obstacles and environmental hazards in disaster scenarios. Ensuring safety may be the role of more than one algorithmic layer in a system, each with varying assumptions and guarantees. This thesis investigates how to provide safety and risk-awareness in a robotic system by leveraging temporal logics, motion planning algorithms, and control theory.

    Traditional control theory approaches interpret the collision avoidance safety task as a `stay-away' task; obstacles are abstracted as collections of geometric shapes, and controllers are designed to avoid each shape individually. We propose interpreting the collision avoidance problem as a `stay-within' task: the obstacle-free space is abstracted into safe regions. We propose control laws based on Control Barrier functions that guarantee that the system remains within such safe regions throughout its mission. Our results demonstrate that our controller indirectly avoids obstacles while providing the system the freedom to move within the safe regions, without the necessity to plan and track a safe trajectory. Furthermore, by extending our idea with Metric Interval Temporal Logic, we are able to consider missions with explicit time bounds. 

    Temporal logics are often used to define hard constraints on motion plans for robotic systems. However, some missions may require the system to violate constraints to make progress. Therefore, we propose to soften the hard constraints when necessary. Such soft constraints, here coined as spatial preferences, are used to account for relations between the system and the environment, such as distance from obstacles. The proposed minimally-violating motion planning algorithm attempts to find trajectories that satisfy the spatial preferences as much as possible, but violate them when needed. We demonstrate the use of spatial preferences on 3D exploration scenarios with Unmanned Aerial Vehicles, where we provide safer trajectories to the system while improving exploration efficiency. 

    In the last part of the thesis, we address safety in scenarios where a precise model of the environment is not available. In such scenarios, the system is required to fulfil the mission while minimizing risk, considering the imprecise model. We leverage Gaussian Processes to build approximate models of the environment, and use their posterior distributions in a risk metric. This risk metric allows us to consider less likely but possible events along the missions. To this end, we propose an online risk-aware motion planning approach, and validate it on disaster scenarios, where exposure to the unmodeled hazards might damage the system. Moreover, we explore risk-awareness between the control and mapping layers, by considering smooth approximations of Euclidean Distance Fields.

    Our results indicate that our algorithms provide robotic systems with i) provably-safe controllers, ii) soft safety constraints, and iii) risk-awareness in unmodeled environments. These three properties contribute to safer and risk-aware robotic systems in the real world.

    Download full text (pdf)
    Towards Safer and Risk-aware Motion Planning and Control for Robotic Systems
  • 44.
    Barbosa, Fernando S.
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Duberg, Daniel
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Jensfelt, Patric
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Tumova, Jana
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Guiding Autonomous Exploration with Signal Temporal Logic2019In: IEEE Robotics and Automation Letters, E-ISSN 2377-3766, Vol. 4, no 4, p. 3332-3339Article in journal (Refereed)
    Abstract [en]

    Algorithms for autonomous robotic exploration usually focus on optimizing time and coverage, often in a greedy fashion. However, obstacle inflation is conservative and might limit mapping capabilities and even prevent the robot from moving through narrow, important places. This letter proposes a method to influence the manner the robot moves in the environment by taking into consideration a user-defined spatial preference formulated in a fragment of signal temporal logic (STL). We propose to guide the motion planning toward minimizing the violation of such preference through a cost function that integrates the quantitative semantics, i.e., robustness of STL. To demonstrate the effectiveness of the proposed approach, we integrate it into the autonomous exploration planner (AEP). Results from simulations and real-world experiments are presented, highlighting the benefits of our approach.

  • 45.
    Barbosa, Fernando S.
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Karlsson, Jesper
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Tajvar, Pouria
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Tumova, Jana
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Formal Methods for Robot Motion Planning with Time and Space Constraints (Extended Abstract)2021In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer Nature , 2021, p. 1-14Conference paper (Refereed)
    Abstract [en]

    Motion planning is one of the core problems in a wide range of robotic applications. We discuss the use of temporal logics to include complex objectives, constraints, and preferences in motion planning algorithms and focus on three topics: the first one addresses computational tractability of Linear Temporal Logic (LTL) motion planning in systems with uncertain non-holonomic dynamics, i.e. systems whose ability to move in space is constrained. We introduce feedback motion primitives and heuristics to guide motion planning and demonstrate its use on a rover in 2D and a fixed-wing drone in 3D. Second, we introduce combined motion planning and hybrid feedback control design in order to find and follow trajectories under Metric Interval Temporal Logic (MITL) specifications. Our solution creates a path to be tracked, a sequence of obstacle-free polytopes and time stamps, and a controller that tracks the path while staying in the polytopes. Third, we focus on motion planning with spatio-temporal preferences expressed in a fragment of Signal Temporal Logic (STL). We introduce a cost function for a of a path reflecting the satisfaction/violation of the preferences based on the notion of STL spatial and temporal robustness. We integrate the cost into anytime asymptotically optimal motion planning algorithm RRT ⋆ and we show the use of the algorithm in integration with an autonomous exploration planner on a UAV.

  • 46.
    Barbosa, Fernando S.
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Lacerda, Bruno
    Univ Oxford, Oxford Robot Inst, Oxford, England..
    Duckworth, Paul
    Univ Oxford, Oxford Robot Inst, Oxford, England..
    Tumova, Jana
    KTH, School of Electrical Engineering and Computer Science (EECS), Centres, ACCESS Linnaeus Centre. KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Hawes, Nick
    Univ Oxford, Oxford Robot Inst, Oxford, England..
    Risk-Aware Motion Planning in Partially Known Environments2021In: 2021 60th IEEE  conference on decision and control (CDC), Institute of Electrical and Electronics Engineers (IEEE) , 2021, p. 5220-5226Conference paper (Refereed)
    Abstract [en]

    Recent trends envisage robots being deployed in areas deemed dangerous to humans, such as buildings with gas and radiation leaks. In such situations, the model of the underlying hazardous process might be unknown to the agent a priori, giving rise to the problem of planning for safe behaviour in partially known environments. We employ Gaussian process regression to create a probabilistic model of the hazardous process from local noisy samples. The result of this regression is then used by a risk metric, such as the Conditional Value-at-Risk, to reason about the safety at a certain state. The outcome is a risk function that can be employed in optimal motion planning problems. We demonstrate the use of the proposed function in two approaches. First is a sampling-based motion planning algorithm with an event-based trigger for online replanning. Second is an adaptation to the incremental Gaussian Process motion planner (iGPMP2), allowing it to quickly react and adapt to the environment. Both algorithms are evaluated in representative simulation scenarios, where they demonstrate the ability of avoiding high-risk areas.

  • 47.
    Barbosa, Fernando S.
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Lacerda, Bruno
    Oxford Robotics Institute, University of Oxford, United Kingdom.
    Duckworth, Paul
    Oxford Robotics Institute, University of Oxford, United Kingdom.
    Tumova, Jana
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Hawes, Nick
    Risk-Aware Motion Planning in Partially Known Environments2021Conference paper (Refereed)
    Abstract [en]

    Recent trends envisage robots being deployed inareas deemed dangerous to humans, such  as buildings with gasand radiation leaks. In such situations, the model of the underlying  hazardous process might be unknown to the agent a priori, giving rise to the problem of planning for safe behaviour inpartially known environments. We employ Gaussian Process regression to create a probabilistic model of the hazardous process from local noisy samples. The result of this regression is then used by a risk metric, such as the Conditional Value-at-Risk, to reason about the safety at a certain state. The outcome is a risk function that can  be employed in optimal motion planning problems. We demonstrate the use of the proposed function in two approaches. First is a sampling-based motion planning algorithm with an  event-based trigger for online replanning. Second is an adaptation to the  incremental Gaussian Process motion planner (iGPMP2), allowing it to quickly react and adapt to the environment. Both algorithms are evaluated in representative simulation scenarios, where they demonstrate the ability of avoiding high-risk areas.

    Download full text (pdf)
    fulltext
  • 48.
    Barbosa, Fernando S.
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Lindemann, Lars
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Dimarogonas, Dimos V.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Tumova, Jana
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Integrated motion planning and control under metric interval temporal logic specifications2019In: 2019 18th European Control Conference, ECC 2019, Institute of Electrical and Electronics Engineers (IEEE), 2019, p. 2042-2049, article id 8795925Conference paper (Refereed)
    Abstract [en]

    This paper proposes an approach that combines motion planning and hybrid feedback control design in order to find and follow trajectories fulfilling a given complex mission involving time constraints. We use Metric Interval Temporal Logic (MITL) as a rich and rigorous formalism to specify such missions. The solution builds on three main steps: (i) using sampling-based motion planning methods and the untimed version of the mission specification in the form of Zone automaton, we find a sequence of waypoints in the workspace; (ii) based on the clock zones from the satisfying run on the Zone automaton, we compute time-stamps at which these waypoints should be reached; and (iii) to control the system to connect two waypoints in the desired time, we design a low-level feedback controller leveraging Time-varying Control Barrier Functions. Illustrative simulation results are included.

  • 49.
    Barbosa, Fernando S.
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Lindemann, Lars
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Dimarogonas, Dimos V.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Tumova, Jana
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Provably safe control of Lagrangian systems in obstacle-scattered environments2020In: 2020 59th IEEE Conference on Decision and Control (CDC), Institute of Electrical and Electronics Engineers (IEEE) , 2020Conference paper (Refereed)
    Abstract [en]

    We propose a hybrid feedback control law that guarantees both safety and asymptotic stability for a class of Lagrangian systems in environments with obstacles. Rather than performing trajectory planning and implementing a trajectory-tracking feedback control law, our approach requires a sequence of locations in the environment (a path plan) and an abstraction of the obstacle-free space. The problem of following a path plan is then interpreted as a sequence of reach-avoid problems: the system is required to consecutively reach each location of the path plan while staying within safe regions. Obstacle-free ellipsoids are used as a way of defining such safe regions, each of which encloses two consecutive locations. Feasible Control Barrier Functions (CBFs) are created directly from geometric constraints, the ellipsoids, ensuring forward-invariance, and therefore safety. Reachability to each location is guaranteed by asymptotically stabilizing Control Lyapunov Functions (CLFs). Both CBFs and CLFs are then encoded into quadratic programs (QPs) without the need of relaxation variables. Furthermore, we also propose a switching mechanism that guarantees the control law is correct and well-defined even when transitioning between QPs. Simulations show the effectiveness of the proposed approach in two complex scenarios.

  • 50.
    Barbosa, Fernando S.
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Tumova, Jana
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
    Risk-Aware Navigation on Smooth Approximations of Euclidean Distance Fields Among Dynamic ObstaclesManuscript (preprint) (Other academic)
    Abstract [en]

    Reasoning about probabilistic outcomes on stochastic models is of essence to safety-critical systems. In this paper we focus on risk-aware collision avoidance approaches in workspaces with static and dynamic obstacles. More specifically, for Lagrangian systems operating in workspaces without perfect information about the obstacle-space. In order to avoid collision with static obstacles, we build smooth approximations of the Euclidean distance field, along with its first and second derivatives, using Gaussian Process implicit surfaces. Since the predictive distance returned by such an approximation is a normal distribution, rather than simply using its mean value, we propose a risk-aware Control Barrier function. Risk metrics provide more coherent measures than chance constraint, with the benefit of distinguishing between tail events. We prove that by using the proposed approach, the Lagrangian system is bound to a smaller, but safer (in terms of risk-awareness), subset of the obstacle-free space. Besides that, we also propose a controller for avoiding collisions with ellipsoidal dynamic obstacles. We compose all the controllers together into a nonsmooth barrier function, and design a Quadratic Program-based optimization controller. The proposed approach is a step-forward towards closer integration between mapping algorithms and feedback controllers. Numerical simulations on synthetic environments highlight the capabilities of the approach proposed.

    Download full text (pdf)
    Risk-Aware Navigation on Smooth Approximations of Euclidean Distance Fields Among Dynamic Obstacles
1234567 1 - 50 of 708
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf