Change search
Link to record
Permanent link

Direct link
BETA
Publications (4 of 4) Show all publications
Rai, A., Antonova, R., Meier, F. & Atkeson, C. G. (2019). Using Simulation to Improve Sample-Efficiency of Bayesian Optimization for Bipedal Robots. Journal of machine learning research, 20, Article ID 49.
Open this publication in new window or tab >>Using Simulation to Improve Sample-Efficiency of Bayesian Optimization for Bipedal Robots
2019 (English)In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 20, article id 49Article in journal (Refereed) Published
Abstract [en]

Learning for control can acquire controllers for novel robotic tasks, paving the path for autonomous agents. Such controllers can be expert-designed policies, which typically require tuning of parameters for each task scenario. In this context, Bayesian optimization (BO) has emerged as a promising approach for automatically tuning controllers. However, sample-efficiency can still be an issue for high-dimensional policies on hardware. Here, we develop an approach that utilizes simulation to learn structured feature transforms that map the original parameter space into a domain-informed space. During BO, similarity between controllers is now calculated in this transformed space. Experiments on the ATRIAS robot hardware and simulation show that our approach succeeds at sample-efficiently learning controllers for multiple robots. Another question arises: What if the simulation significantly differs from hardware? To answer this, we create increasingly approximate simulators and study the effect of increasing simulation-hardware mismatch on the performance of Bayesian optimization. We also compare our approach to other approaches from literature, and find it to be more reliable, especially in cases of high mismatch. Our experiments show that our approach succeeds across different controller types, bipedal robot models and simulator fidelity levels, making it applicable to a wide range of bipedal locomotion problems.

Place, publisher, year, edition, pages
MICROTOME PUBL, 2019
Keywords
Bayesian Optimization, Bipedal Locomotion, Transfer Learning
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:kth:diva-249828 (URN)000463322000001 ()
Note

QC 20190423

Available from: 2019-04-23 Created: 2019-04-23 Last updated: 2019-04-23Bibliographically approved
Rai, A., Antonova, R., Song, S., Martin, W., Geyer, H. & Atkeson, C. (2018). Bayesian Optimization Using Domain Knowledge on the ATRIAS Biped. In: 2018 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA): . Paper presented at IEEE International Conference on Robotics and Automation (ICRA), MAY 21-25, 2018, Brisbane, AUSTRALIA (pp. 1771-1778). IEEE
Open this publication in new window or tab >>Bayesian Optimization Using Domain Knowledge on the ATRIAS Biped
Show others...
2018 (English)In: 2018 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), IEEE, 2018, p. 1771-1778Conference paper, Published paper (Refereed)
Abstract [en]

Robotics controllers often consist of expert-designed heuristics, which can be hard to tune in higher dimensions. Simulation can aid in optimizing these controllers if parameters learned in simulation transfer to hardware. Unfortunately, this is often not the case in legged locomotion, necessitating learning directly on hardware. This motivates using data-efficient learning techniques like Bayesian Optimization (BO) to minimize collecting expensive data samples. BO is a black-box data-efficient optimization scheme, though its performance typically degrades in higher dimensions. We aim to overcome this problem by incorporating domain knowledge, with a focus on bipedal locomotion. In our previous work, we proposed a feature transformation that projected a 16-dimensional locomotion controller to a 1-dimensional space using knowledge of human walking. When optimizing a human-inspired neuromuscular controller in simulation, this feature transformation enhanced sample efficiency of BO over traditional BO with a Squared Exponential kernel. In this paper, we present a generalized feature transform applicable to non-humanoid robot morphologies and evaluate it on the ATRIAS bipedal robot, in both simulation and hardware. We present three different walking controllers and two are evaluated on the real robot. Our results show that this feature transform captures important aspects of walking and accelerates learning on hardware and simulation, as compared to traditional BO.

Place, publisher, year, edition, pages
IEEE, 2018
Series
IEEE International Conference on Robotics and Automation ICRA, ISSN 1050-4729
National Category
Robotics
Identifiers
urn:nbn:se:kth:diva-237160 (URN)000446394501055 ()2-s2.0-85063147098 (Scopus ID)978-1-5386-3081-5 (ISBN)
Conference
IEEE International Conference on Robotics and Automation (ICRA), MAY 21-25, 2018, Brisbane, AUSTRALIA
Funder
Knut and Alice Wallenberg Foundation
Note

QC 20181024

Available from: 2018-10-24 Created: 2018-10-24 Last updated: 2019-08-20Bibliographically approved
Kokic, M., Antonova, R., Stork, J. A. & Kragic, D. (2018). Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation. In: Proceedings of The 2nd Conference on Robot Learning, PMLR 87: . Paper presented at 2nd Conference on Robot Learning, October 29th-31st, 2018, Zürich, Switzerland. (pp. 641-650).
Open this publication in new window or tab >>Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation
2018 (English)In: Proceedings of The 2nd Conference on Robot Learning, PMLR 87, 2018, p. 641-650Conference paper, Oral presentation with published abstract (Refereed)
Abstract [en]

We develop an approach that benefits from large simulated datasets and takes full advantage of the limited online data that is most relevant. We propose a variant of Bayesian optimization that alternates between using informed and uninformed kernels. With this Bernoulli Alternation Kernel we ensure that discrepancies between simulation and reality do not hinder adapting robot control policies online. The proposed approach is applied to a challenging real-world problem of task-oriented grasping with novel objects. Our further contribution is a neural network architecture and training pipeline that use experience from grasping objects in simulation to learn grasp stability scores. We learn task scores from a labeled dataset with a convolutional network, which is used to construct an informed kernel for our variant of Bayesian optimization. Experiments on an ABB Yumi robot with real sensor data demonstrate success of our approach, despite the challenge of fulfilling task requirements and high uncertainty over physical properties of objects.

National Category
Computer Systems
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-248396 (URN)
Conference
2nd Conference on Robot Learning, October 29th-31st, 2018, Zürich, Switzerland.
Note

QC 20190507

Available from: 2019-04-07 Created: 2019-04-07 Last updated: 2019-05-07Bibliographically approved
Antonova, R., Kokic, M., Stork, J. A. & Kragic, D. (2018). Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation. In: : . Paper presented at 2nd Conference on Robot Learning, Zürich, Switzerland, Oct. 29-31 2018.
Open this publication in new window or tab >>Global Search with Bernoulli Alternation Kernel for Task-oriented Grasping Informed by Simulation
2018 (English)Conference paper, Published paper (Refereed)
Abstract [en]

We develop an approach that benefits from large simulated datasets and takes full advantage of the limited online data that is most relevant. We propose a variant of Bayesian optimization that alternates between using informed and uninformed kernels. With this Bernoulli Alternation Kernel we ensure that discrepancies between simulation and reality do not hinder adapting robot control policies online. The proposed approach is applied to a challenging real-world problem of task-oriented grasping with novel objects. Our further contribution is a neural network architecture and training pipeline that use experience from grasping objects in simulation to learn grasp stability scores. We learn task scores from a labeled dataset with a convolutional network, which is used to construct an informed kernel for our variant of Bayesian optimization. Experiments on an ABB Yumi robot with real sensor data demonstrate success of our approach, despite the challenge of fulfilling task requirements and high uncertainty over physical properties of objects.

National Category
Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-249696 (URN)
Conference
2nd Conference on Robot Learning, Zürich, Switzerland, Oct. 29-31 2018
Note

Contribution/Authorship note: Rika Antonova and Mia Kokic contributed equally

QC 20190520

Available from: 2019-04-17 Created: 2019-04-17 Last updated: 2019-05-20Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-3018-2445

Search in DiVA

Show all publications