Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Reinforcement Learning for Pivoting Task
KTH, School of Computer Science and Communication (CSC), Robotics, perception and learning, RPL.
KTH, School of Computer Science and Communication (CSC), Robotics, perception and learning, RPL.
KTH, School of Computer Science and Communication (CSC), Robotics, perception and learning, RPL.
KTH, School of Computer Science and Communication (CSC), Robotics, perception and learning, RPL.ORCID iD: 0000-0003-2965-2953
(English)Manuscript (preprint) (Other academic)
Abstract [en]

In this work we propose an approach to learn a robust policy for solving the pivoting task. Recently, several model-free continuous control algorithms were shown to learn successful policies without prior knowledge of the dynamics of the task. However, obtaining successful policies required thousands to millions of training episodes, limiting the applicability of these approaches to real hardware. We developed a training procedure that allows us to use a simple custom simulator to learn policies robust to the mismatch of simulation vs robot. In our experiments, we demonstrate that the policy learned in the simulator is able to pivot the object to the desired target angle on the real robot. We also show generalization to an object with different inertia, shape, mass and friction properties than those used during training. This result is a step towards making model-free reinforcement learning available for solving robotics tasks via pre-training in simulators that offer only an imprecise match to the real-world dynamics.

Keywords [en]
Reinforcement Learning, Pivoting, Dexterous Manipulation
National Category
Robotics
Identifiers
URN: urn:nbn:se:kth:diva-215138OAI: oai:DiVA.org:kth-215138DiVA, id: diva2:1146611
Note

QC 20171023

Available from: 2017-10-03 Created: 2017-10-03 Last updated: 2019-12-04Bibliographically approved
In thesis
1. Vision-Based In-Hand Manipulation with Limited Dexterity
Open this publication in new window or tab >>Vision-Based In-Hand Manipulation with Limited Dexterity
2019 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

In-hand manipulation is an action that allows for changing the grasp on an object without the need for releasing it. This action is an important component in the manipulation process and helps solving many tasks. Human hands are dexterous instruments suitable for moving an object inside the hand. However, it is not common for robots to be equipped with dexterous hands due to many challenges in control and mechanical design. In fact, robots are frequently equipped with simple parallel grippers, robust but lacking dexterity. This thesis focuses on achieving in-hand manipulation with limited dexterity. The proposed solutions are based only on visual input, without the need for additional sensing capabilities in the robot's hand.

Extrinsic dexterity allows simple grippers to execute in-hand manipulation thanks to the exploitation of external supports. This thesis introduces new methods for solving in-hand manipulation using inertial forces, controlled friction and external pushes as additional supports to enhance the robot's manipulation capabilities. Pivoting is seen as a possible solution for simple grasp changes: two methods, which cope with inexact friction modeling, are reported, and pivoting is successfully integrated in an overall manipulation task. For large scale in-hand manipulation, the Dexterous Manipulation Graph is introduced as a novel representation of the object. This graph is a useful tool for planning how to change a certain grasp via in-hand manipulation. It can also be exploited to combine both in-hand manipulation and regrasping to augment the possibilities of adjusting the grasp. In addition, this method is extended to achieve in-hand manipulation even for objects with unknown shape. To execute the planned object motions within the gripper, dual-arm robots are exploited to enhance the poor dexterity of parallel grippers: the second arm is seen as an additional support that helps in pushing and holding the object to successfully adjust the grasp configuration.

This thesis presents examples of successful executions of tasks where in-hand manipulation is a fundamental step in the manipulation process, showing how the proposed methods are a viable solution for achieving in-hand manipulation with limited dexterity.

Abstract [sv]

In-hand manipulation gör det möjligt att ändra fattningen om ett objekt utan att behöva släppa det. Detta är en viktig komponent och gör det möjligt att lösa många uppgifter.Den mänskliga händen är ett flexibelt instrument som är lämpligt för att flytta föremål inuti handen. Det är dock inte vanligt att robotar är utrustade med lika flexibla händer på grund av utmaningar inom reglerteknik och design av mekaniska system. I själva verket är robotar ofta utrustade med enkla parallel gripper, som är robusta men saknar finmotorik. Denna avhandling fokuserar på att uppnå in-hand manipulation med begränsad finmotorik. De föreslagna lösningarna baseras endast på visuell perception, utan behov av ytterligare sensorer i robotens hand.

Extrinsic dexterity (extrinsisk finmotorik) gör att enkla robothänder kan utföra in-hand manipulation tack vare utnyttjandet av externa stöd. Denna avhandling introducerar nya metoder för att lösa in-hand manipulation med tröghetskrafter, kontrollerad friktion och yttre tryck som ytterligare stöd för att förbättra robotens manipuleringsförmåga. Pivoting ses som en möjlig lösning för enkla greppförändringar: två metoder som hanterar inexakt friktionsmodellering presenteras samt som gungning är framgångsrikt integrerats i en fullständig manipuleringsuppgift. För storskalig in-hand manipulation introduceras Dexterous Manipulation Graph som en ny representation av objektet. Denna graf är ett användbart verktyg för att planera ändring av grepp via in-hand manipulation. Det kan också utnyttjas för att kombinera både in-hand manipulation och regrasping för att öka möjligheterna att justera greppet. Dessutom utvidgas denna metod för att uppnå in-hand manipulation även för föremål med okänd form. För att utföra de planerade objektrörelserna i robothanden utnyttjas dubbelarmade robotar för att förbättra den dåliga färdigheten hos parallel gripper: den andra armen ses som ett ytterligare stöd som hjälper till att skjuta och hålla objektet för att framgångsrikt justera greppkonfigurationen.

Denna avhandling presenterar exempel på framgångsrika utföranden av uppgifter där manuell manipulation är ett grundläggande steg i manipuleringsprocessen och visar hur de föreslagna metoderna är en rimlig och effektiv lösning för att uppnå handmanipulation med begränsad finmotorik.

Place, publisher, year, edition, pages
KTH Royal Institute of Technology, 2019
Series
TRITA-EECS-AVL ; 2019:74
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:kth:diva-263051 (URN)978-91-7873-332-3 (ISBN)
Public defence
2019-11-25, Kollegiesalen, Stockholm, 10:00 (English)
Opponent
Supervisors
Note

QC 20191105

Available from: 2019-11-05 Created: 2019-10-28 Last updated: 2019-11-05Bibliographically approved

Open Access in DiVA

fulltext(5710 kB)27 downloads
File information
File name FULLTEXT01.pdfFile size 5710 kBChecksum SHA-512
06d74b080bbbc4d38ef7866788a59652a16390ee783c675e8f9bf2f693e45b0cc6839304514c6e92baa9012a00121bd70a48ada2d24545afbed5f4eedd496ed9
Type fulltextMimetype application/pdf

Other links

https://arxiv.org/abs/1703.00472

Search in DiVA

By author/editor
Antonova, RikaCruciani, SilviaSmith, ChristianKragic, Danica
By organisation
Robotics, perception and learning, RPL
Robotics

Search outside of DiVA

GoogleGoogle Scholar
Total: 27 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 528 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf