CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Vision-Based In-Hand Manipulation with Limited Dexterity
KTH, School of Electrical Engineering and Computer Science (EECS), Robotics, Perception and Learning, RPL.ORCID iD: 0000-0002-9171-8768
2019 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

In-hand manipulation is an action that allows for changing the grasp on an object without the need for releasing it. This action is an important component in the manipulation process and helps solving many tasks. Human hands are dexterous instruments suitable for moving an object inside the hand. However, it is not common for robots to be equipped with dexterous hands due to many challenges in control and mechanical design. In fact, robots are frequently equipped with simple parallel grippers, robust but lacking dexterity. This thesis focuses on achieving in-hand manipulation with limited dexterity. The proposed solutions are based only on visual input, without the need for additional sensing capabilities in the robot's hand.

Extrinsic dexterity allows simple grippers to execute in-hand manipulation thanks to the exploitation of external supports. This thesis introduces new methods for solving in-hand manipulation using inertial forces, controlled friction and external pushes as additional supports to enhance the robot's manipulation capabilities. Pivoting is seen as a possible solution for simple grasp changes: two methods, which cope with inexact friction modeling, are reported, and pivoting is successfully integrated in an overall manipulation task. For large scale in-hand manipulation, the Dexterous Manipulation Graph is introduced as a novel representation of the object. This graph is a useful tool for planning how to change a certain grasp via in-hand manipulation. It can also be exploited to combine both in-hand manipulation and regrasping to augment the possibilities of adjusting the grasp. In addition, this method is extended to achieve in-hand manipulation even for objects with unknown shape. To execute the planned object motions within the gripper, dual-arm robots are exploited to enhance the poor dexterity of parallel grippers: the second arm is seen as an additional support that helps in pushing and holding the object to successfully adjust the grasp configuration.

This thesis presents examples of successful executions of tasks where in-hand manipulation is a fundamental step in the manipulation process, showing how the proposed methods are a viable solution for achieving in-hand manipulation with limited dexterity.

Abstract [sv]

In-hand manipulation gör det möjligt att ändra fattningen om ett objekt utan att behöva släppa det. Detta är en viktig komponent och gör det möjligt att lösa många uppgifter.Den mänskliga händen är ett flexibelt instrument som är lämpligt för att flytta föremål inuti handen. Det är dock inte vanligt att robotar är utrustade med lika flexibla händer på grund av utmaningar inom reglerteknik och design av mekaniska system. I själva verket är robotar ofta utrustade med enkla parallel gripper, som är robusta men saknar finmotorik. Denna avhandling fokuserar på att uppnå in-hand manipulation med begränsad finmotorik. De föreslagna lösningarna baseras endast på visuell perception, utan behov av ytterligare sensorer i robotens hand.

Extrinsic dexterity (extrinsisk finmotorik) gör att enkla robothänder kan utföra in-hand manipulation tack vare utnyttjandet av externa stöd. Denna avhandling introducerar nya metoder för att lösa in-hand manipulation med tröghetskrafter, kontrollerad friktion och yttre tryck som ytterligare stöd för att förbättra robotens manipuleringsförmåga. Pivoting ses som en möjlig lösning för enkla greppförändringar: två metoder som hanterar inexakt friktionsmodellering presenteras samt som gungning är framgångsrikt integrerats i en fullständig manipuleringsuppgift. För storskalig in-hand manipulation introduceras Dexterous Manipulation Graph som en ny representation av objektet. Denna graf är ett användbart verktyg för att planera ändring av grepp via in-hand manipulation. Det kan också utnyttjas för att kombinera både in-hand manipulation och regrasping för att öka möjligheterna att justera greppet. Dessutom utvidgas denna metod för att uppnå in-hand manipulation även för föremål med okänd form. För att utföra de planerade objektrörelserna i robothanden utnyttjas dubbelarmade robotar för att förbättra den dåliga färdigheten hos parallel gripper: den andra armen ses som ett ytterligare stöd som hjälper till att skjuta och hålla objektet för att framgångsrikt justera greppkonfigurationen.

Denna avhandling presenterar exempel på framgångsrika utföranden av uppgifter där manuell manipulation är ett grundläggande steg i manipuleringsprocessen och visar hur de föreslagna metoderna är en rimlig och effektiv lösning för att uppnå handmanipulation med begränsad finmotorik.

Place, publisher, year, edition, pages
KTH Royal Institute of Technology, 2019.
Series
TRITA-EECS-AVL ; 2019:74
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
URN: urn:nbn:se:kth:diva-263051ISBN: 978-91-7873-332-3 (print)OAI: oai:DiVA.org:kth-263051DiVA, id: diva2:1366179
Public defence
2019-11-25, Kollegiesalen, Stockholm, 10:00 (English)
Opponent
Supervisors
Note

QC 20191105

Available from: 2019-11-05 Created: 2019-10-28 Last updated: 2019-11-05Bibliographically approved
List of papers
1. Reinforcement Learning for Pivoting Task
Open this publication in new window or tab >>Reinforcement Learning for Pivoting Task
(English)Manuscript (preprint) (Other academic)
Abstract [en]

In this work we propose an approach to learn a robust policy for solving the pivoting task. Recently, several model-free continuous control algorithms were shown to learn successful policies without prior knowledge of the dynamics of the task. However, obtaining successful policies required thousands to millions of training episodes, limiting the applicability of these approaches to real hardware. We developed a training procedure that allows us to use a simple custom simulator to learn policies robust to the mismatch of simulation vs robot. In our experiments, we demonstrate that the policy learned in the simulator is able to pivot the object to the desired target angle on the real robot. We also show generalization to an object with different inertia, shape, mass and friction properties than those used during training. This result is a step towards making model-free reinforcement learning available for solving robotics tasks via pre-training in simulators that offer only an imprecise match to the real-world dynamics.

Keywords
Reinforcement Learning, Pivoting, Dexterous Manipulation
National Category
Robotics
Identifiers
urn:nbn:se:kth:diva-215138 (URN)
Note

QC 20171023

Available from: 2017-10-03 Created: 2017-10-03 Last updated: 2019-10-28Bibliographically approved
2. Integrating Path Planning and Pivoting
Open this publication in new window or tab >>Integrating Path Planning and Pivoting
2018 (English)In: 2018 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS) / [ed] Maciejewski, AA Okamura, A Bicchi, A Stachniss, C Song, DZ Lee, DH Chaumette, F Ding, H Li, JS Wen, J Roberts, J Masamune, K Chong, NY Amato, N Tsagwarakis, N Rocco, P Asfour, T Chung, WK Yasuyoshi, Y Sun, Y Maciekeski, T Althoefer, K AndradeCetto, J Chung, WK Demircan, E Dias, J Fraisse, P Gross, R Harada, H Hasegawa, Y Hayashibe, M Kiguchi, K Kim, K Kroeger, T Li, Y Ma, S Mochiyama, H Monje, CA Rekleitis, I Roberts, R Stulp, F Tsai, CHD Zollo, L, IEEE , 2018, p. 6601-6608Conference paper, Published paper (Refereed)
Abstract [en]

In this work we propose a method for integrating motion planning and in-hand manipulation. Commonly addressed as a separate step from the final execution, in-hand manipulation allows the robot to reorient an object within the end-effector for the successful outcome of the goal task. A joint achievement of repositioning the object and moving the manipulator towards its desired final pose saves time in the execution and introduces more flexibility in the system. We address this problem using a pivoting strategy (i.e. in-hand rotation) for repositioning the object and we integrate this strategy with a path planner for the execution of a complex task. This method is applied on a Baxter robot and its efficacy is shown by experimental results.

Place, publisher, year, edition, pages
IEEE, 2018
Series
IEEE International Conference on Intelligent Robots and Systems, ISSN 2153-0858
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:kth:diva-246313 (URN)10.1109/IROS.2018.8593584 (DOI)000458872706008 ()978-1-5386-8094-0 (ISBN)
Conference
25th IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), OCT 01-05, 2018, Madrid, SPAIN
Note

QC 20190319

Available from: 2019-03-19 Created: 2019-03-19 Last updated: 2019-10-28Bibliographically approved
3. Dexterous Manipulation Graphs
Open this publication in new window or tab >>Dexterous Manipulation Graphs
2018 (English)In: 2018 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS) / [ed] Maciejewski, AA Okamura, A Bicchi, A Stachniss, C Song, DZ Lee, DH Chaumette, F Ding, H Li, JS Wen, J Roberts, J Masamune, K Chong, NY Amato, N Tsagwarakis, N Rocco, P Asfour, T Chung, WK Yasuyoshi, Y Sun, Y Maciekeski, T Althoefer, K AndradeCetto, J Chung, WK Demircan, E Dias, J Fraisse, P Gross, R Harada, H Hasegawa, Y Hayashibe, M Kiguchi, K Kim, K Kroeger, T Li, Y Ma, S Mochiyama, H Monje, CA Rekleitis, I Roberts, R Stulp, F Tsai, CHD Zollo, L, IEEE , 2018, p. 2040-2047Conference paper, Published paper (Refereed)
Abstract [en]

We propose the Dexterous Manipulation Graph as a tool to address in-hand manipulation and reposition an object inside a robot's end-effector. This graph is used to plan a sequence of manipulation primitives so to bring the object to the desired end pose. This sequence of primitives is translated into motions of the robot to move the object held by the end-effector. We use a dual arm robot with parallel grippers to test our method on a real system and show successful planning and execution of in-hand manipulation.

Place, publisher, year, edition, pages
IEEE, 2018
Series
IEEE International Conference on Intelligent Robots and Systems, ISSN 2153-0858
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:kth:diva-246311 (URN)10.1109/IROS.2018.8594303 (DOI)000458872702017 ()2-s2.0-85062989451 (Scopus ID)978-1-5386-8094-0 (ISBN)
Conference
25th IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), OCT 01-05, 2018, Madrid, SPAIN
Note

QC 20190319

Available from: 2019-03-19 Created: 2019-03-19 Last updated: 2019-10-29Bibliographically approved
4. In-Hand Manipulation of Objects with Unknown Shapes
Open this publication in new window or tab >>In-Hand Manipulation of Objects with Unknown Shapes
(English)Manuscript (preprint) (Other academic)
Abstract [en]

This work addresses the problem of changing grasp configurations on objects with an unknown shape through in-hand manipulation. Our approach leverages shape priors,learned as deep generative models, to infer novel object shapesfrom partial visual sensing. The Dexterous Manipulation Graph method is extended to build upon incremental data and account for estimation uncertainty in searching a sequence of manipulation actions. We show that our approach successfully solves in-hand manipulation tasks with unknown objects, and demonstrate the validity of these solutions with robot experiments.

National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:kth:diva-263044 (URN)
Note

QC 20191029

Available from: 2019-10-28 Created: 2019-10-28 Last updated: 2019-10-29Bibliographically approved
5. Discrete Bimanual Manipulation for Wrench Balancing
Open this publication in new window or tab >>Discrete Bimanual Manipulation for Wrench Balancing
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Dual-arm robots can overcome grasping force and payload limitations of a single arm by jointly grasping an object.However, if the distribution of mass of the grasped object is not even, each arm will experience different wrenches that can exceed its payload limits.In this work, we consider the problem of balancing the wrenches experienced by  a dual-arm robot grasping a rigid tray.The distribution of wrenches among the robot arms changes due to objects being placed on the tray.We present an approach to reduce the wrench imbalance among arms through discrete bimanual manipulation.Our approach is based on sequential sliding motions of the grasp points on the surface of the object, to attain a more balanced configuration.%This is achieved in a discrete manner, one arm at a time, to minimize the potential for undesirable object motion during execution.We validate our modeling approach and system design through a set of robot experiments.

National Category
Robotics
Identifiers
urn:nbn:se:kth:diva-263041 (URN)
Note

Under review for ICRA 2020. QC 20191029

Available from: 2019-10-28 Created: 2019-10-28 Last updated: 2019-10-29Bibliographically approved

Open Access in DiVA

fulltext(12315 kB)24 downloads
File information
File name FULLTEXT02.pdfFile size 12315 kBChecksum SHA-512
2cc2d63b69a3f3f3badb60befb75232b11cf055ad7ed984ab2f05ee5174832d3d8a276badc56c6a3d1ece77404dc426d4dfcba6d7763e09f30cc5d9f9a6ccc7e
Type fulltextMimetype application/pdf

Authority records BETA

Cruciani, Silvia

Search in DiVA

By author/editor
Cruciani, Silvia
By organisation
Robotics, Perception and Learning, RPL
Computer Vision and Robotics (Autonomous Systems)

Search outside of DiVA

GoogleGoogle Scholar
Total: 24 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 152 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf