Towards Robust 6-DoF Multi-Finger Grasping in Clutter with Explicit Scene Understanding

Thumbnail Image

Files

isbn9789526406824.pdf (16.05 MB)

URL

Journal Title

Journal ISSN

Volume Title

School of Electrical Engineering | Doctoral thesis (article-based) | Defence date: 2022-02-25
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.

Authors

Date

2022

Department

Sähkötekniikan ja automaation laitos
Department of Electrical Engineering and Automation

Major/Subject

Mcode

Degree programme

Language

en

Pages

122 + app. 164

Series

Aalto University publication series DOCTORAL THESES, 15/2022

Abstract

Learning to grasp, be it from experience or data, has transformed how we view robotic grasping. In the last decade alone, the idea of learning has resulted in numerous approaches that can quickly generate successful grasps on a wide variety of unknown objects, outperforming previous non-learning-based methods by a large margin. However, many of the learning-based methods reach such a performance by limiting themselves to the generation of 4-Degree of Freedom (DoF) top-down parallel-jaw grasps on singulated objects. These limitations facilitate learning by constraining the search space, but also prevent sampling 6-DoF multi-finger grasps that are useful in, for example, semantic grasping. This dissertation aims to determine whether explicit scene understanding, such as completely reconstructing object shapes from partial point clouds instead of using the point clouds directly, can lift these limitations. More specifically, it investigates the methods and benefits of including explicit scene understanding when learning singulated objects and objects in clutter 6-DoF parallel jaw and multi-finger grasp samplers. To this aim, we first explore 4-DoF grasping and present a shape reconstruction method that enables 4-DoF top-down grasping methods to generate complete 6-DoF grasps. The same reconstruction method is also applied to enable quick 6-DoF multi-finger grasp sampling. Then, we investigate how to represent object shape and composition uncertainties in grasping, resulting in two grasp planners: one robust grasp plannerover uncertain shape completions and one POMDP planner over object composition uncertainties. Finally, we explore grasping objects in cluttered scenes, where we propose to reconstruct every object in the scene with object segmentation and object reconstruction to facilitate grasping. This process, named scene completion, was fundamental for developing a fast target-driven multi-finger grasp sampler for grasping objects in clutter. Together, all the results indicate that explicit scene understanding does increase the generality, robustness, and performance of approaches that learn 6-DoF parallel jaw and multi-finger grasping, albeit at a higher computational cost. Consequently, we recommend that roboticists consider explicit scene understanding when developing new grasping approaches.

Description

Supervising professor

Kyrki, Ville, Prof., Aalto University, Department of Electrical Engineering and Automation, Finland

Thesis advisor

Verdoja, Francesco, Dr., Aalto University, Finland

Keywords

robotic grasping, deep learning, dexterous manipulation

Other note

Parts

  • [Publication 1]: Jens Lundell, Francesco Verdoja and Ville Kyrki. Robust Grasp Planning Over Uncertain Shape Completions. In International Conference on Intelligent Robots and Systems (IROS), Macau, China. pp. 1526–1532, November 2019.
    DOI: 10.1109/IROS40897.2019.8967816 View at publisher
  • [Publication 2]: Jens Lundell, Francesco Verdoja and Ville Kyrki. Beyond Top-Grasps Through Scene Completion. In International Conference on Robotics and Automation (ICRA), Paris, France. pp. 545–551, November 2020.
    Full text in Acris/Aaltodoc: http://urn.fi/URN:NBN:fi:aalto-202010306202
    DOI: 10.1109/ICRA40945.2020.9197320 View at publisher
  • [Publication 3]: Jens Lundell, Enric Corona, Tran Nguyen Le, Francesco Verdoja, Philippe Weinzaepfel, Gregory Rogez, Francesc Moreno-Noguer and Ville Kyrki. Multi-FinGAN: Generative Coarse-To-Fine Sampling of Multi-Finger Grasps. In International Conference on Robotics and Automation (ICRA), Xi’an, China. pp. 4495–4501, May 2021
  • [Publication 4]: Jens Lundell, Francesco Verdoja and Ville Kyrki. DDGC: Generative Deep Dexterous Grasping in Clutter. In Robotics and Automation Letters (RA-L), pp. 6899–6906, October 2021.
    Full text in Acris/Aaltodoc: http://urn.fi/URN:NBN:fi:aalto-202110139604
    DOI: 10.1109/LRA.2021.3096239 View at publisher
  • [Publication 5]: Joni Pajarinen, Jens Lundell and Ville Kyrki. POMDP Manipulation Planning under Object Composition Uncertainty. Submitted to Transactions on Robotics (T-RO), October 2021

Citation

Permanent link to this item

https://urn.fi/URN:ISBN:978-952-64-0682-4

Collections

[diss] Sähkötekniikan korkeakoulu / ELEC