Grasp Planning for Robotic Manipulation in 6-DoF

Abstract

Robotic grasping is a fundamental capability in manufacturing, healthcare, and domestic environments, enabling robots to manipulate objects efficiently and autonomously. This paper focuses on task-oriented grasping (TOG) in six degrees of freedom, which improves accuracy and translatability to real-world applications. We aim to contribute to the ongoing development of TOG by identifying challenges, evaluation metrics, describing two TOG approaches, and suggesting future research directions towards fully autonomous robotic task execution.

Three-Stage Grasp

method overview

Image adapted from Newburry et al.

First, Grasp Synthesis find one or multiple high-quality grasps based on criterias such as grasp stability, reliability, gripper geometry and kinematics.

Then, Grasp Planning generates a trajectory for the robot to execute the grasp while avoiding collisions. Second, the planned trajectory and grasp are executed. The gripper applies forces and torques to a set of contact points on the object.

Finally, the object is lifted and used for the next task.

Grasping Planning with CaTGrasp

Image taken from CaTGrasp.

CaTGrasp is an exemplar-based, data-driven method for 6-DoF task-oriented grasping with a two-finger parallel gripper, designed for industrial assembly tasks where an object is picked from clutter and then placed at a target location. To avoid costly manual labeling, it learns category-level knowledge such as shape, pose, and task-relevant gripper-object contact points from unlabeled synthetic point clouds by simulating grasps and measuring whether the object can be lifted and placed successfully (Fig. 5a). At execution time, the scene point cloud is segmented, a target object is chosen (Fig. 5b), and the learned category knowledge is transferred to it based on object similarity (Fig. 5c). Grasp candidates are then sampled and ranked by combining grasp stability with task success (Fig. 5d). The approach performs nearly as well in the real world as in simulation, but it only covers three object categories, which limits generalization to unseen objects.

Grasping Planning with FoundationGrasp

Image taken from FoundationGrasp.

FoundationGrasp targets better generalization to unseen objects and tasks by integrating foundation models into task-oriented grasping, trained on the LaViA-TaskGrasp dataset of kitchen objects and tasks. Given an object image, a point cloud with a grasp candidate, and a language instruction for instance "pick up the cup to pour," GPT generates a geometric description of the object and a semantic description of the task, while crowdsourced internet images add extra views to counter occlusion. The semantic information is encoded with BERT, and the geometric information is encoded after CLIP extracts multimodal features from the image. A transformer with self- and cross-attention processes both branches and outputs a score indicating how well each candidate, sampled by a task-agnostic grasp sampler, suits the task. Because the LLM descriptions link geometrically similar objects and related verbs, the method generalizes much better, although it still requires labor-intensive annotation, tends to prefer less robust grasps near handle ends, and is limited to two-finger grippers.

BibTeX


  @misc{banning_grasp-planning_2024,
    author = {Banning, Ole},
    title  = {Grasp Planning for Robotic Manipulation in 6-DoF},
    school = {Technical University of Munich},
    year   = {2024},
    type   = {Seminar Paper},
  }

Grasp Planning for Robotic Manipulation in 6-DoF

Seminar Paper 2024

Example taken from FoundationGrasp.

In this seminar paper, we outline the challenges of task-oriented grasping, two promising approaches, FoundationGrasp and CaTGrasp, and discuss future directions of task-oriented grasping.

Abstract

Three-Stage Grasp

Grasping Planning with CaTGrasp

Grasping Planning with FoundationGrasp

BibTeX