This repository is deprecated. Please access https://github.com/caiobarrosv/selective_grasping for the updated code.

This work has been already sent to a congress. Please avoid copying this repository for publishing to avoid copyright issues.

Single Shot Generative Grasping Convolutional Neural Network (SSGG-CNN)

---

Contents

1. Authors
2. Description
3. Required packages - Kinetic Version
4. Run GGCNN in Gazebo and RVIZ
5. Connecting with the real UR5

---

1.0 - Authors

• Caio Viturino* - [Lattes] [Linkedin] - engcaiobarros@gmail.com
• Kleber de Lima Santana Filho** - [Lattes] [Linkedin] - engkleberf@gmail.com
• Daniel M. de Oliveira* - danielmoura@ufba.br
• Cézar Bieniek Lemos* - cezarcbl@protonmail.com
• André Gustavo Scolari Conceição* - [Lattes] - andre.gustavo@ufba.br

*LaR - Laboratório de Robótica, Departamento de Engenharia Elétrica e de Computação, Universidade Federal da Bahia, Salvador, Brasil

**PPGM - Programa de Pós-Graduação em Mecatrônica, Universidade Federal da Bahia, Salvador, Brasil.

2.0 - Description

This paper proposes a two-step cascaded system with the Generative Grasping Convolutional Neural Network (GG-CNN) and the Single Shot Multibox Detector architecture (SSD) to perform grasping in a vision-based object recognition system. We call the proposed method as Single Shot Generative Grasping Neural Network (SSGG-CNN). The GG-CNN is a powerful object-independent grasping synthesis method well-known for the outstanding performance in open-loop and closed-loop systems using a pixel-wise grasp quality prediction. However, it is not capable of distinguishing between manipulable objects and fixed objects in the workspace. In order to mitigate this problem, the SSD was adopted to perform object detection. It allows the grasping system to perform the grasp only in manipulable objects identified by the SSD. We found an average success rate of 85% over 20 grasps attempts considering open-loop with static and uncluttered objects randomly organized on a planar surface.

The GGCNN was created by Douglas Morrison, Peter Corke, Jürgen Leitner (2018).

3.0 - Required packages - Kinetic Version

This code was developed with Python 2.7 on Ubuntu 16.04 with ROS Kinetic.

• Realsense Gazebo Plugin
• Realsense-ros Release version 2.2.11
• Librealsense Release version 2.31.0 - Install from source
• Moveit Kinetic
• Moveit Python
• Robotiq Gripper
• Universal Robot
• ur_modern_driver
• Gluoncv
• Opencv
• Mxnet Install Mxnet for your CUDA version.

NOTE: This package should be placed into your src folder. Please open an issue if you find any problem related to this package.

Easy install

In order to install all the required packages easily, create a new catkin workspace

mkdir -p ~/catkin_ws_new/src

Clone this repository into the src folder

cd ~/catkin_ws_new/src
git clone https://github.com/lar-deeufba/ssggcnn_ur5_grasping

Run the install.sh file

cd ~/catkin_ws_new/src/ssggcnn_ur5_grasping/install
sudo chmod +x ./install.sh
./install.sh

This repository also need the SSD512 implementation created by czrcbl. Please follow the next procedures provided by the author.

Clone and install the following:

git clone --branch ssggcnn https://github.com/czrcbl/bboxes.git

git clone --branch ssggcnn https://github.com/czrcbl/detection.git

Download the model.params in the following link and move it to the scripts/detection_pkg folder.

4.0 - Run SSGG-CNN in Gazebo

Please follow each following steps:

4.1 - Launch Gazebo:

roslaunch ssggcnn_ur5_grasping gazebo_ur5.launch

4.2 - Run the UR5 control node

Press enter after the following message appears and jump to the step 4.3: "==== Press enter to move the robot to the 'depth cam shot' position!"

rosrun ssggcnn_ur5_grasping ur5_open_loop.py --gazebo

4.3 - Run the SSD node

rosrun ssggcnn_ur5_grasping main.py

4.4 - Run the GG-CNN node

Press enter after the following message appears and jump to the step 4.5: "Press enter to start the GGCNN"

rosrun ssggcnn_ur5_grasping run_ggcnn.py --ssggcnn

4.5 - Spawn the objects in the workspace

rosrun ssggcnn_ur5_grasping spawn_objects.py

4.6 - UR5 control node

After running the GG-CNN node you are able to move the robot and perform the grasp. Press enter to complete each related task specified in ur5_open_loop.py

4.7 - Change the Gazebo properties (OPTIONAL)

It will speed up your Gazebo simulation a little bit :)

rosrun ssggcnn_ur5_grasping change_gazebo_properties.py

4.8 - Visualize the images published by the GG-CNN

You might want to see the grasp or any other image. In order to do that, you can use the rqt_image_view.

rosrun rqt_image_view

4.9 - Visualize depth cloud in RVIZ

If you want to visualize the data being published by the Intel Realsense D435 please run the following node:

rosrun ssggcnn_ur5_grasping rviz_ur5.launch