Detection and Pose Estimation of Cherry Tomatoes Using Deep Keypoints for Automatic Picking

No Thumbnail Available
Journal Title
Journal ISSN
Volume Title
Sähkötekniikan korkeakoulu | Master's thesis
Date
2023-01-23
Department
Major/Subject
Autonomous Systems
Mcode
ELEC3055
Degree programme
Master's Programme in ICT Innovation
Language
en
Pages
54
Series
Abstract
With the development of artificial intelligence and robotics, more and more automatic devices are designed and built to help human production. This work explores a representative application of fruit picking in the agriculture area, focusing on the detection and pose estimation of cherry tomatoes. Since the proper way of picking for cherry tomatoes is by cutting the branched stem of the fruit bunch, the detection of that particular stem and its pose is the main difficulty faced in this task. Two datasets were created for two proposed perception systems both based on YOLO, a deep learning model for object detection, and a keypoint labeling method was proposed for annotating the pose of the bunches for various shapes of cherry tomatoes. Although the first proposed model, which combined the detection for the stem and the bunch, did not meet the requirement of this task, the second model introducing key-point detection successfully tackled the problem inspired by YOLO-Pose, which is designed for the human pose estimation task. Some modifications to the keypoint generation process on the original YOLO-Pose were applied to limit the range of generation area by absorbing the information from the detection box, and brought an increase in both performance and robustness, reaching the AP@0.05 of (0.962) and AP@0.5 of (0.826) on the test set and reducing the normalized keypoint distance error by 10.6% on average.
Description
Supervisor
Kyrki, Ville
Thesis advisor
Chen, Qiming
Keywords
fruit detection, keypoint detection, automatic fruit picking, pose estimation
Other note
Citation