Detection and Pose Estimation of Cherry Tomatoes Using Deep Keypoints for Automatic Picking

Thumbnail Image

URL

Journal Title

Journal ISSN

Volume Title

Sähkötekniikan korkeakoulu | Master's thesis

Authors

Date

2023-01-23

Department

Major/Subject

Autonomous Systems

Mcode

ELEC3055

Degree programme

Master's Programme in ICT Innovation

Language

en

Pages

54

Series

Abstract

With the development of artificial intelligence and robotics, more and more automatic devices are designed and built to help human production. This work explores a representative application of fruit picking in the agriculture area, focusing on the detection and pose estimation of cherry tomatoes. Since the proper way of picking for cherry tomatoes is by cutting the branched stem of the fruit bunch, the detection of that particular stem and its pose is the main difficulty faced in this task. Two datasets were created for two proposed perception systems both based on YOLO, a deep learning model for object detection, and a keypoint labeling method was proposed for annotating the pose of the bunches for various shapes of cherry tomatoes. Although the first proposed model, which combined the detection for the stem and the bunch, did not meet the requirement of this task, the second model introducing key-point detection successfully tackled the problem inspired by YOLO-Pose, which is designed for the human pose estimation task. Some modifications to the keypoint generation process on the original YOLO-Pose were applied to limit the range of generation area by absorbing the information from the detection box, and brought an increase in both performance and robustness, reaching the AP@0.05 of (0.962) and AP@0.5 of (0.826) on the test set and reducing the normalized keypoint distance error by 10.6% on average.

Description

Supervisor

Kyrki, Ville

Thesis advisor

Chen, Qiming

Keywords

fruit detection, keypoint detection, automatic fruit picking, pose estimation

Other note

Citation

Permanent link to this item

https://urn.fi/URN:NBN:fi:aalto-202301291709

Collections

[dipl] Sähkötekniikan korkeakoulu / ELEC