Food object recognition: An application of deep learning

Abstract

Identifying a food from its image can save people’s life. It can be used to know the presence of potential allergens in food or by estimating the nutritional content of food, it may also be used to combat the obesity epidemic. With such applications in mind, we seek to exploit the advances in machine learning and deep learning to train models that identify European food from digital photos. From the literature it was discovered that the Faster RCNN was the current state-of-art CNN based framework which could get local information of object in image and recognize it. Furthermore, we also develop an Android application for recognition of food objects.

Faster RCNN requires a large volume of data with labels and localization infor- mation of the objects present in them. It is very challenging to find such datasets to train our network. We made up a food dataset of 69k images with 445 labels and trained our model using those images. But the dataset was skewed in terms of numbers of images per category that negatively affected the performance of the model. To improve the performance, we tried several approaches like taking only a subset of labels and equalizing the number of training samples for each label. We also used transfer learning to get around the problem of overfitting the network when our training sample size is limited. Finally, by using publicly available data set and adapting it to our needs, our model was able to identify images with 0.37 mean Average Precision. The Android application uses this model to recognize food objects from images.