Detecting trash and valuables with machine vision in passenger vehicles

Abstract

The research conducted here will determine the possibility of implementing a machine vision based detection system to identify the presence of trash or valuables in passenger vehicles using a custom designed in-car camera module. The detection system was implemented to capture images of the rear seating compartment of a car intended to be used in shared vehicle fleets. Onboard processing of the image was done by a Raspberry Pi computer while the image classification was done by a remote server. Two vision based algorithmic models were created for the purpose of classifying the images: a convolutional neural network (CNN) and a background subtraction model. The CNN was a fine-tuned VGG16 model and it produced a final prediction accuracy of 91.43% on a batch of 140 test images. For the output analysis, a confusion matrix was used to identify the correlation between correct and false predictions, and the certainties of the three classes for each classified image were examined as well. The estimated execution time of the system from image capture to displaying the results ranged between 5.7 seconds and 11.5 seconds. The background subtraction model failed for the application here due to its inability to form a stable background estimate. The incorrect classifications of the CNN were evident due to the external sources of variation in the images such as extreme shadows and lack of contrast between the objects and its neighbouring background. Improvements in changing the camera location and expanding the training image set were proposed as possible future research.