Edge Computing-based Multi-camera Multi-object Detection and Tracking

Abstract

The studies of Multi-Camera Multi-Object Tracking (MCMOT) has enabled vision-based Real-Time Location System (RTLS). Vision-based RTLS has various applications realized by monitoring and analyzing movement in a designated location. One of many challenges of a MCMOT system is its level of complexity to process multiple object detection and tracking in real-time on multiple streams. Thanks to the recent advancement in GPU-accelerated edge device, NVIDIA introduced Jetson that allows multiple neural networks to be processed in parallel. This work aims to showcase the MCMOT system that could be run in real-time while being processed entirely by the NVIDIA Jetson device.

The proposed MCMOT system is designed and implemented based on the edge cloud architecture where we process MCMOT on the edge using Azure IoT Edge runtime, and only send insightful data into the cloud. Our system pipeline consists of three modules: (1) Deepstream SDK module for object detection and tracking in each cameras, (2) Multi-camera Tracker module for transforming positions from camera perspective and combining those positions into global 2D coordinates, (3) Cloud integration and Visualization modules for outputting the derived global coordinates.

To validate our system, we created our own dataset and evaluated different object detector backbones and object trackers. Our system can perform real-time MCMOT of four streams at full resolution. However, the results of the experiment suggest that the proposed system struggles heavily on the problem caused by occlusion and inaccurate homography transformation. Based on these experiments, we are able to identify major shortcomings and provide possible direction for improvement in the future work.