Vessel Bearing Estimation Using Visible and Thermal Imaging

Abstract

Maritime awareness and autonomous navigation can be enabled by state-of-the-art deep learning methods, by monitoring and tracking the position of maritime vessels. In our experiment, we acquire ship dataset from Megastar cruise ferry campaigns in Baltic Sea. We detect the nearby vessels using visible and infrared imaging sensors and object detectors You-Only-Look-Once v5 (YOLOv5), and Detectron2 RCNN network and use that information along with DeepSORT method to track the position of the vessel. We obtain the bearing of the vessels detected from both infrared and visible image sequences of ship and fuse them using a Kalman filter (KF) and Rauch-Tung-Striebel (RTS) smoother. We then compare the result to the bearing obtained from the automatic identification system (AIS) of the vessel, and also compare it among object detectors. We obtained a root-mean-square error of 0.17∘ in vessel bearing tracking as compared to AIS.