Sensor fusion approach for unmanned aerial vehicle object tracking and following

Abstract

Uncrewed aerial vehicles are widely used in critical tasks including search and rescue, infrastructure inspection and environmental monitoring, all requiring continuous and reliable object tracking. However, existing trackers often fail under full or partial occlusion, large displacement, motion blur, varying illumination or when limited to a single sensing modality. This thesis presents three models that improve object tracking and following, where two models enhance object tracking performance and one model improves object following performance. The first is the Supervised Machine Learning Hybrid Analytical Tracker which combines supervised machine learning and analytical tracking techniques to improve both accuracy and robustness. The second is a cross-correction and fusion model that enables multiple tracking channels, such as visible light and infrared, to operate in parallel and correct each other’s outputs to maintain target lock even when one channel fails. The third is a motion-aware camera platform control model that improves object following by compensating for rotative motion effects of the platform. Evaluation is only performed on the single-channel tracking algorithms including proposed Supervised Machine Learning Hybrid Analytical Tracker. The models are evaluated using metrics such as accuracy measured by mean Intersection over Union, robustness based on tracking continuity, average computational speed and performance variance across repeated runs. Statistical significance is assessed using the Shapiro–Wilk test followed by the Friedman and Wilcoxon signed-rank tests with Bonferroni correction. Experiments are conducted in both indoor laboratory and outdoor uncrewed aerial vehicle setups to ensure reliability and real-world relevance. The results show that the Supervised Machine Learning Hybrid Analytical Tracker achieves 42% higher accuracy and 25% higher robustness compared to classical trackers such as Channel and Spatial Reliability and Kernelized Correlation Filter. Compared to the vision transformers based Fast Segment Anything Model, it achieves 3% higher accuracy and 15% higher robustness. Although it operates 52% slower in inference compared to Fast Segment Anything, the increased accuracy and robustness indicate its strength in long-term tracking, especially in scenarios involving large displacement and full occlusion.