Real-Time Augmented Reality based Operator Assistance for Driving Cut-to-Length Forest Machines

Abstract

The automation of forestry operations is pivotal for the effective maintenance of forest resources. Forestry operations often involve the utilization of heavy machinery, such as cut-to-length (CTL) machines, designed for tree logging. The success of these operations relies heavily on the proficiency of the operator, highlighting the need for operator assistance. This support aids operators in making real-time decisions, thereby bridging the gap between experienced and inexperienced operators. While automation technologies are prevalent in CTL machines, some lack real-time assistance, and others rely on on-screen displays, requiring the translation of information from offline maps and displays to the real-world. Thus, there is a need for an automation solution that can offer real-time assistance directly to the operator's view. Augmented reality emerges as a fitting solution, where operators view the forest through a Head Mounted Device (HMD), and assistance information, including tree species names and navigation routes, are seamlessly displayed within this view. Therefore, this thesis aims to develop a proof-of-concept system for operator driving with HMDs in outdoor conditions, demonstrating the capability to provide operator assistance information based on the current location of the all-terrain vehicle (ATV) and the gaze of the operator. The platform was implemented on an ATV, facilitating driving with an HMD in a parking area. The goal was to overlay the offline point cloud map of the parking area onto corresponding real-world objects when viewed through the headset. User studies were conducted, and video recordings of the operator's view, coupled with feedback from users, were analyzed to assess the solution's effectiveness in presenting location-based information and overall feasibility. The results indicated that the point cloud partially matched real-world objects for 66.7% of the total duration, with observed discrepancies in depth alignment between point cloud and real-world objects. Ergonomic studies revealed that lighter headsets and improved video quality could enhance operational duration and quality.