Machine Recognition of Engineering Diagrams in Process Industry

Abstract

Engineering diagrams are widely used in process industry as a standard graphic language to represent engineering schematics and convey information. Over the years, a large amount of legacy engineering diagrams has been accumulated in companies so that there is an increasing demand on digitizing the diagrams to improve productivity. With the progress of computer vision, especially deep learning-based object detection, we take advantage of the latest deep learning models and algorithms to process and recognize the legacy diagrams, which fuel the networks. In this thesis, an end-to-end digitizing model is proposed to recognize engineering diagrams as machine encoded format. Due to the complexity of diagrams, the recognition task is divided into three sub-targets: symbols, connectivity and characters. We experiment on multiple state-of-the-art deep learning-based approaches to recognize symbols, such You Only Look Once (YOLO). Line recognition algorithm is proposed based on Hough transformation and Skeletonization. For the characters, we conduct the recognition by two steps, first locating, second recognizing.

The model is evaluated on real industry engineering diagrams with quantitative and visual results provided. Firstly, the results demonstrate that YOLO works well for symbol recognition, reaching over 90% mAP@0.75 of all symbols. YOLO can also be used for character locating, where the characters are regarded as a symbol. Secondly, connection lines can be recognized effectively by the proposed algorithm combining Hough Transformation with region of interests. The result of recognizing the three sub-targets are integrated to generate a DXF format diagram. There is also some discussion on a universal model which can expand the usage of our model to different types of diagrams. One of the most essential steps is the analysis of source diagrams and data preparation, which is time consuming dirty work but can significantly improve the recognition performance.