The Practical Applicability of a CNN for Automated Building Damage Assessment

Thumbnail Image
Journal Title
Journal ISSN
Volume Title
Perustieteiden korkeakoulu | Master's thesis
Machine Learning, Artificial Intelligence andData Science
Degree programme
Master’s Programme in Computer, Communication and Information Sciences
Every year many people are impacted by the effects of natural hazards. For an effective disaster response, a building damage assessment is of great importance. In current practices, these assessments are done manually. Executing these assessments automatically using satellite imagery holds big potential; it decreases workload while increasing safety, consistency, timeliness, spatial coverage, and possibly accuracy. However, before an automated assessment can be used in real-life situations, it should be well designed to fit user needs. Therefore, this research focuses on the practical applicability of a \ac{cnn} for automated building damage assessment. The models' practical applicability is assessed on two requirements: the consistency of performance across disasters; and the performance on a disaster for which no labeled data exists. Firstly, to test the consistency, it is explored how the model performs on a range of 13 disasters, including four different damage types (wind, flooding, tsunami, and volcanic eruptions) across different geographical regions. Testing on this variety of data has never been done before. The results show that performance significantly differs across disasters. Through quantitative analysis it is found that the percentage of buildings belonging to each damage class largely influences the performance, while image and disaster-specific parameters do not show a significant impact. Secondly, a realistic setting of data availability is simulated, where no labeled damage data of the test disaster is available. A model is trained on several sets of training disasters that do not include the test disaster. The best performance is reached when training solely on disasters that have the same damage type (e.g.\@ wind) as the test disaster. Performance in this set-up differed drastically between the two test disasters experimented with, reaching 93\% and 53\% of the macro F1 score compared to when the model was trained on the test disaster itself. Thus, the model definitely has potential to learn from previous disasters, but additional research is required to find out what influences the difference in transferability. Lastly, several modifications to the model were implemented to examine their impact on the performance. The main findings were that the negative impact of data imbalance can be diminished by applying resampling or cost-sensitive learning and that while solely using imagery of the area after the disaster, shows a drop in performance, it could be used in situations where no pre disaster imagery is available. The experiments in this research show that performance differs significantly across disasters. While for some disasters the model gains good performance, even in the real-life context of not having labeled data of the test disaster, for others the performance is disappointing. A first attempt to understand these differences was made, but further research is needed to affirm the results.
Laaksonen, Jorma
Thesis advisor
van den Homberg, Marc
Jung, Alexander
machine learning, building damage, CNN, damage assessment, transfer learning