Earthquake Damage Assessment Based on Deep Learning Method Using VHR Images ^†

Abstract

One of the numerous fundamental tasks to perform rescue operations after an earthquake is to check the status of buildings that have been destroyed. The methods to obtain the damage map are in two categories. The first group of methods uses data before and after the earthquake, and the second group only uses the data after the earthquakes that we want, to offer a flexible damage map according to information that we are available to achieve. In this paper, we work on VHR satellite images of Haiti and UNet which is a convolution network. The learning algorithm’s profound changes to improve the results were intended to identify the damage of the buildings caused by the earthquake. The deep learning algorithms require training data and that is one of the problems that we want to solve. As well as previous studies examining pixel by pixel degradation, ultimate precision to increase that shows the success of this approach felt and has been able to reach the overall accuracy of 68.71%. The proposed method for other natural disasters such as rockets, explosions, tsunamis, and floods also destroyed buildings in urban areas is to be used.

1. Introduction

The world has constantly influenced natural disasters such as earthquakes, floods, and tsunami during civilization. They are considered to be extremely tragic threats and ruined human security and property. A quick evaluation of infrastructure damage after a dangerous event has an essential role in emergency response management and recovery planning [1,2].

The traditional approach to estimating the spatial distribution of earthquake losses in buildings through building field inspection is made by a volunteer group consisting of engineers, architects, and other construction industry professionals. This precise inspection process is essential because evaluations are reliable and give us valuable information on the damaged building’s seismic function. However, the duration of these inspections makes them impossible for emergency support and planning for early recovery. Depending on the availability of qualified specialists and the geographical distribution of damaged buildings, the field inspection process can last for months [3].

Therefore, for decades, remote sensing techniques play an essential role in examining the earthquake’s damage, especially due to its fast availability after catastrophic and large coverage. In most studies, remote sensing measurements have been used to detect collapsed buildings using different methods before and after the event. Some researchers have only used post-event information and the similarity between damaged buildings that use it to distinguish between destroyed and undamaged buildings that limit this method’s accuracy [4,5,6].

As the main sources of remote sensing, optical images usually provide only two-dimensional information that is not suitable for detecting objects; using LIDAR and SAR can improve determination and identification, especially for three-dimensional objects. Information about the height, especially for the three-dimensional interpretation of the building’s state, can detect damaged and collapsed buildings by comparing the altitude information before and after the event in urban areas. The method of producing accurate altitude data is tough and expensive, which leads to the inaccessibility of the precise 3D data before and after the earthquake. UAV allows for higher resolution images and cloud 3D points. However, the preparation of UAV images is challenging before the earthquake because UAV images are not continually covering all regions of the world. However, satellite images are still the primary and commonest source for damage assessment. For this reason, they are more reliable and more accessible to detect the destruction of buildings [7,8,9].

With the rapid improvement of satellite optics sensors’ spatial resolution, optical data is promising data for identifying earthquake damages. However, achieving remote sensing VHR images before the earthquake is not easy. Therefore, in new studies, we have tried to achieve good results without having the before event information. This has led to valuable algorithms that can be used with both access and not access to data before the earthquake. The other problem with satellite images is that they cannot detect damage to the building’s length due to the imaging of the above [10,11].

The most common method is to detect damage based on change detection techniques. Images before and after an event have been collected and create an image difference, representing the difference between the two datasets. However, this method limits the requirement to have two sets of before and after the earthquake that may not always be available. In such circumstances, machine learning methods had been introduced. Deep learning, one of the advanced techniques in the field of machine learning, is the best method for complex and non-linear feature extraction and is at a high level. In recent years, convolutional neural networks (CNN) due to outstanding performance in extracting features on remote sensing have been widely used [12,13].

In terms of operational response to the disaster, many challenges remain, two of them to declare that we have and try to solve these challenges [14].

• Good performance of deep learning algorithms is limited to the size of data available, and the network structure is considered. One of the most critical challenges for using a deep learning method for monitoring the buildings damaged in the disaster is that the training images of damaged targets are usually not very much. In terms of operational response to the disaster, many challenges remain, two of them declare that we have and try to solve these challenges.
• The size of blocks that have been labeled as undamaged or damaged buildings by the algorithm is ultimately a significant impact on overall accuracy. Previous studies only a label on a large block and was allocated. However, this block contains a large number of pixels is irrelevant. Therefore, theoretically, the pixel-based labeling method is more accurate [15,16].

UNet, which is essentially a convolution network, can ultimately reduce the challenges, and we change its layers to make it perform better [17].

In this study, the convolution neural network UNet for monitoring Haiti earthquake damage on pixel-based images with high-resolution remote sensing is implemented.

2. Experiments

2.1. Datasets

On 12 January 2010, an earthquake with a magnitude of seven on the Richter scale hit Port-au-Prince, capital of Haiti. In Port-au-Prince and in the southern areas of Haiti, about 97,294 houses were completely destroyed and 188,383 houses have suffered damage [18]. The study area is part of the city of Port-au-Prince and is shown in Figure 1. In this study, by Worldview 2 satellite imagery, pre-image was acquired on 16 January 2010, and post-image obtained on 1 October 2009. The satellite image consists of four multi-spectral bands with a resolution of 2 m and one high-resolution panchromatic band with 0.5 m resolution. Four high-resolution colored bands are used in this algorithm, through integrating the multi-spectral and panchromatic bands. To assess damaged and undamaged buildings use the International Institute UNITAR / UNOSAT data and Earthquake Geospatial Data Dataverse (CGA, Harvard Univ) dataset with visual interpretation [19,20].

View of the area before and after the earthquake is shown in Figure 2.

2.2. Method

In this study, an approach based on deep learning algorithms and neural networks for monitoring the buildings destroyed by the earthquake is presented. The way that we are going to explain it is based on previous studies where the use of VHR optical data is emphasized. The solution proposed to lack of access to the data before the earthquake [21,22,23].

Deep learning is a neural network with a large number of hidden layers to extract many features from raw data. Data can be an image, pixel, signal, and so on. The different architecture of this kind exists today. The number of layers is greater (deeper), so the more non-linear characteristics are obtained, which is why we are interested in deep learning. Figure 3 shows the general view of the deep learning networks. Unlike deep learning, machine learning extracts features by itself, and they need to identify the characteristics and feature engineering [24,25,26].

The UNet algorithm, due to high precision, high-speed processing, and learning, has no need for large data sets to learn and complex and expensive hardware. In recent years, the objects of the image and image processing has become popular in detection. The characteristics of this network enable us to overcome two major challenges that we mentioned in the introduction [27,28]. Figure 4, is shown an overview of the proposed method in this study.

2.2.1. Pre-Processing

The images collected before and after the event were compiled into a large image. The co-registration procedure was implemented on the pre and post-event images. Bands of pre-post images pan sharped and stacked together [29]. At this stage, each pixel should be assigned a value of zero or one that reflects the state of the destruction of the building. Both images and ground-truth data of building damage were projected into the UTM/WGS84 geo-referenced coordinate system. Theoretically and ideally, the image tiles with the pixel size of an arbitrary 2ⁿ are suitably used as the input [30]. When selecting random patches to do the training, we try to choose the patches that more than half of the pixels within them labeled as undamaged or damaged.

2.2.2. Network Architecture and Training

UNet originated from Ronneberger in 2015 [31]. The blocks of neural network units of UNet, UNet adopted in this study, and Deep Residual UNet that proposed by Zhang in 2018 are shown in Figure 5 [31,32,33].

It has already been shown in many studies that normalizing input data on different architectures accelerates network convergence. The use of the Batch Normalization in deep learning algorithms makes sustainable education and training operations faster [34]. So Residual Unet network, as well as our proposed network of Batch Normalization, is used. We normalize the input layer by adjusting and scaling the activations. For instance, once we have features from 0 to 1 and a few from 1 to 1000, we should always normalize them to hurry up learning. If the input layer is taking advantage of it, why not do an equivalent thing also for the values within the hidden layers, that are changing all the time, and get 10 times or more improvement in the training speed [35,36]. The architecture of the UNet algorithm is shown in Figure 6.

The max-pooling layer used in general U-net was replaced by a convolutional layer with a stride of two because the convolutional layer with increased stride outperforms the max-pooling with regards to several image recognition benchmarks as shown in Figure 5 [37].

To reduce the calculation time cost, we decreased the filter number to 50% of the first. This strategy was recommended in many studies because it was shown useful for remote-sensing recognition tasks [38].

We use a batch size of 25 and a patch size of 256 × 256 pixels for the UNet models. The models were trained for 50 epochs. We trained the network with a learning rate of 0.01 for all epochs. RMSProb is used for parameter optimization that is suitable for large datasets. The employed loss function is cross-entropy.

2.2.3. Development Environment

In this work, we used the Deep Learning Studio (DLS) and Peltarion as the deep learning framework [39,40]. DLS is DeepCognition’s web-based software, that designs networks and trains deep learning algorithms for Windows and Linux. DLS supports deep learning networks for image recognition tasks. All experimentation and modeling environment tasks are implemented in Deep Learning Studio Virtual Machine (DLSVM).

The virtual machine is configured with 32 GB of RAM, a 2.30 GHz 2-core Intel(R) Xeon(R) CPU, and a 1.59 GHz NVIDIA Tesla T4 GPU 16GB DDR6 with 50 GB memory [39,40].

The data are preprocessed and analyzed in Python using the GDAL, NumPy, pandas, OpenCV, Scipy, Scikit-image, Scikit-learn, Pillow, MKL, and Tifffile libraries. The deep learning algorithms are achieved in the Deep Learning Studio (DLS) which is a robust GUI, partially free, and easy-to-use framework. It can be used in the cloud or on our infrastructure.

3. Results

In this study, we completed 50 epochs for both the UNet model and deep residual UNet to get the trained building damage recognition model [31,32]. The relation between the cross-entropy loss and the iteration of epochs is shown in Figure 5. Trend graphs in Figure 7 show that our proposed method considerably and noticeably improves results. Both networks till epoch 15 are close to each other, with not much difference between them.

Here, the UNet has a much lower omission error (28.1% undamaged, and 39.3% damaged) than the deep residual UNet (37.7% undamaged, and 47.2% damaged). The overall accuracy of our proposed approach 69.71 and the overall accuracy of deep residual UNet 62.5%, which is shown the method is proposed in this paper proves the performance of the network. The Kappa value for UNet in this paper is 37.7%.

Some buildings were classified incorrectly, because of the orthographic projection characteristic of the optical remote sensing measurement, the sensor can only record the information on top of each object, and the damage situation under the roof is not reflected. An example is shown in Figure 8 [41,42].

The final result of the buildings damage map shown is in Figure 9. Although the number of buildings damaged and safe is almost equal, about there are three times the pixels dedicated to undamaged buildings more than buildings have been destroyed. The black area relates to parks, slums, and tents survivors as well as other items that are not within the building kind.

From the availability of data, to achieve the final map takes less than 7 h and that is much faster than the field inspection and this is the advantage of this approach.

4. Discussion

This model was demonstrated for mapping the earthquake damage, but the framework also works for other hazards such as floods, missile attacks, hurricanes, and many natural and unnatural disasters. To generalize this framework to other tragic event types, the VHR satellite image and corresponding reference data for different disaster types should be used to train the new model. The proposed model is a supervised classification model. It can be simply implemented to react to future hazards after these models are well developed.

The role of building footprint data is to create training data labels. It should be considered that the label of land covers the jungle, water, etc. The non-built-up regions are also available, and we can train a new model that does not depend on the building footprint data. From this aspect, the proposed framework does not depend on building footprint data and is a generalized framework.

5. Conclusions

Dominance image processing and artificial intelligence in the field of images of remote sensing, especially with the development of algorithms for deep learning, continues to grow. However, unlike other issues that improved very significantly, in remote sensing a little performance increased, so still, we need to research and further studies of the potential of the computer world in the field of geographical sciences and image processing.