The Feasibility of Identifying Defects Illustrated on Building Facades by Applying Thermal Infrared Images with Shadow ^†

Abstract

Infrared thermography (IRT) has been widely employed to identify the defects illustrated in building facades. However, the IRT covered with a shadow is hard to be applied to determine the defects shown in the IRT. The study proposed an approach based on the multiplicated model to describe quantitively the shadow effects, and the IRT can be segmented into few classes according to the surface temperature information recorded on the IRT by employing a thermal infrared camera. The segmented results were compared with the non-destructive method (acoustic tracing) to verify the correctness and robustness of the approach. From the processed results, the proposed approach did correctly identify the defects illustrated in building facades through the IRTs were covered with shadow.

1. Introduction

Infrared thermography (IRT) can be used to surface inspection. Image segmentation is a crucial step in interpreting the visual information from the recorded images. IRT usually can be segmented into several regions such that the surface temperature differences can be applied to identify the defects. However, the segmented results can be affected by several factors, like shadow effects. While an image segmentation technique is used on the IRT covered with shadow effects, the shadow will interfere with the segmented results such that the segmented results are corrupted. Can the IRT cover with shadow effects be analyzed to identify the defects with image segmentation? This paper proposes an approach to analyze IRT with shadow effects by employing the multiplicative model. Vese and Chan introduced piecewise-constant and piecewise-smooth level set functions into the model such that the model can be simplified and numerically implemented the segmentation by employing two level set functions [1]. The proposed approach applies the image segmentation model proposed by Vese and Chan, and the iteration scheme proposed by Zhang [2] et al. to segment the IRT with shadow and the shadow effects can be approximated. The segmented results were verified by comparing the results obtained by applying acoustic tracing method. The remainder of this paper is organized as follows. In the next section, the schematic way of the proposed approach and the related theories are introduced. In Section 3, the processed results by applying the proposed method and taping are illustrated. Section 4 some conclusions are drawn.

2. Methods

A multiplicative model can be applied to interpret the shadow effects presented in IRT. In general, the image model of IRT can be presented as follows:

$$I_0=bT+n$$

(1)

where the $I_0$ is the given IRT, the b is the quantitative presentations of shadow effects, the T is the calibrated image without shadow effects, and the n is noise. Zhang et al. proposed to apply two level set functions (${\varphi }_1 \mathrm{and} {\varphi }_2$), the standard deviations of the segmented regions, and the iteration scheme is employed to approximate the intensity inhomogeneity such that the given image can be correctly segmented [2]. In doing so, the given IRT with shadow effects can be segmented into several regions such that the data distribution in each segmented region is homogeneous and be replaced by a regional constant $c_i$, and that the whole scene $\mathsf{\Omega }$ can be composed by the set of $R_i\left(\mathsf{\Omega }=\bigcup R_i\right)$. Hence, the whole image can be presented as follows:

$$I_0=b\left({\displaystyle \sum }_{i=1}^N{c}_i\right)$$

(2)

where the N indicates the number of the segmented regions. There is an assumption that for a particular location x in the given IRT, its neighbors do have the contributions of shadow effects to the location. With combining the Equations (1) and (2) into the model proposed by Zhang [2], the image model can be illustrated as follows:

$$E\left(\sigma ,c,b\right)={\displaystyle \int }\left({\displaystyle \sum }_{i=1}^N{\displaystyle \int }K\left(y,x\right)\left(\log {\sigma }_i+\frac{{\left(I_0-b\left(x\right)c_i\right)}^2}{2{\sigma }_i^2}\right)M_i\left(\mathsf{\Phi }\right)dx\right)dy$$

(3)

where K is the weight function, ${\sigma }_i \mathrm{is}$ the standard deviation of the segmented region ${\mathrm{R}}_i$, and $M_1\left(\mathsf{\Phi }\right)=H\left({\varphi }_1\right)H\left({\varphi }_2\right)$, $M_2\left(\mathsf{\Phi }\right)=H\left({\varphi }_1\right)\left(1-H\left({\varphi }_2\right)\right)$, $M_3\left(\mathsf{\Phi }\right)=\left(1-H\left({\varphi }_1\right)\right)H\left({\varphi }_2\right) \mathrm{and} M_4\left(\mathsf{\Phi }\right)=\left(1-H\left({\varphi }_1\right)\right)\left(1-H\left({\varphi }_2\right)\right)$. H is defined as the Heaviside function. The regional constants $c_i$ can be found by applying the derivative of Equation (3) with respect to $c_i$, and $c_i$ can be presented as follows:

$$c_i=\frac{{{\displaystyle \int }}_{\mathsf{\Omega }}^{}\left(K\left(y,x\right)⨂b\right)I_0{M}_i\left(\mathsf{\Phi }\right)dydx}{{{\displaystyle \int }}_{\mathsf{\Omega }}^{}\left(K\left(y,x\right)⨂b^2\right)M_i\left(\mathsf{\Phi }\right)dydx}$$

(4)

where $⨂$ is the convolution operator. The standard deviation ${\sigma }_i$ of the segmented region ${\mathrm{R}}_i$ can be obtained by finding the derivative of Equation (3) with respect to ${\sigma }_i$, and ${\sigma }_i$ is mathematically illustrated as follows:

$${\sigma }_i^2=\frac{{{\displaystyle \int }}_{{\mathsf{\Omega }}_i}^{}K\left(y,x\right)⨂\left[{\left(I_0-b\left(x\right)c_i\right)}^2{M}_i\left(\mathsf{\Phi }\right)\right]dydx}{{{\displaystyle \int }}_{{\mathsf{\Omega }}_i}^{}K\left(y,x\right)⨂M_i\left(\mathsf{\Phi }\right)dydx}$$

(5)

The shadow effects b can be estimated by taking the derivative of Equation (3) with respect to b, and it can be shown as follows:

$$b=\frac{{{\displaystyle \sum }}_{i=1}^4\left[K\left(y,x\right)⨂I_0{M}_i\left(\mathsf{\Phi }\right)\right]\frac{c_i}{{\sigma }_i^2}}{{{\displaystyle \sum }}_{i=1}^4\left[K\left(y,x\right)⨂M_i\left(\mathsf{\Phi }\right)\right]\frac{c_i^2}{{\sigma }_i^2}}$$

(6)

Eventually, the segmented results can be approximated as follows:

$$S=C_1{M}_1\left(\mathsf{\Phi }\right)+C_2{M}_2\left(\mathsf{\Phi }\right)+C_3{M}_3\left(\mathsf{\Phi }\right)+C_4{M}_4\left(\mathsf{\Phi }\right)$$

(7)

In doing so, the IRT with shadow can be segmented such that regional boundaries of the segmented regions can be located. The surface temperature differences can be identified.

3. Materials and Processed Results

In this study, Thermal infrared camera InfReC R500Pro made by NEC was employed to record the surface temperature information, and it can simultaneously photo the same objects in a digital format. Each recorded thermal infrared image has 480 × 640 pixels, and it can measure the minimum temperature difference up to 0.03 °C. A series of thermal infrared images were taken from 9:00 AM to 3:00 PM on 30 January 2019, to evaluate the validity of the algorithm to depress the shadow effects illustrated in the thermal infrared images. One image was selected and shown in Figure 1., the parts of the building façade were covered with shadow effects, and parts of the building façade was exposed under the sun. With the proposed approach, the processed results were illustrated in Figure 2. Figure 2a the segmented results were illustrated though the IRT was covered with shadow. The shadow effects were quantified and shown in Figure 2b. The relationships between the numbers of iterations and the defined energy defined by Equation (3) are presented in Figure 2c.

The acoustic tracing measurements were applied to the façade of the building to verify the segmented results of using the proposed algorithms to analyze IRT [3]. The audio signals of the acoustic tracing will reach a maximum while a precursor hit the place in which can be identified as a defect, and an example is given in Figure 3. The façade was examined by the acoustic tracing measurements, and there are totally 180 locations examined. The defect locations identified by the acoustic tracing approach were illustrated in Figure 4. Roughly, those locations are falling into the segmented regions with higher surface temperatures.

4. Conclusions

There are several conclusions made by applying the proposed approach to analyze the given IRT, and are given as follows:

• The proposed method can correctly segment the IRT affected by the shadow effects;
• The defects can be identified from the segmented results by the extracted regional boundaries;
• The proposed approach can quickly reach a convergence;
• The proposed approach can quantitively approximate the shadow effects;
• The acoustic tracing does verify the segmented results of applying the proposed approach.