Vision-Based Damage Detection Method Using Multi-Scale Local Information Entropy and Data Fusion

Abstract

Low-spatial-resolution measurements from contact sensors and excessive measurement noise have impeded the implementation of vibration-based damage detection. To tackle these challenges, we propose a novel vision-based damage detection method combining multi-scale signal analysis theory and data fusion algorithm. For high-spatial-resolution vibration measurements, phase-based optical flow estimation algorithm is adopted to deploy virtual sensors on the structure, yielding reliable mode shapes. We then introduce the concept of entropy into damage detection. A novel damage index, defined in Gaussian multi-scale space and named multi-scale local information entropy (MS-LIE), is proposed. The MS-LIE integrates the multi-scale analysis component and the entropy analysis component, addressing both the issue of detection sensitivity and noise immunity, thereby showcasing enhanced performance. Moreover, a data fusion technique for multi-scale damage information is developed to further mitigate the noise-induced uncertainty and pinpoint damage locations. A series of numerical and experimental scenarios are designed to validate the method, and the results indicate that the proposed method accurately detects single and multiple damages in noisy environments, obviating the need for baseline data as a reference.

1. Introduction

Structural health monitoring (SHM) is critical for tracking the performance and condition of engineering structures over their service life, essential across mechanical, aviation, civil, and infrastructure sectors [1,2,3]. In light of the widely accepted assumption that local damage alters modal parameters, a variety of vibration-based damage detection methods have emerged, such as natural frequency-based [4,5,6], modal damping-based [7,8,9], and mode shape-based [10,11,12]. Although the frequency shift indicates alterations in the state-of-health of structures, its properties of lumped parameter restrict its application to damage localization. Additionally, practical challenges persist in obtaining modal damping. As an alternative, the rich spatial dynamic information of the mode shape-based method renders it highly suitable for the issue of detecting damage [13,14]. Detecting irregularities in mode shapes of damaged structures enables precise damage localization.

The measurement of mode shapes is the crucial step in the application of mode shape-based methods, exerting a direct influence on the accuracy of damage localization. Currently, ways to acquire mode shapes mainly fall into two categories: contact and non-contact methods. Contact methods necessitate the mounting of sensors, such as accelerometers and strain gauges, on the structure’s surface for vibration signal capture [15,16,17,18]. Although contact sensors are generally reliable, they cause undesired changes to the dynamics of lightweight structures, leading to mass-loading effects [19]. Moreover, conventional contact approaches provide only sparse and discrete monitoring points, resulting in low spatial measurement resolution, which is typically inadequate for mode shape-based damage detection [20]. It should also be recognized that long-term monitoring of large-scale structures using contact methods is a time-consuming and labor-intensive task.

In contrast, non-contact vibration measurement techniques like the scanning laser vibrometer (SLV) and vision-based methods offer high-spatial-resolution measurements without sensors attached to the structures. Yang et al. [21] introduced a damage detection method that integrates multiple indices and validated it using mode shapes captured by SLV. Guo et al. [22] used SLV to measure mode shapes and mapped them into the difference of Gaussian space for damage detection. Nevertheless, such devices are costly and tend to be time-consuming when performing high-resolution spatial measurements due to the need for sequential operation [23,24]. As an alternative, vision-based methods provide cost-effective and efficient high-spatial-resolution measurements [25,26]. Yan et al. [27] proposed a framework based on the digital image correlation (DIC) technique for cable force determination aided by mode shapes. Baqersad et al. [28] used the three-dimensional point tracking (PT) technique to extract full-field mode shapes of a wind turbine rotor. Despite the growing research on vision-based SHM [29,30,31], noteworthy insufficiencies persist. Existing vision-based methods, such as DIC and PT, rely excessively on high-quality speckle patterns or high-contrast markers on the surface. This necessitates meticulous surface preparation [32,33], posing difficulties for application on inaccessible or large structures. Furthermore, vision-based methods are susceptible to measurement noise due to the influence of image resolution and the inherent constraints of the optoelectronic system, thus impacting the precision of damage detection.

The aim of this research is to propose a novel vision-based damage detection method that integrates multi-scale signal analysis with data fusion algorithms. The method employs an advanced computer vision algorithm named phase-based optical flow estimation to arrange virtual sensors across the structure. Notably, optical flow represents the motion pattern of objects within an image sequence over time. Phase-based optical flow estimation provides full-field vibration information in sub-pixel accuracy under marker-free conditions. In light of the fact that the damage will lead to localized chaotic changes, a novel damage index, the multi-scale local information entropy (MS-LIE), is introduced to effectively characterize the complexity of mode shapes in Gaussian multi-scale space while mitigating measurement noise. Moreover, according to the assumption that damage feature spatial distribution remains consistent across all scales, a data fusion technique grounded in Dempster-Shafer (D-S) evidence theory is developed to fuse the damage information at all scales, ultimately eliminating the noise-induced uncertainty and enabling the precise determination of the damage location.

The remainder of the paper is organized as follows: Section 2 introduces the workflow and the theory of the proposed method. Section 3 numerically investigates the efficiency and robustness of the proposed method under a range of noise levels and damage states. Section 4 demonstrates experimental verifications on multiple damage detection scenarios. Section 5 draws the conclusions and suggests avenues for future research.

2. Methodology

The workflow of the proposed damage detection method is presented in Figure 1. Steps 1 and 2, combined, form the basis of the vision-based measurement. Utilizing the vibration video captured in step 1, step 2 adopts phase-based optical flow estimation algorithm to acquire vibration signals and identifies high-spatial-resolution mode shapes of the structure through blind source separation (BSS) and independent component analysis (ICA). Step 3 involves the extraction of multi-scale damage features, with the acquired mode shapes being mapped into Gaussian multi-scale space to suppress the measurement noise. Local information entropy (LIE) is computed at each scale to characterize local damage-induced irregularities in mode shapes, resulting in the novel damage index MS-LIE, which manifests multi-scale damage information. Step 4 employs a data fusion algorithm based on D-S theory to fuse the damage information at all scales, thereby facilitating a global assessment of the damage state and enabling a definitive identification of the damage location.

2.1. High-Spatial-Resolution Mode Shapes via Phase-Based Optical Flow Estimation

According to the fact that the video recording process is a projection of structural motion onto the image plane, structural vibration can be measured from video records by analyzing the temporally displaced image intensity $I\left(x+u\left(x,y,t\right),y+v\left(x,y,t\right)\right)$ across frames [34]. In this context, x and y represent pixel coordinates, while u and v denote the optical flow, which incorporates spatially localized and time-varying vibration signals in the x and y directions, respectively. Recent research has shown that optical flow estimation based on phase possesses several appealing properties, including insensitivity to lighting changes and calculation stability [35]. The flow diagram for phase-based optical flow estimation is presented in Figure 2.

Based on Fourier’s shift theorem, which states that a global motion in the time domain corresponds to a related linear phase shift in the frequency domain, local phase information gives way to local motion estimation. In this paper, we use complex Gabor quadrature filters [36,37] to acquire the local phase. The mathematical expression for a 2D Gabor filter is given by

$$g(x,y,\lambda ,\theta ,\psi ,\sigma ,\gamma )=\mathit{exp}(-{\displaystyle \frac{x^{\text{'} 2}+{\gamma }^2{y}^{\text{'} 2}}{2{\sigma }^2}})\mathit{exp}\left(i(2\pi {\displaystyle \frac{x^{\text{'}}}{\lambda }}+\psi )\right)=G_{\theta }^{2D}+\mathrm{i} H_{\theta }^{2D}$$

(1)

In the general form of the 2D Gabor filter in Equation (1), i is the unit imaginary number, x and y are the independent spatial variables, λ represents the wavelength of the sinusoid, θ specifies the orientation of the parallel stripes of the Gabor kernels, ψ is the phase offset, σ is the standard deviation of the Gaussian function modulating the sinusoid, γ is the spatial aspect ratio that determines the ellipticity of the Gabor kernels, $x^{\text{'}}=x\mathit{cos}\theta +y\mathit{sin}\theta$, and $y^{\text{'}}=-x\mathit{sin}\theta +y\mathit{cos}\theta$. For simplicity, the 2D Gabor filter can also be expressed as a real part and an imaginary part, i.e., $G_{\theta }^{2D}$ and $H_{\theta }^{2D}$. As an example, the real and imaginary pairs for 2D Gabor filters in four varying orientations are shown in Figure 3.

For a motion video with image intensity I (x, y, t) at spatial location (x, y) and time t, the local image amplitude and phase in the orientation θ can be derived by convolving the image with a complex 2D Gabor kernel:

$$A_{\theta }\left(x,y,t\right)e^{i{\varphi }_{\theta }\left(x,y,t\right)}=(G_{\theta }^{2D}+i H_{\theta }^{2D})\otimes I\left(x,y,t\right)$$

(2)

where $A_{\theta }$ is the local image amplitude, ${\varphi }_{\theta }$ is the local image phase, and $\otimes$ represents the convolution operation.

It has been demonstrated that the constant contours of the local phase through time provide a good approximation to the motion field and the displacement signal [38], which can be written as

$${\varphi }_{\theta }\left(x,y,t\right)=c,\hspace{1em}c\in R$$

(3)

for some constant c. Differentiating Equation (3) with respect to time t

$$\left({\displaystyle \frac{\partial {\varphi }_{\theta }\left(x,y,t\right)}{\partial x}},{\displaystyle \frac{\partial {\varphi }_{\theta }\left(x,y,t\right)}{\partial y}},{\displaystyle \frac{\partial {\varphi }_{\theta }\left(x,y,t\right)}{\partial t}}\right) \left(u,v,1\right)=0$$

(4)

where u and v represent the pixel shift in the x and y directions, respectively, known as the optical flow described above. When it comes to θ = 0 and θ = 90°, it is approximately the case that ${\displaystyle \frac{\partial {\varphi }_0}{\partial y}}\approx 0$ and ${\displaystyle \frac{\partial {\varphi }_{90°}}{\partial x}}\approx 0$, and the optical flow can be calculated as follows:

$$u=-{\left({\displaystyle \frac{\partial {\varphi }_0\left(x,y,t\right)}{\partial x}}\right)}^{-1}{\displaystyle \frac{\partial {\varphi }_0\left(x,y,t\right)}{\partial t}}, v=-{\left({\displaystyle \frac{\partial {\varphi }_{90°}\left(x,y,t\right)}{\partial y}}\right)}^{-1}{\displaystyle \frac{\partial {\varphi }_{90°}\left(x,y,t\right)}{\partial t}}$$

(5)

The displacement signal in the time domain is acquired by computing the optical flow between consecutive frames and the initial frame. Since each tracked pixel point can serve as a measurement point, phase-based optical flow can achieve high-spatial-resolution vibration measurements. Following capturing the vibration responses, the BSS and ICA techniques are employed to identify the high-spatial-resolution mode shape parameters solely based on the knowledge of vibration signals, without requiring excitation information. Further details on the BSS and ICA are available in [39], which includes an in-depth explanation and derivation of the techniques.

2.2. A Novel Damage Index: MS-LIE

Mode shape-based damage detection can be classified into two principal categories: change ratio analysis and singularity analysis. Singularity analysis is a reference-free damage detection approach that can detect local damage without the requirement for baseline data, and it is this approach that is attracting increasing attention. The damage-induced singularity in mode shapes or their spatial derivatives is analyzed to localize the damage. Traditionally, mode shape curvature has been selected as a sensitive feature to reflect damage locations sensitively. Nevertheless, the direct second-order difference operation on the mode shape renders it susceptible to measurement noise. Emerging signal processing tools such as the fractal dimension-based method [20] have been introduced, which aim to give a quantitative evaluation of damage-induced singularity in mode shapes. However, the inherent conflict between the detection sensitivity and noise immunity of damage indices cannot be adequately resolved, particularly for damage detection in noisy environments. Thus, inspired by the concept of entropy, this work introduces a multi-scale perspective to construct a novel damage index: MS-LIE.

The concept of entropy originated in the field of thermodynamics as a measure of the degree of chaos present within a given system and was later incorporated into statistics and information theory, where it is used to quantify the uncertainty associated with information. As the mode shape is a waveform signal source, its complexity can also be assessed in terms of entropy to reveal damage.

According to information coding theory [40], the information space of an information source X comprising r components can be formulated as

$$\left[\begin{array}{l}X\\ P\end{array}\right]:\left\{\begin{array}{l}X:a_1, a_2, ..., a_r\\ P\left(X\right):p_1, p_2, ..., p_r\end{array}\right.$$

(6)

where X = (a₁, a₂, …, a_r) is the symbol of information source, P = (p₁, p₂, …, p_r) is the discrete probability distribution, which satisfies 0 ≤ p_i ≤ 1 and ∑ p_i = 1 (i = 1, 2, …, r).

The self-information I(a_i) quantifying the total information in a_i can be calculated as

$$I\left(a_i\right)={\mathit{log}}_2{\displaystyle \frac{1}{p_i}}=-{\mathit{log}}_2{p}_i$$

(7)

Equation (7) demonstrates that the self-information is uniquely determined by the priori probability. As the probability approaches 0, the uncertainty increases and the information content becomes more significant. Furthermore, to characterize the information of the entire source X, the information entropy H(X) is put forward, defined as the mathematical expectation of self-information, that is

$$H\left(X\right)=I\left(a_1\right)p\left(a_1\right)+I\left(a_2\right)p\left(a_2\right)+...+I\left(a_r\right)p\left(a_r\right)=-\sum _{i=0}^r{p}_i{\mathit{log}}_2{p}_i$$

(8)

In the context of damage detection, each measurement point can be conceptualized as a component of the information source. Considering the localized nature of structural damage, a sliding window strategy is introduced to calculate the local probability, which reveals the local feature of the mode shapes rather than the overall characteristics:

$$p_i={\displaystyle \frac{y_i}{{\sum }_{j=i-\frac{S-1}{2}}^{i+\frac{S+1}{2}}{y}_j}}$$

(9)

where y_i represents the obtained mode displacement of the i-th measurement point and S is the size of the sliding window.

The process of calculating the local probability using a sliding window is depicted in Figure 4. By substituting Equation (9) into Equation (8), the information entropy gains the local sensitivity performance. Thus, the LIE for each measurement point is formulated as

$$\mathrm{LIE}\left(i\right)=-{\displaystyle \frac{y_i\left({\mathit{log}}_2{y}_i-{\mathit{log}}_2{\sum }_{j=i-\frac{S-1}{2}}^{i+\frac{S+1}{2}}{y}_j\right)}{{\sum }_{j=i-\frac{S-1}{2}}^{i+\frac{S+1}{2}}{y}_j{\mathit{log}}_{2}S}}$$

(10)

In the healthy state of the structure, the mode shapes exhibit geometric smoothness, and the amplitude of the LIE remains small. Conversely, damage to the structure results in local irregularities within the mode shapes, which manifest as an increase in the amplitude of the LIE. Despite its sensitivity in detecting local damage, the LIE is not sufficiently robust against the measurement noise, making the direct application of the damage detection method without modifications a challenging endeavor. To suppress the measurement noise, this paper utilizes the multi-scale space theory for discrete signals [41], mapping the mode shape into Gaussian multi-scale space:

$$Y_{\sigma }\left(x\right)=G_{\sigma }\left(x\right)\otimes y\left(x\right)$$

(11)

where Y_σ is the multi-scale representation of the mode shape, G_σ is the variable-scale Gaussian kernel, the subscript σ represents the scale parameter, and y(x) is the obtained mode shape.

The form of variable-scale Gaussian kernel is written as

$$G_{\sigma }\left(x\right)={\displaystyle \frac{1}{2\pi {\sigma }^2}}{e}^{-{\displaystyle \frac{x^2}{2{\sigma }^2}}}$$

(12)

Utilizing the mapping of the mode shape as described by Equations (11) and (12), the LIE of measurement points across all scales forms a novel damage index, termed MS-LIE, within the Gaussian multi-scale space. The MS-LIE provides insight into multi-scale damage features, calculated as

$$\mathrm{MS}\text{-}\mathrm{LI}{\mathrm{E}}_{\mathsf{\sigma }}(i)=-{\displaystyle \frac{Y_{\sigma }(i)({\mathit{log}}_2{Y}_{\sigma }(i)-{\mathit{log}}_2{\sum }_{j=i-\frac{S-1}{2}}^{i+\frac{S+1}{2}}{Y}_{\sigma }(j))}{{\sum }_{j=i-\frac{S-1}{2}}^{i+\frac{S+1}{2}}{Y}_{\sigma }\left(j\right) {\mathit{log}}_{2}S}}$$

(13)

where MS-LIE_σ(i) is the local information entropy of the i-th measurement point at the σ scale. In particular, where there are closely spaced multiple damages, the scale parameter should be set not exceeding the damage spacing, to obtain better detection results. In this case, the scale step size can be reduced to obtain more damage evidence in a limited scale range. On the other hand, for high noise scenarios, the scale parameter can be set appropriately high to obtain better noise robustness.

To compare the damage characteristics for different scales, the MS-LIE can be processed by a normalization method:

$$\mathrm{MS}\text{-}\mathrm{LI}{\mathrm{E}}_{\mathsf{\sigma }}^{\mathrm{norm}}(i)={\displaystyle \frac{\mathrm{MS}\text{-}\mathrm{LI}{\mathrm{E}}_{\mathsf{\sigma }}\left(i\right)-\mathit{min}\left(\mathrm{MS}\text{-}\mathrm{LI}{\mathrm{E}}_{\mathsf{\sigma }}\right(i\left)\right)}{\mathit{max}\left(\mathrm{MS}\text{-}\mathrm{LI}{\mathrm{E}}_{\mathsf{\sigma }}\right(i\left)\right)-\mathit{min}\left(\mathrm{MS}\text{-}\mathrm{LI}{\mathrm{E}}_{\mathsf{\sigma }}\right(i\left)\right)}}$$

(14)

2.3. Data Fusion for Multi-Scale Damage Information

In the above-established Gaussian multi-scale space, the detection of local singularity peaks with MS-LIE at a specific scale has the potential to identify the damage in theory. However, in practical applications, the selection of scale parameters presents a challenging task. At a smaller scale, the mode shape exhibits a pronounced damage-induced singularity, which is detectable yet may be obscured by noise interference. Conversely, at a larger scale, noise is markedly reduced, but the damage-induced singularity diminishes due to the mode shape’s increased smoothness. To address these deficiencies, in this study, a data fusion algorithm based on D-S evidence theory is developed to fuse the MS-LIE at all scales, which eliminates noise-induced uncertainty and leads to the final determination of damage locations.

Let Ω be a finite set called frame of discernment, whose physical significance is the set of measurement points at which damage may occur:

$$\Omega =\left\{e_1, e_2, ..., e_N\right\}$$

(15)

where e_i (i = 1, 2, …, N) represents measurement points composing the structure, and N is the number of measurement points. All damage localization results can be represented as a subset of Ω, and 2^Ω is defined as the power set including all the subsets of Ω and empty set ($\varnothing$). Furthermore, the basic probability m is assigned in the mapping of the power set:

$$m:2^{\Omega }\to \left[0, 1\right]$$

(16)

where m denotes the basic probability distribution, which must ensure that the assigned probabilities sum to unity and there is no belief in the empty set:

$$\sum _{A\subseteq \Omega }m\left(A\right)=1,\hspace{1em}m\left(\varnothing \right)=0$$

(17)

Considering that the MS-LIE at each scale should be valid evidence of damage localization, the basic damage probability of the i-th measurement point at each scale is defined as

$$m_{\sigma }\left(i\right)={\displaystyle \frac{\mathrm{MS}\text{-}\mathrm{LI}{\mathrm{E}}_{\mathsf{\sigma }}^{\mathrm{norm}}\left(i\right)}{{\sum }_{j=0}^N\mathrm{MS}\text{-}\mathrm{LI}{\mathrm{E}}_{\mathsf{\sigma }}^{\mathrm{norm}}\left(j\right)}}$$

(18)

where m_σ is an N-dimensional vector containing the damage probability distribution at the scale σ.

According to the D-S evidence theory [42], the basic probability assignments can be combined. The combination rule for the two basic damage probability distributions, i.e., m₁ and m₂, is shown as follows:

$$F\left(i\right)={\displaystyle \frac{{\sum }_{j\cap k=i}{m}_1\left(j\right)m_2\left(k\right)}{1-{\sum }_{j\cap k=\varnothing }{m}_1\left(j\right)m_2\left(k\right)}}$$

(19)

where F denotes the fused probability distribution.

Based on Equation (19), the damage probability distributions of two adjacent scales are fused, starting at the lowest scale and progressing upwards to the highest. This fusion operation continues until the final damage probability distribution is obtained, as shown in Figure 5.

Without loss of generality, and briefly, the data fusion process is represented as

$$F^n\left(i\right)={\displaystyle \frac{{\sum }_{j\cap k=i}{F}^{n-1}\left(j\right)m_{{\sigma }_{n+1}}\left(k\right)}{1-{\sum }_{j\cap k=\varnothing }{F}^{n-1}\left(j\right)m_{{\sigma }_{n+1}}\left(k\right)}}\hspace{1em}\left(n\ge 2\right)$$

(20)

where Fⁿ denotes the fused damage probability distribution after n times of data fusion, and σ_n represents the n-th scale. With the above steps, the imprecision in damage localization caused by measurement noise is eliminated, and the damage detection can be user independent.

3. Numerical Investigations of the Methodology

It is worth noting that the proposed method is based on a combination of vision-based measurements and entropy-based modal singularity analysis, which is fundamentally not limited to specific structural forms. Therefore, considering the beam as the most common and fundamental structure in industrial scenarios, the beam structure will be selected as the object of study to validate the proposed method in the following.

3.1. Analytical Model of Cracked Beam

Figure 6a illustrates the physical system under investigation, which is a cantilever beam of length L and uniform rectangular cross-section b × h. There are n cracks in the beam, with the i-th (i = 1, 2, …, n) crack located at L_d,i from the left clamped end of the beam. It is presumed that the cracks are open and have a uniform depth h_d,i. Utilizing the linear elastic fracture mechanics principle, the transverse crack is modeled as a rotational spring [43]. This results in the cracked beam being divided into n + 1 segments via n springs. The corresponding analytical model of the cracked beam is presented in Figure 6b.

During the numerical analysis, as only bending vibrations of the thin beam are considered, it is assumed that the rotational spring constant governs the local flexibility matrix. The stiffness of the rotational spring is given by Cao et al. [44] as

$$K_{t,i}={\displaystyle \frac{EI}{5.346hJ\left(h_{d,i}/h\right)}}$$

(21)

where E is the modulus of elasticity of the beam, I is the area moment of inertia for the beam cross-section, and J(h_d,i/h) is the dimensionless local compliance function.

To be brief, we define dimensionless parameters, relative crack size α_i and relative crack location β_i, to describe damages as

$${\alpha }_i=h_{d,i}/h,\hspace{1em}{\beta }_i=L_{d,i}/L$$

(22)

According to the Euler-Bernoulli beam theory, the governing equations for the free vibrations of beam segments can be written as

$$EI{\displaystyle \frac{{\partial }^4{w}_i\left(x,t\right)}{\partial x^4}}+\rho A{\displaystyle \frac{{\partial }^2{w}_i\left(x,t\right)}{\partial t^2}}=0$$

(23)

where ρ is the mass density, A is the cross-section area, and w_i is the transverse deflection shape of the i-th beam segment. By using the method of separation of variables, the general solution of Equation (23) can be obtained as

$$w_i\left(x\right)=c_{i,1}\mathit{cosh}\left(\lambda x\right)+c_{i,2}\mathit{sinh}\left(\lambda x\right)+c_{i,3}\mathit{cos}\left(\lambda x\right)+c_{i,4}\mathit{sin}\left(\lambda x\right)$$

(24)

with λ⁴ = ω²ρA/EI, where ω is the vibration angular frequency, and c_i,1 to c_i,4 are constant coefficients determined from the boundary conditions. The boundary conditions at the clamped and free ends are denoted by

$$\left\{\begin{array}{l}{w}_1\left(x\right)\left|x=0\right.=0,{\displaystyle \frac{\partial w_1\left(x\right)}{\partial x}}\left|x=0=0\right.\hspace{1em}\hspace{1em}\left(the clamped end\right)\\ {\displaystyle \frac{{\partial }^2{w}_{n+1}\left(x\right)}{\partial x^2}}\left|x=L=0,{\displaystyle \frac{{\partial }^3{w}_{n+1}\left(x\right)}{\partial x^3}}\left|x=L=0 \left(the free end\right)\right.\right.\end{array}\right.$$

(25)

Considering the connection between the two beam segments, constraints can be established to ensure the continuity of displacement, bending moment, and shear. In addition, the condition of equilibrium between the transmitted bending moment and the rotation of the spring should be imposed. Thus, the boundary conditions at the crack are written as follows:

$$\left\{\begin{array}{l}{w}_i\left(x\right)\left|x=L_{d,i}=w_{i+1}\left(x\right)\left|x=L_{d,i}\right.\right.\\ {\displaystyle \frac{{\partial }^2{w}_i\left(x\right)}{\partial x^2}}\left|x=L_{d,i}\right.={\displaystyle \frac{{\partial }^2{w}_{i+1}\left(x\right)}{\partial x^2}}\left|x=L_{d,i}\right.\\ {\displaystyle \frac{{\partial }^3{w}_i\left(x\right)}{\partial x^3}}\left|x=L_{d,i}\right.={\displaystyle \frac{{\partial }^3{w}_{i+1}\left(x\right)}{\partial x^3}}\left|x=L_{d,i}\right.\\ -EI{\displaystyle \frac{{\partial }^2{w}_i\left(x\right)}{\partial x^2}}\left|x=L_{d,i}=K_{t,i}\left({\displaystyle \frac{\partial w_i\left(x\right)}{\partial x}}\left|x=L_{d,i}-{\displaystyle \frac{\partial w_{i+1}\left(x\right)}{\partial x}}\left|x=L_{d,i}\right.\right.\right)\right.\end{array}\right.$$

(26)

Based on the boundary conditions described above, the resulting characteristic equation of the system is solved numerically, and the mode shape of each beam segment is obtained. For other constraints, such as pinned cracked beams, the solution can be performed similarly by changing the boundary conditions at both ends.

3.2. Noise Immunity Evaluations

3.2.1. Evaluations on Single Damage

A steel cantilever beam with single damage is first considered. The parameters of this beam are as follows: mass density ρ = 7800 kg/m³, elastic modulus E = 210 GPa, Poisson’s ratio μ = 0.32, length L = 600 mm, width b = 25 mm, and thickness h = 10 mm. A crack characterized by dimensionless parameters α = 0.3 and β = 0.4 is introduced into the beam. To perform the noise immunity evaluation, white Gaussian noise (WGN) is introduced for this purpose. Once the mode shapes are obtained through the above analytical model, the WGN is added to the pure mode shapes, resulting in different noisy environments. The noise level is assessed by the signal-to-noise ratio (SNR). The selection of SNR in this paper is mainly based on empirical values for vision-based measurements, which is typically between 50 dB and 100 dB in our previous laboratory measurements. We have therefore considered two scenarios: one with lower noise level (SNR = 90 dB) and another with higher noise level (SNR = 60 dB).

In the simulated scenario with an SNR of 90 dB, the MS-LIE at different scales obtained from the first, second, and third mode shapes are shown in Figure 7a, Figure 7b, Figure 7c, respectively, with the Gaussian multi-scale space spanning a scale range of 2–10 and a step size of 0.1. The corresponding damage detection results are demonstrated in Figure 7d–f. The figures reveal that the MS-LIE exhibits sharp amplitude changes at the location corresponding to a damage-induced singularity in the mode shape. The MS-LIE heat map derived from the first mode shape shows some undesired local amplitude fluctuations due to noise. However, as the order of the adopted mode shape increases, the MS-LIE heat maps become progressively smoother and more robust to noise. Furthermore, with the aid of data fusion, the precise location of the damage is identified at the peak of the final damage probability distribution.

In the simulated scenario with an SNR of 60 dB, the MS-LIE across the scale range of 2–10 is displayed in Figure 8a–c. The corresponding damage detection results are shown in Figure 8d–f. An increase in the level of measurement noise results in more chaotic mode shapes, thereby making the noise-induced interference more visible. Specifically, a series of linear sidelobes appeared in the MS-LIE heat maps of the first and second mode shapes, posing challenges in detecting damage at a single scale. The larger scale parameter can reduce this phenomenon of noise interference enhancement, but it leads to increased ambiguity in locating the damage. Nevertheless, since the data fusion technique developed in this paper harnesses damage evidence at all scales, the final damage probability distributions obtained still accurately indicate damage.

3.2.2. Evaluations on Double Damages

This study considers a steel pinned beam with two damages. The structural geometry and material parameters are identical to those used in the single damage simulation. Two cracks, each characterized by dimensionless parameters α₁ = 0.3, β₁ = 0.2 and α₂ = 0.4, β₂ = 0.8, are introduced into the beam. Similarly, the noise is assigned as WGN and added to the mode shapes to simulate noisy environments at varying SNRs.

In the simulated scenario with an SNR of 90 dB, Figure 9a–c illustrates the MS-LIE at various scales obtained from the first, second, and third mode shapes, respectively, where the multi-scale space is established on the scale range of 2–10 and a step size of 0.1. The corresponding damage detection results are presented in Figure 9d–f. The presence of multiple damage does not affect the sharp change in the amplitude of the MS-LIE at each damage location, and the MS-LIE heat map of the higher order mode shape is smoother and more robust to noise. Using the proposed data fusion technique, the precise locations of both damages are determined correctly at the peaks of the final damage probability distribution.

In the simulated scenario with an SNR of 60 dB, the MS-LIE at the scale range of 2–10 is displayed in Figure 10a–c. The corresponding damage detection results obtained from data fusion are shown in Figure 10d–f. The MS-LIE heatmaps indicate that detecting both damages at a single scale does face a significant hurdle due to the high noise level. It is possible to suppress the noise by increasing the scale, which allows the features at the damage location to remain discernible. While the first mode shape is relatively weakly robust to noise, resulting in the omission of one slight damage, the method based on MS-LIE and data fusion can still pinpoint both damages based on the second and third mode shape data.

Based on the above numerical research, the proposed method based on MS-LIE and data fusion exhibits robustness when detecting slight damage in various noisy environments. Moreover, the method does not rely on direct difference analysis with the original healthy data, and is capable of concurrently revealing single and multiple damages without requiring baseline data as a reference.

3.2.3. Compared with Other Methods

To demonstrate the advantages of the proposed method, we compare the performance of the method to two alternative damage detection methods, namely the fractal dimension-based method [20] and the deep learning-based method [10]. The beam specimen used for comparison is the same as mentioned above, based on the 2nd mode shape in the noisy environment with the SNR of 60 dB. The detection results of the three methods are shown in Figure 11 and Figure 12. Although the other two methods can detect damage, some drawbacks still exist. The fractal dimension-based method is heavily disturbed by noise. While local mutations in the fractal dimension can offer a means to detect damage with a high degree of sensitivity, this property is susceptible to interference from noise, resulting in the misidentification of noise as damage features. The deep learning-based method is effective in overcoming noise interference, but the end-to-end regression prediction task makes its results still fluctuate to some degree in non-damaged regions, which complicates the determination of the threshold of damage. Moreover, the implementation of the deep learning-based method necessitates the generation of extensive datasets and the training of complex models, which imposes an additional burden on practical applications. In contrast, the proposed method has the capacity to detect the damage directly and accurately without the aforementioned drawbacks.

Monte Carlo experiments are performed to statistically compare the detection accuracy of the methods. Considering the trade-off between simulation accuracy and computational resource consumption, a convergence analysis has been performed and the results tend to stabilize after the sample size reached 2000, suggesting that this sample size is sufficient for this specific case. Therefore, the Monte Carlo sample size is set to 2000, adhering to Chebyshev’s law. A broad range of SNRs (50 dB–100 dB) is considered with the damage degree maintained at a constant level ($\alpha =0.2$). Two damage cases are studied: single damage and multiple damages (with 2 to 4 damages at random). The results are presented in Figure 13, which demonstrate that the proposed method has better noise immunity compared to the other two methods, especially in the cases with multiple damages.

3.3. Ablation Study

To assess the influence of individual components on the proposed method’s damage detection performance, the ablation study is conducted by sequentially removing specific components. The contribution of each component to the method can be thoroughly analyzed by examining the effect of these alterations on the accuracy of damage detection. Removing the multi-scale analysis component and the entropy analysis component degrades the proposed method to LIE and multi-scale mode shape curvature (MS-MSC), while removing both degrades the proposed method to the original mode shape curvature (MSC). Monte Carlo experiments are performed. Firstly, a broad range of SNRs (50 dB–100 dB) is considered to investigate the noise robustness in the ablation study, with the damage degree maintained at a constant level ($\alpha =0.2$). Two damage cases are studied: single damage and multiple damages (with 2 to 4 damages at random), with the results presented in Figure 14.

At high SNRs, all four methods achieve high detection accuracy due to less noise interference. Incorporating entropy analysis enhances the sensitivity of MS-LIE and LIE to damage, leading to marginally higher accuracy compared to MS-MSC and MSC. With the rise in noise level, the accuracy of LIE and MSC decreases significantly, with LIE experiencing an especially steep drop. This indicates that LIE and MSC are less robust against noise interference. The LIE, with entropy analysis, is more sensitive to damage; however, this sensitivity also extends to noise. In contrast, MS-LIE and MS-MSC consistently retain higher detection accuracy, implying that the multi-scale analysis component is crucial for enhancing the noise immunity of the proposed method.

Damage degrees over a wide range (0.1–0.3) are also considered to assess the detection sensitivity in the ablation study, while maintaining a constant SNR (SNR = 90 dB). Two damage cases are investigated, including a single damage case and a multiple damage case (with 2 to 4 damages at random), whose results are shown in Figure 15. With noise conditions maintained at a favorable level, the detection accuracy of MS-LIE and LIE is consistently superior to that of MS-MSC and MSC, and this gap increases significantly as the damage degree decreases. This demonstrates that the entropy analysis component renders the proposed method markedly more sensitive to damage. The proposed method integrates the multi-scale analysis component and the entropy analysis component, accounting for both the detection sensitivity and noise immunity, thus exhibiting superior performance.

3.4. Computational Complexity Analysis

A computational complexity analysis of the proposed damage detection method is performed. Considering that the computational consumption is mainly focused on the cascading convolution operations during the multi-scale mapping process, the variation of the average computational time with the total number of scales and the total number of measurement points is investigated. Firstly, the total number of scales is varied from 10 to 100, while the total number of measurement points is fixed at 1000. Subsequently, the total number of measurement points is varied from 100 to 1000, while the total number of scales is maintained fixed at 100. The computations are performed on the same terminal (Intel^® i5-10500 CPU) and the results of the computational complexity analysis are shown in Figure 16.

The average calculation time increases almost linearly with the total number of scales and measurement points, but the linear increase is not significant. For the strict case where the total number of scales is 100 and the total number of measurement points is 1000, the average calculation time is 0.7961 s, which is still within the acceptable range for practical real-time applications.

4. Experimental Verifications of the Methodology

This section presents experiments to further verify the proposed damage detection method, divided into two parts. The first part assesses the accuracy of vibration signals obtained via phase-based optical flow estimation by comparing the vision-based approach with the laser vibrometer, focusing on displacement, frequency, and mode shapes. The second part validates the effectiveness of the proposed damage detection method through two experiments. The experiments encompass scenarios involving the detection of single damage and double damages.

4.1. The Accuracy of Vibration Measurement

The schematic diagram of the vision-based experimental system is shown in Figure 17a, while the set-up for the experimental platform is illustrated in Figure 17b. The experimental system primarily comprises a high-speed camera, a laser vibrometer, a pair of lamps, a vibration test rig and a laptop. The cantilever beam employed in the experiment is made of aluminum alloy 6061 with dimensions of 400 × 40 × 5 mm, whose mass density, elastic modulus and Poisson’s ratio are 2700 kg/m⁻³, 69 GPa, and 0.33, respectively. During the experiment, we capture the digital video of the structural vibration at 3000 frames per second (fps) using the high-speed camera (Revealer 5F04, full resolution 2320 × 1718 pixels, pixel size 7 × 7 μm, maximum frame rate 52,800 fps, responsivity ISO 6400). Figure 17c displays a screenshot of the motion video. As a reference, the laser vibrometer (Keyence LK-H155, resolution 0.1 μm, maximum sampling frequency 390 kHz) is used to acquire the response of the vibrating beam.

The comparison of displacement responses obtained from the vision-based system (the scale factor between the image plane and the beam’s physical plane is 31.6 pixels per millimeter) and the laser vibrometer is shown in Figure 18. To enable quantitative analysis of the similarity between the two time-domain responses, we introduce the Pearson correlation coefficient (PCC), calculated as

$$\mathrm{PCC}\left(x_V,x_L\right)={\displaystyle \frac{{\sum }_{t=1}^T\left(x_V\left(t\right)-{\overline{x}}_V\right)\left(x_L\left(t\right)-{\overline{x}}_L\right)}{\sqrt{{\sum }_{t=1}^T{\left(x_V\left(t\right)-{\overline{x}}_V\right)}^2}\sqrt{{\sum }_{t=1}^T{\left(x_L\left(t\right)-{\overline{x}}_L\right)}^2}}}$$

(27)

where x_V(t) and x_L(t) are the responses at time t obtained from the vision-based system and the laser vibrometer, respectively, and T is the calculation period.

A PCC value approaching 1 suggests a strong correlation between the two data. It is worth noting that the laser vibrometer operates at a higher sampling frequency than the video frame rate, resulting in inconsistent sample sizes. Thus, we down sample the laser vibrometer data to enable the calculation of PCC, yielding a PCC of 0.988. Furthermore, we use the fast Fourier transform (FFT) technology to extract the first three frequencies from the responses and compare these with the results of the Finite Element Method (FEM), as detailed in

Table 1. The comparison of frequencies obtained from the vision-based system, laser vibrometer, and FEM.

1st Frequency/Hz	FEM	33.96
	Laser	33.44
	Vision	33.36
2nd Frequency/Hz	FEM	213.32
	Laser	212.45
	Vision	212.55
3rd Frequency/Hz	FEM	597.48
	Laser	596.46
	Vision	596.02

. The findings confirm the accuracy of the vision-based system in vibration measurement, utilizing the phase-based optical flow algorithm.

Based on the captured vibration responses, the high-spatial-resolution mode shapes are identified by the BSS and ICA technique, which is shown in Figure 19a. To quantitatively assess the accuracy of the mode shapes obtained, we use the modal assurance criterion (MAC), defined as

$$\mathrm{MAC}\left({\phi }_V,{\phi }_F\right)={\displaystyle \frac{{‖{\phi }_V^T{\phi }_F‖}^2}{\left({\phi }_V^T{\phi }_V\right)\left({\phi }_F^T{\phi }_F\right)}}$$

(28)

where φ_V and φ_F are the mode shapes obtained from the vision-based system and FEM, respectively.

A high correlation between the two mode shapes is indicated by a MAC value close to 1. The bar chart in Figure 19b depicts the MAC values for the first three mode shapes, validating the strong correspondence between the mode shapes obtained from the vision-based system and those from the FEM. This means that the high-spatial-resolution mode shapes given by vision-based method are reliable.

4.2. Detection of Single Damage

The experiment to detect single damage is performed on an aluminum alloy 6061 beam specimen with a single crack. The open crack is introduced into the specimen via wire-electrode cutting, measuring 4 mm in length and 4 mm in depth. The specimen’s dimensions and the crack’s location are displayed in Figure 20a, and an image of the used specimen is shown in Figure 20b.

In this experiment, to achieve high-spatial-resolution mode shapes, we select 1000 pixels along the beam’s length for measurement, effectively deploying 1000 virtual sensors that do not impart mass loading effects on the beam. The first three mode shapes are obtained and depicted in Figure 21a–c. Despite the presence of damage in the structure, the mode shapes remain relatively smooth overall. This makes it challenging to detect damage through direct observation of the mode shapes.

The results of the proposed MS-LIE at different scales are shown in Figure 22a,c,e, with the Gaussian multi-scale space constructed using a scale range of 2–10 and a step size of 0.1. The corresponding enlarged heatmaps at larger scales are demonstrated in Figure 22b,d,f. At smaller scales, although the damage feature is more sensitive, the presence of strong noise causes the confusion of damage-induced singularities with noise-induced ones, which complicates the assessment. In the area of larger scales, the damage feature is more distinguishable, albeit at the cost of reduced localization resolution. To solve the scale selection dilemma, we use the proposed data fusion algorithm to fuse the damage features across multi-scale space, maximizing the use of damage evidence and eliminating the effects of measurement noise. The final damage localization results obtained from the first, second, and third mode shapes are demonstrated in Figure 23a–c. It is observable that the proposed method pinpoints single damage accurately.

4.3. Detection of Double Damages

For the detection of multiple damages, the experiment is performed on an aluminum alloy 6061 beam specimen featuring two open cracks. The two cracks are introduced into the specimen via wire-electrode cutting, with the length of 3 mm, 4 mm and the depth of 4 mm, 5 mm. The specimen’s dimensions and the cracks’ location are displayed in Figure 24a, and an image of the employed specimen is provided in Figure 24b.

For this experiment, we similarly select 1000 pixels along the beam’s length as virtual sensors to capture high-spatial-resolution mode shapes. The results of the MS-LIE are presented in Figure 25a,c,e, where the Gaussian multi-scale space is established by the scale range of 2–10 with the step 0.1. Enlarged heatmaps corresponding to the larger scales are displayed in Figure 25b,d,f. The behavior of the damage representation in the multiple damage scenario closely resembles that in the single damage case. With the aid of the proposed data fusion algorithm, the advantage of high localization sensitivity at small scales and the advantage of strong noise robustness at large scales are fully integrated. The final damage localization results using the first, second, and third mode shapes are presented in Figure 26a–c, showcasing the proposed method’s ability to clearly reveal multiple damages and achieve precise localization.

5. Conclusions

This study proposes a novel vision-based method for damage detection using multi-scale local information entropy and data fusion. We utilize an advanced phase-based optical flow algorithm to acquire high-spatial-resolution vibration signals. The damage location is precisely determined through the fusion of damage information extracted by the proposed MS-LIE across all scales. A series of numerical and experimental scenarios are designed to validate the method. According to the results of investigations, the following conclusions can be drawn:

(1)

The vibration signals obtained by phase-based optical flow estimation show good agreement with those from the laser vibrometer and FEM in terms of displacement, frequency, and mode shapes. This indicates that the vision-based method can deliver precise vibration measurements obviating the need for speckle patterns or high-contrast markers on the surface, and the high-spatial-resolution mode shapes provided by the vision-based method are reliable. The utilization of the phase-based optical flow estimation technique facilitates the implementation of a non-contact, non-destructive monitoring approach that can be applied to a diverse range of structures.

(2)

The novel damage index MS-LIE integrates the multi-scale analysis component and the entropy analysis component, addressing both the issue of detection sensitivity and noise immunity, thereby showcasing enhanced performance. The MS-LIE effectively reveals damage features in Gaussian multi-scale space, even in the presence of noise. Benefiting from utilizing damage evidence across all scales through data fusion technique, the proposed method demonstrates robustness in detecting damage under various noisy environments. The final damage probability distributions accurately identify instances of single and multiple damages without the necessity of a baseline data set as a reference.

(3)

The results of the ablation study have demonstrated the utility of entropy-based and multi-scale analysis-based approaches in SHM. It can inform future research by encouraging the exploration of other entropy measures for damage detection and promoting the integration of advanced multi-scale analysis-based data fusion techniques to further enhance detection capabilities.

Despite the laboratory success of this paper, there are some noteworthy limitations. On the one hand, vision-based measurements for large structures may suffer from the problem that the imaging area does not fully cover the structure. This needs to be solved by introducing partition measurements and mode shape splicing techniques. On the other hand, the structural geometrical complexity of the structure may cause overlap or obstruction problems in the imaging area. This needs to be addressed by further multi-camera measurements. Future work will aim to overcome these limitations and further verify the generalizability to more complex structures such as bridges, towers, or composite materials. Additionally, we will explore quantifying the sensitivity of key parameters for different systems and structural conditions.