1. Introduction
Structural damage detection (SDD) is one of the most relevant topics in structural health monitoring (SHM). Timely SDD is helpful for finding the potential defects of a structure and preventing its sudden collapse. The early detection methods are mainly on-site inspections, which are labor-intensive, time-consuming, and only effective for visible surface defects. The structural vibration contains real and complete state information of a structure [
1]; therefore, some vibration-based SDD methods are proposed. For example, the SDD methods are based on modal parameters and their derivatives (namely, the parametric method), including the natural frequencies [
2], mode shapes [
3], modal flexibility [
4], mode curvature [
5], and modal strain energy [
6,
7]. The non-parametric method establishes the SDD indicators directly from the real-time vibration signals, including acceleration [
8] and displacement [
9]. Among them, the real-time detection method based on eigen perturbation and a Kalman filter has been well confirmed [
10]. Although these methods have significantly improved the accuracy of the SDD, they still face many challenges. The parametric methods need accurate modal parameter identification, which may be compromised under the influence of many factors (measurement and/or analysis errors). Furthermore, a single modal-based indicator cannot cover all damage scenarios (e.g., the natural frequencies can only detect the existence of damage, but cannot determine the damage location) [
2]; meanwhile, the non-parametric methods require large-scale data analysis, which is affected by the knowledge level of analysts, and the accuracy and efficiency of damage detection are questionable. Even the popular Kalman filter method also needs both accurate structural modeling and external excitation, which will limit its application in real engineering [
10]. Therefore, an automatic and efficient data processing tool to integrate/fuse multiple information sources is urgently needed.
Machine learning (ML) methods provide a new way to solve the above difficulties. The ML enables a system to automatically learn from its experience and predict the corresponding scenario according to the learned knowledge. ML algorithms have been widely used in vibration-based SDD. Classical ML algorithms include the support vector machine (SVM) [
11] and artificial neural network (ANN) [
12], which have achieved encouraging results. In particular, the backpropagation (BP) neural network has been widely applied to the parametric and non-parametric SDD methods, for example, damage detection of a truss [
13], a steel frame [
14], and a bridge model [
15], and its effectiveness was also confirmed on a real steel frame [
16]. However, all the above methods need to extract a set of fixed features, e.g., the modal parameters and/or wavelet transform coefficients [
17], principal component analysis (PCA) [
18], and wavelet decomposition (WD) [
11]. Furthermore, the fully connected neural network (i.e., BP neural network) is prone to over-fitting and is computationally expensive, which will sacrifice the effectiveness of the method in large-scale SDD tasks.
As a deep learning algorithm, a convolutional neural network (CNN) provides a novel method for the SDD due to its excellent feature extraction ability. Meanwhile, a CNN has powerful computing performance and is able to prevent over-fitting due to its weight sharing (in the convolution process) and sparse connection (in the pooling process); it has unprecedented potential in the field of SDD. Zhong et al. [
19] demonstrated that a CNN can extract damage information from the mode shapes; Lin et al. [
20] also showed that a CNN can extract damage information directly from the acceleration signals, and Teng et al. [
21] illustrated a CNN feature extraction process in structural surface defect detection. The effectiveness of a 2-D CNN was demonstrated using numerical [
22] and experimental [
23] models of a benchmark structure by joining the data of 14 accelerometers. As an alternative, a 1-D CNN has attracted attention in electrocardiogram (ECG) detection, engine detection [
24], and voltage/current detection of electronic equipment [
25]. These studies confirmed the excellent performance of a 1-D CNN in damage detection. In the field of civil engineering, a 1-D CNN was used [
26] to detect damage in a laboratory frame, where its effectiveness was validated on the collected acceleration signals using a wireless sensor network (WSN) [
27]. Subsequently, the SDD method based on the vibration and 1-D CNN was also used to detect the mass changes of the real bridges [
28]. Although the CNN-based SDD methods achieved encouraging results, for practical engineering, especially for the long-span bridges, it is difficult to collect the complete bridge vibration information and arrange sufficient signal acquisition points. Therefore, although a CNN has a strong signal processing capability, the damage detection is affected by the non-synchronization and incompleteness of the vibration signals and the interference between multiple sensors. In order to obtain more complete damage information, a new data analysis strategy is necessary.
The strategy of data fusion provides a state-of-the-art SDD method. By fusing multi-channel/multi-scale information, the data fusion technology can provide complete and detailed object information. In medical engineering, computed tomography and magnetic resonance (CT-MR) image fusion can obtain a more accurate lesion location [
29]; in remote sensing image processing, image fusion technology can improve image resolution [
30]. In the field of SHM, the time domain and frequency domain images of the bridge vibration were fused to detect abnormal signals [
31], and the accuracy of damage detection was improved by fusing the modal strain energy (MSE) of multi-modes [
32] and the MSE with dynamic response [
33], and the Dempster–Shafer (D–S) evidence theory and multi-sensor-signals-based SDD method was also implemented [
34]. The damage indicators based on modal parameters and their derivatives need accurate modal identification from the original vibration signal and the accuracy is compromised by the accidental error of measurement and/or analysis. The popular Kalman filters can effectively eliminate the interference of noise [
35]; however, the structural parameter identification method based on eigen perturbation and a Kalman filter still faces many challenges: (1) it can only be used to identify time-invariant structural parameters [
36]; (2) for sub-component (location) damage detection of a structure, the accuracy and robustness need to be further improved [
37]; (3) it cannot be applied to a non-Gaussian parameter system [
38]; (4) there is a certain time delay [
39]; (5) low sampling frequency will affect the stability of the filter [
40]. These often lead to significant implementation difficulties. The vibration signals contain the complete structural state information [
41]; thus, it is of great potential to use the vibration signals as structural damage indicators. The information of a single sensor has a certain ability to detect the structural damage state [
42]; however, the influence of the sensor location on damage detection results is not clear, and the complementarity of multiple sensors is also a topic worthy of further study. The existing methods fuse the original data of multiple sensors as the input of a CNN (namely, data-level fusion); however, the collected signals may be unsynchronized and incomplete, and the signals of multiple sensors may have interference.
In order to further improve the accuracy of damage detection, one solution was to synthesize the information of multiple sensors and avoid mutual interference. In this study, a novel decision-level fusion strategy was applied to the SDD. That is, each acquisition point (accelerometer) was regarded as an independent observation unit. Each accelerometer signal was used to train a 1-D CNN, and the prediction results of the multiple CNN models were integrated to finally predict the structural damage state (decision-level fusion). This work was carried out on a numerical model and two experimental models; meanwhile, a control experiment (data-level fusion) was designed to highlight the advantages of the proposed method.
4. Conclusions
In this study, a 1-D CNN was employed to detect the damage of a bridge and a steel frame structure, and a novel fusion strategy (decision-level fusion) was used to fuse the prediction results of multiple CNNs, which significantly improved the accuracy of the SDD. Specifically, the vibration signal of each acquisition point was used to train a CNN, and the prediction results of these CNN models were fused.
Based on the above results, the following conclusions were drawn:
(1) The proposed fusion strategy (decision-level fusion) could significantly improve the prediction accuracy of the numerical model by 10% compared with the control experiment (data-level fusion).
(2) The proposed fusion strategy (decision-level fusion) was also validated in the experimental bridge model, and the accuracy was improved by 16% compared with the data-level fusion strategy in the control experiment. This was also confirmed regarding the damage detection of the long-span steel frame (improved by 30%).
(3) The proposed fusion strategy also performed better than any CNN trained by the signals of an individual acquisition point.
(4) The proposed method was more competitive than the D–S evidence theory and a Kalman filter.