Multi-Source Fusion Deformation-Monitoring Accuracy Calibration Method Based on a Normal Distribution Transform–Convolutional Neural Network–Self Attention Network

Abstract

In multi-source fusion deformation-monitoring methods that utilize fiber Bragg grating (FBG) data and other data types, the lack of FBG constraint points in edge regions often results in inaccuracies in fusion results, thereby impacting the overall deformation-monitoring accuracy. This study proposes a multi-source fusion deformation-monitoring calibration method and develops a calibration model that integrates vision and FBG multi-source fusion data. The core of this model is a normal distribution transform (NDT)–convolutional neural network (CNN)–self-attention (SA) calibration network. This network enhances continuity between points in point clouds using the NDT module, thereby reducing outliers at the edges of the fusion results. Experimental validation shows that this method reduces the absolute error to below 0.2 mm between multi-source fusion calibration results and high-precision measured point clouds, with a confidence interval of 99%. The NDT-CNN-SA network offers significant advantages, with a performance improvement of 36.57% over the CNN network, 14.39% over the CNN–gated recurrent unit (GRU)–convolutional block attention module (CBAM) network, and 9.54% over the CNN–long short term memory (LSTM)–SA network, thereby demonstrating its superior generalization, accuracy, and robustness. This calibration method provides smoother and accurate structural deformation data, supports real-time deformation monitoring, and reduces the impact of assembly deviation on product quality and performance.

1. Introduction

In modern manufacturing, small assembly deviations or errors caused by structural deformations can considerably impact product quality and performance [1]. Hence, the accuracy of deformation monitoring impacts not only the assembly efficiency, but also the optimization of product functionality and reliability [2]. Nowadays, there are different digital methods to monitor deformations, such as visual monitoring [3], coordinate monitoring [4], laser scanning [5], strain gauge monitoring [6], and sensor monitoring [7]. However, with increased product complexity, ensuring monitoring accuracy using a single method is difficult [8]. Current research on structural deformation monitoring emphasizes the multi-source fusion method [9], which integrates data from multiple sensing systems, thereby ensuring the reliability of monitoring results across complex structures and meeting the requirements for data accuracy and real-time performance in structural deformation monitoring.

In recent years, multi-source fusion deformation-monitoring methods have attracted significant attention. Smith et al. [10] conducted validation experiments on different products using a multi-source fusion method combining vision and haptics, thereby enhancing the deformation-monitoring accuracy by integrating high-fidelity haptic data with visual information. Xu et al. [11] successfully monitored the truss structure of a space telescope with high accuracy by combining data from latitudinal and longitudinal instruments, a laser tracker, and a photogrammetric system. Xiang et al. [12] combined data from a contact probe and laser scanner to monitor the large free-form surfaces of an aerospace product. Gao et al. [13] integrated data from a laser scanner with proximity and tilted photogrammetric data to monitor the deformation of the outer wall of an LNG storage tank. While these multi-source fusion deformation-monitoring methods considerably improve the deformation monitoring accuracy, they often neglect the need for real-time monitoring. To address this issue, researchers have proposed a structural deformation-monitoring technique by incorporating fiber Bragg grating (FBG) data. Qu et al. [14] combined initial state data reconstructed using dual warps with FBG monitoring data, thereby reducing nonlinear measurement errors and successfully monitoring the structural deformation of an arrayed synthetic aperture radar (SAR) of an airborne remote sensing system. Sun et al. [15] used a Gaussian process regression method to fuse binocular scanning, laser tracking, and FBG data to monitor the deformation of thin-walled skin structures. Liu et al. [16] proposed a deformation optimization algorithm by combining FBG and binocular scanning data, demonstrating its effectiveness in accurately capturing structure deformation data. These multi-source fusion deformation-monitoring methods use FBG and other techniques to improve real-time monitoring performance. However, comparisons with high-precision vision measurements reveal that while the accuracy of the FBG area of product deployment is very high, in the above research, the FBG sensor layout often adopts orthogonal distribution to form a quasi-distributed monitoring system. Due to space limitations, the edge area of the product can only realize unidirectional data collection, so the edge area lacks a constraint point limit, and the data accuracy of the edge area of the product fusion result is low.

In order to overcome the problem of the lack of FBG constraint point limitations in the edge area of the structure, this study proposes a multi-source fusion deformation-monitoring calibration method using a normal distribution transformation (NDT)–convolutional neural network (CNN)–self-attention (SA) calibration network. This method uses NDT to integrate the point-cloud data from the fused multi-source deformation-monitoring model, thereby improving the continuity of each point within the point clouds. It extracts features from the mesh using a CNN and employs an SA mechanism to adjust the significance of the features across the various positions in each input. The calibration results are later obtained using a multilayer perceptron. This method addresses fusion edge anomalies while ensuring the real-time performance of the multi-source fusion deformation-monitoring method. Additionally, it improves the accuracy of the multi-source fusion structural deformation-monitoring results of fused FBG data by combining visual monitoring characteristics, which provides a high accuracy and eliminates edge anomalies.

2. Multi-Source Fusion Deformation-Monitoring Calibration Model

FBG is a sensor that detects strain and has the characteristic of real-time monitoring. Its working mechanism is that the surface of a product is affected by pressure and other loads, which causes the product structure to be deformed, so that the wavelength of the FBG optical-fiber-sensing center pasted on the surface is shifted. FBG has a selective effect on optical wavelength, and the wavelength that conforms to the size of the band is transmitted into the modulation region of the optical fiber, so that the optical properties of the corresponding fiber core change to form periodic modulation, and the refractive index of the fiber changes, resulting in the overall deviation of the reflected wave of the optical fiber to obtain the measured parameters.

The multi-source fusion deformation-monitoring method, which combines vision and FBG, aims to ensure high real-time performance while capturing the most accurate and complete structural information. This study presents a calibration method that integrates data from a vision measurement and FBG monitoring system, using scanned high-precision measured data as the output. The calibration model is shown in Figure 1. In the multi-source fusion model that incorporates FBG monitoring, the structural edge regions often exhibit anomalous point data with large errors owing to the limitations of FBG in covering the structural edges of the monitored products. To address this issue, we adopt a method that enhances point-cloud continuity to eliminate edge anomalies and improve the overall monitoring accuracy by calibrating the model. This method does not require coordinate transformation or alignment between high-precision measured data and multi-source fusion data. It neither imposes restrictions on the dense–sparse layer degree of the point cloud nor requires consistency in the number of points.

The workflow of the multi-source fusion deformation-monitoring calibration model begins by scanning the target object using a binocular scanning system at a zero-load static moment, thereby resulting in a high-precision initial state point cloud. This point cloud captures the global structural information in its undeformed state for the multi-source fusion model. Once the deployment is complete, a deformation force is applied to the target object, while the system monitors the changes in the center wavelength of the FBG. The center-wavelength deformation data provide accurate deformation information in the FBG monitoring area. Simultaneously, the binocular scanning system gathers the point-cloud data of the target object during deformation, which represent accurate measurements of the object after deformation. This is later used to optimize the network output.

All the acquired data, including the initial state point cloud, FBG center-wavelength variation, and high-precision measured point cloud, were utilized during calibration. The initial state point cloud and FBG center-wavelength variation were fused using multi-source heterogeneous data fusion to generate a deformed multi-source fusion point cloud. The multi-source fusion point cloud and high-precision measured point cloud were input into the constructed NDT-CNN-SA calibration network after preprocessing. This network enhances the calibration the accuracy of the multi-source fused point-cloud data by using the labeled data as the target output. Through neural network training, it identifies correlations and patterns between the two data types in an unsupervised manner, thereby leveraging the position information of the fused point cloud to achieve accurate multi-source point-cloud data.

The multi-source fusion deformation monitoring calibration model not only addresses the problem of point-to-point correspondence during calibration and reduces the data calibration complexity, but also optimizes the quality of the multi-source fusion point cloud. The multi-source fusion deformation-monitoring and calibration network, which serves as the core component of the model, enhances point-cloud continuity and feature refinement.

3. Construction of Multi-Source Fusion Deformation-Monitoring Calibration Network Using NDT-CNN-SA

The calibration network includes the NDT, CNN, and SA modules. This section elaborates on the construction of these modules and concludes with an overview of the framework of the NDT-CNN-SA calibration network.

3.1. NDT Module

The NDT algorithm is used in point-cloud alignment by dividing the point cloud into specific grid sizes based on the spatial distribution of the reference point cloud. It calculates the multidimensional normal distribution parameters for each grid to identify the optimal transformation matrix that maximizes the probability density between the point clouds, thereby ensuring effective alignment. The NDT module uses a data processing method that divides the point cloud into grids and calculates the normal distribution parameters for each grid, as shown in Figure 2.

The NDT module represents a 2D or 3D point cloud as a multidimensional differential Gaussian distribution, wherein the normal distribution $N(\mu ,C)$ of each NDT cell comprises a mean vector and covariance matrix $C$, expressed as follows:

$$(\mu ,C)=({\displaystyle \frac{1}{n}}{\displaystyle \sum }_n^{k=1}{x}_k,{\displaystyle \frac{1}{n-1}}{\displaystyle \sum }_n^{k=1}(x_k-\mu ){(x_k-\mu )}^T),$$

(1)

where $x_k=1,\cdots ,n$ represents the 3D points in each cell.

Using the NDT module, the multi-source fusion point-cloud data are converted into a gridded representation. This approach preserves the continuity between the points in the point cloud and simplifies the computation process by capturing the characteristics of each grid, thereby eliminating the need for individual point calculations.

3.2. CNN Module

The CNN module is a feature extraction module that uses three 1D convolutional layers to extract features from the NDT units. It incorporates a batch standardization layer after each 1D convolutional layer to standardize the data, thereby preventing bias and deviation by ensuring a consistent output distribution. An activation function introduces nonlinearity to the standardized results, thereby enhancing the ability of the network to express complex relationships. The structure of this module is shown in Figure 3, wherein $P\in R^{m\times 12}$ represents the NDT unit data, $m$ the number of the NDT units, and $F_p\in R^{256\times m}$ the point-cloud feature.

The point cloud serves as the input to the CNN module, thereby undergoing three sets of nonlinear mapping operations, expressed as follows:

$$F_1=ReLu(BN(f^1(P))),$$

(2)

$$F_2=ReLu(BN(f^1(F_1))),$$

(3)

$$F_P=ReLu(BN(f^1(F_2))),$$

(4)

where $f^1$ represents the standard 1D convolution operation with a kernel and step size of 1 each. The $BN$ operation standardizes the input for a batch size, thereby stabilizing the data distribution. $\mathrm{Re}Lu$ represents the activation function.

The feature extraction method implemented using the CNN module can automatically learn point-cloud features. It divides the point-cloud feature extraction process into low- and high-level layers, thereby gradually extracting abstract information from the input data to acquire high-level features of the point-cloud data.

3.3. SA Module

The SA mechanism is a deep learning technique for processing sequence data that enables the model to simultaneously focus on different parts of the sequence rather than relying on a fixed window size or predetermined weights. This model generates a weight for each location by calculating the degree of association between that position and others in the sequence.

This module receives the feature vector output from the CNN feature extraction module, denoted as $F_p$, and changes it into $F_p^{tran}$. After transposing the specified dimensions, the output and transformed feature vector are input into the SA mechanism module, which automatically optimizes the weights of each input feature to enhance the expressive power of the network. The structure of this module is shown in Figure 4.

The input features $F_P$ and $F_P^{tran}$ of the SA mechanism are obtained using the following computation, wherein $transpose$ denotes the transpose operation of the specified dimension:

$$F_P^{tran}=transpose(F_P),$$

(5)

In the forward propagation process, the attention score $scores$, which indicates the importance of each sequence, is obtained using the following computation, wherein $weigh{t}_{init}$ and $bia{s}_{init}$ denote the initialized weights and bias terms, respectively:

$$scores=(F_p^{tran}\times weigh{t}_{init})+bia{s}_{init},$$

(6)

We apply $Soft\mathit{max}$ to normalize the attention scores and obtain the attention weights. This is calculated by placing the elements of the eigenvectors in the range of $[0,1]$ in the specified dimensions and integrating them as $1$. The formula is expressed as follows:

$$weigh{t}_{soft}={\displaystyle \frac{\mathit{exp}(score{s}_i)}{{\displaystyle {\sum }_j\mathit{exp}(score{s}_j)}}},$$

(7)

The weighted summed SA vector is obtained by multiplying $weigh{t}_{init}$ by $F_P^{tran}$ and summing the specified dimensions to obtain the final SA vector, expressed as follows:

$$weight={\displaystyle \sum (weigh{t}_{soft}\times F_p^{tran})},$$

(8)

The final output of the fused features $F^{output}$ is obtained by the following computation, which implements the operation of fusing the SA weighted result with the original feature vector:

$$F^{output}=F_p+weight,$$

(9)

The feature extraction method of the SA module is utilized to weight the point-cloud features, thereby enabling the network to effectively process the point-cloud information at different locations. This improves efficiency and optimizes the calibration of the multi-source fusion deformation monitoring.

3.4. NDT-CNN-SA Network Architecture

Figure 5 shows the architecture of the NDT-CNN-SA, which comprises four components: a normal distribution transform module, a CNN point-cloud feature extraction module, an SA mechanism, and a multilayer perceptron.

The advantages of using the NDT module to calibrate the results of the multi-source fusion deformation monitoring are as follows: (1) The NDT module can effectively extract information from the mathematical model during the fusion process, thereby enhancing the integrity of data sample features. This reduces feature discovery during training, reduces computational complexity, and improves computational efficiency. (2) By gridding the point cloud and calculating the multidimensional normal distribution for each grid, the NDT module can extract connection information between the points within the grid. This approach improves the continuity of points in the point cloud by connecting structural edge information with neighboring areas. It effectively addresses the structural edge anomalies caused by the inability to deploy FBGs at the edges of the product, thereby improving the robustness and accuracy of the multi-source fusion deformation-monitoring model.

The design of the CNN module combines the characteristics of the NDT unit with the advantages of convolutional neural networks. While the NDT module enhances the continuity of points within the point cloud, it might reduce the input feature richness of the data. In this study, the CNN module is used to extract more advanced and effective features. The multifaceted feature extraction of NDT units is achieved using three 1D convolutional layers, combined with batch normalization and nonlinear transformation via the activation function. This combination improves the expressive ability of the NDT units, thereby enhancing the feature extraction effect of the network.

The SA module weights the information at different locations in the feature vector output from the CNN module, filters irrelevant data in the input sequence, and enhances the robustness, performance, and accuracy of the network.

The multilayer perceptron (MLP) layer maps the feature vectors processed through it to the desired output dimensions and captures more complex and abstract representations in the feature space. By adjusting the nonlinear activation function and weights across layers, the MLP effectively captures deep features and underlying patterns in the data, thereby improving the calibration performance.

The NDT-CNN-SA calibration network enhances the point-cloud continuity by gridding the point-cloud data using the NDT module. The CNN module ensures effective feature extraction from the NDT unit data, thereby minimizing the feature loss. The SA module calculates the attention weights for features at different locations, thereby accelerating convergence during training. Finally, the MLP performs a series of mapping transformations on the data to obtain the final output. This approach minimizes the introduction of nonlinear transformations by extracting features and later expanding dimensions, thereby retaining relevant information in the feature space and reducing computational complexity and overfitting. By alternating between dimension expansion and compression, the neural network can model complex data within a smaller implicit space, thereby enhancing the flexibility and efficiency of feature learning. The NDT-CNN-SA calibration network offers a method that enhances the continuity of points between point clouds in multi-source fusion deformation monitoring for fused FBG monitoring systems. Additionally, the multi-source fusion deformation-monitoring method incorporating the FBG monitoring system, combined with a neural network, achieves high efficiency and high quality in dealing with structural edge data anomalies, and effectively improves the accuracy of structural deformation monitoring.

4. Experimental Verification

This section first describes the datasets used in the training experiments of the NDT-CNN-SA network and the network parameter settings. Finally, the performance metrics of the network are analyzed experimentally.

4.1. Dataset Acquisition and Preprocessing

The data collection object used for training the NDT-CNN-SA network is a carbon fiber wing skin model. The input of the network is a multi-source fusion point cloud. The data sources for this fusion model include the zero-load static point cloud captured using a binocular vision measurement system and the center-wavelength variation captured using the FBG monitoring system. The multi-source fused point cloud monitors the data in real time. The output of the network is a high-precision real-time point cloud, which represents the scan data acquired simultaneously with the input-fused model data using the binocular scanner. The main components of the dataset acquisition experiment include a simplified wing model, fixed platform, binocular vision scanning system, and robotic arm, coupled with an FBG monitoring system and different counterweights, as shown in Figure 6. Among them, the FBG monitoring system is very sensitive to ambient temperature, so the experiment was carried out in a constant-temperature calibration room where the ambient temperature changed very little to minimize the influence of ambient temperature on the experiment.

The quasi-distributed FBG sensor network was pasted on the wing model; this project used the Optical System 256 fiber grating demodulation system of Beijing Xizhuo Information Technology Co., Ltd. (Beijing, China), which integrates several modules such as light source, data acquisition, network communication, etc. The wavelength range is 1528~1568 nm, the wavelength resolution is 0.1 pm, and the synchronous frequency is 100 Hz. At the same time, it is equipped with four measurement channels. The sensor network was constructed using wavelength division multiplexing and space division multiplexing technology, and the wavelength division technology was used in each channel to realize the grating wavelength demodulation of a single channel.

A total of six FBGs were used in the experiment, each with a side-mode rejection ratio of 18.75 dB and a gate area length of 10 mm, and center wavelengths of 1530.1795, 1539.6166, 1542.8915, 1547.5220, 1549.2798, and 1552.6104 nm. For FBG-type SMF-28 Acrylate, its coating method is polyimide and its tensile strength is greater than 100 kpsi. This FBG has been used in extensive experiments in the literature [17], with sensitivities ranging from 1.96 pm/m⁻¹ to 50.65 pm/m⁻¹ at curvatures less than 30m⁻¹. In addition, the literature [18] has demonstrated a strain sensitivity of 1.2 $\mathrm{pm}/\mathsf{\mu }\mathsf{\epsilon }$ for this FBG.

To construct the dataset used for the network training, it was crucial not only to calibrate each device during the system setup, but also to conduct a series of processing tasks. First, the optical fiber was connected to the demodulation system to measure the central wavelength in the static zero-load state, while the complete point-cloud data of the wing skin model in this state were measured using a calibrated binocular scanner. Stabilizing pressure was applied to the edge of the wing skin model. The stabilizing pressure was applied to the object using weights suspended on the lower surface of the wing model with a tray. The force value was constantly changing by changing the number of weights, and the range of the force value was between 0 N and 50 N. The magnitude of the applied force was varied by repeated step-by-step incremental and decremental methods, and repetitive experiments were carried out to enrich the dataset and improve the robustness of the model. Once the model reached a stable state, the change in the center wavelength of the FBG was monitored, while the deformation point-cloud data were collected using the multi-source fusion model. The acquired multi-source fusion point cloud was used as input to the network input, while a binocular scanning system simultaneously captured a high-precision measured point cloud of the wing skin model. The binocular vision scanning system used the Creamform factory tracking scanner, which was composed of a C-Track optical tracker and an optical CMM 3D scanner, with a scanning accuracy of 0.064 mm (9.1 m³) and a resolution of 0.05 mm. The binocular vision scanning system was based on the binocular stereo vision measurement principle and the structured light 3D vision measurement principle to quickly obtain the high-density point-cloud data of the product. To reduce the impact of noise on the network, the high-precision measured point cloud was processed using denoising and downsampling to extract the part containing the complete wing model, which was later used as the network output. Finally, the network was trained to calibrate the accuracy of the multi-source fusion point clouds.

The dataset used for NDT-CNN-SA network training included 5000 samples. Each sample comprised a multi-source fusion point cloud, a high-precision measure point cloud, and measurements of the different deformations and angles of the same target.

4.2. Evaluation Metrics

L1 Loss (absolute value loss) is used as a loss function to optimize the neural network model on the training set, thereby minimizing overfitting. L1 Loss is expressed as follows:

$$Loss={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n\left|y_i-{\widehat{y}}_i\right|},$$

(10)

where $y_i$ represents the true output of each sample and ${\widehat{y}}_i$ the predicted value of the network for the sample.

The mean square error (MSE) is used as a loss function to measure the performance of the neural network on the test set and is expressed as follows:

$$MSE={\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n{(y_i-{\widehat{y}}_i)}^2},$$

(11)

The root mean square error (RMSE) is used to assess the degree of fit of the network to the training and test sets and quantify the average prediction error of the network. The RMSE is expressed as follows:

$$RMSE=\sqrt{{\displaystyle \frac{1}{n}}{\displaystyle \sum _{i=1}^n{(y_i-{\widehat{y}}_i)}^2}},$$

(12)

4.3. Parameter Settings

To build the network, the experiment was conducted on Windows 11 using PyTorch 2.0.0 and Python 3.10.9. The experimental setup comprised an NVIDIA GeForce RTX 3080 Laptop GPU with 14 GB of operating memory. The dataset comprised 5000 sample sets, with 3000 sets for training, 1000 for testing, and 1000 for validation. The training and test set data were used for network training, while the validation set data were only used for the final performance evaluation.

The Adam optimization algorithm was used to optimize the training parameters and weights. The batch size was set to eight for training, with an initial learning rate of 0.001. The learning rate was adjusted by multiplying it by 0.7 every 10 iterations. The number of epochs was set to 250, while the comparative result was the average of five calculations. The specific configurations of the network parameters are presented in

Table 1. Configuration of network-specific parameters.

Network Layer	Network Layer Type	Number of Neurons	Weighting Kernel/Convolution Kernel Size	Pacemaker	Activation Function
1	NDT Representation Layer	—	—	—	—
2	Transform Net Layer	—	—	—	—
3	Convolutional Layer	64	1	1	—
4	Batch Normalization Layer	64	—	—	ReLU
5	Convolutional Layer	128	1	1	—
6	Batch Normalization Layer	128	—	—	ReLU
7	Convolutional Layer	256	1	1	—
8	Batch Normalization Layer	256	—	—	ReLU
9	Self-Attention Layer	256	—	—	—
10	Convolutional Layer	128	1	1	—
11	Batch Normalization Layer	128	—	—	ReLU
12	Convolutional Layer	64	1	1	—
13	Batch Normalization Layer	64	—	—	ReLU
14	Convolutional Layer	3	1	1	—
15	Fully Connected Layer	3	—	—	—

.

During the training process of the experiment, the number of iterations and size of the dataset significantly impacted the calibration network. In this study, we designed an iteration-number verification experiment wherein the number of calibration network iterations was adjusted to 1000, as shown in Figure 7a. During the first 150 iterations, the loss value decreased rapidly. Between 150 and 250 iterations, the loss value continued to converge, although the decrease rate reduced. When iterations ranged from 250 to 1000, the fluctuations in loss value were ≤1 × 10⁻⁴, while the perturbation range was less than 1% of the loss value. This experiment demonstrated that more than 250 iterations did not optimize the training results of the network. Consequently, the number of iterations was set to 250 for the network training.

Comparison experiments were conducted on the number of samples in the dataset. When the number of samples was less than 500, the average calibrated deviation was greater than 0.3 mm, which was too large for the calibration task. Hence, these cases were excluded from the comparison experiments. The network training was conducted using dataset sizes ranging from 500 to 5000. The overall validation data were calibrated using the trained network model’s deformation results. The deviation of the calibration results from the actual results was calculated, and the comparison results are shown in Figure 7b. The experimental results indicate that when the dataset size reached 5000, the deviation statistics stabilized and became more tightly distributed. For this dataset size, the values of the upper limit, Q3 (upper quartile), Q2 (median), lower limit, and mean were the lowest, with Q1 (lower quartile) corresponding with the minimum. The detailed results of all deviation statistics are presented in

Table 2. Calibration bias statistics for the different dataset sizes.

Dataset Size	Upper Limit (mm)	Q3 (mm)	Q2 (mm)	Q1 (mm)	Lower Limit (mm)	Mean (mm)
500	0.358	0.183	0.108	0.066	0.001	0.225
1000	0.303	0.154	0.089	0.055	0.0007	0.175
2000	0.305	0.156	0.103	0.057	0.0007	0.146
3000	0.301	0.143	0.101	0.056	0.001	0.128
4000	0.266	0.135	0.091	0.051	0.0007	0.119
5000	0.208	0.117	0.082	0.055	0.0001	0.102

. With a dataset size of 5000 data sets and 250 iterations, the convergence time is 57 min and the training time is 187 min.

4.4. Network Performance Experiment Comparison

To demonstrate the effectiveness of the added module, the following ablation experiments were conducted on the NDT-CNN-SA network framework while maintaining a similar experimental environment and model parameter settings: (1) a CNN network framework, (2) an NDT-CNN network framework, (3) a CNN-SA network framework, and (4) an NDT-CNN-SA network framework. In the feature output phase of the network architecture, it is necessary to reduce the number of features to three again, so the CNN feature extraction module is essential. Therefore, a control experiment without the CNN module was not performed in the ablation experiment.

The training results of the four sets of the ablation experiments on the test set are shown in Figure 8. As shown in Figure 8a,b, adding the NDT module does not reduce the fluctuation in Loss and RMSE during the descent of the test set. This is because the further consideration of connectivity between the point clouds increases the data complexity. However, adding the SA module significantly reduces these fluctuations. When the NDT and SA modules are added simultaneously, the Loss and RMSE descent processes exhibit the greatest stability with minimal perturbations. This indicates that the combined NDT and SA modules improve the robustness of the network in this architecture.

An example sample data from the validation set was selected and processed through each network for calibration, thereby providing a more intuitive understanding of the performance of each module in the multi-source fusion system calibration task. As shown in Figure 9, the calibration results from each network were compared to the labeled data, while the deviation from the labeled data was plotted. The CNN network exhibited larger deviations during calibration, notably in abnormal edge areas. These abnormal edge areas did not interact with the adjacent areas. The NDT-CNN network can eliminate abnormal edge areas by enhancing the continuity of the data between the point clouds compared to the CNN network, but it will relatively increase the deviation of the adjacent areas. However, the CNN-SA network can reduce the point-cloud deviation and improve the calibration accuracy without introducing connectivity issues between edge anomalies and adjacent areas. The NDT-CNN-SA network demonstrated superior point-cloud continuity with less bias and smoother edges than the other networks. The CNN convergence time was 38 min, the CNN-SA convergence time was 69 min, and the NDT-CNN-SA convergence time was 57 min. Although the CNN network converged faster, the accuracy of the NDT-CNN-SA network was improved by 36.57% compared to the CNN, which verifies the higher computational efficiency of the NDT module, and is in line with the advantages of the NDT module described in the previous section.

To further demonstrate the feasibility and superiority of the NDT-CNN-SA network architecture, a comparative experiment was conducted against the CNN–gated recurrent unit (GRU)–convolutional block attention module (CBAM) [19] and CNN–long short-term memory (LSTM)–SA [20] frameworks. These networks were evaluated under similar experimental environments and model parameter settings.

The training results of the three comparison experiments on the test set are shown in Figure 10. In the test set, as shown in Figure 10a,b, the loss of the three networks reached 1 × 10⁻⁴, although the NDT-CNN-SA calibrated network exhibited the smallest fluctuations and highest stability in the descent process. This trend was similar to that of RMSE, with the RMSE results of NDT-CNN-SA being 23.2% better than those of CNN-GRU-CBAM and 5.1% better than those of CNN-LSTM-SA.

To compare the performance of each network in the multi-source fusion system calibration task intuitively, similar validation set sample instances as in the ablation experiment were selected and input into each network for calibration. The deviation of the calibration results from the labeled data was plotted using similar criteria as in the ablation experiment. The results are shown in Figure 11. As shown in Figure 11a–c, the deviation in the data calibration task of the CNN-GRU-CBAM network is relatively large, while the middle region of its wing model is lighter in color and less accurate than the other two networks. Although the CNN-LSTM-SA network shows considerable improvement compared to the CNN-GRU-CBAM network, it still exhibits some distorted regions at the wing edges. Conversely, the NDT-CNN-SA network demonstrates superior point-cloud matching and continuity, with fewer errors and smoother edges than the other networks.

To further evaluate the performance of each network, the full validation set samples were input into the networks used in the ablation experiments and comparison tests. The complete deviation values of the calibration results were compiled and plotted as a box-and-line plot, as shown in Figure 11d. This figure illustrates the distribution of the deviation values for each network. The experimental results reveal that the NDT-CNN-SA network exhibited a more compact and smaller distribution of the deviation statistics. With the aforementioned network architecture, the values of the upper limit, Q3, lower limit, and mean were the smallest, while Q2 and Q1 were almost similar to the minimum values. These findings further confirm the validity and superiority of the NDT-CNN-SA model for multi-source fusion deformation calibration tasks. The detailed deviation statistics are presented in

Table 3. Calibration deviation statistics for the different networks.

Network Architecture	Upper Limit (mm)	Q3 (mm)	Q2 (mm)	Q1 (mm)	Lower Limit (mm)	Mean (mm)
CNN	0.459	0.225	0.111	0.069	0.0007	0.237
NDT-CNN	0.286	0.155	0.107	0.067	0.001	0.153
CNN-SA	0.280	0.147	0.096	0.059	0.0008	0.162
CNN-GRU-CBAM	0.234	0.127	0.085	0.056	0.0006	0.148
CNN-LSTM-SA	0.235	0.125	0.080	0.053	0.0002	0.130
NDT-CNN-SA	0.208	0.117	0.082	0.055	0.0001	0.102

.

By comparing the qualitative and quantitative results, the NDT module proposed in this study improves the continuity between the points in the point cloud, thereby addressing the problem of distorted regions lacking correlation with neighboring regions. Furthermore, the architecture proposed in this paper has been shown to provide an effective calibration of monitoring results for the calibration of deformation in structures monitored by multi-source fusion.

5. Conclusions

In this study, a novel multi-source fusion deformation-monitoring calibration method was proposed to address edge data anomalies in multi-source fusion deformation-monitoring results fused with FBG. This method reduced the maximum absolute error between the multi-source fusion deformation-monitoring data and measured data to below 0.2 mm. The NDT-CNN-SA calibration network served as a core component of this calibration task, with the normal distribution transformation module enhancing the continuity among the points in the point-cloud data, thereby reducing the edge anomalies encountered during the multi-source fusion deformation monitoring. By adding the SA mechanism module, the network adaptively learned the correlation and weight among the data, thereby improving the accuracy of the data calibration. Furthermore, the NDT-CNN-SA network did not require a fixed or similar number of points in the calibrated point clouds. The accuracy of the NDT-CNN-SA network improved by 36.57% compared with the CNN, 14.39% compared with the CNN-GRU-CBAM network, and 9.54% compared with the CNN-LSTM-SA network, thereby indicating significant accuracy improvement in the multi-source fusion deformation-monitoring task. This calibration method will improve assembly accuracy in manufacturing and product quality, and provide more accurate real-time data for visualized assembly. However, the main limitation of this study lies in the slow acquisition of real-time scanning data in the dataset production process. In the future, more efficient data-acquisition methods will be investigated to simplify the calibration data production process.