An Aeromagnetic Compensation Algorithm Based on a Residual Neural Network

Abstract

Aeromagnetic compensation is a crucial step in the processing of aeromagnetic data. The aeromagnetic compensation method based on the linear regression model has poorer fitting capacity than the neural network aeromagnetic compensation algorithm. The existing gradient updating neural network-based aeromagnetic compensation algorithm is subject to the problem that the gradient disappears during the backpropagation process, resulting in poor fitting ability and affecting aeromagnetic compensation accuracy. In this paper, we propose a neural network compensation algorithm with strong fitting ability: residual backpropagation neural network (Res-bp). The algorithm realizes the cross-layer propagation of the gradient through a residual connection so that the network not only preserves the original information but also acquires additional information during training, successfully solving the problem of gradient disappearance and boosting the network’s fitting capacity. The algorithm is applied to the data collected by unmanned aerial vehicles (UAVs) to verify its effectiveness. The results show that the improvement ratio is improved compared with the traditional neural network, demonstrating that the algorithm has a significant compensation effect on aeromagnetic interference and improves the quality of aeromagnetic data.

1. Introduction

Aeromagnetic surveying, as one of the main ways to explore and survey geological resources, is an important aerogeophysical prospecting method [1,2]. Along with the miniaturization of sensors, aeromagnetic measurement based on UAVs is becoming more widely used. However, data obtained by a magnetic sensor on an aircraft contain magnetic interference information, and the effect of compensation for aeromagnetic interference directly determines the quality of aeromagnetic data [3]. Tolles and Lawson established the T-L model based on magnetic interference and aircraft attitude [4]. Based on this model, Leliak designed a compensation equation enabling the elimination of magnetic interference by solving for a compensation coefficient [5]. Leach regarded magnetic compensation as a linear regression problem and proposed a ridge regression (RR) algorithm to solve the multicollinearity problem in the T-L equation [6]. Wu et al. used principal component analysis (PCA) to reduce the multicollinearity of the T-L model [7].

For the aeromagnetic compensation method based on linear regression, if the aeromagnetic data have a complex or nonlinear structure, the algorithm is unstable, and the fitting ability is poor due to the correlation between the data, which affects the quality of aeromagnetic data [8]. Deep learning is based on the representation learning of data and has a strong fitting ability. Williams proposed the use of neural network modeling for aeromagnetic compensation and established a neural network model of aircraft interference [9]. However, due to the limited computing power available at the time, the model has not been widely used. Zhang analyzed the aircraft swaying interference not considered in the T-L model and used a one-dimensional convolutional neural network to remove the effect of tail-boom swing [10]. Xu established a binary classification network for magnetic anomaly detection and a regression network for geomagnetic noise suppression and applied deep learning to magnetic anomaly detection and noise elimination [11].

However, in the backpropagation process of neural networks, owing to factors such as the chain rule, the gradient will disappear, affecting the accuracy of aeromagnetic compensation. Yu used an autoencoder network for aeromagnetic compensation to extract data features by searching the direction of maximum change in the backpropagation process, thereby reducing the multicollinearity of the model. However, in the training process of the autoencoder network, the feature extraction of high-dimensional complex data is not perfect, the gradient is prone to fall into the local minimum, and the problem of gradient disappearance is not considered [12]. Wang considered the gradient disappearance problem in the autoencoder network training and introduced a residual connection into the autoencoder network. The features output from each layer of the encoder are transmitted to the corresponding layer of the decoder through residual connections. The algorithm is applied to fault diagnosis of generator sets [13]. Ma combined a linear model and neural network model and proposed a dual estimation method for aeromagnetic compensation [14]. Although this method can suppress the noise caused by the overfitting problem of the neural network, owing to the excessive number of iterations, the gradient will approach zero infinitely, and the gradient of the network will disappear, making it difficult to reduce the loss function, thus affecting the compensation effect. Yu considered the main factors that affect the generalization ability of the network, analyzed the probability density function that constitutes the regression neural network, and cited the fully connected neural network for aeromagnetic compensation, which solved the problem of overfitting the network and improved the generalization ability of the network, although the process did not consider the problem of gradient disappearance [15].

To address the gradient disappearance problem of neural networks in aeromagnetic compensation algorithms, in this paper, we combine a residual connection with neural networks and propose a neural network aeromagnetic compensation algorithm (Res-Bp). The features output from the low-level information of the network are transmitted to the high-level information through the residual connection without intermediate operation to realize the cross-layer propagation of the gradient [16]. Residual connection changes the structure of the network and makes it have the ability of identification mapping. It effectively retains the gradient characteristics during backpropagation, alleviates the gradient disappearance problem of traditional neural networks to a certain extent, and improves the accuracy of aeromagnetic compensation. The algorithm is applied to the data collected by UAVs to verify its effectiveness. The remainder of this article is organized as follows. In Section 2, we briefly introduce the T-L model and analyze the reason for the gradient disappearance of the primitive neural network, as well as the principle of the algorithm proposed in this paper. In Section 3, we apply real compensation flight data to conduct experiments and analysis.

2. Compensation Model and Method

2.1. T-L Model

The magnetic interference model of aircraft includes three components: the permanent ($H_p$) magnetic field generated by various parts of the aircraft, the induced ($H_i$) magnetic field produced by geomagnetic field magnetization, and the eddy-current ($H_{\mathrm{ec}}$) magnetic field produced by aircraft motion, which can be expressed as:

$$H_p{=c}_1{cos\alpha +c}_2{cos\beta +c}_{3}cos\gamma$$

(1)

$$H_i{=T}_t\left(\begin{array}{c}{c}_4{cos}^2{\alpha +c}_5{cos\alpha cos\beta +c}_{6}cos\alpha cos\gamma \\ {+c}_7{cos}^2{\beta +c}_8{cos\beta cos\gamma +c}_9{cos}^2\gamma \end{array}\right)$$

(2)

$$H_{\mathrm{ec}}{=T}_t\left(\begin{array}{c}{c}_{10}{cos\alpha cos}^{\prime }{\alpha +c}_{11}{cos\beta cos}^{\prime }{\alpha +c}_{12}{cos\gamma cos}^{\prime }\alpha \\ {+c}_{13}{cos\alpha cos}^{\prime }{\gamma +c}_{14}{cos\beta cos}^{\prime }{\gamma +c}_{15}{cos\gamma cos}^{\prime }\gamma \\ {+c}_{16}{cos\alpha cos}^{\prime }{\beta +c}_{17}{cos\beta cos}^{\prime }{\beta +c}_{18}{cos\gamma cos}^{\prime }\beta \end{array}\right),$$

(3)

where $c_1{, c}_2{,\dots , c}_{18}$ are compensation coefficients;${ {cos}^{\prime }\alpha , {cos}^{\prime }\beta , \mathrm{and} cos}^{\prime }\gamma$ are the derivatives of $cos\alpha , cos\beta , \mathrm{and} cos\gamma$, respectively; and $T_t$ is the Earth’s magnetic field.$cos\alpha , cos\beta , \mathrm{and} cos\gamma$ can be calculated based on three-axis fluxgate magnetometer data as:

$$cos\alpha =\frac{T_x}{T}, cos\beta =\frac{T_y}{T}, cos\gamma =\frac{T_z}{T}{, T}_t=\sqrt{T_x{}^2{+T}_y{}^2{+T}_z{}^2}.$$

(4)

The total magnetic interference field can be expressed as:

$$\begin{array}{cc}\hfill H_t& =H_p+H_i+H_{ec}\hfill \\ \hfill & = c_1{x}_1 + c_2{x}_2 + c_3{x}_3 + c_4{x}_4 + c_5{x}_5 + c_6{x}_6\hfill \\ \hfill & + c_7{x}_7 + c_8{x}_8 + c_9{x}_9 + c_{10}{x}_{10} + c_{11}{x}_{11} + c_{12}{x}_{12}\hfill \\ \hfill & + c_{13}{x}_{13} + c_{14}{x}_{14} + c_{15}{x}_{15} + c_{16}{x}_{16} + c_{17}{x}_{17} + c_{18}{x}_{18}\hfill \end{array}$$

(5)

The LS solution of the T-L equation is:

$${XC=H}_t$$

(6)

$$C_{LS}={\left(X^{T}X\right)}^{-1}{X}^T{H}_{t,}$$

(7)

where $H_t$ is a column vector, $C$ consists of${ c}_i$($i=1, 2, \dots , 18$), and:

$$X=\left[\begin{array}{cccc}{X}_{1,1}& X_{1,2}& \dots & X_{1,18}\\ X_{2,1}& X_{2,2}& \dots & X_{2,18}\\ ⋮& ⋮& & ⋮\\ X_{n,1}& X_{n,2}& \dots & X_{n,18}\end{array}\right],$$

(8)

where $X_{n,1}{, X}_{n,2}{,\dots , X}_{n,18}$ are the variables in Equation (5), and the column vector of $X$ has multicollinearity.

When the independent variable has multicollinearity ($\left|X^{T}X\right|\approx 0$), due to the singularity of $X^{T}X$, there is a strong correlation between variables, which results in a considerable distance between the estimated and true values. There is a deviation in the LS algorithm, which affects the compensation accuracy. Ridge regression (RR) [16] is based on LS, adding an identity matrix to the coefficient matrix to improve stability. Its solution is expressed as:

$$C_{RR}={\left(X^{T}X+\lambda I\right)}^{-1}{X}^T{H}_t,$$

(9)

where $\lambda$ is a positive constant, and $I$ is the identity matrix.

2.2. Primitive Neural Model

The neural network structure for aeromagnetic compensation includes an input, hidden, and output layer. The input training samples of the network are $X_{nxm}$. The nodes of the neural network are connected by $\left(W, b\right){=(W}^1{, b}^1{, W}^2{, b}^2{,\dots , W}^l{, b}^l)$, where $b^l$ is the bias term of layer $l$, and $W^l$ is the weight between layers $l$ and $l - 1$.

In the process of forward propagation, the features of layer $l=2$ are:

$$Z_i^{\left(2\right)}=f\left({\displaystyle \sum }_j^m{W}_{ij}^1{X}_j{+b}_i^1\right),$$

(10)

where $i$ is the number of nodes in layer 2, and $j$ is the number of nodes in the input layer, $f (·)$ is the activation function, and the features of layer $l+1 (l > 2)$ are:

$${ Z}_i^{\left(l+1\right)}=f\left({\displaystyle \sum }_j^h{W}_{ij}^l{Z}_j{}^{\left(l\right)}{+b}_i^l\right),$$

(11)

where $Z_j{}^{\left(l\right)}$ is feature $j$ of layer $l$, and $W_{ij}^l$ is the weight between node $j$ of layer$l$ and node $i$ of layer $l+1$.

To improve the speed of training convergence, in this paper, we select a ReLU activation function and MSE as the loss function, expressed as:

$$f(a)=max(0,a)=\left\{\begin{array}{l}\begin{array}{cc}a& a>0\end{array}\\ \begin{array}{cc}0& a\le 0\end{array}\end{array}\right\}$$

(12)

$$\mathrm{Loss}=\frac{1}{n}{{\displaystyle \sum _{k=1}^n(y_k-{\widehat{y}}_k)}}^2,$$

(13)

where $y_k$ is the true value, and ${\widehat{y}}_k$ is the predicted value.

In the process of backpropagation, the gradient descent algorithm is used to calculate the gradient of $\mathrm{loss}$ function on each parameter through the derivative chain rule, the global optimal parameters are obtained, and the parameters are updated according to the gradient [17]. Taking two hidden layers as an example, the gradient process of $\mathrm{loss}$ function to $W^1$ is:

$$\begin{gathered} Z^1=f \left(W^1{X+b}^1\right) \\ {Z}^2=f \left(W^2{Z}^1{+b}^2\right) \\ \hat{y}=f \left(W^3{Z}^2{+b}^3\right) \end{gathered}$$

(14)

$$\frac{\alpha loss}{\alpha W^1} = \frac{\alpha loss}{\alpha \hat{y}}·\frac{\alpha \hat{y}}{\alpha Z^2}·\frac{\alpha Z^2}{\alpha Z^1}·\frac{\alpha Z^1}{\alpha W^1}$$

(15)

When one of the terms in Equation (15) is less than 1, with an increase in the number of layers, the gradient update information decays exponentially, so the shallow network parameters cannot be updated. Therefore, the neural network exhibits the phenomenon of gradient disappearance, which leads to an insufficient fitting ability of the model, thus affecting the accuracy of aeromagnetic compensation.

2.3. Res-Bp

To address the problem of gradient disappearance in the aeromagnetic compensation process of the neural network, we introduce the idea of residual connection and propose an algorithm with strong fitting ability, Res-Bp. The core of residual connection is the data fusion of nonlinear and linear features [18]. In forward propagation, the input signal can be directly propagated from any low level to a high level and has identity-mapping ability.

As shown in Figure 1, the residual connection has an identity-mapping structure compared to the ordinary connection. That is, the output ($Z^l$) of a layer is added to the new information (${F(Z}^l)$) of the layer after nonlinear transformation, and the sum result is input into the activation function as the output of the next layer.

In the process of forward propagation, assuming that $f (·)$ is a linear activation function, the features of layer $L$ are:

$$Z^L=F (F \left(\dots F \left(Z^l{,W}^l\right)\right)).$$

(16)

$$Z^L{=Z}^l+{\displaystyle \sum }_{i=l}^{L-1} F \left(Z^i{,W}^i\right),$$

(17)

where $F$ represents the nonlinear transformation function.

Equations (16) and (17) represent the training process from layer $l$ to layer $L$ in the forward propagation process of ordinary and residual connections, respectively. Residual connection layer $L$ can be expressed as the sum of any layer $l$ ($L$ > $l$) and the residual parts between them, which shows that the residual network always retains the original information in the process of training and obtains new information in the network. The training of an ordinary connection from layer $l$ to layer$L$ can only be formed through the training iteration of each layer between them.

When the dimensions of input $Z^l$ and output $Z^L$ are inconsistent, a weight ($W^{\prime }$) is necessary to adjust the dimension of $Z^l$ to attain consistency. It realizes the residual operation of tensors between different dimensions and jumps out of the limitation of dimensions. It is expressed as:

$$Z^L{=W}^{\prime }{Z}^l+{\displaystyle \sum }_{i=l}^{L-1} F\left(Z^i{,W}^i\right).$$

(18)

In the process of backpropagation, the gradient of loss to $Z^l$ is:

$$\frac{\alpha loss}{{\alpha Z}^l}=\frac{\alpha loss}{{\alpha Z}^L}·\frac{{\alpha Z}^L}{{\alpha Z}^l}=\frac{\alpha loss}{{\alpha Z}^L}·\frac{{\alpha Z}^L}{{\alpha Z}^{L-1}}·\dots ·\frac{{\alpha Z}^{l+1}}{{\alpha Z}^l}$$

(19)

$$\begin{gathered} \frac{\alpha loss}{{\alpha Z}^l}=\frac{\alpha loss}{{\alpha Z}^L}·\frac{{\alpha Z}^L}{{\alpha Z}^l}=\frac{\alpha loss}{{\alpha Z}^L}·\left(1+\frac{\alpha {{\displaystyle \sum }}_{i=l}^{L-1} F\left(Z^i{,W}^i\right)}{{\alpha Z}^l}\right) \\ =\frac{\alpha loss}{{\alpha Z}^L}+\frac{\alpha loss}{{\alpha Z}^L}·\frac{\alpha {{\displaystyle \sum }}_{i=l}^{L-1} F\left(Z^i{,W}^i\right)}{{\alpha Z}^l}. \end{gathered}$$

(20)

Equations (19) and (20) represent the gradient process of finding $\mathrm{loss}$ function pair parameters in the backpropagation process of ordinary and residual connections, respectively. When the residual connection is backpropagation, the high-level information can be directly propagated to the low level without the transformation of the intermediate weight matrix [19]. Even if there is gradient attenuation between layers $L$ and $l$, the gradient of layer $L$ can still be directly transmitted to layer $l$, realizing the cross-layer propagation of the gradient, which can solve the problem of gradient disappearance and improve the accuracy of aeromagnetic compensation. When ordinary connections are backpropagating, gradient information cannot be effectively propagated to the lower layer, so the gradient disappears.

In this paper, the T-L equation is used as the coefficient matrix ${(X}_{nx18}, \mathrm{where} n$ is the sample size) and as the input parameter of the neural network model. The total ($y_n{=H}_{tn}$) magnetic interference field is used as the output parameter of the model. The Res-Bp structure is as shown in Figure 2. Suppose there are two sets of data: a training set ($X_{train}{, y}_{train}$) and a test set ($X_{test}{, y}_{test}$). After the training set is trained by the network model ($C_{train}$), the predicted interference field of the test set is:

$${\hat{y}}_{test} = X_{test}{C}_{train}.$$

(21)

The compensated aeromagnetic data are obtained by subtracting the predicted value (${\hat{y}}_{test}$) from the real value ($y_{test}$).

The improvement ratio ($\mathrm{IR}$) is a measurement scale proposed by the Canadian Aviation Research Center (NAE). At present, this measurement scale is adopted internationally and can reflect the performance of the aeromagnetic measurement system [20,21]. Therefore, the standard deviation ($\mathrm{STD}$) and $\mathrm{IR}$ of the aeromagnetic data before and after compensation are used to evaluate the magnetic interference compensation algorithm:

$$\mathrm{STD}=\sqrt{\frac{1}{n} {\displaystyle \sum }_{i=1}^n {\left(x_i-\mu \right)}^2}$$

(22)

$$\mathrm{IR}=\frac{{\mathrm{STD}}_u}{{\mathrm{STD}}_c},$$

(23)

where $\mu$ is the arithmetic mean of the variable, and ${\mathrm{STD}}_u$ and ${\mathrm{STD}}_c$ are the standard deviations of the uncompensated and compensated data, respectively.

3. Experiment

3.1. Compensation Flight

To verify the feasibility of the method, we used an unmanned aerial vehicle (UAV) to perform a figure-of-merit (FOM) flight to obtain aeromagnetic data. As shown in Figure 3, the UAV platform is equipped with a three-axis fluxgate magnetometer to measure the three components of the geomagnetic field and a potassium pump magnetometer to measure the total geomagnetic field.

The FOM flight [22] includes maneuvers in four directions, each of which includes three ±5° yaws, three ±5° pitches, and three ±10° rolls. The flight altitude is 50 m, the duration of each maneuver is 10 s, the sampling rate is 10 Hz, and the flight duration is 8–10 min. The FOM flight path is shown in Figure 4, including flight A and flight B. The aeromagnetic interference data of the two flights are shown in Figure 5.

3.2. Compensation Experiment

To determine whether Res-Bp can effectively solve the gradient disappearance problem and improve compensation accuracy, we performed compensation experiments with the data of flights A and B. First, we used the data of flights A and B as the test set and training set, respectively, which are compensated by linear regression methods LS and RR.

The results shown in Figure 6 and

Table 1. Comparison of compensation results of LS and RR.

Test Set	Training Set	Method	STDu	STDc	IR
Flight A	Flight B	LS	2.171	0.355	6.115
		RR		0.324	6.701

show that during flight A, LS reduces STD from 2.171 to 0.355, and IR is 6.115. RR reduces STD to 0.324 and IR to 6.701, so the fitting ability of LS is weaker than that of RR, which is a biased estimation regression method for collinear data analysis. By giving up the unbiasedness of LS, the regression coefficient is obtained at the cost of some information loss and reduced accuracy. It has a higher fitting accuracy and stronger stability than LS.

We used the data of flights A and B as the training set and test set, respectively, to compensate flight B and flight A. The compensation results are shown in Figure 7 and Figure 8.

Table 2. Comparison of compensation results of BP and Res-Bp.

Test Set	Training Set	Method	STDu	STDc	IR
Flight B	Flight A	BP	2.334	0.294	7.939
		Res-Bp		0.260	8.977
Flight A	Flight B	BP	2.112	0.285	7.411
		Res-Bp		0.266	7.940

compares the results of Res-Bp and neural networks (BP). In flight B, Res-Bp reduces STD from 2.334 to 0.260, and IR is 8.977. In flight A, Res-Bp reduces STD from 2.112 to 0.266, and IR is 7.940. Compared with the linear regression method (Table 1), Res-Bp has a stronger compensation ability for complex and nonlinear data. Compared with the neural network, Res-Bp improves the compensation accuracy and fitting ability.

Because the parameter update is related to the gradient of the $\mathrm{loss}$ function, the larger the gradient, the faster the parameter update, which is conducive to the convergence of the network. Figure 9 and the $\mathrm{Loss}$ curve (Figure 10a,b) show that the gradient of Res-Bp is larger than that of BP, the convergence speed of Res-Bp is faster than that of BP, and the algorithm is more stable, demonstrating that Res-Bp effectively alleviates the gradient disappearance problem and improves the fitting ability of the network. However, as the network descends along the direction of negative gradient, the principle of weight adjustment is to take the local optimum at each step, leading to the local minimum (Figure 11), and the global optimum solution cannot be obtained. Figure 10 shows that BP and Res-Bp have local minimum values, preventing the network from reaching the expected optimal state, resulting in unstable performance.

The decisive coefficient (R²) measures the goodness of fit [23] and is expressed as:

$$R^2\left(y, \widehat{y}\right) = 1-\frac{{\Sigma }_{k=1}^m{y_k-{\widehat{y}}_k}^2}{{\Sigma }_{k=1}^m{y_k-{\overline{y}}_k}^2},$$

(24)

where $y_k$ is the real value, ${\widehat{y}}_k$ is the predicted value, and ${\overline{y}}_k$ is the mean real value.

We used R² to measure the performance of the model. The closer R² is to 1, the better the fitting degree of the regression line to the real value and the better the model’s performance. Therefore, according to the R² curve (Figure 10c,d), the fitting degree of Res-Bp is stronger than that of the neural network, and it converges faster.

4. Discussion

We demonstrated the effectiveness of combining a residual connection with a neural network in Res-Bp to improve aeromagnetic compensation accuracy. However, some magnetic interference still cannot be fully compensated, with many possible explanations. First, during the FOM flight, the flight control system of the UAV should be manually controlled, using a low-altitude flight to obtain aeromagnetic data. However, due to the influence of control and real-time meteorological factors, the flight attitude of the UAV is unstable and cannot fully comply with the FOM flight. An unstable flight attitude aggravates the multicollinearity of the model and affects the accuracy of the compensation model. Second, the number of training sets may not be sufficient. Multiple FOM flights can be carried out, and multiple flight dataset can be used as a training set to further train the network to improve its generalization ability in order to improve compensation accuracy. Third, it may be affected by factors such as the activation and loss functions in the model. Fourth, there are local minima in Res-Bp and BP in the training process, which affect the fitting ability of the network and therefore the quality of aeromagnetic data. In a future study, the compensation model will be further improved.

5. Conclusions

Based on the linear regression method used to eliminate magnetic interference, problems such as insufficient fitting ability and an unstable algorithm are encountered. Although the neural network is a nonlinear function approximation network with good performance, it’s fitting ability is better than that of linear regression and it is widely used in aeromagnetic compensation owing to the effect of gradient backpropagation and the inevitable problem of gradient disappearance on the fitting ability. Because the FOM flight of the UAV is not standardized, there will be noise in the aeromagnetic data. Even after normalization, the data still contain problems, leading to gradient problems in the network. To improve compensation accuracy, we proposed Res-Bp, which is a neural network fitting compensation algorithm based on the T-L model. By combining residual connection with a neural network, the output of a layer can directly cross several layers as the input of a later layer, which considerably improves the integrity of gradient information in the backpropagation process. Experiments with several groups of FOM flight data show that the algorithm is effective. The results show that Res-Bp has better aeromagnetic compensation ability than traditional neural networks, effectively retaining the gradient in the backpropagation process and avoiding the problem of gradient disappearance. Additionally, the convergence process of the network is accelerated, the propagation of features is strengthened, and the fitting ability of the network and the accuracy of aeromagnetic compensation are improved.