Multi-Layer Perceptron Model Integrating Multi-Head Attention and Gating Mechanism for Global Navigation Satellite System Positioning Error Estimation

Abstract

To better understand and evaluate the GNSS positioning performance, it is convenient to adopt corresponding measures to reduce the impact of errors on positioning. A GNSS positioning error estimation scheme based on an improved multi-layer perceptron model is proposed. The multi-head attention mechanism and gating operation are integrated into the multi-layer perceptron model to adaptively select and filter features, enhancing the model’s ability to understand input features. First, the original positioning error of the satellite is obtained through the Kalman filter positioning method. The data are then preprocessed to extract available features. Finally, the features are input into the constructed model for training and testing to obtain the estimated positioning error value. Two types of comparative experiments were completed. The performance of the presented model is evaluated by the root mean square error. Experimental results show that the proposed method performs well in terms of performance indicators, and has obvious advantages over other state-of-the-art methods. In particular, the root mean square error of the presented method in the first dataset is 0.239 m, which is $39.2\%$ and $17\%$ lower than the current state-of-the-art long short-term memory network and convolutional neural network, respectively. The presented method can provide higher-precision estimated values for studying the GNSS positioning error estimation problem.

1. Introduction

Global navigation satellite system (GNSS) positioning technology has a wide range of applications in aviation, navigation, vehicle navigation, and other fields. GNSS positioning technology is necessary for achieving precise flight trajectory control and target navigation in aviation and navigation [1]. Real-time positioning is required to support autonomous driving and intelligent transportation systems in vehicle navigation [2]. Accurate positioning data are needed to study crustal movement and atmospheric changes in seismic monitoring and climate research [3]. The demand for high-precision and high-reliability positioning is increasing with the continuous development and transformation of GNSS technology. Previously, scholars focused on the research of GNSS high-precision positioning, and less exploration was undertaken on the problem of GNSS positioning error estimation. However, GNSS positioning and GNSS positioning error estimation are equally important [4]. Positioning errors may lead to the inaccuracy and unreliability of GNSS positioning, which has a serious impact on practical applications. Therefore, accurately estimating and controlling the positioning error are the keys to ensuring the accuracy and reliability of GNSS positioning.

GNSS positioning is affected by errors that directly lead to inaccurate positioning results. The error sources in the GNSS positioning process mainly include satellite clock errors, ionospheric delay, tropospheric delay, multipath effect, receiver error, etc. [5]. Early research on GNSS positioning error estimation mainly focused on the estimation of a certain positioning error [6]. To minimize the impact of various factors on positioning results through modeling and analysis, many scholars have explored the problem of single error term estimation [7,8,9]. Han et al. designed a scheme to achieve high-precision positioning by estimating satellite clock errors [10]. This method has achieved excellent positioning results in both static and dynamic positioning. To address the problem that the dual-frequency ionospheric correction method ignores the high-order ionospheric error, Hoque et al. investigated a different approximation formula to estimate and correct the error caused by the excess path length [11]. Due to the lack of joint estimation of other error terms, the positioning error estimation method for a single error term has the problems of low accuracy and poor robustness. Subsequently, scholars have begun to focus on more refined positioning error modeling methods, and GNSS positioning accuracy has been improved to a certain extent [12]. The focus of related research is gradually shifting to error handling and precision improvement in complex environments [13,14,15]. Lesouple et al. studied the GNSS positioning problem in a restricted environment, and applied the sparse estimation theory to estimate the GNSS positioning error caused by different environments [16]. Jiang et al. explored the GNSS positioning problem in an environment with severe occlusion and interference, and presented an assisted joint positioning algorithm that exploits near-Earth Doppler and other measurement data to track the target position [17]. Although the research has promoted the advancement of GNSS positioning error estimation methods in complex environments, it has not solved the problem of limited estimation accuracy caused by estimating only a single error term. Nowadays, multisystem fusion has become a popular research topic [18]. Researchers have begun to study how to apply multisystem satellite observation data for error estimation and correction to improve the accuracy and reliability of positioning. Researchers have gradually shifted from estimating individual errors to estimating the overall error in the GNSS positioning process [19,20,21]. Marais et al. expatiated a GNSS positioning algorithm based on pseudorange error modeling and adaptive filtering [22]. Biswas et al. explored the influence of satellite spatial distribution on GNSS position estimation, and analyzed the GNSS positioning error estimation performance of four Kalman filters in detail [23]. Although estimating the overall error in the positioning process has been proven to be able to effectively mitigate the impact of error factors on positioning results, the method of estimating the overall error has the defect of relying on the assumptions of GNSS observation noise and motion model.

The application of GNSS positioning has introduced many complex and highly dynamic situations. How to solve the problems of error handling in complex environments and reliable positioning in highly dynamic environments urgently needs to be considered. Research on positioning error estimation methods can reduce or eliminate positioning deviations caused by additional positioning error factors. Traditional positioning error estimation methods for single error terms have the problems of low estimation accuracy and the need to make assumptions about GNSS observation noise. Although traditional methods for estimating the overall error can achieve higher estimation accuracy, they also require assumptions about GNSS observation noise. Therefore, a new GNSS positioning error estimation method is urgently needed. In recent years, the GNSS positioning error estimation field has been further developed. Scholars have applied artificial intelligence methods to address the GNSS positioning error estimation problem and obtained good results. The GNSS positioning error estimation method based on artificial intelligence does not rely on prior assumptions, and can exploit GNSS observation data to model complex positioning error factors, providing a new idea for improving the positioning accuracy and robustness of GNSS [24,25]. Closas et al. utilized an iterative approach to deal with multivariate optimization problems, and obtained the maximum likelihood estimator of the position under GNSS [26]. Kuratomi et al. explored a GNSS positioning error estimation method based on the random forest algorithm, and obtained excellent error estimation results [27]. Considering that the least squares method ignores the fact that velocity information is time-dependent, Jiang et al. investigated and implemented a GNSS position estimation method based on graph optimization, using historical states and measured values to estimate the current state [28]. By aiming at the problem that the accuracy and stability of multisource fusion navigation still cannot meet the needs of current practical applications, Bao et al. studied a GNSS position error estimation algorithm based on multitask learning [29]. To obtain more accurate GPS positioning accuracy than existing statistical and linear machine learning methods, Yang et al. utilized the long short-term memory network (LSTM) to dynamically model the factors affecting GPS positioning accuracy within a certain coherent time window and then predict the positioning error in the next second to a few seconds. The effectiveness of the presented method was verified through experiments. Implementing the presented LSTM model on hardware requires further considerations [30]. Liu et al. studied an LSTM model with infinite impulse response (IIR) filtering by constructing a dynamic adaptive filter to remove outliers and interference. This innovative solution enhances the robustness of the LSTM network when processing noisy data and achieves better experimental results [31]. Shen et al. studied the unmodeled error using historical GNSS observation data and proposed a positioning error estimation method based on convolutional neural network (CNN). The effectiveness of the presented method was verified through experiments, especially in the low-frequency component [32]. The research is of great significance to error elimination and reliability estimation in the GNSS positioning process.

Traditional GNSS positioning error estimation methods are mostly based on linear assumptions, and cannot effectively capture complex nonlinear relationships, which limits the accuracy of error estimation. GNSS positioning error estimation methods based on deep learning can comprehensively consider multiple error factors, and have powerful nonlinear modeling capabilities, which can effectively improve the accuracy of error estimation. Therefore, a GNSS positioning error estimation method is studied based on an improved multi-layer perceptron (MLP) model. The multi-head attention (MHA) and gating operation are exploited to selectively weight and transform input features, enabling adaptive feature selection and information filtering. Unlike traditional GNSS positioning error estimation methods that rely on precise mathematical models and assumptions, the proposed method is a data-driven deep learning model. The relationship between error factors is learned through historical GNSS data. The proposed method does not require assumptions about error factors, and can adaptively extract features from GNSS data, thereby providing more accurate error estimates. The main contributions of the paper are as follows:

(1)

A MLP model integrating MHA and gating operation is proposed for GNSS positioning error estimation. To our knowledge, no scholars have utilized this method to address the problem of GNSS positioning estimation.

(2)

Traditional GNSS positioning error estimation methods are usually based on linear models or empirical rules. The presented method can model nonlinear relationships more flexibly, thereby achieving higher estimation accuracy.

(3)

Two types of comparative experiments are performed. The experimental results show that the presented method has higher estimation accuracy than other GNSS positioning error estimation methods.

The remainder of the article is structured as follows. In Section 2, the GNSS positioning error estimation problem is formulated. Section 3 focuses on the presented improved MLP model. In Section 4, the performance of the improved MLP model is evaluated, and the experimental results are analyzed in detail. In Section 5, the proposed method is discussed. Finally, a brief summary is presented in Section 6.

2. Problem Formulation

GNSS positioning adopts the principle of trilateration to determine the position of a receiver by measuring the distance between the receiver and multiple satellites. However, the GNSS positioning process may be affected by various error factors. Therefore, accurately estimating the GNSS positioning error is challenging.

In an Earth-centered, Earth-fixed (ECEF) coordinate system, assuming that there are n satellites, the coordinates of the satellites are $S_i=\left(x_i,y_i,z_i\right)$, where the value of i ranges from 1 to n. The coordinates of the receiver are $P=\left(x,y,z\right)$. Different positioning methods are utilized in the GNSS positioning process, and different observation equations are obtained. The most common GNSS positioning method is pseudorange measurement positioning, which is achieved by measuring the geometric distance from the receiver to the satellite and various error factors. If each error factor is regarded as an independent random variable, the observation equation of the pseudorange measurement model is

$$P_i=∥P-S_i∥+\mathrm{c}\Delta t+T_i+I_i+M_i+{\epsilon }_i$$

(1)

where $∥P-S_i∥$ denotes the geometric distance from the receiver to satellite i, $\Delta t$ expresses the receiver clock error, $T_i$ means the tropospheric delay, $I_i$ represents the ionospheric delay, $M_i$ denotes the multipath error, and ${\epsilon }_i$ expresses the measurement error. Notably, due to space reasons, only several important GNSS positioning error factors are shown in (1). In the GNSS positioning, there are other methods, such as carrier phase measurements, which can be applied to satellite positioning.

An observation equation for each satellite is applied to describe the positional relationship between the receiver and the satellite during the GNSS positioning process. Multiple observation equations are obtained by observing multiple satellites at the same time. Then, by solving the system of simultaneous equations from multiple observation equations, the position coordinates $P=\left(x,y,z\right)$ of the receiver in the ECEF coordinate system can be solved. The above process can be realized by utilizing the least squares method or other positioning algorithms.

Assume that the true position of the receiver is $P_{\mathrm{true}}$. Then, the positioning error is $\Delta P=P_{\mathrm{true}}-P$. Therefore, the GNSS positioning error estimation problem is

$$P_{\mathrm{true}}=f\left(D\right)+\Delta P_{\mathrm{est}}$$

(2)

where $f\left(D\right)$ expresses the positioning algorithm based on the observation data D and $\Delta P_{\mathrm{est}}$ denotes the estimated value of the positioning error.

As shown in Figure 1, the presented method aims to estimate the positioning error $\Delta P$ as accurately as possible, and make the difference between $\Delta P_{\mathrm{est}}$ and $\Delta P$ as small as possible, to obtain better GNSS positioning error estimation performance.

The main application scenario of the proposed method is the positioning needs of daily electronic devices, such as smartphones. Although the positioning accuracy of code observation is lower than that of carrier phase observation, the positioning time required for code observation is shorter than that of carrier phase observation, and it is less affected by environmental changes. Therefore, code observation is more suitable for the application scenario of smartphone positioning.

3. Proposed Method

The solution to the GNSS positioning error estimation problem depends on the analysis and processing of historical data. Currently, the common methods of GNSS positioning error estimation tend to ignore the relationships among different types of data, resulting in unsatisfactory estimation accuracy. With the rapid development of artificial intelligence methods, the excellent ability of artificial intelligence methods in data processing and feature learning has been proved. Therefore, after exploring and experimenting with various artificial intelligence methods, a multi-head attention gating unit (MAGU) that combines the MHA and gating operation was designed, and then the MAGU and MLP models were integrated to address the problem of GNSS positioning error estimation. The MHA is beneficial to improving the feature extraction ability of the model. The gating operation can enhance the model’s dynamic information selection capability.

The traditional MLP model is a feedforward neural network model. It consists of multiple neural layers. Each MLP neuron is connected to all neurons of the previous layer, and receives input from the previous layer. Weighting calculations and nonlinear transformations are then performed on the input and, finally, the result is passed to the next layer. The explored model represents an enhancement of the traditional MLP model, incorporating a gating operation and a MHA, and updating the network structure and parameters.

According to Figure 2, the overall process of the designed scheme is to first extract relevant parameters from GNSS data and adopt them as feature variables. Next, the positioning error is obtained through the positioning solution, and the positioning error is selected as the target variable. Then, the feature variables and target variables are input into the improved MLP model for multiple iterations of training. Finally, the estimated value of the positioning error is output.

3.1. Improved MLP Model

Although the traditional MLP model can be used to perform the nonlinear transformation on all input features in each hidden layer, it cannot selectively process different features. Therefore, the presented method integrates the MAGU module into the network structure of the traditional MLP model. This module can dynamically select the input features so that the model can better adapt to the importance and correlation of different features, thus enhancing the feature selection ability of the model.

3.1.1. Model Network Structure

The improved MLP model mainly consists of an input layer, four fully connected layers, two MAGU modules, an output layer, and some activation functions. Four fully connected layers and two MAGU modules constitute the hidden layer of the improved MLP model. The hidden layer mainly performs a series of linear transformations and feature extractions on the input data. The sigmoid activation function is applied in the MAGU module. The fully connected layer uses the rectified linear unit (ReLU) activation function, which can improve the nonlinear ability of the model and speed up its training.

In Figure 3, assuming that the input of the presented model is x, its output after passing through the first fully connected layer and the first ReLU activation function is

$$x_1=F_1\left(W_{1}x+b_1\right)$$

(3)

where $W_1$ and $b_1$ denote the weight matrix and bias vector of the first linear layer, x means the input data, and $F_1$ expresses the ReLU activation function. The ReLU activation function [33] is

$$F_1\left(x\right)=\left\{\begin{array}{c}x\begin{array}{cc}\hspace{1em}& if\begin{array}{cc}x\ge 0& \hspace{1em}\end{array}\end{array}\\ 0\begin{array}{cc}\hspace{1em}& if\begin{array}{cc}x<0& \hspace{1em}\end{array}\end{array}\end{array}\right.$$

(4)

The output of the second linear layer can be expressed as

$$x_2=W_2{x}_1+b_2$$

(5)

After inputting $x_2$ into the first MAGU, the output is

$$x_3=g_1\odot u_1+\left(1-g_1\right)\odot x_2$$

(6)

where ⊙ means elementwise multiplication, $g_1$ represents the output of the gating operation in MAGU, and $u_1$ denotes the output of the MHA after linear transformation in MAGU.

The output of $x_3$ after the third linear layer and the second ReLU activation function can be expressed as

$$x_4=F_1\left(W_3{x}_3+b_3\right)$$

(7)

The output of the data after the fourth linear layer is

$$x_5=W_4{x}_4+b_4$$

(8)

After taking $x_5$ as the input of the second MAGU module, the output can be expressed as

$$x_6=g_2\odot u_2+\left(1-g_2\right)\odot x_5$$

(9)

The final output of the proposed model is

$$y=F_2\left(W_5{x}_6+b_5\right)$$

(10)

where $F_2$ denotes the linear activation function. The linear activation function is adopted in the output layer of the improved MLP model. The final estimation result is obtained through the identity mapping of the linear activation function.

The number of neurons in the input layer of the presented model is set to be consistent with the number of feature vectors in the dataset. The first and second fully connected layers have 128 neurons. The number of fully connected layer neurons of the first MAGU module is 128. The third and fourth fully connected layers have 64 neurons. The number of fully connected layer neurons of the second MAGU module is 64. The manuscript is studying a regression problem, and the output is a scalar. Therefore, the number of neurons in the output layer is 1.

3.1.2. Model Training and Parameter Setting

The design process of the improved MLP model mainly includes steps such as data preprocessing, model initialization, training parameter setting, loss function definition, and model performance evaluation. First, the input data are standardized and each feature is scaled to a range with a mean of 0 and a standard deviation of 1. The training data and test data in the dataset are divided at a ratio of 7:3. Then, an initialized model, which includes an input layer, a fully connected layer, an MAGU module, and an output layer, is built. Finally, the relevant parameters of the model are set and the positioning error estimation accuracy of the model is evaluated.

The inputs of the presented model in the training phase are feature variables and target variables. Feature variables include carrier-to-noise ratio, satellite clock deviation, etc. The target variable is the positioning accuracy obtained through high-precision positioning solution technology, which is regarded as the true value of the positioning error. The output of the presented model during the training phase is estimated value of the positioning error.

The adaptive moment estimation (Adam) optimizer and $L2$ regularization are adopted in the improved MLP model. Gradients are calculated by the backpropagation algorithm, and the parameters of the model are updated by adopting the optimizer during training. The Adam optimizer combines the characteristics of the momentum method and the adaptive learning rate, effectively updating the parameters and adaptively adjusting the learning rate. $L2$ regularization penalizes model weights, making the weight values tend to a smaller range. It is mainly employed to control model complexity and prevent overfitting to enhance the generalizability of the model. $L2$ regularization is

$$L{2}_{reg}=L+\lambda \sum _{i=1}^n{w}_i^2$$

(11)

where L denotes the original loss function, $\lambda$ means the weight decay coefficient, w expresses the weight, n represents the number of data samples, and i is the index.

The improved MLP model adopts the mean square error (MSE) as the loss function, and the MSE is

$$\mathrm{MSE}=L\left(y,\widehat{y}\right)={\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2$$

(12)

where n means the number of samples, $y_i$ and ${\widehat{y}}_i$ represent the true value and its estimated one. The smaller the MSE value, the smaller the difference between estimated value and true value.

3.2. Multi-Head Attention Gating Unit

One defect of the traditional MLP model is that it ignores the interdependencies between different time steps in the sequence data when processing sequence data. The presented model aims to solve this defect by adding MAGU to the MLP model. MAGU mainly consists of three parts, namely MHA, gating operation, and linear transformation. The MHA can extract features from different subspaces in parallel and better capture the dependencies between time series data. The gating operation can selectively control the information flow by ignoring irrelevant information or amplifying important information, enabling the improved MLP model to better process sequence data. A linear transformation can map input data into a new representation space, which usually includes matrix multiplication operations and offset addition operations.

The MAGU network structure in Figure 4 mainly includes an input layer, a multi-head attention layer, a gating subunit, a linear transformation operation, and an output layer. The gating subunit contains a linear layer and a sigmoid activation function, which can generate gating weights through the sigmoid function to achieve gating operations. The linear transformation layer includes a linear transformation layer and a linear activation function. Linear transformation refers to linear transformation of the input data and the hidden state of the previous time step, and then mapping them to a new feature space.

During the forward propagation, the model works as follows: First, an input is received. Then, it is passed to the multi-head attention layer. Subsequently, it is passed to the gating subunit. In the gating subunit, the gating signal is calculated by the sigmoid activation function, and after the calculation is completed, the input is passed to the linear transformation layer. Afterward, the input is linearly transformed in the linear transformation layer. The final output is the result of multiplying the gating signal by the transformation result and adding the input residual.

MAGU is a module that combines MHA and gating operation to enhance the model’s ability to process time series data. MHA can capture complex dependencies in the input sequence. Gating operation can improve the nonlinear expression ability of the model by dynamically adjusting the flow of information. Next, the two important components of the MAGU module, the MHA and the gating operation, will be introduced in detail.

3.2.1. Multi-Head Attention Mechanism

The MHA mechanism can be utilized to capture long-term dependencies in the input sequence [34]. Its principle is to adopt multiple attention heads in parallel to capture the features of different subspaces in the input sequence. First, each attention head independently performs a linear transformation on the input. Then, the attention weights are calculated. Finally, the outputs of all heads are concatenated and linearly transformed to obtain the final output.

According to Figure 5, the MHA is mainly composed of four linear layers, a scaled dot product attention layer, and a connection layer. The scaled dot product attention layer includes dot product operation, scaling calculation and softmax operation. The execution process of the MHA is to first pass the input vector through three different linear layers to generate the query matrix, key matrix, and value matrix, respectively. Afterward, the attention score matrix is obtained by calculating the dot product of the query matrix and the key matrix for each attention head. Then, the obtained attention score matrix is divided by a scaling factor to prevent the gradient from disappearing or exploding. Subsequently, a softmax operation is performed on the scaled attention score matrix to obtain the attention weight matrix. Then, the attention weight matrix is multiplied by the value matrix to obtain the weighted value matrix. Repeat the above steps for each attention head. Finally, the outputs of all attention heads are connected and passed through a linear layer to obtain the final output of the MHA mechanism.

Assume that the input vector passes through a linear layer to obtain the query, key, and value of each attention head.

$$Q_i=x{W}_i^Q,K_i=x{W}_i^K,V_i=x{W}_i^V$$

(13)

where W expresses the weight matrix and the subscript i represents the i-th attention head.

The output of each attention head can be expressed as

$$H_i=\mathrm{softmax}\left({\displaystyle \frac{Q_i{K}_i^T}{\sqrt{d_k}}}\right)V_i$$

(14)

where $d_k$ represents the dimension of the key vector.

Then, the outputs of all attention heads are concatenated and passed through another linear transformation to obtain the final output.

$$M\left(Q,K,V\right)=\mathrm{Concat}\left(h_1,\dots ,h_n\right)W^O$$

(15)

where $W^O$ means the weight of the last linear transformation layer, and h denotes the total number of attention heads.

The presented model can focus on information at different positions in the input sequence through the MHA, which improves the proposed model’s ability to understand and express the input data. It is worth noting that traditional recurrent neural networks, which are commonly used for time series feature extraction, cannot be computed in parallel, while the MHA can be computed in parallel.

3.2.2. Gating Operation

The gating mechanism regulates the flow of information by introducing control signals. The gating weights are generated by the sigmoid function, and the input and the transformed results are weighted and summed to achieve feature screening. Since the sigmoid function has a smooth curve characteristic, it can perform a nonlinear transformation on the input and help the model adapt to nonlinear relationships to learn more complex features.

The gating operation consists of a linear layer and a sigmoid activation function. Its execution process is to first receive the input from the MHA. Then, the input is linearly transformed through the linear layer. Subsequently, the gating weight is generated through the sigmoid activation function. Finally, the gating weight is passed to the next layer of the network.

The output after the gating operation can be expressed as

$$g_i=\sigma \left(W_g{H}_i+b_g\right)$$

(16)

where $W_g$ denotes the weight matrix of the gating operation, $b_g$ is the bias vector of the gating unit, $H_i$ expresses the output of the MHA in the corresponding layer, and $\sigma$ represents the sigmoid activation function.

The sigmoid activation function [35] is

$$f\left(x\right)={\displaystyle \frac{1}{1+e^{-x}}}$$

(17)

where e means the basis of the natural logarithm. The sigmoid function can map the input x into a continuous value between 0 and 1.

Since the activation function in the gating operation is differentiable, this facilitates the use of the backpropagation algorithm to calculate the gradient and adopt the optimizer to update the model parameters during training to minimize the loss function. The gating operation can promote the stable propagation of gradients, which can alleviate the problem of gradient vanishing or gradient exploding to a certain extent.

The MHA helps the model focus on important features. The gating operation can further filter the features. The adoption of MHA and gating operation enables the MLP model to effectively capture long-term dependencies in the input data, thereby improving the performance of the model.

4. Experimental Process and Result Analysis

To clearly demonstrate the performance of the proposed method, three types of experiments are conducted. First, the estimation accuracy of the presented method is comprehensively demonstrated. Then, the proposed method is compared with the most commonly used methods and the current state-of-the-art methods. Finally, an ablation experiment is performed to verify the effectiveness of the MAGU module.

Two datasets were adopted in the experiments. The datasets were collected on tree-lined streets and roads with wide views. The data were collected dynamically with a collection interval of 1 s. The true position information of the receiver was obtained from the NovAtel SPAN system. The adopted dataset contains code measurements and does not contain carrier phase measurements. The test environment is Python 3.9. The hardware is supported by RTX 3090 GPU, whose total memory is 24 GB. The first dataset was collected and publicly released by Fu et al., which consists of GNSS positioning data collected by multiple low-cost receivers in the San Francisco area of the United States [36]. The second dataset was collected and publicly released by Kuratomi et al. [27]. Both datasets were used for comparative experiments and ablation experiments. Applying the same dataset for comparison can improve the persuasiveness of the comparison experiment to a certain extent.

The experimental process of employing the first dataset is mainly as follows. First, the dataset collected and publicly released by Fu et al. was obtained. Secondly, the original positioning error of the receiver is obtained by the Kalman filter (KF) method. Then, the parameters that may affect the positioning error are extracted from the dataset as feature variables, and the obtained positioning error is adopted as the target variable. Finally, the feature variables and target variables are input into the improved MLP model together, and the estimated results of the positioning error are output after multiple iterations of training. Next, each step will be introduced in detail, and corresponding diagrams will be given.

4.1. Obtain the Original Positioning Error

The KF method is used to obtain the original positioning error in the first dataset. The positioning process not only includes a series of processing and analysis of satellite observation data and navigation message data, but also includes the correction and elimination of various error factors such as satellite clock deviation and ionospheric delay. The second dataset includes original positioning error data. Therefore, the step of calculating the original satellite positioning error can be ignored during the experiment using this dataset, and the subsequent steps can be directly carried out.

According to Figure 6, the calculated positioning error fluctuates greatly, with a minimum error of about 0.1 m and a maximum error of more than 6 m. The average positioning error is calculated to be 2.840 m. Notably, the purpose of the designed method is mainly to explore positioning error estimation, which does not depend too much on the accuracy of the original positioning error. Therefore, not only the KF method can be used to obtain the value of the positioning error, but also other GNSS positioning methods can be utilized to obtain the value of the positioning error.

4.2. Preprocessing the Experimental Dataset

The original satellite observation and navigation message files contain many parameters. However, not all of these parameters are directly related to the positioning error estimation. Therefore, the data in these files need to be preprocessed. Data preprocessing includes unifying the data value range and preliminary feature screening. Since the value ranges of different data are different, it is necessary to standardize the data. The standardization operation is to subtract the mean of the same type of data from the data to be standardized and divide it by the standard deviation, so as to achieve the purpose of scaling different data in proportion.

The main features in the dataset and their corresponding descriptions are listed in

Table 1. The main features in the dataset and their corresponding descriptions.

Feature	Description
Satellite clock difference	The difference in time between an atomic clock on a satellite and a reference clock on Earth.
Satellite clock deviation	The offset of an atomic clock on a satellite from its theoretically precise time.
Satellite elevation	The angle between the satellite and the horizon where the receiver is located.
Ionospheric delay	The ionospheric delay means that when a satellite signal passes through the ionosphere, the free electrons in the ionosphere will scatter and reflect the satellite signal, thereby changing the propagation path of the satellite signal. The ionospheric delay is obtained by the Klobuchar model.
Satellite azimuth	The angle between the line from the ground observation point to the satellite and the due north direction.
Tropospheric delay	The tropospheric delay means that when a satellite signal passes through the atmosphere, the water vapor in it will cause the satellite signal to refract, thereby changing the propagation path of the satellite signal. The tropospheric delay is obtained by the Saastamoinen model.
Carrier-to-noise ratio	The ratio of the carrier power of the received signal to the noise power in the surrounding environment.
Satellite coordinate information	The satellite position information in the ECEF coordinate system is calculated using the satellite broadcast ephemeris.

. These features are utilized in the training and testing processes of the improved MLP model. The quality of the feature selection strategy will directly affect the GNSS positioning error estimation accuracy of the presented method. The improved MLP model mainly selects features through MHA and gating operation. Then, the weights are updated according to the gradient, thereby gradually optimizing the performance of the model.

4.3. Performance Analysis of the Improved MLP Model

Next, the improved MLP model is trained and tested with the two datasets. To demonstrate the performance of the presented model more intuitively, the root mean square error (RMSE) is adopted in the experiments as the evaluation index of the estimation accuracy of the model. The smaller the RMSE value is, the closer the estimated value is to the true value, and the higher the accuracy of the model. The formula for calculating the RMSE [37] is

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(y_i-{\widehat{y}}_i\right)}^2}$$

(18)

where n denotes the number of samples, $y_i$ represents the true value, and ${\widehat{y}}_i$ represents the estimated value.

The x-axis and y-axis in Figure 7 are the training epochs and loss values of the model during the training process, respectively. A total of 500 epochs are iterated during the model training process. In accordance with Figure 7, the loss of the model is continuously reduced during the training process and is ultimately maintained at a low value, demonstrating that the model has converged normally and achieved better performance.

The general idea of GNSS positioning is adopted, and the output results of the model are divided into three components: East, North, and Up. Figure 8 shows the experimental results of the proposed model on the three components of East, North, and Up. It can be seen from Figure 8 that the trained and converged model has a high estimation accuracy.

After the two datasets are input into the model separately, the model gradually converges after several iterations of training. The presented method obtains RMSE values of 0.239 m and 0.029 m for the first and second datasets, respectively. The random forest (RF) algorithm is widely adopted in the field of GNSS positioning error estimation, and it has high estimation accuracy. Therefore, to demonstrate the performance of the presented method more clearly, the experimental results of the presented method and the RF algorithm will be displayed and comparatively analyzed next.

The gray portion in Figure 9 indicates the confidence interval for the positioning error estimate. The confidence interval is denoted by the area of the estimated value minus two standard deviations and the estimated value plus two standard deviations. The confidence interval is a concept from statistics, which means that the true value has a certain probability of falling within a certain interval around the estimated value. The confidence interval is an interval range and is often used to indicate the credibility of the estimated value. Applying confidence intervals to the problem of positioning error estimation is mainly used to describe the uncertainty range of positioning error estimation, which is conducive to evaluating the reliability and stability of the proposed model.

As shown in Figure 9, the estimated values of the presented method have a large repetition range with respect to the true values, and they are basically within the confidence interval, which shows that the estimation results of the positioning errors of the presented method have high accuracy. However, the RF algorithm is sensitive to noise and outliers in the data, which affects the estimation accuracy of the algorithm. On the whole, the presented method has a larger repetition range of estimated values and true values compared to the RF algorithm. The estimation accuracy of the presented method is significantly better than the RF algorithm.

It can be seen from Figure 10 that most of the estimation errors of the presented method are approximately distributed around 0, and the difference does not generally exceed 0.1. This shows that the estimated values obtained by the presented method are more accurate and have better performance. The RF algorithm is a model based on decision trees that cannot handle the interdependence in time series data well. Compared with the RF algorithm, the presented method has a more concentrated error distribution and the overall error estimate is smaller.

The Stanford diagram can show how well the estimated positioning error matches the original positioning error. The closer to the original positioning error the estimated positioning error is, the closer the data points are to the diagonal. According to Figure 11, most of the data points of the presented method are in the lower left corner of the graph, and only a very small number of data points are distributed on either side of the diagonal. This shows that the presented method obtains more accurate estimates. However, part of the data in the positioning error estimate obtained by the RF algorithm is biased, and there are a large number of data points where the positioning error estimate is less than the true value of the positioning error.

A residual is defined as the difference between the true value and an estimated value. The residual histogram clearly shows the residual distribution of the proposed method. The x-axis and y-axis in Figure 12 are the size and frequency of the residual value, respectively. Based on Figure 12, the residual histogram of the presented method has a higher kurtosis than the residual histogram of the RF algorithm, and the peak value is closer to 0. This shows that the residual distribution of the presented method is relatively concentrated, the overall value is small, and it has better estimation performance.

Table 2. Quantitative analysis experimental results of the proposed method and random forest algorithm (unit: m).

Methods	RMSE Value	MAE Value	Standard Deviation
RF	0.047	0.022	0.016
Ours	0.029	0.014	0.010

records the quantitative analysis results of the proposed method and the random forest algorithm. RMSE, mean absolute error (MAE), and standard deviation are used as evaluation indicators. It can be seen from Table 2 that the performance of the proposed method is better than that of the random forest algorithm in all three evaluation indicators.

4.4. Comparative Experiments

To better demonstrate the performance of the presented method, two comparative experiments are carried out. The first comparative experiment is between the presented method and other common methods, and the second comparative experiment is between the presented method and methods studied by other scholars.

4.4.1. Comparison of the Presented Method with Other Common Methods

Common methods in the GNSS positioning error estimation research mainly include the extended Kalman filter (EKF), linear regression (LR), support vector machine (SVM), decision tree (DT), and RF. These five common methods and the presented method are compared in the first comparative experiment.

By incorporating the MAGU module into the MLP model, the presented method minimizes the influence of unimportant features on the model, enabling the model to achieve better performance. It can be seen from

Table 3. Comparative experimental results of the proposed method and five common methods (unit: m).

Methods	RMSE Value on the First, Dataset	RMSE Value on the Second, Dataset
EKF	1.324	0.118
LR	0.864	0.071
SVM	0.618	0.065
DT	0.527	0.060
RF	0.473	0.047
Ours	0.239	0.029

that the presented method obtains smaller RMSE values for both datasets and has obvious advantages over other common methods. This shows that the estimation accuracy of the presented method is higher.

4.4.2. Comparison of the Presented Method with the Current State-of-the-Art Method

To our knowledge, the current state-of-the-art methods are the long short-term memory network model studied by Yang et al. [30] and the convolutional neural network model designed by Shen et al. [32] when dealing with the GNSS positioning error estimation problem. Therefore, the proposed method is compared with two state-of-the-art GNSS positioning error estimation methods to show the superiority of the presented method in the second comparative experiment.

The LSTM model is a method commonly applied to predict time series data. It usually consists of input gate, forget gate, output gate, and cell state. Compared with the MLP model, it requires additional gating units and loop connections. The CNN models are mostly used to process image data, and are not suitable for processing data containing geographic information. It can be seen from

Table 4. Comparative experimental results of the proposed method and two state-of-the-art methods (unit: m).

Methods	LSTM [30]	CNN [32]	Ours
RMSE value on the first dataset	0.393	0.288	0.239
RMSE value on the second dataset	0.039	0.033	0.029

that the RMSE value of the presented method is significantly smaller than that of the other two methods, especially compared with the LSTM method. The proposed method can adaptively choose which features to retain based on the importance of the input features. This characteristic can improve the representation ability and generalization performance of the model to a certain extent. Since the experiment was conducted with the same dataset, the results of the comparative experiment are convincing to a certain extent.

4.5. Ablation Experiment

To verify the effectiveness of the MHA mechanism and gating operation, ablation experiments are performed. The ablation experiment consists of four groups of experimental objects. The control variable method was adopted in the ablation experiment. Four groups of experimental subjects were tested separately under the premise that other parameters remained unchanged.

The MLP model without the MHA mechanism and the gating operation has difficulty effectively processing different features in the data due to the lack of feature importance screening ability. Therefore, it can be seen from

Table 5. Ablation experiment results of the proposed model (unit: m).

Gating Operation	MHA	RMSE on the First, Dataset	RMSE on the Second, Dataset
		0.449	0.042
✓		0.427	0.041
	✓	0.398	0.038
✓	✓	0.239	0.029

that the RMSE values obtained by the MLP model with the MHA mechanism and the gating operation are smaller than those of the MLP model without the MHA mechanism and the gating operation on both datasets. This shows that the proposed model gives full play to the advantages of the MHA mechanism and the gating operation, and can better learn and extract features in the data and obtain more accurate estimation accuracy. In addition, from the experimental results of the MLP model with only the gating operation and the MLP model with only the MHA mechanism, it can be seen that the MHA mechanism improves the model performance more than the gating operation.

5. Discussion

In applications that require high positioning accuracy, if the errors of the GNSS positioning system are not promptly warned or corrected, serious consequences may occur. The purpose of studying GNSS positioning error estimation methods is to evaluate and correct errors, ensure that GNSS can still provide high-quality positioning services in complex environments, and avoid large deviations in positioning results. As can be seen from Table 3, the proposed method achieves a lower RMSE value than the traditional method. This is because the traditional GNSS positioning error estimation method has the following shortcomings:

(1)

Traditional positioning error estimation methods rely on precise assumptions, and have difficulty accurately reflecting the error size in complex environments. Therefore, the estimation accuracy is usually low.

(2)

There are large differences in orbital errors, clock biases, and other factors between different satellite systems when multiple GNSS systems are used together. Traditional GNSS positioning error estimation methods have difficulty compensating for errors between multiple systems at the same time, and are prone to cause accumulation and cross-interference of errors between multiple systems.

However, the GNSS positioning error estimation method based on deep learning has powerful nonlinear modeling capabilities. It can learn the trend of error changes from historical data to effectively handle complex error factors, and has the potential to further improve accuracy with the support of large-scale data. Therefore, a new multi-layer perceptron model is proposed for GNSS positioning error estimation. Multi-head attention mechanisms and gating operations are designed to improve the standard MLP model. The improved MLP model can better capture the relationship between input features by combining multi-head attention mechanism and gating mechanism. It can be seen from the two types of comparative experiments that the proposed model not only has better performance than other commonly used methods, but also has obvious advantages over the current state-of-the-art methods.

Our study aims to demonstrate the potential of the presented method in dealing with real-time position error estimation problems. However, in actual applications, factors such as data transmission delay, data reliability, and application performance requirements may need to be considered. In the face of building occlusion in urban environments and signal interference in natural environments, additional measures need to be taken to deal with data uncertainty to improve the stability and reliability of position estimation.

6. Conclusions

Traditional GNSS positioning error estimation methods rely on prior assumptions about observation noise and motion models, and have difficulty handling the complex nonlinear relationships between different error factors. However, the GNSS positioning error estimation method based on deep learning does not require prior assumptions. It can extract effective information from a large amount of historical data, and capture the complex nonlinear relationship between different error factors, thereby improving the accuracy of positioning error estimation. Therefore, a GNSS positioning error estimation method based on deep learning is explored. The nonlinear modeling ability and feature selection ability of a MLP model are improved by introducing a multi-head attention mechanism and gating operations into the MLP model. Experimental results show that the presented method has better performance than that of the extended Kalman filter, linear regression, decision tree, support vector machine, and other methods for two datasets, and the presented method is also better than the current state-of-the-art long short-term memory network and convolutional neural network considering comparative experiments. This shows that the presented method can reduce the influence of errors on the positioning results to a certain extent, and has more accurate positioning error estimation capabilities. Currently, the proposed model only estimates the positioning error based on GNSS data. However, other sensors, such as IMU and visual sensors, also provide valuable information in practical applications. In the future, we will explore how to fuse data from different sensors to further improve the error estimation accuracy of the model in complex environments.