Transformer Aided Adaptive Extended Kalman Filter for Autonomous Vehicle Mass Estimation

Abstract

Vehicle mass is crucial to autonomous vehicles control. Affected by the nonlinearity of vehicle dynamics between vehicle states, it is still a tough issue to estimate vehicle mass precisely and stably. The transformer aided adaptive extended Kalman filter is proposed to further improve the accuracy and stability of estimation. Firstly, the transformer-based estimator is introduced to provide an accurate pre-estimation of vehicle mass, with the nonlinear dynamics among vehicle states being learned. Secondly, on the basis of comparing the real-time input and training data of neural network, the weight adjustment module is designed to present an adaptive law. Finally, the adaptive extended Kalman filter is proposed to meet the demand of accuracy and stability, where the pre-estimation of transformer-based estimator is integrated with the adaptive law. Dataset is collected by conducting heavy-duty vehicle simulation. The mean absolute percentage error, mean absolute error, root mean square error and convergence rate averaged over simulation tests are 0.90%, 256.47 kg, 357.01 kg and 184 steps, respectively. The results show the outperformance of the proposed method in terms of accuracy and stability.

1. Introduction

With the rapid emergence of various advanced technologies in different fields, such as computers and automobiles, the development of autonomous vehicles has become thriving in recent years. Autonomous vehicles have landed in various scenarios with the help and combination of those technologies. These vehicles are always equipped with different electronic control systems, which are driven with feedback regulation [1,2]. Therefore, the accurate estimation of vehicle real-time states and parameters can significantly improve the performance of vehicle control systems. As one of them, vehicle mass has a significant impact on the performance of systems that rely on vehicle dynamics, such as electronic stability program (ESP), anti-lock brake system (ABS), etc. [3]. Moreover, vehicle mass holds significant importance in vehicle dynamic modeling, and the estimation of other vehicle states, such as lateral velocity and sideslip angle, heavily depends on such dynamic model [4,5]. However, it is faced with a challenging problem since vehicle mass, especially for coaches and trucks, changes greatly depending on real-time payloads. Therefore, developing a method for determining the vehicle mass is critical to the safety and performance of autonomous vehicles, and has been extensively investigated by an increasing number of researchers in recent years so as to tackle the challenge.

Generally speaking, there are three types of parameter estimation methods for vehicle mass: sensor-based, model-based, and data-driven [6,7,8,9,10,11,12]. Sensor-based can be further divided into two kinds: fixed sensors and on-board sensors. Fixed sensors like weighbridges are the main weighing equipment used by factories, mines and businesses for bulk cargo measurement because of their benefits of sturdy construction and convenient use. However, its application has been constrained by a lack of deactivation activity. On-board sensor research naturally evolved to address this issue [6]. With the installation of on-board sensors, mass weighing may be conducted while driving at any moment. However, the performance of these sensors is subject to the complex and changeable working environment, and the installation of additional sensors increases the cost of vehicle manufacturing.

The model-based approach primarily comprises three essential components, namely the sensor data, vehicle model, and the core algorithm, which are of lower cost compared to sensor-based ones [13,14,15,16]. In terms of sensor data, Rajamani and Hedrick adopted an additional linear variable differential transformer (LVDT) for observational purposes when constructing an adaptive vehicle states observer [17]. Jensen et al., in contrast, designed an approach that utilized solely parameters obtained from the on-board diagnostics (OBD) system to estimate mass [18]. Depending on the specific perspective chosen, various types of models can be established, including the lateral dynamic model, suspension dynamic model, longitudinal dynamic model, and others [17,18,19,20]. Given its dominant role throughout the driving process and exclusive use of pre-existing sensors, the longitudinal dynamic model facilitates the utilization of a larger amount of collected data for calculations and is frequently utilized by researchers.

With regard to the core algorithms, the majority of model-based approaches utilize filtering methods. Cai et al. presented a two-stage structured algorithm based on an extended Kalman filter (EKF) for estimating load and slope [21]. However, EKF suffers from inaccurate approximations for highly nonlinear models due to its reliance on first-order Taylor series expansions. In contrast, the unscented Kalman filter (UKF) offers a more precise approximation of nonlinear models using the unscented transform, even when they are highly nonlinear. Therefore, Jin et al. employed UKF in their dual-structure approach for online estimation of inertial parameters, including vehicle mass [22]. Nonetheless, it should be noted that Kalman filter-based suboptimal approximations are limited by their reliance on Gauss approximation. In an effort to overcome this limitation, Sun et al. utilized a particle filter (PF) to estimate the electric vehicle mass, which is a non-linear, non-Gauss filter and entirely unconstrained by the limitations of the Kalman filtering framework [23]. However, PF may suffer from particle degeneracy and sampling error. Hence, the pursuit of a universal and precise method that remains impervious to the vagaries of nonlinearity is of great importance for the accurate estimation of vehicle mass.

The data-driven approach presents an appealing alternative for vehicle state estimation, as it does not require costly additional hardware facilities and can learn nonlinear input-output relationships [24,25,26]. In a recent study, Korayem et al. compared a model-based method with a data-driven approach based on a multi-layer perceptron (MLP) for estimating the mass of a trailer tractor and found that the latter produced superior results [27]. However, these methods rely heavily on training data and can lead to unpredictable results if the distribution of test data does not match that of the training data, thus causing a reduction in accuracy and stability, and possibly leading to harm in control systems. To address this issue, Sieberg et al. suggested a hybrid estimating method using long short-term memory (LSTM) and UKF with a confidence level to enhance reliability in estimating other vehicle states [28].

Nevertheless, LSTM may suffer from vanishing gradients in long sequences, leading to diminished performance. As an alternative to LSTM, Transformer has gained increasing popularity in fields like computer vision (CV) and natural language processing (NLP), with its accuracy in time series regression having been proved [29]. While UKF is a popular choice of filter for state estimation, it requires more tuning parameters and may suffer from numerical stability issues. Moreover, the confidence level calculation method proposed in [28] can be time-consuming. To address these challenges and improve the accuracy and stability of autonomous vehicle mass estimation, we propose the transformer aided adaptive extended Kalman filter (TA-AEKF) method, which combines transformer and EKF with a fast confidence level calculation method. Our paper makes the following contributions:

• A novel estimator based on transformer is introduced to provide an accurate pre-estimation of vehicle mass by learning nonlinear dynamics from vehicle data under diverse conditions, thus acting as a virtual observation for EKF.
• A fast confidence level calculation method is designed for weight adjustment based on the conformity level between the real-time input and the training data, to adaptively determine the impact of transformer pre-estimation.
• TA-AEKF is proposed to integrate the transformer pre-estimation into EKF with the adaptive weight adjustment in order to precisely and stably estimate vehicle mass.

The rest of this paper is organized as follows: In Section 2, the proposed method is explained in detail. In Section 3, simulation data are collected. In Section 4, experiments are conducted for method validation. Conclusions are made in Section 5.

2. Transformer Aided Adaptive Extended Kalman Filter

2.1. Framework

The proposed method comprises primarily three components. First, the acquired dataset is used to train the transformer neural network offline to dig deep information. When a new sequence of input variables is fed into the neural network for online estimation, a pre-estimated mass value will be produced. Second, the weight $\tau$ is determined by inputting the feature vector into the weight adjustment module at the same time. Finally, the pre-estimated mass value will be embedded into the EKF as the expanded observation quantity, and the weight $\tau$ will be utilized to balance the weight between the neural network and EKF. The algorithm structure of TA-AEKF is shown in Figure 1.

2.2. Transformer-Based Vehicle Mass Pre-Estimation

2.2.1. Preliminaries

A neural network is used for vehicle mass estimation and adaptive EKF (AEKF) design in this paper. Different kinds of neural networks, such as MLP, LSTM have been used for parameter estimation or even controller design in various fields for their excellence in approximating nonlinear functions [30,31,32,33]. Compared with MLP, LSTM is more frequently employed in time series regression problems to extract more hidden information. However, LSTM can suffer from vanishing gradients in long sequence data. Shown in Figure 2, a transformer neural network, where no decoder is adopted to solve this problem, for time series regression is proposed, which is different from traditional ones originally adopted in NLP with the encoder-decoder structure [34].

Firstly, the input feature will be projected from w-dimension features into n-dimension. $z_t$, the sum of the embeddings of the input signal and positional embeddings of the input signal, will then be passed into the encoder block. Here adopt the learnable positional embeddings in order to mine the sequential information in the time series. The encoder block is composed of several encoder layers. Each encoder layer mainly includes the fully connected layer and the self-attention layer, the core component of the transformer. The output Z of attention is calculated by:

$$Z=softmax\left(\frac{Q{K}^T}{\sqrt{d_k}}\right)V$$

(1)

where Q, K, and V are the queries, keys, and values matrix, respectively, and $d_k$ the dimension of the tensor, is used to scale the attention output.

In multi-head attention, the output of all attention heads will be concatenated and then multiplied by the weight matrix W to produce the output of the encoder layer $Z_C$:

$$Z_C=\mathrm{Concatenate}\left(Z_1,\dots ,Z_h\right)W$$

(2)

where h is the number of attention heads.

Features extracted by the encoder $E=[e_1;...;e_t;...;e_w]\in {\mathbb{R}}^{n\times r}$ are input to a fully connected layer in the linear block.

$$E^{\prime }=flatten\left(E\right)\in {\mathbb{R}}^{n·r}$$

(3)

$$\widehat{y}=W_o{E}^{\prime }+b_o$$

(4)

where $W_o\in {\mathbb{R}}^{l\times (n·r)}$ and $b_o\in {\mathbb{R}}^l$ are used to project the $(n·w)$-dimension features into l-dimension, $\widehat{y}\in {\mathbb{R}}^l$ is the estimated regression output of neural network.

For the regression problem, the quadratic loss function will be adopted:

$$L=\parallel \widehat{y}{-y\parallel }^2$$

(5)

where y is the ground truth of the output variable, in this case, the total mass of the vehicle ($m_{total}$) and $l=1$.

2.2.2. Input Feature Selection

Input features should be first decided properly to better study the nonlinear relationships between vehicle states. During the entire driving process, vehicle longitudinal motions is more often than lateral one. Then the vehicle longitudinal dynamic model can be chosen to select neural network features and design the adaptive EKF subsequently.

With the help of the transmission system, automobile driving torque $T_{tq}$ can finally go to the driving wheels. The corresponding reaction force of the ground will propel the vehicle move forward. The various forces on the vehicle during the uphill process are as shown in Figure 3. Considering only the longitudinal dynamics, the model of the vehicle can be developed as follows:

$$F_t=F_f+F_w+F_j+F_i$$

(6)

where $F_t$, $F_f$, $F_w$, $F_j$, and $F_i$ refer to the driving force, rolling resistance, air resistance, acceleration resistance and slope resistance, respectively.

The driving force $F_t$ can be expressed as:

$$F_t=\frac{T_{tq}{i}_g{i}_0{\eta }_T}{r}$$

(7)

where $i_g$ is the gear ratio of the transmission, $i_0$ is the gear ratio of the final drive, ${\eta }_T$ is the transmission mechanical efficiency, r is the effective radius of rotating tire, respectively.

The acceleration resistance $F_j$ is defined as:

$$F_j=\delta m{a}_x$$

(8)

where $\delta$ is the rotating mass conversion factor, m is the total vehicle mass, $a_x$ is the vehicle longitudinal acceleration, respectively.

The air resistance $F_w$ when there is no wind is as follows:

$$F_w=\frac{1}{2}\rho C_d{A}_f{u}_x^2$$

(9)

where $\rho$ is the air density, $C_d$ is the aerodynamic drag coefficient, $A_f$ is the frontal area of the vehicle, $u_x$ is the vehicle longitudinal speed, respectively.

The rolling resistance $F_f$ is defined as:

$$F_f=mgfcos\alpha$$

(10)

where g is the acceleration of gravity, f is the coefficient of rolling resistance, and $\alpha$ is the angle of slope, respectively.

The slope resistance $F_i$ is defined as:

$$F_i=mgsin\alpha$$

(11)

Integrate formulas of each force into (6) yields:

$$\frac{T_{tq}{i}_g{i}_0{\eta }_T}{r}=\delta m{a}_x+\frac{1}{2}\rho C_d{A}_f{u}_x^2+mgfcos\alpha +mgsin\alpha$$

(12)

Given that the transmission ratio is not always available in advance and that the effective tire radius varies and is challenging to determine, the transmission to wheel ratio is approximated here using the vehicle’s speed and engine speed:

$$i_r=\frac{i_g{i}_0}{r}=\frac{\pi }{30}\frac{n}{u_x}$$

(13)

where $i_r$ is defined as the ratio of $i_g{i}_0$ to $\mathrm{r}$.

Therefore, (12) is transformed into:

$$\delta m{a}_x=\frac{\pi }{30}\frac{T_{tq}{\eta }_{T}n}{u_x}-\frac{1}{2}\rho C_d{A}_f{u}_x^2-mg(fcos\alpha +sin\alpha )$$

(14)

As seen in (14), at the very least, driving force, longitudinal acceleration, longitudinal velocity, and slope angle should be gathered in order to determine vehicle mass. They can be collected using sensors, such as IMU and GNSS, which are frequently used in autonomous vehicles.

The neural network input feature vector is expressed as follows:

$$A_{input}={\left[\begin{array}{ccccc}{F}_t& F_w& \delta a_x& (fcos\alpha +sin\alpha )g& v_x\end{array}\right]}^T$$

(15)

where $v_x$ is velocity expressed in km/h.

The transmission of information in a classic feedforward neural network is unidirectional, with the current input being transmitted forward layer by layer through the neural network, corresponding to the current output. In real tasks, however, the output of the current time is influenced not just by the current time input, but also by the historical time. As a result, the back step of $A_{input}$ is taken into account and supplied into the neural network.

2.2.3. Neural Network Architecture

For multi-head attention, 16 parallel attention heads are adopted. The amount of dropout applied to all linear layers is 0.3. The encoder block applied consists of 3 encoder layers.

2.3. Confidence Level for Weight Adjustment

The weight between EKF and the transformer neural network estimation results is represented by the weight factor, which impacts their weight in the final estimation results. The confidence level is calculated by comparing the training and test datasets for similarity. A subspace partition of training data and mapping of test data is presented in [28]. However, it necessitates a significant amount of computing. A new approach is being developed here. Assume the variable $x^{\prime }={\left[\begin{array}{cccc}{F}_t& a_x& v_x& \alpha \end{array}\right]}^T$ in the training dataset’s state quantity vector x follows the multivariate normal distribution:

$$p(x^{\prime };\mu ,\Sigma )=\frac{1}{{\left(2\pi \right)}^{\frac{n}{2}}{|\Sigma |}^{\frac{1}{2}}}exp\left(-\frac{1}{2}{(x^{\prime }-\mu )}^T{\Sigma }^{-1}(x^{\prime }-\mu )\right)$$

(16)

where $\mu$ is the mean vector, $\Sigma$ is the covariance vector, n represents the number of features of x, and $|\Sigma |\equiv det\Sigma$ is the determinant of $\Sigma$. Therefore, the probability density at $\mu$ of the training data is:

$$p_{max}=\frac{1}{{\left(2\pi \right)}^{\frac{n}{2}}{|\Sigma |}^{\frac{1}{2}}}$$

(17)

During the application of estimation, every time a new datapoint is fed into the neural network, substitute its value into (16), and get the corresponding probability density:

$$p_k=\frac{1}{{\left(2\pi \right)}^{\frac{n}{2}}{|\Sigma |}^{\frac{1}{2}}}exp\left(-\frac{1}{2}{(x_k^{\prime }-\mu )}^T{\Sigma }^{-1}(x_k^{\prime }-\mu )\right)$$

(18)

Here, the probability density of the test dataset point will be divided by the maximum probability density in the training dataset, i.e., $p_{max}$ to get the final confidence level:

$${\tau }_k=\frac{p_k}{p_{max}}=exp\left(-\frac{1}{2}{(x_k^{\prime }-\mu )}^T{\Sigma }^{-1}(x_k^{\prime }-\mu )\right)$$

(19)

2.4. AEKF

In comparison to RLS, KF is a correction-based recursive algorithm. It realizes the parameter estimation of a linear system with the minimum variance criterion. KF and its variants are frequently adopted for state estimation in linear systems [35,36]. For systems with non-linear transition equations or measurement equations, EKF is derived to linearize the equations. Different Kalman filter extensions (e.g., UKF, cubature KF (CKF)) have been adopted in various fields [37,38,39]. However, EKF performs better in terms of real-time performance, computing efficiency and parameter tuning in engineering applications. It is assumed that the nonlinear system has the following nonlinear state transition equation and observation equation:

$$\begin{array}{cc}\hfill x_k& =g(x_{k-1},{\mu }_{k-1})+w_{k-1}\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill y_k& =h\left(x_k\right)+v_k\hfill \end{array}$$

(21)

where $x_k$ is the state quantity at step k, $y_k$ is the observation quantity, $\mu$ is the control quantity, g is the transition function, h is the observation function, w is process noise, subject to $N\left(0,Q^w\right)$, v is observation noise, subject to $N\left(0,R^v\right)$.

The procedure of EKF is a series of successive operations of prediction based on state input and update based on measurement output.

First, the prior state estimation of state quantity ${\widehat{x}}_k^{-}$ is calculated by:

$${\widehat{x}}_k^{-}=g({\widehat{x}}_{k-1},{\mu }_{k-1})$$

(22)

and then the prior state estimation of covariance matrix $P_k^{-}$ is calculated by:

$$P_k^{-}=A{P}_{k-1}{A}^T+Q^w$$

(23)

The Kalman gain $K_k$ is calculated by:

$$K_k=P_k^{-}{H}^T{\left(H{P}_k^{-}{H}^T+R^v\right)}^{-1}$$

(24)

Therefore, the posterior state estimation of state quantity ${\widehat{x}}_k$ is corrected with measurements by:

$${\widehat{x}}_k={\widehat{x}}_k^{-}+K_k(y_k-H{\widehat{x}}_k^{-})$$

(25)

and the posterior state estimation of covariance matrix $P_k$ is updated by:

$$P_k=\left(I-K_{k}H\right)P_k^{-}$$

(26)

Here, A and H are the Jacobian matrices, calculated by the partial derivative of state transition function g and observation function h to state quantity x, respectively.

$$\begin{array}{c}\hfill A=\frac{\partial g}{\partial x}\end{array}$$

(27)

$$\begin{array}{c}\hfill H=\frac{\partial h}{\partial x}\end{array}$$

(28)

In order to apply EKF, the state space model is established. The longitudinal speed and vehicle mass are selected as the state quantity:

$$x_k={\left[\begin{array}{cc}{v}_x& \frac{1}{m}\end{array}\right]}^T$$

(29)

The state equation form is derived to be:

$$\left[\begin{array}{c}{v_x}_k\\ \frac{1}{m_k}\end{array}\right]=\left[\begin{array}{c}{g}_1\\ g_2\end{array}\right]+w_{k-1}=\left[\begin{array}{c}{v}_{k-1}+a_x\Delta T\\ \frac{1}{m_{k-1}}\end{array}\right]+w_{k-1}$$

(30)

The observation quantity is chosen as follows:

$$y_k=\left[\begin{array}{c}{v}_{x_k}\end{array}\right]$$

(31)

The Jacobian matrix is:

$$A=\left[\begin{array}{cc}\frac{\partial g_1}{\partial v}& \frac{\partial g_1}{\partial \frac{1}{m}}\\ \frac{\partial g_2}{\partial v}& \frac{\partial g_2}{\partial \frac{1}{m}}\end{array}\right]=\left[\begin{array}{cc}{A}_{11}& A_{12}\\ 0& 1\end{array}\right]$$

(32)

where

$$\begin{array}{cc}\hfill A_{11}& =1-\frac{3.6\Delta T}{\delta m}\left(\frac{3\pi }{25}\frac{T_{tq}{\eta }_{T}n}{v^2}+\frac{1}{12.96}\rho C_{D}A{v}_x\right)\hfill \end{array}$$

(33)

$$\begin{array}{cc}\hfill A_{12}& =\frac{3.6\Delta T}{\delta }\left(\frac{3\pi }{25}\frac{T_{tq}{\eta }_{T}n}{v}-\frac{1}{25.92}\rho C_{D}A{v}_x^2\right)\hfill \end{array}$$

(34)

The new observation quantity is chosen by expanding the original observation $y_k$ with the transformer pre-estimation ${y_{nn}}_k$ as follows:

$$\begin{array}{cc}\hfill {\tilde{y}}_k& =\left[\begin{array}{cc}{y}_k& {y_{nn}}_k\end{array}\right]\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill {y_{nn}}_k& =\frac{1}{{m_{nn}}_k}\hfill \end{array}$$

(36)

where ${m_{nn}}_k$ will be the estimated mass value of neural network.

Based on the weight adjustment module, an adaptive law is utilized to integrate the pre-estimation of the transformer into the AEKF. The flow chart of AEKF is shown in Figure 4. The pseudocode of TA-AEKF is shown in Algorithm 1, where $R_{nn}^v={\sigma }_{m_{nn}}^2$ represents the observation covariance matrix increased by the neural network. Instead of using ${\tau }_k$ and $(1-{\tau }_k)$ as the weight factor, here divide them by ${\tau }_k$ and use $\left({\tau }_k\right)/{\tau }_k=1$ and $(1-{\tau }_k)/{\tau }_k$ to adjust the observation covariance matrix of neural network.

Algorithm 1 Transformer Aided Adaptive Extended Kalman Filter

function Deep Adaptive EKF  ${\widehat{x}}_0\leftarrow$ Initialization  $P_0\leftarrow$ Initialization  for $k\leftarrow 1$ to N do    ${\widehat{x}}_k^{-}\leftarrow g({\widehat{x}}_{k-1},{\mu }_{k-1})$    ${\tilde{Q}}_k^w\leftarrow Q_k^w$    $P_k^{-}\leftarrow A{P}_{k-1}{A}^T+{\tilde{Q}}_k^w$    ${\tilde{R}}_k^v\leftarrow \left[\begin{array}{cc}{R}^v& 0\\ 0& \left(1-{\tau }_k\right)/{\tau }_k{R}_{nn}^v\end{array}\right]$    $K_k\leftarrow P_k^{-}{H}^T{(H{P}_k^{-}{H}^T+{\tilde{R}}_k^v)}^{-1}$    ${\tilde{y}}_k\leftarrow {\left[\begin{array}{cc}{y}_k& {y_{nn}}_k\end{array}\right]}^T$    ${\widehat{x}}_k\leftarrow {\widehat{x}}_k^{-}+K_k\left({\tilde{y}}_k-H{\widehat{x}}_k^{-}\right)$    $P_k\leftarrow \left(I-K_{k}H\right)P_k^{-}$  end for  return ${\widehat{x}}_N,P_N$ end function

As $\tau \in (0,1]$, therefore $(1-{\tau }_k)/{\tau }_k\in [0,+\infty )$. In case when the new input data of the neural network is far from the original dataset, $\tau$ approaches zero. $(1-{\tau }_k)/{\tau }_k$ approaches $+\infty$. The posterior state ${\widehat{x}}_k$ relies more on the matching degree between the physical model and the actual driving condition. As the similarity grows, the value $\tau$ increases gradually. If $\tau$ equals one, the observation covariance part based on the neural network equals zero. The final estimation result is completely based on a neural network.

3. Data Acquisition

For a data-driven approach such as the transformer, the amount and richness of the data may directly affect how the network functions. The dataset must include an adequate amount of data samples and should resemble actual ones. The neural network is tested using simulation data produced by the high-fidelity Truck-Sim model.

By configuring the settings for the truck model and the testing environment in TruckSim, gather sensor data in the virtual environment, and assess the accuracy of the parameter estimate technique. The main parameters of simulation vehilc e is shown in

Table 1. Main parameters of simulation vehicle.

Parameter	Value
vehicle curb weight	6042 kg
vehicle payload	4000 to 40,000 per 4000 kg
Frontal area	6.8 ${\mathrm{m}}^2$
efficiency	0.99
sample time	0.01 s

. The simulated vehicle’s curb weight is 6042 kg. A range of payloads is selected, ranging from 4000 to 40,000 kg every 4000 kg.

Different slope conditions are used to generate enough simulation data. Five kinds of slope conditions are shown in Figure 5a, and the highway fuel economy driving schedule (HWFET) speed profile condition shown in Figure 5b are adopted. Each payload value and slope condition is combined to collect enough information. Each simulation is 100 s long.

In order to verify the accuracy and stability of the proposed method, two simulation tests are conducted. The gross vehicle weight is set to be 28,542 kg. The other parameters remain the same as the vehicle used for simulation data collection. Two kinds of slope conditions are used for test data generation, shown in Figure 6.

4. Results

The proportion of the training set, test set, and validation set are 6:2:2. The training of the neural network is implemented in Pytorch and Tsai [40]. The experiments were conducted on Ubuntu 20.04 operating system computer, which is equipped with Intel(R) Core(TM) i5-10600 CPU @ 3.30 GHz CPU, 64 G RAM and 12GB GeForce RTX 3080.

4.1. Evaluation Metrics

The root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) are used as evaluation metrics to assess the accuracy of mass estimate results:

$$\begin{array}{cc}\hfill \mathrm{RMSE}& =\sqrt{\frac{1}{n}\sum _{i=1}^n{\left(\widehat{m_i}-m\right)}^2}\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill \mathrm{MAE}& =\frac{1}{n}\sum _{i=1}^n\left|{\widehat{m}}_i-m\right|\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill \mathrm{MAPE}& =\frac{100\%}{n}\sum _{i=1}^n\left|\frac{{\widehat{m}}_i-m}{m}\right|\hfill \end{array}$$

(39)

where $\widehat{m_i}$ is the estimated value of mass, m is the ground truth of mass, n is the total number of estimated mass values.

In order to evaluate the stability of estimation results, here the convergence rate (CR) is adopted to evaluate the stability of estimation [41]. The estimation result is deemed to be convergent when the following requirements are satisfied:

$$\begin{array}{c}\hfill c_v\left(\widehat{m}\right)=\frac{\sigma \left(\widehat{m}\right)}{\mu \left(\widehat{m}\right)}<c_{v0}\end{array}$$

(40)

$$\begin{array}{c}\hfill \frac{\left|{\widehat{m}}_i-m\right|}{m}<2\%\end{array}$$

(41)

where $c_v$ is the coefficient of variation, defined as the standard variance of estimation $\sigma \left(\widehat{m}\right)$ divided by the mean of estimation $\mu \left(\widehat{m}\right)$, and $c_{v0}$ is the threshold of $c_v$ and set to be 0.02. CR is defined as the calculation steps cost when $c_v$ converges to lower than $c_{v0}$.

4.2. Performance Evaluation

The estimation results of each test are shown in Figure 7 and Figure 8. The blue dashed line represents the result of ground truth in the middle subgraph and $c_{v0}=2\%$ in the bottom subgraph, respectively.

In Figure 7a, the weight level of test 1 changes due to the difference between the training set and the test set. Shown in Figure 7b, the estimation results of the neural network estimator vary and fluctuate occasionally. The transformer shows higher oscillations, followed by LSTM and MLP. However, these oscillations are not allowed in the control system as they would break the stability of the system. On the other hand, the estimation result of EKF is more stable. To start with, the EKF result undershoots to some high value but then begins to converge to ground truth gradually, showing more stability. The estimation results of the proposed method neither overshoot nor oscillates. Figure 7c shows the coefficient of variation and demonstrates the stability of each method. $c_v$ of neural network methods gradually rises and is higher than $2\%$ in the end, while the result of EKF overshoots and converges to lower than $2\%$. The result of the proposed method rises like neural network estimators, but will finally converge to a stable value lower than $2\%$.

In Figure 8a, the weight level of Test 2 changes more frequently, and the results of neural network estimators will also oscillate, which also shows the instability of neural network results, and might do harm to control systems. The changing trend of results also shows the advantages of the proposed method.

The metrics listed in

Table 2. Evaluation metrics of simulation test 1 results.

Method	MAPE (%)	MAE (kg)	RMSE (kg)	CR (steps)
EKF	5.85	1669.77	1719.58	3790
MLP	2.28	652.05	1117.12	nan
LSTM	1.80	514.51	1124.30	nan
Transformer	1.42	404.91	972.35	nan
TA-AEKF (Proposed)	0.89	255.41	356.42	156

,

Table 3. Evaluation metrics of simulation test 2 results.

Method	MAPE (%)	MAE (kg)	RMSE (kg)	CR (steps)
EKF	5.98	1706.40	1745.78	3338
MLP	3.57	1018.14	2031.73	nan
LSTM	2.95	843.07	2018.19	nan
Transformer	2.15	613.11	1068.24	nan
TA-AEKF (Proposed)	0.90	257.53	357.61	212

and

Table 4. Evaluation metrics of simulation test 1 & 2 average results.

Method	MAPE (%)	MAE (kg)	RMSE (kg)	CR (steps)
EKF	5.91	1688.08	1732.68	3564
MLP	2.93	835.09	1574.42	nan
LSTM	2.38	678.79	1571.25	nan
Transformer	1.78	509.01	1020.29	nan
TA-AEKF (Proposed)	0.90	256.47	357.01	184

demonstrate the superiority of the proposed method in terms of accuracy, with lower values in MAPE, RMSE, and MAE. RMSE and MAE are used to measure the deviation between the ground truth value and the predicted value, whereas MAPE measures the relative size of the deviation. RMSE and MAE have the same dimensions as the quantity, while MAPE is dimensionless. RMSE is more sensitive to outliers and can magnify prediction errors due to the square operation. In contrast, MAE and MAPE are more robust to extreme values.

In test 1, the MAE of LSTM is smaller than that of MLP, but LSTM has a higher value of RMSE, indicating that there are higher extreme values in LSTM’s estimation. As for test 2, EKF has the highest value of MAE, indicating the maximum number of estimation errors. However, LSTM and MLP have higher values of RMSE, indicating higher deviations in the results of neural network estimators. MAPE is easier to understand, even when the ground truth values are different, whereas MAE and RMSE need to combine the real value to judge the difference. These discrepancies are also apparent from Figure 7 and Figure 8.

In terms of the time needed to converge to the threshold of efficiency variation, the proposed method is faster, demonstrating its stability advantage. Furthermore, among the three neural network estimators, the transformer neural network demonstrates superior performance. However, it should be noted that all neural network estimators exhibit instability, as they are unable to converge below the threshold of 2%. These further prove the necessity of integrating EKF with the transformer.

4.3. Input Feature Demonstration

To prove that the input feature is selected for the neural network, here the permutation method is used. In addition to the original features, lateral acceleration $a_y$ and yaw rate $\omega$ are also fed into the neural network. The MAPE increase when a feature is removed is calculated, shown in Figure 9. The removal of features included in $A_t$ would result in higher MAPE than $a_y$ and $\omega$. Therefore, just selecting the input features in $A_t$ is enough to recharacterize the nonlinear dynamics.

4.4. Confidence Level Demonstration

4.4.1. Calculation Speed Demonstration

To better demonstrate the calculation speed of the confidence level module, the computing time of each method is calculated as shown in

Table 5. Evaluation computing time of weight adjustment.

Method	Computing Time (s)
	Test1	Test2	Average of Test1 & Test2
Method 1 ([28])	280.21	293.68	286.94
Method 2 (Proposed)	1.00	0.99	1.00

. Results prove the outperformance of the proposed method in terms of calculation speed of confidence level.

4.4.2. Improtance Demonstration

To better demonstrate the importance of the weight adjustment module, here different values are chosen to balance the relationship between the transformer and AEKF. The weight factor is set to be ${10}^{-15}$, 1 and the adaptive weight ${\tau }_x$, respectively. Furthermore, the results are shown in Figure 10 and

Table 6. Evaluation metrics of different weight factors.

Method	MAPE (%)	MAE (kg)	RMSE (kg)	CR (steps)
$\tau =1e-15$	8.06	2301.15	2305.04	154
$\tau =1$	1.42	404.91	972.35	nan
$\tau ={\tau }_k$	0.89	255.41	356.42	156

. When $\tau$ equals one, $(1-{\tau }_k)/{\tau }_k$ equals zero. In this case, the result of TA-AEKF is the same as the estimation of the transformer. ${\displaystyle \underset{x\to 0}{lim}}$ is represented by setting $\tau ={10}^{-15}$. Furthermore, the result would be more reliant on the dynamic model. TA-AEKF estimator results when $\tau ={\tau }_k$ has the lowest MAPE, MAE, and RMSE. As for convergence steps, the values of $\tau ={10}^{-15}$ and $\tau ={\tau }_k$ are very close. Therefore, the adaptive $\tau ={\tau }_x$ does work.

4.5. Real Time Test

To better evaluate the real-time performance of the proposed method, the corresponding computing time is calculated. For neural network estimators, the model can be trained offline and loaded before estimation. As shown in

Table 7. Evaluation computing time of simulation test.

Method	Computing Time (s)
	Test1	Test2	Average of Test1 & Test2
EKF	0.53	0.57	0.55
MLP	0.17	0.21	0.19
LSTM	0.17	0.18	0.17
Transformer	0.44	0.46	0.45
TA-AEKF (Proposed)	1.07	1.09	1.08

, LSTM costs the lowest time and TA-AEKF costs the highest. Besides, for a certain driving process, the total mass remains constant. Therefore, the calculation of mass can be done just once by collecting enough data. In this case, the computing time of the proposed method meets the need for the requested real-time performance.

5. Conclusions

In order to further improve the accuracy and robustness of autonomous vehicle mass estimation, the transformer-aided adaptive extended Kalman filter is proposed by integrating the transformer pre-estimation into an extended Kalman filter using a weight adjustment module. Simulation tests are conducted to validate the appropriateness of neural network feature selection, the adaptive law of the proposed weight adjustment module and the accuracy and stability of the proposed method.

In future work, we extend to collect more real vehicle data under more complex working conditions for performance verification. Furthermore, the proposed method will be extended to the estimation of other parameters in autonomous vehicles and different fields.