Remaining Useful Life Prediction of Rolling Bearing Based on Multi-Domain Mixed Features and Temporal Convolutional Networks

Abstract

For the remaining useful life (RUL) prediction of rolling bearing under strong background noise, it is hard to get accurate results based on the non-stationary vibration signals because of complex degradation characteristics and difficult extraction of key features. The framework of RUL prediction for rolling bearing is established by integrating multi-domain mixed features and temporal convolutional network (TCN). The variational mode decomposition method based on the dung beetle optimization algorithm is developed to reduce signal noise by determining the optimal parameters adaptively. To construct a health indicator of rolling bearing effectively, an isometric feature mapping algorithm is introduced to reduce the dimensionality of multi-domain mixed features, integrating time-domain, frequency-domain, and entropy features of vibration signals under non-stationary and nonlinear conditions. By considering the advantages of a multi-head attention mechanism (MA) and bidirectional gated recurrent unit (BiGRU), a TCN-based multi-head attention and bidirectional gate (TCNMABG) is developed to predict the RUL of rolling bearing accurately, whose detailed implementation process of TCNMABG is described based on XJTU-SY dataset. To verify the performance of TCNMABG, the FEMTO-ST dataset is introduced to perform the numerical experiments, and the results show that prediction error is reduced by 65.96% on average.

1. Introduction

Rotating machinery is one of the most widely used equipment in the industrial field [1]. As an important part of rotating machinery equipment, rolling bearings play a vital role in ensuring the normal operation of equipment. However, due to the harsh operating environment of the equipment and the highly variable workload, rolling bearing faces inevitable risks of wear and failure [2]. The prediction of equipment’s remaining useful life is one of the key technologies in fault prediction and health management (PHM) [3]. If the rolling bearing degrades to the required threshold, the machine will fail. How to accurately determine the RUL of rolling bearing based on monitoring data is crucial for developing a reasonable maintenance plan and reducing downtime and cost losses [4]. For RUL prediction of rolling bearing under strong background noise, due to the complex degradation characteristics of vibration signals and the difficulty in extracting key features, it is difficult to obtain accurate prediction results based on non-stationary vibration signals. Therefore, it is necessary to explore RUL prediction methods based on multi-domain features, which have important practical significance in equipment maintenance management.

Currently, the remaining useful life (RUL) prediction methods for rolling bearing equipment can be mainly divided into two types: traditional methods based on physical models and data-driven methods [5]. The remaining useful life prediction method based on physical models [6] relies on degradation mechanisms, expert rules, and empirical knowledge, resulting in limited applicability of the model. In recent years, with the advancement of sensor technology and the development of deep learning [7], data-driven remaining useful life prediction models have received widespread attention [8]. This type of model learns degradation features and trends from data for prediction, reducing the need for prior knowledge and complex physical models. It can adapt to changes in different systems and environments, and has strong adaptability and scalability. Data-driven methods can be further divided into shallow machine learning-based methods and deep learning-based methods. Methods based on shallow machine learning include statistical regression analysis [9], support vector machines [10,11], neural networks [12,13,14], and so on. However, the hierarchical structure of shallow machine learning-based methods is relatively simple, which limits the model’s ability to extract deep-level structures and abstract features from the data. Life prediction methods based on deep learning typically perform better in feature extraction on large-scale data and can automatically learn feature representations that are suitable for tasks. The model-building process of this method generally requires three major steps: signal denoising, feature extraction and health indicator construction, and prediction model.

Signal noise reduction is an important prerequisite for predicting the remaining useful life prediction of rolling bearings. Yao et al. [15] proposed a denoising network model based on convolutional denoising autoencoder. The noise component was removed from the original data by stacking convolutional autoencoders, and the RUL of rolling bearings was predicted by the bidirectional long short-term memory network model. Li et al. [16] use a wavelet packet algorithm to denoise the direct current component of the gear pump pressure signal and then extract state evaluation indexes to predict the remaining useful life of the gear pump. Ren et al. [17] proposed a joint denoising algorithm based on adaptive white noise complete set empirical mode decomposition and improved adaptive wavelet threshold to address the issue of wavelet threshold denoising algorithms not being able to adaptively select decomposition levels and wavelet bases. The processed intrinsic mode functions (IMF) components were reconstructed to better extract signal features from noisy signals. Zhao et al. [18] proposed an improved stacked denoising autoencoder method, which extracts features from noisy signals through the encoder and reconstructs the signal using the decoder to achieve effective signal denoising. At present, the above method has the advantages of automatic feature extraction and improved prediction accuracy in signal noise reduction processing, but there are also challenges, such as high algorithm complexity and great demand for parameter adjustment, and it is difficult to determine the method parameters adaptively, resulting in increased sensitivity of the model to noise, and the effect of the remaining useful life prediction model needs to be further improved.

Feature extraction and health index construction are the key steps of remaining useful life prediction of rolling bearing. Zhou et al. [19] proposed a depth feature extraction method based on multi-dimensional self-attention temporal convolutional networks and designed a pattern-weighted feature fusion method to obtain degradation indicators. She et al. [20] proposed a health index construction method based on the canonical resolution autoencoder model and predicted the remaining useful life of rolling bearings through the RUL prediction model based on a particle filter. Zhao et al. [21] proposed a data-driven feature extraction method, namely the fitting curve derivative method of maximum power spectrum density. This method extracts the performance degradation features of rolling bearings throughout the life cycle from historical data, thereby establishing a RUL prediction model. Peng et al. [22] proposed a multi-sensor health indicator (HI) construction method based on reinforcement learning, which can realize automatic learning and find the best sensor combination rules, thereby improving the RUL prediction performance of the model. Li et al. [23] proposed a method for constructing composite health indicators by weighted fusion of multi-source sensors to characterize the evolution trend of equipment degradation and achieve remaining useful life prediction. Zhang et al. [24] used the self-organizing mapping method to extract features based on similar sample sets and proposed a health indicator construction method based on the minimum feature circle. The existing methods have achieved certain results in feature extraction and health indicator construction, with high efficiency and accuracy, but there are still some shortcomings, such as the single-domain feature is difficult to fully express the nonlinear degradation law of equipment in feature extraction, and the interaction between features is not fully considered in feature selection. Traditional feature dimensionality reduction methods based on linear models [25,26] cannot adequately capture complex dynamic characteristics in nonlinear and non-stationary vibration signals, which may lead to information loss and other problems during feature dimensionality reduction.

The construction of the remaining useful life prediction model has an important impact on the prediction effect. Cao et al. [27] proposed a remaining useful life prediction method combining a self-attention mechanism with a long short-term memory neural network to solve the problem that the correlation between components was not fully considered in the RUL prediction of mechanical equipment. Zhang et al. [28] proposed a remaining useful life prediction model based on a transformer, which can simultaneously extract features of different sensors and time steps in parallel and finally verify the model performance using turbofan engine data sets. Lin et al. [29] proposed an attention-based gated recurrent unit neural network model to effectively use feature information to predict the remaining useful life of equipment. Liu et al. [30] proposed an enhanced encoder-decoder framework, which inputs the time series feature data into the encoder-decoder network model based on LSTM and calculates the RUL value at the end of the acquired signal combined with the linear regression algorithm of the output layer. Cao et al. [31] combined kernel principal component analysis and long short-term memory network method to predict the remaining useful life of rotating machinery in view of the difficulty in extracting degraded information caused by redundant data from multiple sensors. The above RUL prediction model has been applied in the field of intelligent fault diagnosis and remains useful for life prediction to a certain extent. However, traditional remaining life prediction models, such as the convolutional neural network model (CNN), recurrent neural network model (RNN) and, long short-term memory network model (LSTM), Transformer model are difficult to capture long-term temporal dependence effectively and have insufficient feature extraction ability for signal key information. The accuracy of model prediction still needs to be further improved.

In order to overcome the limitations of the above-mentioned method for predicting the remaining useful life of rolling bearings, the main contributions of this paper are as follows: (1) Dung Beetle algorithm optimized VMD combined with correlation coefficient method is proposed to reduce the noise of the original signal. It realizes automatic optimization of VMD initial parameters, reduces manual intervention, and captures degradation information in noisy signals more accurately. (2) A feature dimension reduction method based on a multi-domain mixed feature and isometric feature mapping (ISOMAP) algorithm is proposed. To capture the characteristics of vibration signals in different fields, obtain comprehensive information on signals, and combine the ISOMAP algorithm to better solve the problem of nonlinear vibration signal feature dimensionality reduction. (3) A remaining useful life prediction model of TCNMABG is proposed. Enhance the feature extraction ability of the model in the process of rotary machinery equipment degradation and improve the prediction accuracy of the remaining useful life prediction model.

The structure of this article is as follows. Section 2 introduces the general framework of the remaining useful life prediction method for TCNMABG rolling bearing. Section 3 introduces the implementation process of the proposed model in detail based on the XJTU-SY dataset, and the results of the comparison experiment and ablation experiment of the model are analyzed and discussed. Section 4, the FEMTO-ST bearing dataset, is used to verify the performance of the model further. Finally, the whole research content is summarized, and the prospect is put forward.

2. Framework of RUL Prediction Based on TCNMABG

2.1. Overall Framework

The overall framework of RUL prediction based on TCNMABG for rolling bearing is shown in Figure 1, which includes data acquisition and preprocessing, feature extraction and selection, and RUL prediction. Firstly, reduce the noise of the vibration signals of rolling bearing based on the dung beetle algorithm optimization-based variational mode decomposition (DBOVMD). Secondly, extract the multi-domain mixed features by integrating the features of the time domain, frequency domain, and entropy. Then, the isometric feature mapping algorithm reduces dimensionality and compresses the extracted features to get the health indicator curve effectively, and the kernel-based fuzzy C-mean clustering method is introduced to determine the first prediction time (FPT) of the rolling bearing. Finally, construct the TCNMABG prediction model, and train the model with the goal of minimizing the loss function to obtain the RUL prediction results for rolling bearing.

2.2. Signal Acquisition and Data Preprocessing

2.2.1. Vibration Signal Acquisition of Rolling Bearing

The vibration signals of rolling bearings are important to determine the degradation characteristics, including wear, cracks, and so on. By monitoring and collecting the vibration signal of the rolling bearing, it can effectively evaluate the operating state of the equipment and predict the remaining useful life. However, it is difficult to accurately extract the effective features because of the influence of strong background noise and other factors in the signal acquisition process. Therefore, it is essential to use the denoise method to reduce the noise in the collected vibration signals.

2.2.2. Signal Noise Reduction Method Based on DBOVMD

When using the variational mode decomposition (VMD) method for signal noise reduction, the number of modal decomposition k and the penalty factor α are crucial to the performance and noise reduction effect of the algorithm. Choosing an appropriate k value can retain effective information when decomposing the signal structure, but a high k value may introduce unnecessary details or noise. The penalty factor α regulates the sparsity and smoothness of the modal function. Too large a value may lead to excessive smoothness, while too small a value may make the decomposition too sparse. Therefore, the correct selection of k and α is crucial for balancing the retention of effective signals and the suppression of noise. By adjusting these two parameters, optimizing the VMD algorithm to adapt to different signal characteristics and noise reduction requirements is a key step in improving the noise reduction effect and algorithm performance.

In this paper, a signal denoising algorithm based on Dung beetle optimization VMD (DBOVMD) is used to find the best parameter combination k and α to reduce the noise of the original data. This method solves the complex optimization problem by simulating the path-planning process of dung beetles when they search for food, with fast convergence speed and high precision.

The fitness function in the DBOVMD algorithm has a significant impact on the parameter optimization results. Envelope entropy can be used to analyze the time-varying characteristics, dynamic behavior, and nonlinear characteristics of time series signals, effectively representing the sparse characteristics of the original signal, especially suitable for non-stationary signals. Therefore, envelope entropy is chosen as the fitness function for the DBOVMD algorithm. When there is more noise and less feature information in the IMF components, the envelope entropy value is larger; otherwise, the envelope entropy value is smaller. The detailed DBOVMD process is summarized as follows, which is shown in Figure 2.

(1) Initialize the key parameters of the DBO algorithm [32], including dung beetle population size, iteration number, and other parameters k and α.

(2) Perform VMD based on the determined parameter combination.

(3) Divide the population randomly into four types, including ball-rolling dung beetles, breeding dung beetles, foraging dung beetles, and stealing dung beetles.

(4) Calculate the fitness values of all the dung beetle positions by envelope entropy, which is shown in Equation (1).

$$\left\{\begin{array}{c}{E}_p=-{\displaystyle \sum _{j=1}^N{p}_j\lg p_j}\\ p_j=a(j)/{\displaystyle \sum _{j=1}^{N}a(j)}\end{array}\right.$$

(1)

where $a(j)$ is the envelope signals obtained by Hilbert demodulation of k modal components decomposed by VMD. $p_j$ is the probability distribution sequence obtained by normalizing $a(j)$. N is the number of sampling points, and the envelope entropy $E_p$ can be computed by evaluating the entropy of the probability distribution sequence $p_j$.

(5) Update the positions of ball-rolling dung beetles, breeding dung beetles, foraging dung beetles, and stealing dung beetles by different strategies.

(6) Calculate the fitness values of the updated population.

(7) Update and save the optimal solution as the current population with the best fitness value when it is better than the previous optimal solution.

(8) If the current iteration exceeds the maximum number of iterations, output the optimal parameter combination; otherwise, return to Step 2.

According to the optimal parameter combination of k and α, the vibration signal is decomposed into modes of different frequencies, and each internal modal component IMF can be evaluated based on the correlation coefficient between the IMF and the raw signals. The effective modes whose correlation coefficients are higher than the threshold should be selected for superposition reconstruction. Therefore, the optimal parameter combination can be obtained by DBOVMD, which can retain the key signal features and remove the noise flexibly.

2.3. Multi-Domain Mixed Feature Extraction and Selection

2.3.1. Multi-Domain Mixed Feature Extraction

The vibration signals can reflect the real-time status information and degradation trend of the equipment. To fully characterize the time change characteristics, frequency components, energy distribution, and other information of the signal. In this paper, dimensional time domain features, dimensionless time domain features, frequency domain features, entropy features, and spectral kurtosis features of signals are extracted respectively to form a multi-domain mixed feature set, and the equipment degradation process is described from multiple dimensions to predict the remaining useful life of rolling bearing equipment.

The dimensional time domain feature is sensitive to the signal characteristics of the equipment running state, but it is also susceptible to changes in load, speed, and other working conditions. The dimensionless time domain features are less affected by environmental disturbance but less sensitive to fault information. Therefore, the comprehensive use of dimensionless and dimensionless time domain features for feature extraction can give full play to their respective advantages so as to monitor the operating state of the equipment more effectively and obtain the degradation information of the equipment. The 17 time-domain feature calculation formulas extracted in this paper are shown in

Table 1. Calculation formula of time domain feature.

Features	Meaning	Calculation Formula	Features	Meaning	Calculation Formula
F1	Max	$x_{\max }=\max \left\{\left|x_i\right|\right\}$	F2	Min	$x_{\min }=\min \left\{\left|x_i\right|\right\}$
F3	Mean	$\overline{x}=\frac{1}{N}{\displaystyle \sum _{i=1}^N{x_i}^2}$	F4	Median	$x_{med}=X\left(\frac{N+1}{2}\right)$
F5	Peak-to-peak value	$x_{pp}=x_{\max }-x_{\min }$	F6	Average rectified value	$x_{arv}=\frac{1}{N}{\displaystyle \sum _{i=1}^N\left|x_i\right|}$
F7	Variance	$x_{\mathrm{var}}=\frac{1}{n-1}{\displaystyle \sum _{i=1}^N{(x_i-\overline{x})}^2}$	F8	Standarddeviation	$\sigma =\sqrt{\frac{1}{N-1}{\displaystyle \sum _{i=1}^N{(x_i-\overline{x})}^2}}$
F9	Kurtosis	$x_{kurt}=\frac{\frac{1}{N}{\displaystyle \sum _{i=1}^N(x_i-\overline{x}}{)}^4}{{\left[\frac{1}{N}{\displaystyle \sum _{i=1}^N(x_i-\overline{x}}{)}^2\right]}^2}$	F10	Skewness	$x_{\mathrm{skew}}=\frac{\frac{1}{N}{\displaystyle \sum _{i=1}^N(x_i-\overline{x}}{)}^3}{{\left[\frac{1}{N}{\displaystyle \sum _{i=1}^N(x_i-\overline{x}}{)}^2\right]}^3}$
F11	Root mean square	$x_{rms}=\sqrt{\frac{1}{N}{\displaystyle \sum _{i=1}^N{x}_i^2}}$	F12	Mean square value	$x_{ms}=\frac{1}{N}{\displaystyle \sum _{i=1}^N{x}_i^2}$
F13	RMS amplitude	$x_{rmsa}={\left(\frac{1}{N}{\displaystyle \sum _{i=1}^N\sqrt{\left|x_i\right|}}\right)}^2$	F14	Waveform factor	$S_f=\frac{x_{rms}}{\frac{1}{N}{\displaystyle \sum _{i=1}^N\left|x_i\right|}}$
F15	Peak factor	$I_P=\frac{\max \left\{\left|x_i\right|\right\}}{\sqrt{\frac{1}{N}{\displaystyle \sum _{i=1}^N{x}_i^2}}}$	F16	Impulse factor	$C_f=\frac{\max \left\{\left|x_i\right|\right\}}{{\displaystyle \sum _{i=1}^N\left|x_i\right|}}$
F17	Clearance factor	$C_{\mathrm{e}}=\frac{\max \left\{\left|x_i\right|\right\}}{{\left(\frac{1}{N}{\displaystyle \sum _{i=1}^N\sqrt{\left|x_i\right|}}\right)}^2}$

Note: In the table, $x_i$ is the time series value of the signal, $i=1,2,\cdots ,N$; $N$ is the number of sample points.

.

Frequency domain features can reflect the distribution of different frequency components in the signal, the degree of energy concentration, and the change of frequency. The calculation formulas of 5 frequency domain features extracted in this paper are shown in

Table 2. Calculation formula of frequency domain feature.

Features	Meaning	Calculation Formula	Features	Meaning	Calculation Formula
F18	Centroid Frequency	$CF=\frac{{\displaystyle \sum _{k=1}^K{f}_k\cdot s(k)}}{{\displaystyle \sum _{k=1}^{K}s(k)}}$	F19	Frequency Variance	$VF=\frac{{\displaystyle \sum _{k=1}^K{(f_k-FC)}^2\cdot s(k)}}{{\displaystyle \sum _{k=1}^{K}s(k)}}$
F20	Mean Square Frequency	$MSF=\frac{{\displaystyle \sum _{k=1}^K{f}_k^2\cdot s(k)}}{{\displaystyle \sum _{k=1}^{K}s(k)}}$	F21	Root Frequency Variance	$RVF=\sqrt{\frac{{\displaystyle \sum _{k=1}^K{(f_k-FC)}^2\cdot s(k)}}{{\displaystyle \sum _{k=1}^{K}s(k)}}}$
F22	RMS Frequency	$RMSF=\sqrt{\frac{{\displaystyle \sum _{k=1}^K{f}_k^2\cdot s(k)}}{{\displaystyle \sum _{k=1}^{K}s(k)}}}$

Note: In the table, $s(k)$ is the spectrum of signal $x_i$, $k=1,2,\cdots ,K$; $k$ is the number of spectral lines; $f_k$ is the frequency value of the kth spectral line.

.

The entropy feature can effectively measure the uniformity of the signal and the complexity of the probability distribution and can quantify the confusion and uncertainty of the signal, which helps understand the information and randomness contained in the signal. In this paper, three entropy feature indicators (F23, F24, F25) are extracted, including power spectrum entropy, singular spectrum entropy, and energy entropy. The spectral kurtosis feature can effectively identify the transient shock and its distribution in the frequency band from the signal containing background noise and can deal with the non-stationary signal well. This article extracts four features related to spectral kurtosis (F26, F27, F28, F29), including mean value, standard deviation, skewness, and kurtosis of spectral kurtosis.

Therefore, a total of 29 feature indexes are extracted from the vibration signal after noise reduction to form a multi-domain mixed feature set.

To better select appropriate and effective degradation features from the constructed multi-domain mixed feature set, a comprehensive evaluation index is developed by the weighted sum of correlation and monotonicity, which can better consider the correlation of time-series features and monotonicity in the degradation process. The comprehensive evaluation index can be evaluated by Equation (2).

$$Cri={\omega }_{1}Corr+{\omega }_{2}Mon$$

(2)

where $Corr$ is the correlation; $Mon$ is the monotonicity [33]; ${\omega }_1$ and ${\omega }_2$ are the weights of the two evaluation indexes, respectively. To better evaluate the comprehensive score, priority is given to considering the impact of correlation on equipment performance evaluation within the comprehensive evaluation index, setting ${\omega }_1$ to 0.7 and ${\omega }_2$ to 0.3.

2.3.2. ISOMAP Feature Dimension Reduction

The extracted multi-domain mixed features represent the degraded features of the equipment operating state from multiple dimensions, which makes the fault feature set with high-dimensional, non-linear, redundant, and other characteristics, and even leads to the occurrence of “dimensional disaster”, which affects the results of the prediction model. Manifold learning [34] This kind of nonlinear dimensionality reduction method [35] can effectively discover low-dimensional manifold components embedded in high-dimensional space, complete dimensionality reduction or data visualization, and is suitable for the dimensionality reduction processing of high-dimensional fault features of rolling bearing equipment.

The Isometric Feature Mapping Algorithm (ISOMAP) utilizes the geodesic distance matrix between the sample data points instead of the Euclidean distance matrix in the multidimensional scale analysis (MDS) algorithm to obtain the low-dimensional manifold components that keep the geodesic distances between the samples unchanged. Compared with the classical linear dimension reduction methods, the ISOMAP can better mine the nonlinear manifold components hidden in the high-dimensional data, which can deal with the data with complex structures efficiently. Therefore, this paper uses the ISOMAP algorithm to compress the multi-domain mixed features for determining the sensitive features in the degradation process. The implementation process of the ISOMAP algorithm is summarized as follows.

(1) Construct a neighbor graph by the k nearest neighbors based on the distance or similarity.

(2) Calculate the geodesic distance between nodes based on the shortest path algorithm. The distance takes into account not only the distance of direct neighbors but also the distance through other node paths.

(3) Map the high-dimensional data to a low-dimensional space to keep the geodetic distance of the original data as similar as possible.

2.3.3. Identifying the First Prediction Time

The first prediction time (FPT) is an important index to evaluate the rolling bearing degrading from the normal state to a degenerate state, which is important for RUL prediction. To better illustrate the degradation trend of rolling bearings, a segmented function is used to characterize the degradation process of bearings, which is shown in Equation (3).

$$f(t_i)=\left\{\begin{array}{cc}1& t_i\le t_j\\ \left(\frac{1}{t_j-t_n}\right)\cdot t_i+\left(\frac{t_n}{t_n-t_j}\right)& t_i\ge t_j\end{array}\right.$$

(3)

where $t_j$ is the initial degradation time, the health status of the device remains unchanged until the $t_j$ moment, and then linearly declines to the $t_n$ moment when the device completely fails.

Traditional methods identify FPT by using artificially set thresholds, such as $3\sigma$ criteria for constructed health indicators, which introduce subjective factors and affect the reliability of the results. Moreover, the method of using quadratic differentiation to determine FPT for health indicators has problems, such as difficulty in observing and determining the results. Therefore, this article adopts the Kernel Fuzzy C-means (KFCM) clustering method [36] to identify FTP. KFCM considers the fuzzy affiliation between the data points and the clustering centers and the kernel similarity between the data points, which can more accurately curve the complex structure and intrinsic correlation of the data. The detailed process of KFCM is summarized as follows.

The KFCM clustering algorithm is an improved fuzzy C-means algorithm, which maps data to high-dimensional space by introducing a kernel function to deal better with nonlinear data. The process includes initializing the cluster center and membership matrix, iteratively updating the cluster center and membership matrix, using the kernel function to perform spatial transformation, and finally outputting cluster results with fuzzy membership.

2.4. Remaining Useful Life Prediction Based on TCNMABG

TCN is a neural network structure based on dilated causal convolution, which combines the local receptive field and parameter-sharing advantages of CNN with the long-term dependency modeling ability of RNN to overcome the limitations of traditional time series prediction models. The basic residual module in TCN consists of two convolution layers for nonlinear mapping. In this paper, we use hyperparameters k = 3 and d = 2 for dilated causal convolution operations, and each layer also adds WeightNorm and Dropout to regularize the network.

TCNMABG is constructed based on TCN by combining MA and BiGRU, and the structure of TCNMABG is shown in Figure 3. TCNMABG mainly consists of a TCN module, multi-head attention module, bidirectional gated recurrent unit module, and remaining useful life prediction module. The detailed RUL prediction of rolling bearing based on TCNMABG is summarized as follows.

Firstly, this model extracts the degradation features of device monitoring data through a temporal convolutional network composed of multiple basic residual modules. When constructing the TCN network, the standard convolution is injected with a void processing input time series by setting dilated causal convolution parameters to increase the receptive field and capture longer time dependencies. Introducing a residual connection between each convolutional layer helps solve the problem of disappearing gradients and speeds up the training process. The efficient processing and feature extraction of time series data is realized through dilated causal convolution and residual connection.

Secondly, after the TCN layer, a multi-head attention mechanism is introduced that can focus on different parts of the input sequence. Each head will calculate the attention weight independently to enhance the model’s ability to learn between different feature representations.

Thirdly, by introducing the BiGRU structure layer, the model enhances the ability to extract features from sequence data at different time scales by processing both forward and backward sequence information.

Finally, the remaining useful life prediction module converts the high-dimensional input data into one-dimensional vectors through the flatten layer of the neural network model and maps these features to the final RUL as an output result through the fully connected layer.

3. Implementation of RUL Prediction Based on TCNMABG

3.1. Dataset Description

In this paper, the XJTU-SY dataset is introduced in the experiments to verify the effectiveness of the proposed method [37,38]. The experimental platform of XJTU-SY is shown in Figure 4.

The platform is composed of an AC motor, motor speed controller, support shaft, tested bearing, hydraulic loading system, and so on. The accelerated degradation test of LDK UER204 rolling bearing was carried out under different speed and load conditions. By using two PCB 25C6 acceleration sensors mounted on the tested bearing, the XJTU-SY dataset records the whole-life-cycle vibration acceleration data of the rolling bearing from normal operation to failure in both horizontal and vertical directions, respectively. The sampling frequency is set to 25.6 kHz, the sampling interval is 1 min, each sampling duration is 1.28 s, and 32,768 data points can be recorded per sample.

In the accelerated degradation experiment of the XJTU-SY dataset, three different operating conditions were set, and five bearings were tested under each operating condition. In this paper, Bearing1_1, Bearing1_2, and Bearing1_3 in dataset operating condition 1 (2100 rpm, 12 kN) were selected as the training set data of the model, and bearing1_5 was selected as the test set data to verify the performance of the proposed method. Because Bearing1_4 is only a sudden fault and the degradation time is too short, it does not participate in the training.

3.2. Evaluation Indicators

To better verify the effectiveness of the proposed method, four evaluation indexes are used to assess the prediction results, which include the mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE), and the coefficient of determination R².

$$MSE=\frac{1}{N}{\displaystyle {\sum }_{\mathrm{i}=1}^N{(Y_i-P_i)}^2}$$

(4)

$$RMSE=\sqrt{\frac{{\displaystyle {\sum }_{\mathrm{i}=1}^N{(Y_i-P_i)}^2}}{N}}$$

(5)

$$MAE=\frac{{\displaystyle {\sum }_{i=1}^N|Y_i-P_i|}}{N}$$

(6)

$$R^2=1-\frac{{\displaystyle {\sum }_{i=1}^N{(Y_i-P_i)}^2}}{{\displaystyle {\sum }_{i=1}^N{({\overline{Y}}_i-Y_i)}^2}}$$

(7)

where $Y_i$ is the real value of remaining useful life, $P_i$ is the prediction value of the proposed method, and N is the total number of data samples.

The smaller the values of RMSE, MSE, and MAE, the higher the prediction accuracy of the proposed method. The coefficient of determination R² is used to measure the fitting of the prediction methods based on the observed data, whose value ranges from 0 to 1. The closer R² is to 1, the better the fitting of the prediction methods based on observed data.

3.3. Model Verification and Comparison

3.3.1. Denoising of Raw Signals

The DBOVMD algorithm is used to denoise the raw vibration signals based on the XJTU-SY dataset. The initialization settings of modal component k are [3, 10], and the penalty factor α is [100, 2500]. The fitness curve in the noise reduction process of the algorithm is shown in Figure 5.

When the fitness function reaches the minimum value, the optimal parameter combination can be determined by DBOVMD, and k is taken as 10, and α is taken as 2338. Then, by calculating the correlation coefficient between the intrinsic mode function and the raw signals, the standard deviation of the correlation coefficient is taken as the threshold value, and the IMF component whose correlation coefficient is larger than the threshold value is retained for signal reconstruction, and the signal after noise reduction is obtained.

3.3.2. Multi-Domain Mixed Feature Extraction

Due to the small number of original bearing data samples, it is difficult to meet the requirements of neural network model training, and generally, the raw signals are long; in order to reduce the calculation cost, the time window division method is used to expand the number of samples. In this paper, the length of the time window is selected as 4096 (32,768/8), and the spacing between Windows is also 4096; that is, the samples do not overlap, and the signal after noise reduction is segmented into multiple samples on the premise of not losing data information. This method is shown in Figure 6.

Taking bearing1_1 as an example, the sampling frequency is 25.6 kHz, 32,768 data points are obtained once per minute, the window length is 4096, and the total number of samples of bearing1_1 is 984 (123 × 32,768/4096). For each time step sample data, the 29-dimensional features in Section 2.3 are extracted from horizontal and vertical vibration signals, respectively, and a total of 58-dimensional multi-domain mixed features are obtained.

The obtained multi-domain mixed feature set is selected according to monotonicity, correlation, and comprehensive evaluation index. The evaluation index results are shown in Figure 7. Taking 0.9 times the maximum value of the comprehensive evaluation index as the threshold, feature visualization was performed on the top 20 features larger than the threshold, and the results are shown in Figure 8.

3.3.3. Feature Dimension Reduction and FPT Recognition

The feature selection results were further processed by the ISOMAP algorithm for feature dimensionality reduction, and the one-dimensional HI curve was obtained, as shown in Figure 9.

KFCM was used for clustering of HI index data, and the results are shown in Figure 10. The blue dot represents the normal stage of the device, and the red dot represents the degradation stage. The early fault point FPT of the device is obtained according to the critical moment position of the two clusters of data in the clustering results. The specific time position is shown in

Table 3. XJTU-SY dataset condition 1 bearing FPT recognition results.

Number	Total Experiment Time/(×7.5 s)	FPT/(×7.5 s)
Bearing1_1	984	628
Bearing1_2	1288	510
Bearing1_3	1264	875
Bearing1_5	416	318

.

3.3.4. Model Training

Before starting model training, it is necessary to determine the RUL labels of the training set and test set data. According to the segmented function definition in Section 2.3.3, add RUL labels to the feature data of each bearing. Due to the significant differences in the full life cycle of different bearings, this paper normalizes the remaining useful life of each bearing in the dataset. The health status is one, and the complete failure is zero. The RUL tag of Bearing1_1 is shown in Figure 11.

The parameters of the TCNMABG model are set as follows. The training batch size is 64, the initial learning rate of the model is 0.001, the number of training rounds epoch is 100, the optimizer is “Adam”, the number of BiGRU layers is 1, the dimensionality of the hidden layer is 64, the output dimension of each layer of the TCN module is [4], the dropout size is 0.1, the time step of the input data is 30, and the linear output layer is 1 neuron. The output dimension of each layer of the module is [4], the dropout size is 0.1, the time step of the input data is 30, and the linear output layer is 1 neuron.

The hardware environment configuration of all experiments is set as follows. Intel (Santa Clara, CA, USA) Core (TM) i9-13900 HX processor, 16 GB RAM, RTX4060 graphics card, Python version 3.8.16, torch version 1.13.1, CUDA version 11.7, MATLAB R2021b, and the Windows 11 operating system.

3.4. Experimental Results

The RUL prediction results of performing RNN, GRU, LSTM, BiLSTM, and TCNMABG are shown in Figure 12. Where (a) is the prediction result of the test set, and (b), (c), (d) is the prediction result of the training set.

In order to illustrate the effectiveness of TCNMABG for predicting the RUL of rolling bearing, the error indexes MSE, RMSE, MAE, and R² are introduced to evaluate the accuracy of the proposed method. By comparing with RNN, GRU, LSTM, and BiLSTM, the results of TCNMABG are obtained by performing all the methods five times, which is shown in

Table 4. Comparison experiments of RUL prediction for bearing1_5 in XJTU-SY dataset.

Prediction Model	MSE	RMSE	MAE	R2
RNN	0.011753	0.108411	0.065320	0.871729
GRU	0.028311	0.168260	0.134098	0.691013
LSTM	0.010972	0.104475	0.089342	0.880250
BiLSTM	0.007165	0.084649	0.058105	0.921797
TCNMABG	0.006753	0.082175	0.056021	0.926302

. It can be seen that the RUL prediction results of TCNMABG, the MSE, RMSE, MAE, and R² in the test set are 0.006753, 0.082175, 0.056021, and 0.926302, respectively. The results show that TCNMABG has an excellent performance in RUL prediction.

Comparing the prediction results in Figure 12, the RNN model has the largest error in prediction results; GRU, LSTM, and BiLSTM have improved the traditional recurrent neural network to some extent and have a better ability to describe time series information, but in this experiment, the prediction results still have large errors and fluctuations. The prediction results of the TCNMABG model can not only better characterize the degradation trend of the remaining useful life of the equipment but also have a good fit with the real RUL value, and the prediction of every bearing in the training set and test set is the most stable, without overfitting phenomenon. The prediction results in Table 4 show that compared with the other four mainstream methods, the MSE, RMSE, and MAE of the TCNMABG model on the test set of the XJTU-SY dataset decreased by 40.72%, 24.91%, and 28.34% on average, respectively, and the R2 increased by 11.51% on average.

3.5. Ablation Experiments

3.5.1. Design of Ablation Experiments

In order to verify the impact of the bidirectional gated recurrent neural network and the multi-head attention mechanism on the performance of the TCNMABG model, the ablation experiments are implemented based on the XJTU-SY dataset under condition 1. The ablation experiments are introduced based on the modules in TCNMABG, which is shown in

Table 5. Usage of different modules in ablation experiment.

Number	Experimental Scheme	BiGRU	MA	TCN
1	TCN	×	×	√
2	TCN-MA	×	√	√
3	TCN-BiGRU	√	×	√
4	TCNMABG	√	√	√

.

In order to verify the impact of bidirectional gated recurrent neural network and multi-head attention mechanism on the performance of the TCNMABG model, the ablation experiments are implemented based on the division of the training set and the test set of the XJTU-SY dataset in Section 3.1. According to the use of different modules, with TCN as the benchmark model, BiGRU and MA as the controllable modules, and four experimental schemes of TCN, TCN-MA, TCN-BiGRU, and TCNMABG models can be obtained, as shown in Table 5.

3.5.2. Results of Ablation Experiments

The RUL prediction results of Bearing1_5 based on four methods are shown in Figure 13. As can be seen from the figure, the RUL prediction curve obtained by the TCNMABG model is closer to the real RUL value than the other three ablation models, and the error curve is concentrated around 0 value, with smaller fluctuation amplitude compared with other models.

To better illustrate the errors between the proposed method and the other three methods, the results are listed in

Table 6. Comparison of prediction results of ablation experiment test set.

Prediction Model	MSE	RMSE	MAE	R2
TCN	0.035751	0.189080	0.137930	0.609812
TCN-MA	0.030283	0.174019	0.123644	0.669498
TCN-BiGRU	0.008670	0.093111	0.064587	0.905379
TCNMABG	0.006753	0.082175	0.056021	0.926302

. The results show that the prediction of the TCNMABG model on the test set is better than the other three ablation methods, with higher prediction accuracy and smaller prediction error.

As can be seen from Table 6, the MSE predicted by RUL based on TCNMABG in the test set is 81.11%, 77.70%, and 22.11% lower than the other three methods, respectively, compared with TCN, TCN-MA, and TCN-BIGRU. Compared to the TCN benchmark model, the MSE metrics of the TCN-MA and TCN-BiGRU models are reduced by 15.29% and 75.75% for the test set, respectively. The results of the ablation experiment results show that the multi-head attention module and the bidirectional gated recurrent neural network module can improve the RUL prediction effectiveness of TCN. Therefore, the added MA and BG are essential to improve the performance of TCN.

4. Validation Analysis of TCNMABG Based on FEMTO-ST Dataset

In order to further verify the generalization performance of TCNMABG, the bearing dataset released by the French FEMTO-ST Institute is used for experimental verification. The FEMTO-ST dataset [39] collects the full life cycle data of the test bearings through the PRONOSTIA experimental platform, which is capable of performing accelerated bearing degradation tests under different working conditions.

The platform is mainly composed of asynchronous motors, drive shafts, couplings, test bearings, pneumatic jack loading systems, and digital regulators, which are shown in Figure 14. The adjustable working condition parameters are the radial force applied to the test bearings and the rotational speed. The vibration signals were collected by two Dytran 3035 B micro-accelerometers at 90 degrees to each other with a sampling frequency of 25.6 kHz. The specific parameters of the test bearing are shown in

Table 7. Specific parameters of bearings tested on the FEMTO-ST dataset.

Parameter	Value	Parameter	Value
Outer ring raceway diameter/mm	29.1	Outer race diameter/mm	32
Inner ring raceway diameter/mm	22.1	Inner race diameter/mm	20
Static load rating/N	2470	Thickness/mm	7
Dynamic load rating/N	4000	Middle diameter/mm	25.6

.

In this paper, the bearings under the load of 4000 N and speed of 1800 rpm working conditions are selected to test the general performance of TCNMABG. Bearing1_1, Bearing1_2, and Bearing1_3 are set as training sets, and Bearing1_4 is set as a test set. According to the proposed framework of TCNMABG, the extraction of multi-domain mixed features can be obtained by the comprehensive index, and the extracted features of Bearing1_1 are shown in Figure 15.

The FPT of the early degradation point is determined by KFCM, and the results of the FPT of the four bearings in working condition one are shown in

Table 8. Bearing FPT Identification Results from the FEMTO-ST Dataset.

Number	Total Experiment Time/(×10 s)	FPT Position /(×10 s)
Bearing1_1	2803	1640
Bearing1_2	871	827
Bearing1_3	2375	1648
Bearing1_4	1428	1112

.

Based on the determined FPT, the segmented linear degradation labels are used to evaluate the RUL of bearings, which is shown in Figure 16. To better analyze the error indexes of TCNMABG, RNN, GRU, LSTM, and BiLSTM are used to analyze the RMSE, MSE, MAE, and R² based on FEMTO-ST dataset, and the results are listed in

Table 9. Evaluation Indicators of Bearing1_4 for RUL prediction based on FEMTO-ST.

Prediction Model	MSE	RMSE	MAE	R2
RNN	0.014471	0.120294	0.059549	0.765822
GRU	0.013153	0.114687	0.061900	0.787143
LSTM	0.012544	0.111998	0.053856	0.797008
BiLSTM	0.009873	0.099364	0.057263	0.840223
TCNMABG	0.004194	0.064760	0.037322	0.932131

.

According to the results of Bearing1_4 in Table 9, compared with RNN, GRU, LSTM, and BiLSTM, the mean square error of TCNMABG is reduced by 71.02%, 68.11%, 66.57%, and 57.52%, respectively, with an average reduction of 65.96%. The root mean square error of TCNMABG is reduced by 46.17%, 43.53%, 42.18%, and 34.83%, respectively, with an average reduction of 41.68%. The mean absolute error of TCNMABG is reduced by 37.33%, 39.71%, 30.70%, and 34.82%, respectively, with an average reduction of 35.64%. In addition, according to the R² results, the R² of TCNMABG is increased by 17.01% on average compared with the other four methods, and the RUL prediction curve fits the real remaining useful life label better, which improves the accuracy of RUL prediction results.

5. Conclusions

This paper proposes an RUL prediction framework of rolling bearing based on multi-domain mixed features and improved TCN. To better extract the key features, a signal noise reduction method based on DBOVMD adaptively determines the optimal parameter combinations of VMD to reduce the signal noise, and ISOMAP is used to reduce the dimension of multi-domain mixed features. KFCM is used to determine the FPT of early degradation time points to mark the label of data. TCNMABG can predict the bearing RUL better, with an average reduction of 40.72% in the MSE value of the bearing RUL prediction error based on the XJTU-SY dataset and an average reduction of 65.96% in the MSE value of the bearing RUL prediction error based on the FEMTO-ST dataset.

The method proposed in this paper offers novel suggestions and ideas for researchers in the field of remaining useful life prediction. However, further improvements are required to enhance the model complexity. In future studies: (1) A feature extraction method for multi-modal data fusion can be designed by integrating information from various sensors and data sources, such as vibration, temperature, sound, and other signals. This will enhance the overall perception of the deteriorating state of rolling bearing. (2) By incorporating embedded transfer learning and domain adaptive methods, valuable degradation features can be accurately extracted under different working conditions and environments. (3) Research on the uncertainty of remaining useful life prediction models to improve the reliability and interpretability of prediction results.