Prediction of Slope Safety Factor Based on Attention Mechanism-Enhanced CNN-GRU

Abstract

This paper proposes a new method for predicting slope safety factors that combines convolutional neural networks (CNNs), gated recurrent units (GRUs), and attention mechanisms. This method can better capture long-term dependencies, enhance the ability to model sequential data, and reduce the dependence on noisy data, thereby reducing the risk of overfitting. The goal is to improve the accuracy of slope safety factor prediction, detect potential slope stability issues in a timely manner, and take corresponding preventive and control measures to ensure the long-term stability and safety of infrastructure and promote sustainable development. The Pearson correlation coefficient is used to analyze the relationship between the target safety factor and the collected parameters. A one-dimensional CNN layer is used to extract high-dimensional features from the input data, and then a GRU layer is used to capture the correlation between parameters in the sequence. Finally, an attention mechanism is introduced to optimize the weights of the GRU output, enhance the influence of key information, and optimize the overall prediction model. The performance of the proposed model is evaluated using metrics such as the mean absolute error (MAE), mean absolute percentage error (MAPE), mean squared error (MSE), root-mean-square error (RMSE), and R². The results show that the CNN-GRU-SE model outperforms the GRU, CNN, and CNN-GRU models in terms of prediction accuracy for slope safety factors, with improvements of 4%, 2%, and 1%, respectively. Overall, the research in this paper makes valuable contributions to the field of slope safety factor prediction, and the proposed method also has the potential to be extended to other time-series prediction fields, providing support for a wide range of engineering applications and further promoting the realization of sustainable development.

1. Introduction

As the Chinese economy develops and material demand increases, the number of slope projects in China will continue to rise. The frequent occurrence of slope instability and damage poses significant potential safety hazards to workers and also causes serious economic losses [1]. Consequently, the establishment of a secure, dependable, and efficacious slope stability prediction model represents a pivotal challenge in geotechnical engineering. Nevertheless, the unpredictability of geological and geotechnical properties renders slope instability and failure a profoundly intricate geological process [2].

In recent years, researchers have conducted extensive research on slope stability prediction through theoretical calculation [3], numerical simulation [4], machine learning [5], and other methods, with notable outcomes. Both theoretical calculations and numerical simulations can be employed to consider the stress exerted on a slope, analyze deformation and stability, and elucidate the failure mode of a slope. Nevertheless, theoretical calculations cannot take into account the constitutive relationship of soil, especially in the case of multi-layer soil. Consequently, in slope stability analysis, this method is unable to accurately reveal the requisite safety and reliability. However, a numerical model necessitates a substantial amount of engineering geological investigation at the outset, which is costly and has limitations.

The advent of big data and machine learning has led to the widespread adoption of machine learning-based slope stability prediction techniques. M. G. Sakellariou et al. [6] applied an Artificial Neural Network (ANN) model to the analysis of slope stability, achieving preliminary results. Subsequently, H.B. Wang et al. [7] evaluated the data of the Yudonghe River landslide in Hubei Province of China through a BPNN and concluded that the landslide was in a critical stable state. Behrouz Gordan et al. [8] combined an ANN and PSO (Particle Swarm Optimization) to optimize the hyperparameters of the ANN model, thereby enhancing the model’s capacity to predict slope stability during an earthquake. Li et al. [9] studied the application of an RBFNN model in the prediction of open-pit slope stability and compared the optimization effects of BSO, GA, and MLP on it. Arsalan Mahmoodzadeh et al. [10] applied six kinds of machine learning algorithms to the analysis and prediction of slope stability. Huang et al. [11] proposed the application of a deep learning model, LSTM, to solve the problem of slope stability prediction. Table 1 provides incomplete statistics on the application of machine learning algorithms in slope stability prediction.

In other aspects, Tang et al. [12] performed a prediction study of ionospheric TEC based on a CNN-LSTM–attention mechanism and found that the model could maintain stability during different months and under different geomagnetic conditions. Liu et al. [13] used the earthworm algorithm to optimize support vector regression for predicting reservoir slopes. The experimental results showed that the model could accurately predict the displacement of reservoir landslides. Ma et al. [14] proposed a backend model for automated machine learning (AutoML), which is easy for field personnel to use and meets practical needs.

The limitations of traditional machine learning models include their inability to accurately capture complex patterns and relationships in data, their lack of capacity to deal with noise or missing data, and their inability to adapt to changes in the operating environment. In comparison with traditional machine learning, deep learning has a greater number of network layers and is better able to express the features of objects through the automatic extraction of feature information [11]. Consequently, an increasing number of scholars are utilizing deep learning techniques to address the limitations of traditional machine learning models. LSTM incorporates three gate functions: the input gate, forgetting gate, and output gate, which regulate the input value, memory value, and output value, respectively. However, it is challenging to train with numerous parameters. In the GRU (gated recurrent unit) model, there are only two gates: the update gate and the reset gate. When all hyperparameters are tuned, the performance of the two gates is comparable. The GRU structure is simpler, the number of training samples is smaller, and it is easier to implement. The gated recurrent unit network (GRU), as a recurrent neural network model, may encounter difficulties such as gradient disappearance, gradient explosion, and the loss of important information when processing ultra-long sequences. In order to address these issues, a convolutional neural network is employed to preprocess data and filter out irrelevant information [15]. Concurrently, the SE (Squeeze-and-Excitation) attention mechanism is incorporated into a CNN-GRU model, enabling it to dynamically adjust the importance of features, focus on key features, and enhance the performance of sequence data processing. The model is capable of more effectively capturing long-term dependencies, enhancing its ability to model sequence data, reducing its dependence on noisy data, and reducing the risk of overfitting [16].

The prediction of the slope safety factor is closely related to sustainable development. This study aims to promote the development of the slope stability prediction field and explore cutting-edge technologies to improve prediction accuracy. The article proposes a new method that combines an attention mechanism and CNN-GRU to enhance the predictive ability of the model. By helping the model focus on the most relevant input data features, attention mechanisms potentially enhance the model’s robustness to noise and data. This model is capable of capturing complex relationships and patterns in data and is applicable to a wide range of slope systems. The accurate prediction of slope stability can provide a foundation for engineering design, ensure construction safety, reduce geological hazard risks, and ensure the safety of life and property. This study makes valuable contributions to the accuracy and effectiveness of slope stability prediction and analysis by integrating advanced technologies and filling knowledge gaps.

Table 1. Machine learning methods applied to slope stability prediction.

Reference	Input Parameters	Year	Algorithm/Methods	Data Number
[6]	H, c, $\gamma ,\beta ,\phi ,r_u$	2004	ANN	46
[7]	H, c, $\gamma ,\beta ,\phi$	2005	BPNN	27
[17]	H, c, $\gamma ,\beta ,\phi ,r_u$	2008	SVM	46
[18]	H, c, $\gamma ,\beta ,\phi ,r_u$	2011	ANN	46
[19]	H, c, $\gamma ,\beta ,\phi$	2013	ANN	675
[20]	H, c, $\gamma ,\beta ,\phi ,r_u$	2014	ELM	97
[21]	H, c, $\gamma ,\beta ,\phi ,r_u$	2015	FA-LS-SVC	168
[22]	$X_{1-7}$	2015	GA-BP-ANN	120
[8]	H, c, PGA, $\beta ,\phi$	2016	PSO-ANN	699
[23]	H, c, $\gamma ,\beta ,\phi ,r_u$	2016	FNS, MARS, MGGP	103
[24]	c, $\beta ,\phi$, pp	2016	ANN	100
[25]	H, c, $\gamma ,\beta ,\phi ,r_u$	2017	NBC	69
[26]	H, c, $\gamma ,\beta ,\phi ,r_u$	2017	PSO-ANN	83
[27]	H, c, $\gamma ,\beta ,\phi ,r_u$	2017	PSO-LSSVM	46
[28]	H, c, $\gamma ,\beta ,\phi ,r_u$	2018	LR, DT, RF, GBM, SVM, MLPNN	168
[29]	H, c, $\gamma ,\beta ,\phi ,r_u$	2018	GSA, RF, SVM, Bayes	107
[30]	H, c, $\gamma ,\beta ,\phi ,r_u$	2018	GPC, QDA, SVM, ANN, ADB-DT, KNN	168
[31]	H, c, $\gamma ,\beta ,\phi ,r_u$	2019	GBM	221
[32]	$c_u,\beta$, w, b/B	2019	MLP, GPR, MLR, SLR, SVR	630
[33]	w, c, $\gamma ,\beta ,\phi$	2019	HHO-ANN	75
[34]	H, c, $\gamma ,\beta ,\phi$	2020	M5Rules–GA	450
[10]	H, c, $\gamma ,\beta ,\phi ,r_u$	2021	GRP, SVM, DT, LSTM, DNN, KNN	327
[9]	H, c, $\gamma ,\beta ,\phi$	2021	BSO-RBFNN, GA-RBFNN, MLP-RBBBFNN	495
[35]	$F_{1-12}$	2022	RF-XGBoost	786
[36]	H, c, $\beta ,\phi$, PGA	2022	DT, RF, AdaBoost	700
[11]	H, c, $\gamma ,\beta ,\phi$	2023	LSTM	2640
[37]	H, c, $\gamma ,\beta ,\phi ,r_u$	2023	DeepBoost	444
[1]	H, c, $\gamma ,\beta ,\phi$	2023	SVM, LR, DT, RF, KNN, NB, LDA	77
[5]	H, c, $\gamma ,\beta ,\phi ,r_u$	2023	SVM, RF, KNN, DT, GB	117

Here, $X_1$ is the elastic modulus, $X_2$ is the rock mass classification, $X_3$ is the installation height of the instrument, $X_4$ is the excavation height of the slope, $X_5$ is the measurement start time, $X_6$ is the measurement time period, and $X_7$ is the actual excavation height after measurement; PGA is the peak ground acceleration; b/B is the retracement distance ratio; $F_1$ is the elevation of the front edge, $F_2$ is the elevation of the back edge, $F_3$ is the slope height, $F_4$ is the slope angle, $F_5$ is the lithological property, $F_6$ is the inclination angle, $F_7$ is the dip direction, $F_8$ is the structure type, $F_9$ is the plane morphology, $F_{10}$ is the profile shape, $F_{11}$ is the landslide volume, and $F_{12}$ is the influence degree of human activities.

Table 1. Machine learning methods applied to slope stability prediction.

Reference	Input Parameters	Year	Algorithm/Methods	Data Number
[6]	H, c, $\gamma ,\beta ,\phi ,r_u$	2004	ANN	46
[7]	H, c, $\gamma ,\beta ,\phi$	2005	BPNN	27
[17]	H, c, $\gamma ,\beta ,\phi ,r_u$	2008	SVM	46
[18]	H, c, $\gamma ,\beta ,\phi ,r_u$	2011	ANN	46
[19]	H, c, $\gamma ,\beta ,\phi$	2013	ANN	675
[20]	H, c, $\gamma ,\beta ,\phi ,r_u$	2014	ELM	97
[21]	H, c, $\gamma ,\beta ,\phi ,r_u$	2015	FA-LS-SVC	168
[22]	$X_{1-7}$	2015	GA-BP-ANN	120
[8]	H, c, PGA, $\beta ,\phi$	2016	PSO-ANN	699
[23]	H, c, $\gamma ,\beta ,\phi ,r_u$	2016	FNS, MARS, MGGP	103
[24]	c, $\beta ,\phi$, pp	2016	ANN	100
[25]	H, c, $\gamma ,\beta ,\phi ,r_u$	2017	NBC	69
[26]	H, c, $\gamma ,\beta ,\phi ,r_u$	2017	PSO-ANN	83
[27]	H, c, $\gamma ,\beta ,\phi ,r_u$	2017	PSO-LSSVM	46
[28]	H, c, $\gamma ,\beta ,\phi ,r_u$	2018	LR, DT, RF, GBM, SVM, MLPNN	168
[29]	H, c, $\gamma ,\beta ,\phi ,r_u$	2018	GSA, RF, SVM, Bayes	107
[30]	H, c, $\gamma ,\beta ,\phi ,r_u$	2018	GPC, QDA, SVM, ANN, ADB-DT, KNN	168
[31]	H, c, $\gamma ,\beta ,\phi ,r_u$	2019	GBM	221
[32]	$c_u,\beta$, w, b/B	2019	MLP, GPR, MLR, SLR, SVR	630
[33]	w, c, $\gamma ,\beta ,\phi$	2019	HHO-ANN	75
[34]	H, c, $\gamma ,\beta ,\phi$	2020	M5Rules–GA	450
[10]	H, c, $\gamma ,\beta ,\phi ,r_u$	2021	GRP, SVM, DT, LSTM, DNN, KNN	327
[9]	H, c, $\gamma ,\beta ,\phi$	2021	BSO-RBFNN, GA-RBFNN, MLP-RBBBFNN	495
[35]	$F_{1-12}$	2022	RF-XGBoost	786
[36]	H, c, $\beta ,\phi$, PGA	2022	DT, RF, AdaBoost	700
[11]	H, c, $\gamma ,\beta ,\phi$	2023	LSTM	2640
[37]	H, c, $\gamma ,\beta ,\phi ,r_u$	2023	DeepBoost	444
[1]	H, c, $\gamma ,\beta ,\phi$	2023	SVM, LR, DT, RF, KNN, NB, LDA	77
[5]	H, c, $\gamma ,\beta ,\phi ,r_u$	2023	SVM, RF, KNN, DT, GB	117

Here, $X_1$ is the elastic modulus, $X_2$ is the rock mass classification, $X_3$ is the installation height of the instrument, $X_4$ is the excavation height of the slope, $X_5$ is the measurement start time, $X_6$ is the measurement time period, and $X_7$ is the actual excavation height after measurement; PGA is the peak ground acceleration; b/B is the retracement distance ratio; $F_1$ is the elevation of the front edge, $F_2$ is the elevation of the back edge, $F_3$ is the slope height, $F_4$ is the slope angle, $F_5$ is the lithological property, $F_6$ is the inclination angle, $F_7$ is the dip direction, $F_8$ is the structure type, $F_9$ is the plane morphology, $F_{10}$ is the profile shape, $F_{11}$ is the landslide volume, and $F_{12}$ is the influence degree of human activities.

2. Obtaining, Analyzing, and Processing Data

2.1. Factors Affecting Slope Stability

The selection of samples determines the upper limit of the model’s predictive ability. Given the intricate mechanism of slope stability and the multitude of influencing factors, scholars at home and abroad generally concur that slope instability is contingent upon the slope shape, rock and soil mass characteristics, and external influencing factors in machine learning or comprehensive evaluation methods. Among these factors, the most significant are the slope height and slope angle. The influence of rock and soil mass characteristics is also significant, with the bulk density, cohesion, internal friction angle, and pore pressure ratio of rock and soil masses being particularly relevant [38]. Figure 1 illustrates the factors that influence slope stability. The external factors mentioned, such as earthquakes or other human factors, were not considered in this study. The Factor of Safety (FOS) is a comprehensive index for evaluating slope stability. In the context of slope stability analysis, the FOS can be defined as the ratio of the slope sliding resistance to the slope sliding force. This ratio is directly related to the shear strength of the soil, as outlined in reference [39].

2.2. Establishment of Database

A total of 183 sets of slope stability samples were obtained from published public materials at home and abroad, with no duplicates or absences; see Appendix A [6,40,41]. All of the samples contain the six factors (H, $\alpha$, $c$, $\phi$, $\gamma$, and $r_u$) affecting slope stability that were previously discussed. In order to more effectively illustrate the distribution law and range of features, the slope stability prediction index system was analyzed in the form of a box diagram, a scatter diagram, and a half-violin diagram, as shown in Figure 2. In comparison to the conventional single violin or box plot, this integrated approach enables a more comprehensive illustration of the central tendency, dispersion, distribution density, and outliers of the data, thereby facilitating a more nuanced understanding of the data characteristics. At the same time, the Pearson correlation coefficient was employed to assess the six influencing factors. The resulting Pearson correlation coefficient and correlation scatter distribution diagram are presented in Figure 3. According to the literature, a Pearson correlation coefficient value between 0.4 and 0.6 is indicative of a moderate correlation between two factors. A correlation coefficient of 0.0 to 0.2 is indicative of a very weak or non-existent correlation. The analysis of the chart allows for a clear understanding of the degree of correlation between various factors. The observation of the scatter chart also permits the observation of a possible nonlinear relationship between factors. Consequently, in order to enhance the precision of the model, it is advisable to standardize and scale the data, with the objective of ensuring that they fall within the range of 0 to 1.

3. Establishment of Model

The CNN is employed to extract the overarching characteristics of the slope stability prediction model, whereas the GRU is utilized to discern the interrelationships between disparate sequences. The attention mechanism module is designed to extract key information from the GRU output, thereby assisting models in making more efficient use of the data.

3.1. CNN Model

The primary objective of a CNN is to identify and extract key features from the input data [42]. A typical convolutional neural network (CNN) architecture comprises multiple layers, including a convolutional layer, a pooling layer, a dropout layer, and a fully connected layer. In the process of feature extraction, the convolutional layer plays a pivotal role in capturing task-related feature information through the operation of a convolutional filter on input data [43]. As the number of convolutional cores increases, the abstraction level of extracted features will gradually increase, which will facilitate a more comprehensive understanding and analysis of the internal structure and underlying laws of the data, thereby enhancing the performance and generalization ability of the model [44].

3.2. GRU

A recurrent neural network (RNN) is a neural network architecture that has been specifically designed for the processing of sequential data. It can be scaled in order to efficiently utilize historical information. The Long Short-Term Memory Network (LSTM), an enhanced variant of the recurrent neural network (RNN), is designed to address the gradient disappearance issue and enhance the stability of the model [45]. The gated cycle unit (GRU) employs the gated mechanism to streamline the LSTM [46] process, as illustrated in Figure 4. Nevertheless, GRUs may be prone to the loss of crucial information when confronted with exceedingly lengthy sequences. To address this issue, convolutional neural networks can be employed to preprocess data and filter out irrelevant information, thereby enhancing the accuracy of slope stability prediction. Consequently, the gated structure of the GRU network, when combined with a convolutional neural network, represents an effective method for the processing of long-term dependencies and sequential data.

3.3. Squeeze-and-Excitation Attention

Attention mechanisms can extract valuable information from features and focus on local information without increasing computational complexity and are widely used to enhance the performance of network models. This approach does not increase the computational complexity, which has led to its widespread use in enhancing the performance of network models. In order to enhance the efficacy of the model, this study introduces Squeeze-and-Excitation attention, which facilitates the modeling of relationships between channels, thereby enhancing the capacity to extract efficacious features from constrained data. The enhancement of the response to important features and the suppression of the response to minor features result in the effective recalibration of features [47]. This approach to feature recalibration enables the network to focus more on useful information. Figure 5 depicts the structure of the attention mechanism, which is divided into four steps.

Step 1: Mapping. The essence of the implementation is convolution.

$$U=F_{tr}\left(X\right)$$

(1)

Here, $XϵR^{H^{\prime }\times W^{\prime }\times C^{\prime }}$, $UϵR^{H\times W\times C}$, $H^{\prime }$ and $H$ represent the heights of $X$ and $U$, $W^{\prime } \mathrm{a}\mathrm{n}\mathrm{d} W$ denote the widths, and $C^{\prime }$ and $C$ indicate the number of channels [48].

Step 2: Compression. Global Average Pooling (GAP) is used to compress the height and width of each channel to 1, and the final dimension is 1 × 1 × $C$.

$$Z=F_{sq}\left(U\right)={\displaystyle \frac{1}{h·w}}\sum {\sum }_{j=1}^{w}U(i,j)$$

(2)

Step 3: Excitation. The Z obtained in step 2 is passed through two fully connected layers to obtain the weight value.

$$S=F_{ex}\left(Z\right)=\sigma \left(w_2\delta \right(w_{1}z\left)\right)$$

(3)

where S is the generating weight, $w_1$ and $w_2$ represent two fully connected operations, $\delta$ is ReLU, and $\sigma$ is sigmoid. The two fully connected layers of this step first reduce the dimension of Z and then increase the dimension, and the generalization ability of the model is enhanced [49].

Step 4: Dot product. The results of step 3 and step 1 are calculated from the dot product of the corresponding channel through the following formula:

$$\tilde{X}=F_{scalc}\left(U,S\right)=S\times U$$

(4)

3.4. Model Frame

Figure 6 depicts the slope stability prediction model developed in this research, which is based on the CNN-GRU-SE method. This model comprises an input layer, a convolutional neural network (CNN), a gated recurrent unit (GRU), an attention layer, and an output layer. The input to the convolutional neural network (CNN) layer is the historical slope data, which are increased by the convolution operation, and the number of parameters is compressed. Furthermore, the feature dimension is reduced by a pooling process. The fully connected layer then converts the feature into a one-dimensional structure, thus completing the feature vector extraction. Concurrently, dropout layers are incorporated to offset the consequences of overfitting. The GRU and the attention layer learn the internal change rule from the extracted features, thereby enabling the prediction of the slope FOS. Furthermore, they fully extract relationships from historical data, expand important information about the acceptance domain, and reduce the feature dimension. The dynamic changes in CNN features are modeled and learned by the GRU network in order to extract the correlation between multiple features. The attention layer, in turn, employs the attention mechanism to assign different hidden-state probability weights to the GRU, thereby focusing on important information related to slope stability.

The proposed safety factor prediction model based on CNN-LSTM was optimized and implemented in Python 3.11.0 using the TensorFlow framework. The experimental hardware includes an Intel (R) Core(TM) i7-13700KF CPU and 32 GB RAM. The Adam optimization algorithm updates the network parameters with an initial learning rate of 0.01. The neural network training parameters are set to a maximum iteration count of 625 and a batch size of 16.

Table 2. Specific structural parameters of the model.

Layer Category	Neurons	Remark
Input layer	6 × 1
Convolution layer	5 × 32	Weights 2 × 32 Bias 1 × 32
Batch normalization	5 × 32	Offset 1 × 32 Scale 1 × 32
Re LU	5 × 32
Maximum pooling	5 × 32
Full connection	1 × 16	Weights 16 × 64 Bias 16 × 1
Re LU	8 × 16
Dimensional global averaging pooling	8 × 32
Full connection	1 × 16	Weights 16 × 32 Bias 16 × 1
GRU	1 × 16	Input weights (48 × 16) Recurrent weights (48 × 16) Bias (48 × 1)
Self-attention	1 × 16	Query weights (2 × 16) Key weights (2 × 16) Value weights (2 × 16) Output weights (16 × 2) Query Bias (2 × 1) Key Bias (2 × 1) Value Bias (2 × 1) Output Bias (16 × 1)
Full connection	1 × 1	Weights 1 × 16 Bias 1 × 1
Output layer	1 × 1

provides the specific structural parameters of the model.

4. Analysis of FOS Prediction Results of Slope

The training set (80%) and the test set (20%) should be split. In order to process inputs of different dimensions and sizes, it is necessary to normalize the sample data, as demonstrated in Equation (5):

$$\left\{\begin{array}{c}{X}_{ij}={\displaystyle \frac{x_{ij}-\mathrm{m}\mathrm{i}\mathrm{n}\left(x_{ij}\right)}{max\left(x_{ij}\right)-\mathrm{m}\mathrm{i}\mathrm{n}\left(x_{ij}\right)}}\\ Y_j={\displaystyle \frac{y_j-\mathrm{m}\mathrm{i}\mathrm{n}\left(y_j\right)}{max\left(y_j\right)-\mathrm{m}\mathrm{i}\mathrm{n}\left(y_j\right)}}\end{array}\right.$$

(5)

where $X_{ij}$ is the JTH input sample value of the ith attribute after normalization, and $x_{ij}$ is the jth input sample value of the ith attribute. $Y_j$ is the jth output sample value after normalization, and $y_j$ is the jth output sample value.

In order to test the validity of the model and evaluate the prediction effect, a number of evaluation indexes were selected, including the mean absolute error (MAE), mean absolute percentage error (MAPE), mean square error (MSE), root-mean-square error (RMSE), and R². The following is a description of the calculation formulae for each evaluation index:

$$MAE={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n\left|\widehat{y_i}-y_i\right|$$

(6)

$$MAPE={\displaystyle \frac{100\mathrm{\%}}{n}}{\sum }_{i=1}^n{\displaystyle \frac{\left|\widehat{y_i}-y_i\right|}{y_i}}$$

(7)

$$MSE={\displaystyle \frac{1}{m}}{\sum }_{i=1}^m{(y_i-\widehat{y_i})}^2$$

(8)

$$R^2=1-{\displaystyle \frac{\sum _i{(\widehat{y_i}-y_i)}^2}{\sum _i{(\overline{y_i}-y_i)}^2}}$$

(9)

where $y_i$ is the actual value, $\widehat{y_i}$ is the predicted value, and n is the number of data.

4.1. Performance Evaluation

This paper presents four models (CNN, GRU, CNN-GRU, CNN-GRU-SE) for the purpose of predicting the FOS of a slope to ascertain the superiority and prediction performance of the proposed model. The selection of the loss function and RMS error curves enables the assessment of the model’s performance during training. The change process is illustrated in Figure 7 and Figure 8. Figure 9 shows the prediction results of each model.

Figure 7 illustrates that the CNN model’s loss function value declines rapidly in the initial stage and then gradually stabilizes. As the number of iterations increases, the fluctuation of the loss value gradually decreases, indicating that the model becomes increasingly stable during the training process. The loss function value of the GRU model exhibits a relatively high degree of fluctuation following a period of decline, which may suggest that the model’s stability during training is inferior to that of the CNN model. The CNN-GRU model exhibits a similar decreasing trend in the loss function value to those of the CNN and CNN-GRU-SE models, although the amplitude of fluctuations during the iterative process is between those of the CNN and GRU models. The decline in the loss function value of the CNN-GRU-SE model is analogous to that observed in the CNN model. The loss function value is initially high and then decreases rapidly as the number of iterations increases, indicating that the model is learning and gradually improving its parameters to minimize the loss function. After approximately 500 iterations, the loss values begin to stabilize and fluctuate in a lower range, which may indicate that the model is approaching convergence and that further iterations will not significantly improve model performance. The overall loss function value of the CNN-GRU-SE model is lower than those of other models, indicating that it performs better than these models in predicting the FOS of the slope.

Figure 8 illustrates that in the initial stage of the CNN model, the error rate declines rapidly and then gradually stabilizes. Overall, although the error exhibited some fluctuations, it remained within the range of 0.5 to 1.5, and there were no discernible indications of overfitting or underfitting. In the initial iteration of the GRU model, the error exhibited a rapid decline, reaching a value of approximately 0.5, and remained at a low level throughout the training process. In comparison to the CNN model, the error fluctuations of the GRU model are more stable, although there are still small-amplitude fluctuations. The CNN-GRU model exhibits a pronounced decline in error at the outset of the iteration, accompanied by a subsequent pronounced fluctuation, particularly following the completion of 1000 iterations. The overall error level is comparable to that of the GRU model but exhibits greater volatility, which may indicate that the model adapts to the training data in a distinct manner. The CNN-GRU-SE model exhibits the most stable error variation during training. The error rapidly declines below 0.5 and remains at a low level throughout the training period. This suggests that the CNN-GRU-SE model may exhibit superior generalization and robustness.

4.2. Performance Comparison of Different Models

To further verify the effectiveness and applicability of the proposed model for predicting the FOSs of slopes, 37 test sets were used to compare the CNN-GRU-SE model with three widely used regression prediction models (ANN, DT, RF). MAE, MAPE, MSE, RMSE, and R² were used as evaluation metrics, as shown in Figure 10.

The CNN-GRU-SE slope FOS prediction model presented in this paper demonstrated high accuracy through a comprehensive analysis of the comparison graphs of various indexes. In comparison to the GRU model, the CNN-GRU-SE model demonstrated a higher level of accuracy, with an RMSE measurement of 0.09 in the prediction of the FOS of the slope. With regard to the mean absolute percentage error (MAPE) measure, the CNN-GRU-SE model yielded an increase in the FOS of 0.04 in comparison to the GRU prediction. Furthermore, the RMSE of the CNN-GRU-SE model for the FOS was found to be 0.04 greater than that of the CNN-GRU model. The mean absolute error (MAE) measurements demonstrated that the CNN-GRU-SE model exhibited an increase in the FOS of 0.06 compared to the GRU prediction and 0.03 compared to the CNN-GRU prediction. For R², the CNN-GRU-SE model demonstrated increases of 0.22 and 0.09, respectively, in comparison to the GRU and CNN-GRU models. Conversely, when the CNN-GRU-SE model is compared with the other three conventional regression models, it becomes evident that the CNN-GRU-SE model outperforms the conventional models in terms of the five aforementioned indicators. The comparison results demonstrate that the CNN-GRU-SE hybrid model exhibits superior prediction accuracy and stability in practical FOS prediction applications.

Figure 11 presents the error bar plots for the distinct models. It can be observed from the figure that the prediction effect of the CNN, GRU, DT, RF, and BP models applied independently is not satisfactory. This is because these models lack effective combinatorial optimization, which results in their prediction performance not being fully demonstrated. In contrast, the CNN-GRU-SE model demonstrates superior performance. With the exception of an outlier, the prediction results of this model are significantly superior to those of common regression models and are closer to the real values. Consequently, integrating the attention mechanism into the CNN-GRU model is a viable approach to enhancing the prediction precision.

The application of machine learning technology enables researchers to more fully and deeply mine the underlying information in a data set, thereby facilitating an understanding of the data, the discovery of the nature of the problem, and the identification of complex relationships. As engineering practice data sets continue to be enriched, it is anticipated that the accuracy and reliability of slope FOS prediction will be further enhanced. Researchers in the field of slope engineering can utilize the advantages of the attention mechanism to investigate complex phenomena associated with slope instability, with data serving as the primary focus.

To more intuitively display the contribution and relative importance of each feature to the model’s predictions, this paper introduces importance (Figure 12) and variable contribution (Figure 13) to quickly understand the ranking of feature importance.

Shap (Shapley Additive explanations) generates a predicted value for each sample model, and the Shap value is the numerical value assigned to each feature in the sample. Similar to the addition method of linear models, assuming the model’s baseline score (usually the mean of the target variable for all samples) is $y_{base}$, the ith sample is $x_i$, the ith feature of the jth sample is $\left(x_i,j\right)$, and the Shap value of this feature is $f\left(x_i,j\right)$, then the model’s predicted value for sample $x_i$ is

$$y_i=y_{base}+f\left(x_i,1\right)+f\left(x_i,2\right)+\cdots +f\left(x_i,j\right)$$

(10)

When $f\left(x_i,j\right)$ > 0, the feature plays a positive role in predicting the target value; conversely, the feature has an opposite effect on the target prediction value. Therefore, Shap not only gives the magnitude of the influence of the feature but also reflects the positive and negative influences of the features in each sample.

Figure 12 takes the absolute value average of the SHAP values for each feature to obtain the distribution of feature importance, which is equivalent to blurring the positive and negative effects in the above figure. The vertical axis of Figure 13 ranks the features according to the sum of the SHAP values of all samples, and the horizontal axis is the SHAP value (the distribution of the impacts of features on the model output); each point represents a sample, and the sample size is accumulated vertically. The color represents the feature value (red corresponds to high values, and blue corresponds to low values).

As can be seen in Figure 12 and Figure 13, in the CNN model, the slope angle and unit weight have a significant impact on the model output, with a more dispersed distribution of SHAP values, and significant effects in both the positive and negative directions. The effects of cohesion and slope height are relatively small, with most SHAP values concentrated near zero. The effects of the angle of internal friction and the water ratio on the model output are moderate, with SHAP values distributed near zero, with a small positive and negative impact. In the GRU model, the unit weight and slope angle still have a significant impact on the model output, but the order of importance of the two has been reversed, and their impact has increased compared to the CNN model, with a wide distribution of SHAP values. The impacts of feature variables in the CNN-GRU model are different from those in the GRU model, with slightly increased effects of the angle of internal friction and the water ratio, and more SHAP values are distributed in both positive and negative directions. By combining the SE attention mechanism, the impact of the water ratio in the SNN-GRU-SE model is significantly increased, surpassing that of the angle of internal friction, which may be due to the attention mechanism improving the model’s ability to extract features.

5. Conclusions

This study proposes a novel method, the CNN-GRU-SE model, with the objective of enhancing the precision of the prediction of the slope’s FOS. This research method employs a combination of convolutional neural networks (CNNs), gated recurrent units (GRUs), and attention mechanisms to more effectively capture long-term dependencies, enhance the ability to model sequence data, reduce the dependence on noisy data, and reduce the risk of overfitting. The principal findings of this study are as follows:

• The integration of an attention mechanism into the model has been demonstrated to significantly enhance the accuracy of weight allocation while simultaneously promoting rapid error convergence and a reduction in the error value. Concurrently, the performance and accuracy of the model are enhanced, and more favorable outcomes are achieved in the training and prediction processes of the model.
• The CNN-GRU-SE model demonstrates superior performance in terms of accuracy, outperforming traditional deep learning models. This advantage is particularly evident in the FOS prediction accuracy of the slope, which significantly improves the accuracy and reliability of prediction results.

The findings indicate that the CNN-GRU-SE model exhibits superior accuracy and reliability in the prediction of the slope’s FOS, which is of paramount importance for the prevention of slope instability incidents. The introduction of an attention mechanism into the prediction of the slope’s FOS opens up new avenues for the application of artificial intelligence technology in this field. The prediction results of the CNN-GRU-SE model enable site workers to effectively prevent and control slope instability accidents, thereby enhancing the efficiency and safety of slope engineering. In conclusion, the CNN-GRU-SE model proposed in this study demonstrates considerable potential for enhancing the accuracy of slope safety factor prediction. The model combines convolutional neural networks (CNNs), gated recurrent units (GRUs), and an attention mechanism to construct a high-performance model with robust and generalizable capabilities. This study employs a quantitative approach to assess the factors influencing slope stability, identifying six evaluation indexes to construct a predictive model. This represents an initial attempt to predict the safety factor of a slope. Although it is challenging to convert qualitative factors into quantitative factors, factors that have a significant impact on slope stability, such as rainfall and existing joints, must be considered. In particular, a significant incident occurred on the Meida Expressway in Guangdong, China, which was primarily attributable to the increase in the proportion of pore water caused by rainfall, resulting in slope instability and, ultimately, pavement collapse. Consequently, it is imperative to reinforce the research and prediction of slope stability, enhance the precision and dependability of these predictions, and furnish more support and assurance for sustainable development. Concurrently, the selection of a more impartial and rational evaluation metric for the slope safety factor will emerge as a focal point and challenge of future research.