Machine Learning Approaches for Slope Deformation Prediction Based on Monitored Time-Series Displacement Data: A Comparative Investigation

Abstract

Slope deformation prediction is one of the critical factors in the early warning of slope failure. Establishing an accurate slope deformation prediction model is important. Time-series displacement data of slopes directly reflect the deformation characteristics and stability properties of slopes. The use of existing data analysis approaches, such as statistical methods and machine learning algorithms, to establish a reasonable and accurate prediction model based on the monitored time-series displacement data is a common solution to slope deformation prediction. In this paper, we conduct a comparative investigation of machine learning approaches for slope deformation prediction based on monitored time-series displacement data. First, we established eleven slope deformation prediction models based on the time-series displacement data obtained from seven in situ monitoring points of the Huanglianshu landslide using machine learning approaches. Second, four evaluation metrics were used to comparatively analyze the prediction performance of all models at each monitoring point. The experimental results of the Huanglianshu landslide indicated that the long-short-term memory (LSTM) model with an attention mechanism and the transformer model achieved the highest prediction accuracy. The comparative analysis of model characteristics suggested that the Transformer model is better adapted to predict nonlinear landslide displacements that are affected by multiple factors. The drawn conclusion could help select a suitable slope deformation model for early landslide warnings.

1. Introduction

Landslides are one of the most serious geological disasters, which frequently occur worldwide and seriously affect people’s property and environmental safety. Therefore, slope deformation monitoring and early warning are of great significance [1,2]. Slope deformation prediction is one of the key factors in the early warning of slope failure. Establishing an accurate prediction model for forecasting slope deformation is critical. Slope deformation prediction models are mainly divided into physics-driven models and data-driven models [3]. Data-driven models, which are widely used and possess higher prediction accuracy, fully consider the complexity and nonlinearity of the deformation characteristics of slopes. [4,5].

The time-series forecasting method [6,7] is a typical data-driven approach for slope deformation, which predicts the future by learning the potential laws in the historical time-series displacement data of slopes. The time-series displacement data of slopes, which can directly reflect the deformation characteristics of slopes, are a series of displacement observation values obtained by the slope field monitoring equipment at a certain time interval. In practical engineering applications, field monitoring devices of the slopes are expensive, and the monitoring process is influenced by many factors [8,9]. Data analysis approaches such as statistical methods and machine learning approaches have been widely applied to various engineering problems with outstanding achievements in recent years. Therefore, exploiting the existing time-series forecasting approaches to establish a reasonable and accurate prediction model based on the monitored displacement data is a common solution to slope deformation prediction.

Currently, time-series forecasting methods mainly include three types: statistical approaches, traditional machine learning approaches, and deep learning approaches. The most popular statistical model is the autoregressive integrated moving average (ARIMA). The ARIMA model is mainly used to predict linear time-series data with accurate forecasting performance on short-term time series [10,11]. However, although the ARIMA can be adopted to achieve a highly accurate linear prediction, it cannot accurately forecast nonlinear time series. The ARIMA model needs to be combined with machine learning methods such as neural networks to improve the prediction performance on nonlinear time series [12,13]. For example, Shi et al. [14] pointed out that prediction models combining machine learning approaches and ARIMA outperformed a single ARIMA model.

The slope system is a complex nonlinear dynamic system, and the stability and deformation of slopes are affected by many uncertain factors, such as rock and soil properties, geomorphological features, rainfall, human engineering activities, and temperature changes. The interaction of these features leads to the complexity, ambiguity and randomness of slope deformation. Therefore, simple statistical methods are not suitable for complex nonlinear time-series problems of slope deformation, while machine learning is a promising alternative to traditional statistical methods due to its nonlinear learning capabilities.

The most applied traditional machine learning approaches for time-series forecasting are support vector regression (SVR) [15,16] and extreme gradient boosting (XGBoost). Compared with statistical approaches, machine learning approaches more easily capture the complexity, dynamics, and nonlinear characteristics of the nonlinear time-series displacement data of slopes. Currently, machine learning approaches have made some achievements in slope displacement prediction research [17,18]. For example, Miao et al. [19] proposed a coupled model of GA-SVR based on time series and a genetic algorithm and achieved an accurate landslide displacement prediction.

In recent years, deep learning has developed rapidly and performed better than traditional machine learning models in many studies [20]. The common deep learning approaches for time-series forecasting mainly include deep neural networks (DNNs) [21,22], convolutional neural networks (CNNs) [23], recurrent neural networks (RNNs) [24] and their variant models, as well as the transformer. Among them, RNNs have a strong model fitting ability for serialized data; and the most widely used RNN variants are the long-short-term memory (LSTM) [25] and gate recurrent unit (GRU) [26], which improve the shortcoming of the gradient explosion problem in conventional RNNs.

The LSTM and GRU are more effective on long sequence data than conventional RNNs and have achieved satisfactory prediction results in many engineering applications [27,28]. For example, Yang et al. [25] proposed an LSTM-based dynamic prediction model for landslide displacement and verified the performance of the model with observed data from two stepped landslides. Ma et al. [29] presented a novel deep learning method, T-GCN, which combined a GCN and a GRU to predict the slope deformation. However, when the input sequence is long, the performance of the LSTM model is still poor. LSTM models that include an attention mechanism are effective to improve the performance of the LSTM and have been applied to slope deformation in recent years [24,30,31].

The transformer is a new deep learning model proposed in 2017. The transformer model replaces the traditional sequential structure of RNNs with a self-attention mechanism, which allows the model to be trained in parallel and have access to global information. Currently, transformer models have been successfully applied to time-series forecasting problems [32,33]. However, few studies have applied the transformer to slope deformation prediction.

Currently, most studies for slope deformation focus on individual machine learning approaches compared to the ARIMA to show the nonlinear prediction performance of machine learning approaches. However, there are significant performance differences among different machine learning approaches, and few studies have conducted a comprehensive comparative analysis of the common machine learning models. In addition, some of the latest deep learning methods, such as the transformer, although outstanding in time series prediction problems in other fields, have hardly been studied for and applied to slope deformation prediction. Each model has different characteristics and application scenarios; thus, choosing an accurate model for slope deformation prediction is critical.

To address the above problem, in this paper, we conducted a comparative investigation of machine learning approaches for slope deformation prediction based on monitored time-series displacement data. The contributions in this paper can be summarized as follows:

(1)

We established two traditional machine learning models of the SVR and XGBoost and eight deep learning models, including the conventional DNN, CNN, RNN, LSTM, BiLSTM, Attention-LSTM, GRU, and transformer, while employing the linear ARIMA model as a baseline for comparison. The displacement data of seven in situ monitoring points of the Huanglianshu landslide located in eastern Chongqing were used as an example for verification.

(2)

We comparatively analyzed the prediction performance of the employed models based on four error metrics and provided a suggestion for selecting an appropriate prediction model for slope deformation.

The rest of this paper is organized as follows. Section 2 introduces the overview of the study area. Section 3 describes the applied prediction models in detail. Section 4 describes the predicted results of the models. Section 5 comparatively analyses the results. Section 6 concludes the paper.

2. Study Area

2.1. Overview of the Huanglianshu Landslide

The Huanglianshu landslide is located in Dabao village, Anping Town, Fengjie County, Eastern Chongqing, on the south bank of the Changjiang River in the upper reaches of the Qutang gorge. It is adjacent to the Yangtze River in the north and approximately 30 km upstream of the Qutang gorge. The geographical location is illustrated in Figure 1.

The Huanglianshu landslide is an old landslide that has been deformed many times in history. Local deformation of the landslide occurred after 836 mm rainfall on 30 May 2012. The landslide as a whole slid to form a sliding area with an area of approximately 6.25 × ${10}^4$ m² and a volume of approximately 80 × ${10}^4$ m³. In the early morning of 31 May, a large-scale overall sliding occurred in the lower deformation area of the landslide, and the landslide deformation area disintegrated seriously. After the landslide occurred, its deformation continued.

The Huanglianshu landslide is a representative landslide in the Three Gorges reservoir area. Since the storage operation of the Three Gorges Project, the original equilibrium of the landslides in the reservoir area has been seriously damaged, and the Three Gorges reservoir area has become a landslide-disaster-prone area. This paper analyzes the Huanglianshu landslide as an example, which is of great practical significance for the early warning of such landslides.

2.2. Monitoring Data

Fourteen GPS displacement monitoring points (FJ001–FJ014) were established in the Huanglianshu landslide working area, and the monitoring equipment recorded landslide displacements from May 2012 to June 2013. Since the large-scale local sliding resulted in the destruction of several GPS monitoring devices, we selected the monitoring points FJ003, FJ004, FJ005, FJ006, FJ009, FJ010, and FJ011 with complete records for research to reveal the deformation characteristics of the slope after local deformation and establish an appropriate prediction model. The employed data in this paper was provided by the Center for Hydrogeology and Environmental Geology of the China Geological Survey. The geographical location relationship of the monitoring points studied in this paper is illustrated in Figure 2.

Figure 3 shows the displacement change curve of the monitoring points of the Huanglianshu landslide. There is a complicated nonlinear deformation characteristic of the Huanglianshu landslide. After the local large deformation of the Huanglianshu landslide occurred on 31 May 2012, the deformation characteristics of each monitoring point were different in the following years. Among the points, the monitored average displacements on FJ003, FJ004, FJ005, and FJ006 were small, while the monitored average displacements on FJ009, FJ010, and FJ011 were large.

The deformation curves in Figure 3 illustrate that the displacement changes of monitoring points were closely related to the relative positions of the monitoring points and the deformation area. As is shown in Figure 2, monitoring points FJ010 and FJ011 were located in the local deformation area; monitoring point FJ009 was located on the right side of the deformation area; and the monitoring points FJ003, FJ004, FJ005, and FJ006 were located outside the local deformation area. Within one year after the occurrence of the local deformation, the sliding velocity of the monitoring points within the deformation was approximately 10 times higher than that of monitoring points above the deformation area.

3. Machine Learning Approaches for Slope Deformation Prediction

3.1. Overview

The prediction process for slope deformation is shown in Figure 4. First, the displacement data at seven monitoring points of the Huanglianshu landslide are divided into a training set and a test set after pre-processing. Second, the training set and test data are divided into multiple datasets according to the time-series forecasting strategy. Third, various statistical and machine learning models are established for training datasets. Finally, the prediction accuracy of each model is verified on the test set data. In this paper, we established a statistical model and 10 machine learning models for slope deformation prediction. The structures of the employed models are illustrated in Figure 5.

3.2. Data Preprocessing

3.2.1. Linear Interpolation

The collected data at the Huanglianshu landslide include the daily recorded data of seven monitoring points from June 2012 to June 2013, with an original data size of 7 × 158. However, since the recording interval of each month was different, some data included more than one day. Since the recording conditions at each monitoring point were the same, we took the FJ003 monitoring point as an example, and the recording interval is shown in Figure 6. On this basis, we adopted the linear interpolation method to transform the original data into daily data from 7 June 2012 to 31 May 2013. The total amount of data after interpolation was 7 × 359.

3.2.2. Data Standardization

Before establishing the time-series prediction model, it is necessary to standardize the series data to eliminate the influence of dimensions. In this paper, the maximum and minimum normalization method was used to normalize the displacement data to [0, 1]. The calculation method is as shown in Equation (1).

$$x^{{}^{\prime }}=\frac{x-x_{min}}{x_{max}-x_{min}}$$

(1)

where $x^{{}^{\prime }}$ represents the calculated value, $x_{max}$ is maximum value in the sequence data, and $x_{min}$ is minimum in the sequence data.

3.3. Time-Series Forecasting Strategy

The time-series problem can generally be expressed as follows. For a time-series containing N historical observation values [$y_1$, $y_2$, …, $y_N$], the recent d historical observation values [$y_{N-d+1}$, …, $y_N$] are used to predict the values [$y_{N+h}$, …, $y_{N+h}$] at the next H time points. When predicting according to the direct strategy [34], first, the sliding window method [35] is used to obtain H datasets that contain input data and target data, i.e., the d historical data as the input and next h data as the output. Second, H independent regression models are trained according to the functional relationship shown in Equation (2). Finally, H historical datasets are input into each model to predict the value at the corresponding time.

$${\widehat{y}}_{N+h}={\widehat{f}}_h(y_{N-d+1},\dots ,y_N),h\in 1,\dots ,H$$

(2)

where [$y_{N-d+1}$, …, $y_N$] is the input data, $y_{N+h}$ is the target data, H is the length of the prediction time, and ${\widehat{f}}_h$ is the regression model.

In this paper, first, the time-series displacement data at each monitoring point were divided into a training set and a test set. Second, each model was trained to obtain optimal parameters based on the training data. Third, the test data were predicted based on the trained model, as shown in Figure 4.

Specifically, in the training process, first, the training data were divided into an array of size 283 × 7 based on a sliding window with a window size of seven and a time step of one. Each array has a size of seven, where the first six data are used as inputs to the deep learning model, and the seventh data is used as the output. Second, the model learned the features of 283 displacement arrays and established a correspondence between the input data and output data. In the predicting process, first, the test data were divided into an array of size 66 × 7 based on the sliding window method. Second, the first six data in each array were fed into the trained model to predict the seventh displacement data. Finally, the predicted displacements were compared with the true displacements to evaluate the model performance.

3.4. Employed Statistical and Machine Learning Approaches

3.4.1. Statistical Approach: ARIMA

The most popular statistical model for time-series prediction is the ARIMA. The ARIMA is a traditional linear prediction model that has three undetermined parameters: autoregressive term order p, sequence difference order d, and moving average term order q. The process of determining p, d, and q is called the ARIMA model order determination.

In the entire order determination and optimation process of the ARIMA model, three tests need to be conducted: (1) the sequence stationarity test; (2) the residual white noise test; and (3) the parameter significance test. The workflow of the ARIMA model is shown in Figure 5a. In this paper, the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) were used to automatically determine the optimal parameters (p, d, and q). The optimal parameters of the ARIMA model for the time-series displacement data of the Huanglianshu landslide are shown in

Table 1. The optimal parameters of the ARIMA model for the displacement data of the Huanglianshu landslide.

Parameter	FJ003	FJ004	FJ005	FJ006	FJ009	FJ010	FJ011
p	1	3	1	1	0	1	1
d	0	1	0	1	1	2	2
q	0	0	2	1	0	0	0

.

3.4.2. Traditional Machine Learning Approaches

(1)

SVR

The SVR involves the application of a support vector machine (SVM) [36] to a regression problem. As is shown in Figure 5b, the principle of the SVR is to find a hyperplane so that the distance between the sample point farthest from the hyperplane and the plane is the shortest. In the nonlinear regression problem, the SVR can achieve a better regression effect by adding a kernel function. The choice of the kernel function is critical for performance. The main optional kernel functions include the linear function, polynomial function, and radial basis function (RBF).

(2)

XGBoost

As shown in Figure 5c, XGBoost is a tree-based model. In this paper, the sliding window representation was used to transform the time-series dataset of Huanglianshu landslide into supervised learning, and the k-fold cross validation method was exploited to evaluate the performance of the model. When the input is a time series and the output is a predicted value, each leaf node in the model is no longer a category but instead is a value, i.e., the prediction score. The calculation principle is shown in Equation (3).

$$\widehat{y_i}={\displaystyle \sum _{k=1}^K}{f}_k\left(x_i\right)$$

(3)

where $\widehat{y_i}$ is the predicted value, $x_i$ is the feature vector, $f_k\left(x_i\right)$ is the calculated value for each tree, and K is the total number of trees; in this paper, K is 1000.

3.4.3. Deep Learning Approaches

(1)

Conventional DNN

The DNN is a neural network that contains multiple hidden layers. DNNs are divided into three types of neural network layers: the input layer, hidden layers, and the output layer. Generally, the first layer is the input layer, the middle layers are hidden layers, and the last layer is the output layer. In this paper, we established a conventional DNN model as a comparison baseline, and the layers of the model were fully connected to each other.

(2)

CNN

The CNN is type of feedforward neural network. Common CNNs consist of a convolution layer, a pooling layer, and a fully connected layer. CNNs play an important role in the field of computer vision; furthermore, CNNs can also be adapted to time-series problems. The difference between them is that a two-dimensional convolution kernel is applied to an image, while a one-dimensional convolution kernel is applied to a time series.

(3)

Conventional RNN, LSTM, GRU, BiLSTM and Attention-LSTM

The RNN is a recurrent neural network that is mainly used to model sequence data. Common RNNs consist of an input layer, hidden layers, and an output layer. In contrast to the DNN and CNN, the RNN is quite effective for data with sequential characteristics and possesses a strong model fitting ability, which can mine information between time-series data.

The LSTM and GRU are two variants that improve the limitation of RNNs for long-term dependence. The LSTM is a special network structure with three “gates” that can be used to remove or increase the information of the “cell state”. The gate used to increase information is called the “update gate”, the gate used for forgetting is called the “forget gate”, and the “output gate” is used for the output. The GRU is a variant of the LSTM model that combines the input gate and forget gate into an update gate. The internal structures of the RNN, LSTM, and GRU are shown in Figure 7.

BiLSTM [37] is a combination of the forward LSTM and backward LSTM. Modeling time series with an LSTM cannot encode back-to-back information, but a BiLSTM can better capture the relationships between sequence data in both directions.

The Attention-LSTM model is an LSTM model with an attention mechanism. In recent years, the attention mechanism, which can focus on the key part of time-series data to reduce the impact of nonkey data for prediction and is considered a fully connected layer and softmax function, has been widely used.

(4)

Transformer

The transformer is a sequence-to-sequence (seq2seq) structure [38] with encoder and decoder networks that use a self-attention mechanism. Compared with the traditional RNN and CNN, the transformer model proposes a new position encoding mechanism to capture the time-series information between input data; thus, the transformer model can fully capture the time-series information with richer dimensions. In the time-series prediction problem, the encoder takes the history of the time-series as the input, while the decoder predicts the future value via an autoregression. The employed transformer model in this paper included the encoder part of the transformer and finally connected a fully connected layer.

4. Results

4.1. Results of Predicted Displacements

In this paper, a total of eight deep learning models were established. Hyperparameters [39] are essential for optimizing the training performance of deep learning models. The optimal hyperparameters for the eight models were finally determined based on multiple training experiments, as are shown in

Table 2. Hyperparameters of the deep learning models.

Learning Rate	Batch Size	Number of Epochs	Optimizer	Loss Function
0.002	100	1000	Adam	MSE

.

The predicted displacements of the eleven models and the measured displacements on the test data at seven monitoring points of the Huanglianshu landslide are shown in Figure 8b. As is shown in Figure 8b, the eleven models generally showed that there was a lag in the predicted displacements, i.e., the predicted value at time t was usually close to the measured value at time $t-$1. The lag in the predicted displacements suggests the existence of autocorrelation in the time-series displacement data of the Huanglianshu landslide.

We conducted an autocorrelation function on the seven time-series displacement data to verify the autocorrelation of the landslide displacements. The autocorrelation function describes the correlation between the data at the current moment and the subsequent moments. As are shown in Figure 8a, the autocorrelation results indicate the varying degrees of autocorrelation in the seven sets of displacement data, which were consistent with the predicted results in Figure 8b.

4.2. Prediction Metrics and Accuracy

In this paper, we used four metrics to evaluate the prediction performance of the eleven models for slope deformation: the mean absolute error (MAE), mean absolute percentage error (MAPE) [40], root mean square error (RMSE), and $R^2$, which are defined in

Table 3. Metrics of the prediction performance.

Metric	Definition	Description
Mean absolute error	MAE = $\frac{1}{n}{\displaystyle \sum _{t=1}^n}|y_t^{{}^{\prime }}-y_t|$	$y_t^{{}^{\prime }}$ is the predicted value, $y_t$ is the measured value, and n is the number of test data. MAE represents the average absolute error between measured and predicted values.
Mean absolute percent error	MAPE = $\frac{100\%}{n}{\displaystyle \sum _{t=1}^n}\frac{|y_t^{{}^{\prime }}-y_t|}{y_t}$	MAPE represents the percentage of errors between measured and predicted values of all samples.
Root mean square error	RMSE = $\sqrt{\frac{1}{n}{\displaystyle \sum _{t=1}^n}{(y_t^{{}^{\prime }}-y_t)}^2}$	RMSE represents the root mean square error between measured and predicted values.
Coefficient of determination	$R^2=1-\frac{{\displaystyle \sum _{t=1}^n}{(y_t^{{}^{\prime }}-y_t)}^2}{{\displaystyle \sum _{t=1}^n}{(\overline{y}-y_t)}^2}$	The closer the value of $R^2$ is to 1, the better the prediction performance of the model.

. The results of the prediction accuracy when using the above four metrics are shown in

Table 4. Prediction accuracies of all models.

Metric	Model	FJ003	FJ004	FJ005	FJ006	FJ009	FJ010	FJ011
MAE	ARIMA	2.7514	2.6987	1.7134	2.0778	4.5172	4.2154	2.5601
	SVR	3.1167	3.0527	1.9669	2.0236	4.5256	4.5768	3.2575
	XGBoost	3.2107	2.7615	1.9541	2.2226	5.6654	4.2474	2.4652
	DNN	3.4244	2.9413	1.8209	2.1979	5.6416	4.4444	3.0304
	CNN	2.9597	3.0787	1.7600	2.2304	6.1500	4.9373	2.6765
	RNN	2.8005	2.8538	1.7716	2.1441	4.7417	4.6331	2.6581
	LSTM	2.7278	3.0349	1.7750	2.0582	5.3751	4.6890	3.0843
	BiLSTM	2.7548	3.0843	1.8252	2.0733	5.1470	4.5105	2.5058
	Attention-LSTM	2.8236	3.0341	1.7601	2.0081	5.2749	4.6290	2.3208
	GRU	2.6942	2.7917	1.7928	2.0549	4.8912	4.2850	2.5024
	Transformer	2.4117	3.0025	1.7117	2.0222	5.9667	3.9941	2.6732
MAPE	ARIMA	0.1761	0.3937	0.1310	0.4546	0.0152	0.0113	0.0229
	SVR	0.1901	0.4882	0.1571	0.4632	0.0153	0.0122	0.0289
	XGBoost	0.2018	0.4364	0.1433	0.4997	0.0188	0.0114	0.0220
	DNN	0.2202	0.5330	0.1397	0.4914	0.0194	0.0120	0.0271
	CNN	0.1879	0.5294	0.1398	0.5062	0.0211	0.0133	0.0241
	RNN	0.1731	0.4448	0.1352	0.5038	0.0159	0.0123	0.0236
	LSTM	0.1674	0.5202	0.1340	0.4649	0.0180	0.0124	0.0275
	BiLSTM	0.1689	0.5408	0.1390	0.4733	0.0172	0.0120	0.0224
	Attention-LSTM	0.1735	0.5218	0.1305	0.4779	0.0181	0.0125	0.0208
	GRU	0.1698	0.4322	0.1375	0.4739	0.0165	0.0114	0.0223
	Transformer	0.1614	0.4946	0.1367	0.4444	0.0203	0.0106	0.0238
RMSE	ARIMA	4.2655	3.8868	2.5959	2.9387	7.2359	6.9043	4.4374
	SVR	4.6544	4.0596	2.7387	2.8294	7.4112	7.3795	5.7565
	XGBoost	5.5068	4.1025	3.0221	3.2335	9.0456	6.8872	4.2473
	DNN	4.8558	4.0541	2.5911	3.0223	8.1783	6.4410	4.8310
	CNN	4.6576	4.1445	2.6057	3.0068	8.5456	6.7313	4.5103
	RNN	4.3923	3.9846	2.5707	2.9604	7.4221	6.8591	4.4002
	LSTM	4.3140	4.0351	2.5216	2.9264	8.2447	7.0540	4.5134
	BiLSTM	4.3429	3.9516	2.5771	2.9241	7.8710	6.6652	4.3341
	Attention-LSTM	4.3721	3.9583	2.5433	2.7559	7.4804	6.5511	4.2251
	GRU	4.3724	3.9943	2.5824	2.8576	7.6167	6.5304	4.2470
	Transformer	3.9680	3.9450	2.5914	2.8247	8.4681	6.3129	4.3167
$R^2$	ARIMA	0.8423	0.5088	0.5170	0.3253	0.8759	0.5672	0.4649
	SVR	0.8122	0.4642	0.4625	0.3746	0.8698	0.5056	0.0995
	XGBoost	0.7371	0.4528	0.3454	0.1832	0.8061	0.5693	0.5098
	DNN	0.7956	0.4656	0.5188	0.2864	0.8415	0.6233	0.4057
	CNN	0.8119	0.4415	0.5134	0.2937	0.8269	0.5886	0.4472
	RNN	0.8327	0.4838	0.5264	0.3153	0.8695	0.5728	0.4738
	LSTM	0.8386	0.4706	0.5443	0.3310	0.8389	0.5482	0.4464
	BiLSTM	0.8365	0.4923	0.5240	0.3320	0.8532	0.5967	0.4895
	Attention-LSTM	0.8343	0.4906	0.5364	0.4067	0.8674	0.6103	0.5149
	GRU	0.8342	0.4813	0.5220	0.3621	0.8625	0.6128	0.5036
	Transformer	0.8635	0.4940	0.5187	0.3768	0.8301	0.6382	0.4936

Bold of numbers denotes the best performance value for each data set.

. The smaller the MAE or the RMSE, or the MAPE value, the higher the prediction accuracy of the model, and MAPE = 0 indicates a perfect model; the closer the value of $R^2$ is to 1, the higher the prediction accuracy of the model.

5. Comparative Analysis

5.1. Stability Test of the Displacement Data and Evaluation of Model Performance

5.1.1. Stability Test of the Mointored Displacement Data

In order to investigate the deformation pattern of the time-series displacement data within and outside the deformation area of the Huanglianshu landslide after local sliding, we conducted a stability test on all monitoring points using the augmented Dickey Fuller (ADF) test [41]. The ADF test determines whether there is a unit root in the time series. If the time series is stationary, there is no unit root. Therefore, the ADF test assumes the existence of a unit root; and if the obtained test statistic p-value is less than significance levels 10, 5, or 1%, it is equivalent to having 90, 95, or 99% certainty against the original hypothesis, respectively. In this paper, we adopted 5% as the significance level, i.e., the situation of the p-value > 5% indicated that the time-series was nonstationary.

As are listed in

Table 5. Results of the ADF test for time-series displacement data at 7 monitoring points.

Monitoring Point	FJ003	FJ004	FJ005	FJ006	FJ009	FJ010	FJ011
p-value	0.000071	0.009987	0.006490	0.013365	0.002463	0.074062	0.797659

, the displacement data at monitoring points FJ003, FJ004, FJ005, FJ006, and FJ009 were stationary, and the displacement data at monitoring points FJ010 and FJ011 within the deformation area were not stationary. The above results indicate that the displacement data within the deformation area had more complex nonlinear relationships.

5.1.2. Evaluation of Model Prediction Performance

According to Table 4, we evaluated the prediction performance of the eleven models when using the four different evaluation metrics in Figure 9. The evaluation results were as follows:

(1)

$R^2$ reflected the overall fit of the prediction models to the real displacement data. As shown in Figure 9b, when using $R^2$ as the evaluation index, all models had the highest prediction accuracies at the monitoring points FJ003 and FJ009, while the prediction accuracies at monitoring points FJ004, FJ005, FJ006, and FJ011 were relatively low. Specifically, monitoring point FJ006 had the lowest prediction accuracy. As a whole, except for the monitoring point FJ006, the $R^2$ of all other monitoring points reached more than 0.5, which showed that, after large-scale local sliding of the slope, the displacement data both within the deformation area and its surrounding area had a certain linear or nonlinear change characteristic, i.e., the slope deformation could be learned to achieve accurate prediction through a specific model.

(2)

Upon combing through the results of the four evaluation metrics in Figure 9, it can be concluded that the optimal prediction models at each monitoring point were the same when using the MAE and MAPE as evaluation metrics, and the optimal prediction models at each monitoring point were the same when using the $R^2$ and RMSE as evaluation metrics. In summary, the ARIMA, the Attention-LSTM, and the transformer model had higher prediction accuracies. In order to compare the accuracy differences of the models for the displacement data within and outside the deformation area, we list the best and worst prediction models at the seven monitoring points of the Huanglianshu landslide in

Table 6. The best and worst prediction models for displacement data within and outside the local deformation area.

Monitoring Point	Outside the Deformation Area					Inside the Deformation Area
	FJ003	FJ004	FJ005	FJ006	FJ009	FJ010	FJ011
Best model	Transformer	ARIMA	Attention-LSTM	Attention-LSTM	ARIMA	Transformer	Attention-LSTM
Worst model	XGBoost	CNN	XGBoost	XGBoost	XGBoost	SVR	SVR

. Table 6 indicated that, although the ARIMA model performed with outstanding prediction accuracy for the displacement data outside the deformation area, it did not perform as well as the transformer and Attention-LSTM models for the displacement data inside the deformation area. Second, the traditional machine learning models XGBoost and SVR had the worst prediction accuracies.

5.2. Comparative Analysis of the Model Characteristics Based on Prediction Results

Typically, the prediction performance of the model is mainly affected by the model characteristics and the original displacement data. The analysis of the influence of the characteristics of the models on the prediction results is as follows.

The ARIMA statistical method was effective for predicting short-term stationary time-series data. In this paper, we employed the displacement data of the next two months for prediction, which belonged to the range of short-term prediction; and the prediction results proved the ability of the ARIMA model to predict short-term problems. However, the ARIMA utilized a traditional linear method, and it is difficult to capture the nonlinear relationships between the displacement data of slopes.

As is shown in Table 6, the prediction accuracy of the ARIMA on the nonstationary displacement data at the monitoring points of FJ010 and FJ011 within the deformation area was much lower than that of most machine learning models, which suggests that ARIMA is better suited to predicting more stable displacement data at monitoring points outside the deformarion area. However, for the actual time-series displacement data of slopes, because of the various uncertain factors, such as weather changes and rainfall, the original data were normally nonstationary. Therefore, the ARIMA model was actually not the best choice for slope deformation prediction in most cases.

The prediction performance of the SVR model was closely related to the selection of the model kernel function. As mentioned in Section 3.4.2, we adopted three kernel functions to predict the displacement data at each monitoring point separately. The calculation results show that the displacement data at monitoring points FJ006, FJ010, and FJ011 were suitable for the RBF kernel function, and other monitoring points were suitable for the linear kernel function. As is shown in Table 3, the SVR model achieved more accurate prediction results for the displacement data of monitoring points FJ006, FJ010, and FJ011, which indicates that the SVR was better at handling nonstationary displacement data within the deformation area than the stationary displacement data outside the deformation area.

Moreover, the traditional machine learning models SVR and XGBoost have outlier prediction values. As shown in Figure 10, the SVR model had outlier prediction values that deviated seriously from the real situation at monitoring point FJ011 (Figure 10a), and the XGBoost model had outlier prediction values at monitoring point FJ003 (Figure 10b). Although the SVR model and XGBoost model can roughly predict the changing trend of displacement data, they have difficulty capturing unusual fluctuations in the data. In other words, the SVR and XGBoost models have more stable performance fitting the data in the normal range, while the ARIMA model is better than the SVR and XGBoost for outlier fitting. Therefore, when the displacement data have noise, the traditional machine models SVR and XGBoost would be more sensitive to overfitting.

For the deep learning models, overall, the conventional RNN and its variant LSTM, GRU, BiLSTM, and Attention-LSTM models achieved more accurate predictions compared with the conventional DNN model and CNN model, and the RNN variant model had better prediction performance than the conventional RNN at the monitoring points within and outside the deformation area. In a conventional RNN model and its variant models, the network structure can store the information on each neuron; whereas in a conventional DNN, information only passes through the network in the forward direction. As mentioned in Section 4.2, there was a first-order autocorrelation between the before and after data in the time-series displacement data of the Huanglianshu landslide; thus, the conventional RNN model and its variant models were more suitable for slope deformation prediction.

Among all deep learning models, the CNN model employed in this paper accomplished the lowest prediction accuracy at all monitoring points. Actually, the CNN method was placed before the input of the LSTM model for the first step to reduce the features [24,42] in most applications. The CNN method was used to capture the deep features in the data, and then the LSTM method was adopted to extract the time-series features in the data. Therefore, using a single CNN model is not suitable for predicting slope deformation.

Among the conventional RNN and its variant models, the BiLSTM and the Attention-LSTM models were more prominent, especially the Attention-LSTM model. The better prediction performance of the LSTM and GRU models was due to their improvement of the long-term dependence problem of the RNN models. As shown in Figure 10, when using the $R^2$ as the metric, the prediction accuracy of the Attention-LSTM model at five monitoring points of FJ004, FJ006, FJ009, FJ010, and FJ011 was much higher than that of the ordinary LSTM. As listed in Table 5, the values of the ADF test at FJ006, FJ010, and FJ011 were greater than 0.01, and the value at FJ004 was close to 0.01. Therefore, when the displacement data are more unsteady, the fitting performance of the Attention-LSTM model is more outstanding.

The above-predicted results at seven monitoring points of the Huanglianshu landslide indicate that the introduction of the attention mechanism has an important influence on the nonlinear displacement data. The deformation of a slope is influenced by many factors, such as rainfall and weather changes, and the influence weight of these potential factors on slope deformation is reflected in the time-series displacement data. The conventional LSTM model cannot focus on the important parts of the displacement data. However, applying the attention mechanism to the LSTM model allows the neural network to adaptively focus on the local displacement data of input time-series that are more important to the current output. Therefore, the LSTM model with an attention mechanism was more effective for slope deformation prediction.

As listed in Table 4, the values of the four prediction metrics of the transformer model on monitoring points FJ010 and FJ011 within the deformation area were significantly higher than other models, while the displacement data of monitoring points FJ003 and FJ005 outside the deformation area also had high prediction accuracy. This indicates that the transformer model has strong nonlinear processing ability, and the displacement within and outside the deformation area of the Huanglianshu landslide can be accurately predicted by the transformer model. The network structure of the transformer model is entirely composed of the self-attention mechanism. The self-attention mechanism is a variant of the attention mechanism, is less dependent on external information, and is better at capturing internal correlations between displacement data. The self-attention mechanism focuses on the important parts of the time-series data by assigning different weights, which reduces the degree of influence of abnormal points on the prediction results to improve the prediction accuracy.

In addition, for some special slopes, such as the rainfall-induced landslides [43,44], the landslide displacement is mainly influenced by the amount of rainfall; furthermore, the rainfall in various regions of the world shows obvious annual periodicity, which means that the displacement data of the slope have year-long dependency. Compared with the LSTM model, the transformer model had stronger long-term dependency modeling capabilities and has a better effect on long sequences. Therefore, the transformer model is better adapted to predict nonlinear landslide displacements affected by multiple factors.

5.3. Discussion

Based on the above analysis, we first conclude that the displacement data at the monitoring points within the deformation area of the Huanglianshu landslide have more complex nonlinear relationshisps than the displacement data at the monitoring points outside the deformation area. Second, the ARIMA statistical method is effective for predicting short-term stationary time-series displacement data within the deformation area, and the machine learning models are better at handling the nonstationary displacement data outside the deformation area. Third, the traditional machine learning models of the SVR and XGBoost were less effective than the ARIMA model at fitting outlier points in displacement data. Finally, deep learning approaches have better application prospects in slope deformation prediction problems, especially LSTM models with an attention mechanism, as well as transformer models.

At present, there is almost no research on applying the transformer model to slope deformation predictions. For time-series forecasting problems, the transformer model far surpassed the traditional long- and short-term memory networks in a variety of application scenarios due to its unique network structure. In recent years, with the rapid development of data science, deep learning has made outstanding achievements in various disciplines, which are not limited to the natural language processing (NLP) field. The application of deep learning to solve slope deformation prediction problems will become a powerful solution. In this paper, we established different prediction models based on the monitoring data of the Huanglianshu landslide and the development history of the time-series prediction method. We hope to apply the latest and most applicable computer technologies to slope prevention and early warning work, and to make advanced computer technologies an effective means of solving various geological disasters to compensate for the limitations of traditional physical means.

In the future, we will first comprehensively consider multiple influencing factors of slope deformation and exploit the existing deep learning approaches to establish a multivariable slope displacement prediction model. Second, more data interpolation methods will be added to the analysis. Finally, at present, spatiotemporal prediction [45,46] has been widely supported in traffic flow forecasting, that is, forecasting from two aspects, time and space, which is a great inspiration for multimonitoring landslides. Therefore, we will develop a spatiotemporal model for slope deformation prediction according to the development of the spatiotemporal prediction in the transportation field.

6. Conclusions

In this paper, to obtain an accurate slope deformation prediction, we conducted a comparative investigation between the ARIMA statistical method and 10 machine learning approaches based on monitored time-series displacement data of the Huanglianshu landslide located in eastern Chongqing. Four error metrics were used to comparatively analyze the prediction performance of all prediction models. The prediction results and comparative analysis of the model characteristics indicate the following: (1) the displacement data within the deformation area of the Huanglianshu landslide have more complex nonlinear relationships than the displacment data outside the deformation area; (2) machine learning approaches have a higher capacity to capture the nonlinear relationships between the time-series displacement data of slopes than the ARIMA model; and (3) among all the models, the Attention-LSTM model and the transformer model achieved the highest prediction accuracies for slope deformation, and the transformer model was better adapted to predict nonlinear landslide displacements affected by multiple factors. In the future, we will consider more factors that affect slope deformation and employ state-of-the-art deep learning approaches to address multivariable slope deformation prediction problems.