A Hybrid Time Series Model for Predicting the Displacement of High Slope in the Loess Plateau Region

Abstract

The physical and mechanical properties of the loess differ from other kinds of soil due to its collapsibility, which has resulted in the complex displacement development law of the loess slope. Therefore, the accurate estimation of the displacement of high slopes in a loess gully region is critical for the safety of people and in construction activities. In the present study, to improve the accuracy of traditional methods, the original cumulative displacement curve was decomposed into trend and fluctuation terms using Empirical Mode Decomposition (EMD) and Wavelet Decomposition (WD). Subsequently, the results were estimated using the Support Vector Machine (SVR) and Long Short-Term Memory Network (LSTM) optimized by Biogeography-based Optimization (BBO), respectively. To select the most appropriate model, SVR, LSTM, EMD-SVR-LSTM, EMD-BBO-SVR-LSTM, and WD-BBO-SVR-LSTM were employed to predict the deformation of a loess slope in the Loess Plateau of China. According to the results, the displacement increases rapidly at the starting stage, and then gradually stabilizes, which is the same as the trend in reality. On comparing the predicted results with field data, it was found that the models with decomposition algorithms achieved higher accuracy. Particularly, the determination coefficient of the EMD-BBO-SVR-LSTM model reaches 0.928, which has better algorithm stability and prediction accuracy than other models. In this study, the decomposition algorithm was applied to the loess slope displacement innovatively, and the appropriate machine learning algorithm adopted for the displacement components. The method improves the accuracy of prediction and provides a new idea for instability warning of loess excavation slopes. The research has implications for urban construction and sustainable development in loess mountainous areas.

1. Introduction

The loess is a kind of widespread stratum in China. With the development of urbanization in the Loess Plateau region, the high side slopes caused by deep excavation and high fill are frequently encountered in highways, bridges, and architectural engineering. The failure of high slopes in the deep excavation or high fill during construction is becoming more and more prominent [1,2,3]. The high slope threatens the safety of buildings/structures, constructors, and surrounding villages. Therefore, the establishment of displacement prediction and early warning methods for high loess slopes is of great significance for the development and construction of the loess gully region [4,5,6,7].

Before the 1980s, most of the prediction methods were proposed based on field monitoring, subjective judgment of experts, and empirical formulas. For example, the three-stage empirical model proposed by Saito in 1968 can be used to predict the landslide through the displacement time curve at the accelerated deformation stage of the slope [8]. Since then, Hoek [9], Hayashi [10], Stevenson [11], Federico [12], etc. extended the model one after another. Among them, Miao et al. [13] proposed an improved four-stage Saito curve based on the Saito method, which can be used to estimate the slope displacement. With the development of modern mathematical theory, many statistical analysis models based on probability theory, fuzzy mathematics theory, and grey system theory have been used to predict slope deformation. For instance, Rice et al. [14] established a prediction equation based on the landslide information in California and proposed a linear discriminant function. Li et al. [15] and Liu et al. [16] introduced the improved grey models (GM) (1,1) of grey system theory to estimate slope deformation. Hayashi et al. [17] calculated the displacement rate based on the experimental and the site data and plotted the analysis chart. Then, the displacement rate-time equations for different stages were established and used to predict slope displacement. However, the research focus of the above methods is mainly on the prediction models, ignoring the processing of monitoring data and the characteristics of slope deformation. Therefore, the accuracy of traditional models is relatively poor due to the high complexity and nonlinearity of the field data [18,19].

In recent years, as Machine Learning (ML) techniques advance by leaps and bounds, many intelligent algorithms, such as Artificial Neural Network (ANN), Support Vector Machine (SVM), and Long Short-Term Memory Network (LSTM), have been applied to civil engineering [20,21,22,23,24]. In particular, many researchers have used data mining techniques such as ML algorithms for slope displacement prediction due to their powerful nonlinear processing capabilities. For example, Gade et al. [25] developed a hybrid model combining ANN and Newmark method for slope displacement prediction and achieved a good prediction effect. Although the application of ANN improves prediction accuracy, the shortcomings are also obvious. Specifically, the network structure is difficult to establish, and the model parameters are difficult to determine. Therefore, SVM without the above disadvantages is widely used in slope displacement prediction due to the advantages of small sample requirements and strong robustness. Liu et al. [26] predicted the slope displacement by SVM, and the results showed that SVM has higher accuracy than ANN. Considering the difficulty of hyperparameter selection, Liu et al. [27] used SVM optimized by Particle Swarm Optimization (PSO) to develop a slope displacement prediction model, and the prediction results showed that the optimization of hyperparameters can improve the model performance. In addition, due to the poor generalization ability caused by the selection of SVM parameters, Cai et al. [28] proposed the displacement prediction model of the left bank slope of Xiluodu Hydropower Station on the Jinsha River based on Wavelet Decomposition (WD) and Genetic Algorithm-Least Squares SVM (GA-LSSVM).

According to the time series analysis theory, Du et al. [29] decomposed the slope displacements into trend and period terms for prediction by the binomial regression model and Back Propagation Neural Network (BPNN), respectively. The predictions indicated the hybrid ML model achieved high accuracy. Zhang et al. [30] separated the displacement into the trend term and periodic term, then the two terms were fitted by the least square method and PSO-SVR respectively. The final results, which were superimposed from the above two terms, showed that the accuracy of the coupled model based on time series and PSO-SVR is significantly higher than that of Grid Search (GV)-SVR and BPNN. In addition, LSTM with the “long-time memory” function has also achieved high accuracy in slope displacement prediction. Zhang et al. [31] developed an algorithm based on LSTM and obtained a better prediction effect by adjusting the parameters of the network structure, such as the learning rate, the iterations, and the number of hidden layer neurons in the process of prediction. Xie et al. [32] decomposed the displacement of Laowuji landslide into trend and periodic components. The trend component is predicted by empirical mode decomposition and the periodic component is predicted by LSTM. The results show that the hybrid model has a better prediction effect than the traditional mechanical model.

Although the research on slope displacement prediction based on ML algorithms has made great progress, there has been a lack of study on displacement prediction for loess slope using ML [33,34]. Moreover, the most commonly used algorithms for displacement decomposition based on signal decomposition theory include Empirical Mode Decomposition (EMD) and Wavelet Decomposition (WD) [35,36], but the comparison of the effects of the two algorithms on the loess slopes requires further study. In addition, the use of suitable prediction models for different displacement components is still unclear.

The purpose of this paper was to propose an effective prediction model of loess slope displacement to achieve accurate prediction after excavation, so as to avoid potential slope instability disasters and provide a reference basis for early warning. Therefore, in the present research, the optimized algorithms named “EMD-SVR-LSTM”, “EMD-BBO-SVR-LSTM”, and “WD-BBO-SVR-LSTM” were proposed. The displacement obtained from the field was firstly decomposed into trend and fluctuation terms using EMD and WD techniques, and then the two terms were predicted by SVR and LSTM, which were optimized by Biogeography-based Optimization (BBO). In addition, SVR, LSTM, and EMD-SVR-LSTM were established. Finally, the deep excavated loess slope of a logistics park in Shaanxi, China was investigated as a case study to analyze the prediction results of different models. Five evaluation indices (MAE, RMSE, R², VAF, PI) were selected to assess the precision of the models and determine the optimal one.

2. Methodology

As ML techniques advance, many intelligent algorithms have been used to predict slope displacements, such as Grey Model, ANN, SVM, etc. ML algorithms have good learning ability, which can better identify the nonlinear characteristics of displacement to predict accurately. However, due to the complexity of the displacement changes, the accuracy of the prediction results directly using the above algorithms is low. Therefore, this research selects the decomposition algorithms based on signal decomposition theory (i.e., EMD and WD) to decompose the original displacement based on previous studies [36,37]. In addition, since the characteristics of fluctuation and trend displacements are different, appropriate algorithms are required to improve the prediction accuracy, respectively. In summary, SVR and LSTM optimized by BBO are used to predict the trend and fluctuation terms respectively, thereby establishing EMD-BBO-SVR-LSTM and WD-BBO-SVR-LSTM for displacement prediction of loess slope.

2.1. Support Vector Machine (SVM)

SVM is a supervised learning algorithm established by Vapnik [38,39]. The theoretical basis of SVM is the Vapnik–Chervonenkis dimension theory and the structural risk minimization criterion. The calculation process of SVM can be summarized as follows: (i) extracting the support vector located on the class boundary; (ii) constructing the optimal separating hyperplane which can minimize the probability of misclassification of data points through the support vector; (iii) splitting the original sample based on the margin maximization principle using the optimal classification hyperplane; (iv) transforming into equal-order minimization problem when solving the margin maximization problem, and obtaining its duality problem by the Lagrange multiplier method; (v) solving the duality problem based on Kuhn–Tucker conditions, and obtaining the final decision function.

Support Vector Regression (SVR) is obtained on the basis of further extension of SVM. The method is to convert SVR problems into SVM problems [40,41,42]. When SVR is used for linear regression, the decision function is as follows:

$$f(x)=y=wx+b$$

(1)

where w is the coefficient vector, and b is a constant. For the features of the samples, the corresponding y is obtained through the above equation.

When SVR is used for nonlinear regression, there will be other constraints in Equation (1). Therefore, the slack variable ${\zeta }_i$ needs to be introduced to relax the constraints and transform the optimal hyperplane problem.

$$\underset{w,b,\zeta ,{\zeta }^{\ast }}{\min }\frac{1}{2}{\Vert w\Vert }^2+C{\displaystyle \sum _{i=1}^m\left({\zeta }_i+{\zeta }_i^{\ast }\right)}$$

(2)

$$s.t.\left(w\cdot x_i\right)+b-y_i\le \epsilon +{\zeta }_i, i=1,2,3,\cdots ,m$$

(3)

$$y_i-b-(w\cdot x_i)\le \epsilon +{\zeta }_i, i=1,2,3,\cdots ,m$$

(4)

$${\zeta }_i,{\zeta }_i^{*}\ge 0,i=1,2,3,\cdots ,m$$

(5)

According to the SVM derivation process, the duality problem is derived using the Lagrange multiplier method for the optimal solution, and then the kernel function is introduced to transform the problem into the following equation:

$$K\left(x,x^{’}\right)=\left(\theta \left(x\right)\cdot \theta \left(x^{’}\right)\right)$$

(6)

$$\underset{\alpha ,{\alpha }^{\ast }}{\min }\frac{1}{2}{\displaystyle \sum _{i=1,j=1}^m\left({\alpha }_i^{\ast }-{\alpha }_i\right)}\left({\alpha }_j^{\ast }-{\alpha }_j\right)K\left(x_i\cdot x_j\right)+\epsilon {\displaystyle \sum _{i=1}^m\left({\alpha }_i+{\alpha }_i^{\ast }\right)}-{\displaystyle \sum _{i=1}^m{y}_i\left({\alpha }_i^{\ast }-{\alpha }_i\right)}$$

(7)

$$s.t.{\displaystyle \sum _{i=1}^m\left({\alpha }_i^{\ast }-{\alpha }_i\right)}=0$$

(8)

$$0\le {\alpha }_i,{\alpha }_i^{*}\le C,i=1,2,3,\cdots ,m$$

(9)

Equations (7)–(9) give the general expression of SVR, which is solved by the Sequential Minimal Optimization (SMO) method. The decision function is obtained as follows:

$$\begin{array}{cc}\tilde{y}\hfill & =\tilde{w}x+\tilde{b}\hfill \\ \hfill & ={\displaystyle \sum _{i=1}^m\left({\tilde{\alpha }}_i^{\ast }-{\tilde{\alpha }}_i\right)}\left(\theta \left(x_i\right)\cdot \theta \left(x\right)\right)+\tilde{b}\hfill \\ \hfill & ={\displaystyle \sum _{i=1}^m\left({\tilde{\alpha }}_i^{\ast }-{\tilde{\alpha }}_i\right)}K\left(x_i,x\right)+\tilde{b}\hfill \end{array}$$

(10)

Considering the characteristics of the samples used, the Radial Basis Function (RBF) is chosen as the kernel function of the SVR. The hyperparameters of RBF are C and γ. For different problems, it is necessary to search for optimal hyperparameters to obtain the best performance. The commonest way is to use intelligent optimization algorithms for automatic optimization. Accordingly, the BBO is used to search for the best combination of parameters.

2.2. Long Short-Term Memory Network (LSTM)

LSTM is an improved Recurrent Neural Network (RNN) algorithm proposed by Hochereiter and Schimdhuber [43] in 1997, which solves the long-term dependency problem encountered in traditional RNN. LSTM modifies the hidden layer structure of the network for long-term memory, and the main improvement of LSTM is the introduction of cell states and the proposed gate mechanism, i.e., forget gate, input gate, and output gate. LSTM can update cell states through gates [37]. In the first step, outdated information needs to be dropped from the cell state when new information is entered. This step is controlled by the forget gate layer. Like RNN, LSTM also uses the output gate layer, where h_t is the output at time t, σ is the sigmoid activation function, W is the weight matrix, b is the bias vector, and the forget gate equation is as given below:

$$f_t=\sigma \left(W_f\cdot \left[h_{t-1},x_t\right]+b_f\right)$$

(11)

The second step discusses the new information stored in the cell state through the following equation:

$$f_t=\sigma \left(W_f\cdot \left[h_{t-1},x_t\right]+b_f\right)$$

(12)

$${\tilde{C}}_t=\tanh \left(W_C\cdot \left[h_{t-1},x_t\right]+b_C\right)$$

(13)

where ${\tilde{C}}_t$ is a candidate vector derived from the tanh layer.

The third step is to renew the cell state. The formula is given below:

$$C_t=f_t\ast C_{t-1}+i_t\ast {\tilde{C}}_t$$

(14)

The final step determines the output of the LSTM, in which the output of the network is controlled by the cell state and needs to be “filtered” by the output gate:

$$o_t=\sigma \left(W_o\left[h_{t-1},x_t\right]+b_o\right)$$

(15)

$$h_t=o_t\ast \tanh \left(C_t\right)$$

(16)

The LSTM modifies the hidden layer by decoupling the hidden states from the cell state through the gate mechanism which allows the model to increase its storage capacity and selectively store and forget information for long-term memory purposes. Generally, the performance of the LSTM algorithm is mainly influenced by two hyperparameters: the number of hidden layers and the learning rate [44]. To improve the estimation accuracy, the optimal hyperparameters are searched using BBO.

2.3. Optimization of the Prediction Models

For solving different problems, ML models have different hyper-parameters. The purpose of hyper-parameter optimization is to find the optimal value that makes the model perform best on the sample data. Common optimization techniques include grid search, stochastic search, Bayesian optimization, and intelligent optimization algorithms. Because hyperparameter optimization has some complicated problems, for example, since the relationship between hyperparameters and the final evaluation index is not necessarily linear, it is not possible to fix all parameters and optimize one alone. In addition, there are many combinations of different hyperparameters, and the search is computationally intensive and time costly. Consequently, intelligent optimization algorithms are more effective in hyperparameter optimization due to the advantages of high efficiency, large search space, and avoidance of local optima [45]. In the present research, BBO was selected to optimize SVR and LSTM. Since the slope displacements are a combination of trend displacements and nonlinear displacements, the trend and fluctuation terms decomposed from the original displacements by the decomposition algorithm provide better predictions than the original data directly substituted. In this section, the original displacement is decomposed using EMD and WD, and then SVR and LSTM, optimized by BBO, are used to estimate the two displacement components respectively, and finally the EMD-BBO-SVR-LSTM and WD-BBO-SVR-LSTM are established.

2.3.1. Hyperparameter Optimization Using BBO

Inspired by biogeography, Simon [46] put forward the BBO algorithm by revealing the behavioral patterns of species migration between habitats. The algorithm solves the optimization problem by simulating the migration of species inside and outside the habitat. A habitat is defined as an island that is geographically isolated from other areas. As a feasible solution to the problem, it can be represented by a set of integers. Each solution is regarded as a habitat with a Habitat Suitability Index (HSI). HSI is the dependent variable of the habitat, which is the same as the fitness in other swarm intelligence optimization algorithms. Habitats that are more suitable for species to live in have higher HSI. The influencing factor of HSI is the Suitability Index Variables (SIVs), which represent the decision variables in the candidate solution (e.g., temperature, rainfall, species, etc.). The higher the SIV of the habitat, the higher is HIS [47,48,49].

Migration behavior is a core step of BBO. High-HSI habitats prefer to share their SIVs with low-HSI habitats (emigration), while low-HSI habitats accept many new SIVs from high-HSI habitats (immigration). Migration is probabilistic and is used to share characteristic information among habitats. Figure 1 illustrates a linear pattern of habitat species distribution. S₀ represents the equilibrium between the emigration rate and the immigration rate [50,51]. The relationship between HSI and the immigration rate (λ) and the emigration rate (μ) is expressed by the following formulas:

$${\lambda }_i=I(1-\frac{S_i}{S_{\max }})$$

(17)

$${\mu }_i=E(\frac{S_i}{S_{\max }})$$

(18)

where I is the maximum immigration rate, E is the maximum emigration rate, and S is the number of species.

The mutation is another core step of BBO, which traverses all SIVs in the habitat, and randomly selects and replaces them with a new SIV through the mutation rate m_i. The mutation will bring new features to the habitats, which can prevent the model from being trapped into a local optimal solution. The mutation rate is inversely proportional to the habitat probability, which can be expressed as follows:

$$m_i=m_{\max }\left(1-p_i/p_{\max }\right)$$

(19)

where m_i is the mutation rate, m_max is a custom value, and p represents the habitat probability. $i=1,2,3,\cdots ,n$, n is habitat quantity. The algorithm flow of BBO is illustrated in Figure 2.

2.3.2. EMD-BBO-SVR-LSTM

Because the slope displacement is generated and developed under the combined action of internal geotechnical mechanical properties and external inducing factors, the cumulative displacement can be decomposed into deterministic trend term displacement caused by its geotechnical conditions and nonlinear fluctuation term displacement caused by external factors. Therefore, it is difficult to obtain more accurate results by directly substituting the original data into the prediction models. In this study, the commonly used EMD algorithm is used for the original cumulative displacement decomposition to acquire the trend and fluctuation terms. In addition, since trend term usually exhibits monotonically increasing properties, prediction is more accurate using SVR, while LSTM can better reflect the periodic changes of fluctuating terms. Accordingly, the BBO-SVR and BBO-LSTM algorithms are applied to estimate the two displacement components respectively, and the final results are obtained by superimposing the two parts of displacement.

EMD is a data-driven technology proposed by N.E Huang [52]. The method can decompose a complicated signal into a limited number of Intrinsic Mode Functions (IMFs), and the IMFs contain the local feature signals of the primitive signal at different time scales. Different from wavelet decomposition and Fourier transform, EMD does not pre-set the basis function and self-adaptively decomposes into several IMFs based on its characteristic time scale [53]. In addition, several IMFs will be obtained after EMD, including the instantaneous frequency and other important information in the original complex non-stationary signal [54,55].

The primary purpose of the EMD technique is to decompose a nonlinear and non-stationary series into a limited number of IMFs and a trend term [56,57]. The steps are as follows:

(i) Obtain all maximum points and all minimum points of the primitive series x(t), and fit the upper and lower envelopes using the cubic spline interpolation function, and then take the average of the upper and lower envelopes as $m_1(t)$.

(ii) Subtract from the primitive series to obtain the new series.

$$h(t)=x(t)-m_1(t)$$

(20)

If $h(t)$ meets the IMF conditions, the first IMF $c_1(t)$ is obtained. Otherwise, $h(t)$ is considered as a new series, and the above steps are repeated until the IMF condition is satisfied to obtain the first IMF component. The remaining part is expressed as:

$$r_1(t)=x(t)-c_1(t)$$

(21)

(iii) Regarding $r_1(t)$ as a new series, the above decomposition is repeated until the residual series in the stage $n$ is monotonous or less than the preset value then the process ends. The remaining series is $r_n(t)$, which represents the overall trend of the original series. The original series can be represented as:

$$x(t)={\displaystyle \sum _{i=1}^n{c}_i(t)+r_n(t)}$$

(22)

The formula for the termination condition of the decomposition process is [44]:

$$tc={\displaystyle \sum _{i=1}^T\left[\frac{{\left|h_{k-1}(t)-h_k(t)\right|}^2}{h_{k-1}^2(t)}\right]},k=1,2,\cdots$$

(23)

In this research, the cumulative displacement is decomposed into several IMF components and a residual component by EMD. The remaining part represents the trend of the original series, which is named the trend term. Then the IMFs are accumulated and reconstructed into a new fluctuation component, which is named the fluctuation term. After the above decomposition and reconstruction, the original displacement is divided into trend and fluctuation terms. Subsequently, the trend and fluctuation terms are estimated using BBO-SVR and BBO-LSTM, respectively, and the ultimate estimated displacement is acquired by superimposing the prediction results of different components at the same moment [35]. The flow of the hybrid algorithm is illustrated in Figure 3.

2.3.3. WD-BBO-SVR-LSTM

Wavelet Transform (WT) proposed by J. Morlet (1974) is a time-frequency analysis technique that evolved and improved on the basis of the Fourier transform [58,59]. Based on the localization idea of the short-time Fourier transform, it overcomes the drawback of constant window size and provides a “time-frequency” window that varies with frequency. WT is a multi-scale local transform that can be used to obtain local frequency information of a signal. If the frequency information resolution of the WT at the current scale does not satisfy the requirements, a larger-scale WT can be performed on the signal to obtain higher-resolution frequency information [60,61,62].

The basic functions of the Fourier transform are sine and cosine functions with infinite length and no attenuation. WT transforms the sine function and cosine function in Fourier transform into the wavelet function with finite length, attenuation, and orthogonality. The limited length means that it has the feature of a “window”. Meanwhile, the length of the “window” can be adjusted through the dilation of the wavelet function, which can obtain any local frequency information in the original signal, because any length of the “window” can be obtained through the dilation of the wavelet function. The formula of WT can be expressed as below:

$$WT(a,\tau )=\frac{1}{\sqrt{a}}{\displaystyle {\int }_{-\infty }^{+\infty }f(t)\ast \psi (\frac{t-\tau }{a})dt}$$

(24)

where * represents convolution, $\psi (t)$ represents wavelet function, $\psi (\frac{t-\tau }{a})$ is the wavelet basis function obtained by dilation and translation of wavelet function. The independent variables are $a$ and $\tau$, $a$ is the degree of dilation, $\tau$ is the translation distance [63].

WT of a signal is the process of fitting appropriate coefficients to a signal using a wavelet basis function. The coefficients obtained by WT can be used to reconstruct the original signal, and the fitting accuracy of WT can be measured by the error of the reconstructed signal. WD is often used to process discrete signals. The coefficients are obtained by WD and then combined with the wavelet basis function, and the signal component with the same length as the original signal is obtained. The low-frequency part and high-frequency part of the WD contain the approximate information and detailed information of the primitive signal, respectively.

To compare the applicability of EMD and WD in the displacement decomposition of loess slopes, EMD-BBO-SVR-LSTM and WD-BBO-SVR-LSTM are established respectively [64]. Similar to the EMD, WD is also used to decompose the displacement curve into trend term (low-frequency part) and fluctuation term (high-frequency part) [65]. In this way, the same strategy can be used to predict the two displacements and then superimposing into the final results. The flowchart of WD-BBO-SVR-LSTM is illustrated in Figure 4.

3. A Case Study of a Loess Slope in China

In order to compare the advantages and disadvantages of the above models more intuitively, this section selects a loess slope located in the Loess plateau area of Shaanxi Province, China, for a case study. The displacement monitoring data of 70 days after the slope excavation are substituted into the prediction model for time series prediction to obtain the displacement prediction results. The optimal model is judged by analyzing the prediction effects of various models and the quantitative comparison of the evaluation indices.

3.1. Introduction of the Slope Case

For the purpose of verifying the effect of the models on predicting the displacement of the loess slope, a deep excavation loess slope at the construction site of a logistics park in Tongchuan, Shaanxi, China was selected as a case study. The construction site is about 756 m long from east to west and 605 m wide from north to south, with an area of about 4.57 × 105 m². The site is inclined to the south-east. The maximum and minimum elevation of the loess slope is 992.83 m and 951.51 m, respectively. The elevation difference reaches 71.3 m. According to the field engineering geological survey, the formation lithology in this area is mainly Quaternary Holocene compacted fill (Q₄^ml), Quaternary Late Pleistocene Malan loess(Q₃^eol) and paleosol (Q₃^el), Quaternary Middle Pleistocene Lishi loess (Q₂^eol) and paleosol (Q₂^el), Quaternary Early Pleistocene alluvial silty clay (Q₁^al). In addition, no groundwater was found within the depth of 80 m of on-site in situ drilling. As shown in Figure 5, a high slope with a height of 30 m and a length of 420 m in the north–south direction was formed after excavation at the project site. Below the slope is the planned logistics park, and above the slope is a village. A large free surface was formed after the excavation. Therefore, to prevent the sudden instability of the slope, a displacement monitoring device was set above the slope for real-time displacement. Then the displacement change law of the slope was analyzed through the monitor data which also provides samples for predicting displacement development using the ML algorithm.

3.2. Data Preparation and Evaluation Indices

In order to establish a time series prediction model, the displacement monitor data of 70 days at the monitoring site were collected, and the recorded data used to analyze and predict the future displacement trend. The data used are shown in Figure 6. Sixty early samples were selected to predict the displacement data in the next 10 periods.

For the practical purpose of rapid discrimination of the slope stability state, it is necessary to perform slope displacement time series prediction. In this study, the first sixty samples in the slope displacement datasets were used as learning samples according to the common methods. The displacement samples of 10 days were used to predict the displacement on the 11th day. By analogy, the model is trained to establish a time series prediction model. The 61–70 displacement monitor value was used as the test sample, and the optimal prediction model obtained through the comparison of model accuracy evaluation indicators.

Many indices are applied to assess the reliability of predictive models. The representative indices in the ML algorithm include the Mean Absolute Error (MAE), the Determination Coefficient (R²), the Root Mean Square Error (RMSE), and the Variance Accounted For (VAF) [66]. In theory, the optimal values of an ideal model are 0, 1, 0 and 100% for MAE, R², RMSE, and VAF respectively [67,68]. However, it is inappropriate to determine the best model solely from these three indicators. Yagiz put forward a Performance Index (PI) to comprehensively consider three indicators [69]. In this research, MAE, RMSE, R², VAF, and PI were selected as the model performance evaluation index. The calculation formulas of the above indices are as follows:

$$\mathrm{MAE}=\frac{1}{m}{\displaystyle \sum _{i=1}^m\left|y_i-{\widehat{y}}_i\right|}$$

(25)

$$\mathrm{RMSE}=\sqrt{\frac{1}{m}{\displaystyle \sum _{i=1}^m{\left(y_i-{\widehat{y}}_i\right)}^2}}$$

(26)

$${\mathrm{R}}^2=1-\frac{{\displaystyle \sum _{i=1}^m{(y_i-{\widehat{y}}_i)}^2}}{{\displaystyle \sum _{i=1}^m{(y_i-\overline{y})}^2}}$$

(27)

$$\mathrm{VAF}=\left[1-\frac{\mathrm{var}(y_i-{\widehat{y}}_i)}{\mathrm{var}(y_i)}\right]\times 100$$

(28)

$$\mathrm{PI}=[{\mathrm{R}}^2+(\mathrm{VAF}/100)-\mathrm{RMSE}]$$

(29)

where m represents the number of samples, $y_i$ and ${\widehat{y}}_i$ denote the monitored and predicted values respectively, $\overline{y}$ is the average value of monitor data.

4. Results and Discussion

To establish a high-accuracy prediction model for estimating the displacement of high loess slopes, the original displacement is decomposed into trend term and fluctuation term based on EMD and WD, and then the SVR and LSTM, optimized by BBO, are applied to estimate the displacement of trend and fluctuation terms respectively. The ultimate estimated displacement results are acquired by superimposing the results of the two components. To validate the prediction effect of the hybrid algorithm, a deep excavation high slope located in the Loess Plateau of China was selected for a case study. This section compares the applicability of the two decomposition algorithms through a case study and determines the optimal loess slope displacement prediction model by comparing the results of several models.

4.1. The Decomposition of the Slope Displacement by EMD and WD

The samples in this study are the field monitoring displacement data of a high loess slope. To build an effective hybrid model for the displacement estimation of the loess slope, the monitoring displacement of the first 70 days was selected. However, since the direct prediction of the original displacements was poor, it is reasonable to decompose the original data into finite components by EMD and WD respectively. Afterwards, the decomposed components are combined into fluctuation term and trend term for prediction, respectively. The purpose of using the two decomposition algorithms is to compare the prediction results and choose a more appropriate method for the displacement prediction of the loess slope.

As shown in Figure 7, the measured displacement is decomposed by the EMD method to obtain IMF components (IMF01~IMF03) and the residual term. The IMF components have strong volatility and clear peaks and troughs, indicating that different components have their specific frequencies. All components show the law of gradually stable fluctuation. Moreover, the residual part called the trend term displacement represents the degree of stability of the series and presents the general trend of the primitive series change. As can be seen from the EMD results, the decomposition achieves two purposes. On the one hand, the “riding wave” is eliminated, the local oscillation of the original displacement curve is weakened, the basic oscillation trend is retained, and the whole waveform is simpler. On the other hand, the displacement component is made into a waveform in which the local maxima and minima are symmetrical to the mean value as the axis 0. In EMD, the original displacement can be obtained by superimposing all components, indicating that the method is recoverable and reversible, which proves the completeness of the EMD.

As shown in Figure 8, the new curve obtained by superimposing all IMF components is called the fluctuating term displacement, which is part of the perturbation term in the original signal, making the originals complex, irregular, and unpredictable. From the Figure 8, it can be seen that the trend term displacement after excluding the fluctuation part shows to be smooth and monotonic overall, which is the same development trend as the original displacement curve and represents the displacement change pattern. It also shows that under the original geotechnical conditions, the displacement change of the excavated slope is relatively regular and monotonically increasing.

Figure 9 exhibits the decomposition curve of actual displacement changes by WD. In WD, the Symlet wavelet basis function is used, and the amount of decomposition levels has a value of 5. As can be seen from the figures, the components D1–D5 show the local fluctuations of the original signal. The overall fluctuations are stronger and the frequency is higher. Yet the fluctuations are rather chaotic, and the peak and valley value changes are irregular, which increases the difficulty of forecasting. In addition, after the separation operation, the remaining low-frequency term (the trend term) increases smoothly and monotonically in an approximate parabola shape, which also largely matches the original displacement variation. This decomposition result also shows that the low-frequency parts of the WD are similar to the original trend, and the high-frequency parts represent the details of the local disturbance.

Figure 10 contrasts the original displacement and the fluctuation and trend terms decomposed by WD. Compared with the results of EMD, the variation range of the fluctuation term is larger, namely WD (−1.48~1.14) and EMD (−0.83~0.70), indicating that more local interference has been eliminated. Although that makes the trend term smoother, it can also be seen that the curve has slightly deviated from the original curve, resulting in the distortion of the trend term. This is in obvious contrast to the synchronization of the EMD trend curve with the original curve, which is further explained by the prediction results below. In addition, the periodicity of fluctuating term changes in WD is worse than that in EMD, which is not conducive to model training and reduces the generalization of the model [70].

4.2. Comparison of Prediction Results

In the light of the common data processing method, the displacement in the first 10 days is selected to predict the displacement in the 11th day, and so on. All data can be divided into 60 groups, so there are 11 displacement values in each group. Similar to an ordinary ML prediction model database, the first 10 values and the 11th value of each group can be considered as input parameters and output parameter respectively. Then, 1–50 groups of samples are used as the training set to train the established models, and the last 10 groups are used as the testing set to verify the model’s accuracy. Different from ordinary prediction models, time series prediction models need to divide the training and test sets in chronological order, and cannot be divided randomly. Otherwise, the displacement later in time will be used to predict the displacement earlier in time, which contradicts the actual situation.

The testing set is employed to validate the accuracy of the model after it has been trained. The comparison between the predicted results and the actual data before and after displacement decomposition is shown in Figure 11. Before displacement decomposition, SVR and LSTM are used separately for prediction. As can be seen from the Figure 11, the prediction results of SVR are closer to the real displacement variation compared with LSTM. However, more accurate prediction results were obtained when the displacement was decomposed into trend and fluctuation terms using EMD and then predicted by SVR and LSTM respectively. It is obvious from the Figure 11 that the precision of the hybrid algorithm is superior to that of the ordinary SVR and LSTM. Specifically, the maximum error between the estimated displacement and the real displacement of the SVR is 0.23, and the maximum error of LSTM is 0.34, while the maximum error of EMD-SVR-LSTM is 0.16. The estimated value of EMD-SVR-LSTM is close to the change law of actual displacement, and the error also decreases [71,72]. As shown in Figure 11, the estimated results of the hybrid model reflect the development of the actual displacement, which indicates that the estimation accuracy can be increased by decomposing the displacement. The reason is that the change law of different displacement terms is more obvious and easier to predict after the displacement is decomposed.

Appropriate hyperparameters can improve the performance of ML models. In the present research, the BBO technique is often applied to search for the optimal hyperparameters of SVR and LSTM. Generally, the hyperparameters of SVR with RBF kernel function are C and γ, which are taken to be 13 and 0.01 respectively after the optimization. The hyperparameters of LSTM (the number of hidden layers n, the learning rate v) are eventually taken to be 11 and 0.007 respectively. After displacement decomposition by EMD, the comparison between the predicted results and the actual displacement before and after hyperparameter optimization is shown in Figure 12. For the trend term displacement, the predictions of BBO-SVR are similar to the actual displacement in Figure 12a. In Figure 12b, the prediction accuracy of the fluctuation term is improved obviously after the hyperparameter adjustment of LSTM. Similarly, the comparison of prediction results based on WD before and after hyperparameters optimization is represented in Figure 13. It is clear from the Figure 12 and Figure 13 that the prediction accuracy of the model is improved for both trend and fluctuation terms after hyperparameter optimization.

The estimation results of the two hybrid algorithms (EMD-BBO-SVR-LSTM and WD-BBO-SVR-LSTM) on the whole time series are shown in Figure 14. Both hybrid models were able to accurately estimate the slope displacement development process. To determine which model is more accurate, Figure 14 illustrates the performance of the hybrid algorithms on the testing set. It is obvious that the prediction result of EMD-BBO-SVR-LSTM is more accurate. In terms of error, the maximum error of EMD-BBO-SVR-LSTM on the testing set is 0.092, while that of WD-BBO-SVR-LSTM is 0.195. Therefore, the EMD-BBO-SVR-LSTM model is more appropriate for the displacement prediction of the loess slope. To further compare the prediction accuracy of several models, evaluation indices (MAE, RMSE, R², VAF, PI) are calculated below.

The regression results of predicted values and monitoring values of the hybrid models on the testing set are illustrated in Figure 15. As can be seen from the Figure 15, the results of the hybrid models are close to y = x, which indicates that there is little error between the predicted results and the measured values. In particular, the prediction results of the EMD-BBO-SVR-LSTM are basically distributed around y = x, which proves that the predicted value has a strong correlation with the actual value. In addition, the MAE, RMSE, R², VAF, and PI of the EMD-BBO-SVR-LSTM model are 0.058, 0.081, 0.887, 82.40, and 1.630 respectively, which is closer to the criteria of an ideal model compared to WD-BBO-SVR-LSTM.

To compare the advantages and disadvantages of the several models mentioned above more clearly, the five evaluation indices are shown in

Table 1. Evaluation indices of different prediction models.

	MAE	RMSE	R2	VAF	PI
SVR	0.075	0.105	0.794	67.69	1.366
LSTM	0.083	0.116	0.751	65.75	1.293
EMD-SVR-LSTM	0.063	0.092	0.845	78.46	1.538
WD-BBO-SVR-LSTM	0.058	0.081	0.887	82.40	1.630
EMD-BBO-SVR-LSTM	0.050	0.074	0.928	89.48	1.749

. It can be seen that EMD-BBO-SVR-LSTM is the most accurate model according to the evaluation indices. To display the indices more intuitively, they are normalized and plotted as a radar chart in Figure 16. Generally, the larger R², VAF, and PI, the smaller RMSE and MAE, the higher is the model accuracy. It is obvious that the results of EMD-BBO-SVR-LSTM are optimal in five aspects among the established models. Most importantly, the accuracy of the models after displacement decomposition is significantly improved according to the indices. It is shown that the method of decomposing the displacement into fluctuation term and trend term is effective in the problem of slope displacement prediction.

Although the models are effective in predicting slope displacement, there are still many limitations. For instance, the developed models belong to the time series prediction model. If a nonlinear trend is generated in the data series, the prediction results have a hysteresis. In addition, the developed models use a database derived from a certain loess excavation slope, which still needs to be improved if extended to general slopes.

5. Conclusions

In the present research, the optimized prediction models were established based on decomposition algorithms, i.e., EMD-BBO-SVR-LSTM and WD-BBO-SVR-LSTM. Then, models were employed to predict the displacement of the loess slopes in the Loess Plateau region of China. The effectiveness of these models was verified by comparing the estimation results of different models, and the optimal model was also determined.

Because the cumulative displacement of slope is generated and developed under the joint action of geotechnical conditions and external inducing factors, this research used the time series decomposition algorithm to decompose the cumulative displacement into trend and fluctuations terms. SVR is more suitable for the trend displacement which is basically monotonous growth. LSTM is more suitable for the fluctuation displacement of periodic changes. Different machine learning algorithms were used for different displacement components, which can better exploit the prediction performance of the algorithms. In addition, BBO was used to search the optimal hyperparameters of SVR and LSTM which can overcome the poor prediction accuracy caused by the randomness of parameters to a greater extent. The developed model has two advantages. On the one hand, it decomposes the cumulative displacement into the more predictable components, and on the other hand, it use SVR and LSTM which have been searched for the optimal hyperparameters to make the prediction more accurate and reasonable. This method not only avoids the large error caused by direct prediction of cumulative displacement, but also optimizes SVR and LSTM to exploit their optimal performance. The accuracy of the model is of great value for the actual slope instability problem.

In the case study, the displacement of the loess slopes was predicted by SVR, LSTM, EMD-SVR-LSTM, EMD-BBO-SVR-LSTM, and WD-BBO-SVR-LSTM, respectively. By analyzing the decomposition results of the two algorithms, namely EMD and WD, it was found that the components of EMD were more conducive to the model prediction. In addition, from the model performance on the testing set, it could be seen that the estimation accuracy was significantly improved by decomposing the displacement and predicting the trend term and fluctuation term with SVR and LSTM, respectively. It also indicated that the decomposition components of EMD are more appropriate than that of WD for predicting the variation of the actual displacement. Given the above, EMD-BBO-SVR-LSTM is the best one of the developed models, to be used to assist the early warning of slope instability with the advantages of high precision and strong robustness.

There are many factors affecting slope deformation, such as geological structure, groundwater, rainfall, etc., which were not considered in this research. More external factors should be introduced into the prediction model to further improve the accuracy of displacement prediction. Moreover, the ML algorithm requires a large number of samples for training to improve the model performance. A database of loess slope displacements could be considered in the future.