6.3.1. Analysis of EALL
The dam deformation sequence is predicted using the EALL by setting up reasonable trials, and the efficacy is validated by comparing the experimental results to those of the ARIMA model, the LSTM model, and the conventional combination of ARIMA-LSTM model. The construction and prediction process of EALL is as follows:
(1) EEMD decomposition of deformation sequence: The dam deformation sequence is decomposed by EEMD. First, the parameters of EEMD are set, and then the number I of adding Gaussian white noise is set to 100, and the standard deviation of Gaussian white noise is 0.05 times. The obtained EEMD decomposition diagram is shown in
Figure 6. The original deformation sequence is decomposed into nine IMF components
, and a trend component
.
Where have high frequencies, showing irregular and fluctuating characteristics, which belong to the nonlinear part of the deformation sequence, have low frequencies and show a certain trend, so it can be considered that temperature and aging are the main factors causing the deformation of this part and therefore belong to the linear part of the sequence.
(2) Judging the frequency of deformation sequence: after the deformation sequence is decomposed into nine IMF components, the zero-crossing rate approach is used to determine the frequency of each component. According to the experience of dam deformation sequence decomposition, 10% is a suitable definition standard for high and low frequency components [
29].
This standard is used in this study to define as high frequency components, and as low frequency components.
(3) Prediction of low-frequency series by ARIMA model: ARIMA model is used to predict low-frequency series .
The other sequences are the same, and
is used as an example in this section. The sequence is first put through the ADF test. If it is determined that the sequence is unstable, the differential operation is performed. The sequence is stable after three differential operations, and the differential order
is 3. Second, the order of ARIMA model is determined, the difference order
, autoregressive order
, and moving average order
are calculated. One of them has been identified as
. AIC criterion is used to identify the parameters of
ranges, and the autocorrelation coefficient diagram (ACF) and partial autocorrelation coefficient diagram (PACF) are used to determine the values of
. The parameters
of the ARIMA model are determined as
by experiments. The model is then trained to forecast the IMF7 sequence, and its performance is assessed in accordance with the evaluation criteria. The evaluation results are shown in
Table 1.
According to
Table 1, it can be seen that the values of RMSE, MAPE, and MAE are relatively small, which indicates that ARIMA has a good prediction effect on the IMF7 sequence and meets the experimental requirements. Similarly, according to the above experimental steps,
are predicted, respectively, and the prediction results of all low-frequency series are obtained.
(4) LSTM model prediction of high-frequency series. The LSTM prediction model is constructed, and the sequence IMF3 is used as the training and prediction sequence. The specific process is as follows:
Step 1: Determine the input sequence of the LSTM model and the feature sequence composed of water level, temperature, and aging, and normalize them to construct the input dataset of the LSTM model.
Step 2: Determine the parameters of the LSTM model. Set up the LSTM’s parameters, setting to 3, to 0.001, and training step size to 10. The original random gradient descent technique is also replaced with the Adam algorithm. The training parameters for the LSTM model are determined with sets to 10 and sets to 200. If it is too small, the network will be underfitted, while if it is too large, the network will be overfitted. The model’s batch size is a crucial parameter that influences the model’s training time and the speed at which the parameters are updated.
Step 3: LSTM realizes prediction. According to the parameters determined in the previous two steps, the LSTM model is constructed and trained to obtain the prediction results of the IMF3 sequence. Similarly, the LSTM prediction result of can be obtained.
The above steps are performed on the sequences
and are predicted by ARIMA model and LSTM model, respectively, and the results obtained are shown in
Figure 7 after integration.
As can be seen from
Figure 7, different models are selected to predict different sequences after the original dam data are decomposed by EEMD and have a good performance. In order to further judge the prediction effect of the model, RMSE, MAPE, and MAE are used to evaluate each sequence, and the results are shown in
Table 2.
As can be seen from
Table 2, ARIMA and LSTM models are used to predict the original sequence after decomposition, and the overall performance is better, which gives full play to the advantages of the respective models and reduces the error of prediction.
(5) LSTM builds the final prediction result. According to the idea of combined model and the prediction process of LSTM in the previous step, the second LSTM model is constructed. The input training data are the linear ARIMA prediction results, of low-frequency sequences, the LSTM prediction results of high-frequency sequences , and the original sequence .
The historical relationship between them was learned through LSTM model, then the final predicted value
was obtained. Parameters of the LSTM model were initialized, trained, and predicted, and the final prediction result of the deformation sequence
was obtained, as shown in
Figure 8a.
In order to evaluate the feasibility and prediction effect of this model more reasonably and objectively, the ARIMA model, LSTM model, and ARIMA-LSTM model are respectively used to predict the selected dam deformation sequence, and the prediction results of the four models are compared and analyzed with the EALL model, as shown in
Figure 8b.
As seen in
Figure 8b, single-point prediction models like ARIMA and LSTM lack the ability to account for both linear and nonlinear components in the deformation sequence, resulting in a substantial error in the prediction result. The ARIMA-LSTM model adopts a combination model format, and the prediction result is obviously superior to that of a single-point prediction model. Nevertheless, because it just uses a simple linear addition method to combine the prediction results of the two models, and does not decompose the deformation sequence to forecast the deformation sequence, the prediction result is still a bit different from the actual value. It can be seen from the figure that the prediction result of the EALL prediction model proposed in this paper is closer to the original value of the deformation sequence, which is better than the three models of ARIMA, LSTM, and ARIMA-LSTM. The prediction results were evaluated by that evaluation criteria, and the results are shown in
Table 3.
It can be seen from
Table 3 that the RMSE value, MAPE value, and MAE value of the EALL proposed in this paper are lower than those of the ARIMA, LSTM, and ARIMA-LSTM models. As can be observed, the EALL predicts more accurately than the ARIMA, LSTM, and ARIMA-LSTM models. Moreover, the change regulation of the deformation sequence can be more precisely mined, so as to realize more accurate deformation prediction.
6.3.2. Analysis of IEALL
In this experiment, the comparison algorithm of IGWO is particle swarm optimization (PSO) and original gray wolf optimization (GWO). The initial population of IGWO algorithm is set to 30, and the maximum number of population iterations is 500. Due to the optimization process of the algorithm, there is a certain randomness that makes the results inevitably have some deviations. In this experiment, all the algorithms are run 10 times, and the average of 10 times is the final result of the experiment, which can reduce the deviation caused by randomness and evaluate the performance of the algorithm more accurately.
Experiment 1: Test and comparison of IGWO optimization performance.
In order to prove that the IGWO algorithm proposed in this paper has stronger optimization performance than the classical optimization algorithm, this experiment is based on the Sphere function, Rastrigin function, and Grienwank function in the benchmark function, the results are compared and analyzed by using PSO algorithm, GWO algorithm, and IGWO algorithm, respectively. It is mainly evaluated from two aspects: global optimization accuracy and stability of the algorithm.
(1) Analysis and comparison of global optimization accuracy
In the experiment, PSO, GWO, and IGWO algorithms were used to optimize the Sphere function, Rastrigin function, and Grienwank function, respectively. The results of 10 experiments and averaging are shown in
Table 4.
According to
Table 4, the IGWO algorithm found the optimal solution 0 after 10 experiments under the test of single-peak benchmark function Sphere. Therefore, under the test of unimodal benchmark function, IGWO optimization effect is better than PSO and GWO algorithm. The results of IGWO are all equal to 0 or the closest to 0 under the tests of multi-peak benchmark functions Rastrigin and Grienwank. Therefore, under the multi-peak benchmark function, IGWO can successfully jump out of the local optimal solution to find the global optimal solution, and the optimization effect is better than PSO and GWO algorithms. In general, IGWO algorithm has better global optimization performance than PSO and GWO under different benchmark functions.
(2) Stability analysis and comparison of algorithms
Algorithm stability is also an important evaluation criterion for optimization algorithms. The mean square error results of 10 times of the above three models are shown in
Table 5.
It can be seen from
Table 5 that under the unimodal benchmark function Sphere test, IGWO algorithm found the optimal solution in 10 trials, and the mean square error was 0. Therefore, under the unimodal benchmark function test, IGWO optimization stability is better than PSO and GWO algorithm. Under the multi-modal benchmark functions, Rastrigin and Grienwank test, the mean square error results of IGWO are smaller than those of PSO and GWO algorithms. In general, under different benchmark functions, IGWO algorithm has more stable optimization effects than PSO and GWO.
The benchmark function optimization test results of the IGWO, PSO, and GWO algorithms were compared, and it was found that the IGWO method had higher convergence accuracy, faster convergence speed, and stronger stability than the PSO and GWO algorithms. As a consequence, the comprehensive optimization performance was higher, which serves as a useful foundation for the second experiment.
Experiment 2: Test and verification of IEALL optimization performance.
The IGWO algorithm is used to optimize the LSTM model parameters in the EALL. Then, in order to verify whether the optimized parameters are optimal, the IEALL model is constructed, constantly changing the current parameters and bringing them into the model for prediction while keeping other parameters constant, recording the RMSE values of the model when each parameter takes different values. The parameters to be optimized are , , , and , etc. In order to verify the impact of the optimized parameters on the model accuracy, the above parameters are experimented and analyzed one by one. Firstly, , , , and are initialized, then IGWO is used to optimize each parameter. The specific experimental procedure is as follows.
(1) The , , , and are optimized by the IGWO algorithm. Where is set to 10 and the range is set to [10, 200], is set to 1 and the range is set to [1, 100], is set to 2 and the range is set to [2, 50], is 0.001 and the range is set to [0.001–0.01], IGWO is used to optimize the current parameters within the parameter range whille keeping the other three parameters unchanged. The experimental results show that , , , and , which are the selected parameters of EALL in this section.
(2) To verify the effectiveness of the parameters after the IGWO optimization, the trend of the RMSE values of the predicted results of the EALL under the change of the current parameters is calculated by keeping the other three parameters unchanged. The experimental results are shown in
Figure 9.
As can be seen from
Figure 9a, when other parameters remain unchanged, the RMSE value of EALL model first decreases with the increase of training times. When
, the model RMSE value reaches the minimum value. However, with the continuous increase of training times, the model slowly enters the state of overfitting, and the RMSE value gradually increases.
Batch size refers to the number of samples fed into the network training at each time. When it is too small, the convergence speed is slow and less stable, and it is easy to fall into the partial optimum, while if the batch size is too large, it is easy to cause problems such as missing the nadir during gradient descent and thus reducing the accuracy of the model. As can be seen from
Figure 9b, the minimum value is obtained when
.
The number of hidden layer units also has a certain influence on the prediction results, it can be seen from
Figure 9c that as the number of hidden layers increases, the internal structure of the model becomes increasingly complex, which is prone to problems such as overfitting and long training time, then leading to gradual increase in the RMSE of the model; the RMSE value of the model is the smallest when
.
Learning rate determines whether the target can converge to the global minimum and the convergence speed. When the learning rate is too small, the convergence process is slow, while if it is too high will lead to overfitting or failure to converge. As can be seen from
Figure 9d, the overall effect of the learning rate on the prediction accuracy decreases first and then improves, and the RMSE reaches the minimum value at
.
It can be concluded through the above verification experiments that the IGWO algorithm can effectively optimize the parameter of the EALL, thereby improving the predictive accuracy of the model. Each reaches the optimum when , , , , which are also set as the optimal parameters obtained after IGWO optimization.
Experiment 3: IEALL prediction results and comparison.
The optimal parameters obtained after IGWO optimization are brought into the EALL to obtain the prediction results of the IEALL model. The results are compared to the model with the EALL and shown in
Figure 10.
As can be seen from the figure, the predicted values of the IEALL model are closer to the actual values and the predicted effect is better. The results of the evaluation of the prediction effects of the two models are shown in
Table 6.
From
Table 6, it can be seen that the RMSE, MAPE, and MAE values of the IEALL model are closer to 0 than EALL, which can be judged that the IEALL model is more effective in dam deformation series prediction compared to EALL.
6.3.3. Analysis of STAGCN-IEALL
- (1)
Parameter setting
The basic parameters of the experiment are time dimension is 2409, the number of nodes is 5, the dimension of measurement point features is 5, and the prediction time is 230. The hyperparameters are set mainly by grid search to determine the optimal parameters, keeping the other hyperparameters constant when determining a parameter. The is taken from the set [8, 16, 32, 64, 128, 256], is taken from the set [0.0001, 0.0005, 0.001, 0.005, 0.01], the optimizer is selected from [Adam, SGD, Adagrad]. Finally, the results are obtained through multiple experiments on the verification set. The selected is 128, is 0.001, the optimizer selects Adam. At the same time, all results are taken as the average of multiple experimental results to reduce the experimental error.
- (2)
Baseline methods
In order to more comprehensively verify the advantages of the method proposed in this section in predicting the dam deformation sequence, the following prediction methods are used for comparative experiments, as follows:
ARIMA-LSTM: The dam deformation prediction model combining autoregressive moving average model and long short-term memory network has stronger prediction ability than a single model.
EALL: The algorithm proposed in
Section 4, based on the common dam deformation combination model, decomposes the deformation sequence through EEMD, and gives full play to the respective prediction advantages of ARIMA and LSTM models. Finally, the relationship between the two predicted values is constructed through LSTM model, and the final predicted value is obtained.
IEALL: Based on EALL, IGWO is proposed to optimize two LSTM networks in EALL, so that the LSTM networks can reach the best training parameters, thus improving the prediction accuracy.
ST-ARIMA [
21]: Temporal-spatial autoregressive moving average model. By constructing the spatial weight matrix and constructing the spatio-temporal stationary sequence by using the methods of adjacency weighting and inverse distance weighting, good prediction results have been achieved in the multi-station prediction.
FC-LSTM [
30]: A prediction model with fully connected LSTM layers is used in both encoder and decoder.
DCRNN [
31]: Diffusion convolutional recurrent neural network, which uses random walk on graph to model the prediction task, has achieved good prediction results.
ST-GCN [
32]: Spatio-temporal graph convolutional neural networks, which combine graph convolutional layers and convolutional sequences to make predictions.
Among them, the first three methods are single measuring point prediction methods only considering time series, and the last four methods are multi-measuring point prediction methods considering spatial correlation.
- (3)
Analysis of experimental results
Experiment 1: Prediction results and analysis of each model.
The deformation series are predicted by the above seven models as well as the STAGCN-IEALL model proposed in this paper, where the single measurement point prediction model uses the single measurement point data and the multi measurement point prediction model uses the deformation data combining the spatial distribution of multiple measurement points to predict the value of a measurement point for 230 days. Through a large number of experiments, the average results of 10 experiments are taken to minimize the error caused by randomness, and the final results are shown in
Table 7.
It can be seen from
Table 7 that the prediction results of single measurement point prediction models such as ARIMA-LSTM, EALL, and IEALL are poor, because these three models ignore spatial features. As ST-ARIMA, FC-LSTM, and DCRNN take into account the spatial association of multiple measurement points, the prediction effect of the three models is better than the above three one-sided point models. ST-GCN uses graph convolution model to fully mine the spatial correlation between multiple measurement points and achieves better results than traditional spatio-temporal models. STAGCN-IEALL proposed in this paper uses GCN to extract spatial features and IEALL to extract temporal features, and on this basis a spatio-temporal attention mechanism is introduced to fully consider the dynamic changes between measurement points, so as to ensure that the model can learn the dynamic correlation between different measurement points at any time and the dynamic correlation dependence of any measurement point at different times.
Compared with the comparison model, the values of RMSE, MAE, and MAPE are the closest to 0, which are 16.06%, 14.72%, and 21.19% lower than those of ST-GCN, respectively. Therefore, the prediction performance of STAGCN-IEALL model proposed in this paper is better.
Experiment 2: Comparative analysis of predictive results under different prediction lengths of each model.
In order to more comprehensively verify the prediction performance and stability of the dam deformation spatio-temporal prediction model based on the STAGCN-IEALL, this experiment uses the above eight models to predict the deformation series under different prediction durations, continuously changing the value of the time dimension
in the model and at the same time calculating the prediction performance under the three indicators of different models, then the results are analyzed. The experimental results are shown in
Figure 11.
It can be seen from
Figure 11 that for the three single-point prediction models, the IEALL model has the best prediction effect. Because the single measuring point prediction model does not consider the spatial correlation between measuring points, the prediction performance gradually decreases with the increase of the prediction time. The four traditional spatio-temporal prediction models, ST-ARIMA, FC-LSTM, DCRNN and ST-GCN, have better prediction performance than the single-point model because of considering the spatial relationship of multiple measurement points. With the increase of prediction time, the prediction performance decreases, but the decline rate is lower than that of the single-point prediction model. It shows that the spatio-temporal prediction model considering spatial factors is more suitable for long-term prediction deformation tasks. STAGCN-IEALL proposed in this paper not only considers the spatio-temporal correlation, but also fully considers the dynamic changes between the measurement points to ensure that the model can learn the dynamic correlation between different measurement points at any time and the dynamic correlation dependence of any measurement point at different times, so as to improve the prediction accuracy of the model. It can be seen from the figure that with the increase of the prediction time, the prediction accuracy of the model is improved. It has always maintained the lowest RMSE, MAE, and MAPE values, and the growth rate is relatively stable. In conclusion, STAGCN-IEALL model is superior to other models.
Experiment 3: Intuitive comparison of prediction effect.
In order to observe the predicted results and true values of the model and the performance of the comparison model more intuitively,
Figure 12 shows the predicted performance of three models: STAGCN-IEALL, IEALL, and ST-GCN when the prediction duration is set to 230:
As can be seen from
Figure 12, the STAGCN-IEALL model proposed in this paper is better than the IEALL model and the classic spatio-temporal prediction model ST-GCN, which can accurately predict the trend of dam deformation and effectively reduce the prediction error.
Experiment 4: Analysis of ablation study.
The STAGCN-IEALL model proposed in this paper is made up of four major components: a spatial attention mechanism, a temporal attention mechanism, a spatial feature extraction module, and a temporal feature extraction module. This section removes a part from the STAGCN-IEALL model in turn, and tests the performance of each model through experiments, in order to further analyze the effectiveness of each module. The following are variant models: (1)-SA: Removing spatial attention module; (2)-TA: the temporal attention module is removed on the basis of model (1); (3)-GCN: the GCN spatial feature extraction module is removed on the basis of (2); (4)-IEALL: based on the model (3), the IEALL time feature extraction module is removed, which is a common encoder-decoder model.
The STAGCN-IEALL model’s prediction table, after each module has been removed one at a time, is shown in
Figure 13. Calculations show that the RMSE, MAPE, and MAE of the encoder-decoder model are lowered by 8.43%, 8.12%, and 11.86%, respectively, after the addition of the IEALL time feature extraction module. The prediction accuracy of the model is effectively improved after the time dimension features were successfully extracted using the IEALL model. On this basis, the RMSE, MAPE, and MAE of the model are decreased by 15.97%, 17.38%, and 22.19%, respectively. It can be concluded that GCN’s spatial feature extraction of deformation data from multiple measurement points improves the model’s spatial dimension expression capacity. The model’s predictability has significantly increased. Due to the temporal attention module, the three indexes of the model are lowered by 6.12%, 9.37%, and 12.39%, respectively. This shows that by taking into account the dynamic correlation between various moments, the model’s prediction accuracy can be significantly increased. Finally, the three indexes increased by 7.78%, 5.93%, and 7.91% after the spatial attention module was implemented. The dynamic correlation of different measurement points was considered by the spatial attention module, which effectively improved the prediction performance. Therefore, it can be concluded that the cooperation between the modules of STAGCN-IEALL model effectively improves the prediction accuracy of the model and achieves more accurate dam deformation prediction.