Time Series Prediction of Reservoir Bank Slope Deformation Based on Informer and InSAR: A Case Study of Dawanzi Landslide in the Baihetan Reservoir Area, China

Abstract

Reservoir impoundment significantly impacts the hydrogeological conditions of reservoir bank slopes, and bank slope deformation or destruction occurs frequently under cyclic impoundment conditions. Ground deformation prediction is crucial to the early warning system for slow-moving landslides. Deep learning methods have developed rapidly in recent years, but only a few studies are on combining deep learning and landslide warning. This paper proposes a slow-moving landslide displacement prediction method based on the Informer deep learning model. Firstly, the Sentinel-1 (S1) data are processed to obtain the cumulative displacement time-series image of the bank slope by the Small-BAseline Subset Interferometric Synthetic Aperture Radar (SBAS-InSAR) method. Then, combining data on rainfall, humidity, and horizontal and vertical distances of pixel points from the water table line, this study created a dataset with landslide displacement as the target feature. After that, this paper improves the Informer model to make it applicable to our dataset. This study chose the Dawanzi landslide in the Baihetan reservoir area, China, for validation. After training with 50-time series deformation data points, the model can predict the displacement results of 12-time series deformation data points using 12-time series multi-feature data, and compared with the monitoring values, its Mean Square Error (MSE) was 11.614. The results show that the multivariate dataset is better than the deformation univariate data in predicting the displacement in the large deformation zone of bank slopes, and our model has better complexity and prediction performance than other deep learning models. The prediction results show that among zones I–IV, where the Dawanzi Tunnel is located, significant deformation with the maximum deformation rate detected exceeding –100mm/year occurs in Zones I and III. In these two zones, the initiation of deformation relates to the drop in water level after water storage, with the deformation rate of Zone III exhibiting a stronger correlation with the change in water level. It is expected that deformation in Zone III will either remain slow or stop, while deformation in Zone I will continue at the same or a decreased rate. Our proposed method for slow-moving landslide displacement forecasting offers fast, intuitive, and economically feasible advantages. It can provide a feasible research idea for future deep learning and landslide warning research.

1. Introduction

The construction of large reservoirs in high and narrow valleys can significantly change the geo-hydro-mechanical conditions of the slopes around the reservoirs, thereby increasing the frequency of geological disasters, especially landslide disasters [1,2,3]. The reservoir area of Baihetan on the Jinsha River, as an essential representative of the high landslide hazard area, has seen the distribution of a large number of landslides in the area since 2021, seriously threatening the safety of the neighboring residents, infrastructures, and ecological environments [1,4]. The main problem in the research on predicting bank slope deformation in reservoirs is the lack of effective monitoring means and prediction methods. Currently, the monitoring of bank slope deformation mainly relies on ground monitoring data from the Global Navigation Satellite System (GNSS) and optical satellite images [5]. Although these data can provide helpful information, due to the limited temporal and spatial extent of these methods, they often cannot comprehensively, accurately, and intuitively reflect the deformation of reservoir bank slopes [6,7]. For GNSS, it can only provide ground deformation in one location and cannot monitor a wide range of research objectives. For optical remote sensing images, it cannot perform all-weather quantitative deformation monitoring on a landslide.

Interferometric synthetic aperture radar (InSAR) technology effectively solves the problem of monitoring and predicting the deformation of reservoir bank slopes because of the ability to perform large-scale quantitative ground deformation monitoring and a stable revisit period to form a continuous time series (e.g., the satellite revisit period of Sentinel-1 is 12 days). Over the past two decades, InSAR technology has made significant progress and has been widely applied to detecting and monitoring slow-moving landslides [8,9]. In detecting and monitoring reservoir bank slope deformation, the time-series InSAR (TS-InSAR) technique has demonstrated its ability to monitor surface deformation associated with water levels [10,11,12]. In addition, the time-series deformation and cumulative displacement maps of the TS-InSAR technique can record the spatial distribution of deformation, which is an essential help in studying the deformation process of reservoir bank slopes [13,14]. Many researchers have examined detecting and monitoring reservoir bank slope deformation using InSAR technology. For example, Li et al. [15] improved the post-processing method of long-time series InSAR results, conducted a study on the relationship between water level change and slope stability in the Xiluodu reservoir area, and derived the long-term evolution trend of the bank slopes in the reservoir area through the analyses. Dun et al. [16] used TS-InSAR to obtain deformation information on the bank slopes in the Xiangbiling section of the Baihetan Reservoir area. They combined it with Google Earth images to identify 14 potential landslide areas.

The importance of predicting ground deformation is self-evident in determining the bank slope development trend and deformation manner. Sentinel-1 (S1) SAR data provide a rich data source for the InSAR time series displacement predicting task [17]. At the same time, the deformation prediction model also has a relatively mature system after a long development period. Currently, prediction methods mainly include empirical models, semi-empirical models, numerical models, and traditional artificial neural network models [18,19,20,21]. Usually, traditional prediction methods are the deterministic calculation of future trends in slope movements or acceleration. Du et al. [22] used backpropagation neural network models to predict the displacement of active alluvial landslides in the Three Gorges reservoir area in China via rainfall-water level factors. However, the deformation mode of the reservoir bank slope is complex and diverse. These traditional prediction methods have various limitations and can only be applied to small-scale single landslides with clear deformation mechanisms. Therefore, traditional prediction methods do not apply to large-scale, complex research areas [18].

In recent years, with the rapid development of Graphics Processing Units (GPUs), deep learning models have shown great potential in natural language processing, computer vision, and time series forecasting. Many scholars have researched the InSAR displacement forecast. For example, Zhou et al. [23] proposed a new prediction process that uses gated recurrent units (GRUs) to split the InSAR deformation of landslides into trend displacement and cyclic displacement and then sum the predicted values of these two to obtain the predicted total displacement. They validated this method in the Three Gorges reservoir area and achieved good results. On the other hand, the Long Short-Term Memory (LSTM) network successfully solves the problems of gradient vanishing and explosion that may occur in traditional Recurrent Neural Networks (RNNs) when learning long-term dependencies. Studies have shown that using LSTM networks as the time series InSAR deformation forecasting model can improve prediction performance [24]. These research results show the potential of deep learning models in applying InSAR displacement forecast and provide an essential reference for future research. Of course, the prediction result obtained by the deep learning method is statistical, and there is no calculation of a physical model. Although existing studies have provided valuable experience, some issues still need to be solved. Firstly, the training data for these models are limited to InSAR time-series deformation data without considering other meteorological or hydrological data such as temperature, precipitation, and reservoir level. Second, these models’ ability to extract InSAR time series deformation features of bank slopes needs to be improved. In addition, the applicability of these models could be better for deformation mechanisms in which the change in reservoir level significantly affects deformation.

The Transformer model provides a practicable solution to the above problem. ChatGPT, the most talked about AI tool in the last two years, is based on the Transformer model [25,26]. The Transformer model has efficient parallel computation, self-attention, positional encoding, and other mechanisms that perform well in time series forecasting tasks [27]. Some researchers have attempted to combine Transformer and InSAR for deformation forecasting. For example, Wang et al. [28] used a modified Transformer model to predict the deformation generated by the freeze-thaw behavior of the permafrost around a salt lake. They discussed and evaluated the expansion trend around the salt lake based on the predicted deformation and the extracted expansion results. Their results show that the Transformer model performs well in time series forecasting tasks. Therefore, we can use the Transformer model for multi-featured bank slope deformation forecasting and deformation mechanism analysis.

Previous studies have focused on InSAR time series deformation forecasting using GRU or LSTM-based models. The InSAR time series deformation prediction method based on the Transformer model still needs to be improved in evaluating the trend of the reservoir bank slope. The Transformer model cannot output all the predicted values of a long series at once; instead, it needs to predict them one by one, which will affect the accuracy and efficiency of the prediction. Therefore, we chose the Informer [29] model with better performance in time series forecasting. Informer has better time and space complexity, which can substantially improve the efficiency of self-attention computation and model prediction. In this study, an Informer model-based InSAR time-series deformation prediction method for reservoir bank slopes is developed and combined with the SBAS-InSAR algorithm to compute and analyze deformations for long time-series for the Dawanzi reservoir area of Baihetan Reservoir. It is the first successful application of the Transformer-based network analysis method to the time series displacement prediction of bank slopes in the reservoir area. In addition, we also used the predicted deformation information combined with the water level change rule to analyze the deformation trend of the landslide. Our deformation prediction method can provide critical early warnings for the prevention and control of bank slope landslides and quantitatively assess the trend of deformers based on the predicted deformations, which is of guiding significance to the project.

2. Study Area and Datasets

2.1. Study Area

The study area lies on the southeastern edge of the Tibetan Plateau, specifically southwest of Sichuan and northeast of Yunnan. It is a middle mountainous area belonging to the lower reaches of the Jinsha River basin. The area’s topography is generally high in the northwest and low in the southeast, with great topographic relief. Moreover, the gullies here are developed, crisscrossed, and deeply cut, most of which are in the shape of a “V”. The Jinsha River bounds the trend of the main gorges and mountains, with the left bank orientated in a nearly north–south direction and the right bank orientated in a north–east direction. The elevation of the top of the left bank is generally around 2000 m, and the relative height difference of the terrain is about 1400 m; the elevation of the top of the right bank is generally between 2500~3000 m, and the elevation of the top of a few mountains reaches more than 3000 m. The relative height difference of the terrain reaches about 2000 m. In general, slope landform and weathering erosion types dominate the area from Heishui River to Aizigou. The road elevation of the study area is 903–916 m, located in Qiluogou Township, Ningnan County. The northern end (downstream) outlet is about 22 km away from the dam site of Baihetan Hydropower Station (Figure 1b). The regional faults developed around the study area mainly include the Daliangshan fault zone, the Zemuhe fault zone, the Xiaojiang fault zone, etc. The most close regional fault is the Daliangshan fault zone [1,30,31]. According to the geological conditions, structural conditions and macroscopic deformation characteristics of the bank slope along the Dawanzi Tunnel, the bank slope is divided into I, II, III and IV zones, a total of 4 sections, as shown in Figure 1c.

The study area lies within the subtropical monsoon climate zone, where distinct dry and wet seasons occur throughout the year. The dry season is from November to April next year, and the rainfall is scarce. The wet season is from May to October, and the precipitation is abundant, accounting for more than 80% of the annual precipitation. The warm and dry climate has an average annual temperature of 15~21°C. Jinsha River Valley is a famous dry-hot valley. The climate has obvious zonality in the vertical direction. With the increase in altitude, the temperature decreases and the precipitation increases [32].

The rock formation in the study area is composed of the middle-thick dolomite of the lower Cambrian (${Є}_1$), the thin sandstone of the middle Xiwangmiao Formation (${Є}_{2\mathrm{x}}$) and the dolomite of the upper Erdaoshui Formation (${Є}_{3\mathrm{e}}$), overlying the Quaternary ($\mathrm{Q}$). Engineering geological mapping and drilling have revealed that the overlying strata consist of colluvial deposits (${\mathrm{Q}}^{\mathrm{c}\mathrm{o}\mathrm{l}+\mathrm{d}\mathrm{l}}$) and slip deposits (${\mathrm{Q}}^{\mathrm{d}\mathrm{e}\mathrm{l}}$). The bedrock is the Middle Cambrian Xiwangmiao Formation (${Є}_{2\mathrm{x}}$) and the Upper Cambrian Erdaoshui Formation (${Є}_{3\mathrm{e}}$) [1].

2.2. Datasets

2.2.1. SAR Datasets

We acquired SAR data from the Sentinel series of SAR satellites launched by the European Space Agency in 2014. It is C-band (wavelength ~5.6 cm [33]) SAR data, known for its high sensitivity to small deformations. In this study, we collected 86 scenes of S1A SAR data with VV polarization from 3 January 2021 to 29 December 2023 to obtain the historical deformation of deformers in the Dawanzi landslide and predict its deformation trends. Unfortunately, no SAR data are available for June 2023-July 2023 in this region. The satellite has a revisit period of 12 days in the region with a line-of-sight incidence angle of around 42°. Before multilooks processing, its resolution in the radar coordinate system is 2.33m in range direction and 13.97m in azimuth direction. The multilooks coefficients are 10 and 2, respectively. After multilooks processing, we achieved a range resolution of 23.29 m and an azimuth resolution of 27.94 m. Because the deformation area mainly lies on the left bank of the Jinsha River. The descending track data have severe shadows and layovers in the target area. Therefore, this study utilizes the ascending SAR data, which offers better performance.

2.2.2. Other Datasets

The hydrological monitoring station stands at the Baihetan hydropower station. It records the water level and precipitation at 8:00 a.m. every day. The water level accuracy is at a centimeter level, and the rainfall accuracy is at 0.1 mm. This study used data from January 2021 to January 2024. Since the water storage in April 2021, the water level changed from 650 m to fluctuate from a low water level of 775 m to a high water level of 825 m (Figure 2), which significantly impacts the stability of the reservoir bank slopes.

We obtained the Digital Elevation Model (DEM) using a UAV DJI M300 RTK combined with a Zen L2 lens, and we conducted the UAV flight in October 2023. The L2 lens equipped with Light Detection And Ranging (LiDAR) emits a 905 nm electromagnetic wave, effectively penetrating the vegetation. The DEM created can better extract the proper elevation of the study area.

This study also used the fifth-generation European Centre for Medium-Range Weather Forecasts (ECMWF) reanalysis data (ERA5) [34] distributed on a 0.25° × 0.25° grid daily 2-m dew point temperature data, which is another manifestation of soil moisture.

Moderate-resolution Imaging Spectroradiometer (MODIS) is a large-scale space remote sensing instrument developed by NASA to understand global climate changes and human activities’ impact on climate [35,36]. This study used the normalized difference vegetation index (NDVI) data of the MODIS from January 2021 to January 2024. The time resolution of the data was 16 days, and the spatial resolution was 250 m × 250 m. We perform linear interpolation sampling on the corresponding date to obtain NDVI data corresponding to the same InSAR date.

3. Methodology

Figure 3 illustrates the workflow. In this study, we first preprocess the various features required for a forecast related to deformation. Subsequently, we utilized 86 scenes of S1 data and a Systeme Probatoire d’Observation de la Terre (SPOT) DEM with a resolution of 6m for SBAS processing, resulting in 86 cumulative deformation images with a reference date of 3 January 2021. The time and deformation characteristics were obtained by cropping, grid turning point, grid sampling and other operations in ArcGIS Pro 3.0. The deformation feature is the target feature in this multivariate prediction univariate task.

We extract the corresponding elevation information for each pixel using the precise DEM generated by UAV with LiDAR. Then, we obtain the water level line based on the time series water level information and DEM. By subtracting the elevation information of the pixels from the water level information over the time series, we calculate the vertical drainage distance (VDD) feature. The number of VDD features is the product of the number of pixels and the time series. We calculate the nearest distance between the pixel point and the water level line over the time series using ArcGIS Pro 3.0, which yields the horizontal drainage distance feature (HDD). The number of HDD features is the same as that of VDD. NDVI and ERA5 features are acquired by sampling each pixel’s spatial position and corresponding time series. In contrast, precipitation features are sampled over the time series and applied to all pixels.

Since the initial Informer model does not apply to our dataset, we have improved the data loading part of the model. Because the deformation results of S1 data processed by the SBAS method are volatile, we use local weighted linear regression (LOESS) for time series deformation filtering so that the data can better reflect the deformation trend. Then, we use the K-means clustering method to group pixels with different deformation characteristics. After grouping, we conduct group training to obtain the most suitable model for different deformation characteristics. After verifying the model’s accuracy, we analyze the predicted results and the deformation in the whole time series and combine other factors to predict the deformation trend of the slope.

3.1. Datasets

In this paper, we use GAMMA software (https://www.gamma-rs.ch/software, accessed on 20 June 2024) to process S1A data. We use the traditional differential InSAR (D-InSAR) technology [37,38] and the technique based on improved SBAS-InSAR to calculate the cumulative deformation since 3 January 2021 [39,40,41,42].

The interferogram pairs’ perpendicular baselines are below 300 m, so all spatial baselines are short. Some studies have shown that when using two Sentinel-1 SAR images with a time difference of more than one month, the coherence of differential interferometry images significantly reduces. In order to maximize temporal coherence and promote phase unwrapping by minimizing the deformation phase, we consider the shortest time interval, so one date data can only form interferograms with the data of the following three dates at most. Figure 4 shows the spatial perpendicular baseline of the interferograms. We selected 252 pairs of interferograms with a minimum perpendicular baseline of 0.1 m and a maximum perpendicular baseline of 272.8 m. The minimum time interval was 12 days, and the maximum time interval was 96 days. The maximum time interval is due to the lack of data in S1 in the study area in June and July 2023. We use a 10 and 2 (range direction and azimuth direction) multilooks ratio to perform multilooks processing on all Sentinel-1 interferograms.

For the NO. $j$ differential interferogram generated from the SAR image acquired at the moment of the slave image $t_A$ and the master image $t_B$ ($t_B$ > $t_A$), the interferometric phases of the pixels with azimuthal coordinates of $x$ and distance coordinates of $r$ can be written as follows:

$$\begin{array}{ll}\delta {\varphi }_j\left(x,r\right)& =\varphi \left(t_B,x,r\right)-\varphi \left(t_A,x,r\right)\\ & \approx {\displaystyle \frac{4\pi }{\lambda }}\left[d\left(t_B,x,r\right)-d\left(t_A,x,r\right)\right]+{\displaystyle \frac{4\pi }{\lambda }}{\displaystyle \frac{B_{\perp j}\mathsf{\Delta }z}{rsin\vartheta }}\\ & +[{\varphi }_{\mathrm{atm}}\left(t_B,x,r\right)-{\varphi }_{\mathrm{atm}}\left(t_A,x,r\right)]+\mathsf{\Delta }{n}_j\end{array}$$

(1)

The first term represents the deformation phase, while the second represents the residual phase that arises due to terrain errors $\mathsf{\Delta }z$. The effect of this phase depends on the perpendicular baseline $B_{\perp j}$, the azimuth and range coordinates $\left(x,r\right)$, and the observation angle $\vartheta$. The third is the influence of the atmosphere. The fourth item is the phase contribution of decorrelation and other noises. Since the time baseline of this study is short, the impact of the fourth item is small.

Precise DEM and track files can remove the terrain residual phase. We use SPOT 6 m resolution DEM as the terrain reference to calculate the differential interferogram and unwrap the phase. For the atmospheric phase, unwrapping errors often occur. Regions experiencing unwrapping errors exhibit significantly higher phase standard deviation in the time series. Therefore, spatial coherence low-pass filtering and temporal coherence high-pass filtering can screen and remove the atmospheric phase. At the same time, we also use the linear de-slope method to minimize the residual tropospheric artifacts [43].

After removing the above effects, we can simplify Equation (1) as follows:

$$\delta {\varphi }_j\left(x,r\right)={\varphi }_B\left(x,r\right)-{\varphi }_A\left(x,r\right)\approx {\displaystyle \frac{4\pi }{\lambda }}[\mathrm{d}(t_B,x,r)-\mathrm{d}(t_A,x,r)]$$

(2)

To obtain a physically meaningful settlement sequence, we express the phase in Equation (2) as the product of the average phase velocity and the time interval between two acquisition times:

$${\nu }_j={\displaystyle \frac{{\varphi }_j-{\varphi }_{j-1}}{t_j-t_{j-1}}}$$

(3)

The phase value of the $j$-interferogram can be written as follows:

$$\sum _{k=t_{A,j}+1}^{t_{B,j}}(t_k-t_{k-1}){\nu }_k=\delta {\varphi }_j$$

(4)

That is, the integral of the speed of each time period over the master and slave image time intervals. Written in matrix form as follows:

$$\overrightarrow{B}\overrightarrow{v}=\delta \overrightarrow{\varphi }$$

(5)

Equation (5) is a matrix with a shape of M × N. Since the SBAS-InSAR adopts a multi-master image strategy, the matrix $\overrightarrow{B}$ is prone to rank deficiency. By utilizing the Singular Value Decomposition (SVD) method, we can obtain the generalized inverse matrix of the matrix $\overrightarrow{B}$, enabling us to derive the minimum norm solution of the velocity vector. Subsequently, we can obtain the deformation for each period by integrating the velocity within each period. The main results are the average deformation rate and time series deformation. The deformation obtained by InSAR technology is the direction of the radar LOS. For the convenience of comparison, projection transformation is sometimes needed. Suppose that $\alpha$ is the azimuth angle of the satellite orbit trace, $\theta$ is the incident angle of the radar wave, $U_e$, $U_n$, and $U_v$ are the components of the surface deformation in the three directions of East, North and Vertical, respectively. The $d_{los}$ is the observation value of InSAR in the line of sight, then there are

$$d_{los}=-U_e\mathit{cos}\alpha \mathit{sin}\theta +U_n\mathit{sin}\mathsf{\alpha }\alpha \mathit{sin}\theta +U_v\mathit{cos}\theta$$

(6)

Assuming that the deformation is in the vertical direction $U_{horizontal}=0$, there are

$$U_v=d_{los}/\mathit{cos}\theta$$

(7)

3.2. Time Series Deformation Prediction of Informer

Using the K-means clustering algorithm, we cluster different pixel groups with similar deformation characteristics based on the cumulative deformation map of the time series of a single master image. By sampling various characteristic data according to the location of the pixels, we generate time series datasets for training, verification, and prediction of the Informer model. These comprehensive datasets include time series deformation, VDD, HDD, 2 m dew point temperature, and precipitation. We then proceed with the deformation prediction with models constructed for different deformation zones.

3.2.1. Model Architecture

Figure 5 illustrates the deformation prediction model based on the Informer architecture [29].

On the left side of the model architecture in Figure 5, the encoder receives many long sequence inputs (green series). The ProbSparse self-attention proposed by Informer replaces the normative self-attention. The blue trapezoid is a self-attention distillation operation used to extract the dominant attention and significantly reduce the network size. Layer-stacked replicas improve the robustness of the model. On the right side is the decoder. The decoder receives long sequence input, fills the target element to zero, measures the feature map’s weighted attention composition, and generatively predicts the output element (orange series).

3.2.2. Efficient Mechanism in Model

Informer addresses three issues found in classical Transformer models: Firstly, it reduces the high complexity of self-attention calculation. Secondly, it mitigates the high complexity associated with Encoder stacking. Thirdly, it improves the slow prediction speed. Specifically, the Informer employs ProbSparse Self-attention and Self-attention Distilling in the Encoder and introduces the Start token in the Decoder.

We use the time series deformation of each pixel in the InSAR deformation image as the target feature for the multivariable prediction univariate task. Additionally, we use the other features of the pixel as the input dataset. Throughout the training process, the model progressively learns the weight matrix. Subsequently, it assigns this matrix based on the training weight of the fully connected layer to derive the query vectors Q, the key vector K, and the value vector V.

The definition of the self-attention mechanism of the specification is rooted in queries vectors Q, keys vectors K and values vectors V [26]. $\mathcal{A}(\mathbf{Q},\mathbf{K},\mathbf{V})=\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}(\mathbf{Q}{\mathbf{K}}^{\top }/\sqrt{d})\mathbf{V}$, where $\mathbf{Q}\in R^{L_Q\times d}$, $\mathbf{K}\in R^{L_K\times d}$, $\mathbf{V}\in R^{L_V\times d}$ and $d$ is the input dimension. According to Tsai [44], the attention of the i-th query is defined as a kernel smoother in the probability form:

$$\mathcal{A}({\mathbf{q}}_i,\mathbf{K},\mathbf{V})=\sum _i{\displaystyle \frac{k({\mathbf{q}}_i,{\mathbf{k}}_j)}{{\displaystyle \sum _l}k({\mathbf{q}}_i,{\mathbf{k}}_l)}}{\mathbf{v}}_j=E_{p({\mathbf{k}}_j|{\mathbf{q}}_i)}[{\mathbf{v}}_j]$$

(8)

Tsai also highlighted that attention calculation requires secondary dot product calculation, leading to memory usage reaching $\mathcal{O}(L_Q{L}_K)$, where $L_Q$ represents the number of queries and vectors and $L_K$ represents the number of key vectors. This unreasonable memory usage is the bottleneck, constraining the enhancement of model prediction ability. Some studies have suggested that the self-attention probability distribution $p({\mathbf{k}}_j|{\mathbf{q}}_i)$ exhibits potential sparsity, indicating that only a few dot product results significantly influence the self-attention outcomes. Only a few inner products of query and key vectors play a pivotal role. ProbSparse Self-attention efficiently addresses this issue by calculating self-attention effectively.

In Equation (6), the degree of dispersion of $p({\mathbf{k}}_j|{\mathbf{q}}_i)$ influences the contribution of the i-th attention to the primary attention. For instance, when $p({\mathbf{k}}_j|{\mathbf{q}}_i)$ follows a uniform distribution, its $q({\mathbf{k}}_j|{\mathbf{q}}_i)=1/L_K$ distribution results in the self-attention’s calculated contribution to the value $\mathit{V}$ being insignificant. To differentiate between ‘important’ p and q, our model uses the Kullback–Leibler divergence:

$$KL(q||p)=ln\sum _{l=1}^{L_K}{e}^{{\mathbf{q}}_i{\mathbf{k}}_l^{\top }/\sqrt{d}}-{\displaystyle \frac{1}{L_K}}\sum _{j=1}^{L_K}{\mathbf{q}}_i{\mathbf{k}}_j^{\top }/\sqrt{d}-ln{L}_K$$

(9)

After deleting the constant, we can define the sparsity measure of the i-th query as follows:

$$M({\mathbf{q}}_i,\mathbf{K})=ln\sum _{j=1}^{L_K}{e}^{{\displaystyle \frac{{\mathbf{q}}_i{\mathbf{k}}_j^{\top }}{\sqrt{d}}}}-{\displaystyle \frac{1}{L_K}}\sum _{j=1}^{L_K}{\displaystyle \frac{{\mathbf{q}}_i{\mathbf{k}}_j^{\top }}{\sqrt{d}}}$$

(10)

The first term is the Log-Sum-Exp (LSE) of qi on all keys, and the second is the arithmetic mean on them.

For each query ${\mathbf{q}}_i\in R^d$ and ${\mathbf{k}}_j\in R^d$ in the key set K, there is a lower bound: $\mathit{ln}{L}_K\le M({\mathbf{q}}_i,\mathbf{K})\le \underset{j}{\mathit{max}}\{{\mathbf{q}}_i{\mathbf{k}}_j^{\top }/\sqrt{d}\}-{\displaystyle \frac{1}{L_K}}{\displaystyle \sum _{j=1}^{L_K}}\{{\mathbf{q}}_i{\mathbf{k}}_j^{\top }/\sqrt{d}\}+\mathit{ln}{L}_K$. When ${\mathbf{q}}_i\in \mathbf{K}$, it also holds. Therefore, there is

$$\overline{M}({\mathbf{q}}_i,\mathbf{K})=\underset{j}{max}\{{\displaystyle \frac{{\mathbf{q}}_i{\mathbf{k}}_j^{\top }}{\sqrt{d}}}\}-{\displaystyle \frac{1}{L_K}}\sum _{j=1}^{L_K}{\displaystyle \frac{{\mathbf{q}}_i{\mathbf{k}}_j^{\top }}{\sqrt{d}}}$$

(11)

Based on the proposed metric, ProbSparse self-attention is obtained by allowing each key to focus only on $u=c\cdot \mathit{ln}{L}_Q$ dominant queries:

$$\mathcal{A}(\mathbf{Q},\mathbf{K},\mathbf{V})=\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}({\displaystyle \frac{\overline{\mathbf{Q}}{\mathbf{K}}^{\top }}{\sqrt{d}}})\mathbf{V}$$

(12)

Through the above derivation, we find that ProbSparse self-attention only requires calculating the sub-dot product per query key lookup, maintaining the time and space complexity of each layer at $\mathcal{O}(L_K\mathit{ln}{L}_Q)$. The process and effect verification of formula derivation can be seen in the reference [29]. In summary, the ProbSparse self-attention mechanism enables the selection and computation of factors with significant influence on the results during attention calculation, thereby enhancing computational efficiency and mitigating the impact of irrelevant information on the results.

As a direct result of the ProbSparse self-attention mechanism, the encoder produces feature maps with redundant combinations of values V. Through distillation operations, the encoder significantly reduces the temporal dimension of the input, effectively addressing the challenge posed by excessively long inputs for stacking. The extraction process from layer j to layer j + 1 proceeds as follows:

$$X_{j+1}^t=MaxPool\left(ELU\left(Conv1d\left({\left[X_j^t\right]}_{AB}\right)\right)\right)$$

(13)

For the hindrance that the dynamic decoding results in too long output to make a rapid prediction, the key is to extract a shorter sequence from the input sequence. This study uses the known 6-period (72-day) data as the label length to predict the last 12-period (144-day) predicted length.

3.2.3. Training and Evaluation

The model presented in this paper uses the deep learning architecture Pytorch. We experimented on a Windows OS computer with an NVIDIA GeForce GTX 2080Ti GPU with 16 GB of RAM and a dual-core Intel (R) Xeon (R) CPU E5-2637. Using 12 multi-feature data with InSAR deformation features as the target features, the Informer model predicts the following 12 InSAR deformation features. In this study, we use multi-feature data ($x_1$,$x_2$, …,$x_{12}$) as the input of the encoder and incorporate multi-feature data ($x_7$,$x_8$, …,$x_{12}$) along with null values for 12-time serves as the input of the decoder. As a result, the final output of the decoder includes target feature data ($x_7$,$x_8$, …,$x_{12}$) and 12-time series deformation data ($x_{13}$,$x_{14}$, …,$x_{24}$).

To enhance the generalization ability and robustness of the model, we adopt data augmentation techniques in this paper. Each pixel has 86 time series deformations. However, due to lacking data, there is a time interval of 5 time series steps (60 days) between steps 74 and 75. Consequently, only 74 time steps remain for training and forecasting. We consider each 24 input (12) and output (12) length as a unit and shift the unit one step backward, thereby adding a training sample. During training, the last 12 time steps (63–74), which do not appear in the time series, are utilized for forecast and effectiveness verification. Following that, we use the subsequent time steps (75–86) to validate the accuracy of the trend confirmed by the prediction results. This process makes each pixel have 50 samples.

The training batch is 32, the number of iterations is 6, the loss function is Mean Square Error (MSE), and the activation function is Gaussian Error Linear Unit (gelu). The learning rate of initial training is 0.0001. The Adam optimizer [45] iteratively updates the learning rate. Each sublayer dropout is 0.1. The model dimension is 512, the number of multi-head attention mechanisms is 8, the number of encoder layers is 3, and the number of decoder layers is 1. The training, validation, and test sets accounted for 70%, 20% and 10%, respectively. In the reasoning process, the trained Informer is used to predict, and the normalized results are restored at the end, which is convenient for the visualization of the results.

4. Results

4.1. Time Series Deformation Result

The results in Figure 6 show the deformation rate in the study area. The right curve boundary of the deformation area near the Jinsha River corresponds to the 825 m water level line of spot 6 m dem. Warm colors signify ground deformation farther from the satellite, while cold colors indicate deformation closer to the satellite. Most areas exhibit deformation rates between −10 mm/year and 10 mm/year, with the maximum deformation rate detected around the bank slope exceeding −100 mm/year. The average deformation rate is −5.5 mm/year, with a median of −4.6 mm/year and a standard deviation of 15.8 mm.

This study primarily analyses deformation near the Dawanzi tunnel. We find two distinct deformation zones, Zone I and Zone III, within the bank slope section where the tunnel is situated. By analyzing engineering geological zoning (see Figure 6) and time-series deformation results (see Figure 7), we observe differences in the distribution of deformation rates between these zones. Zone I shows a fan-shaped deformation along the 825 m water level line. It spans approximately 660 m in length with a leading edge width of about 680 m. Higher deformation rates occur in the coastal area and the prominent part of the middle and rear terrain. In contrast, Zone III exhibits an oblate shape, measuring around 890 m in length and 490 m in width at its widest point. The elevated deformation rates concentrate on the front bank collapse and the sub-landslide area on the right front side.

The impoundment of Baihetan hydropower station commenced in April 2021 (refer to Figure 2). Within approximately three months of impoundment, it reached the current low water level of 775 m. The deformation of the bank slope beside the Jinsha River mainly starts from the data on 10 January 2022, and this period is the time when the water level is declining. Over time, the Dawanzi landslide consistently exhibits deformation away from the satellite. Analysis of Figure 2 and Figure 7 reveals a notable correlation between the water level fluctuations of the Jinsha River and the development of deformation within the landslide. As of 19 December 2023, the maximum deformation monitored by SBAS was −180 mm and 100 mm.

4.2. Time Series Deformation Prediction Results

Of the 86 data, only the first 74 can be used for model training and verification because of missing data in June–July 2023. To better show the training effect of the prediction model and verify it, we selected the data from 3 January 2021 to 5 January 2023 as the model of the training set. The data from 26 August 2022 to 5 January 2023 are used as the model’s input to obtain the prediction results from 17 January 2023 to 29 May 2023. The results are compared with the results obtained by the SBAS-InSAR, and the results are shown in Figure 8. The results generated by the Informer model are close to those obtained from the SBAS-InSAR method, exhibiting little difference to the naked eye. On 17 May 2023, the SBAS-InSAR method observed a slight blue-deepening phenomenon; however, upon comparison with results from preceding and subsequent periods, we attribute this to the influence of noise. It shows that noise has less influence on the forecasting results of our model.

To further demonstrate the predictive capabilities of our model, Figure 9c shows the cumulative deformation histogram for prediction and monitoring on 29 May 2023. We observe minimal difference between the prediction results and those obtained through the SBAS-InSAR method, especially in small deformation. Our model tends to yield more conservative results than the SBAS-InSAR method. The standard deviation of the two histograms is 0.0216, with average and median values of the model prediction results and SBAS-InSAR results being 3.3 mm and 3.6 mm, 4.7 mm and 5.3 mm, respectively. We analyze that the SBAS-InSAR method exhibits more deformation in the direction close to the sensor due to vegetation growth.

4.3. Deformation Analysis of Typical Points

To analyze the predictive capability of our model across different deformation regions, we randomly select six points with different conditions in Figure 10 and examine the deformation characteristics of each region. Among these points, points A and D are situated in Zone I, point B in Zone III, point C in the upper right bank landslide area of Zone IV, point E in Zone II, and point F in a non-deformation area distant from the Jinsha River. Figure 11 shows the line chart of time series deformation of the six points predicted by the Informer model. Landslide susceptibility is affected by various factors such as soil moisture, drainage distance, vegetation, and precipitation [46]. We compare the prediction results of univariate ablation experiments and those of the multivariate Informer model. This comparison aims to explain the impact of these factors on the prediction of reservoir bank slope deformation. The input data of the univariate ablation experiment based on the Informer model only contains time series deformation data.

Figure 11 illustrates that the multivariate data more accurately approximates the deformation curves of the study area obtained by the SBAS-InSAR method than the univariate data, a conclusion evident in Figure 11a–c,e. Models trained on univariate data exhibit an “inertia”, as shown in Figure 11a,c,d. For more straightforward trends, such as when the pre-deformation information has a linear decreasing trend, this ‘inertia‘ will cause the model to fail to change accordingly when other factors change. The corresponding representation is that the univariate training model overestimates the deformation results in Figure 11a,c, and the deformation prediction in Figure 11b is conservative; however, the effect shown in Figure 11d is similar to the model trained by multivariate data. Even though the data of 86 periods are missing for two months, we can observe from the trend shown in the last 12 periods of these figures in Figure 11 that the prediction of the deformation trend by the model trained with multivariate data is highly reliable. The difference between multivariate and univariate prediction is insignificant for the F point with small deformation and far away from Jinsha River in SBAS-InSAR results. From Figure 11f, we can also observe that the small deformation data processed by the SBAS-InSAR method for S1 data over a long time series exhibit significant volatility, demonstrating low reliability. In summary, the influencing factors we selected significantly impact the deformation of the reservoir bank slope. This impact aids in predicting deformations in large deformation areas of the bank slope but has minimal effect on non-deformation areas.

Next, we will analyze the deformation characteristics of the regions represented by these six points. In Figure 11b, we observe that even at points A-E in the great deformation zone, only small deformation occurs before the water level reaches 816m for the first time in October 2021. However, significant deformation occurs at points A–E approximately two months after the water level begins to decline. This phenomenon is particularly typical at points B and C, and the same deformation acceleration also occurs during the decline of the water level after secondly reaching the high water level from January 2023 to May 2023; it indicates that the deformation of zones III and IV is more obviously affected by the water level. Zones I, where points A and D are located, except for being more affected by the water level during the initiation of the deformation, show a tendency to slow down the deformation in the last 12 periods of the data. Considering the engineering control situation and the region’s deformation characteristics, we predict that deformation will slow down or cease when the water level drops for the third time from its peak in December 2023. The relationship between deformation and water level change at point E is unclear. It shows a lifting of the ground surface, probably related to the change in buoyancy due to the change in water level. Point F is in a state of no deformation, and the deformation data only relates to the noise and vegetation.

5. Discussion

5.1. Informer Performance on This Dataset

We conducted a comparative study to assess the impact of the ProbSparse self-attention mechanism and self-attention distillation mechanism on the training effectiveness of our dataset. The results in

Table 1. Training effects of different mechanisms.

Self-Attention Type	Distilling	MSE	MAE	Complexity
Prob	True	11.614	2.524	$\mathcal{O}(L \mathit{log} L)$ 1
Prob	False	14.296	2.82
Full	True	11.376	2.371	$\mathcal{O}(L^2)$
Full	False	15.242	2.961

¹ L is the number of vectors.

show that the ProbSparse self-attention mechanism and the self-attention mechanism in Transformer have similar effects, with nearly MSE and Mean Absolute Error (MAE) values. However, the self-attention distillation mechanism demonstrates a practical improvement in training our dataset. Furthermore, ProbSparse self-attention and self-attention distillation mechanisms exhibit excellent complexity and can predict more extended time series under similar hardware conditions [29].

We trained and predicted using LSTM, RNN, GRU, and Informer models to evaluate our method’s accuracy of time series deformation prediction. Besides the loss function MSE, we used MAE, Root Mean Squared Error (RMSE), and MAPE to evaluate each model’s performance. The Informer model predicted values closely match those obtained by the SBAS-InSAR method, with a Mean Absolute Percentage Error (MAPE) of only 1.631%, indicating that the average difference between all predicted values and actual values is 1.631%, demonstrating the effectiveness of our prediction method.

Table 2. The training results of the dataset in this paper on deep learning models.

Models	MAE	MSE	RMSE	MAPE
LSTM	4.041	25.030	5.003	24.603
RNN	4.535	34.319	5.858	11.104
GRU	6.988	64.192	8.012	10.223
Informer	2.371	11.376	3.373	1.631

shows that, for our dataset, the accuracy of the Informer model surpasses others. It suggests that the Informer model with ProbSparse self-attention mechanism performs better on sequences containing periodic fluctuation signals of water level. It can effectively handle long input sequences and achieve high deformation prediction performance at the point scale. Overall, the Informer model exhibits excellent performance and can effectively predict long-term series of surface deformation of reservoir bank slope.

This paper represents the first attempt to predict the deformation trend of the reservoir bank slope using the Transformer-based model. Due to the gradient disappearance and explosion problems in RNN or LSTM and the limitations of convolutional filters, they have limitations in modeling long-term complex relationships in sequence data. In contrast, the Informer model retains a direct link to all previous time stamps, allowing information to spread over a more extended sequence, which makes it suitable for the deformation prediction of long-time series. Moreover, our proposed method comprehensively considers various deformation factors of the bank slope, including rainfall, water level changes, drainage distance, and other information. By learning the deformation patterns of the bank slope, it achieves accurate prediction results. The time series deformation results capture essential signals of periodic changes around the bank slope, reflecting trends of sinking or rising over time due to rainfall and surface runoff effects. The prediction results depicted in Figure 11 demonstrate the method’s ability to simulate both water level-related and non-water level-related deformation signals accurately.

5.2. Limitations and Prospects

The main disadvantage of this method is that the trained results of one slope are not universal for other bank slopes. The time series features considered in the model, such as water level and drainage distance, are not the most critical factors affecting the deformation of the bank slope. It is difficult to quantitatively describe the rock formation and geological structures of pixels. Moreover, these characteristics are not time-varying. However, the location of the points in the dataset describes the factors that affect these factors. In other words, the location of the points represents the geological information of the pixel area. Since the calculation work of the time series prediction task is less than that of computer vision and natural language processing tasks, the Informer model has excellent computational complexity. So, it is practicable to recreate and train new models to analyze other bank slopes. Nevertheless, when using the dataset of another slope, it is essential to consider its richness. Finally, this method only applies to forecasting the slow-moving reservoir bank slope deformation.

Despite these limitations, there are several avenues for future research and improvements. Incorporating additional critical factors, such as geological characteristics and material properties, into the model could enhance its predictive accuracy and generalizability. Advanced techniques like transfer learning might be explored to adapt the model to different slopes without extensive retraining. Integrating more comprehensive datasets with diverse slope conditions and deformation patterns could also help create a more robust model. Future work could also focus on real-time monitoring and prediction, leveraging sensor data and Internet of Things (IoT) technology to provide more timely and precise forecasts. By addressing these challenges and expanding the model’s scope, we can significantly advance the application of time series analysis in geotechnical engineering and slope stability prediction.

6. Conclusions

This paper studies a time series prediction method for slope deformation using the Informer model, demonstrating its effectiveness in application to the Dawanzi landslide. The research proves that the model has great potential in predicting the deformation of the deformation zone of the reservoir bank slope. Based on the above research, we can draw the following conclusions:

(1)

The deformation rate in most parts of the study area is between −10 mm/year and 10 mm/year, and the maximum deformation rate detected around the bank slope is more than −100 mm/year;

(2)

Based on the Informer model, the displacement mode of the reservoir bank slope can be predicted. Compared with other existing methods, the model performs better in predicting the deformation trend of the reservoir bank slope. The RMSE and MAPE of the predicted displacement of the 144-day time series of 12 periods is 3.373 mm and 1.631%;

(3)

According to the deformation characteristics of the random points, we have the following understanding of the Dawanzi landslide: the four areas where the Dawanzi tunnel is located have obvious deformation in zones I and III. The start of the deformation of the two is related to the decline of the water level after the first impoundment. Zone III’s deformation rate strongly correlates with the water level change. We predict that the deformation in Zone III will remain slow or stop, and the deformation in Zone I will continue, but the rate will decrease.

In the future, to achieve more accurate prediction of bank slope deformation and improve the robustness of the prediction model. We will incorporate more data on the other reservoir bank slopes into the model, improve the model to consider the spatial relationship between different variables and learn the spatial and temporal relationship between deformation and other factors from the data.