Slope Deformation Prediction Combining Particle Swarm Optimization-Based Fractional-Order Grey Model and K-Means Clustering

Abstract

Slope deformation poses significant risks to infrastructure, ecosystems, and human safety, making early and accurate predictions essential for mitigating slope failures and landslides. In this study, we propose a novel approach that integrates a fractional-order grey model (FOGM) with particle swarm optimization (PSO) to determine the optimal fractional order, thereby enhancing the model’s accuracy, even with limited and fluctuating data. Additionally, we employ a k-means clustering technique to account for both temporal and spatial variations in multi-point monitoring data, which improves the model’s ability to capture the relationships between monitoring points and increases prediction relevance. The model was validated using displacement data collected from 12 monitoring points on a slope located in Qinghai Province near the Yellow River, China. The results demonstrate that the proposed model outperforms the traditional statistical model and artificial neural networks, achieving a significantly higher coefficient of determination $R^2$ up to 0.9998 for some monitoring points. Our findings highlight that the model maintains robust performance even when confronted with data of varying quality—a notable advantage over conventional approaches that typically struggle under such conditions. Overall, the proposed model offers a robust and data-efficient solution for slope deformation prediction, providing substantial potential for early warning systems and risk management.

1. Introduction

Slope deformation is a critical factor influencing the stability of hillsides, often leading to slope failures and landslides, which pose significant risks to infrastructure, ecosystems, and human lives [1]. Early and accurate prediction of slope deformation is therefore essential for mitigating potential slope failures and landslides, enabling timely intervention and improving safety measures in vulnerable areas [2]. Over the years, numerous methods have been explored to predict slope deformation, ranging from traditional numerical simulations and analytical models to hybrid models that combine different techniques [3,4,5,6]. These approaches have provided valuable insights into the behavior of slopes under various conditions.

Developing a deformation prediction model based on monitoring data allows for forecasts grounded in actual conditions that directly reflect real-world conditions, offering significant advantages over traditional numerical simulations and analytical methods [7,8,9]. Compared to structures like dams and bridges, slope monitoring data are often limited, making reliable predictions more challenging. Engineering structures are valuable assets, justifying higher monitoring costs, which enables the installation of more precise and frequent monitoring equipment [10]. Furthermore, these structures are typically more accessible, allowing for easier maintenance and timely repairs of monitoring equipment. In contrast, slope monitoring devices are harder to reach, and repairs are less frequent, leading to potential data gaps and less reliable monitoring results, highlighting the need for more efficient modeling techniques capable of handling limited data [11].

Grey models have proven advantageous in handling time series prediction with limited data, making them particularly suitable for slope deformation prediction, where data scarcity often challenges traditional modeling methods [12,13]. The fractional-order grey model (FOGM) offers greater flexibility and accuracy in capturing underlying trends in limited datasets compared to the standard grey model [14,15]. However, one critical challenge is determining the optimal fractional order $\alpha$ for the model, as an inappropriate choice can significantly degrade the model’s performance, leading to inaccurate predictions and compromising its reliability in real-world applications [16,17]. While most machine learning techniques, such as support vector machines or deep learning models, require large datasets to function effectively and can be computationally expensive, they are less suitable for problems with scarce data or highly non-linear search spaces [18,19,20,21]. Additionally, gradient descent-based methods like those used in neural networks can easily become trapped in local minima, particularly in non-convex optimization problems, which may make them ineffective for optimizing the fractional order in grey models [22,23,24]. On the other hand, for example, genetic algorithms are computationally intensive and often require a considerable amount of time to converge to an optimal solution, especially in complex and high-dimensional optimization problems [25]. In this study, we use particle swarm optimization (PSO) to optimize the fractional order $\alpha$ for the FOGM, which is particularly suited for this task due to its ability to efficiently search large, multidimensional spaces for optimal solutions. The PSO model does not rely on gradient-based methods, which can struggle with non-linear and complex objective functions, making it a robust choice for this optimization task [26,27].

In addition to optimizing the grey model, this study also emphasizes the need for effective data clustering when predicting deformation from multi-point monitoring data [28]. Given the spatial–temporal variations in the monitoring points, clustering offers a way to group similar points and account for their interrelationships [29]. To address this, we apply k-means clustering to classify monitoring points based on their temporal data characteristics [30]. This clustering step enables us to capture the temporal and spatial patterns within the monitoring data. Once the monitoring points have been clustered, we apply the PSO-based FOGM to each cluster, improving the accuracy and relevance of the predictions for each group.

The contributions of this study are twofold: (1) it introduces a PSO-optimized fractional-order grey model for predicting slope deformation, and (2) it provides a data clustering approach that accounts for temporal and spatial dependencies using k-means clustering, enhancing the predictive power of the model. The combination of these techniques offers a promising solution for accurate, data-efficient slope deformation prediction. This article is organized as follows: Section 2 describes the development of the model, including the PSO-based fractional-order grey model and k-means clustering. Section 3 presents the case study as well as the dataset used in this study. Section 4 discusses the results, covering both the clustering outcomes and prediction results. Section 5 evaluates the advantages and limitations of the proposed model. Finally, Section 6 summarizes the key findings of this study.

2. Model Development

2.1. Particle Swarm Optimization-Based Fractional-Order Grey Model

In this section, we present the frame of the particle swarm optimization (PSO)-based fractional-order grey model (FOGM) for predicting the displacement of monitoring data on a slope. We first establish the FOGM, which incorporates grey system theory and fractional calculus to capture the dynamic behaviors of slope displacement. We then use PSO to optimize the fractional order $\alpha$ within the FOGM, improving the model’s predictive performance. Figure 1 shows the flow chart of the PSO-FOGM combined model.

The FOGM is a predictive modeling technique within grey system theory, which enhances the traditional grey model by introducing fractional-order accumulation, thereby allowing the model to better capture complex, non-linear, and long-memory dynamics in time series data.

Consider time series displacement data $\left\{\delta \right(1),\delta (2),\dots ,\delta (n\left)\right\}$, where $\delta \left(k\right)$ represents the displacement at time k. Fractional accumulation of order $\alpha$ is defined as follows:

$${\mathbf{\Delta }}^{\left(\alpha \right)}\left(k\right)=\sum _{i=1}^k\left({\displaystyle \genfrac{}{}{0pt}{}{k-i+\alpha -1}{\alpha -1}}\right)\delta \left(i\right)$$

(1)

where $\mathbf{\Delta }$ represents the fractional derivative and $\left({\displaystyle \genfrac{}{}{0pt}{}{k-i+\alpha -1}{\alpha -1}}\right)$ is the binomial coefficient given by

$$\left({\displaystyle \genfrac{}{}{0pt}{}{k-i+\alpha -1}{\alpha -1}}\right)={\displaystyle \frac{\Gamma (k-i+\alpha )}{\Gamma \left(\alpha \right)\Gamma (k-i+1)}}$$

(2)

where $\Gamma (·)$ is the Gamma function.

The FOGM is defined by the following first-order differential equation:

$${\displaystyle \frac{d{\mathbf{\Delta }}^{\left(\alpha \right)}\left(k\right)}{dk}}+a{\mathbf{\Delta }}^{\left(\alpha \right)}\left(k\right)=b$$

(3)

where a and b are coefficients to be determined. The solution to this differential equation is assumed to be of the form

$${\mathbf{\Delta }}^{\left(\alpha \right)}\left(k\right)=C{e}^{-ak}+{\displaystyle \frac{b}{a}}$$

(4)

where C is a constant determined by the initial conditions.

Using the initial condition ${\mathbf{\Delta }}^{\left(\alpha \right)}\left(1\right)=\delta \left(1\right)$, we find C as follows:

$$C=\delta \left(1\right)-{\displaystyle \frac{b}{a}}$$

(5)

Thus, the complete solution is

$${\mathbf{\Delta }}^{\left(\alpha \right)}\left(k\right)=\left(\delta \left(1\right)-{\displaystyle \frac{b}{a}}\right)e^{-ak}+{\displaystyle \frac{b}{a}}$$

(6)

The predicted value $\widehat{\delta }\left(k\right)$ is obtained by applying the inverse fractional accumulation to ${\mathbf{\Delta }}^{\left(\alpha \right)}\left(k\right)$:

$$\widehat{\delta }\left(k\right)={\mathbf{\Delta }}^{\left(\alpha \right)}\left(k\right)-{\mathbf{\Delta }}^{\left(\alpha \right)}(k-1)$$

(7)

Parameters a and b are estimated using the least squares method. Define the error function E as

$$E=\sum _{k=2}^n{\left(\delta \left(k\right)-\widehat{\delta }\left(k\right)\right)}^2$$

(8)

The values of a and b that minimize E are taken as the optimal parameters for the model.

Particle swarm optimization (PSO) is a heuristic optimization algorithm inspired by the social behavior of birds [31,32]. Here, we use the PSO model to optimize the fractional order $\alpha$ in the FOGM. The objective function for PSO is the Mean Squared Error (MSE) between the actual values $\delta \left(k\right)$ and the predicted values $\widehat{\delta }\left(k\right)$:

$$\mathrm{MSE}={\displaystyle \frac{1}{n}}\sum _{k=1}^n{(\delta \left(k\right)-\widehat{\delta }\left(k\right))}^2$$

(9)

where n is the amount of data, and k counts the number of data sequences. Then, define the search space for the fractional order $\alpha$:

$${\alpha }_{\min }\le \alpha \le {\alpha }_{\max }$$

(10)

where ${\alpha }_{\min }$ is the lower limit and ${\alpha }_{\max }$ is the upper limit.

Then, generate initial particles $\{{\alpha }_1\left(0\right),{\alpha }_2\left(0\right),\dots ,{\alpha }_m\left(0\right)\}$ randomly within $[{\alpha }_{\min },{\alpha }_{\max }]$. Define the initial velocities $\{v_1\left(0\right),v_2\left(0\right),\dots ,v_m\left(0\right)\}$ randomly. Define the PSO parameters, including the inertia weight $\omega$, the cognitive learning factor $c_1$, the social learning factor $c_2$, and the maximum number of iterations. The update velocity can be written as follows:

$$v_i(t+1)=\omega v_i\left(t\right)+c_1{r}_1(p_{i,\mathrm{best}}-{\alpha }_i\left(t\right))+c_2{r}_2(g_{\mathrm{best}}-{\alpha }_i\left(t\right))$$

(11)

where $v_i\left(t\right)$ is the velocity of particle i at iteration t, $\omega$ is the inertia weight, $c_1$ denotes the cognitive factor, $c_2$ denotes the social factor, $r_1,r_2$ are random numbers between 0 and 1, $p_{i,\mathrm{best}}$ is the personal best position of particle i, and $g_{\mathrm{best}}$ is the global best position. Ensuring ${\alpha }_i(t+1)$ remains within $[{\alpha }_{\min },{\alpha }_{\max }]$, the update position can be expressed as follows:

$${\alpha }_i(t+1)={\alpha }_i\left(t\right)+v_i(t+1)$$

(12)

Update $p_{i,\mathrm{best}}$ and $g_{\mathrm{best}}$ if the current fitness is better than the previous one:

$$p_{i,\mathrm{best}}={\mathrm{argmin}}_{{\delta }_i}\mathrm{MSE}\left({\delta }_i\right)$$

(13)

$$g_{\mathrm{best}}={\mathrm{argmin}}_{{\delta }_i}\mathrm{MSE}\left({\delta }_i\right)$$

(14)

The iteration is terminated once one of the following conditions is reached: maximum number of iterations is reached; MSE falls below a predefined threshold; or minimal change in the fitness value across iterations.

The optimal fractional order $\alpha$ is given by

$${\alpha }_{\mathrm{optimal}}=g_{\mathrm{best}}$$

(15)

where $g_{\mathrm{best}}$ is the global best position found during the optimization process.

2.2. K-Means Clustering Method

The primary goal of clustering the dataset is to group similar wave characteristics together, allowing us to generate predictive models for each cluster. To provide comprehensive insights into the displacement behavior of the slope, three clustering criteria are selected: the coefficient of determination of the statistical model $R_s^2$, altitude z, and displacement difference between the initial and final time of the dataset $\Delta \delta$.

• $R_s^2$ reflects the goodness of fit of the statistical model used to predict displacement, ensuring that points with more reliable and predictable displacement trends are grouped together.
• z accounts for the topographical variations on the slope, which can significantly influence deformation due to factors like soil composition and water infiltration.
• $\Delta \delta$ captures the rate and intensity of displacement changes over time, helping to distinguish between regions with steady movements and those experiencing more dynamic shifts.

Together, these three criteria enable a robust clustering approach that improves the prediction of displacement by considering statistical, topographical, and temporal factors that affect slope stability.

As a clustering method, we use the K-means method. Figure 2 illustrates the flow chart of the K-means clustering algorithm. The process starts by defining the number of clusters, K, and initializing the centroids. The next step involves calculating the distance between each data point and the centroids [33,34]. Each data point is then assigned to the centroid that is closest, based on this distance metric. Once the initial assignment is complete, the algorithm checks for any changes in the group membership of the data points. If no data points have switched clusters, the algorithm terminates, as it has reached convergence. If any points have moved to a different group, the centroids are updated by recalculating their positions based on the mean of the points assigned to each cluster. This process is repeated iteratively, with the points being reassigned and the centroids recalculated until the algorithm converges, meaning no further changes in cluster membership occur.

The mathematical details are as follows:

We first select the K centroids ${\mu }_1,{\mu }_2,\cdots ,{\mu }_K$ using the K-means+ initialization technique in the 3-dimensional (3D) space formed by the parameters $R_s^2$, z, and $\Delta \delta$. Each centroid can be represented using a 3D vector:

$${\mu }_k=({\mu }_{k1},{\mu }_{k2},{\mu }_{k3})$$

(16)

where ${\mu }_{k1}$, ${\mu }_{k2}$, and ${\mu }_{k3}$ are the coordinates of centroid ${\mu }_k$ for parameters $R_s^2$, z, and $\Delta \delta$, respectively.

For each data point $x_i=(R_{si}^2,z_i,\Delta {\delta }_i)$, assign it to the nearest centroid ${\mu }_k=({\mu }_{k1},{\mu }_{k2},{\mu }_{k3})$ based on the Euclidean distance. The assignment rule is as follows:

$$c_i=arg\underset{k}{min}{\parallel (R_{si}^2,z_i,\Delta {\delta }_i)-({\mu }_{k1},{\mu }_{k2},{\mu }_{k3})\parallel }^2$$

(17)

where $\parallel (R_{si}^2,z_i,\Delta {\delta }_i)-({\mu }_{k1},{\mu }_{k2},{\mu }_{k3}){\parallel }^2$ is the squared Euclidean distance between $x_i$ and centroid ${\mu }_k$, and $c_i$ is the cluster assignment for the data point $x_i$. The squared Euclidean distance can be written as follows:

$$\parallel (R_{si}^2,z_i,\Delta {\delta }_i)-({\mu }_{k1},{\mu }_{k2},{\mu }_{k3}){\parallel }^2={(R_{si}^2-{\mu }_{k1})}^2+{(z_i-{\mu }_{k2})}^2+{(\Delta {\delta }_i-{\mu }_{k3})}^2$$

(18)

As data points have been assigned to clusters, the centroids can be updated using the mean of the points in each cluster. For each cluster k, the updated centroid can be written as follows:

$${\mu }_k^{\left(new\right)}=\left({\displaystyle \frac{1}{|C_k|}}\sum _{x_i\in C_k}{R}_{si}^2,{\displaystyle \frac{1}{|C_k|}}\sum _{x_i\in C_k}{z}_i,{\displaystyle \frac{1}{|C_k|}}\sum _{x_i\in C_k}\Delta {\delta }_i\right)$$

(19)

where $|C_k|$ is the number of points in the cluster and $C_k$ the set of points assigned to cluster k.

The algorithm continues iterating between the assignment and update steps until convergence. Convergence occurs when the centroids stop changing, i.e., the difference between centroids of consecutive iterations is negligible:

$$\parallel {\mu }_k^{\left(new\right)}-{\mu }_k^{\left(old\right)}\parallel \le ϵ, \forall k$$

(20)

Alternatively, the algorithm has the option to halt either after a pre-established number of iterations has been completed or when the variation in the total sum of squares of the within-cluster error (SSE) reaches a acceptable level. The total SSE is calculated as follows:

$$\mathrm{SSE}=\sum _{k=1}^K\sum _{x_i\in C_k}{\parallel (R_{si}^2,z_i,\Delta {\delta }_i)-({\mu }_{k1},{\mu }_{k2},{\mu }_{k3})\parallel }^2$$

(21)

The final output includes the cluster assignments $c_1,c_2,\cdots ,c_n$ for each data point, and the final centroids ${\mu }_1,{\mu }_2,\cdots ,{\mu }_K$. The objective of the K-means algorithm is to minimize the total within-cluster SSE:

$$J=\sum _{k=1}^K\sum _{x_i\in C_k}{\parallel (R_{si}^2,z_i,\Delta {\delta }_i)-({\mu }_{k1},{\mu }_{k2},{\mu }_{k3})\parallel }^2$$

(22)

3. Case Study and Dataset

The case study is conducted on a slope located in Qinghai Province near a reservoir of the Yellow River, China. This region is characterized by rugged topography with steep gradients and a complex geological setting. The area features rock formations interspersed with active fault zones, which have been affected by the water level of the reservoir. This condition contributes to pronounced slope deformation, making the area an ideal natural laboratory for evaluating and validating the performance of the proposed PSO-FOGM model. The monitoring system was established after initial signs of deformation were detected, with continuous monitoring of the slope’s stability since that time. A variety of techniques have been employed to observe and measure the deformation, including displacement monitoring sensors, boreholes, exploration tunnels, etc. All monitoring results consistently indicate that the slope is undergoing continuous deformation, and that there is an ongoing risk of slope failure. For comprehensive information on the geological details of the slope, refer to the work by Yuan et al. (2024) [35]. We selected surface displacement data from 12 monitoring points located on the slope’s surface, that is, TP 1-8, TP 2-7, TP 2-8, TP 2-9, TP 3-13, TP 3-12, TP 4-7, TP 5-9, TP 5-8, TP 4-6, TP 4-2, and TP 5-5. The monitoring points were selected to ensure a comprehensive representation of the slope’s deformation patterns, based on criteria that include extensive spatial coverage, geotechnical significance, and practical feasibility for long-term monitoring. Points were strategically distributed across different segments of the slope to capture both localized and overall deformation trends, with a focus on areas exhibiting critical geological features such as fault zones, zones of potential weakness, or high-stress regions. Additionally, accessibility and the ability to sustain reliable data acquisition were key considerations in the selection process. Together, these criteria guarantee that the collected data accurately reflect the complex deformation behavior of the slope, thereby strengthening the robustness and validity of the case study. See Figure 3 for the distribution of these monitoring points. These monitoring points are strategically positioned to capture representative deformation patterns across different sections of the slope. The data span from 1 June 2011 to 31 October 2013, providing a comprehensive time series for analysis.

Figure 4 illustrates the spatial coordinates of the selected monitoring points on the slope, with coordinates along the X-, Y-, and Z-axes representing the location of each monitoring point on the slope. The monitoring points are distributed across the slope’s surface, capturing the deformation behavior at different spatial locations.

Figure 5 presents the 2-dimensional projections of the coordinates of the selected monitoring points across three different projections: (a) the X-Y projection, (b) the X-Z projection, and (c) the Y-Z projection. In Figure 5a, the monitoring points are distributed along the X-axis ranging from 2600 m to 3000 m and the Y-axis ranging from 5600 m to 6400 m. In Figure 5b, the points are projected along the X-axis and the Z-axis ranging from 2500 m to 2850 m. Figure 5c shows the Y-Z projection.

Figure 6 presents the time variation of the monitored slope displacement at each monitoring point, with the displacement values plotted over a period from June 2011 to October 2013. Figure 6a shows the displacement data for the monitoring points TP1-8, TP2-7, TP2-8, and TP2-9, while Figure 6b displays data for the remaining monitoring points. The displacement at TP1-8, TP2-7, TP2-8, and TP2-9 remains relatively low throughout the monitoring period, with the maximum displacement for these points not exceeding 250 mm. Specifically, the highest displacement recorded for TP1-8 is 230 mm, for TP2-7 is 210 mm, for TP2-8 is 240 mm, and for TP2-9 is 220 mm. In contrast, the displacement data for the other monitoring points show much larger values. For example, the maximum displacement for TP3-12 reaches 6800 mm, for TP3-13 it is 7100 mm, for TP4-2 it is 5500 mm, and for TP5-8 it is 9000 mm. The largest displacement observed across all monitoring points is 11,200 mm at TP4-5. In addition, the time series data for TP1-8, TP2-7, TP2-8, and TP2-9 exhibit considerable fluctuation with rapid increases, suggesting minor yet significant deformation within these regions. On the other hand, the displacement data for other monitoring points show smoother and more gradual increases, indicating a steadier rate of displacement over time. This variation in the monitoring data is important for understanding the overall displacement behavior of the slope and must be considered when developing predictive models, as it may affect the models’ accuracy and the generalizability of their predictions.

4. Results

4.1. Clustering Results of Monitoring Points

As discussed in Section 2.2, the three parameters used as clustering criteria adopted by the k-means clustering method include the following: the coefficient of determination of the statistical model $R_s^2$, elevation of monitoring point z, and displacement varying difference $\delta$ for each monitoring point. The three criteria parameters serve as the input of the k-means clustering model. The values for $R_s^2$ range from 0.6032 at TP2-8 to 0.9745 at TP4-7. The elevations z vary from 2498 m at TP5-5 to 2830 m at TP1-8. The displacement varying difference $\Delta \delta$ ranges from 0.3317 at TP4-2 to 0.9979 at TP5-8, with the highest displacement varying difference observed at TP4-7, TP5-5, and TP5-9. These parameters were used to categorize the monitoring points into distinct clusters. See

Table 1. The input data of the k-means clustering method for all monitoring points.

Monitoring Points	${\mathit{R}}_{\mathit{s}}^2$	z	$\mathbf{\Delta }\mathit{\delta }$
TP1-8	0.9594	2830	0.4379
TP2-7	0.9834	2803	0.5726
TP2-8	0.6445	2702	0.6688
TP2-9	0.6032	2602	0.8081
TP3-12	0.9285	2593	0.8808
TP3-13	0.9713	2703	0.8449
TP4-2	0.9656	2548	0.3317
TP4-6	0.9618	2603	0.9981
TP4-7	0.9745	2783	0.9979
TP5-5	0.9608	2498	0.6358
TP5-8	0.9797	2732	0.9953
TP5-9	0.9722	2826	0.9979

for the clustering criteria of all monitoring points.

Figure 7 exhibits the graphical representation of the clustering results. Each monitoring point is categorized into one of three clusters. The clustering results show that TP2-9 and TP3-12 are assigned to class 1, TP2-7, TP1-8, and TP5-9 are grouped into class 2, and TP2-8, TP4-2, TP4-6, TP4-7, TP3-13, and TP5-5 fall into class 3. These clusters are determined based on the relationship between the three parameters used in the clustering method, providing valuable insights into the spatial distribution and deformation behaviors of the slope.

4.2. Prediction Results of the Proposed Model

After the monitoring points have been clustered into three classes using the k-means clustering method, we proceed to train the dataset in each class using the particle swarm optimization-based fractional-order grey model (PSO-FOGM). The training process involves using the first 90 % of the time series data sequence as training data, while the last 10 % is reserved for testing the model’s performance. The PSO-FOGM is optimized for each class individually, ensuring that the unique characteristics of the displacement trends within each class are captured effectively. By separating the data into distinct classes based on their clustering results, the model can account for different patterns in displacement behavior, leading to more accurate predictions for each group.

Figure 8 compares the displacement $\delta$ at five monitoring points in class 1. In Figure 8a, TP 2-9 exhibits considerable fluctuations in monitoring data, which is notably more variable than the smoother trends observed at other monitoring points. These fluctuations suggest that the displacement behavior at TP 2-9 is more erratic, leading to a lower prediction accuracy compared to the other points where the displacement trends are more gradual and consistent, as reflected by the $R^2$ value of 0.8032 for TP 2-9. In contrast, the $R^2$ values for TP 3-12, TP 4-2, TP 4-6, and TP 5-5 are much higher—0.9926, 0.9998, 0.9998, and 0.9957, respectively—indicating a much closer fit between the monitoring and predicted data at these points. A higher $R^2$ corresponds to smoother, more predictable displacement trends, which are easier for the proposed model to capture accurately. The coefficient of determination $R^2$ in temporal prediction, evaluating how well the model’s predictions fit the overall data, is given by Equation (23):

$$R^2=1-{\displaystyle \frac{\sum {(y_t-{\widehat{y}}_t)}^2}{\sum {(y_t-\overline{y})}^2}}$$

(23)

where $y_t$ represents the observed values, ${\widehat{y}}_t$ are the predicted values, and $\overline{y}$ is the mean of the observed values over the entire dataset. Normally, the model can be validated once $R^2$ is larger than 0.8. The results suggest that in addition to the performance of the prediction model itself, the quality of monitoring data plays a critical role in the prediction performance. That is, points with more stable, less fluctuating data yield higher prediction accuracy, whereas points with more variable data lead to lower prediction accuracy due to the model’s difficulty in capturing such irregular patterns.

Figure 9 presents a comparison of the predicted and monitoring displacement data for four monitoring points in class 2. The displacement trends at all monitoring points are generally well captured by the proposed model, as indicated by the high $R^2$ values: 0.9621 for TP 1-8, 0.9678 for TP 2-7, 0.9998 for TP 4-7, and 0.9967 for TP 5-9. These high $R^2$ values suggest that the model performs well across different monitoring points, with the predicted data closely matching the observed data, particularly for points with more consistent and gradual displacement increases. For TP 4-7 and TP 5-9, the displacement values grow steadily, reaching 5000 mm and 6000 mm, respectively, with minimal deviations between predicted and monitoring data throughout the majority of the observation period. However, a noticeable deviation is observed in subplot (d) for TP 5-9 at the final stage of the data, which corresponds to the last 10% of the dataset, representing the testing data. This deviation highlights the challenges associated with extrapolating predictions beyond the range of the training data, as the first 90% of the dataset was used for training, while the last 10% served as testing data. Despite this final deviation, the overall performance remains strong, with $R^2$ values approaching 1 for TP 4-7 and TP 5-9, demonstrating that the model is highly effective for most of the testing period.

Figure 10 compares the predicted displacement data with the monitoring data for three monitoring points in class 3. The displacement trends at all three points are well captured by the proposed model, as indicated by the $R^2$ values: 0.8407 for TP 2-8, 0.9977 for TP 3-13, and 0.9989 for TP 5-8. At TP 2-8, the displacement values fluctuate between 20 mm and 40 mm, and while the model tracks the trend, the $R^2$ value of 0.8407 suggests some difficulty in predicting the more variable data compared to the other points. On the other hand, TP 3-13 and TP 5-8 exhibit smoother, more consistent displacement patterns, with displacement values increasing steadily to around 2000 mm and 3000 mm, respectively. The $R^2$ values for these points indicate a near-perfect fit between the predicted and monitoring data, reflecting the model’s high accuracy in predicting smoother displacement trends.

Figure 11 presents the relative residuals between the predicted and monitoring data across each monitoring point organized by class. Figure 11a–e correspond to monitoring points in class 1, with TP 2-9 showing the highest relative residuals, peaking at around 3 before stabilizing around 0.1. This high fluctuation corresponds to a lower $R^2$ value of 0.8032, indicating a less accurate prediction. Similarly, TP 2-8 in class 3 shows high initial fluctuations, with residuals peaking around 0.8, which aligns with its relatively low $R^2$ value of 0.8407. These two monitoring points (TP 2-9 and TP 2-8) demonstrate the largest discrepancies between predicted and observed data, reflecting their difficulty in capturing the displacement patterns accurately. Other monitoring points exhibit lower and more stable residuals, with residuals generally staying below 0.1, suggesting a better predictive performance for these points, as evidenced by their higher $R^2$ values. The overall trend indicates that the accuracy of predictions improves as the residuals decrease, with higher $R^2$ values corresponding to more stable and predictable displacement behaviors.

In addition to the coefficient of determination $R^2$, we also estimated the Root Mean Square Error (RMSE) for each monitoring point. The RMSE provides a measure of the average magnitude of the errors between the observed and predicted values, whose expression is as follows:

$$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{n}}\sum _{t=1}^n{(y_t-{\widehat{y}}_t)}^2}$$

(24)

where $y_t$ represents the observed values, ${\widehat{y}}_t$ are the predicted values, and $\overline{y}$ is the mean of the observed values over the entire dataset. See Figure 12 for the $R^2$ and RMSE of different models for each monitoring point.

4.3. Comparison with Other Models

To evaluate the performance of the proposed PSO–fractional grey model (PSO-FOGM), two benchmark models were implemented: an artificial neural network model (ANN) and a statistical model (SM). The neural network model was designed as a feed-forward architecture consisting of an input layer, two hidden layers, and an output layer. The first hidden layer comprises 32 neurons, and the second hidden layer comprises 16 neurons. Both hidden layers use the Rectified Linear Unit activation function to introduce non-linearity, while the output layer employs a linear activation function to produce continuous predictions. The network was trained using the Adam optimizer with a learning rate of 0.001, a batch size of 32, and over 100 epochs. See [36,37] for the mathematical details of the ANN. The statistical model used for comparison is the most commonly used model in the field of slope deformation prediction, which is often serves as a baseline for assessing the performance improvements achieved by other models. In this study, we implemented a multiple linear regression model estimated via the Ordinary Least Squares (OLS) method. For a detailed mathematical description of this statistical model, please refer to [35].

Figure 13 presents the time variation of $\delta$ for the proposed model, the statistical model (SM), and the artificial neural network model (ANN) across different monitoring points. It can be seen from the figure that the proposed model outperforms both the SM and ANN, particularly for the testing data. For monitoring points TP 2-9 and TP 2-8, which exhibit significant fluctuations in the data, all three models show limited performance. However, the proposed model significantly improves the accuracy of the predictions compared to the SM and ANN, effectively capturing the high fluctuations at these points. Specifically, TP 2-9 and TP 2-8 display large deviations in the predicted data, with residuals peaking in some instances, which suggests that the erratic behavior of the monitoring data presents challenges for all models. Despite this, the proposed model still manages to provide a better fit than the SM and ANN for these fluctuating data sequences. Furthermore, the quality of the monitoring data plays a key role in prediction accuracy. For points such as TP 2-8, TP 2-9, TP 1-8, and TP 2-7, which may be affected by issues with the monitoring equipment, the data quality is relatively low, which in turn impacts the performance of all models. Nonetheless, the proposed model exhibits a notable advantage in handling these data sequences with higher fluctuations, showcasing its robustness in comparison to traditional statistical and machine learning models.

Figure 14 illustrates the $R^2$ for the statistical model (SM), the artificial neural network model (ANN), and the proposed model across different monitoring points. The black bars, representing the proposed model, are generally higher than the green (SM) and red (ANN) bars, indicating that the proposed model consistently outperforms both the SM and ANN in terms of $R^2$ for the majority of the monitoring points. The proposed model demonstrates strong performance across all monitoring points, with $R^2$ values close to 1 for most points, reflecting high accuracy in its predictions. In contrast, the SM and ANN show more variation, with lower $R^2$ values, especially for monitoring points such as TP 2-9, TP 2-8, and TP 2-7, where the models struggle to accurately predict the data. This highlights the superiority of the proposed model, especially in handling challenging data with higher fluctuations, as evidenced by its consistently higher $R^2$ values compared to the other models.

5. Discussion

In this study, we investigated the performance of a particle swarm optimization-based fractional-order grey model (PSO-FOGM) for predicting slope displacement, comparing it with traditional statistical models (SMs) and artificial neural networks (ANNs). The results clearly highlight the significance of data quality and demonstrate the superior performance of the proposed model over both the SM and ANN.

5.1. Effect of Data Quality and Model Performance

First, data quality plays a crucial role in the prediction accuracy of all models tested in this study. Monitoring points with fluctuating or noisy data, such as TP 2-9, TP 2-8, TP 2-7, and TP 1-8, presented challenges for all models, leading to lower prediction accuracy. For instance, TP 2-9, with an $R^2$ value of 0.8032 for the proposed model, and TP 2-8, with an $R^2$ value of 0.8407, demonstrate that even the proposed model struggled to handle data with significant fluctuations. This highlights the need for high-quality, stable data to achieve optimal model performance. The proposed model was, however, more robust in these cases compared to the SM and ANN, showing less deviation in its predictions. In contrast, the SM and ANN showed even greater difficulty in handling these points, with lower $R^2$ values and higher relative residuals, indicating that data quality not only affects model performance but also influences the ability of the model to generalize from training to testing data. In real-world applications, monitoring devices may experience errors or instability, which can lead to poor-quality data, and the ability of the model to still deliver relatively accurate predictions, as seen in the proposed model, is a significant advantage.

Second, the performance comparison between the proposed PSO-FOGM model, SM, and ANN was based on $R^2$ values and relative residuals, both of which demonstrated the proposed model’s superior accuracy. For most monitoring points, the proposed model achieved $R^2$ values very close to 1, with values such as 0.9998 for TP 4-2 and TP 4-7, indicating an excellent fit with the monitoring data. In comparison, the SM and ANN generally exhibited lower $R^2$ values, especially for points with more challenging data. For example, TP 2-9 had an $R^2$ value of 0.8032 for the proposed model, but the corresponding values for the SM and ANN were significantly lower, reflecting poorer prediction performance. The residuals further support this conclusion: the proposed model showed residuals consistently below 0.1 for most points, even in the testing phase, while the SM and ANN exhibited higher residuals, particularly for monitoring points with high variability, such as TP 2-9 and TP 2-8. The proposed model also demonstrated notable improvements in handling these fluctuations compared to the SM and ANN, with smaller discrepancies between the predicted and observed data. The results of this comparison highlight the effectiveness of the PSO-FOGM in modeling complex displacement trends. The optimization of the fractional order within the grey model, combined with the particle swarm optimization algorithm, enhances the model’s ability to adapt to different types of data, making it more robust than traditional statistical methods or machine learning approaches like ANNs. This ability to manage data with both high variability and stable trends provides the proposed model with a significant advantage in practical applications where data quality may not always be ideal.

In some monitoring points—such as TP 4-2 and TP 4-7—the proposed model achieved an unusually high coefficient of determination ($R^2$ values close to 1). While this might raise concerns about overfitting, closer examination reveals that these points exhibited relatively stable or smoothly varying displacement trends with minimal fluctuations, which naturally favors higher predictive accuracy. Additionally, we employed a standard train–test approach to ensure that the testing set remained independent, thereby mitigating the risk of overfitting. Not all monitoring points reached such high $R^2$ values; those with more fluctuating or noisy data demonstrated lower, though still satisfactory, performance. Therefore, rather than indicating a universal overfit, the exceptionally high $R^2$ reflects specific slope behaviors and data characteristics at these particular monitoring points.

In conclusion, the findings demonstrate that the proposed PSO-FOGM offers substantial improvements over the SM and ANN in terms of both prediction accuracy and resilience to fluctuating data, while also emphasizing the critical importance of high-quality monitoring data for optimal model performance. The results also suggest that the PSO-FOGM has the potential to be applied in a wide range of geotechnical and other fields requiring accurate time series prediction, especially in scenarios with imperfect or noisy data.

5.2. Limitations

While the proposed PSO-FOGM model has shown notable advantages in handling noisy or fluctuating data compared to traditional statistical models (SMs) and artificial neural networks (ANNs), several limitations must be acknowledged. First, the data used in this study were collected between 2011 and 2013, which may not reflect the current state of the slope under investigation. These open access datasets provided a comprehensive foundation for model validation; however, more recent monitoring data are currently unavailable for publication. This constraint potentially restricts the relevance of the findings to evolving slope conditions, and future work should incorporate updated datasets to further validate and refine the model’s predictive capabilities.

Second, although the PSO-FOGM model showed greater robustness and smaller residuals than the SM and ANN for monitoring points with highly fluctuating data, its performance still declined relative to points with more stable data. This underscores the importance of maintaining high-quality, continuous monitoring records and highlights a practical limitation: real-world sensor errors or instability may lead to incomplete or poor-quality datasets. Nevertheless, the proposed model’s ability to handle data of varying quality more effectively than the SM and ANN remains a significant advantage, particularly in operational settings where data irregularities are common.

Third, like many data-driven methods, the PSO-FOGM model relies on parameter tuning—especially in determining the optimal fractional order. Inconsistent parameter selection can result in suboptimal performance and potentially reduce the model’s ease of use for practitioners lacking expertise in optimization techniques. Future research could explore adaptive or automated parameter selection methods to minimize human intervention while maintaining or improving model accuracy. In addition, although our study demonstrates strong performance in the given slope environment, additional case studies with diverse slope conditions would help validate its generalizability. Future directions could include integrating heterogeneous data sources (e.g., remote sensing and real-time sensors) and examining the performance of the PSO-FOGM model across a wider range of geological scenarios.

6. Conclusions

In this study, we developed a particle swarm optimization-based fractional-order grey model (PSO-FOGM) for predicting the displacement of monitoring data on a slope. The proposed model was evaluated and compared against traditional statistical models (SMs) and artificial neural networks (ANNs) across multiple monitoring points. The results demonstrated that the PSO-FOGM outperforms both the SM and ANN, particularly for the testing data, providing more accurate and reliable predictions. The model showed superior performance, especially in handling monitoring points with fluctuating data, such as TP 2-9 and TP 2-8, where both the SM and ANN struggled to capture the displacement behavior accurately. Additionally, the analysis revealed the importance of data quality, with points affected by lower data quality due to monitoring device limitations yielding less accurate predictions for all models. However, the proposed model still exhibited notable advantages in these cases, underscoring its robustness in real-world applications. The high $R^2$ values obtained for the majority of the monitoring points further confirm the effectiveness of the proposed model in predicting displacement trends. This study highlights the potential of the PSO-FOGM as a powerful tool for slope stability monitoring and prediction, providing an important contribution to the field of geotechnical engineering. Future work could focus on improving the model’s performance in cases with highly erratic data and exploring its application to other geotechnical problems.