Interpretable Landslide Susceptibility Evaluation Based on Model Optimization

Abstract

Machine learning (ML) is increasingly utilized in Landslide Susceptibility Mapping (LSM), though challenges remain in interpreting the predictions of ML models. To reveal the response relationship between landslide susceptibility and evaluation factors, an interpretability model was constructed to analyze how the results of the ML model are realized. This study focuses on Zhenba County in Shaanxi Province, China, employing both Random Forest (RF) and Support Vector Machine (SVM) to develop LSM models optimized through Random Search (RS). To enhance interpretability, the study incorporates techniques such as Partial Dependence Plot (PDP), Local Interpretable Model-Agnostic Explanations (LIMEs), and Shapley Additive Explanations (SHAP). The RS-optimized RF model demonstrated superior performance, achieving an Area Under the Curve (AUC) of 0.965. The interpretability model identified the NDVI and distance from road as important factors influencing landslides occurrence. NDVI plays a positive role in the occurrence of landslides in this region, and the landslide-prone areas are within 500 m from the road. These analyses indicate the importance of improved hyperparameter selection in enhancing model accuracy and performance. The interpretability model provides valuable insights into LSM, facilitating a deeper understanding of landslide formation mechanisms and guiding the formulation of effective prevention and control strategies.

1. Introduction

Landslides are a widespread and frequent natural disaster, causing substantial economic damage and causing significant risks to human life and property safety [1,2,3]. The ability to accurately predict landslide locations is critical for effective risk assessment and for implementing mitigation strategies. To address this need, Landslide Susceptibility Mapping (LSM) was developed as an important tool for the systematic identification of areas prone to landslides [4]. LSM provides a foundation for decision-making in land use planning and emergency management [5]. In recent years, the application of machine learning (ML) in LSM has increased due to its ability to achieve high prediction accuracy using various models [6], such as Logistic Regression (LR) [7], Artificial Neural Networks (ANN) [8], Random Forest (RF) [9], and Support Vector Machines (SVM) [10], among others. When using machine learning models, choosing the right combination of hyperparameters is very important for LSM results. In some studies, a meta-heuristic optimization algorithm was used to achieve better fitting and generalization of LSM in the study area, thus improving the model’s performance [11]. Among these algorithms, Random Search (RS) is notable for its ability to explore the search space more comprehensively, rather than just relying on deterministic steps or fixed rules. However, in some studies, even though the accuracy of the model is low, it is widely used because it is easy to explain [12]. Consequently, balancing the model’s accuracy with its interpretability remains a central research challenge.

To enhance decision-making in tasks assisted by machine learning, it is crucial to thoroughly understand the output capabilities of predictive models. This understanding not only enhances the models themselves but also deepens our comprehension of the modeling process [13,14]. Landslides, due to their complex genetic mechanisms, show pronounced regional characteristics and heterogeneity influenced by various evaluation factors. Analyzing these factors is thus vital for accurate landslide prediction [15]. In order to further improve the interpretability of machine learning models and establish and build trust among users and models, some researchers have proposed a series of interpretability methods. Existing studies have demonstrated the effectiveness of interpretability models in improving the interpretability and accuracy of models [16,17]. The introduction of the post hoc interpretation algorithm offers a novel approach to explain the landslide susceptibility model based on machine learning. The principle is to interpret a model that has already been trained, rather than train the model directly with interpretability in mind. Post hoc interpretation algorithms try to explain the prediction results of the model through different methods to help people understand the decision-making process and the laws behind the model [18,19]. This algorithm can improve the understanding of the internal interpretation of the model, thus improving the fairness and robustness of the decision and ensuring that the causality of the model’s reasoning is better guaranteed [20]. Common machine learning post hoc interpretation algorithms include Partial Dependence Plot (PDP), Local Interpretable Model-Agnostic Explanations (LIME), Shapley Additive Explanations (SHAP), etc.

Some studies have used post hoc interpretation algorithms to analyze the contribution of individual predicted values, with remarkable results. Alqadhi et al. [21] used deep learning and interpretable artificial intelligence (XAI) to predict landslides, in which the PDP model accurately predicted rainfall, slope, soil texture, and river line density as key parameters triggering landslides. Based on this, Chen [22] used the LIME algorithm to make a localized interpretation of tunnel geothermal hazard susceptibility evaluation and concluded that surface temperature, fault density, and peak seismic acceleration are the most important factors in tunnel geothermal hazard susceptibility evaluation. The SHAP model addresses this limitation by calculating the Shapley value of the feature in the sample and showing the contribution of the feature to the predicted value [23]. Pradhan et al. [24] introduced the SHAP algorithm for landslide susceptibility modeling and explained the relationship between landslide and slope.

Many machine learning models lack the necessary transparency and interpretability for decision-making, making LSM difficult to apply in real-world scenarios. Machine learning-based interpretability not only provides an understanding of the model’s predictions, but also provides decision-makers and community residents with insight into landslide risk factors. By analyzing how various factors contribute to landslide predictions, a deeper understanding of landslide triggers is achieved. This improves the accuracy of predictions and allows for timely preventive actions. Wei et al. [25] used the machine learning model to point out the potential correlation between elevation, land-cover type, and safety factor and the occurrence of landslides, so as to help decision makers to better assess and manage landslide risks. In addition, explainability helps community residents better understand landslide risks and enhance their awareness and response capacity. Explaining the logic and basis behind model predictions to residents promotes their participation and cooperation in addressing landslides, providing effective technical support and decision-making for prevention and mitigation. By enhancing the algorithm and the model’s interpretability, and by focusing on improving performance, we can also boost the interpretability of the machine learning model. This approach addresses the “black box” issue inherent in LSM, clarifying the model’s internal workings. Such improvements are of great practical importance for the effective management and mitigation of landslides.

In view of this, Zhenba County of Shaanxi Province in China was selected as the study area, with 15 evaluation factors identified for LSM. These factors encompass landform, geological conditions, hydrological conditions, environmental conditions, and human activities. To optimize the hyperparameters of the RF and SVM models, an RS approach was employed, resulting in the development of the RS-RF and RS-SVM models. These optimized models were then applied as the basis for landslide susceptibility evaluation within the study area. Furthermore, a comparative analysis was conducted to assess the predictive performance of these models. Finally, the interpretability model was used to study the understandability of the landslide susceptibility model, and the correlation between the evaluation factors and landslides was explored in order to provide reference for the geological disaster prevention and control research in Zhenba County, Shaanxi Province, China.

2. Study Area and Data

2.1. Study Area

Zhenba County, Hanzhong City, Shaanxi Province, China (Figure 1), with a total area of 3437 km², is located in the west of Bushang and the east section of Micangshan. The main ridge of Bashan Mountain runs from east to west, forming two geographical units, north and south, and acts as the watershed for the Jialing River and the Han River, containing six major rivers. The geological composition is predominantly characterized by folds and faults; karst landforms are well developed, featuring numerous karst caves and underground rivers. The surface rocks are mostly sedimentary and metamorphic. Within the deep gorges, the steep mountains exhibit a vertical height difference of 2118 m. The terrain is mainly characterized by Zhongshan landform; with elevations being higher in the northwest and lower in the southeast, featuring gentler southern slopes and steeper northern slopes. The average annual rainfall in the region ranges from 1250 to 1350 mm, with rainfall concentrated between June and September. The steep terrain, active geological structures, and heavy rainfall make Zhenba County not only prone to landslides, but also a region that is frequently affected by landslides.

2.2. Data

2.2.1. Landslide Conditioning Factors

Given the complexity of landslide hazards and their evolution, selecting appropriate evaluation factors for LSM is crucial [26]. Based on the current research on landslide mechanisms [27,28], the availability of data [1], and the characteristics of landslides in the study area, this study used five major indexes, namely terrain and geomorphology, geological structure, meteorology and hydrology, human activities, and environmental conditions, as the first-level factor indexes of LSM. It collects historical landslide records, digital elevation model (DEM), geological data, administrative zoning, river network, road source and description of network, normalized difference vegetation index (NDVI), and land use data (

Table 1. Data and data sources.

Data Name	Data Name	Type	Accuracy
Historical landslides	Resources and Environmental Sciences and Data Center	Vector
DEM	Global digital elevation model (GDEM)	Grid	30 m
Geological data	National Data Center for Geological Information	Grid	1:200,000
Administrative zoning	Shaanxi Municipal Bureau of Land	Vector	1:100,000
River network	Resources and Environmental Sciences and Data Center	Vector	1:100,000
Road network	Resources and Environmental Sciences and Data Center	Vector	1:100,000
NDVI	Landsat 8 OLI	Grid	30 m
Land use	Shaanxi Municipal Bureau of Land	Grid	10 m

). According to the data source, 15 evaluation factors were obtained by different processing methods on the ArcGIS 10.5 platform, and the obtained factors were reclassified. Factors include elevation, slope, topographic relief, curvature, aspect, sediment transport index (STI), stream power index (SPI), topographic wetness index (TWI), lithology, distance to faults, NDVI, distance to rivers, land use, distance to roads, and POI kernel density (Figure 2).

2.2.2. Prepare Training and Test Datasets

To enhance the data management and calculation efficiency, this study employed a 30 m × 30 m grid unit as the minimal evaluation unit. This decision was based on the observation that the impact zones of most historical landslides are small, typically not exceeding 30 × 30 m, and smaller pixel grid units could negatively impact calculation efficiency [29,30]. For layers derived from raster data of varying resolutions, factor extraction within the ArcGIS 10.5 platform involved initially converting vector data into raster format. Subsequently, the resampling tool was utilized to standardize the raster data resolution of each factor to 30 × 30 m, ensuring uniformity across all data resolutions.

According to Bui et al. [31], LSM is characterized as a binary classification task, including landslide (value 1) and non-landslide (value 0), and the probability of landslide susceptibility is 0 to 1. Using 14 evaluation factors as input variables, the model transformed 510 landslide points from landslide areas into 4162 landslide grid units (labeled 1). Similarly, 4162 non-landslide grid units (labeled 0) were generated from areas located 500 m away from landslide sites, ensuring a balanced representation of both landslide and non-landslide conditions in the analysis [32]. Landslide raster units and non-landslide raster units were combined in a 1:1 ratio [33] into a dataset of 8342 samples. The dataset was randomly divided into the training set and the test set at a ratio of 7:3 [34]. The training set was used to build the model, and the test set was used to evaluate the model’s performance and generalization ability, together enabling effective machine learning tasks.

3. Materials and Methods

3.1. Landslide Susceptibility Model

3.1.1. Random Forest

RF selects feature variables through independent sampling and random sampling, constructs multiple decision trees for each sample, and obtains the final prediction and classification results according to the majority votes obtained from each tree [35,36]. Due to the limited generalization error generated by setting up a set of decision trees for prediction, the use of RF models can effectively avoid the problem of model overfitting and significantly improve the effectiveness and superiority of the model. The node splitting of each decision tree takes the Gini Index as the impurity function, and the expression is as follows:

$$\mathit{Gini} \mathit{Index}=1 -\sum _c{p}^2\left(c\left|t\right.\right)$$

(1)

where c is the number of classification categories, t is the decision tree node, and p is the relative frequency of c. The growth process of the decision tree in RF involves randomly extracting a specific number of characteristic variables for node splitting, and the number of extracted variables is generally the square root of the total number.

3.1.2. Support Vector Machine

SVM is a supervised machine learning algorithm based on the statistical learning theory and practical risk minimization principle [37]. The principle of SVM uses kernel function to map data from low-dimensional nonlinearity to high-dimensional space and transform nonlinear data into linear divisible, so as to find the optimal hyperplane. The core of SVM is to construct the optimal hyperplane and kernel function. The optimal hyperplane is derived as follows:

$$f\left(x\right)=\mathrm{sign}\left[\sum _{i=1}^m{a}_i{L}_{j}K(X,X_i)+b\right]$$

(2)

where f(x) is SVM regression function; a_i is a normal real number; L_j is the landslide category vector, L_j = {1, −1}; m is the total number of samples; b is the offset term; and K(X, X_j) is the kernel function.

The kernel function affects the mapping problem of samples. In this study, rbf kernel function, which has significant advantages in LSM, was adopted [38], and the expression is as follows:

$$K(x_i,x_j)=\exp \left({-\gamma ‖x_i-x_j‖}^2\right)$$

(3)

where K(x_i, x_j) is the kernel function; γ is the parameter of the kernel function, γ > 0; and ‖x_i − x_j‖² is the square Euclidean distance between two eigenvectors.

3.2. Hyperparameter Optimization Model

In the given hyperparameter search range, random search generates the selected hyperparameters by random sampling and then combines them. The arrangement of the combination points is random and non-repetitive, allowing for the identification of superior model parameters by efficiently exploring a larger configuration space with fewer trials. The number of computations in random search is a fixed value n, and the computational complexity O(n) helps mitigate the issue of search times increasing exponentially with the expansion of the hyperparameter range, thus preserving efficiency [39]. For parameters that vary continuously, random search (RS) was used to samples from the distribution.

3.3. Model Evaluation and Comparison

The frequency ratio can be used to evaluate the validity and reliability of evaluation results of the landslide sensitivity model [40]. The objective of the method is to divide the proportion of landslide samples in each grade by the proportion of grid in the grade to obtain the frequency ratio, so as to verify the accuracy of the model evaluation results. The expression for the frequency ratio is as follows:

$$\mathit{Frequency} \mathit{ratio}=\frac{\raisebox{1ex}{$L_j$}\!\left/ \!\raisebox{-1ex}{$L$}\right.}{\raisebox{1ex}{$S_j$}\!\left/ \!\raisebox{-1ex}{$S$}\right.}$$

(4)

where L_j is the number of landslides in the interval, L is the total number of landslides in the study area, S_j is the number of interval grid areas, and S is the total grid area in the study area. The greater the frequency ratio, the greater the influence of the interval on landslide occurrence

After the model is trained, the performance of the model is reflected through the calculation results of each precision evaluation index on the test set. In this study, the accuracy rate, recall rate, and FI-score based on confusion matrix are used as statistical evaluation indicators of the model, and the calculation formula of each indicator is as follows:

$$\mathit{Precision}=\frac{\mathit{TP}}{\mathit{TP}+\mathit{FP}}$$

(5)

$$\mathit{Recall} =\frac{\mathit{TP}}{\mathit{TP}+\mathit{FN}}$$

(6)

$$F1-\mathit{score} =\frac{2\mathit{Precision}·\mathit{Recall}}{\mathit{Precision}+\mathit{Recall}}$$

(7)

where TP represents the number of samples in which both the actual value and the predicted value are positive. FP indicates the number of samples with a negative actual value and a positive predicted value. FN represents the number of samples for which the actual value is positive but the predicted value is negative. TN represents the number of samples in which both the actual and predicted values are negative [41,42]. The values of accuracy rate, recall rate, and F1-score range are [0, 1]. The accuracy rate is the proportion of predicted positive samples and true-positive samples in all predicted positive samples. The closer it is to 1, the smaller the probability of misjudgment is in the results predicted by the model as landslides. The recall rate is the proportion of samples predicted by the model to be positive and true to be positive in all samples, which is close to 1, indicating that the model has a stronger ability to predict landslides. The F1-score can comprehensively reflect the model’s recognition ability of positive and negative samples through the harmonic average of accuracy rate and recall rate, and the larger the value, the better the prediction performance of the model [43,44]. In addition, the effectiveness of the model was measured using the Receiver Operator Characteristic (ROC) curve, an evaluation method that evaluates the model’s ability to predict a specific probability threshold. The area under the ROC curve is the AUC value, and the value range is [0.5, 1]. The larger the AUC value is, the better the model performance is [45,46].

3.4. Interpretability Model of Landslide Susceptibility Mapping

3.4.1. Partial Dependence Plot

Partial Dependence Plots (PDPs) aim to modify the key features within a model, while keeping all other features constant. This approach allows for the observation of how changes in these key features impact the accuracy of the model’s predictions [47]. PDP can also explain the relationship between characteristic variables and predicted results [48]. The expression is as follows:

$$\widehat{f_s}\left(x_s\right)=E_{x_C}\left[\widehat{f}\left[\left(x_s,x_C\right)\right]\right]=\int \widehat{f}\left(x_s,x_C\right)\mathit{dP}\left(x_C\right)$$

(8)

where x_s is a characteristic variable, usually there are only 1 or 2 features in the set, s; $\widehat{f_s}\left(x_s\right)$ is the estimated label value corresponding to each value of x_s; x_C is the characteristic variable other than x_s; and E represents the mathematical expectation, which is to find the average value of x_C when the characteristic variable x_s is fixed and its corresponding marginal distribution changes. $P\left(x_C\right)$ = $\int P\left(x_{\mathit{all}}\right)d{x}_s$, where x_all represents all characteristic variables, and $\int P\left(x_{\mathit{all}}\right)$ is the joint probability density of all characteristic variables in the model.

3.4.2. Local Interpretable Model-Agnostic Explanations

LIME is a model-independent, locally interpretable model that explains a single model prediction by approximating the model locally around a given prediction [49]. For a single sample, the local model is assumed to be a simple linear model, and the local data points are explained. LIME makes small perturbations around local points to observe the prediction behavior of the model and assigns weights according to the distance between the disturbance points and the original data to obtain an interpretable model and prediction results. For example, x’s interpretation model, model g; the approximation of model g; and the original model, model f, are compared by minimizing the loss function, as follows:

$$M_{\mathit{explanation}}\left(x\right)=\underset{g\in G}{\mathit{arg}\mathit{min}}L\left(f,g,{\pi }_x\right)+{ \mathsf{\Omega }}_{\left(g\right)}$$

(9)

where Ω(_g) represents the model complexity of the explanatory model g, G represents all possible explanatory models, and π_x defines the neighborhood of x and makes the model interpretable by minimizing L.

3.4.3. Shapley Additive Explanations

SHAP, as an interpretive tool for machine learning model predictions, is a game theory approach to interpret machine learning model predictions based on Shapley values [50]. Compared with traditional feature interpretation methods, SHAP effectively unifies the global or local interpretation of models [51]. SHAP uses an additive feature attribution method to produce interpretable models, where the output model is defined as a linear addition of the input variables. SHAP simplifies input z by mapping x to z by x = h_x(z). Based on z, the original model, f(x), can be expressed as a linear function of binary variables, and the calculation formula is as follows [52,53]:

$$f\left(x\right)=g\left(z\right)={\phi }_0+\sum _{i=1}^M{\phi }_i{z}_i$$

(10)

where z = {0, 1}^M; M is the number of input features; φ₀ = f(h_x(0)); and φ_i is the feature attribute value, and the formula is as follows:

$${\phi }_i=-\sum _{S\in F\backslash \left\{i\right\}}\frac{\left|S\right|!\left(M -\left|S\right|! - 1\right)!}{M!}\left[f_x\left(S\cup \left\{i\right\}\right)-f_x\left(s\right)\right]$$

(11)

$$f_x\left(s\right)=f\left(h_x^{-1}\left(z\right)\right)=E\left[f\right(x\left)\left|x_s\right.\right]$$

(12)

where F is the non-zero set input in z; S is a subset of F, and the i feature is not included in F; and φ_i is the uniform measure of the additive feature attribute, called the SHAP value.

4. Results

4.1. Results of Landslide Susceptibility Mapping

Multicollinearity is a phenomenon in which the independent variables are closely related. While multicollinearity does not affect the predictive power of the model as a whole, it does affect the individual contribution of each variable [54]. In this study, the Variance Inflation Factor (VIF) and Minimum Tolerance (TOL) were used to assess multicollinearity. A VIF > 5 or TOL < 0.2 indicate significant multicollinearity between these factors [55]. The study results showed (

Table 2. Multicollinearity analysis of the evaluation factors.

Factor	Original Factor		New Factor
	Tolerances	VIF	Tolerances	VIF
Elevation	0.596	1.679	0.596	1.678
Slope	0.241	4.150	0.241	4.141
Aspect	0.966	1.035	0.967	1.035
Topographic relief	0.172	5.828	0.237	4.213
Curvature	0.770	1.299	0.774	1.292
Lithology	0.947	1.055	0.948	1.055
NDVI	0.842	1.188	0.845	1.184
Land use	0.911	1.098	0.948	1.055
TWI	0.142	7.039	/	/
STI	0.218	4.582	0.299	3.342
SPI	0.208	4.808	0.274	3.652
Distance to faults	0.887	1.127	0.888	1.127
Distance to rivers	0.972	1.028	0.972	1.028
Distance to roads	0.589	1.698	0.589	1.697
POI kernel density	0.894	1.119	0.895	1.118

) that the TOL and VIF of topographic relief were 0.172 and 5.828, indicating a certain degree of collinearity with the other 15 factors. At the same time, the TOL and VIF of the TWI were 0.142 and 7.039, indicating a strong collinearity between TWI and other factors. After removing the TWI, the smallest TOL is 0.274, the largest VIF is 3.651, and the remaining factors all have VIF values greater than 5 or TOL less than 0.2, indicating that there is no multicollinearity between the factors used in the study. Therefore, the remaining 14 evaluation factors were used as input factors of LSM for model training.

In this study, both the RF and SVM models were optimized using random search. The optimized parameters of the RF model are n_estimators and max_depth. The n_estimators parameter is used to specify the number of decision trees in the forest, and max_depth is used to control the maximum depth or number of layers per decision tree. The optimized n_estimators of RF model are 267, and the max_depth is 16. The optimized parameters of SVM model are c and γ: c is the penalty term or regularization parameter that controls the model, and γ is mainly used to control the influence of the kernel function after the model is optimized. The optimization c of the SVM model is 1.33, and γ is 0.43.

After model training is completed, a susceptibility index is assigned to each grid, and the trained model is imported into ArcGIS 10.5 to obtain the landslide occurrence index of each grid range. The index ranges from 0 to 1, indicating the probability of landslide occurrence. The closer to 1, the higher the probability of landslide occurrence. According to Hu et al.′s study [56], using the natural discontinuation method, the study area was divided into five susceptibility levels: very low susceptibility levels (0–0.07), low susceptibility levels (0.07–0.27), medium susceptibility levels (0.27–0.47), high susceptibility levels (0.47–0.67), and very high susceptibility levels (0.67–1) (Figure 3). It can be seen from the susceptibility zoning map that the evaluation results of the four models are roughly similar, but there are some differences in the susceptibility classification region of each model. The RF model predicted that the extremely high area of the LSM was 252.45 km², followed by the high area (334.56 km²), the middle area (584.38 km²), the low area (914.40 km²), and the very low area (1351.20 km²) (Figure 3a). The extremely high area of LSM predicted by the SVM model is 287.46 km², followed by high area (310.80 km²), middle area (427.55 km²), low area (778.99 km²), and very low area (1632.20 km²) (Figure 3b). The extremely high area of LSM predicted by the RS-RF model is 177.98 km², followed by the high area (359.54 km²), the middle area (677.18 km²), the low area (997.61 km²), and the very low area (1224.70 km²) (Figure 3c). The RF-SVM model predicted that the extremely high area of LSM was 207.86 km², followed by the high area (258.84 km²), the middle area (439.09 km²), the low area (726.11 km²), and the very low area (1805.10 km²) (Figure 3d). Among them, the RS-RF model predicts the smallest area of the extremely high region of LSM, and the largest area of the very low region. The area of the middle region and the low region are more detailed, indicating that the prediction area of the RS-RF model is more accurate.

A reliable landslide susceptibility model should be able to accurately predict the probability or likelihood of landslide occurrence. If a model shows high reliability on validation sets or historical data that are consistent with actual observations, landslides may occur more frequently in areas or time periods predicted by the model. The calculated frequency ratio results (Figure 4b) show that the four models range from very low susceptible areas to very high susceptible areas, and the frequency ratio gradually increases with higher susceptibility levels, indicating that the evaluation results of the models are reasonable and reliable and can accurately identify landslide-susceptible areas. Among them, the RS-RF model has the largest frequency ratio and reflects higher susceptibility in LSM overall, which is significantly different from the other three models. The extremely high prone area of the RS-RF model is the smallest, accounting for 5.18%, but 80.75% of the landslide points in the study area are included, indicating that the RS-RF model has a better modeling performance (Figure 4a).

4.2. Model Performance Evaluation

The LSM model’s performance was evaluated using the precision, recall, and F1-score. According to the obtained index results (

Table 3. Model evaluation indexes.

Model	Precision	Recall	F1-Score
RF	87.97%	87.91%	87.94%
SVM	84.43%	84.40%	84.41%
RS-RF	90.49%	90.47%	90.48%
RS-SVM	90.24%	90.23%	90.23%

), the precision of the RF model is 87.97%, the recall is 87.91%, and the F1-score is 87.97%. The precision, recall, and F1-score of SVM model were 84.43%, 84.40%, and 84.41%, respectively. The precision, recall, and F1-score of the RS-RF model were 90.49%, 90.47%, and 90.48%, respectively. The precision of the RS-SVM model was 90.24%, and the recall and F1-score were 90.23% and 90.23%, respectively. These results clearly show that the RS-RF model has the highest index values. After hyperparameter optimization, the performance of the models was improved.

Use cross-validation techniques to evaluate the model’s performance on data other than the training dataset. Reduce the possibility of overfitting and improve the model’s ability to generalize on independent datasets. The following results were obtained by using 5-fold cross-validation (

Table 4. Results of 5—fold cross-validation of RS-RF model.

Evaluation Indexes	Fold-1	Fold-2	Fold-3	Fold-4	Fold-5
Precision	90.82%	90.33%	90.63%	90.58%	90.94%
Recall	90.75%	90.57%	90.99%	90.63%	91.17%
F1-score	90.75%	90.45%	90.99%	90.39%	90.74%

). The precision ranges from 90.33% to 90.94% in each fold, indicating that the precision of the model in identifying positive class samples is quite high. The recall in each fold is 90.57% to 91.17%, indicating that the model can capture positive class samples well. F1-scores ranged from 90.39% to 90.99% across the folds, indicating that the model performed well in maintaining a balance of precision and recall.

In this study, four models were evaluated using the AUC-ROC curve pairs (Figure 5). The AUC values of RF, SVM, and RS-SVM models were 0.943, 0.904, and 0.946, respectively, indicating that the 12 evaluation factors were significantly correlated with the distribution of landslide points, and the evaluation results were accurate. The AUC value for the RS-SVM model is 0.965, indicating a strong correlation between the 12 evaluation factors and the distribution of landslide points, resulting in highly accurate evaluation results. The AUC values of all four models exceeded 0.9, demonstrating their effective performance in LSM evaluation. Since the AUC value of the RS-SVM model surpassed the other models, ranking RS-SVM > RS-SVM > RF > SVM, the RS-RF model was given priority for a more in-depth evaluation.

4.3. Interpretability of the Model

4.3.1. Partial Dependence Plot

To identify the influence of each evaluation factor on landslide prediction, Partial Dependence Plots (PDPs) were used to detect the impact of each evaluation factor on landslide prediction probability. The PDP shows a visualization of the average model response as a function of the selected variable, revealing similarities or differences in responses to the predicted model [57]. The horizontal axis represents the value range of the evaluation factor, while the vertical axis represents the predicted value.

It can be seen from the Figure 6 that different factors have different effects on the model prediction. Landslide probability increases with the increase in slope, topographic relief, SPI, and STI, while it decreases with NDVI. The elevation, distance from fault, distance from river, and distance from road were negatively correlated with the probability of landslide occurrence. For land use type, forest has the highest probability of landslide, while residential has the lowest probability of landslide. The PDP shows that, within a distance of less than 1500 m from a river, the probability of a landslide decreases as the distance increases. When the distance is greater than 1500 m, the probability of landslide increases with the increase in distance. When the distance is greater than 2500 m, the probability of landslide decreases again. In general, the POI kernel density shows a positive correlation with landslide occurrence probability. When the POI kernel density is greater than 0.07, landslide prediction probability increases significantly. However, in order to fully understand the model, it is important to consider other factors and potential interactions.

4.3.2. Local Interpretable Model-Agnostic Explanations

LIME uses a locally linear model, and the generated interpretive graph shows the features that have the greatest impact on the model’s decisions in the local region, with positive numbers in the middle indicating that the factor has a positive impact on the outcome, and negative numbers indicating the opposite [58]. In this study, four groups of samples were randomly selected, two groups were landslide occurrence samples, and two groups were non-landslide samples. Each group contained 14 variables.

As shown in the figure (Figure 7a,b), the results show the contribution of various factors to the prediction results of landslide hazards. The probability of landslide occurrence predicted by the model is 88% and 91%, respectively. In terms of the probability of landslide occurrence, the model predicts that the primary factors contributing to landslides include low NDVI, proximity to roads, and a smaller degree of relief. In the samples without landslides (Figure 7c,d), the model predicted a landslide probability of 94% and 91%, respectively. The model’s mispredictions could be attributed to factors such as longer distances from roads, high NDVI, low elevation, etc. Both sets of randomly sampled data, during and without landslides, indicate that the stratigraphic lithology is Jurassic. The LIME analysis may suggest that the Jurassic layer plays a significant role in the model’s predictions for certain parts of these datasets. LIME explains that NDVI is the main driving factor of landslide, which helps to understand the decision-making process of the model in this local area.

4.3.3. Shapley Additive Explanations

To understand the interaction and importance of variables in data-driven modeling, the SHAP model’s importance ranking diagram and summary plot are used to interpret the model globally [59], showing the magnitude and direction of each factor’s influence on the landslide susceptibility.

Each point in the SHAP summary diagram represents the magnitude of a factor value for a landslide sample within an evaluation factor, along with the corresponding SHAP value. The more intense the red color, the larger the factor value. Similarly, the closer a point’s position is to the right, the larger the SHAP value, indicating a higher contribution to landslide susceptibility; in other words, it signifies a greater likelihood of landslides. As can be seen from the Figure 8a, the factors that have the greatest impact on the susceptibility in the study area are NDVI, followed by the distance from the road. Analyzing the variation in the SHAP value of the NDVI evaluation factor with its factor value reveals that the point color for NDVI tends to be red, suggesting that a higher NDVI value contributes negatively to landslide susceptibility, thereby indicating a reduced likelihood of landslides. Generally, closer proximity to roads positively contributes to landslide occurrence, increasing the likelihood of landslides. Elevation is the most influential factor in topography, while lithology is the most significant factor in formation structure, and SPI dominates in terms of meteorology and hydrology. By averaging the SHAP values for all samples of each factor, the importance-ranking diagram of landslide hazard evaluation factors can be obtained. This diagram quantitatively analyzes the contribution degree of each factor to landslide hazard through SHAP values. It can be seen from the figure (Figure 8b) that SHAP values of land use type, curvature, and distance from fault are all less than 0.025, indicating that these factors have little influence on landslide hazards.

5. Discussion

5.1. Regional Analysis of Landslide Susceptibility Based on Model Optimization

This study used machine learning to predict the probability of landslide occurrence and evaluated the response of each model in landslide prediction. To achieve optimal Landslide Susceptibility Mapping (LSM), some studies utilize between 9 and 15 evaluation factors [60,61,62]. Through a collinearity diagnosis, it can be seen in this study that TWI has the highest VIF, while both STI and SPI have a relatively high VIF. It is observed that TWI is positively correlated with STI, while SPI is negatively correlated with SPI. The relationship between SPI and STI reflects the interaction between topography and water flow in soil erosion. Soil erosion compromises the stability of surface soil, potentially leading to landslides [63,64,65]. The steep terrain in the study area may contribute to high VIF values for slope and topographic relief, both derived from DEM data, thereby indicating potential multicollinearity [66,67]. After considering the landslide characteristics of the study area and conducting further extensive research, the remaining 14 evaluation factors were applied to the study of Zhenba County, excluding TWI. The selection of factors for exclusion and their impact on landslide occurrence differ across various regions. Therefore, selecting the appropriate evaluation factors is helpful to improve the accuracy of the evaluation results of susceptibility. The performance of the hyperparameter optimization method depends on the nature of the learning task and data distribution, and the optimization effect will be better when the search scope of hyperparameter is both extensive and detailed [68].

In this study, the evaluation indexes were significantly improved through random search, and the frequency ratio of the predicted results further proved that the RS-RF model was superior to other models, which revealed that hyperparameter optimization can effectively improve the accuracy of machine learning models and their performance in LSM. Some studies have used machine learning models such as RF, SVM, and DNN to predict landslide susceptibility [69,70,71]. However, these studies did not consider the optimization of input features and parameters. In contrast, both the RS-RF model and RS-SVM model considering the combination of features showed the best generalization ability and prediction accuracy. From the perspective of evaluation indicators, the RS-RF model and RS-SVM model are similar in performance, but significant variations are observed in the LSM produced by each, which may be due to the difference in model architecture caused by different combinations of hyperparameters, resulting in drivers output results. This also confirms that the choice of hyperparameter combinations has different effects on the effectiveness of the landslide susceptibility assessment based on machine learning. Secondly, negative samples are selected through random sampling, and the model’s performance is comprehensively evaluated using various evaluation metrics. The results indicate that the model achieved a good predictive performance. However, selecting non-landslide samples from low-sensitivity regions may further enhance the model’s performance [56,72]. Given the uncertainties associated with this approach, it is essential to fine-tune multiple machine learning models using more optimized algorithms and establish specific methods for negative sample selection to thoroughly assess the effectiveness of optimization algorithms in Landslide Susceptibility Mapping.

5.2. Interpretability of the Model and Constraints of Landslide Hazards

Previous studies on landslide susceptibility models have primarily focused on the significance of factors and the spatial distribution of landslide susceptibility regions. However, these studies have often neglected the internal influence of each factor on landslide prediction. Consequently, they have failed to address the complexity inherent in machine learning. In this study, RF showed an excellent model performance, and some studies also confirmed that RF has a better performance in LSM compared with other machine learning models [73,74]. Although RF can explain the main disaster factors of landslide in the study area to a certain extent through factor importance ranking [31], for a single landslide event, the RF model is unable to detail the contribution of each evaluation factor to the occurrence of the landslide. This limitation restricts the practical engineering application of LSM [75]. This study combines interpretability algorithms to explain the vulnerability of landslides in the study area at both global and local levels, address how the model comes up with prediction and potential decision criteria, and gain insight into the overall trend of the model and the model’s behavior in some specific situations.

In this study, global interpretation helps to explain the model’s behavior on the dataset. SHAP shows that NDVI is the most important leading factor of LSM in the study area, and NDVI is a good indicator of vegetation growth and spatial distribution density of vegetation [76], which is closely related to the occurrence of landslides. According to the division of NDVI by previous studies [77], the study area is basically in the middle and high vegetation cover area (0.6–1). Some studies have shown [78] that there is a close relationship between the change in NDVI and human activities. Vegetation change is a long-term and complex dynamic process, enhanced to some degree through the implementation of projects aimed at converting farmland back to forest and grassland, along with natural forest conservation efforts [79,80]. These measures can help mitigate the occurrence of landslides in the study area [81,82]. The local interpretation focuses on the prediction outcome for a single data point, and PDP shows that NDVI is negatively correlated with the probability of landslide occurrence. LIME further indicates that low NDVI promotes the occurrence of landslides, while high NDVI inhibits the occurrence of landslides. The reason may be that human activities diminish the vegetation cover in some areas, thus reducing the soil and water conservation ability and shear strength of the slope. The higher the vegetation coverage, the greater the resistance to water infiltration and the stability of the slope, leading to a reduced likelihood of landslides. This observation aligns with the actual distribution patterns of landslides observed [83]. The proximity to roads is identified as the second most significant factor for LSM in the study area, with the highest incidence of landslides occurring when the distance from a road is less than 500 m. This phenomenon may be attributed to soil loosening during road construction, leading to the destruction of vegetation and consequently increasing the likelihood of landslides. The construction and maintenance of roads may also change the terrain, thus affecting the stability of the surrounding land, which can lead to landslides.

The interpretability model of this study explains that human activities play an important role in landslides in this region. In road construction, man-made activities introduce new factors that can trigger landslide. To accommodate human transportation needs, construction often involves cutting through mountains, tunneling, and filling valleys, disrupting the original slope stability and, thereby, reducing it. Secondly, after the completion of the road, the movement of vehicles will cause the vibration of the slope, and the slope will then deform over time, which may lead to the occurrence of landslide geological disasters. Quantifying the contribution of NDVI and human activities of different vegetation types and their interactions is worthy of further research.

6. Conclusions

This study optimized the performance of RF and SVM models for landslide susceptibility in the study area and used interpretability models to reveal how the evaluation factors respond to landslide susceptibility. Prior to modeling, a multicollinearity diagnosis was employed to preprocess the data of landslide evaluation factors, followed by training RF and SVM models in combination with random search to achieve the highest performance. The landslide susceptibility prediction model was evaluated. Finally, an interpretability model was constructed by a post hoc interpretation algorithm to reveal the relationship between landslide evaluation factors and landslides.

The performance of RF, SVM, RS-RF, and RS-SVM models falls within a reasonable range, with the RS-RF model exhibiting the best performance. This model demonstrates a strong predictive capability for future landslides in the study area, offering comprehensive insights for managing and mitigating future landslides. Furthermore, human activities emerge as the primary factors contributing to landslides in the study area. Within this context, NDVI has a positive influence on mitigating landslide occurrences, with areas situated less than 500 m from roads being at a higher risk of landslides.