A Comparative Study of Susceptibility and Hazard for Mass Movements Applying Quantitative Machine Learning Techniques—Case Study: Northern Lima Commonwealth, Peru

Abstract

This study addresses the importance of conducting mass movement susceptibility mapping and hazard assessment using quantitative techniques, including machine learning, in the Northern Lima Commonwealth (NLC). A previous exploration of the topographic variables revealed a high correlation and multicollinearity among some of them, which led to dimensionality reduction through a principal component analysis (PCA). Six susceptibility models were generated using weights of evidence, logistic regression, multilayer perceptron, support vector machine, random forest, and naive Bayes methods to produce quantitative susceptibility maps and assess the hazard associated with two scenarios: the first being El Niño phenomenon and the second being an earthquake exceeding 8.8 Mw. The main findings indicate that machine learning models exhibit excellent predictive performance for the presence and absence of mass movement events, as all models surpassed an AUC value of >0.9, with the random forest model standing out. In terms of hazard levels, in the event of an El Niño phenomenon or an earthquake exceeding 8.8 Mw, approximately 40% and 35% respectively, of the NLC area would be exposed to the highest hazard levels. The importance of integrating methodologies in mass movement susceptibility models is also emphasized; these methodologies include the correlation analysis, multicollinearity assessment, dimensionality reduction of variables, and coupling statistical models with machine learning models to improve the predictive accuracy of machine learning models. The findings of this research are expected to serve as a supportive tool for land managers in formulating effective disaster prevention and risk reduction strategies.

1. Introduction

More than 2.8 million people [1] live on the hillsides of Metropolitan Lima, occupying the territory in a disordered, disjointed articulation, with complete informality, resulting in high vulnerability and exposure to natural hazards (Figure 1). Among them, mass movements (MMs) are induced by El Niño phenomenon and earthquakes, especially on the central coast of Peru, which is located in an area with a 278-year seismic gap since the 1746 earthquake [2], and a seismic event of a magnitude greater than 8.8 Mw is expected. If this occurs, due to the level of ground shaking, landslides, topples, and rockfalls are expected, leading to loss of life, damage to infrastructure, and blockage of access routes that connect Lima with the most vulnerable districts and the highly populated areas settled on the hillsides of Metropolitan Lima [3]. On the other hand, the El Niño phenomenon has occurred with greater recurrence in recent years, with torrential rains that caused debris flow, landslides, and collapses in many departments along the Peruvian coast, including Lima, causing damage to infrastructure, and affecting the lives and health of people [4]. In the Northern Lima Commonwealth (NLC), it has been reported that approximately 60% of emergency reports correspond to MMs [5], resulting in human and economic losses. The lack of an effective system for assessing and predicting these hazards, as well as adequate mitigation strategies, poses a latent threat.

The assessment of mass movement susceptibility (MMS) is part of the first aspect of disaster risk management, which is important for urban planning, response, and post-disaster reconstruction [6]. In recent decades, MMS mapping has been widely used to zone probable areas for future MM based on identifying areas of past occurrences and areas with similar or identical physical characteristics [7]. To perform MMS mapping, various models based on Geographic Information Systems (GISs) have been used. These models can be classified into qualitative or knowledge-based methods such as the heuristic method [8], semi-quantitative methods such as the analytic hierarchy process [9], quantitative or data-driven methods such as bivariate or multivariate statistical methods [6,8,10,11,12,13], machine learning [14,15,16,17], and hybrid approaches [18,19,20,21,22,23]. Additionally, physics-based models have been employed in the assessment of landslide susceptibility [24,25,26] and could be used for MM to enhance the understanding of the mechanisms that generate these phenomena. It is acknowledged that, for our study area, as recognized by [27], the availability of input data for the models limits the results of the models.

The Geological, Mining, and Metallurgical Institute of Peru (INGEMMET) has developed the regional scale MMS model in Metropolitan Lima [28] using a qualitative (heuristic) expert judgment approach. This input is utilized by the National Center for Estimation, Prevention, and Reduction of Disaster Risk (CENEPRED) to conduct risk scenarios by adding triggering variables and hazard exposure analysis. The free availability of conditioning and triggering variables data from national and international technical–scientific institutions, coupled with significant advancements in GIS tools, software, and open-source codes for processing high-resolution spatial information over large areas of land, provides the necessary resources to close the gaps in exhaustive studies for exploring data-driven models for MMS mapping. These models include weight of evidence (WoE) and machine learning (ML) techniques such as logistic regression (LR), multilayer perceptron (MLP), random forest (RF), support vector machine (SVM), and naive Bayes (NB) for estimating susceptibility and hazard due to MMs. Addressing this knowledge gap is imperative to enhance the prediction capability and response to such hazards.

2. Materials and Methods

2.1. Study Area

The study area covers seven districts of the NLC, Los Olivos, San Martin de Porras, Independencia, Comas, Carabayllo, Ancon, and Puente Piedra; it covers an area of 793 km², ranging from 11.572° to 12.039° S latitude and from 77.199° to 76.810° W longitude. The elevation of the study area ranges from 26.5 to 2748.0 m above sea level (m asl), with a maximum elevation difference of 2721.5 m asl. The NLC districts are located on the western–central boundary of the South American continent, in the subduction zone of the Nazca plate in relation to the South American. On the other hand, historically, during the El Niño phenomenon, anomalies in precipitation triggering MM phenomena have been demonstrated on the coast of Lima [29,30]. The oldest geological units belong to the Upper Jurassic and Pleistocene, which outcrop in the high reliefs of the Andean foothills in the highest areas of the study region; in contrast, the more recent units form the valley and plain fill that descends to the Pacific Ocean coast [31]. The rock types vary in nature, including andesitic volcanic rocks interbedded with marine shales, limestones, and sandstones; intrusive rocks, such as diorites, tonalites, granodiorites, and granites, outcrop in the peripheries of urban areas and are altered and easily disintegrated; fluvio-alluvial deposits, consisting of a matrix of sands and clays that incorporate gravels, pebbles, and boulders, cover most of Lima; lastly, aeolian deposits of fine sand and silt host numerous human settlements [32].

2.2. Mass Movement Inventory (MMI)

The MMI represents information on the spatial distribution of the location of MMs and also provides crucial information in the study of the relationships between the occurrence of MM and the conditioning and triggering causal factors [12,33]. The MMI used in this study was conducted by INGEMMET (from 1960 to 2023, last update), photointerpreted in the office at a scale of 1/25,000 from aerial photographs and satellite images from Landsat-5 and the Google Earth platform. In addition, the MMI was verified in extensive fieldwork, which was carried out by the technical team of INGEMMET. It should be noted that the types of MM phenomena considered in this study are landslides, topple, and rockfall, which could be associated with El Niño and seismic events. Non-MMs were mapped using high spatial-resolution satellite images from Google Earth, employing a hybrid approach. Initially, flat areas such as rivers, streams, and slopes less than 5° were mapped as proposed by [34]. Furthermore, the MMS model developed by INGEMMET was used to complement the non-MM samples in areas of very low, low, and medium susceptibility that coincide with geological zones of alluvial deposits and geomorphological features such as slopes or alluvial-fan piedmonts. These are moderate slope areas where mass movements such as landslides, rockfalls, and topples do not occur. As suggested by [35,36], the hybrid approach for mapping non-negative samples improves the performance and reliability of the models.

A total of 236 MM polygons were randomly selected from a total of 329 mapped using the vector > research tool > random selection tool in QGIS. Among these, 189 (80%) were used for training, and 47 (20%) were used for evaluation. It is worth mentioning that debris flows were not considered because these polygons are mostly located spatially in the lower parts of the valley, which could affect the results of the MMS models.

2.3. Data

The

Table 1. Type and description of variables used in the study. Information provided by national and international sources.

Type	Description	Type	Scale or Spatial Resolution	Year	Source	Link
Vectorial	Lithology	Categorical	1/100,000	-	INGEMMET	https://geocatmin.ingemmet.gob.pe/geocatmin/ (last access 2 February 2024)
	Geomorphology	Categorical	1/100,000	-	INGEMMET	https://geocatmin.ingemmet.gob.pe/geocatmin/ (last access 2 February 2024)
	Hydrogeology	Categorical	1/100,000		INGEMMET	https://geocatmin.ingemmet.gob.pe/geocatmin/ (last access 2 February 2024)
	Mass movements inventory	Categorical	1/50,000	2021	INGEMMET	https://geocatmin.ingemmet.gob.pe/geocatmin/ (last access 2 February 2024)
Raster	Vegetation cover	Categorical	1/100,000		INGEMMET	https://www.datosabiertos.gob.pe/dataset/cobertura-vegetal-ministerio-del-ambiente (last access 2 February 2024)
	Digital elevation model (DEM)	Continuous	12.5 m	2010	USGS	https://earthexplorer.usgs.gov/ (last access 2 February 2024)
	Seismic microzonation	Categorical	-	-	IGP/CISMID	https://www.igp.gob.pe/servicios/infraestructura-de-datos-espaciales/componentes/webservice (last access 2 February 2024)
	Precipitation anomalies in El Niño phenomenon	Continuous	100 m	2021	SENAMHI	https://idesep.senamhi.gob.pe/portalidesep/ (last access 2 February 2024)

INGEMMET, Instituto Geológico, Minero y Metalúrgico; USGS, United States Geological Survey; IGP, Instituto Geofísico del Perú; CISMID, Centro Peruano Japonés de Investigaciones Sísmicas y Mitigación de Desastres; SENAMHI, Servicio Nacional de Meteorología e Hidrología del Perú.

shows the vector and raster data used in the research. All variables were standardized to a 12.5 m pixel size raster format, including geological, environmental, and triggering variables. Finally, the final working scale of the susceptibility and hazard mapping by MM was 1/10,000.

In addition,

Table 2. Research variables names.

Class	Name	Variable	PCA	Type of Variable
Conditioning factor
Geological and environmental	Lithology	U1	-	Categorical
	Geomorphology	U2	-	Categorical
	Hydrology	U3	-	Categorical
	Vegetation cover	U4	-	Categorical
Topographical	Slope	T1	PCA-1 PCA-2 PCA-3	Continuous
	Aspect	T2		Continuous
	Topographic Wetness Index (TWI)	T3
	Terrain Roughness Index (TRI)	T4		Continuous
	Flow direction	T5		Continuous
	Profile curvature	T6		Continuous
	General curvature	T7		Continuous
Triggering factors
Seismic 8.8 Mw (seismic microzonation) Precipitation anomalies in El Niño phenomenon		D1	-	Categorical
		D2	-	Continuous

presents the DEM-derived topographic products, geological and environmental variables of study for use in the susceptibility and hazard models for the NLC, their representation is presented in Figure A1 and Figure A2.

2.4. Methods

Figure 2 shows the flowchart of this study, which is divided into five steps: the first step is data downloading and preparation, followed by variable exploration, geospatial modelling of the MMS, evaluation of MMS models using the area under the curve (AUC), and estimation of MM hazard under two scenarios.

2.4.1. Step 1: Input Datasets

Vector and raster data were downloaded from national and international geospatial repositories as indicated in Table 1. The data were standardized to the raster format at the same spatial resolution, using QGIS. Training and testing data were selected using the random selection tool in QGIS.

2.4.2. Step 2: Exploratory Variable Methods

Before applying the MMS model, it is important to ensure that there is no dependence between variables (variable correlation) and that the variables are not influenced by multicollinearity.

Pearson Correlation

To discard the linear correlation of the variables, the Pearson correlation was applied, which is considered an effective method for this purpose [13]. The following formula was used:

$$r={\sigma }_{xy}/{\sigma }_x{\sigma }_y$$

(1)

where ${\sigma }_{xy}$ is the covariance of variables “x” and “y”; and ${\sigma }_x$ and ${\sigma }_y$ are the standard deviations of variables “x” and “y”, respectively. The value of the Pearson coefficient can vary from −1 to 1, where if r < 0, the correlation is negative and stronger as it approaches r = −1; if r > 0, the correlation is positive and stronger as it approaches r = 1; finally, if r = 0, then there is no relationship between the variables. Additionally, as indicated by [37], a value greater than ±0.8 could lead to multicollinearity issues.

Multicollinearity

Multicollinearity is a condition where there are high relationships between two or more independent variables in a multiple regression model [38]. To verify multicollinearity, the variance inflation factor (VIF) was used, where, in practical terms, a VIF value greater than 5 or 10 [39] indicates that the association of regression coefficients is poorly estimated due to multicollinearity issues [40].

$$VIF=1/(1-R_j^2)$$

(2)

where $R_j^2$ is the coefficient of determination for the regression of $x_j$ on other explanatory variables.

Principal Component Analysis (PCA)

PCA is a multivariate mathematical procedure that performs the orthogonal transformation of a set of correlated variables into a smaller number of uncorrelated variables or principal components [41], which are mutually independent [42]. Once the multicollinearity among variables has been assessed and confirmed, it is common practice to exclude highly correlated variables that influence each other. However, PCA allows for addressing the multicollinearity issue among influential factors without the need to eliminate variables [10]. This is considered a good option because natural processes are integral, and excluding a variable could lead to loss of information in the analysis of the studied phenomenon. Additionally, PCA allows for evaluating the impact of different influencing factors on MMS. A more extensive discussion on the mathematical foundations of PCA can be found in [43,44,45], and the PCA summary procedure assumes that the original data are as follows:

$$X= \left[\begin{array}{cccc}{x}_{11}& x_{12}& \dots & x_{1m}\\ .& .& \dots & .\\ .& .& \dots & .\\ x_{n1}& x_{n2}& \dots & x_{nm}\end{array}\right]$$

(3)

where m is the number of causal factors, n is the number of MM, and each x_nm represents the value of a MM factor, and the mean and standard deviation of these factors can be calculated. For this matrix, the eigenvalue and eigenvector can be determined by the following:

$$\left(R-{\lambda }_{i}I\right)l_i=0$$

(4)

where λ_i and l_i are the eigenvalues and eigenvectors, respectively; l_i corresponds to the principal components; and λ_i corresponds to the variance obtained from each principal component. For a specific feature vector, its cumulative contribution rate can be calculated by the following equation:

$$\alpha = \frac{{\lambda }_i+{\lambda }_i+\cdots +{\lambda }_k}{{\lambda }_i+{\lambda }_i+\cdots +{\lambda }_m}\times 100\%$$

(5)

The majority of information from input variables is found in the first principal component, which can be expressed as shown in the following equation [46]:

$$PCm=a_{m1}{y}_1+a_{m2}{y}_2+\dots +a_{mn}{y}_n$$

(6)

where PC is the principal component in the “m-th” place, and $a_{mn}$ is the weight for the m-th PC for the n-th variable.

The number of PCs was determined based on the level of variance explanation, which ranges from 0 to 1 or 0% to 100%. In this study, a minimum threshold of 0.6 for variance explanation was used; however, in some similar studies, values as low as 0.40 [42] or as high as 0.68 [47] for variance explanation based on the number of PC have been found. The choice of the variance explanation value is based on the objective of reducing the correlation and multicollinearity of the variables.

2.4.3. Step 3: Mass Movement Susceptibility Modelling

After performing an exploratory analysis of topographic variables and reducing their dimensionality to PCA-1, PCA-2, and PCA-3, geological and environmental variables were integrated into U1, U2, U3, and U4. The MMS models were constructed using two composite variables, the sum of the topographic PCAs (PCA_123), and geological–environmental components (U_1234), after applying the WoE process. These MMS models were reclassified into quintiles at 20, 40, 60, 80, and 100% using the Numpy Python library. Finally, susceptibility areas were calculated using QGIS.

Weights of Evidence (WoE)

Quantitative techniques based on statistics establish functional relationships between instability factors and the past and present distribution of mass movements [42]. The WoE method proposed by [48] is a bivariate statistical technique based on Bayesian probability theory. The stability or instability of certain regions can be estimated through a set of conditioning factors, which are measured by the relationship and spatial distribution of areas known and affected by MM [47]. In this study, WoE was used to determine the relationship between factors influencing the occurrence of MM; to map the MMS; and as input variables for the LR, MLP, SVM, RF, and NB models.

The first step in determining the WoE for each class involves obtaining the prior probability of finding MM, which is estimated as the area affected by MM (L) in the past over the total study area (A).

$$P(L)=N(\mathrm{L})/N(\mathrm{A})$$

(7)

Refs. [48,49] present a mathematical development of the fundamental of the methodology. The method assigns positive (W⁺) or negative (W⁻) weights to each class of conditioning variables based on the degree of association between the variable class and the spatial distribution and density of evidence of MM. $B$ and $\overline{B}$ represent the presence and absence of the conditioning factor in potential mass movements, respectively, and $\overline{L}$ indicates the absence of mass movements.

$${Wi}^{+}=\ln (P\left(B|L\right)/P\left(B|\overline{L}\right))$$

(8)

$${Wi}^{-}=\ln (P\left(\overline{B}|L\right)/P\left(\overline{B}|\overline{L}\right))$$

(9)

where Wi⁺ with a positive value (>0) indicates that the variable is present where there is the presence of mass movements; additionally, its magnitude represents the positive correlation between the presence of the factor and MMs. On the other hand, Wi⁺ with a negative value (<0) indicates that the absence contributes to the generation of MMs. Wi⁻ is used to evaluate the importance of the absence of the factor in the occurrence of an MM: when Wi⁻ is positive (>0), the absence of the variable is favorable in the generation of the MM; the opposite (<0) is not [49]. Both Wi⁺ and Wi⁻ are estimated for each class of variables.

The contrast value or final weight, $W_f$ (Equation (7)), indicates the measure of correlation between the conditioning factor and the MM. If $W_f=0$, then the spatial distribution of MM is independent of the considered variable. If $W_f>0$, then there is a positive association between the variable and the generation of the MM. Lastly, if $W_f<0$, there is a negative association, meaning that the absence of the factor contributes to the generation of the MM [49,50].

$$W_f=W^{+}-W^{-}$$

(10)

Finally, to obtain the MMS map of the conditioning factors by WoE, the algebraic sum of all the contrast values of each variable is calculated. This means that the weights of each class of the variables are summed pixel by pixel, and the sum is the susceptibility map.

$${MMS}_{WoE}=W_{f,Variable1}+W_{f,Variable2}+\cdots W_{f,Variable“n”}$$

(11)

Logistic Regression (LR)

LR is considered one of the most popular statistical methods for multivariate regression analysis used to investigate binary response from a set of measurements [51] in the earth sciences. In other words, it estimates the relationship between a dependent variable and multiple independent variables [52]. The variables can be continuous, discrete, or a combination of both; they can have a normal or non-normal distribution, and the dependent variable is dichotomous [53].

In the analysis of MMS, the dependent variable is the absence (probability of occurrence, 0) or presence (probability of occurrence, 1) of an MM, which, for this study, was taken from the MMI of INGEMMET. LR transforms the dichotomous dependent variable into a logit variable, which can be used to form a multivariate regression relationship between the dependent variable and the independent variables, which, in this case, are the multiple conditioning factors [54]. The results of the LR estimate the probability of the presence or absence of an MM based on the predictor variables, according to the following equation.

$$p=1/(1+e^{-z})$$

(12)

where $p$ is the probability that the dependent variable will be 1 (maximum probability of MM presence) or 0 (minimum probability of MM presence), forming an S-curve; e is the Napierian number, and $z$ is the linear combination of the independent variables. The linear combination, $z$, can be expressed by the following formula [51].

$$z=B_0+B_1{X}_1+B_2{X}_2+B_3{X}_3+\cdots B_n{X}_n$$

(13)

where $X$ is the independent variable (conditioning factors), which can be represented as $X(x_1,x_2,\dots x_n)$, and B is the estimated coefficient for each independent variable, $B(b_1,b_2,\dots ,b_n)$. B₀ is the intercept of the model, and n is the number of independent variables [55]. Based on Equations (12) and (13), the LR model can be expressed in its extended form as Equation (14). It is worth mentioning that the statsmodels module of Python 3.6 was used.

$$Logit(P)=(\frac{1}{1+e^{-(B_0+B_1{X}_1+B_2{X}_2+B_3{X}_3+\cdots B_n{X}_n)}})$$

(14)

Multilayer Perceptron (MLP)

The MLP is an artificial neural network composed of multiple layers of interconnected neurons, commonly used in supervised learning. It is used to solve complex classification and regression problems, thanks to its ability to learn nonlinear representations of data based on mathematical algorithms to mimic the learning process of the human brain [56]. Structurally, the network consists of input layers, hidden layers with different numbers of neurons, and an output layer (i.e., the MMS model). The neurons in the layers are connected through weight values, which are trained and tested to form a stable network structure with decision-making capabilities [8]. Considering X = X_{i(i=1,2,…,n)} as the vector of factors conditioning the MM, and Y_j = (Y₁, Y₂) indicating the class of MM or non-MM, MLP can be expressed by the following equation:

$$Y_i={f(X}_i)+b_j$$

(15)

where b_j is the bias value of the neuron, and ${f(X}_i)$ is an hidden function that is optimized by the adjustable network weights during the training process for a given network architecture [57,58]. The MLP model was implemented using the scikit-learn module of Python 3.6.

Support Vector Machine (SVM)

Proposed by [59], it is a supervised ML method based on the concept of an optimal separating hyperplane in the sample space, such that the distance to the classification hyperplane of the two class groups is a maximum function between the margins of the class boundaries [60,61,62]. This classification capability makes SVM used for solving non-linear classification and regression problems; thus, it is one of the most used ML techniques in assessing MMS. Consider a matrix of conditioning factors (X = X_{i(i=1,2,…,n)}); Y_j = (Y₁, Y₂) is a vector of MM classes (non-MM and MM), and the optimal hyperplane can be obtained by solving the classification function as follows:

$$f\left(x\right)=sign\left[{\sum }_{i=1}^n{a}_i{Y}_{j}K\left(X,X_i\right)+b\right]$$

(16)

where a_i is the positive real constant, n is the number of conditioning factors, b is the bias, and K(X, X_i) is kernel function whose binary solution can be reviewed in [63,64,65]. The SVM model was implemented using the scikit-learn module of Python 3.6.

Random Forest (RF)

Proposed by [66], random forest is a supervised classification algorithm that is based on classification and regression trees used to create each decision tree. It utilizes a random subset of variables at each node based on a Bootstrap sample, and RF generates thousands of random binary trees to form a forest [17]. In cases with large amounts of sample data, the more feature elements present, the fewer errors and overfitting are generated [61,67]. RF is frequently used to determine the MMS [17,62,67,68]. Equation (17) represents the algorithm of RF.

$${\widehat{C}}_{rf}=majority vote\{{{\widehat{C}}_n(x)\}}_{n=1}^N$$

(17)

where ${\widehat{C}}_{rf}$ is the final predicted class by the RF, ${\widehat{C}}_n$ is the class predicted by the n-th decision tree in the forest for the observation (x) and N indicates the number of decision trees in the forest [69]. In this study the Python scikit-learn module was applied to implement the RF model.

Naive Bayes (NB)

NB is an important algorithm in the field of ML and data mining [61], applied to various fields, and is based on the Bayes probability theorem [70], which is suited for when the data have a high dimension and is not affected by the distribution of the data [71]. NB is a classifier with absolute independence assumptions between attributes [70]. The NB classification process within a set of factors affecting the prior probability of an MM occurrence can be expressed as follows:

$$Y_{NB}={argmax}_{Yi=\left\{non-MM,MM\right\}}P\left(Y_i\right){\prod }_{i=1}^{n}P(X_i/Y_i)$$

(18)

where X = (X₁, X₂, …, X_n) is the vector of factors affecting the MM, Yi = (Y₁, Y₂) is the vector of categorical variables (non-MM or MM), P(Y_i) is the prior probability of event Y_i, P(X_i/Y_i) is the conditional probability, and n is the number of conditioning factors [63,65]. In this study, the Python scikit-learn module was applied to implement the NB model.

Machine Learning Hyperparameters

In this research, there is no intention to evaluate the influence of hyperparameter optimization on the results of the MMS mapping. The choice of hyperparameters has been made based on the literature, specifically on the values and configurations most used in susceptibility mapping studies, as presented in

Table 3. Hyperparameters used in this research.

Model	Hyperparameters
LR	method = “bfgs”,
MLP	lr = 0.1, architecture [1, 4, 4, 4], epochs = 1000, activation “relu”
SVM	Kernel = “linear”
RF	n_estimators = 360, max_depth = 11, criterion = “gini”, min_samples_split = 5, min_samples_leaf = 1
NB	priors = None, var_smoothing = 1 × 10−9

. As mentioned by [60], most ML models can achieve excellent accuracy with their default set of hyperparameters, as it is nearly impossible to manually search through combinations due to the countless possibilities of trial and error.

2.4.4. Step 4: Model Accuracy Evaluation

In Step 4, the MMS models were evaluated using the Receiver Operating Characteristic (ROC), the area under the curve (AUC), the F-1 score, and the accuracy (ACC).

Curve ROC y AUC

The ROC curve is a graphical representation of sensitivity (true positive rate) on the y-axis and 1-specificity (false positive rate) on the x-axis [10]. The AUC of the ROC allows for the assessment of model fit and prediction [42] and is commonly applied as a criterion for selecting the most appropriate model to determine the MMS [10]. The AUC value indicates the percentage of observed positive pixels that are correctly predicted, quantifying the probability that susceptibility models correctly classify the presence or absence of mass movement phenomena based on a set of independent variables or conditioning factors (Figure A1).

The AUC ranges from 0 to 1. AUC values between 0.5 and 0.6 are considered poor models. AUC values between 0.6 and 0.7 are considered average models. AUC values between 0.7 and 0.8 are considered good models. AUC values between 0.8 and 0.9 can be considered very good models, and an AUC value greater than 0.9 is considered an excellent model [11].

F-1 Score

The F-1 score shows the performance of the model, and high values are optimal [61]. This indicator unifies accuracy and sensitivity [15]. Accuracy is determined by dividing the true positives by the total number of pixels classified as MM. Sensitivity is the proportion of true positives predicted in the real positive class sample. Therefore, the F-1 score is the harmonic mean of precision and sensitivity [72].

Accuracy (ACC)

The ACC indicates the ratio of correct prediction to the total number of evaluation samples [71]. It is calculated as the ratio between the number of true predictions and the total number of samples in the dataset.

Cross-Validation (CV)

Cross-validation is a resampling method used to test the predictive robustness of a statistical model [73,74]. The idea is to further split the training dataset, fitting the model on one part and evaluating it on another. A data-splitting strategy of five equal parts (k = 5 folds) was used, performed randomly using the scikit-learn library in Python. After splitting the data into k-folds, the model was trained and validated k times. This procedure was applied to all ML models, obtaining the average and standard deviation of AUC (AUC-CV), F-1 score (F-1score-CV), and accuracy (ACC-CV).

2.4.5. Step 5: Hazard Mass Movement

In Step 5, the hazard was determined under two scenarios; the first scenario involved a seismic event of a magnitude greater than 8.8 Mw and the El Niño phenomenon. For the El Niño phenomenon, anomalies of the maximum accumulated rainfall values for January, February, March, and April during the last El Niño events with extraordinary rainfall in the country were used, namely 1983/84, 1997/98, 2017, and 2023 (where anomalies of precipitation ranging from 100% to 350% have been predominantly recorded within the study area) [75]. Regarding the seismic event, the seismic–geotechnical microzonation of the soil was used under a seismic scenario of 8.8 Mw, characterized into five zones related to the dynamic response of the soil to the earthquake. It is worth noting that, although the complete spatial distribution of the seismic microzonation in the study area is not available, it covers a large part of the area where the population and the livelihoods of the population are located.

To determine the hazard of MMs, the heuristic method was used. The MMS model with the highest training and evaluation metric was assigned a weight of 0.75, and each triggering variable was assigned a weight of 0.25 using the raster calculator in QGIS. Finally, the hazard model for each scenario was obtained and reclassified into quintiles, like the MMS.

3. Results

3.1. Exploration of Variables

To determine the MMS using the WoE, LR, MLP, SVM, RF, and NB methods, the correlation of the topographic variables was analyzed, and then the multicollinearity of the variables was analyzed using VIF. Figure 3 shows the Pearson correlation of topographic variables derived from a DEM.

From the figure, it is observed that there are variables that are correlated, as there are variables with a correlation coefficient ≥ 0.8. The correlation coefficient between the Terrain Roughness Index (T4) and the slope (T1) was 0.99, and the correlation coefficient between the general curvature (T7) and the profile curvature (T6) was 0.8. This indicates a high correlation between the influencing factors. Therefore, we proceeded to conduct a multicollinearity analysis using the VIF statistic (

Table 4. Multicollinearity analysis of topographic variables.

Name	Variable	VIF
Intercept	-	10.1
Slope	T1	67.3
Aspect	T2	1.5
Topographic Wetness Index (TWI)	T3	-
Terrain Roughness Index (TRI)	T4	67.3
Flow direction	T5	1.4
Profile curvature	T6	3
General curvature	T7	3

), where a value greater than 5 indicates multicollinearity issues in the fitting of susceptibility models, which are sensitive to the linear correlation of influencing factors.

From the multicollinearity result, it is observed that the slope and Terrain Roughness Index are affected by multicollinearity. Therefore, a dimensionality reduction of the variables was performed using PCA to exclude variable correlation and multicollinearity issues. In Figure 4, the contribution of the variance of the seven PCAs is observed. Assuming a variance explanation level of ≥0.65, 3 PCAs are required, resulting in 0.85 variance explanation, with each component being independent of the others.

Table 5. Weights of topographic variables in the PC.

PCA	Weights	T1	T2	T3	T4	T5	T6	T7
PCA-1	0.377	0.565	−0.102	−0.502	0.566	−0.038	0.237	0.203
PCA-2	0.250	0.240	0.104	−0.065	0.238	0.087	−0.648	−0.667
PCA-3	0.218	−0.019	−0.689	0.035	−0.034	−0.705	−0.112	−0.114

shows the importance and contribution of each influential variable in the three PCAs. The higher the value (in absolute terms), the greater the contribution to the PCA. For PCA-1, the most relevant variable was slope. For PCA-2, the general curvature variable showed the highest relevance. Lastly, orientation was the most relevant factor for PCA-3. Figure 5 displays the selected PCAs as input variables for the MMS.

Finally, the PCAs were reclassified into five quintiles, and contrast values were determined by applying WoE to both PCA and geological–environmental factors to determine the relationship between the study variables and MM phenomena; these variables were used in the WoE model and the coupled models of LR, MLP, SVM, RF, and NB.

3.2. Mass Movement Susceptibility (MMS) Modelling

The results of the MMS models were reclassified into five quintiles, very low (VL), low (L), medium (M), high (H), and very high (VH). The spatial distribution expressed in the area is shown in

Table 6. Area of MMS levels for all models.

Models	VL (km2)	L (km2)	M (km2)	H (km2)	VH (km2)
WoE	92.352	257.426	180.316	145.113	116.071
LR	157.959	162.505	157.243	153.230	162.130
MLP	128.209	192.514	170.778	152.981	146.793
SVM	158.280	162.000	156.579	155.760	160.448
RF	122.548	194.783	158.764	154.008	162.963
NB	145.511	174.507	156.579	155.573	160.895
Heuristic *	137.610	203.527	205.480	168.482	77.181

* Model of MMS generated by INGEMMET, provided by CENEPRED.

.

The highest MMS values (very high and high) were generated by the RF models, followed by the NB, SVM, and LR models, all with 20% of the surface of the study area in the very high and high susceptibility levels, as shown in Figure 6. The MMS levels by the heuristic method were generated by INGEMMET based on lithology, hydrogeology, slope, land use, and vegetation cover.

3.3. Model Validation

As indicated in the methodology section, three metrics were used to evaluate and validate the susceptibility models: the AUC value, F-1score, and ACC. The overall performance of the models was evaluated using AUC analysis [76,77]. All MMS models applied in this study, namely LR, MLP, SVM, RF, and NB, surpassed an AUC value of >0.9. The highest AUC value for the training of the models was observed in the RF model, AUC = 1.000, followed by the SVM model, AUC = 0.994, MLP, and LR, both with AUC = 0.986. Additionally, the difference between the maximum and minimum AUC values was only 1.9%. On the other hand, regarding the AUC value for the evaluation data, it was revealed that all the MMS models behave as excellent models in terms of correctly classifying the presence and absence of MM phenomena, as they all are very close to 1 (Figure 7). Finally, the CV performance is calculated as the average of the values from the k folds to clearly compare the datasets. The AUC-CV values for all different datasets of the ML models are high, averaging above 0.97.

Regarding the F-1 score value, it was determined that all the ML models surpass the F-1 score value of >0.950, with the RF model having the highest value of this metric, F-1 = 0.991. Additionally, the CV value of F-1score-CV shows values above 0.943, with the RF model presenting the highest average of 0.959. Finally, regarding the ACC of the ML models, the RF model was found to be the most accurate (ACC = 0.989); this was also confirmed in the CV, with an average ACC-CV value of 0.947.

Table 7. MMS methods and model validation metrics.

Model	AUC-Train	AUC-Test	AUC-CV	F-1 Score	F-1 Score-CV	ACC	ACC-CV
LR	0.986	1.000	0.981 ± 0.027	0.957	0.946 ± 0.021	0.952	0.946 ± 0.036
MLP	0.986	0.998	0.980 ± 0.021	0.963	0.947 ± 0.022	0.958	0.944 ± 0.039
SVM	0.994	1.000	0.979 ± 0.039	0.951	0.943 ± 0.057	0.947	0.937 ± 0.064
RF	1.000	0.996	0.981 ± 0.029	0.991	0.959 ± 0.024	0.989	0.947 ± 0.038
NB	0.981	1.000	0.980 ± 0.034	0.961	0.955 ± 0.042	0.958	0.945 ± 0.043

presents the training, evaluation, and CV metrics of the ML models generated in this study.

3.4. MM Hazard Scenarios

The MM hazard for El Niño phenomenon and earthquakes greater than 8.8 Mw was determined using the heuristic method, generated from the geospatial product between the MMS and the triggering factors. As mentioned in the previous section, the RF-derived MMS model was used because it was the model that presented the highest metrics in relation to the other models for both training and evaluation events. For both the El Niño phenomenon scenario and the earthquake exceeding 8.8 Mw, approximately 40% and 35% respectively, of the study area is classified as high and very high hazard levels. Conversely, the lowest hazard levels for both scenarios are found in the lower areas of the study region, which coincide with alluvial plains not susceptible to MMs. Figure 8 presents the hazard maps by MMs for both El Niño and the seismic scenario greater than 8.8 Mw.

4. Discussion

In this study, six quantitative MMS models were generated: the first was generated using WoE, a bivariate statistical method; and the other five were ML models, namely LR, SVM, NB, MLP, and RF, which were coupled with WoE. The models generated with ML exhibit the same spatial pattern; that is, in the elevated areas where the MMs are distributed in the study area, the highest susceptibilities are found, translating into values close to 1. This coincides with geomorphological units of mountains composed of intrusive, volcanic, or sedimentary rocks with steep slopes. From a geomechanical perspective, these rocks are fractured, altered, or weathered, generating high susceptibilities to MMs, which have been recognized by the ML models. Regarding WoE, although it captures spatial patterns similar to the ML models, in areas where the conditioning factors and the MMIs indicate zones of high probability of MMs, high and very high susceptibilities were not identified, particularly in the northern and southeastern extremes of the study area, where steep slopes and fractured, weathered rocks are present. This demonstrates that ML techniques achieve a better representation of MMS in the study area, with the RF model standing out as the one that most homogeneously represented the areas of susceptibility to MM and the conditioning factors that predispose their occurrence. The opposite is observed in the lowland areas, where values close to 0 predominate, with very low and low susceptibility levels for all MMS models; this coincides with geomorphological units of alluvial plains, aeolian deposits, and alluvial-torrential deposits with low slopes. In terms of area, around 33% to 40% of the study area for the six MMS models fall into the highest susceptibility levels, high and very high, while the lowest susceptibilities, that is, very low and low, cover a larger area, ranging from 40% to 44% of the study area. It is also noted that the trend in terms of area across all models, that is, the area covered by each level in the different ML models, is similar. Spatially, the highest susceptibilities in the study area are found on the eastern edge of the MLN, where the highest parts of the Andean foothills are located.

The validation process is crucial in MMS mapping. Several studies have suggested that an AUC value between 0.8 and 0.9 indicates a very good model, while a value higher than 0.9 indicates an excellent model [10,11]. The results revealed that all MMS models demonstrate an excellent performance in accurately predicting the presence and absence of MM phenomena during the training and evaluation of the models; among them, the RF model stands out in terms of AUC, F-1 score, and accuracy, both in the training and test data, as well as in cross-validation. For the latter, high values of AUC-CV, F-1 score-CV, and ACC-CV were obtained across different datasets. Similar results have been found in studies by [64,71,78], where the susceptibility to landslides was compared by applying different ML techniques, such as NB, k-nearest neighbors, RF, deep neural network, LR, boosted regression tree, and SVM. They found that the model with the best training and evaluation metrics is the RF model, with AUC values exceeding 0.920 in training and up to 1.000 in evaluation.

To compare the MMS results of WoE, LR, MLP, SVM, RF, and NB with the heuristic method, the susceptibility levels were standardized into five classes. Subsequently, the susceptibility levels were extracted for the point-type vectors (centroids of the test polygons) of MM in the study area. It was determined that 69.7% of the points are in the high and very high MMS levels for the heuristic model. On the other hand, for RF, SVM, LR, NB, and WoE, 97.0%, 90.9%, 90.7%, 90.9%, 87.9%, and 78.8% of the points are at the highest susceptibility levels, namely high and very high (Figure 9). The above indicates that the proposed machine learning-based models for determining MMS exhibit good performances in discriminating MM events compared to the heuristic method and that coupling statistical methods with ML models generates accurate and reliable models, as indicated by several authors [20,21,22]. This is because they are designed to automatically obtain the optimal nonlinear relationship between the study variables [17,64,78].

The hazard levels suggest that, in the event of an El Niño phenomenon, close to 40% of the surface area of the NLC would be in the highest hazard levels, high and very high. Similarly, more than half of the surface area of the Carabayllo district (54.2%) would be under the same high and very high hazard levels, followed by the districts of Comas, Independencia, and Ancon, with high and very high hazard levels of approximately 37.0%. On the other hand, Los Olivos, Puente Piedra, and San Martin de Porras have less than 8% of their surface area at the highest hazard levels. Regarding the seismic hazard scenario, the seismic microzonation did not cover the entire study area spatially; it represented only 4.9%, 24.6%, 75.3%, 63.2%, 94.3%, 79.7%, and 70.4% of the surface area of Ancon, Carabayllo, Comas, Independencia, Los Olivos, Puente Piedra, and San Martin de Porras, respectively. Therefore, the percentages shown refer to the proportion of the total area covered by the seismic microzonation spatial coverage. The districts of Ancon, Carabayllo, Comas, Independencia, and Puente Piedra have between 36% and 52% of their surface area under high and very high seismic hazard levels in the event of a magnitude greater than 8.8 Mw.

Table 8. Hazard levels for MM under El Niño phenomenon and seismic greater than 8.8 Mw.

District	El Niño Phenomenon—Hazard Level (km2)					Seismic—Hazard Level (km2)
	VL	L	M	H	VH	VL	L	M	H	VH
Ancon	38.711	73.605	80.316	47.446	69.538	1.356	5.573	2.152	6.112	0.043
Carabayllo	41.491	50.580	50.618	81.983	86.703	35.802	1.854	2.355	11.626	25.039
Comas	15.012	9.032	6.663	8.591	9.473	21.883	0.588	0.998	4.292	9.004
Independencia	5.441	0.678	3.817	5.087	0.987	4.220	0.332	0.333	2.095	3.174
Los Olivos	12.621	3.600	1.325	0.678	0.000	15.772	0.239	0.442	0.729	0.000
Puente Piedra	20.155	14.179	12.097	3.849	0.026	16.493	5.144	2.882	8.553	6.946
San Martin de Porres	26.692	6.196	2.599	0.477	0.000	26.636	1.572	2.499	2.948	0.109
Sum	160.122	157.868	157.435	148.114	166.725	122.163	15.302	11.661	36.355	44.316
%	20.3	20.0	19.9	18.7	21.1	53.2	6.7	5.1	15.8	19.3

shows the hazard levels expressed in surface area for the El Niño phenomenon and the seismic event.

In this study, MMS mapping was implemented with the purpose of identifying the areas most prone to MM, as well as to evaluate the associated hazard under two scenarios: the first one considering El Niño phenomenon and the second one considering a seismic above 8.8 Mw. Susceptibility and hazard mapping are fundamental processes in disaster risk management, as they enable the identification of areas prone to risk to propose prevention and risk reduction strategies. Therefore, errors in susceptibility and hazard mapping can lead to false conclusions, resulting in the loss of lives and livelihoods [71].

As evidenced in both this study and previous research [8,79,80], the quantitative approach based on ML techniques offers a precise and efficient methodology for processing large and complex datasets; this includes geological, topographic, hydrological, climatic, environmental, and anthropogenic factors. In contrast, classical qualitative and semi-quantitative methods determine subjective and artificial weights based on expert judgment and experience.

It is relevant to highlight that there are variables that can introduce uncertainty in the application of ML models, such as the number and type of variables, the data quality, and the number of inventories of MM and non-MM for training, among others [17,79]. In this research, the aim was not to control all variables but to maximize the available resources; therefore, the uncertainty regarding the number and type of variables was minimized by conducting a comprehensive analysis of the topographic variables in the models, including correlation, multicollinearity, and dimensional reduction using PCA. This approach allowed us to exclude variable correlations, reduce noise, and mitigate the risk of overfitting, thus improving the accuracy of the models [45,81]. Additionally, the WoE analysis was employed to identify causal relationships between instability factors and the distribution of MM. In summary, methodologies were integrated and combined to enhance MMS accuracy, resulting in hybrid models based on PCA and coupled using WoE as a foundation for ML models such as LR, MLP, SVM, RF, and NB.

However, it is important to recognize that the success of applying ML models depends on the information provided by experts and the quality of the input data. Therefore, its implementation at a national or regional scale in other territories must be carefully evaluated, ensuring proper methodological flow and the availability of high-quality inputs, especially regarding geological, topographical, and environmental factors. Additionally, it is necessary to establish an appropriate spatial resolution of the triggering factors in relation to their spatial and temporal resolution and variability.

4.1. Limitations

Regarding the limitations of this study, it is noteworthy that there is a lack of information about the triggering events of the MM, meaning that it is not specified whether they were triggered by extreme rainfall, earthquakes, anthropogenic causes, etc. Additionally, the spatial resolution of the geological, geomorphological, and hydrogeological inputs used in this study (1/100,000) may not be suitable if decisions need to be made at a detailed scale. On the other hand, the DEM used was generated at the beginning of the last decade, so there could be changes in the topography that are not considered. In the MMS models, the use of different sets of negative samples and their impact on the final MMS results were not evaluated; however, to overcome this limitation, negative samples were mapped using a hybrid approach to map negative samples not only in flat areas, as indicated in the methodology section. Additionally, it is recognized that hyperparameter optimization was not carried out, as it was not the objective of the research; however, satisfactory results were obtained in the training, evaluation, and CV metrics of the models. Finally, it is noted that there is a need for further studies to improve seismic microzonation in the study area, especially due to the longitudinal growth in the periphery of Lima.

4.2. Perspectives

In terms of future perspectives, six MMS mapping were presented, five of them based on machine learning techniques with excellent results and one based on bivariate statistics with good results. All models showed better classification metrics for MM events compared to the classic heuristic method. These ML models offer a valuable tool for disaster risk management, particularly in the processes of estimation, prevention, reduction, and reconstruction of disaster risk management. The application of ML techniques, supported by available data, has the potential to significantly improve MM zoning and, ultimately, contribute to the resilience of communities against these natural events.

5. Conclusions

Six quantitative models were constructed, trained, and evaluated to model MMS mapping: WoE, LR, MLP, SVM, RF, and NB. Their purpose was to identify the most susceptible areas to MM in the NLC and to assess the associated hazards under two scenarios: El Niño phenomenon and a seismic with a magnitude greater than 8.8 Mw. Before modelling the MMS mapping, an exploratory analysis of the variables was conducted, including correlation analysis, multicollinearity assessment, and dimensionality reduction, using PCA for the topographic variables. Subsequently, a combination of methods was applied, incorporating the WoE technique to topographic, geological, and environmental variables, which served as inputs for the ML models applied. The models were constructed, trained, and evaluated using metrics including the AUC, F-1score, and ACC. The findings demonstrate the excellent performance of the ML models, as all exhibit high metrics in training, evaluation, and CV, with the RF model standing out for its predictive capability. Additionally, the results of the quantitative susceptibility models were compared with those of a heuristic model, revealing that the latter exhibits between 10 and 20% less ability to discriminate MM events compared to the quantitative models. Regarding the hazard levels, in the event of El Niño phenomenon and a seismic event exceeding 8.8 Mw, approximately 40% and 35% respectively of the NLC area would be exposed to the highest danger levels. The rapid growth of cities on the outskirts of Lima will increase pressure on land occupation. Therefore, the findings of this research are expected to serve as a tool to support decision-makers, the technical–scientific community, and civil society in developing effective strategies for disaster prevention and risk reduction. Ultimately, this will contribute to enhancing the resilience of communities in the face of disasters.