Ensemble Model Development for the Prediction of a Disaster Index in Water Treatment Systems

Abstract

The quantitative analysis of the disaster effect on water supply systems can provide useful information for water supply system management. In this study, a total disaster index (TDI) was developed using open-source public data in 419 water treatment plants in Korea with 23 input variables. The TDI quantifies the possible effects or damage caused by three major disasters (typhoons, heavy rain, and earthquakes) on water supply systems. The four components (regional factor, risk factor, urgency factor, and response and recovery factor) were calculated using input variables to determine the disaster index (DI) of each disaster. The weight of the input variables was determined using principal component analysis (PCA), and the weights of the DI of three natural disasters and four components used to calculate the TDI were determined by the analytical hierarchy process (AHP). Specifically, two ensemble machine learning models, random forest (RF) and XGBoost (XGB), were used to develop models to predict the TDI. Both models predicted the TDI with the coefficient of determination and root-mean-square error-observations standard deviation ratio of 0.8435 and 0.3957 for the RF model and 0.8629 and 0.3703 for the XGB model, respectively. The relative importance analysis suggests that the number of input variables can be minimized, which improves the models’ practical applicability.

1. Introduction

Various natural disasters, such as floods and earthquakes, cause considerable damage to water supply systems. This damage includes the destruction of plants, intake systems, pipelines, and electric systems, and the consequent interruption of water supply to the public [1]. The assessment of damage to water supply systems caused by natural disasters is important for proper management and decision-making processes to prevent and restore the damage caused by natural disasters [2,3].

Assessing risk and measuring disaster resilience are the keys for predicting possible events, quantifying contributing factors, and identifying potential consequences. One good example is the lone house that remained standing after Hurricane Ike in 2008. It was rebuilt based on the experience from Hurricane Rita in 2005 on elevated ground, with an appropriate roof pitch and windows that were designed to withstand winds of up to 209 km/h, thus surviving Hurricane Ike with its winds of 177 km/h [4]. Although there have been many efforts to develop quantitative and indicator-based assessments, such as the comprehensive disaster resilience index (CDRI) [5,6], there is no universal standard for the measurement of disaster and related consequences [7]. A reliable disaster resilience framework with unified terminology and its quantitative evaluation would be an important tool in the decision-making processes for both policymakers and engineering professionals [8].

Statistical methods such as principal component analysis (PCA) or analytic hierarchy process (AHP) are often applied for the evaluation of disaster effects on civil infrastructures. For example, Park et al. [9] suggested a disaster risk index for 51 high-speed railroad stations in Korea. The index was calculated from a linear equation of four main indices (hazard, exposure, vulnerability, and emergency response and recovery capability) suggested by Rossi and Gilmartin [10] where the weights of each main index were determined by PCA. Recent studies have also used statistical analysis based on survey data for the assessment of disaster risk on flooding or water security [11,12].

In recent decades, advanced technologies of data mining and machine learning (ML) have been used to manage disasters such as typhoons and earthquakes [13,14,15,16,17,18,19], and various emerging remote sensing technologies have also been increasingly used for monitoring and detecting data related to disaster managements [20]. The continuous increase in available data due to advanced data collection technologies, such as remote sensors or unmanned aerial vehicles (UAV), has accelerated the application and accuracy of ML models [21,22,23]. Ofli et al. [21] used aerial images captured from UAVs for identifying features of interest such as damaged shelters and blocked loads to assist with disaster response. The features of interest in the image were annotated and used for training ML models, including support vector machine (SVM) and random forest (RF), where the overall accuracy of the model’s classification results ranges from 0.73 to 0.85. Sheykhmousa et al. [24] analyzed satellite images of land cover and land use using an SVM classifier. The image data during Typhoon Haiyan, which caused massive damage in the Philippines in 2013, was compared with the image data in 2017, four years after Typhoon Haiyan, to assess the post-disaster recovery process. Chen et al. [15] analyzed the impact indices of flood disasters using RF and developed a risk assessment model based on the neural network method. Various data, including rainfall and socioeconomic data in the Yangtze River Delta area between 2008 and 2018, were used for model development. More recently, Kao et al. [19] used an advanced deep learning algorithm, long short-term-memory (LSTM), to forecast flood events.

Recent studies also used social platforms with text information about disasters to analyze the characteristics of disasters such as typhoons and earthquakes [14,25,26]. Resch et al. [25] analyzed earthquake characteristics from social media information during an earthquake in Napa, California, USA, in 2014 using latent Dirichlet allocation (LDA), which is widely used for topic analysis. The spatial hot spots of the earthquake were determined from the LDA model with 86.45% accuracy compared with the United States Geological Survey earthquake footprint report. More recently, Yu et al. [14] analyzed text information in social media during Typhoon Anemone along the coast of China in August 2012 to develop a typhoon disaster classification system using a model based on a convolutional neural network.

ML is also increasingly used in environmental management. Zhang et al. [27] predicted air pollution by PM 2.0 with a fusion model based on three gradient boosted decision tree (GBDT) algorithms. The root-mean-squared error (RMSE) of the fusion model was 32.300. Bi et al. [28] also used a GBDT-based model, light gradient boost, coupled with a fast Fourier transform for the assessment of a liquefaction disaster. However, even with substantial efforts on the classification and analysis of disaster, its quantitative and indicator-based assessments on the water infrastructure have not been thoroughly conducted.

In this study, the effects of various disasters on water supply systems from the perspective of management are quantified by statistical data analysis methods, PCA, and analytic hierarchy process (AHP). From the statistical approach, a total disaster index (TDI) was developed. In the second part, tree-based ensemble models (i.e., RF and GBDT) were used to predict TDI, which provides valuable information for the safety management of water supply systems.

2. Methods

2.1. Data Sources

Total 23 input variables of facility specification and operational data in 419 water treatment plants in Korea were used to develop a TDI. The data were obtained from statistical yearbooks and open-source public data (

Table 1. Data sources.

Data	Reference
Water treatment plant operational information and facility specification	Statistical yearbook of water treatment system [32]
Meteorological data	Meteorological administration information portal [30]
Financial status in local government	Ministry of the interior and safety information portal [31]
Design standard for wind speed	Korean Design Standard [33]
Local peak ground acceleration by earthquake	Korea Seismicity Map [29]

). The 23 input variables provide information about the water supply systems, including water supply capacity, pipeline density, number of customers, management labor, and regional characteristics of natural conditions where the water treatment plants are located (

Table 2. Input variables.

Variables	Description
CUSTOMER	Population that receives drinking water from the water treatment plant
EMPLOYEES_AREA	Employees per management area of the authority * (person/km2)
EMPLOYEES_SITE	Employees per number of water treatment plants of the authority * (person/ea)
EQ1	Seismic design application status (applied: 1, not applied: 0)
LINE_DENSE	Total pipeline length per management area of the authority * (m/km2)
LOCAL_EMPLOYEES	Number of employees in the water supply plant
MONEY	Financial independence of the local government (%)
PGA_500	500 years frequency peak ground acceleration (%)
PGA_1000	1000 years frequency peak ground acceleration (%)
PGA_2400	2400 years frequency peak ground acceleration (%)
PUMP	Number of water supply pumps in the water treatment plant (ea)
PUMP_EP	Sum of electrical capacity of all water supply pumps in the water supply plant (kW)
Q	Water supply capacity of the water treatment plant (m3/day)
Q_DAILY	Daily average water production capacity of the water treatment plant (m3/day)
Q_MAX	Daily maximum water production capacity of the water treatment plant (m3/day)
Q_PRO	Total annual water production capacity of the water treatment plant (m3)
QT	Maximum water supply capacity per hour of the water treatment plant (m3/hr)
QW	Total annual electric power usage of the water treatment plant (kWh)
QY	Total annual amount of water treated by the water treatment plant (m3/year)
RAIN	Number of flood warning advisories between 2015 and 2019 in the region where the water treatment plant is located (times/km2)
SWIND	Number of strong wind advisories between 2015 and 2019 in the region where the water treatment plant is located (times/km2)
TYPHOON	Number of typhoon warning advisories between 2015 and 2019 in the region where the water treatment plant is located (times/km2)
WIND_R	Regional standard wind speed (m/s) in the region where the water treatment plant is located

* authority: the owner of the water supply plant (i.e., the local government) and one authority may manage multiple plants.

). The local peak ground acceleration by an earthquake at each water treatment plant was estimated from the Korea Seismicity Map program developed by Cao et al. [29]. The data for regional natural conditions were obtained from meteorological data available from the national meteorological administration information portal [30]. The financial status of a local government that manages the water treatment plant was collected from the public data portal of the Ministry of the Interior and Safety in Korea [31].

2.2. Disaster Index

2.2.1. Type of Disaster

Typhoons and heavy rains are among the most frequent disasters in Korea, while Korea has been known to be relatively safe from earthquakes. However, interest in earthquakes in Korea has increased since the two earthquakes with magnitudes of 5.8 and 5.4 on the Richter scale in 2016 and 2017, respectively. In this study, three natural disasters, typhoons, heavy rains, and earthquakes, were selected as the most influential disasters on the water supply system and used for the TDI development considering natural characteristics in Korea.

2.2.2. Component of Disaster Index

The four components (i.e., regional factor, risk factor, urgency factor, and response and recovery factor), describing the level of the damage caused by each type of disaster, were used to determine the disaster index (DI) of three natural disasters as follows.

• Regional factor (RE) represents regional characteristics such as the frequency of natural disaster occurrence in the selected areas;
• Risk factor (RI) represents the quantity of possible damage caused by natural disasters. For example, the RI increases as the capacity of water treatment plants or the length of water supply pipelines increases;
• Urgency factor (UR) represents the urgency of recovery after a disaster. For example, the UR increases with a larger population in the area receiving drinking water; and,
• Response and recovery factor (RR) represents the recovery ability during and after a disaster, which is estimated by the financial status or manpower of the authority of a water treatment plant, such as the local government.

A total of 23 input variables obtained from open-source public data were used to determine the four components (RE, RI, UR, and RR), as summarized in Table 2.

2.2.3. PCA Analysis for Index Weight

The weights of each variable for the DI of three natural disasters (typhoon, heavy rain, and earthquake) were determined using PCA. PCA is a statistical method that reduces the dimension of variables and determines each variable’s relative importance using an eigenvector. The input variables were standardized as an average of zero and standard deviation of one for PCA analysis [34,35,36].

2.2.4. AHP Analysis

The DI of three natural disasters and four components were used for the calculation of TDI in 419 water treatment plants. However, there was limited data available for the statistical determination of the relative weight of each natural disaster and four components for the TDI calculation. In addition, although the effect of earthquakes on the water supply system is expected to be extremely large, there were only two significant earthquakes in Korea that occurred in 2016 and 2017. Thus, it should be noted that quantitative data for the analysis of the effect of earthquakes was limited.

The weights of three natural disasters and four components used to calculate the TDI were determined by the AHP suggested by Saaty [37,38]. The AHP is a structured data analysis method for complex decision-making, which is also widely used to analyze disaster data [9,23,39]. In AHP, a pairwise comparison matrix of each element for the decision-making process is structured. This structure relates to the matrix’s eigenvector, which represents the weight of each element in the decision-making process [37,40].

The survey results from 62 experts or engineers currently working in water treatment plants were used for AHP analysis. The survey data with a consistency ratio (CR) of less than 0.2 was used to calculate the weight of each input variable to maintain the consistency of the AHP analysis result [40,41,42].

$$\mathrm{CI}=\frac{{\lambda }_{max}-n_f}{n_f-1}\mathrm{and}\mathrm{CR}=\frac{\mathrm{CI}}{\mathrm{RI}}$$

(1)

where

${\lambda }_{max}$: principal eigenvalue in the pairwise comparison matrix,

n_f: number of features,

CI: consistency index,

RI: random consistency index (RI = 0.90 for n = 4 and RI = 0.58 for n = 3), and

CR: consistency ratio.

2.2.5. Disaster Index Model

The TDI is determined by the weighted sum of the DI for three natural disasters using the following equations (Equations (2)–(5)).

TDI = a(TI) + b(HI) + c(EI)

(2)

TI = a_t(RE_t) + b_t(RI_t) + c_t(UR_t) − d_t(RR_t)

(3)

HI = a_h(RE_h) + b_h(RI_h) + c_h(UR_h) − d_h(RR_h)

(4)

EI = a_e(RE_e) + b_e(RI_e) + c_e(UR_e) − d_e(RR_e)

(5)

where

TI: DI for typhoon;

HI: DI for heavy rain;

EI: DI for earthquake;

a, b, and c: weight of each natural DI; and

a_t, b_t, c_t, d_t, a_h, b_h, c_h, d_h, a_e, b_e, c_e, and d_e: weight of each component.

Subscripts (i.e., t, h and e) from Equations (3)–(5) represents typhoon, heavy rain, and earthquake respectively.

2.3. Disaster Prediction Model

2.3.1. Model Selection

Two ensemble models, RF and GBDT, have been increasingly used as ML models to manage the water environment. Both models show good performance, even for nonlinear relationship analysis, and data with outliers are also applicable for both classification and regression [43,44].

RF is a tree-based ensemble model in which a random data selection approach generates multiple decision trees. RF randomly selects several sets of input features from the original input features by a bagging method before generating the decision trees, which increases the independence and variability of each decision tree. The final RF prediction is determined by averaging the predictive results from individual decision trees in RF [45]. Consequently, the prediction performance of RF can be dramatically improved [46,47,48] and outperforms other ML models [49]. RF has shown high performance in various domains and has also been continuously applied to environmental research, such as water quality prediction [50,51].

GBDT is an ensemble model based on a gradient boosting method (GBM), called a sequential tree-based calculation process [45,52,53], and a set of decision trees. Unlike RF which determines the final prediction by voting (for classification) or averaging (for regression), GBDT uses the decision tree, called a weak learning model, from a previous stage in the ML process to improve model performance in the following stage. Residual errors of the prior stage are included in developing the decision tree in the current stage to reduce the residual errors by optimizing a specified loss function [45,52]. This optimization process is sequentially performed until the predefined number of decision trees is reached, which is a major difference with RF, where the calculation of each tree is independent.

GBDT is optimized by minimizing an objective function, J, for a training data set with n samples. The regularization term can be added to avoid overfitting of the model [44,54]. Equation (6) shows an illustrative example of the objective function of GBDT [44,54].

$$\mathrm{J}={\sum }_{i=n}^{n}L\left(y_i,{\widehat{y}}_i\right)+{\sum }_{k=1}^K\Omega \left(f_k\right)$$

(6)

where

$f_k$: function of the kth decision tree,

$L$: loss function that calculates the difference between an observation ($y_i$) and model prediction (${\widehat{y}}_i$) in each decision tree,

$\Omega$: regularization function that penalizes the complexity of the model, and

n: number of data samples.

The schematics of RF and GBDT are compared in Figure 1, where X denotes input features as X = x₁, x₂, …, x_n, h(X, θ_k), (k = 1, 2, …, K) is a collection of decision trees, and the ${\theta }_k$ are independent and identically distributed random vectors [44,45,54].

In this study, both RF and GBDT models were used for the TDI estimation of 419 drinking water treatment plants. The Python open-source libraries of Scikit-learn (for RF) and XGBoost (for GBDT) were used for regression model development [55,56]. XGBoost (XGB) is one of the most popular GBDT implementations developed by Chen and Guestrin [45,54]. Scikit-learn is also a popular Python-based ML library developed by Pedregosa et al. [55].

2.3.2. Model Optimization

The hyperparameters of RF and XGB were optimized by a trial and error method with ten-fold cross-validation using the grid search library in Scikit-learn [57]. The models were developed with 23 input variables of 419 water treatment plants, where the ratio of data used for training and testing of the models was 8:2.

2.3.3. Feature Importance (FI) of Input Variables

The relative importance of input variables on RF and XGB model performance was calculated using the feature importance (FI) algorithm in Scikit-learn [57]. The FI in the tree-based model was computed as the total impurity reduction of the model brought by that feature [55,58,59].

2.3.4. Model Evaluation

The model performance was evaluated by three evaluation indexes (Equations (7)–(9)), RMSE, coefficient of determination (R²), and RMSE-observation standard deviation ratio (RSR). RSR ranges from 0 to 1 and approaches 0 when the model shows a good fit with observation. The model is considered to predict the observation when RSR < 0.70 [60,61].

$$R^2=1-\frac{{{\displaystyle \sum }}_{i=1}^n{\left(M_{i,obs}-M_{i,model}\right)}^2}{{{\displaystyle \sum }}_{i=1}^n{\left(M_{i,obs}-\overline{M_{i,obs}}\right)}^2}$$

(7)

$$RMSE=\sqrt{\frac{{{\displaystyle \sum }}_{i=1}^n{\left(M_{i,obs}-M_{i,model}\right)}^2}{n}}$$

(8)

$$RSR=\frac{\sqrt{{{\displaystyle \sum }}_{i=1}^n{\left(M_{i,obs}-M_{i,model}\right)}^2}}{\sqrt{{{\displaystyle \sum }}_{i=1}^n{\left(M_{i,obs}-\overline{M_{i,obs}}\right)}^2}}$$

(9)

where

$M_{i,obs}$: observed values,

$\overline{M_{i,obs}}$: mean of observed values, and

$M_{i,model}$: model predicted value.

3. Results and Discussion

3.1. Characteristics of Input Variables

Total 23 input variables for the development of DI were identified from open-source public statistical data. The characteristics of the input variables are summarized in

Table 3. Characteristics of input variables.

Variables	Average	Max	Min	Standard Deviation
CUSTOMER	89,902.749	3,030,917.000	0.000	276,768.491
EMPLOYEES_AREA	0.195	3.017	0.006	0.449
EMPLOYEES_SITE	25.819	304.333	0.833	50.505
EQ1	0.222	1.000	0.000	0.416
LINE_DENSE	2718.442	25,817.066	274.779	3812.635
LOCAL_EMPLOYEES	8.043	134.000	0.000	16.030
MONEY	18.972	78.450	4.020	15.198
PGA_500	8.532	11.000	3.199	1.792
PGA_1000	11.515	14.000	5.431	1.884
PGA_2400	16.695	19.000	9.222	1.959
PUMP	2.852	176.000	0.000	9.093
PUMP_EP	444.527	14,400.000	0.000	1637.036
Q	48,083.905	1,600,000.000	30.000	141,017.670
Q_DAILY	30,932.308	1,081,369.000	6.000	90,686.216
Q_MAX	37,629.160	1,221,400.000	20.000	106,454.278
Q_PRO	11,278,843.986	394,699,507.000	2288.000	33,103,181.424
QT	1818.253	69,120.000	0.000	6465.596
QW	2,406,152.687	78,414,779.000	0.000	7,988,910.560
QY	11,623,330.642	402,072,337.000	3276.000	33,936,442.393
RAIN	0.084	0.900	0.009	0.123
SWIND	0.077	0.813	0.002	0.108
TYPHOON	0.016	0.060	0.001	0.013
WIND_R	29.084	44.000	24.000	5.143

. The frequency of warning advisories of natural disasters was calculated at each water treatment plant from the sum of the three variables in Table 3 (i.e., RAIN, SWIND, and TYPHOON). The frequency of warning advisories ranged from 0.017 to 1.29 times/km² and tended to be higher in areas near the ocean as shown in Figure 2 using ArcGIS pro.

3.2. Disaster Index (DI) Model Development

3.2.1. PCA Analysis

The weights for each natural disaster index were determined from PCA with 23 input variables (

Table 4. PCA analysis for weight of each component.

Disaster (Index)	Component	Input Variable (Symbol)	Weight
Typhoon (TI)	REt	WIND_R (REt1)	0.309
		TYPHOON (REt2)	0.345
		SWIND (REt3)	0.346
		sum	1.000
	RIt	Q (RIt1)	0.143
		QY (RIt2)	0.143
		Q_PRO (RIt3)	0.144
		PUMP (RIt4)	0.052
		PUMP_EP (RIt5)	0.140
		QT (RIt6)	0.132
		QW (RIt7)	0.136
		LINE_DENSE (RIt8)	0.110
		sum	1.000
	URt	Q_DAILY (URt1)	0.334
		Q_MAX (URt2)	0.334
		COSTUMER (URt3)	0.332
		sum	1.000
	RRt	LOCAL_EMPLOYEES (RRt1)	0.248
		MONEY (RRt2)	0.235
		EMPLOYEES_SITE (RRt3)	0.263
		EMPLOYEES_AREA (RRt4)	0.254
		sum	1.000
Heavy rain (HI)	REh	WIND_R (REh1)	0.500
		RAIN (REh2)	0.500
		sum	1.000
	RIh	Q (RIh1)	0.143
		QY (RIh2)	0.143
		Q_PRO (RIh3)	0.144
		PUMP (RIh4)	0.052
		PUMP_EP (RIh5)	0.140
		QT (RIh6)	0.132
		QW (RIh7)	0.136
		LINE_DENSE (RIh8)	0.110
		sum	1.000
	URh	Q_DAILY (URh1)	0.334
		Q_MAX (URh2)	0.334
		COSTUMER (URh3)	0.332
		sum	1.000
	RRh	LOCAL_EMPLOYEES (RRh1)	0.248
		MONEY (RRh2)	0.235
		EMPLOYEES_SITE (RRh3)	0.263
		EMPLOYEES_AREA (RRh4)	0.254
		sum	1.000
Earthquake (EI)	REe	PGA_500 (REe1)	0.333
		PGA_1000 (REe2)	0.336
		PGA_2400 (REe3)	0.331
		sum	1.000
	RIe	Q (RIe1)	0.143
		QY (RIe2)	0.143
		Q_PRO (RIe3)	0.144
		PUMP (RIe4)	0.052
		PUMP_EP (RIe5)	0.140
		QT (RIe6)	0.132
		QW (RIe7)	0.136
		LINE_DENSE (RIe8)	0.110
		sum	1.000
	URe	Q_DAILY (URe1)	0.334
		Q_MAX (URe2)	0.334
		COSTUMER (URe3)	0.332
		sum	1.000
	RRe	LOCAL_EMPLOYEES (RRe1)	0.234
		EQ1 (RRe2)	0.056
		MONEY (RRe3)	0.222
		EMPLOYEES_SITE (RRe4)	0.249
		EMPLOYEES_AREA (RRe5)	0.239
		sum	1.000

). The eigenvectors were calculated from PCA and normalized to make the sum of weights of each component to be 1.

3.2.2. AHP Analysis

The weights for each disaster type were determined from the AHP analysis using the survey data (CR < 0.2) (

Table 5. Analytical hierarchy process (AHP) analysis results.

(a) Weights for Disaster Type.
Disaster	Weight
Typhoon	0.481
Heavy rain	0.198
Earthquake	0.321
Sum	1.000
CR	0.054
(b) Weights for Each Component.
Disaster	Component	Weight
Typhoon	REt	0.275
	RIt	0.265
	URt	0.216
	RRt	0.244
	Sum	1.000
	CR	0.017
Heavy rain	REh	0.279
	RIh	0.247
	URh	0.221
	RRh	0.253
	Sum	1.000
	CR	0.004
Earthquake	REe	0.215
	RIe	0.370
	URe	0.235
	RRe	0.180
	Sum	1.000
	CR	0.040

). The response rate of the survey was in the range between 52 and 69% for each item. The weights of each disaster are in the order of typhoons, earthquakes, and heavy rain.

3.2.3. Disaster Index (DI)

The TDI was determined using the following model (Equations (10)–(13)) which were developed from PCA and AHP analysis (Table 4 and Table 5).

TDI = 0.481(TI) + 0.198(HI) + 0.321(EI)

(10)

TI = 0.275(RE_t) + 0.265(RI_t) + 0.216(UR_t) − 0.244(RR_t)

(11)

where

RE_t = 0.309(RE_t1) + 0.345(RE_t2) + 0.346(RE_t3),

RI_t = 0.143(RI_t1) + 0.143(RI_t2) + 0.144(RI_t3) + 0.052(RI_t4) + 0.140(RI_t5) + 0.132(RI_t6) + 0.136(RI_t7) + 0.110(RI_t8),

UR_t = 0.334(UR_t1) + 0.334(UR_t2) + 0.332(UR_t3), and

RR_t = 0.248(RR_t1) + 0.235(RR_t2) + 0.263(RR_t3) + 0.254(RR_t4).

HI = 0.279(RE_h) + 0.247(RI_h) + 0.221(UR_h) − 0.253(RR_h)

(12)

where

RE_h = 0.500(RE_h1) + 0.500(RE_h2),

RI_h = 0.143(RI_h1) + 0.143(RI_h2) + 0.144(RI_h3) + 0.052(RI_h4) + 0.140(RI_h5) + 0.132(RI_h6) + 0.136(RI_h7) + 0.110 (RI_h8),

UR_h = 0.334(UR_h1) + 0.334(UR_h2) + 0.332(UR_h3), and

RR_h = 0.248(RR_h1) + 0.235(RR_h2) + 0.263(RR_h3) + 0.254(RR_h4).

EI = 0.215(RE_e) + 0.370(RI_e) + 0.235(UR_e) − 0.180(RR_e)

(13)

where

RE_e = 0.333(RE_e1) + 0.336(RE_e2) + 0.331(RE_e3),

RI_e = 0.143(RI_e1) + 0.143(RI_e2) + 0.144(RI_e3) + 0.052(RI_e4) + 0.140(RI_e5) + 0.132(RI_e6) + 0.136(RI_e7) + 0.110(RI_e8),

UR_e = 0.334(UR_e1) + 0.334(UR_e2) + 0.332(UR_e3), and

RR_e = 0.234(RR_e1) + 0.056(RR_e2) + 0.222(RR_e3) + 0.249(RR_e4) + 0.239(RR_e5).

Using the developed models, TDI values of 419 water treatment plants were determined with the range between −0.526 and 3.813 with an average of 0 and a standard deviation of 0.343. A higher TDI represents a higher potential of effect or damage by a disaster in water treatment systems. The TDI tends to be higher in water treatment plants near metropolitan cities as well as the areas near ocean.

The TDI was developed considering the natural status of Korea. For example, there were only two earthquakes in 2016 and 2017, which were considered to have caused actual damage to water treatment plants in Korea. As the data available for the quantification of damage by earthquakes is minimal, the AHP based on survey data was used for the DI calculation.

Although there were not many cases of damage in water treatment systems from earthquakes, the weight of the earthquake was larger than that of heavy rain. The AHP results represent that, although earthquakes have been rare in Korea, the damage and consequences by an earthquake would not be negligible when it occurs, indicating that a preventive plan against earthquakes should be prepared in advance. In addition, given that most of the facilities already experience heavy rain and are relatively well prepared for these instances, it is expected that the actual damage caused by heavy rain is relatively small compared to other disasters.

3.3. Ensemble Model Simulation

3.3.1. Total Disaster Index (TDI) Prediction using Ensemble Models

Two ensemble ML models, RF and XGB, were used to develop a model to predict TDI. The model performance with the test data set was evaluated by three indices, as summarized in

Table 6. Summary of model evaluation results.

Model	RMSE	R2	RSR
RF	0.100	0.8435	0.3957
XGB	0.093	0.8629	0.3703

. The R² and RSR were 0.8435 and 0.3957 for the RF model and 0.8629 and 0.3703 for the XGB model, respectively.

The observed data and model predictions are compared in Figure 3. The model prediction shows a similar good fit with observations both in the RF and XGB models, while XGB showed a slightly better performance for all three evaluation indexes (Table 6 and Figure 3).

3.3.2. Feature Importance (FI) Analysis

The FI of 23 input variables for both RF and XGB models to predict DI are shown in Figure 4. The FI was different between RF and XGB, while the variables that represent the scale of water treatment plants such as PUMP_EP and Q tend to have a higher effect on model performance for both models. For RF, the sum of FI in the highest nine input variables was more than 80%, while for XGB, the sum of FI in the highest four variables was more than 80% of the total FI for XGB.

The performance of the models was compared between RF and XGB using fewer input variables, starting with 1 and adding up to 10 input variables with the order from the highest FI (Figure 5). The RF model showed a tendency to improve the performance of the model as the number of input variables increased from one to ten, and even when using three input variables, the RSR was 0.6954, indicating that the model accurately predicted the observation. XGB shows better performance when using fewer input variables. The RSR is 0.5323 when only three input variables were applied, which reduces to 0.3937 when using ten input variables. The FI analysis shows that several input variables with higher feature importance have a considerable effect on model performance. The analysis results show that both the RF and XGB models show similar performance when using five or more input variables with higher FI. The FI is one of the factors and not an absolute standard considered for model structure. The necessary input variables are not always obtainable from the actual operation and management of water treatment systems. Thus, the practical applicability of the model would be improved as fewer input variables are used. The FI analysis suggests that the model shows acceptable performance if only part of the input variables with the highest FI would increase the practical applicability of the model.

4. Summary and Conclusions

In this study, a disaster index (DI) for predicting the effect or damage caused by three major natural disasters in Korea (i.e., typhoons, heavy rain, and earthquakes) was newly developed to quantify each natural disaster’s effect on water utilities.

Although the operational data in water utilities provided a good understanding regarding the effect of disasters, the data is usually collected in an individually specified format often site-specific, making it difficult to collect, organize, and analyze the data. In addition, the operational data for water utilities was not easily accessible, limiting the comprehensive development of the DI. Therefore, in this study, the DI of natural disasters in water treatment systems was developed using statistical open-source public data. Two well-defined statistical data analysis methods (i.e., AHP and PCA) were used for the determination of DI.

The open-source public data have greater accessibility and are updated regularly, so the DI can also be updated considering the current status, which is also a significant benefit of using open-source public data. The DI developed in this study may be site-specific at a given location and conditions of water utilities, but the developed framework would be applicable for quantifying the effect of disasters on water treatment systems in other regions with different natural status.

In the second part, two ensemble models (i.e., RF and XGB) were used to develop models to predict TDI. Both RF and XGB showed similar satisfactory performance for prediction of the DI, while the XGB showed a slightly better performance in general. The FI analysis also suggested that the models have sufficient performance for practical use with only several input variables of the highest FI, which can improve the practical applicability of the models.

Quantitative assessment of disaster effects on water treatment systems is essential for better management of the water treatment systems and stable supply of drinking water to the public. However, data related to disaster analysis are often limited and even hardly quantifiable. One of the possible solutions would be to keep collecting data, analyze them statistically, while facilitating frequent discussions from experts experiencing the disasters in their utilities [11,35]. The recent advance of information and communication technologies, such as sensor-based real-time monitoring methods, can provide various continuous monitoring data about the operational condition of water treatment plants and related infrastructure, which can improve the pre- and post-management planning processes [20,22]. However, the quantification and assessment of disasters on water treatment systems are still in an early stage, and the use of field operational data and responses, in particular during disaster events, is currently limited at this time.

This study provided quantified information on the impact of various natural disasters on water treatment systems with open-source public data, which would be useful for creating a plan to reduce damage to water supply systems caused by natural disasters. Further study is warranted to use high-frequency real-time data to improve the model performance and practical applicability.