A Global Resilience Analysis-Based Benchmark Framework for Comparing Reliability Surrogate Measures of Water Distribution Systems

Abstract

Various reliability surrogate measures have emerged over the last three decades to design water distribution systems. However, existing comparative studies cannot assess surrogate measures from the resilience perspective considering the dynamic absorption–recovery process imposed by pipe failures. In this work, we propose a novel benchmark framework based on the global resilience analysis to examine surrogate measures’ performance. Surrogate measures were compared via the stress–strain curve derived from the global resilience analysis under extended period simulation. In particular, we identify the comparable stress range to articulate the differences among surrogate measures and significantly reduce the computational burden. Then, we develop the normalized resilience score (NRS) to evaluate the quality of solutions to network design. Five well-known measures are compared for the multiobjective design of two benchmark networks. Results show that the Network Resilience Index achieves 2.5% to 10.1% better NRSs than the mean NRSs over five surrogate measures, implying that both nodal surplus energy and pipe diameter uniformity greatly impact the network system’s resilience. The uniformity of pipe diameters is more significant than the uniformity of flow rate. Our findings contribute to the design of new and better surrogate measures for network resilience evaluation.

1. Introduction

In the 2030 Agenda for Sustainable Development, all member states of the United Nations (UN) agree to achieve 17 goals (well known as SDGs) for the period 2015 to 2030. Among these, the 11th goal is to make cities and human settlements inclusive, safe, resilient, and sustainable. As one of the most important underground infrastructures, distribution systems play a key role in ensuring reliable and resilient water supply service from treatment plants to various types of customers in cities and towns worldwide. However, water distribution systems (WDSs) always face various uncertain threats (e.g., continuously ageing pipelines and demand surge due to extreme heat wave); therefore, improving system performance is of great significance to meet the 11th SDG. Performance indicators can be expressed in terms of reliability [1], vulnerability [2], resilience [3], availability [4], robustness [5], etc. Among these, reliability is widely used to guide WDS design tasks [6,7,8]. In a narrow sense, reliability of WDSs includes hydraulic and mechanical reliability [9], in which the former and the latter refer to the system’s reliability in coping with demand fluctuations and components failure events, respectively. Due to the computationally hard nature of reliability, using surrogate measures is a popular workaround.

Over the last three decades, researchers have proposed various surrogate measures. Well-known surrogate measures include flow entropy (FE) [10], resilience index (RI) [11], network resilience index (NRI) [12], modified resilience index (MRI) [13], diameter sensitive flow entropy (DSFE) [14], redundancy (REDU) [15], available power index (API), pipe hydraulic resilience index (PHRI) [16], combined entropy-resilience index (CERI) [17], and combined entropy-network resilience index (CENRI), to name a few. The surrogate measures are calculated using two main network features: nodal surplus energy [18] and entropy [19]. Thus, existing surrogate measures can be categorized into three groups: (1) energy-based, including RI, NRI, MRI, REDU, API, and PHRI; (2) entropy-based, including FE and DSFE; and (3) combined measures, such as CERI and CENRI.

However, identifying a good enough surrogate measure is still challenging for researchers and practitioners. Raad et al. [20] assessed FE, RI, NRI, and a mixed reliability index using three benchmark networks, making correlation analysis on the Pareto-optimal set from optimization. None of the measures were satisfactory to represent both hydraulic and mechanical reliability, although RI and NRI performed better than FE. Later, Liu et al. [16] proposed API and PHRI, and compared these two new measures with RI, NRI, MRI, and DSFE under demand and pipe failure uncertainties. They found that energy-based measures performed better than entropy-based ones. Wang et al. [21] compared a collection of surrogate measures by establishing a many-objective optimization model. They considered single-pipe failure mode to analyze the correlation between mechanical reliability and surrogate measures, revealing that energy-based measures were more consistent with reliability and cost than entropy-based ones. The REDU was recommended due to its satisfactory performance, which is closer to other energy-based measures with the lowest computational overhead. Sirsant et al. [17] proposed two surrogate measures, CERI and CENRI. They investigated the correlation between surrogate measures and reliability in four networks under demand fluctuations and single-pipe failure scenarios, revealing that CERI performed better when facing uncertain threats in different network structures.

As mentioned above, previous comparative studies leave some research gaps. First, they considered only single-pipe failure scenarios, neglecting multipipe failure scenarios due to Black Swan events (e.g., extremely-low-temperature weather or earthquakes) that have catastrophic effects. Second, few studies compared the surrogate measures’ performance from the decision-making perspective in practice, that is, conducting the cost–benefit analysis at a set of comparable cost levels. Third, the pipe failure scenarios were time-independent, which cannot reflect the absorption–recovery process over the disturbance period.

In the last decade, resilience analysis has attracted increasing interest in academia. Resilience is an extension of reliability, considering more pipe failures and dynamic system performance over time. Diao et al. [22] proposed global resilience analysis (GRA) to acquire dynamic system performance over time in three failure modes (i.e., pipe burst, fire, and contaminant intrusion). Later, Meng et al. [23] enriched the GRA framework, considering a key aspect called the recovery stage during the failure period. They extracted six strain indicators to reflect network performance, including time to strain, duration, failure rate, recovery rate, magnitude, and severity. Therefore, comparing surrogate measures under the GRA framework can reflect the dynamic impact of external threats over the disturbance period. Furthermore, performance evaluation based on resilience analysis, rather than the traditional reliability analysis, strengthens the connections between academia and the 11th SDGs.

The specific objective of this study is to establish a novel GRA-based benchmark framework for comparing reliability surrogate measures. This study also attempts to address the question of why some surrogate measures perform better than others from the resilience perspective. This paper’s novelty lies in its provision of a new decision-making tool for the design of a more resilient water distribution system. This study also provides insights into developing better surrogate measures.

2. Materials and Methods

2.1. The Global Resilience Analysis-Based Benchmark Framework

Figure 1 demonstrates the flowchart of the proposed benchmark framework, based on the global resilience analysis (GRA) [22,23]. The benchmark framework consists of four main steps.

First, a collection of reliability surrogate measures was selected as needed. A set of multiobjective optimization models, taking the minimization of cost and maximization of each surrogate measure as objective functions, were set up and solved by NSGA-II [24] to obtain the Pareto-optimal front of each surrogate measure. At the design stage, decisionmakers usually favor more economical design solutions. As such, three cost levels (i.e., low, medium, and high) were chosen within the low-cost range, where a small cost increase significantly improved network performance. As a result, a set of representative solutions were generated from the intersections between each cost level and the Pareto-optimal front of each surrogate measure.

Then, global resilience analysis was conducted to determine the impact of pipe failure scenarios on the target network through extended period simulation, generating the network performance curve and stress–strain curves. A set of pipe failure scenarios, i.e., progressively increasing external stresses, were assumed to happen to water supply of a representative design solution during the peak demand period, mimicking the worst situation during a daily operation. The corresponding adaptive behavior of a network system under failure scenarios was quantified by six strain indicators [23], including time to strain (TTS), duration (DUR), failure rate (FR), recovery rate (RR), magnitude (MAG), and severity (SEV), derived from the network performance curve in Figure 2.

This curve illustrates the system’s capabilities for absorption, recovery, and adaptation in response to external stresses [25]. As shown in the network performance curve in Figure 2, the minimum performance corresponds to the worst performance moment during the pipe failure period, specifically at time T₂. The difference between the threshold and minimum performance represents the magnitude of the failure event. The threshold can be defined as 80% of normal water supply [17,26], below which the entire network is regarded as breakdown. Some pipes are assumed to burst at time T₀. Between T₀ and T₁, the network exhibits a high absorptive capability and withstands disturbances, with performance ranging between the normal and threshold. This period is referred to as time to strain. The network begins to fail at time T₁. From T₁ to T₃, the network performance remains below the threshold, indicating the duration of network failure. The area, enclosed by the network performance curve and the threshold, represents the severity of the failure event. The rate at which the network performance decreases from the threshold to the minimum during time T₁ to T₂ is known as the failure rate, while the rate at which it recovers from the minimum to the threshold during time T₂ to T₃ is termed the recovery rate.

As such, three stress–strain curves (i.e., min, mean, and max) for each strain indicator were obtained after running the global resilience analysis for each representative network solution. Although the stress–strain curves may vary significantly among different scenarios, this paper used only the mean stress–strain curves to compare surrogate measures, avoiding the bias caused by focusing on extreme events with very low probabilities.

Next, the comparable stress range (CSR) was identified. As with increasing the stresses in the global resilience analysis, the representative solutions of surrogate measures had different strain values. The whole system would malfunction when the stress reached a critical threshold. In that case, the difference of strain indicators was not significant (i.e., relative range less than 5%), which had no need to continue to compare surrogate measures. At each cost level, the same stress had different effects for six strain indicators. Therefore, the comparison among surrogate measures focused on the common range between the minimum stress and critical stress across all strain indicators based on the mean stress–strain curves of surrogate measures.

Finally, the normalized resilience score (NRS) of each surrogate measure was calculated within the comparable stress range at each cost level. The strain indicators were first combined into a normalized resilience score at each stress (see Equations (7)–(9) below). Afterwards, a network design solution’s resilience equals the average normalized resilience scores within the comparable stress range.

Note that this study only considered the threats imposed by pipe failure events, which are the most critical events for network resilience [3], ranging from single-pipe failure to a set of multipipe failures. The proposed framework can adapt to other types of threats, such as demand fluctuations and contamination intrusion.

2.2. Simulation-Optimization Approach

The multiobjective optimization model was set up for each selected surrogate measure to obtain the Pareto fronts. The objective was to design a resilient network while minimizing cost and maximizing a surrogate measure. The decision variables are options for commercial pipe diameters. The constraints contain implicit constraints, such as mass balance at nodes and energy conservation in loops, and explicit constraints, such as minimum nodal heads. The definition of the network design problem can be expressed as follows:

$$\mathit{minimize}: {obj}_1=\sum _{j=1}^{np}u\left(d_j\right)\times L_j$$

(1)

$$\mathit{maximize}: {obj}_2=Surrogate measure$$

(2)

$$s.t.\left\{\begin{array}{c}{H}_i\ge H_i^{req} \\ d_j\in D=d_1,d_2,...,{ d}_{max}\end{array}\right.$$

(3)

where $d_j$ is the diameter of pipe j; $u\left(d_j\right)$ is the unit cost of pipe j depending on the pipe diameter; $L_j$ is the length of pipe $j$; $D$ denotes the collection of commercially available pipe diameters; ${ d}_{max}$ denotes the maximum diameter option; $H_i$, and $H_i^{req}$ are the actual and required head at node $i$, respectively.

The NSGA-II algorithm [24] can solve the optimization models effectively and provide alternative solutions in the trade-off between cost and each surrogate measure. Wang et al. [27] proved that it uses less computation and finds a Pareto front closer to the true one than other algorithms such as AMLAGAM, Borg, etc. Some adjustable parameters can influence the quality of Pareto fronts, including population size, number of function evaluations, crossover probability, mutation probability, crossover distribution index, and mutation distribution index. Fine-tuning these parameters will approach the actual Pareto front [28].

2.3. Comparable Stress Range

After running the global resilience analysis, three stress–strain curves were obtained for each strain indicator. However, only the mean stress–strain curve was used to continue the comparison. At a given cost level, the surrogate measures’ stress–strain curves had a convergence phenomenon (see Step 3 in Figure 1). Specifically, the divergent strains gradually converged into the same level as the stress increased. The relative range was used to describe the convergence degree of a numerical series of strains by their minimum, mean, and maximum values [29,30] (see Equation (4)). When it is below or equal to 5%, it indicates sufficient convergence within the dataset [31]. This study uses the comparable stress range (CSR) to distinguish the strains of representative solutions identified by different surrogate measures. The CSR’s upper bound is called the critical stress, and its lower bound is the stress of single-pipe failure. The formulas are as follows.

$$rr={\displaystyle \frac{max\left\{{\epsilon }_s\right\}-min\left\{{\epsilon }_s\right\}}{mean\left\{{\epsilon }_s\right\}}}\times 100\%$$

(4)

$$\left\{\begin{array}{c} rr\ge 5\%, {\sigma }_i\le {\sigma }_c\\ rr<5\%, {{\sigma }_i>\sigma }_c\end{array}\right.$$

(5)

$$CSR=[{\sigma }_1,\mathit{min}\left\{{\sigma }_c\right\}]$$

(6)

where ${\epsilon }_s$ means the strain by surrogate measure $s$ at a specific cost level and stress, and $rr$ is the relative range; ${\sigma }_i$ means the $i$th stress, equal to the percentage of burst pipes to a total number of pipes, ranging from 0 to 100%; ${\sigma }_c$ is the critical stress, beyond which the strains by surrogate measures have a relative range of less than 5%; ${\sigma }_c$ depends on the strain indicators and cost levels, and the minimum of ${\sigma }_c$ is the upper bound of the comparable stress range; ${\sigma }_1$ is the lower bound of the comparable stress range, meaning the single-pipe failure. The strain indicator should be excluded when its critical stress is equal to the lower bound of the comparable stress range. In this case, the differences among various surrogate measures from the viewpoint of this strain indicator vanish.

2.4. Normalized Resilience Score

After identifying the comparable stress range, the normalized resilience score (NRS) was calculated for surrogate measures’ representative solutions. The strain scores (SS), ranging from 0 to 100, were calculated based on the strains at critical stress. The higher the strain score is, the better the network system’s resilience achieves. The average of strain scores represents the normalized resilience score at a stress, and the average of normalized resilience scores across the comparable stress range is the network system’s resilience score of a representative solution. The equations are as follows.

$${SS}_{ij}=100\times \left(1-{\displaystyle \frac{{\epsilon }_{ij}}{{\epsilon }_{cj}}}\right)$$

(7)

$${NRS}_i={\displaystyle \frac{1}{m}}\sum _{j=1}^m\left(1-{\displaystyle \frac{{\epsilon }_{ij}}{{\epsilon }_{cj}}}\right)$$

(8)

$$NRS={\displaystyle \frac{1}{n}}\sum _{i=1}^n{NRS}_i$$

(9)

where ${\epsilon }_{ij}$ and ${\epsilon }_{cj}$ represent the values of strain indicator $j$ at stress $i$ and critical stress $c$, respectively; ${SS}_{ij}$ is the strain score between 0 and 100 for the strain indicator $j$ at stress $i$; ${NRS}_i$ denotes the normalized resilience score at stress i; m is the number of strain indicators; A solution’s $NRS$ is the average of all normalized resilience scores within the comparable stress range, ranging from 0 to 1; A higher NRS means better performance; n is the number of stress levels.

2.5. Surrogate Measures Selected for Comparison

Our aim is to identify the best formula of surrogate measure that reflects a resilient network. To this end, five surrogate measures were selected from the literature, which are good representatives in different categories, including energy-based, entropy-based, and combined measures. The selected energy-based measures were network resilience index [12], redundancy [15], and pipe hydraulic resilience index [16]. The selected entropy-based measure was diameter-sensitive flow entropy [14]. The last selected measure was combined entropy-resilience index [17]. These selected measures are most effective in their categories for designing reliable networks, as proved in comparative studies [14,16,17,21,32].

Each category of surrogate measures has distinct formulas to capture network characteristics. In energy-based measures, we identified three formulas for characterizing system energy: summing the head of each node, summing the pressure of each node, and summing the head loss in each pipe. Only one formula was identified in both entropy-based measures and combined measures. The entropy-based measures focus on flow distribution within networks, while the combined measures represent a weighted sum of energy- and entropy-based measures.

2.5.1. Energy-Based Surrogate Measures

The first measure, resilience index (RI), describes energy transfer, including input, dissipation, and surplus power, in looped networks [11]. A network requires more nodal surplus energy to improve its ability to respond to random disasters. A higher resilience index means the network is safer to maintain customers’ water supply needs. The detailed formula can be expressed as follows.

$$RI={\displaystyle \frac{\sum _{i=1}^{nn} Q_i\left(H_i-H_i^{req}\right)}{\sum _{r=1}^{nr} Q_r{H}_r+\sum _{k=1}^{npu} \left(\frac{P_k}{\gamma }\right)-\sum _{i=1}^{nn} Q_i{H}_i^{req}}}$$

(10)

where $nr$, $npu$, and $nn$ are the numbers of reservoirs, pumps, and demand nodes, respectively; $Q_r$ and $H_r$ are the discharge and the head of reservoir $r$, respectively; $P_k$ is the power input by pump $k$, and $\gamma$ is the water-specific weight; $Q_i$ is the demand of node $i$.

However, the resilience index only considers nodal surplus energy, ignoring diameter uniformity. Later, Prasad and Park [12] proposed the network resilience index (NRI) by adding a nodal diameter uniformity coefficient to RI, which can express the uniformity of pipe diameters connected to each node. The equations to calculate NRI are as follows.

$$NRI={\displaystyle \frac{\sum _{i=1}^{nn} U_i{Q}_i\left(H_i-H_i^{req}\right)}{\sum _{r=1}^{nr} Q_r{H}_r+\sum _{k=1}^{npu} \left(\frac{P_k}{\gamma }\right)-\sum _{i=1}^{nn} Q_i{H}_i^{req}}}$$

(11)

$$U_i={\displaystyle \frac{\sum _{j=1}^{n{p}_i} d_{ij}}{n{p}_i\times max\left\{d_{ij}\right\}}}$$

(12)

where ${np}_i$ is the number of pipes connected to node i; $U_i$ is diameter uniformity connected to node $i$; $d_{ij}$ is diameter of the pipe from node $i$ to $j$.

Liu et al. [15] proposed the nodal head redundancy (REDU) without water demand to express the ability to absorb failures or fluctuations. The formula is as follows.

$$REDU={\displaystyle \frac{1}{nn}}\sum _{i=1}^{nn}{\displaystyle \frac{P_i-P_{min,i}}{P_{max,i}-P_{min,i}}}$$

(13)

where $P_i$, $P_{min,i,}$, and $P_{max,i}$ represent the actual, minimum, and maximum pressures at node $i$, respectively.

Liu et al. [16] proposed the pipe hydraulic resilience index (PHRI), adopting a pipe’s hydraulic gradient between upstream and downstream nodes to reflect source friction losses to users. Less energy consumption of pipes will provide more nodal surplus energy for downstream nodes. The equations are as follows.

$$PHRI={\displaystyle \frac{\sum _{j=1}^{np} \left(H_{ds,j}-H_{req}\right)L_{pro,j}}{\sum _{j=1}^{np} \left(H_{us,j}-H_{req}\right)L_{pro,j}}}$$

(14)

$$L_{pro,j}=\sqrt{L_j^2-{\left(Z_{us,j}-Z_{ds,j}\right)}^2}$$

(15)

where $np$ is the number of pipes; $H_{req}$ is the required head of the system; $H_{ds,j}$ and $Z_{ds,j}$ mean nodal head and elevation at the downstream end of pipe $j$; $H_{us,j}$ and $Z_{us,j}$ are corresponding values at the upstream end of pipe $j$; $L_j$ and $L_{pro,j}$ are the actual length and projected length of pipe $j$, respectively.

2.5.2. Entropy-Based Surrogate Measures

Tanyimboh and Templeman [10] proposed the flow entropy (FE) to design a resilient network using a probability distribution to estimate the degree of uncertainty. Networks with higher entropy values have fewer uncertainties and more easily cope with unpredictable conditions such as demand fluctuation and component failure randomly. The flow entropy reflects the multiple available supply paths. When a specific path fails, another can transport water to meet the required nodal demand [25]. The formula is as follows.

$$FE=-\sum _{r=1}^{nr}{\displaystyle \frac{Q_r}{T}}\mathit{ln}\left({\displaystyle \frac{Q_r}{T}}\right)-{\displaystyle \frac{1}{T}}\sum _{i=1}^{nn} T_i\left[{\displaystyle \frac{Q_i}{T_i}}\mathit{ln}\left({\displaystyle \frac{Q_i}{T_i}}\right)+\sum _{i=1}^{{nps}_i}{\displaystyle \frac{q_{ij}}{T_i}}\mathit{ln}\left({\displaystyle \frac{q_{ij}}{T_i}}\right)\right]$$

(16)

where $T_i$ is the sum of flow to node $i$; $T$ is the total flow supplied from all sources; $T_i/T$ is the ratio of the inflow reaching node $i$ to the total flow of sources; ${nps}_i$ means the pipe set starting from node $i$; $q_{ij}$ means pipe flow from node $i$ to $j$.

Liu et al. [14] proposed the diameter-sensitive flow entropy (DSFE) by combining flow entropy with pipes’ velocities. The improvement is an additional coefficient (i.e., $v_C/v_{ij}$) in the last term of flow entropy, reflecting the diameter connected to nodes. The equation is as follows.

$$DSFE=-\sum _{r=1}^{nr}{\displaystyle \frac{Q_r}{T}}\mathit{ln}\left({\displaystyle \frac{Q_r}{T}}\right)-{\displaystyle \frac{1}{T}}\sum _{i=1}^{nn} T_i\left[{\displaystyle \frac{Q_i}{T_i}}\mathit{ln}\left({\displaystyle \frac{Q_i}{T_i}}\right)+\sum _{i=1}^{{nps}_i}{\displaystyle \frac{v_C}{v_{ij}}}·{\displaystyle \frac{q_{ij}}{T_i}}\mathit{ln}\left({\displaystyle \frac{q_{ij}}{T_i}}\right)\right]$$

(17)

where $v_C$ is the velocity constant and equals 1 m/s; $v_{ij}$ represents the flow velocity from node $i$ to $j$.

2.5.3. Combined Surrogate Measures

Sirsant and Reddy [17] integrated the flow entropy [7] and resilience index [8] into a combined entropy-resilience index (CERI), considering pipe diameter uniformity and nodal surplus energy. The measure performed well in networks with different scales and loop degrees, as assessed by scenarios of demand fluctuations and component failures. The equation is as follows.

$$CERI=w_1\cdot \left({\displaystyle \frac{FE}{{FE}_{max}}}\right)+w_2\cdot \left({\displaystyle \frac{RI}{{RI}_{max}}}\right)$$

(18)

where ${FE}_{max}$ depends on the specific network; ${RI}_{max}$ is constantly equal to 1. The sum of weights $w_1$ and $w_2$ equals 1; both vary from 0 to 1. The recommended $w_1$ and $w_2$ are equal to 0.4 and 0.6, respectively.

3. Case Studies

3.1. Benchmark Networks

This research chose two benchmark networks to compare the selected surrogate measures. The network layouts are shown in Figure 3.

The first well-known case is the Hanoi network (HAN) from Fujiwara and Khang [33], which contains 34 pipes organized in 3 loops, 31 demand nodes, and 1 reservoir with a fixed head of 100 m. The Hazen–Williams roughness coefficients of all pipes are equal to 130. All nodes require a head above the ground elevation at least 30 m. Six commercial pipe sizes range from 304.8 mm to 1016.0 mm. Therefore, the problem has a vast search space equal to 6³⁴ ≈ 2.87 × 10²⁶ discrete combinations.

The second case is a real-world network adapted from a town named Fossolo (FOS) in Italy [34]. The network includes 58 pipes, 36 demand nodes, and 1 reservoir with a fixed head of 121.0 m. All the polyethene pipes have a considerably high roughness coefficient of 150. All demand nodes have the same minimum pressure head requirement equal to 40 m, but each node has a different maximum pressure head. In addition, the flow velocity in each pipe is enforced to be no more than 1 m/s. There are 22 commercially available diameters ranging from 16.0 mm to 409.2 mm; thus, the search space equals 22⁵⁸ ≈ 7.25 × 10⁷⁷ discrete combinations.

3.2. Experimental Setup

An actual demand pattern derived from the real-world network [35] was adopted to run an extended period simulation under pipe failure conditions for both network cases. Only the period from 19:00 to 22:00 was chosen to mimic the pipe(s) failure, which was a three-hour breakdown with peak demand in the network [22].

The optimization models were established for each selected surrogate measure in HAN and FOS cases. In order to obtain convergent Pareto fronts, the optimization parameters of NSGA-II were fine-tuned according to [28]. Some parameters were the same in two cases: population size of 100, crossover probability of 0.9, crossover distribution index of 20, and mutation distribution index of 20. The mutation probabilities were equal to the inverses of the number of pipes in two cases, which were 1/34 and 1/58, respectively. The number of generations were set to 500 and 1000 for HAN and FOS cases, respectively, to ensure sufficient convergence. To eliminate the effect of random initial populations, 30 independent optimization runs were carried out to obtain an aggregate set of Pareto front solutions. Then, we used the fast nondominated sorting [24] to generate the Pareto-optimal front for each surrogate measure. The WNTR [36] was used to run hydraulic simulations under pipe failure scenarios, and the NSGA-II in Pymoo [37] was used to complete the optimization calculation.

After obtaining the Pareto fronts, the newly proposed resilience-based benchmark framework was adopted to compare surrogate measures. The normalized resilience score (NRS) was calculated using the global resilience analysis within the comparable stress range. In network simulation of pipe failure scenarios, the pressure-dependent analysis was conducted rather than the demand-driven analysis to obtain more accurate nodal demand [38]. Valves were assumed to be installed following the N-rule, meaning that each pipe has two valves at both ends and can ultimately be isolated from the network [39]. Figure 2 shows detailed information on the global resilience analysis. The traditional reliability-based framework was also used for comparison purposes, but only the single-pipe failure was simulated. The corresponding network performance indicator under the reliability-based framework was the mechanical reliability score (MRS) [18], which represents the level of water demand satisfaction at all nodes in the event of pipe failures, ranging from 0 to 1. The formulas are as follows.

$${DS}_{ij}=\left\{\begin{array}{cc}1,& P_{ij}\ge P_{req,ij}\\ {\displaystyle \frac{P_{ij}}{P_{req,ij}}},& P_{min,ij}<P_{ij}<P_{req,ij}\\ 0,& P_{ij}\le P_{min,ij}\end{array}\right.$$

(19)

$$MRS={\displaystyle \frac{1}{npf}}\sum _{j=1}^{npf} \sum _{i=1}^{nn} w_{ij}\times {\mathrm{DS}}_{ij}$$

(20)

where $nn$ is the number of nodes, and $npf$ is the number of scenarios considered in pipe failure analysis. In single-source networks, some pipes directly connected to the source should be excluded because the failure of these pipes will inevitably result in the breakdown of the entire system. ${DS}_{ij}$ is the nodal demand score of node $i$ under pipe failure scenario $j$, depending on the relationship between the actual nodal pressure $P_{ij}$ and the required and minimum pressures $P_{req,ij}$,$P_{min,ij}$, defined in the pressure-dependent analysis. $w_{ij}$ is the ratio of actual demand at node $i$ to the total demand under pipe failure scenario $j$.

4. Results and Discussion

4.1. Comparison between Resilience- and Reliability-Based Frameworks

4.1.1. Pareto Fronts Obtained by Various Surrogate Measures

Five surrogate measures’ Pareto fronts were obtained by solving the optimization models of network design in two cases, as shown in Figure 4. Three cost levels were selected within the low-cost range, including USD 6.75 million, USD 7.25 million, and USD 7.75 million in the HAN case and EUR 0.06 million, EUR 0.18 million, and EUR 0.30 million in the FOS case. The intersections of the three cost lines with Pareto fronts are the low, medium, and high solutions. Next, surrogate measures were evaluated by calculating the reliability and resilience of the representative solutions.

4.1.2. Representative Solutions’ Performance under Pipe Failure Scenarios

The five surrogate measures were compared by the resilience-based and reliability-based frameworks among the representative solutions of two network cases, as shown in

Table 1. HAN case’s solution performance based on reliability and resilience framework.

Benchmark Framework	Surrogate Measure	Low-Cost Solution		Medium-Cost Solution		High-Cost Solution
Mechanical reliability score (MRS)	NRI	0.907	2.5%	0.922	4.1%	0.958	4.9%
	DSFE	0.885 *	0.0%	0.886 *	0.0%	0.913 *	0.0%
	REDU	0.897	1.4%	0.914	3.2%	0.940	3.0%
	PHRI	0.896	1.2%	0.909	2.6%	0.915	0.2%
	CERI	0.916	3.5%	0.937	5.8%	0.951	4.2%
Normalized resilience score (NRS)	NRI	0.673	16.8%	0.741	15.8%	0.764	13.7%
	DSFE	0.576 *	0.0%	0.640 *	0.0%	0.672 *	0.0%
	REDU	0.632	9.7%	0.718	12.2%	0.746	11.0%
	PHRI	0.642	11.5%	0.698	9.1%	0.679	1.0%
	CERI	0.689	19.6%	0.765	19.5%	0.754	12.2%

Note: The MRS and NRS in bold indicate the best performance, and the MRS and NRS with an asterisk (*) indicate the worst performance. Relative increases in percentages were calculated based on worst performance.

and

Table 2. FOS case’s solution performance based on reliability and resilience framework.

Benchmark Framework	Surrogate Measure	Low-Cost Solution		Medium-Cost Solution		High-Cost Solution
Mechanical reliability score (MRS)	NRI	0.998	3.5%	1.000	0.5%	1.000	0.0%
	DSFE	0.965 *	0.0%	0.995 *	0.0%	1.000	0.0%
	REDU	0.975	1.1%	1.000	0.5%	1.000	0.0%
	PHRI	0.976	1.2%	1.000	0.5%	1.000	0.0%
	CERI	0.999	3.6%	0.999	0.4%	1.000	0.0%
Normalized resilience score (NRS)	NRI	0.784	42.3%	0.902	35.0%	0.915	20.2%
	DSFE	0.551 *	0.0%	0.668 *	0.0%	0.788	3.5%
	REDU	0.595	8.0%	0.792	18.6%	0.886	16.4%
	PHRI	0.589	6.9%	0.800	19.8%	0.885	16.3%
	CERI	0.758	37.6%	0.780	16.8%	0.761 *	0.0%

Note: The MRS and NRS in bold indicate the best performance, and the MRS and NRS with an asterisk (*) indicate the worst performance. Relative increases in percentages were calculated based on worst performance.

. The traditional reliability was only calculated under single-pipe failure scenarios [17,21]. With drastic climate change and continuous pipeline deterioration, multipipe failure is likely to happen [40]. The reliability-based framework that only considered single pipe failure scenarios cannot cope with such a situation, where the differences among various surrogate measures vanish due to the insignificant threats. In other words, it is necessary to extend to multipipe failure scenarios for benchmarking surrogate measures under vital stresses.

Comparison by the resilience-based framework effectively captured the performance differences of the surrogate measures in networks with different loop degrees. For instance, in the low-cost solutions of the HAN case, the mechanical reliability scores of NRI and CERI were 2.5% and 3.5% higher than DSFE respectively. In contrast, their corresponding values of the normalized resilience scores were 16.8% and 19.6% higher than DSFE. The resilience-based framework amplified the performance differences by additional multipipe scenarios. This phenomenon was more significant in a network with higher loop degree like the FOS case. In the low-cost solutions, the mechanical reliability scores of NRI and CERI were 3.5% and 3.6% higher than DSFE respectively, while their corresponding values of the normalized resilience scores were 42.3% and 37.6% higher than DSFE. As with increasing the cost levels to medium-cost and high-cost in the FOS case, using the mechanical reliability scores failed to distinguish performance differences of surrogate measures (i.e., most mechanical reliability scores equal to one in Table 2). However, using the normalized resilience scores can effectively distinguish performance differences of surrogate measures in networks with more loops. For instance, the performance of NRI was 20.2% higher than CERI in the high-cost solution of the FOS case. Compared using the resilience-based framework, NRI was the best surrogate measure while DSFE was the worst one. Notably, CERI had significant performance fluctuations in the FOS case, which performed second best in the low-cost solution but worst in the high-cost solution.

4.2. Comparison of Surrogate Measures by the Resilience-Based Framework

4.2.1. Comparable Stress Range of Representative Solutions

In the resilience-based framework, the comparable stress range was from one single pipe failure (SPF) to a critical number of failed pipes, where the strain indicators had significant differences (i.e., more than 5% in this study). Due to multiple strain indicators, the minimum critical stress among six strain indicators was set as the upper bound of the comparable stress range, as shown in Figure 5. The time to strain (TTS) was excluded for comparison because the corresponding critical stresses of this strain indicator were always equal to the values obtained under single-pipe failure scenarios.

As seen from Figure 5, the strain indicators have different critical stresses among three cost levels. The recovery rate has the highest critical stress, and the duration has the lowest critical stress in the two cases. As increasing cost levels, the critical stresses of some strain indicators (i.e., duration, magnitude, and severity) remained within 5% variation. The critical stresses of the other strain indicators, such as failure rate and recovery rate, had greater variations over 5%. For instance, the critical stresses for recovery rate were 21.875%, 37.5%, and 25% at the low, medium, and high costs in the HAN case, respectively. The critical stresses for recovery rate rose to 61.404%, 64.912%, and 77.193% at the low, medium, and high costs in the FOS case, respectively.

A network with more loops had a wider comparable stress range. The CSR was from 3.125% to 15.625% in the HAN case, meaning the failure scenarios from one pipe to five pipes. In contrast, the CSR was from 1.754% to 40.351% in the FOS case, meaning the failure scenarios from 1 pipe to 23 pipes. Networks with fewer loops (e.g., HAN) have less redundancy of flow paths from water source to demand nodes; thus, a small number of failed pipes would have a significant impact on network strain indicators. In contrast, a highly looped network (e.g., FOS) has more water supply paths, requiring more simultaneous pipe failures to compare surrogate measures.

4.2.2. Normalized Resilience Scores of Selected Surrogate Measures

The normalized resilience scores of selected surrogate measures were compared among representative solutions in the HAN and FOS cases, as shown in Figure 6. The NRI had overall higher NRSs than the mean values (red dashed lines) of surrogate measures in two cases with different loop degrees. A previous study [17] compared the mechanical reliability and hydraulic reliability of surrogate measures and found that NRI was an excellent surrogate measure, mainly used for designing medium-scale networks. The CERI was sensitive to network topology. Specifically, it was superior to the other surrogate measures in the HAN case but performed poorly in medium- and high-cost solutions in the FOS case. The overall scores of REDU and PHRI were close to the mean level and worse than NRI and CERI. The DSFE performed worst overall among the selected surrogate measures.

Identifying a good enough surrogate measure is significantly essential in network optimization design. The complex network model can be mathematically understood as coupling topological and hydraulic information [41]. Since such information is used to calculate surrogate measures, it is noteworthy to discuss the relationship between surrogate measures’ resilience scores and network meta-information (shown in

Table 3. Network meta-information for calculating surrogate measures.

	Node Meta Information	Link Meta Information
TopologicalInformation	Node count	Pipe count
Hydraulic Information	Required demand, actual demand, head, and pressure	Flowrate, diameter, velocity, length, and head loss

). The overall NRSs of surrogate measures and the corresponding meta-information used to calculate them are demonstrated in

Table 4. NRSs of surrogate measures compared with network meta-information.

Surrogate Measure	HAN Case’s NRS	FOS Case’s NRS	Node Information			Link Information
			$\mathit{n}\mathit{n}$	$\mathit{Q}$	$\mathit{H}\left(\mathit{P}\right)$	$\mathit{n}\mathit{p}$	$\mathit{q}$	$\mathit{d}$	$\mathit{v}$	$\mathit{L}$
NRI	0.726	0.867	√	√	√	√		√
DSFE	0.629 *	0.669 *	√	√		√	√		√
REDU	0.699	0.758	√		√
PHRI	0.673	0.758			√	√				√
CERI	0.736	0.766	√	√	√	√	√

Note: $nn$ = number of nodes; $Q$ = required or actual demand; $H\left(P\right)$ = head (pressure); $np$ = number of pipes; $q$ = flowrate; $d$ = diameter; $v$ = velocity; $L$ = length. The NRS in bold indicates the best performance, and the NRS with a star (*) indicates the worst performance.

.

A network system’s resilience depends on some necessary network meta-information. The optimal NRI had the number of nodes, demand, head, number of pipes, and diameter, reflecting essential resilience features of nodal surplus energy and diameter uniformity. The CERI focused on flow uniformity rather than diameter uniformity, which was worse than NRI in the highly looped network (i.e., FOS). Based on the results presented in Figure 6 and Table 4, pipe diameter turns out to be more important than flow rate to maintain the network system’s resilience. The REDU and PHRI were based on nodal pressure, which lacked the demand to assess the node criticality. The entropy-based measure DSFE ignored the nodal head or pressure, which had the worst network resilience scores. Therefore, we suggest that decisionmakers use NRI with the best performance to obtain a rational network layout with a proper diameter distribution.

As more evidence, Jeong and Kang [42] proposed a hydraulic uniformity index (HUI) using only pipe information without any node information, including the number of pipes, flow rate, length, and head loss. We also tested this surrogate measure in WDS design tasks, but failed to obtain a satisfactory Pareto front as other surrogate measures (see Figure 4). Existing studies found that nodal meta-information is vital in network resilience evaluation [43]. The exclusion of node metainformation made the surrogate measure lose the ability to capture the most critical nodes with higher demand. As such, when new surrogate measures are proposed, the necessary network meta-information should be fully considered.

4.2.3. Impact of Stress on WDS’s Resilience

The normalized resilience score was calculated at each stress within the comparable stress range of surrogate measures, as shown in Figure 7. The HAN and FOS cases showed significant declining trends in network system’s resilience when stress increased. The HAN’s stress–NRS curve of each surrogate measure showed a nearly linear decline. Under single-pipe failure, normalized resilience scores of surrogate measures were all close to 1 among representative solutions. Then, the average scores of surrogate measures decreased to 0.26, 0.39, and 0.39 at critical stress of 15.625% in three cost solutions, respectively.

The FOS stress–NRS curve of each surrogate measure showed a nonlinear decline, especially in medium- and high-cost solutions. In three cost solutions, normalized resilience scores were almost equal to 1 under single-pipe failure for all surrogate measures, which decreased slowly at lower stress. When the stress increased to greater than 10%, normalized resilience scores showed evident differences among surrogate measures. When the stress reached the critical point (i.e., 40.351%), the average scores of surrogate measures decreased to 0.23, 0.33, and 0.41 in three cost solutions, respectively.

4.2.4. Impact of Stress on Network Strain Indicators

The impact of stress on network strain indicators was examined within the comparable stress range of surrogate measures. Except for time to strain, the remaining five strain indicators were assessed by strain scores (see Equation (7)) among representative solutions in two cases, as shown in Figure 8 and Figure 9. In strain-score radars, a larger pentagon means the surrogate measure performed better. Again, in both cases, NRI showed consistently better performance compared with the other surrogate measures, especially when coping with higher stresses. As stresses increased, the five strain scores decreased simultaneously with different degrees. The DUR strain shrank most significantly due to having the minimum critical stress (see Figure 5), followed by the FR and RR strains. The MAG and SEV strains had similar shrinking trends but with smaller magnitude. Additionally, the pentagons associated with different surrogate measures did not intersect with each other, implying that various strain indicators deteriorated in a consistent way.

Networks with more loops can effectively differentiate network strain indicators at higher stresses. In the HAN case with fewer loops, the strain scores of surrogate measures were all between 80 and 100 at the minimum stress, most of which were close to the full score of 100. In the FOS case with more loops, all strain scores were equal to 100 at minimum stress. As the stress increased to the median of the comparable stress range, the FOS case had a more significant difference among strain scores. The maximum gap for the two surrogate measures was 20 in the HAN case (i.e., CERI and DSFE’s medium-cost solution) and 60 in the FOS case (i.e., NRI and DSFE’s low-cost solution).

By increasing cost, the network layout can significantly improve strain performance at higher stresses within a lower cost range. This phenomenon is not evident in single-pipe failure scenarios at the minimum stress but is noticeable at the median and critical stresses. In the FOS case, PHRI’s DUR score at the median stress was 40, 80, and 100 in low-, medium-, and high-cost solutions; in contrast, NRI’s DUR score at the critical stress was 5, 30, and 40 in three cost solutions. In both networks, we found a more significant increase in strain scores from low to medium costs. It suggests that decisionmakers should focus on the cost–benefit analysis on solutions in the lower cost range of Pareto fronts, which have higher marginal benefits of WDS resilience. The benchmark framework proposed in this study provides an effective tool for this purpose.

4.3. Implications of Results and Their Practical Significance

Based on the results demonstrated above, the insights provided for researchers and practitioners are twofold. First, upgrading the traditional reliability-based benchmark framework to the resilience-based framework is necessary to distinguish various surrogate measures for network system performance evaluation rationally. A superior surrogate measure identified by the resilience-based framework can effectively tell decisionmakers how to cope with pipe failure events by enhancing the network system layout. This will, in turn, benefit the vast and costly underground infrastructure maintenance activities with a limited budget, especially in adapting to the SDGs set by the United Nations under ageing infrastructure threats.

Second, this study makes a substantial step forward by pointing out the correlation between the network meta-information and the formulas of various selected surrogate measures. Despite the difficulty of quantifying such a correlation, it at least tells which meta-information plays a key role and which should not be used. This may inspire the creation of new and better surrogate measures for network system resilience assessment in a deterministic manner rather than random tests.

5. Conclusions

This paper developed a novel benchmark framework based on the global resilience analysis to compare reliability surrogate measures for the design of water distribution systems. The proposed framework assessed five selected surrogate measures using two well-known cases, i.e., the Hanoi and Fossolo networks. The main conclusions are drawn as follows.

(1) Based on the global resilience analysis, this paper establishes a novel framework for comparing reliability surrogate measures, which reveals the variations of network strains with increasing stresses (triggered by pipe failure events with different magnitude) at multiple cost levels. Compared to existing benchmark frameworks dependent on snapshot simulation, the global resilience analysis-based framework considers a network’s dynamic response to external threats (e.g., pipe burst) by extended period simulation, thus accurately evaluating the rationality of surrogate measures. The normalized resilience score obtained from the proposed benchmark framework provides an essential decision-support tool for network layout design. We demonstrate that using the normalized resilience score can magnify the differences of surrogate measures than the commonly used methods, helping decisionmakers screen out more reasonable surrogate measures in the literature. Thus, the proposed global resilience analysis-based benchmark framework is recommended to replace the existing decision-making approaches for WDS design and rehabilitation purposes. This new approach contributes to the water research community as a more cost-effective decision-making methodology in adapting to the sustainable development goals set by the United Nations under ageing infrastructure threats.

(2) The comparable stress range plays a critical role in the proposed benchmark framework. On the one hand, it captures the stress range in which the surrogate measures are distinct, which helps to articulate the performance differences of the surrogate measures. On the other hand, it significantly reduces the computational burden by identifying the critical stress threshold. As such, there is no need to enumerate all scenarios from single-pipe failure to all-pipe failure. This feature benefits the comparative studies on large-scale networks with high loop degrees.

(3) Under the proposed benchmark framework, the surrogate measure considering nodal surplus energy and pipe diameter uniformity (i.e., NRI) correlates highly with network system’s resilience. The correlation between the surrogate measures that consider only nodal energy (i.e., REDU, PHRI) and network system’s resilience declines, suggesting that reasonable surrogate measures should consider not only nodal energy but also the distribution of pipe diameters. The surrogate measures related to flow entropy (i.e., CERI and DSFE) show significant performance fluctuations correlated with network system resilience. As such, NRI is the best surrogate measure to guide the network system layout optimization. Furthermore, we reveal that the uniformity of pipe diameters has a more significant impact on resilience than the uniformity of flow rate. These findings suggest that nodal energy redundancy and diameter uniformity are crucial for maintaining network system’s resilience under pipe failure scenarios. This paper provides insight into the future design of new and better surrogate measures by identifying key meta-information (e.g., nodal head and pipe diameter uniformity) and less critical factors (e.g., velocity and pipe length).

The present study has certain limitations that can be improved in the future. Case selection has a significant impact on the performance of surrogate measures. Some researchers found that entropy-based measures were superior to other measures for multisource networks [44] and large-scale networks [17]. In this study, only two single-source cases were considered. Therefore, it is necessary to consider more network cases with varying topological features (e.g., larger numbers of sources, junctions, and pipes). Using cases with complex hydraulic components (e.g., pumps, tanks) and networks containing district metered areas is also worthy of investigation. Additionally, this study only discusses pipe failure scenarios in the proposed benchmark framework. The method should also be adapted to threats like fire disasters and contaminant intrusions.