Next Article in Journal
Effectiveness of a Dam-Breach Flood Alert in Mitigating Life Losses: A Spatiotemporal Sectorisation Analysis in a High-Density Urban Area in Brazil
Next Article in Special Issue
NaOH-Activated Natural Glauconite for Low-Cost Adsorption of Congo Red Dye
Previous Article in Journal
Evaluation of Various Turbulence Models and Large Eddy Simulation for Stall Prediction in a Centrifugal Pump
Previous Article in Special Issue
A Concise Review of Theoretical Models and Numerical Simulations of Membrane Fouling
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

An Evaluation System for Assessing the Operational Efficiency of Urban Combined Sewer Systems Using AHP—Fuzzy Comprehensive Evaluation: A Case Study in Shanghai, China

1
Key Laboratory of Urban Water Supply, Water Saving and Water Environment Governance in the Yangtze River Delta of Ministry of Water Resources, National Engineering Research Center for Urban Pollution Control, State Key Laboratory of Pollution Control and Resource Reuse, College of Environmental Science and Engineering, Tongji University, Shanghai 200092, China
2
Shanghai Urban Construction Design and Research Institute (Group) Co., Ltd., Shanghai 200125, China
*
Author to whom correspondence should be addressed.
Water 2023, 15(19), 3434; https://doi.org/10.3390/w15193434
Submission received: 23 August 2023 / Revised: 22 September 2023 / Accepted: 27 September 2023 / Published: 29 September 2023
(This article belongs to the Special Issue Sustainable Water and Wastewater Treatment)

Abstract

:
In recent years, China’s urban smart water management has focused on enhancing the quality, efficiency, energy conservation, flood prevention, and control of combined sewer overflow (CSO) and urban waterlogging. To evaluate the combined sewer system’s operation efficiency effectively, a comprehensive evaluation system was established. This system incorporated expert scoring and the coefficient of variation method, considering 31 specific indexes to assess six key aspects: CSO control, waterlogging control, stable wastewater transportation, pipeline management and maintenance, energy conservation, and smart water affairs. The AHP—Fuzzy comprehensive evaluation was employed for evaluation, combining AHP for index weighting and fuzzy comprehensive evaluation for quantitative scoring. The system was applied to assess the operation efficiency of a specific area in Shanghai from 2020 to 2022. Results showed progress in CSO control and energy conservation, with operation efficiency improving from low in 2020 to moderate in 2021 and good in 2022. However, waterlogging control and pipeline management still require improvement in the combined sewer area. Overall, the evaluation system provides valuable insights into the system’s performance, identifying areas for targeted enhancement and emphasizing the need for further improvements to achieve optimal operation efficiency.

1. Introduction

Evaluating the operational efficiency of urban sewer systems holds immense significance, as it guides their development direction, informs multi-objective decision making, and aids in system optimization. Such evaluations enable us to identify system issues, assess the current state, predict various risks, and provide valuable insights for the development, decision making, and optimization of drainage systems. In China, the rise of new technologies, such as artificial intelligence, digital twins, and the Internet of Things, has catalyzed the rapid development of ‘smart water affairs’ [1]. Guided by principles like ‘low-carbon’, ‘green’, ‘energy-saving’, ‘scientific’, and ‘harmonious’, several ‘key technologies for intelligent urban sewer system treatment’ have emerged. Within this context, evaluating the operational efficiency of sewer systems has become a crucial focus. This paper delves into the evaluation of operation efficiency for combined sewer systems.
Practical research on the evaluation of sewer system operational efficiency encompasses various aspects. Researchers such as Chen F [2], Leimgruber J et al. [3], and Zhang D et al. [4] have examined combined sewer overflow control and waterlogging management. Zheng M et al. [5], Okwori E et al. [6], and Ghavami S et al. [7] have assessed the performance of drainage networks. Ananda J et al. [8] have devised performance evaluation systems for sewer management, while Baah K [9], Ba Z et al. [10], and Liu W et al. [11] have investigated safety risk assessments of sewer systems. Additionally, Nam S [12] and Wang J et al. [13] have conducted comprehensive evaluations of drainage systems.
As evident from the above, comprehensively evaluating sewer system operational efficiency involves multiple facets and angles. This multi-dimensional challenge is often referred to as ‘multi-criteria evaluation’, and it employs various methods, as outlined in Table 1.
The primary methods employed in the multi-criteria evaluation approach are the Analytic Hierarchy Process (AHP) and the Entropy Weight Method. AHP is a subjective method for determining weights, relying on expert judgments. It finds its suitability in scenarios where both qualitative and quantitative indicators coexist. In contrast, the Entropy Weight Method is an objective technique that derives weights from indicator information entropy. It is particularly well suited for quantitative indices. In addressing issues related to evaluation uncertainty, subjectivity, and the quantification of qualitative indices, several additional methods are employed. These include fuzzy mathematics theory, gray correlation analysis (GRA), artificial neural networks (ANNs), data envelopment analysis (DEA), and the Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS). When these methods are used in combination, it is often referred to as a comprehensive evaluation approach.
In this study, the index system contains numerous elements necessitating qualitative evaluation, making the AHP a fitting choice. Moreover, owing to the presence of several qualitative indicators, quantifying the evaluation results becomes a challenging task. To address this challenge, we have introduced the principles of fuzzy mathematics into our evaluation framework. It is noteworthy that the integration of AHP and fuzzy mathematics typically can be divided into two types: one is Fuzzy Analytic Hierarchy Process (FAHP) and the other is AHP–Fuzzy Comprehensive Evaluation (FCE), which are depicted in Figure 1. FAHP involves fuzzifying the assigned weights, while Fuzzy Comprehensive Evaluation pertains to the fuzzification of the evaluation results. Due to the authoritative experts in this field who were consulted to assign the weights in this study, fuzzy mathematics was only used for the evaluation results. As such, this study leans towards an AHP-FCE approach (Figure 1b) to establish a set of evaluation index systems for assessing the operation efficiency of an urban combined sewer system.

2. Evaluation System for Operation Efficiency of Combined Sewer System

2.1. Establishment of the Evaluation Index System

The operational efficiency of a combined sewer system encompasses various facets, necessitating the development of a comprehensive multi-objective evaluation system. This system is designed to provide a precise, objective, and scientifically grounded assessment of both the system’s overall performance and its individual components. The ultimate goal is to offer invaluable guidance for the development and construction of efficient drainage systems. In this study, we conducted several expert assessments and thoughtfully selected evaluation metrics after carefully analyzing the real-world conditions of the combined sewer system. These metrics underwent rigorous screening, considering six critical aspects: combined sewer overflow (CSO) control, waterlogging management, seamless wastewater transport, pipeline maintenance, energy efficiency, and smart water management. By incorporating a wide range of perspectives and aspects into the evaluation process, our multi-objective comprehensive evaluation system aims to provide a comprehensive and precise assessment of the operational efficiency of the combined sewer system. The insights gained from this assessment will serve as a valuable reference, offering essential guidance for optimizing and enhancing the overall performance of the sewer system.
Throughout the index selection process, we adhered to principles of scientific rigor, completeness, principal components, and independence. This process involved various methodologies, including literature frequency analysis, expert consultations, and theoretical analysis.
The primary sources of reference for conducting literature frequency analysis are categorized into two main sections. The first comprises indexes that have been utilized in previous studies published in research papers. The second section encompasses indexes recommended for use by international organizations. In line with relevant studies on the assessment of sewer system operational efficiency, these can be broadly classified into four categories based on their evaluation objectives: (1) combined sewer overflow control and waterlogging management, (2) prediction and diagnosis of pipe network performance, (3) sewer system management and operation, and (4) comprehensive sewer system evaluation. A compilation of common indexes employed in these studies is presented in Table 2.
Several international organizations, including the International Water Association (IWA) [22], the International Benchmarking Networking of Water Supply and Drainage Performance (IBNET) [23], the Water Services Regulation Authority (OFWAT) [24], the U.S. Environmental Protection Agency (US EPA) [25], and the American Water Works Association (AWWA) [26], have introduced diverse evaluation metrics, as detailed in Table 3. These metrics comprehensively address various aspects of effectiveness evaluation in drainage system operation and planning. Based on the characteristics of these indicators, they can be categorized into four distinct types: ecological effects, social effects, economic effects, and network planning.
Table 2. Summary of classification of sewer system operation efficiency evaluation objectives.
Table 2. Summary of classification of sewer system operation efficiency evaluation objectives.
Evaluation ObjectivesCommon IndexesAuthors
Combined sewer overflow control and waterlogging controlDescriptive indexes: waterlogging nodes, waterlogging volume, waterlogging duration, waterlogging area, waterlogging depth, overflow water volume, overflow duration, overflow distribution, etc.;
Predictive indexes: proportion of waterlogged pipe sections, potential waterlogging hazards, proportion of overflow volume, potential overflow pollution, etc.;
Guiding indexes: waterlogging reduction potential, overflow reduction potential, regulation and storage capacity, dispatching capacity, etc.
Chen F(2016) [2], Leimgruber J et al.(2018) [3], Jiang Z(2020) [16], Zhang D et al.(2020) [4], Cai Z et al.(2020) [18]
Prediction and diagnosis of pipe network performancePipeline network design indexes: drainage pipe diameter, storm water outlet elevation, dispatching capacity, storage capacity, etc.;
Hydraulic performance indexes: pipeline water depth, pipeline flow velocity, pipeline fullness, pipeline bearing capacity, etc.;
Pipeline status indexes: pipe age, pipe material, burial depth, cushion, pipeline blockage, pipeline inspection, pipeline maintenance, etc.
Zheng M et al.(2020) [5], Okwori E et al.(2020) [6], Ghavami S et al.(2020) [7], Yang L et al.(2021) [20], Jin H et al.(2021) [19], Wang Z et al.(2018) [17], He F et al.(2023) [21]
Management and operation of sewer systemsEcological effect indexes: greenhouse gas emissions, drainage pollution, environmental benefits, etc.;
Social effect indexes: service population, customer service satisfaction, etc.;
Economic effect indexes: maintenance costs, investment in fixed assets, etc.;
Pipe network planning indexes: pipe network coverage, pipe network density, pipe network structure at all levels, etc.;
Young J (2017) [15], Lee J et al. (2018) [27], Ananda J (2020) [8]
Comprehensive evaluation of sewer systemsSewage treatment capacity, drainage pipe network density, sewage treatment rate, low carbon conservation, recycling, natural symbiosis, value creation, smart management and control, etc.Wang Z et al. (2018) [17], Nam S et al.(2019) [12], Wang J et al.(2022) [13]
Drawing from the referenced drainage system indicators and considering the specific conditions of Shanghai’s combined sewer network, we initially identified a total of forty-two indexes, spanning six critical dimensions. Detailed definitions and the significance of these indexes can be found in Table 4.

2.2. Optimization of Evaluation Indexes

The coefficient of variation (CV) is a useful metric for comparing data dispersion among multiple datasets. It becomes particularly valuable when dealing with datasets that have vastly different measurement scales or dimensions. Directly comparing standard deviations in such cases may not be appropriate, as it requires eliminating measurement and dimensional influences. The CV, in contrast, is the ratio of the standard deviation to the mean of the original data. This makes the coefficient of variation dimensionless, allowing for objective comparisons among multiple datasets.
In this study, the evaluation indexes were optimized using a combination of the questionnaire scoring method and the coefficient of variation method. Initially, the 42 indexes were reassessed to identify representative, highly important, and reliable evaluation indicators. Due to differences in indicator dimensions, the coefficient of variation was employed to analyze the scoring results.
First, we designed a questionnaire to gauge the importance of the initial 42 indexes. The questionnaire was distributed to experts and personnel involved in various aspects of the drainage system, who provided ratings for the significance of these primary indicators. We collected a total of 49 valid questionnaires. Next, we analyzed the scoring results from the 49 experts to create an expert evaluation quantification table. For each index, we calculated the average score and variation coefficient. Based on these values, we selected evaluation indicators with high average scores and small coefficients of variation for subsequent quantitative or qualitative evaluation.
To ensure the reliability of the survey results, we introduced an expert-level coefficient to adjust the scores, considering factors such as work positions, professional titles, and years of experience of the 49 scoring experts. The correction coefficient values are provided in Table 5, and the composition of the expert group is illustrated in Figure 2.
The scoring experts represented institutions including the Shanghai Municipal Design Institute, Shanghai Drainage Department, Shanghai Water Group, Shanghai Urban Construction Design Institute, and Tongji University.
The index screening process is outlined in Table 6. Indexes with high mean importance values were retained and low coefficients of variation, while removing those with low mean importance values and low coefficients of variation. Indexes with significant coefficients of variation were subject to further discussion.
Following a comprehensive analysis and summarization of the experts’ scoring results, we have compiled and presented the expert group’s scoring outcomes in Table 7. In this table, the average score reflects the collective judgment of the experts regarding the importance of each specific index. A higher average score signifies greater importance, aligning with the consensus view of the experts. Conversely, the coefficient of variation indicates the degree of divergence among the experts in their assessments of the importance of each index. A higher coefficient of variation score indicates a greater level of disparity among the experts regarding the significance of a particular index.
After applying the Analytic Hierarchy Process (AHP) to weight the indexes, it became evident that the weight assigned to certain indexes was disproportionately small. To maintain the consistency of the judgment matrix and ensure the representativeness of the selected indexes, we decided to exclude three specific indexes: interception capacity, amount of sludge removed, and operator level.
Six indexes were removed after extensive discussions among experts for the following reasons:
(1)
Pumping station discharge time: Despite some experts believing it could effectively represent overflow duration, it was removed due to the challenges in accurately measuring pumping station discharge into the river, especially during rainy days.
(2)
Wastewater treatment plant overflow time: Similar to the previous index, it was removed because accurately measuring wastewater treatment plant discharge into the river, especially during rainy days, posed significant challenges.
(3)
Ponding area ratio: Although some experts saw potential in this indicator for characterizing waterlogging severity, it was removed because, in practice, ponding areas can result from topographical factors unrelated to waterlogging.
(4)
Fullness degree: While some experts thought this index could characterize hydraulic properties within drainage pipes and aid in predicting flooding, others believed it was mostly reflective of full pipe flow during rainy days, rendering it irrelevant for evaluation. It was removed based on the study’s actual context.
(5)
Waterlogging potential hazards: This index was challenging to measure accurately, leading to its removal.
(6)
Variation coefficient of inflow quality: Although considered by some to reflect overflow pollution extent and hazards, experts argued that changes in this index were not directly related to overflows and waterlogging. As a result, this indicator was deleted based on expert recommendations.
Following two rounds of screening, encompassing the initial index selection and subsequent re-evaluation, we arrived at a final set of 31 specific indexes. These indexes comprehensively cover six critical dimensions: CSO control, waterlogging control, stable wastewater transportation, pipeline management and maintenance, energy conservation, and smart water affairs.
Following two rounds of screening, namely, the primary screening of indexes and the rescreening of indexes, a final set of 31 specific indexes were obtained, covering six crucial aspects: CSO control, waterlogging control, stable transportation of wastewater, pipeline management and maintenance, energy conservation, and smart water affairs.

3. AHP—Fuzzy Comprehensive Evaluation Method

The AHP—Fuzzy Comprehensive Evaluation method, also known as the AHP-FCE method, combines the Analytic Hierarchy Process (AHP) with Fuzzy Comprehensive Evaluation (FCE). This approach aims to capitalize on the strengths of both AHP and fuzzy comprehensive evaluation methods, making it particularly suitable for multi-level evaluation and analysis models that involve difficult-to-quantify indexes. Essentially, AHP-FCE is an improved version of the AHP method that incorporates fuzzy mathematics to handle uncertainties and imprecise data effectively.
The AHP-FCE method involves four fundamental steps: (1) Construction of the Hierarchical Structure, (2) Pairwise Comparisons and Judgmental Matrix, (3) Construction of the membership matrix, and (4) Calculation of Comprehensive Fuzzy Evaluation.

3.1. Construct the Hierarchical Structure of the Evaluation Index System

Construct a hierarchical framework system of evaluation indexes based on the screened indexes mentioned above, with the hierarchical structure shown in Figure 3.
The first level is the Target Layer; its target is to evaluate the operation efficiency of the combined sewer system, which is coded as A.
The second level is the Rule Layer, including the six aspects of “CSO control”, “Waterlogging control”, “Stable transportation of wastewater”, “Management and maintenance of pipeline”, “Energy conservation”, and “Smart water affairs”, which are coded as B1 to B6.
The third level is the Index Layer, which is the specific index corresponding to each rule layer. For example, the six indexes of “pumping station discharge volume”, “pumping station discharge frequency”, “wastewater treatment plant overflow ratio”, “wastewater treatment plant overflow frequency”, “overflow potential hazard”, and “area of influence of overflow” under the “CSO control” are coded as C11 to C16; the six indexes of “ponding depth”, “ponding frequency”, “ponding time ratio”, “ponding point ratio”, “overloaded pipeline ratio”, and “waterlogging reduction potential” under the “Waterlogging control” are coded as C21 to C26.

3.2. Make Pairwise Comparisons and Obtain the Judgmental Matrix

3.2.1. Pairwise Comparisons Scoring

The evaluation indexes under each rule layer were compared in pairs and scored using the 1–9 scale method, following the importance of Saaty elements [28]. The specific scoring method is presented in Table 8 below, and the matrix formed by the scoring results is referred to as the judgment matrix.
The elements of the judgment matrix have the following properties [28]:
a i j = 1 a j i
To ensure the reliability and effectiveness of the inquiry results, an expert authority coefficient (Cr) is introduced. The expert authority coefficient (Cr) is determined based on the expert’s academic title (C), expert judgment basis (Ca), and expert familiarity (Cs). The scoring method for these factors is presented in Table 9, and the calculation method for the expert authority coefficient (Cr) is shown in Equation (2), as follows:
C r = 1 C a + C s + C
Cr > 0.7 is usually considered acceptable.
Based on the valid part of the expert survey results, construct a judgment matrix and calculate its eigenvalues; “λmax” and feature vector “w” are used to empower the weights of various indexes.

3.2.2. Computation of Weights and Consistency of Comparisons

Consistency testing is to check whether there are contradictions between the weights of the indicators obtained by the Analytic Hierarchy Process. The RI values of the 2–7 order judgment matrix are detailed in Table 10.
C I = λ m a x N N 1
C R = C I R I
When CR < 0.1 [28], it is considered that the consistency of constructing the judgment matrix is acceptable, that is, the results of index weighting can be used. Equations (3) and (4) come from the literature [28].

3.3. Construct the Membership Matrix

This study adopts a two-level fuzzy comprehensive evaluation model.

3.3.1. Determine and Divide the Factor Set

Determine the elements in the factor set U = {u1, u2, …,u3} corresponding to 31 evaluation indexes [29].
Divide the factor set U = {u1,u2, …,u31} into six groups U = {U1, U2, …, U6} corresponding to six rule layers, and have the relationship in Equation (5)
U = i 6 U i   a n d   U i U j ( i j )

3.3.2. Determine the Comment Set

Determine the comment set V = {v1,v2, …,vm} [29], according to relevant standards, statistical yearbooks, engineering materials, literature, and policy documents and divide the evaluation level domain into five evaluation levels: A+, A, A−, B, and C, that is, V = {v1, v2, …, v5}.
The domain of discourse is divided into two types: quantitative indexes and qualitative indexes, as shown in Table 11 and Table 12.

3.3.3. Second-Level Factor Set Evaluation Calculation

The fuzzy comprehensive evaluation matrix is obtained by comprehensively evaluating the sets of secondary factors Ui = {u1(i), u2(i), …,uni(i)} [29], and the membership function is used to calculate the membership degree, as shown in Equation (6):
R i = r 11 i     r 12 i     r 15 i r 21 i     r 22 i     r 25 i                                       r n 1 i     r n 2 i     r n 5 i     i = 1 ,   2 , ,   6
Through the AHP method, it can be known that the weight of each secondary factor set Ui = {u1(i), u2(i), …,uni(i)} is W = {w1(i), w1(i), …,wni(i)} [29], then the comprehensive judgment of each secondary fuzzy judgment matrix is:
B i = W i × R i   i = 1 ,   2 , ,   6
In this study, the membership function of semi-trapezoidal fuzzy distribution, which as shown in Figure 4, was used to calculate the membership degree for quantitative indicators, and the formula is as follows [29]:
According to the classification of indexes domains, A+ correspond to formulas (8), which is L-function (Figure 4a); the three hierarchical domains A, A−, and B correspond to normal function (Figure 4b) formulas (9); C correspond to formulas (10), which is R-function (Figure 4c).
μ x = 0 ,   x < a x a b a ,   a x b 1 ,   x > b
μ x = 0 ,   x < a x a b a ,   a x < b 1 ,   b x < c d x d c ,   c x < d 0 ,   x b
μ x = 1 ,   x < c d x d c ,   c x d 0 ,   x > d
For qualitative indexes, adopt fuzzy statistical method, invite experts in the industry to evaluate the domain level of qualitative indexes, count the results of expert evaluation, and use the frequency of membership to calculate the degree of membership.

3.4. Calculation of Comprehensive Fuzzy Evaluation

Comprehensively evaluate the elements in the first-level factor set U = {U1, U2, …, U6}. In this study, the six aspects of rule layers of the first-level factors—CSO control, waterlogging control, stable transportation of wastewater, management and maintenance of pipeline, energy conservation, and smart water affairs—are subjectively weighted, and the weights are respectively given to 0.2, 0.2, 0.18, 0.12, 0.15, and 0.15, according to the characteristics of the research object, that is, W = {w1, w2, w3, w4, w5, w6} = {0.2, 0.2, 0.18, 0.12, 0.15, 0.15}, then [29]:
R = B 1 , B 2 , B 3 , B 4 , B 5 , B 6   T
Comprehensive evaluation is:
B = W × R
Finally, the corresponding comments or grades are determined according to the principle of the maximum degree of membership. In order to improve the accuracy of the results, this study adopts the method of the weighted average to determine the degree of membership.

4. Case Application

4.1. Overview of the Situation in a Certain Combined Sewer Area of Shanghai

Figure 5 illustrates the combined sewer area, which is one of several large areas in the central urban region of Shanghai. This combined sewer system covers an extensive area of 335 square kilometers and serves a substantial population of approximately 6 million people. The drainage network in this area connects with four urban wastewater treatment plants with a combined capacity of about 2.282 million cubic meters per day (m3/d). Specifically, the first wastewater treatment plant has a design capacity of 1.7 million m3/d, while the second wastewater treatment plant has a design capacity of 500,000 m3/d. Following the implementation of upgrades and expansions, the replenishment capacity of the system will be increased to 800,000 m3/d. Consequently, the capacity of the first wastewater treatment plant will be reduced to 1.1 million m3/d, and the capacity of the second wastewater treatment plant will be reduced to 300,000 m3/d. Despite these changes, the overall treated-water volume in the combined sewer area will remain at 2.2 million cubic meters per day to address rainwater overflow issues effectively. Furthermore, the construction of a new wastewater treatment plant with a capacity of 1.2 million m3/d is planned, which will further enhance the operational efficiency of the entire drainage network in the combined sewer area. These improvements and expansions will play a vital role in efficiently managing wastewater and rainwater overflow, contributing to the sustainable development and environmental protection of the urban region.
The study focuses on the evaluation of the combined sewer system in the combined sewer area of Shanghai, encompassing the following components: (1) the trunk line and main branch pipeline network of the combined sewer phase 1 and the combined sewer phase 2; (2) the wastewater treatment plants at the end of the pipeline network; (3) the pumping station for the trunk line and the main branch line of the combined sewer phase 1 and the combined sewer phase 2. Figure 6 provides an overview of the combined sewer area of Shanghai, serving as the geographic representation of the study object.
The research objective is to comprehensively evaluate the operational efficiency of the combined sewer system in the Shanghai combined sewer area over a three-year period (2020, 2021, and 2022) and analyze the results of the evaluation.

4.2. Evaluating the Operation Efficiency of the Combined Sewer Area

4.2.1. Empowerment of Indexes Use the AHP Method

In the process of using the Analytic Hierarchy Process (AHP) to assign weights to each index, seven experts specializing in drainage systems were consulted. These experts were asked to compare the indicators under each criterion layer using the 1–9 scale method. Based on their level of authority, the relative importance of each indicator was determined using a weighted average method. Subsequently, a consistency judgment matrix was constructed to calculate the weight of each indicator, and consistency testing was performed to ensure the validity of the results. Taking the indexes in the CSO control of rule layer B1 as an example, the scoring statistics of the importance of Saaty elements for each expert are presented in Table 13. Among the experts, three have an authority coefficient Cr > 0.7 in expert correspondence, with their authority coefficients being 0.8, 0.97, and 0.83, respectively. The pairwise comparison matrix formed by the experts is organized in Table 14. After conducting the consistency test, the calculation results are shown in Table 15.

4.2.2. Determining the Degree of Membership Function

In order to handle quantitative indexes, the semi-trapezoidal fuzzy distribution membership function will be applied to calculate the membership degree of each indicator for the years 2020 to 2022. On the other hand, for qualitative indexes, a panel of seven experts in the field of sewer systems will be invited to evaluate the qualitative indicators for the same three-year period (2020 to 2022). The degree of membership for each indicator will be determined using the fuzzy statistical method, taking the evaluation results for the year 2022 as an example, as shown in Table 16.

4.2.3. Multi-Level Fuzzy Comprehensive Evaluation and Quantitative Scoring

According to formulas (6) and (7), the fuzzy judgment matrix is constructed and calculated for the six aspects of the rule layer for the three years 2020 to 2022. The evaluation results for the rule layer are presented in Table 17.
Next, the fuzzy membership degree matrix of the target layer is constructed according to formulas (11) and (12). Finally, the evaluation results for the three years 2020 to 2022 are calculated and presented in Table 18.
To provide a more direct representation of the evaluation of various indexes of combined sewer operation efficiency, quantitative scoring is performed for each indicator using the degree of membership obtained from the criterion layer. The quantification method involves assigning grades A+, A, A−, B, and C, with corresponding scores of 100, 90, 80, 70, and 60 points, respectively. The sum of these scores, multiplied by the corresponding evaluation grade proportion, yields the final quantitative score, as shown in Table 17 and Table 18.

4.3. Analysis of Operation Efficiency Evaluation Results

According to Figure 7 and Figure 8, the ranking of the relative importance of the index layer relative to the target layer is Total energy consumption (C51) > Variation coefficient of inflow volume (C31) > Pumping station discharge volume (C11) > Ponding depth (C21) > Storage capacity (C32) > Inflow monitoring ratio (C61) > Overloaded pipeline ratio (C25) > Wastewater treatment plant overflow ratio (C13) > Pumping station dispatch capability (C33) > Ponding frequency (C22) > Pumping station discharge frequency (C12) > Layout of pipeline monitoring points (C62) > Wastewater transportation unit consumption (C52) > Wastewater delivery capacity (C43) > Urban pipeline network investment (C42) > Wastewater treatment plant overflow frequency (C14) > Pipeline network length per capita (C41) > Ponding time ratio (C23) > Ponding point ratio (C24) > Intelligent control (C64) > Digitalization degree of pipe network (C65) > Intelligent decision (C63) > Forebay water level of outlet pump room (C34) > Area of influence of overflow (C16) > Wastewater treatment rate (C46) > Waterlogging reduction potential (C26) > Pipeline network density (C45) > Environmental benefits (C53) > Overflow potential hazard (C15) > Maintenance and update cost (C44) > Pipeline inspection (C47).
Note that there are 12 indexes with a total weight ratio of more than 0.04, including Total energy consumption (C51), Variation coefficient of inflow volume (C31), Pumping station discharge volume (C11), Ponding depth (C21), Storage capacity (C32), Inflow monitoring ratio (C61), Overloaded pipeline ratio (C25), Wastewater treatment plant overflow ratio (C13), Pumping station dispatch capability (C33), Ponding frequency (C22), Pumping station discharge frequency (C12), and Layout of pipeline monitoring points (C62); the sum of the weights of these indexes is 0.6697, which is an important index affecting the operation efficiency of the combined sewer system.
It can be seen from Table 17 that in 2020, the CSO control (B1) of the combined sewer system in the combined sewer area of Shanghai can be judged as C, the Waterlogging control (B2) can be judged as B, the Stable transportation of wastewater (B3) can be judged as A−, the Management and maintenance of pipeline (B4) can be judged as A−, the Energy saving (B5) can be judged as C, and the Smart water level (B6) can be judged as A. In 2021, the CSO control (B1) of the combined sewer system in the combined sewer area of Shanghai can be judged as C, the Waterlogging control (B2) can be judged as A−, the Stable transportation of wastewater (B3) can be judged as A−, the Management and maintenance of pipeline (B4) can be judged as A−, the Energy saving (B5) can be judged as A−, and the Smart water level (B6) can be judged as A. In 2021, the CSO control (B1) of the combined sewer system in the combined sewer area of Shanghai can be judged as A, the Waterlogging control (B2) can be judged as A−, the Stable transportation of wastewater (B3) can be judged as A, the Management and maintenance of pipeline (B4) can be judged as A−, the Energy saving (B5) can be judged as A, and the Smart water level (B6) can be judged as A.
The comprehensive evaluation of the operation efficiency of the combined sewer area of Shanghai for the three years from 2020 to 2022 is shown in Table 15. In 2020, the comprehensive evaluation result has a probability of A+ of 11.29%, a probability of A of 12.06%, a probability of A− of 24.95%, a probability of B of 20.24%, and a probability of C of 31.04%. In 2021, the comprehensive evaluation result has a probability of 12.97% at A+, 14.92% at A, 40.17% at A−, 13.46% at B, and 18.41% at C. In 2022, the comprehensive evaluation result has a probability of 23.77% at A+, 36.18% at A, 26.17% at A−, 4.70% at B, and 9.12% at C. In summary, it can be considered that the operation efficiency of the combined sewer system in the combined sewer area will be at a relatively low level in 2020, will increase to a moderate level in 2021, and will further increase to a good level in 2022.
Based on the three-year comprehensive evaluation results, it can be concluded that the operational efficiency of the combined sewer system in the Shanghai combined sewer area has been continuously improving. Figure 9 illustrates that this improvement is mainly attributed to the significant advancements in CSO control and energy conservation within the combined sewer system. The consistent enhancement of integrated overflow control in the area can be attributed to the upgrading and replenishment project of wastewater treatment plants in Shanghai, which has effectively curtailed overflow flow and frequency, while reducing the scope of overflow pollution.
Furthermore, the continuous improvement in energy conservation within the combined sewer system indicates substantial progress in enhancing quality and efficiency. With decreasing wastewater transportation consumption and total energy consumption, environmental benefits have also been on the rise.
However, relatively low scores in Waterlogging control and Management and maintenance of pipeline aspects highlight areas that require more attention and improvement in the future. These aspects represent the main directions for future enhancement in the combined sewer area. Strengthening drainage and waterlogging prevention infrastructure and increasing investment in pipeline maintenance are crucial steps in further improving the operational efficiency of the combined sewer system in the area. Addressing these aspects will help in achieving even higher levels of efficiency and environmental protection.
Figure 10 and Figure 11 depict statistical maps illustrating changes in quantitative scores for the index layer between 2020 and 2022. Notably, several indicators, including Pumping station discharge volume (C11), Wastewater treatment plant overflow ratio (C13), Wastewater treatment plant overflow frequency (C14), Ponding frequency (C22), Overloaded pipeline ratio (C25), Wastewater delivery capacity (C43), Maintenance and update cost (C44), Pipeline network density (C45), Total energy consumption (C51), and Wastewater transportation unit consumption (C52), exhibit significant score improvements during this period.
Conversely, Pumping station discharge frequency (C12), Wastewater treatment plant overflow ratio (C13), Ponding depth (C21), Pipeline network length per capita (C41), Inflow monitoring ratio (C61), and Intelligent decision (C63) indicators, which previously had lower scores, represent key focus areas for future development and enhancement of the drainage system.
According to the analysis and discussion of the above results, it can be seen that applying this evaluation system can give us a preliminary understanding of the aspects for the improvement of comprehensive operational efficiency and what are the more important indicators that affect the improvement of comprehensive efficiency. We will conduct in-depth research on the contribution, correlation, sensitivity, and other issues between upper- and lower-level indicators, as well as those under the same rule layer, which is the foundation to find applicable theories and algorithms, achieve multi-criteria decision analysis, and even explore the possibility of interdisciplinary cooperation [45].

5. Conclusions

The previous studies mainly focus on the evaluation of a specific objective of the sewer system, such as overflow control evaluation, safety risk evaluation, performance evaluation, and so on. A set of comprehensive evaluation systems for assessing the operation efficiency of an urban combined sewer system based on AHP-FCE is established, which originally introduces the evaluation indexes and their evaluation rules related to the level of smart water affairs and the stable transportation of drainage. This study arrived at the following conclusions:
(1)
Multiple rounds of screening were conducted using methods such as the coefficient of variation to identify specific indexes that can accurately characterize the operation efficiency of the combined sewer system. Ultimately, a total of 31 specific indexes were identified, covering six essential aspects: CSO control, Waterlogging control, Stable transportation of wastewater, Management and maintenance of pipeline, Energy conservation, and Smart water affairs. This comprehensive evaluation index system model was constructed to assess the operation efficiency of urban combined sewer systems.
(2)
The constructed operation efficiency model for the combined sewer system was then applied to the combined sewer area of Shanghai, enabling a thorough evaluation of its operational efficiency over the period from 2020 to 2022. The results highlighted 12 critical influencing indexes in the combined sewer system in this area: total energy consumption, variation coefficient of inflow volume, pumping station discharge volume, ponding depth, storage capacity, inflow monitoring ratio, overloaded pipeline ratio, wastewater treatment plant overflow ratio, pumping station dispatch capability, ponding frequency, pumping station discharge frequency, and layout of pipeline monitoring points.
(3)
The comprehensive evaluation results for the three-year period indicate continuous improvement in the combined sewer system’s operational efficiency in this area. The system has progressed from a relatively low level to a relatively good level. Notably, there have been significant enhancements in CSO control and energy conservation. However, there is still scope for improvement in waterlogging control and the management and maintenance of pipelines, highlighting these aspects as primary areas of focus for further enhancement in the future.

Author Contributions

Conceptualization, H.W. and M.Y.; methodology, H.W. and M.Y.; software, M.Y.; validation, Y.G., Y.W. and X.D.; formal analysis, M.Y.; investigation, Wang Hongwu and M.Y.; resources, H.W.; data curation, M.Y.; writing—original draft preparation, H.W. and M.Y.; writing—review and editing, H.W. and M.Y.; visualization, M.Y.; supervision, Y.G.; project administration, X.D.; funding acquisition, H.W. and X.D. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Key Research and Development Program of China: No. 2021YFC3201504 and No. 2021YFC3201305; the Project of Science and Technology Commission of Shanghai Municipality: 20dz1204605.

Data Availability Statement

Data cannot be shared publicly, because data from this study contain information about the national security and geography.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Yin, H.; Zhang, H.; Xu, Z. Review of Intelligent Decision-Making Technologies for Urban Drainage System. J. Tongji University. Nat. Sci. 2021, 49, 1426–1434. [Google Scholar]
  2. Chen, F. Research on Evaluation and Optimisation of Flood and Overflow Control Capacity of Urban Drainage System; Tsinghua University: Beijing, China, 2016. [Google Scholar]
  3. Leimgruber, J.; Steffelbauer, D.; Krebs, G.; Tscheikner-Gratl, F.; Muschalla, D. Selecting a series of storm events for a model-based assessment of combined sewer overflows. Urban Water J. 2018, 15, 453–460. [Google Scholar] [CrossRef]
  4. Zhang, D.; Shen, J.; Liu, P.; Zhang, Q.; Sun, F. Use of Fuzzy Analytic Hierarchy Process and Environmental Gini Coefficient for Allocation of Regional Flood Drainage Rights. Int. J. Environ. Res. Public Health 2020, 17, 2063. [Google Scholar] [CrossRef] [PubMed]
  5. Zheng, M.; Liu, S.; Liu, Y.; Li, H. Structural Condition Assessment of Urban Drainage Pipes Based on Particle Swarm Optimization-Extreme Learning Machine. J. Tongji Univ. (Nat. Sci.) 2020, 48, 513–516+551. [Google Scholar]
  6. Okwori, E.; Viklander, M.; Hedstrom, A. Performance assessment of Swedish sewer pipe networks using pipe blockage and other associated performance indicators. H2Open J. 2020, 3, 46–57. [Google Scholar] [CrossRef]
  7. Ghavami, S.; Borzooei, Z.; Maleki, J. An effective approach for assessing risk of failure in urban sewer pipelines using a combination of GIS and AHP-DEA. Process Saf. Environ. Prot. 2020, 133, 275–285. [Google Scholar] [CrossRef]
  8. Ananda, J. Assessing the operational efficiency of wastewater services whilst accounting for data uncertainty and service quality: A semi-parametric approach. Water Int. 2020, 45, 921–944. [Google Scholar] [CrossRef]
  9. Baah, K.; Dubey, B.; Harvey, R.; McBean, E. A risk-based approach to sanitary sewer pipe asset management. Sci. Total Environ. 2015, 505, 1011–1017. [Google Scholar] [CrossRef]
  10. Ba, Z.; Fu, J.; Liang, J.; Liang, K.; Wang, M. Risk Assessment Method of Drainage Network Operation Based on Fuzzy Comprehensive Evaluation Combined with Analytic Network Process. J. Pipeline Syst. Eng. Pract. 2021, 12, 04021009. [Google Scholar] [CrossRef]
  11. Liu, W.; Dong, W. Risk assessment on the drainage pipe network based on the AHP-entropy weight method. J. Saf. Environ. 2021, 21, 949–956. [Google Scholar]
  12. Nam, S.; Nguyen, T.; Oh, J. Performance Indicators Framework for Assessment of a Sanitary Sewer System Using the Analytic Hierarchy Process (AHP). Sustainability 2019, 11, 2746. [Google Scholar] [CrossRef]
  13. Wang, J.; Xiong, Y.; Zhang, X.; Deng, Y. Evaluation of the state and operational effectiveness of urban drainage pipe network based on AHP-fuzzy comprehensive evaluation method: Taking Huai’an District of Huai’an City as an example. J. Environ. Eng. Technol. 2022, 12, 1162–1169. [Google Scholar]
  14. Xu, D.; Zhang, Q. Risk assessment of the integrated sewage pipeline conduction based on the AHP–GRA formula. J. Saf. Environ. 2019, 19, 1149–1154. [Google Scholar]
  15. YOUNG, J.; Suho, B. Evaluating the Efficiency of Public Sewer Systems in Korean Local Governments: Focusing on Economies of Scale, Density, and Scope. Korean J. Local Public Enterp. 2017, 13, 1–28. [Google Scholar]
  16. Jiang, Z. Research on Multi-Scenario Waterlogging Risk Assessment Based on Entropy Weighting and GIS; Nanchang University: Nanchang, China, 2020. [Google Scholar]
  17. Wang, Z.; Zhou, P.; Liu, C.; Xu, H.; Wang, S.; Hou, W. A potential evaluating methods of sponge city and on the underground drainage system. J. Harbin Inst. Technol. 2018, 50, 118–127. [Google Scholar]
  18. Cai, Z.; Li, D.; Deng, L. Risk evaluation of urban rainwater system waterlogging based on neural network and dynamic hydraulic model. J. Intell. Fuzzy Syst. 2020, 39, 5661–5671. [Google Scholar]
  19. Jin, H.; Jin, X.; Jin, J. Performance Assessment Framework for Reinforced Concrete Pipes in Rainwater Drainage System Using a Combined Weights-Fuzzy Theory. J. Perform. Constr. Facil. 2021, 35, 04021001. [Google Scholar] [CrossRef]
  20. Yang, L.; Xing, W.; Zhang, L.; Zhao, C.; Zhou, Y.; Lu, P.; Li, H. Evaluation of the health status of sewage pipeline system based on GA optimized BP neural network model. Water Wastewater Eng. 2021, 47, 123–131. [Google Scholar]
  21. He, F.; Cheng, S.; Zhu, J. Enhancing the Vulnerability Assessment of Rainwater Pipe Networks: An Advanced Fuzzy Borda Combination Evaluation Approach. Buildings 2023, 13, 1396. [Google Scholar] [CrossRef]
  22. Matos, R.; Duarte, P.; Ashley, R.; Molinari, A.; Schulz, A. Performance Indicators for Wastewater Services; IWA: London, UK, 2003. [Google Scholar]
  23. Danilenko, C.; Macheve, B.; Mofitt, L. The IBNET Water Supply and Sanitation Blue Book 2014; World Bank: Washington, DC, USA, 2014. [Google Scholar]
  24. OFWAT. Key Performance Indicators—Guidance; Water Services Regulation Authority: Birmingham, UK, 2013.
  25. EPA US. Guide for Evaluation Capacity, Management, Operation, and Maintenance (CMOM) Programs at Sanitary Sewer Collection Systems; United States Environmental Protection Agency: Cincinnati, OH, USA, 2005.
  26. AWWA. Benchmarking Performance Indicators for Water and Wastewater Utilities–2013 Survey Data and Analyses Report; American Water Works Association: Denver, CO, USA, 2015. [Google Scholar]
  27. Lee, J.; Chung, G.; Park, H.; Park, I. Evaluation of the Structure of Urban Stormwater Pipe Network Using Drainage Density. Water 2018, 10, 1444. [Google Scholar] [CrossRef]
  28. Saaty, T.L. How to make a decision—The analytic hierarchy process. Eur. J. Oper. Res. 1990, 48, 9–26. [Google Scholar] [CrossRef]
  29. Chen, J.; Hsieh, H.N.; Do, Q.H. Evaluating teaching performance based on fuzzy AHP and comprehensive evaluation approach. Appl. Soft Comput. 2015, 28, 100–108. [Google Scholar] [CrossRef]
  30. Dong, L.; Shi, Z. Discussion on Control of the Overflow from the Bumping Stations In the Main City of Shanghai. Shanghai Water 2014, 30, 5–7. [Google Scholar]
  31. Shanghai Municipal Water Affairs Bureau. 2020–2022 Water Situation Bulletin, Shanghai, China. 2023. Available online: http://swj.sh.gov.cn/fxft-sqtb/index.html (accessed on 20 May 2023).
  32. Ministry of Housing and Urban-Rural Development of the People’s Republic of China; State Administration for Market Regulation. Code for Design of Outdoor Wastewater Engineering GB 50014-2021; Publisher: Beijing, China, 2021.
  33. Ministry of Housing and Urban-Rural Construction of the People’s Republic of China; General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China. Technical Code for Urban Flooding Prevention and Control GB 51222-2017; Publisher: Beijing, China, 2017.
  34. Liao, Q.; Tan, Q.; Shi, Z. Research on optimization of stable transportation operation of urban wastewater treatment plant and pipeline network linkage. Water Wastewater Eng. 2016, 52, 20–24. [Google Scholar]
  35. Shanghai Municipal Statistics Bureau. 2020 Shanghai Statistical Yearbook. Shanghai, China. 2021. Available online: https://tjj.sh.gov.cn/tjnj/20210303/2abf188275224739bd5bce9bf128aca8.html (accessed on 20 May 2023).
  36. Shanghai Municipal Statistics Bureau. 2021 Shanghai Statistical Yearbook. Shanghai, China. 2022. Available online: https://tjj.sh.gov.cn/tjnj/20220309/0e01088a76754b448de6d608c42dad0f.html (accessed on 20 May 2023).
  37. Shanghai Municipal Statistics Bureau. 2022 Shanghai Statistical Yearbook. Shanghai, China. 2023. Available online: https://tjj.sh.gov.cn/tjnj/20230206/804acea250d44d2187f2e37d2e5d36ba.html (accessed on 20 May 2023).
  38. Ministry of Housing and Urban-Rural Construction of the People’s Republic of China. 2019 China Urban Construction Statistical Yearbook. China. 2020. Available online: https://www.mohurd.gov.cn/file/2022/20221012/20f60c18-a721-4f75-85fb-e3ed8e0113d9.xls?n=2021%E5%B9%B4%E5%9F%8E%E5%B8%82%E5%BB%BA%E8%AE%BE%E7%BB%9F%E8%AE%A1%E5%B9%B4%E9%89%B4 (accessed on 20 May 2023).
  39. Ministry of Housing and Urban-Rural Construction of the People’s Republic of China. 2020 China Urban Construction Statistical Yearbook. China. 2021. Available online: https://www.mohurd.gov.cn/file/2021/20211012/f88e6161ebf4bcbd4ffde8d1e9b64f0e.xls?n=2020%E5%B9%B4%E5%9F%8E%E5%B8%82%E5%BB%BA%E8%AE%BE%E7%BB%9F%E8%AE%A1%E5%B9%B4%E9%89%B4 (accessed on 20 May 2023).
  40. Ministry of Housing and Urban-Rural Construction of the People’s Republic of China. 2021 China Urban Construction Statistical Yearbook. China. 2022. Available online: https://www.mohurd.gov.cn/file/old/2020/20201231/w02020123122485271423125000.xls (accessed on 20 May 2023).
  41. Shanghai Municipal Statistics Bureau. 2022 Statistical Bulletin of Shanghai National Economic and Social Development. Shanghai, China. 2023. Available online: http://bx2200.com/tjgongbao/huadong/5563.html (accessed on 20 May 2023).
  42. State Council in China. Water Pollution Prevention and Control Project. China. 2019. Available online: http://www.gov.cn/zhengce/content/2015-04/16/content_9613.htm (accessed on 20 May 2023).
  43. Shanghai Municipal Water Affairs Bureau. Notice of Shanghai Municipal Water Administration on the Maintenance and Supervision of Drainage Pipe Networks in 2022. Shanghai, China. 2023. Available online: http://swj.sh.gov.cn/swyw/20230329/e0fc142d9ffa4063910d64d44f915995.html (accessed on 20 May 2023).
  44. Shanghai Municipal Engineering Design Institute (Group) Co. Ltd.; Qidi Water (Group) Co. Ltd. Technical Specification for On-line Monitoring of Urban Drainage Pipe Network T∕CECS 869-2021; Publisher: Beijing, China, 2021. [Google Scholar]
  45. Kut, P.; Pietrucha-Urbanik, K. Most Searched Topics in the Scientific Literature on Failures in Photovoltaic Installations. Energies 2022, 15, 8108. [Google Scholar] [CrossRef]
Figure 1. Flow chart of the evaluation method combining analytic hierarchy process and fuzzy mathematics. Panel (a) shows the flow chart of FAHP evaluation; panel (b) shows the flow chart of AHP-fuzzy comprehensive evaluation.
Figure 1. Flow chart of the evaluation method combining analytic hierarchy process and fuzzy mathematics. Panel (a) shows the flow chart of FAHP evaluation; panel (b) shows the flow chart of AHP-fuzzy comprehensive evaluation.
Water 15 03434 g001
Figure 2. Component of the members of the expert group for rescreen of evaluation indexes of the combined sewer system. Panel (a) shows the component of work position of the advisory expert group; panel (b) shows the component of professional title of the advisory expert group; panel (c) shows the component of working years of the advisory expert group.
Figure 2. Component of the members of the expert group for rescreen of evaluation indexes of the combined sewer system. Panel (a) shows the component of work position of the advisory expert group; panel (b) shows the component of professional title of the advisory expert group; panel (c) shows the component of working years of the advisory expert group.
Water 15 03434 g002
Figure 3. Hierarchy tree for the evaluation of operation efficiency in a combined sewer system.
Figure 3. Hierarchy tree for the evaluation of operation efficiency in a combined sewer system.
Water 15 03434 g003
Figure 4. Trapezoidal membership function distribution. Panel (a) shows the L-function distribution; Panel (b) shows the normal distribution; Panel (c) shows the R-function distribution.
Figure 4. Trapezoidal membership function distribution. Panel (a) shows the L-function distribution; Panel (b) shows the normal distribution; Panel (c) shows the R-function distribution.
Water 15 03434 g004
Figure 5. The overall map of a certain combined sewer area of Shanghai.
Figure 5. The overall map of a certain combined sewer area of Shanghai.
Water 15 03434 g005
Figure 6. Monitoring map of points in a certain combined sewer area of Shanghai.
Figure 6. Monitoring map of points in a certain combined sewer area of Shanghai.
Water 15 03434 g006
Figure 7. Distribution of the global weight of 31 indexes in index layer by AHP.
Figure 7. Distribution of the global weight of 31 indexes in index layer by AHP.
Water 15 03434 g007
Figure 8. Overall prioritization of all 31 indexes by the AHP.
Figure 8. Overall prioritization of all 31 indexes by the AHP.
Water 15 03434 g008
Figure 9. The 2020–2022 histogram of quantified score at the rule layer in combined sewer area of Shanghai.
Figure 9. The 2020–2022 histogram of quantified score at the rule layer in combined sewer area of Shanghai.
Water 15 03434 g009
Figure 10. Changes in Indicator Tier Scores from 2020 to 2022, Part 1.
Figure 10. Changes in Indicator Tier Scores from 2020 to 2022, Part 1.
Water 15 03434 g010
Figure 11. Changes in Indicator Tier Scores from 2020 to 2022, Part 2.
Figure 11. Changes in Indicator Tier Scores from 2020 to 2022, Part 2.
Water 15 03434 g011
Table 1. Typical multi-criteria evaluation method.
Table 1. Typical multi-criteria evaluation method.
MethodFeaturesApplication Cases
AHPSubjective weighting method; suitable for consulting experienced experts, convenient and efficient, but too subjective; applicable to index systems with both quantitative and qualitative indexesEvaluate the performance of the operation of the Korean sewer system [12]
AHP-FCESubjective weighting method; qualitative indexes can be quantified through membership functionsComprehensive evaluation of the state and operation efficiency of the drainage network in a region of Huai’an City, China [13]
AHP-DEASubjective weighting method; suitable for multi-criteria evaluation and subsequent optimization; significantly improving the optimality and fairness of the evaluation systemCombining Bayesian Networks to evaluate the probability of failure and consequences in sewer pipelines [7]
AHP-GRASubjective weighting method; applicable to problems where the sample data are relatively small and regression analysis cannot be conducted; quantifying scores through grey correlation analysisRisk assessment of combined sewer pipes [14]
FAHPSubjective weighting method; reduced the difficulty of determining the consistency of the matrix, which is a fuzzification of AHP, resulting in increased uncertaintyResearch the allocation of the flood drainage rights about each administrative region [4]
ANP-FCESubjective weighting method; allow mutual control and influence between indexes; suitable for complex systemsRisk assessment of the industrial park drainage system [10]
DEAObjective weighting method; a planning model established from the perspective of input–output can be used for multi-objective optimization and decision makingMeasuring the economic efficiency of Australian wastewater treatment services [8]; Using tobit regression analysis to evaluate the efficiency of wastewater company [15]
EntropyObjective weighting method; determine indicator weights based on information entropy theory; applicable to quantitative indexes
Entropy-TOPSISObjective weighting method; applicable to ranking or comparison problems of multi-objective evaluationsAssessment of waterlogging control capacity in 31 provinces in China [16]
Entropy-FCEObjective weighting method; qualitative indexes can be quantified through membership functionsUrban waterlogging risk assessment [17]
ANN-FCEObjective weighting method, with adaptive learning ability; suitable for situations with a large amount of high-quality data availableComprehensive assessment of urban waterlogging control capabilities and risks [18]; Using the PSO-ELM model to predict and diagnose drainage pipelines [5]
AHP-EntropySubjective and objective combination weighting method; this not only reduces the subjectivity of AHP, but also avoids the sensitivity of entropy weight method to data, and uses membership function to quantify the evaluation resultsPerformance evaluation of rainwater concrete pipes [19]; The operation efficiency of a regional sewer pipe network in a city was evaluated [11]
AHP-ANNSubjective and objective combination weighting method; suitable for situations with a large amount of high-quality data availableEstablish of an evaluation model of GA-optimized BP neural network to evaluate the health status of sewer pipes [20]
Fuzzy BordaSubjective and objective combination weighting method; combining multiple evaluation methods to improve the reliability and applicability of evaluationCombining entropy, TOPSIS, efficiency coefficient, FCE, etc. through Borda combination evaluation method to evaluate the vulnerability of rainwater pipe network [21]
Table 3. Operation efficiency of sewer system indexes published by international organizations.
Table 3. Operation efficiency of sewer system indexes published by international organizations.
IWAIBNETOFWATUS EPAAWWA
Water ResourcesService CoverageSewer overflowServing populationPipe service life
Personnel IndexWater Consumption and ProductionApplicable drainage non-infrastructureTotal length of drainage pipeWastewater collection cost per km of pipeline
Operating IndexTreated wastewater volumeApplicable drainage infrastructurePipe internal and external statusCustomer service satisfaction
Fixed AssetsPipe network performanceLeakagePipe slopeOperation and maintenance cost per km of pipeline
Service QualityCost and personnelGHG emissionPipe length, diameter, material, etc.Number of wastewater treatment facilities
Financial SituationService qualitySerious pollution incidentConstruction status of ancillary sewer facilitiesTreated wastewater volume
Fixed asset investmentSewage Discharge Permit ComplianceUse of auxiliary drainage facilitiesSewage treatment rate
Revenue and expenditureEconomic index Income and expenditure status
Financial performance Pipe network coverage
Table 4. Explanation of evaluation indexes for operation efficiency of combined sewer system.
Table 4. Explanation of evaluation indexes for operation efficiency of combined sewer system.
Rule LayerIndex LayerIndex Meaning
CSO controlPumping station discharge volume (m3)The discharge volume of trunk line pumping stations discharged into rivers in rainy days.
Pumping station discharge frequency (%)The frequency of discharge of trunk line pumping stations into the river within a year.
Pumping station discharge time (min)The average time for the trunk line pumping station to discharge the river.
Wastewater treatment plant overflow ratio (%)The ratio of the overflow of the wastewater treatment plant to the water inflow at the end of the pipe network.
Wastewater treatment plant overflow frequency (%)Overflow frequency of wastewater treatment plant in one year.
Wastewater treatment plant overflow time (min)Average duration of wastewater treatment plant overflow.
Overflow potential hazardHazardous consequences of overflow (Pumping station discharge to the river + overflow in wastewater treatment plant).
Overflow reduction potentialConstruction level of overflow pollution control facilities.
Area of influence of overflowArea affected by pollution from overflow (Pumping station discharge to the river + overflow in wastewater treatment plant).
Interception capacityOverall interception capacity of trunk line pumping station.
Waterlogging controlPonding depth (m)The maximum rainwater depth at which ponding occurs within the catchment area.
Ponding frequency (%)Frequency of ponding during rainfall.
Ponding time ratioThe ratio of the maximum duration of ponding to the rainfall duration.
Ponding area ratio (%)Percentage of catchment areas that are flooded.
Ponding point ratio (%)The ratio of points with ponding water within one year to the total monitoring points.
Fullness DegreeAverage fullness degree of downstream trunk pipelines during rainfall.
Overloaded pipeline ratio (%)The ratio of pipelines overloaded during rainfall to total pipelines.
Waterlogging reduction potentialConstruction level of waterlogging facilities.
Waterlogging potential hazardsHarmful consequences of waterlogging.
Stable transportation of wastewaterVariation coefficient of inflow volumeChanges in wastewater volume at the end of the drainage pipeline network.
Variation coefficient of inflow qualityChanges in wastewater quality at the end of the drainage pipeline network.
Storage capacityThe storage capacity of the drainage system.
Pumping station dispatch capabilityDispatching capability of drainage system trunk line pumping station.
Forebay water level of outlet pump room (m)Forebay wastewater level of outlet pump room at end of pipeline network.
Management and maintenance of pipelinePipeline network length per capita
(km/10,000 people)
The length of the pipeline network per 10,000 people in the drainage pipe network service area.
Urban pipeline network investment
(10,000 yuan/km2)
Urban drainage investment divided by urban built-up area.
Wastewater delivery capacity (%)The water delivery of the pipeline network divided by the design flow of the wastewater treatment plant.
Maintenance and update cost
(10,000 yuan/km)
Equipment maintenance and renewal costs per kilometer of drainage pipeline network facilities.
Pipeline network density (km/km2)Drainage pipe network construction density.
Wastewater treatment rate (%)Matching degree of wastewater supply in the service area and terminal treatment capacity.
Operator levelWorking years and professional titles of sewer system operators.
Pipeline inspection (%)Percentage of lengths of sewer pipes inspected annually.
Amount of sludge removed (t/km)Amount of pipeline sludge removed per kilometer of pipeline.
Energy conservationTotal energy consumption (kW·h)The total energy consumption of sewer trunk line transportation, flood control, etc.
Wastewater transportation unit consumption (kW·h/1000 m3)Power consumption per thousand cubic meters of water transported by the drainage pipe network.
Environmental benefitsComprehensive benefits to the urban environment.
Increase energy consumption valueTotal energy consumption (tons of standard coal) divided by industrial added value (10,000 yuan).
Smart water affairsInflow monitoring ratio (%)Proportion of liquid level monitoring installed in the inflow section of the main branch line on the trunk sewer line.
Layout of pipeline monitoring points (points/10 km)On-line monitoring points in every kilometer of pipeline.
Intelligent decisionIntelligent decision-making level in drainage network diagnosis.
Intelligent controlSewage plant–drainage network–river joint dispatch and intelligent control level.
Digitalization degree of pipe network (%)Proportion of digitally managed pipe segments.
Table 5. The value of the correction coefficient of the expert group for rescreen of evaluation indexes.
Table 5. The value of the correction coefficient of the expert group for rescreen of evaluation indexes.
Work Positionk1Professional Titlek2Working Yearsk3
Research1Senior1Over 10 years1
Design0.9Deputy Senior0.95–10 years0.8
Management0.7Intermediate0.73–15 years0.7
Operation0.5Primary0.5Less than 3 years0.5
Note: k1 is the correction coefficient for different work positions; k2 is the correction coefficient for different professional titles; k3 is the correction coefficient for different working years.
Table 6. Description of the method of screening indexes using the coefficient of variation method.
Table 6. Description of the method of screening indexes using the coefficient of variation method.
MeanCVSituation DescriptionProcess Result
≥4≤0.3A relatively large number of experts believe that the importance of this index is highAdopt
≥4>0.3The importance of this index is relatively high, but there are large differences in the scores among expertsFurther argumentation, prudently adopt
<4≤0.3A relatively large number of experts believe that the importance of this index is lowEliminate
<4>0.3The importance of this index is relatively low, but there are large differences in the scores among expertsFurther argumentation, prudently eliminate
Note: CV is the abbreviation for coefficient of variation.
Table 7. Index rescreening expert group scoring results.
Table 7. Index rescreening expert group scoring results.
Rule LayerIndex LayerMeanCVResult
CSO controlPumping station discharge volume4.370.2611Adopted
Pumping station discharge frequency3.970.3191Adopted after expert argumentation
Pumping station discharge time3.630.3847Eliminated after expert argumentation
Wastewater treatment plant overflow ratio4.170.2809Adopted
Wastewater treatment plant overflow frequency3.980.3243Adopted after expert argumentation
Wastewater treatment plant overflow time3.740.3532Eliminated after expert argumentation
Overflow potential hazard4.070.2647Adopted
Overflow reduction potential3.890.2754Eliminated
Area of influence of overflow4.140.2418Adopted
Interception capacity4.370.2355Eliminated due to small weight
Waterlogging controlPonding depth4.570.2366Adopted
Ponding frequency4.210.2678Adopted
Ponding time ratio4.370.2677Adopted
Ponding area ratio3.820.3360Eliminated after expert argumentation
Ponding point ratio4.140.2740Adopted
Fullness Degree3.790.3569Eliminated after expert argumentation
Overloaded pipeline ratio3.940.2865Adopted after expert argumentation
Waterlogging reduction potential4.080.2833Adopted
Waterlogging potential hazards3.730.3182Eliminated after expert argumentation
Stable transportation of wastewaterVariation coefficient of inflow volume4.320.2523Adopted
Variation coefficient of inflow quality3.890.3191Eliminated after expert argumentation
Storage capacity4.190.2317Adopted
Pumping station dispatch capability4.200.2390Adopted
Forebay water level of outlet pump room3.960.2827Adopted after expert argumentation
Management and maintenance of pipelinePipeline network length per capita4.270.2431Adopted
Urban pipeline network investment4.160.2453Adopted
Wastewater delivery capacity4.600.2109Adopted
Maintenance and update cost4.190.2553Adopted
Pipeline network density4.400.2127Adopted
Wastewater treatment rate4.270.2426Adopted
Operator level4.210.2498Eliminated due to small weight
Pipeline inspection4.000.2387Adopted
Amount of sludge removed4.370.2154Eliminated due to small weight
Energy conservationTotal energy consumption4.210.2678Adopted
Wastewater transportation unit consumption3.930.2845Adopted after expert argumentation
Environmental benefits4.100.2401Adopted
Increase energy consumption value3.860.2826Eliminated
Smart water affairsInflow monitoring ratio4.230.2350Adopted
Layout of pipeline monitoring points4.160.2345Adopted
Intelligent decision4.240.2272Adopted
Intelligent control4.070.2527Adopted
Digitalization degree of pipe network4.100.2448Adopted
Table 8. Saaty’s nine-point preference scale.
Table 8. Saaty’s nine-point preference scale.
Intensity of Importance of aijCompare Index of i and j
1Equally Important
3Weakly Important
5Strongly Important
7Very Strongly Important
9Extremely Important
2, 4, 6, 8Intermediate value between adjacent
Table 9. Expert self-assessment authority coefficient (Cr) score sheet.
Table 9. Expert self-assessment authority coefficient (Cr) score sheet.
Authoritative Coefficient Sub-ItemsGrading Rules
Basis for judgment (Ca) LargeMediumSmall
Theoretical knowledge0.30.20.1
Experience0.50.40.3
Personal intuition0.10.10.1
References0.10.10.05
Familiarity (Cs)very familiarfamiliargeneralless familiarno familiar
10.80.50.20
Academic title (C)SeniorDeputy SeniorMediumPrimary
10.90.80.7
Table 10. RI values comparison table.
Table 10. RI values comparison table.
N234567
RI0.000.580.901.121.241.32
Table 11. Classification of discourse domain of quantitative evaluation index.
Table 11. Classification of discourse domain of quantitative evaluation index.
CodeIndex LayerLevelClassification Reference
A+AA−BC
C11Pumping station discharge volume (m3)252266280294308Engineering data, Literature [30]
C12Pumping station discharge frequency (%)1020304050Engineering data, Water situation bulletin [31]
C13Wastewater treatment plant overflow ratio (%)246810Engineering data
C14Wastewater treatment plant overflow frequency(%)3040506070Engineering data
C21Ponding depth (m)0.270.40.60.71Standard [32,33]
C22Ponding frequency (%)1015202530Engineering data
C24Ponding point ratio (%)48121620Engineering data
C25Overloaded pipeline ratio (%)48121620Engineering data
C31Variation coefficient of inflow volume (%)0.10.150.200.250.3Standard [32]
C34Forebay water level of outlet pump room (m)2.0–2.51.8
3.2
1.5
3.5
1.2
3.8
0.9
4.0
Literature [34]
C41Pipeline network length per capita
(km/10,000 people)
282318138Statistical yearbook [35,36,37,38,39,40]
C42Urban pipeline network investment
(10,000 yuan/km2)
65050035020050Statistical yearbook, Statistical bulletin [41]
C43Wastewater delivery capacity (%)9095100110120Statistical yearbook
C44Maintenance and update cost
(10,000 yuan/km)
3025201510Statistical yearbook [35,36,37,38,39,40]
C45Pipeline network density (km/km2)19151173Statistical yearbook [35,36,37,38,39,40]
Literature [42]
C46Wastewater treatment rate (%)99.898.697.496.295Statistical yearbook [35,36,37,38,39,40]
C48Pipeline inspection (%)1801401008050Policy document [43]
C51Total energy consumption (kW·h)76508075850089259350Engineering data
C52Wastewater transportation unit consumption (kW·h/1000 m3)57.660.86467.270.4Engineering data
C61Inflow monitoring ratio (%)9070503010Standard [44]
C62Layout of pipeline monitoring points (points/10 km)2010410.5Standard [44]
C65Digitalization degree of pipe network (%)9080706050Standard [44]
Table 12. Classification of discourse domain of qualitative evaluation index.
Table 12. Classification of discourse domain of qualitative evaluation index.
CodeIndex LayerLevelClassification
Reference
A+AA−BC
C15Overflow potential hazardvery smallsmallgenerallargevery largeExpert Consultation
C16Area of influence of overflowvery smallsmallgenerallargevery large
C23Ponding time ratiovery shortshortgenerallongvery long
C26Waterlogging reduction potentialvery largelargegeneralsmallvery small
C32Storage capacityvery largelargegeneralsmallvery small
C33Pumping station dispatch capabilityvery goodbettergeneralpoorworse
C53Environmental benefitsvery goodbettergeneralpoorworse
C63Intelligent decisionvery goodbettergeneralpoorworse
C64Intelligent controlvery goodbettergeneralpoorworse
Table 13. Expert scoring statistics of CSO control in the evaluation of combined sewer system.
Table 13. Expert scoring statistics of CSO control in the evaluation of combined sewer system.
Pairwise Comparison IndexesExpert Scoring Results Combined with Expert Authority Coefficient Weighted StatisticsWeighted Average Scaled Results
1/91/71/51/313579
C11–C12000.97001.630001.96
C11–C1300001.800.8002.23
C11–C14000.97000.8300.803.19
C11–C1500000.970.80.83002.89
C11–C1600000.9701.63003.51
C12–C13000.970.83000.8001.72
C12–C1400001.80.80001.62
C12–C1500000.970.830.8002.87
C12–C1600000.971.630002.26
C13–C14000.97001.630001.96
C13–C150000001.630.9705.74
C13–C16000000.80.830.9705.13
C14–C1500001.770.830001.64
C14–C1600000.971.630002.26
C15–C160000.831.7700000.79
Table 14. The pairwise comparison matrix of CSO control in the evaluation of combined sewer system.
Table 14. The pairwise comparison matrix of CSO control in the evaluation of combined sewer system.
C11C12C13C14C15C16Eigenvalues (Weights)
C1111.962.233.192.893.510.3062
C120.5111.721.622.872.260.2070
C130.450.5811.965.745.130.2259
C140.310.620.5111.642.260.1283
C150.350.350.170.6110.790.0598
C160.280.440.200.441.2710.0698
CR = 0.045 < 0.1
Table 15. Evaluation index weight of combined sewer system in a certain combined sewer area of Shanghai.
Table 15. Evaluation index weight of combined sewer system in a certain combined sewer area of Shanghai.
Rule LayerCodeWeightIndex LayerCodeLocal WeightGlobal Weight
CSO
control
B10.2Pumping station discharge volumeC110.30620.0612
Pumping station discharge frequencyC120.20700.0414
Wastewater treatment plant overflow ratioC130.22590.0452
Wastewater treatment plant overflow frequencyC140.12830.0257
Overflow potential hazardC150.05980.0120
Area of influence of overflowC160.06980.0140
Waterlogging
control
B20.2Ponding depthC210.29610.0592
Ponding frequencyC220.21510.0430
Ponding time ratioC230.09830.0197
Ponding point ratioC240.08950.0179
Overloaded pipeline ratioC250.23470.0469
Waterlogging reduction potentialC260.06610.0132
Stable transportation of wastewaterB30.18Variation coefficient of inflow volumeC310.35990.0648
Storage capacityC320.31780.0572
Pumping station dispatch capabilityC330.24110.0434
Forebay water level of outlet pump roomC340.08130.0146
Management
and
maintenance
of
pipeline
B40.12Pipeline network length per capitaC410.17490.0210
Urban pipeline network investmentC420.22340.0268
Wastewater delivery capacityC430.22560.0271
Maintenance and update costC440.09650.0116
Pipeline network densityC450.10190.0122
Wastewater treatment rateC460.11030.0132
Pipeline inspectionC470.06740.0081
Energy
conservation
B50.15Total energy consumptionC510.73060.1096
Wastewater transportation unit consumptionC520.18840.0283
Environmental benefitsC530.08100.0122
Smart
Water
affairs
B60.15Inflow monitoring ratioC610.37860.0568
Layout of pipeline monitoring pointsC620.27320.0410
Intelligent decisionC630.11350.0170
Intelligent controlC640.11890.0178
Digitalization degree of pipe networkC650.11580.0174
Table 16. Membership degree of the evaluation indexes of operation efficiency in a certain combined sewer area of Shanghai.
Table 16. Membership degree of the evaluation indexes of operation efficiency in a certain combined sewer area of Shanghai.
CodeIndex LayerActual Value2022 Actual Value of Membership
202020212022A+AA−BC
C11Pumping station discharge volume (m3)292.69325.78223.7610000
C12Pumping station discharge frequency (%)47.4047.1245.750000.4250.575
C13Wastewater treatment plant overflow ratio (%)12.516.962.690.6550.345000
C14Wastewater treatment plant overflow frequency(%)91.4477.7238.360.1640.836000
C15* Overflow potential hazard0.2860.4290.28600
C16* Area of influence of overflow0.1430.5710.28600
C21Ponding depth (m)10.98100001
C22Ponding frequency (%)27.8618.6218.1800.3640.63600
C23*Ponding time ratio0.1430.4290.2860.1430
C24Ponding point ratio (%)12.5010.719.3800.6550.34500
C25Overloaded pipeline ratio (%)13.6711.289.5700.6080.39300
C26* Waterlogging reduction potential0.1430.4290.42900
C31Variation coefficient of inflow volume (%)0.210.220.1700.60.400
C32Storage capacity0.2860.4290.28600
C33Pumping station dispatch capability0.1430.2860.57100
C34Forebay water level of outlet pump room (m)2.642.562.5010000
C41Pipeline network length per capita
(km/10,000 people)
11.6311.6811.620000.7240.276
C42Urban pipeline network investment
(10,000 yuan/km2)
530.59577.94532.280.2150.785000
C43Wastewater delivery capacity (%)119.12111.19100.44000.9560.0440
C44Maintenance and update cost (10,000 yuan/km)17.5713.3924.2000.840.1600
C45Pipeline network density (km/km2)13.9219.0820.0310000
C46Wastewater treatment rate (%)96.6896.8997.38000.9830.0170
C47Pipeline inspection (%)12011810200.050.9500
C51Total energy consumption (kW·h)9217.878455.608096.0700.9500.05000
C52Wastewater transportation unit consumption (kW·h/1000 m3)69.7761.1359.720.3380.663000
C53Environmental benefits0.5710.1430.28600
C61Inflow monitoring ratio (%)50505000100
C62Layout of pipeline monitoring points (points/10 km)20202010000
C63Intelligent decision00.1430.4290.2860.143
C64Intelligent control0.1430.4290.1430.2860
C65Digitalization degree of pipe network
(%)
10010010010000
Note: The indexes marked with * are qualitative indexes, and their actual values are scored by experts.
Table 17. Evaluation results and quantitative score of combined sewer system operational efficiency rule layer in a certain combined sewer area of Shanghai from 2020 to 2022.
Table 17. Evaluation results and quantitative score of combined sewer system operational efficiency rule layer in a certain combined sewer area of Shanghai from 2020 to 2022.
YearsRule LayerCodeEvaluation Level Proportion %Quantitative
Score
A+AA−BC
2020CSO controlB112.7110.4134.8550.7466.61
Waterlogging controlB23.298.4526.2020.1241.9171.09
Stable transportation of wastewaterB314.4926.6748.2010.64084.50
Management and maintenance of pipelineB44.5628.5915.4925.9925.3776.10
Energy conservationB54.632.311.1626.4265.4865.42
Smart water affairsB643.928.3446.121.62089.42
2021CSO controlB103.7018.3019.5158.1966.54
Waterlogging controlB22.3521.5045.582.9227.6476.79
Stable transportation of wastewaterB315.4217.7648.9917.84083.08
Management and maintenance of pipelineB421.8013.7710.0543.9810.4179.27
Energy conservationB53.4726.8469.680083.37
Smart water affairsB643.926.7246.123.24089.13
2022CSO controlB150.2325.073.708.8011.9089.05
Waterlogging controlB22.3535.0031.621.4029.6177.89
Stable transportation of wastewaterB320.6542.1037.250088.34
Management and maintenance of pipelineB415.0025.9840.3613.844.8383.26
Energy conservationB510.9983.085.940090.51
Smart water affairsB640.606.7244.426.641.6287.80
Table 18. The final results evaluation and quantified scores of the operation efficiency of the sewer system in the combined sewer area of Shanghai from 2020 to 2022.
Table 18. The final results evaluation and quantified scores of the operation efficiency of the sewer system in the combined sewer area of Shanghai from 2020 to 2022.
YearsEvaluation Level Proportion %ResultScore
A+AA−BC
202011.2912.0624.9520.2431.40B75.15
202112.9714.9240.1713.4618.41A−79
202223.7736.1826.174.709.12A86
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Wang, H.; Yan, M.; Gao, Y.; Wang, Y.; Dai, X. An Evaluation System for Assessing the Operational Efficiency of Urban Combined Sewer Systems Using AHP—Fuzzy Comprehensive Evaluation: A Case Study in Shanghai, China. Water 2023, 15, 3434. https://doi.org/10.3390/w15193434

AMA Style

Wang H, Yan M, Gao Y, Wang Y, Dai X. An Evaluation System for Assessing the Operational Efficiency of Urban Combined Sewer Systems Using AHP—Fuzzy Comprehensive Evaluation: A Case Study in Shanghai, China. Water. 2023; 15(19):3434. https://doi.org/10.3390/w15193434

Chicago/Turabian Style

Wang, Hongwu, Ming Yan, Yuan Gao, Yanqiong Wang, and Xiaohu Dai. 2023. "An Evaluation System for Assessing the Operational Efficiency of Urban Combined Sewer Systems Using AHP—Fuzzy Comprehensive Evaluation: A Case Study in Shanghai, China" Water 15, no. 19: 3434. https://doi.org/10.3390/w15193434

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop