A Systems Analysis Approach to Identifying Critical Success Factors in Drinking Water Source Protection Programs
Round 1
Reviewer 1 Report
The manuscript aims to first analyze the operational hazards of a typical drinking water source protection program, using a process hazard analysis; secondly, to identify control measures to ensure safe operations.
The article is interesting and well presented. The study design is appropriate, as well as the methods are adequately described and the results are clearly presented. For this reason, the manuscript could be considered for publication after some minor revisions.
In particular, even though the Introduction section provides a sufficient background, it could be useful to underline the problems related to the presence in the supplied water of the disinfection by-products.
The Authors refer to DBPs at lines 48-49, but it could be more interesting to the readers if they deepen this important issue. In fact, the presence of DBPs in drinking water represents a frequent issue that affects the quality of the water intended for human consumption worldwide. For example, in Europe, several Countries faced (and they are still facing) with the presence of chlorites.
In order to cover a wider international panorama, I suggest underlining this public health threat. As a possible reference, the Authors could see "Derogation from drinking water quality standards in Italy according to the European Directive 98/83/EC and the Legislative Decree 31/2001 - a look at a recent past". Ann Ig 2018; 30(6):517-526. doi:10.7416/ai.2018.2263.
Finally, another suggestion is to edit the table no. 9, as the heading of "Mean" and "Rank" are difficult to read, and to use italics for the titles of all the paragraphs (see lines: 309; 326; 345; 359; 436; 474; 517; 525; 549.
Overall, the manuscript is well organized and interesting, and I think that it would be well received by the readers.
Author Response
Response to Reviewer 1
1. The Authors refer to DBPs at lines 48-49, but it could be more interesting to the readers if they deepen this important issue. In fact, the presence of DBPs in drinking water represents a frequent issue that affects the quality of the water intended for human consumption worldwide. For example, in Europe, several Countries faced (and they are still facing) with the presence of chlorites. In order to cover a wider international panorama, I suggest underlining this public health threat. As a possible reference, the Authors could see "Derogation from drinking water quality standards in Italy according to the European Directive 98/83/EC and the Legislative Decree 31/2001 - a look at a recent past". Ann Ig 2018; 30(6):517-526. doi:10.7416/ai.2018.2263.
Reply: We agree that DBPs have been and remain an ongoing public health concern in the supply of safe drinking water. In response to the comments made we have added further detail on the issues of DBPs and the experiences in Europe with Chlorites in lines 51-57.
2. Another suggestion is to edit the table no. 9, as the heading of "Mean" and "Rank" are difficult to read, and to use italics for the titles of all the paragraphs (see lines: 309; 326; 345; 359; 436; 474; 517; 525; 549.
Reply: The headings of table now have been adjusted to be more readable. We reviewed the headings mentioned and made sure that formatting matched the template formatting for heading 3.
Author Response File: 
Author Response.docx
Reviewer 2 Report
Specific comments
Line 27, “aide” with this spelling is not a verb, is a noun.
The authors are opening the abstract talking about only four deciding factors. However, then in lines 294 through 306, it is mentioned that so many variables need to be simplified using PCA. What are these so many factors? I am not convinced that PCA is a relevant tool to use in this study. Please clarify.
Lines 297 to 298: the statement “factor analysis through PCA and is …” does not make any sense. Please rephrase. PCA and factor analysis are two different methods.
Structure of tables 2 and 5 is very awkward and hard to follow, please revise/ reproduce.
In figure 2, what is the difference between dashed lines and solid lines?
Headings of table 3 are poorly chosen, what do you mean by “not providing”?
Line 473, why 1 % level is chosen? Explain. Why then use 0.05 on line 538. Inconsistent analysis.
Presentation of table 10 is very poor and awkward. Nobody can understand anything from it. Why do you have to present the correlation results?
What’s this weird alignment with the heading of table 9? Words cannot be read.
Why do you use Pearson correlation? Why not Spearman’s rank? Pearson means that you are confident that the correlations are linear? How do you know this? Please explain or clarify.
Line 643, the tone is like report writing and not a scientific research. Please change throughout the paper. Also, usually past tense is used in the conclusion section.
Major comments
In the abstract, line 22, this sentence is not clear, “Using Principle Components Analysis (PCA) of the questionnaire responses, 22 the study identified four critical success factors (CSFs) for DWSP”. Please rephrase.
The fact that the authors have chosen Taiwan, Australia and Greece is nice as it diversifies the experiment in this research study, however, they authors had better explain why these three and not other countries. In other words, it’d be nice to explain the differences of the three water authorities to justify the diversity and accordingly the statistical significance of the results.
How can the authors guaranty the relevance and efficacy of PCA in finding the results of this study? In other words based on what PCA is used to compare the four deciding factors, why did not you use multi criteria decision making for instance? PCA is a method rarely used in engineering science, what has inspired the authors to use PCA and what gives PCA the edge to be utilized in here? This should be clarified in this paper before delving into the methodology sections.
In general, the writing of the paper can be a bit improved even though the structure of the paper seems to be acceptable. Please pay attention to all the major and specific comments and address them all before attempting to resubmit. In addressing my comments please refer to line numbers so that I can specifically follow the changes you will have made to the manuscript.
Author Response
Response to Reviewer 2
Specific comments
1. Line 27, “aide” with this spelling is not a verb, is a noun.
Reply: Spelling has been changed as shown in line 29.
2. The authors are opening the abstract talking about only four deciding factors. However, then in lines 294 through 306, it is mentioned that so many variables need to be simplified using PCA. What are these so many factors? I am not convinced that PCA is a relevant tool to use in this study. Please clarify.
Reply: Lines 297 to 306 has been revised to provide a better explanation of the process used to refine the deciding factors.
3. The paragraph from line 210 in the introduction has been rewritten to better explain why PCA was used.
Reply: The paragraph from line 219-232 is now updated to explain the rationale behind using PCA as the method for dimensional reduction.
4. Lines 297 to 298: the statement “factor analysis through PCA and is …” does not make any sense. Please rephrase. PCA and factor analysis are two different methods.
Reply: Text has been amended to remove reference to factor analysis.
5. Structure of tables 2 and 5 is very awkward and hard to follow, please revise/ reproduce.
Reply: Tables 2 & 5 have been revised and reformatted to improve readability.
6. In figure 2, what is the difference between dashed lines and solid lines?
Reply: The explanation provided with the figure has been updated to better describe the differences in the solid lines to show the direction of control actions and dashed lines to show the direction of feedback.
7. Headings of table 3 are poorly chosen, what do you mean by “not providing”?
Reply: Table 3 headings have been updated to be clearer to the reader and match the accepted practice of the STPA methodology.
8. Line 473, why 1 % level is chosen? Explain. Why then use 0.05 on line 538. Inconsistent analysis.
Reply: The reference to 0.05% on line 538 referred to the correlation matrix and was there to highlight the number of significant correlations in the matrix. With the matrix removed there is no longer any inconsistency in the significance levels applied.
9. Presentation of table 10 is very poor and awkward. Nobody can understand anything from it. Why do you have to present the correlation results?
Reply: The table has now been removed as the key matrix results of interest are included in the text and the correlations.
10. What’s this weird alignment with the heading of table 9? Words cannot be read.
Reply: Alignment has been changed to make it more readable
11. Why do you use Pearson correlation? Why not Spearman’s rank? Pearson means that you are confident that the correlations are linear? How do you know this? Please explain or clarify.
Reply: See changes at line 549. The distribution of the data in this case can be assumed to have a normal distribution under central limits theorem, due to the sample size. We also tested the distributions using the Shapiro-Wilk Test to ensure they were normally distributed. Therefore, parametric methods such as Pearson correlation are suitable for data analysis. Additional references have been included to justify the making of this assumption
12. Line 643, the tone is like report writing and not a scientific research. Please change throughout the paper. Also, usually past tense is used in the conclusion section.
Reply: Numerous changes have been made in the conclusion and throughout the paper to better match the tone and structure used in scientific research.
Major comments
1. In the abstract, line 22, this sentence is not clear, “Using Principle Components Analysis (PCA) of the questionnaire responses, the study identified four critical success factors (CSFs) for DWSP”. Please rephrase.
Reply: In response to this comment, lines 21 to 24 have been rephrased to be clearer to the reader.
2. The fact that the authors have chosen Taiwan, Australia and Greece is nice as it diversifies the experiment in this research study, however, the authors had better explain why these three and not other countries. In other words, it’d be nice to explain the differences of the three water authorities to justify the diversity and accordingly the statistical significance of the results.
Reply: Information is now included from lines 310 to 323 which provides and overview on the differences in organisational structures and relationships in the provision of water services. This information shows there is diversity in the water supply structures between each country.
3. How can the authors guaranty the relevance and efficacy of PCA in finding the results of this study? In other words, based on what PCA is used to compare the four deciding factors, why did not you use multi criteria decision making for instance? PCA is a method rarely used in engineering science, what has inspired the authors to use PCA and what gives PCA the edge to be utilized in here? This should be clarified in this paper before delving into the methodology sections.
Reply: In this study, the focus is on reducing the numerous countermeasures identified through using STPA to a select number of critical success factors based on surveys of the perceptions of industry professionals. We selected PCA as it is very useful for this approach and is used widely for complex management structures and programme management, such as those found in construction management, project management, project planning etc. We refer to the following selected references in the text which take a similar approach as this study to finding critical success factors:
Iyer K.C.; Banerjee P.S.; Measuring benchmarking managerial efficiency of project execution schedule performance, International Journal of Project Management, 2016, 34, pp219-236 http://dx.doi.org/10.1016/j.ijproman.2015.10.008
Lingard H.; Francis V.; Michelle T.; Work–life strategies in the Australian construction industry: Implementation issues in a dynamic project-based work environment. International Journal of Project Management, 2012, 30, pp282-295
Li Y.; Ning Y.; Chen W.T.; Critical Success Factors for Safety Management of High-Rise Building Construction Projects in China. Advances in Civil Engineering 2018, ID 1516354.
To address the concerns, the introduction from line 219 onwards has been updated to explain the rationale for choosing PCA for the review of CSFs for management of drinking water catchments. Also included are additional references to better justify the use of PCA in this study.
Author Response File: Author Response.docx
Round 2
Reviewer 2 Report
The paper has been improved compared to the previous version, which is good. However, I still have some questions:
Why is there still reference to factor analysis then?
Table 5 is still hard to read and hard to follow. please left align the content and present in two columns.
Left align all columns on table 2 as well.
Table 8, ANOVA, what is df? explain. not found in the text.
On table 10, left align the content on the first column.
Line 222: “a large number” of and not “the large number of”. pay more attention to the grammar for the next round of revision.
Author Response
1. Why is there still reference to factor analysis then?
Reply: We have made several minor changes in the text to ensure that there is no confusion between PCA and Factors analysis. Furthermore, a number of references while referring to factor analysis either include content on PCA or see PCA as comparable method. To improve clarity reference 46 on line 583 in the previous version has been removed and replaced with reference 34 in current revision
2. Table 5 is still hard to read and hard to follow. please left align the content and present in two columns. Left align all columns on table 2 as well.
Reply: The tables have been reformatted based on the feedback received
3. Table 8, ANOVA, what is df? explain not found in the text.
Reply: df refers to degrees of freedom with the group. A note has been added to the bottom of the table on line 503 to provide and explanation.
4. On table 10, left align the content on the first column.
Reply: The first column in table 10 has been re aligned
5. Line 222: “a large number” of and not “the large number of” pay more attention to the grammar for the next round of revision.
Reply: The suggested change on line 222 has been made. Additionally the grammar of the entire paper has been reviewed again.
Author Response File: Author Response.docx
