Analysis of Uncertainty in Internal Erosion Simulations for DLBreach and WinDAM C

Round 1

Reviewer 1 Report

1.In lines 113-117, how to determine the number of simulations required for this research system? If there is a corresponding analysis for this research. And where did the 100 simulation numbers for modern systems come from？Please explain briefly.

2.In lines 224-230, based on the data in the table, please quantify the impact of each parameter on the indicator and make a comparison.

3.In line 255-256, how to understand what you mentioned“but not time to peak unlike what the variance results showed perhaps indicating some room for alternative explanations there .”，especially“some room for alternative explanations”.

4.This paper has strong discussion, and it is suggested to add data analysis content, so that readers with different academic backgrounds can better understand the actual effect of the research results, and the experimental data and simulation results can be specifically and scattered to the theoretical analysis and quantitative expression.

5.The expression style is too colloquial, which is difficult for readers to understand. It is suggested to improve it appropriately.

Comments for author File: Comments.docx

This paper has strong discussion, and it is suggested to add data analysis content, so that readers with different academic backgrounds can better understand the actual effect of the research results, and the experimental data and simulation results can be specifically and scattered to the theoretical analysis and quantitative expression.

Author Response

Reviewer 1 - Thank you for your many thoughtful comments and thorough review.

1. In lines 113-117, how to determine the number of simulations required for this research system? If there is a corresponding analysis for this research. And where did the 100 simulation numbers for modern systems come from？Please explain briefly.

Modified to include in lines 144-146 that some experimentation was done on the number of simulation runs, and that 100 runs was chosen as a tradeoff between data quality and execution time.

1. In lines 224-230, based on the data in the table, please quantify the impact of each parameter on the indicator and make a comparison.

These observations are clarified in lines 208-214 and 217-221 and the quantification of these patterns are provided in table 3.

1. In line 255-256, how to understand what you mentioned “but not time to peak unlike what the variance results showed perhaps indicating some room for alternative explanations there .” especially “some room for alternative explanations”.

The mention to alternative explanations has been removed. The discussion section covers possible explanations for the results as seen including topics such as designer and modeler intentions, artifacts of the models, etc.

1. This paper has strong discussion, and it is suggested to add data analysis content, so that readers with different academic backgrounds can better understand the actual effect of the research results, and the experimental data and simulation results can be specifically and scattered to the theoretical analysis and quantitative expression.

The background has been expanded, descriptions of patterns simplified, and explanations expanded upon.

1. The expression style is too colloquial, which is difficult for readers to understand. It is suggested to improve it appropriately.

The general wording has been simplified throughout the document for clarity.

Reviewer 2 Report

1.In the abstract, there are few methods and innovative descriptions, so it is suggested to increase illustration of the novel methods or models. In addition, the discussion of the research results is too brief, and it is suggested to modify them.

2. The introduction is not exact and rigorous enough to describe the significance of the manuscript, including the related literature review, the questions raised, etc.

3. In section 2, where is the innovation of the method reflected? The author's contribution cannot be reflected. In addition, 2.4 is the same as the topic in 2.5, which is not allowed.

4. In section 3, the result analysis is a little confusing, and it is not clear what the author wants to express.

5. It is suggested to add a conclusion section.

It is suggested that the English expression of the manuscript be revised and improved, since some are very confusing.

Author Response

Reviewer 2 - Thank you for your many thoughtful comments and thorough review.

1. In the abstract, there are few methods and innovative descriptions, so it is suggested to increase illustration of the novel methods or models. In addition, the discussion of the research results is too brief, and it is suggested to modify them.

The abstract has been updated to clarify the purpose of the research as it pertains to the goals of the original initiative.

1. The introduction is not exact and rigorous enough to describe the significance of the manuscript, including the related literature review, the questions raised, etc.

The introduction has been updated to expand upon the history of the field and how the goals of the initiative extend our current knowledge.

1.  In section 2, where is the innovation of the method reflected? The author's contribution cannot be reflected. In addition, 2.4 is the same as the topic in 2.5, which is not allowed.

The findings of the study have been clarified, in particular the comparison between the performance of the models. The section titles have been updated.

1. In section 3, the result analysis is a little confusing, and it is not clear what the author wants to express.

Section three has been updated to clarify the language used and to attempt to explain some of the reasons behind the results that are seen.

1. It is suggested to add a conclusion section.

A conclusion summarizing the results and suggesting future work has been added.

Reviewer 3 Report

1-      The data in Table 1 is shifted and to the left and right and not clear to the reader

2-      In Table 2. Peak Flow Wate should be corrected

3-      In Table 2. There is a huge difference in Peak flow between Big Bay and HERU PI. Either check the number and or explain why?

4-      In Table 2. There are relatively not much difference the in time between collapse Big Bay and HERU PI  with large difference between all other parameters

5-      Subsection 2.4 and 2.5 have the same title?

6-      The figures should have the same scale to present the difference in a better way.

7-      The manuscript is mainly tables and numbers. The paper should be written better

8-      Figure 3 is not clear and the Authors should provide R²  value

9-      According to the figures it looks an excellent relationship between the actual and predicted value, however in line 350 the authors wrote” there is a fairly relationship….”  

1-   There is no conclusion and the results in one section (Sec. 3) and the discussion in other section (Sec. 4)?

Author Response

Reviewer 3 - Thank you for your many thoughtful comments and thorough review.

1. The data in Table 1 is shifted and to the left and right and not clear to the reader

Table 1 has been updated for clarity.

1. In Table 2. Peak Flow Wate should be corrected

Peak Flow Wate updated to Peak Flow Rate

1. In Table 2. There is a huge difference in Peak flow between Big Bay and HERU PI. Either check the number and or explain why?

The difference in values such as peak flow rate are because of the difference in size. The language has been updated to emphasize that Big Bay is a large-scale dam and P1 is a mid-sized dam.

1. In Table 2. there is relatively not much difference the in time between collapse Big Bay and HERU PI with large difference between all other parameters

Lines such as 363-369 have been added to describe how the choice in starting point affects the time measurements. Additionally, again because of the difference in composition and scale of these dams, metrics such as flow rate may be different while the time to peak flow rate may be similar.

1. Subsection 2.4 and 2.5 have the same title?

Section titles have been updated. (line 147)

1. The figures should have the same scale to present the difference in a better way.

When adjusted to the same scale, the relative relationship between peaks and the observed value can become too small to see for some metrics. Table 3 is provided for a more quantitative understanding, while Figure 1 is left for a visual/intuitive understanding.

1. The manuscript is mainly tables and numbers. The paper should be written better

Tables have been condensed/removed where appropriate. Explanation of results have been expanded upon.

1. Figure 3 is not clear and the Authors should provide R2 value

Added the root mean squared error to the description of Figure 2. This is a more typical measure of accuracy for prediction results compared to the R^2 value.

1. According to the figures it looks an excellent relationship between the actual and predicted value, however in line 350 the authors wrote” there is a fairly relationship….” 

The wording has been updated. (line 326)

1. There is no conclusion and the results in one section (Sec. 3) and the discussion in other section (Sec. 4)?

Separating the results and discussion is the required format of the provided article template. Conclusions have been added to summarize the results and suggest future work.

Reviewer 4 Report

MS GEOHAZARDS - 2905775

OVERVIEW

As remarked in the Abstract, this study is part of an international initiative on the evaluation of dam safety simulation software for internal erosion analysis and best practices. The primary experiments involve simulation uncertainty in the failure events of two real-world dams with two different programs. Further analyses on variance, sensitivity, and optimization are performed. The results would improve our understanding of the influences that users bring to using the considered tools.

The manuscript is of interest to Geohazards journal. It is well written and pleasant to read. However, I have some concerns about it: (i) The content of the manuscript appears more suitable for a Technical Note than a Research Paper or Article; (ii) Novelties and advances in knowledge in terms of new methodologies/approaches are unclear; (iii) The organization of the manuscript needs to be improved.

Here below specific comments are provided in the hope they can help to improve this manuscript. A re-review would be desirable.

SPECIFIC COMMENTS

[Abstract] The Abstract is well concise and explanatory. Maybe it is too qualitative: some quantitative information could be useful. Also some key information on the two real-world dams could be attractive.

[Keywords] The keyword "hydrologic simulation" appears too broad. I would replace it with a more specific one.

[Introduction] At the end of this section I would emphasize the novelties and advances in knowledge this study would provide, mainly in terms of new methodologies or approaches.

[Materials and Methods] (i) It reads "2.1. Dokata", but it should read "2.1. Dakota"; (ii) Maybe a couple of figures showing Big Bay and HERU dams in plan would be nice; (iii) In Table 1 where it reads "cm3", "m3", "m2", and so forth, the superscript font should be used for "3", "3", "2", and so forth; (iv) In Table 2 it reads "Peak Flow Wate", maybe it should read "Peak Water Flow"; (v) The subsections 2.4 and 2.5 have the same title! (vi) It reads "2.4. Optimization Analysis", but it should read "2.6. Optimization Analysis".

[Results] I have appreciated the statistical analysis performed in this section. However, how did the Authors validate their results?

[Conclusions] This section is missing! I know that in MDPI papers this section is optional. However, in the present case it would appear important. The Authors could apply for a critical synthesis of key points and could recommend new areas for future researches.

Author Response

Reviewer 4 - Thank you for your many thoughtful comments and thorough review.

OVERVIEW

As remarked in the Abstract, this study is part of an international initiative on the evaluation of dam safety simulation software for internal erosion analysis and best practices. The primary experiments involve simulation uncertainty in the failure events of two real-world dams with two different programs. Further analyses on variance, sensitivity, and optimization are performed. The results would improve our understanding of the influences that users bring to using the considered tools.

The manuscript is of interest to Geohazards journal. It is well written and pleasant to read. However, I have some concerns about it: (i) The content of the manuscript appears more suitable for a Technical Note than a Research Paper or Article; (ii) Novelties and advances in knowledge in terms of new methodologies/approaches are unclear; (iii) The organization of the manuscript needs to be improved.

Here below specific comments are provided in the hope they can help to improve this manuscript. A re-review would be desirable.

SPECIFIC COMMENTS

[Abstract] The Abstract is well concise and explanatory. Maybe it is too qualitative: some quantitative information could be useful. Also some key information on the two real-world dams could be attractive.

[Keywords] The keyword "hydrologic simulation" appears too broad. I would replace it with a more specific one.

[Introduction] At the end of this section I would emphasize the novelties and advances in knowledge this study would provide, mainly in terms of new methodologies or approaches.

[Materials and Methods] (i) It reads "2.1. Dokata", but it should read "2.1. Dakota"; (ii) Maybe a couple of figures showing Big Bay and HERU dams in plan would be nice; (iii) In Table 1 where it reads "cm3", "m3", "m2", and so forth, the superscript font should be used for "3", "3", "2", and so forth; (iv) In Table 2 it reads "Peak Flow Wate", maybe it should read "Peak Water Flow"; (v) The subsections 2.4 and 2.5 have the same title! (vi) It reads "2.4. Optimization Analysis", but it should read "2.6. Optimization Analysis".

[Results] I have appreciated the statistical analysis performed in this section. However, how did the Authors validate their results?

[Conclusions] This section is missing! I know that in MDPI papers this section is optional. However, in the present case it would appear important. The Authors could apply for a critical synthesis of key points and could recommend new areas for future researches.

Response:

(i). Reduced the number of graphs and figures and narrowed the focus of the results.

(ii). Clarified purpose and findings for results.

(iii). Major revision to organization of entire document.

[Abstract]

Updated abstract, including some qualitative data about the dams.

[Keywords]

Changed hydrologic simulation to embankment simulation.

[Introduction]

Clarified the contents of the article. Together with the expansion of the background information, this hopefully clarifies the goals and how this expands upon the current state of the field.

[Materials and Methods]

(i) Updated Dokata to Dakota

(ii) Added a rough comparison of the side profiles of the two dams as a new figure.

(iii) Fixed superscript of units.

(iv) Updated Peak Flow Wate to Peak Flow Rate

(v) Updated Section title of section 2.4

(vi) Updated Section numbering of section 2.5

[Results]

Clarified the analysis results. In line 63 we now mention that the results were presented and compared with those of the other participants in the initiative. Further validation may be carried out in the future by interested researchers.

[Conclusions]

Conclusions section added to summarize the results and suggest future work.

Round 2

Reviewer 1 Report

1.The data needs to be presented more clearly, the language needs to be more concise, there are too few graphs and too many words.

2.In FIG. 3, the scatterplot can be modeled into a spline curve to make the result more intuitive.

The language is slightly colloquial and needs more professional expression.

Author Response

Thank you for your thoughtful comments and thorough review.

1. The data needs to be presented more clearly, the language needs to be more concise, there are too few graphs and too many words.

The language of some sections such as 2.1 and 4 have been revised to be more concise.

2. In FIG. 3, the scatterplot can be modeled into a spline curve to make the result more intuitive

The line of perfect prediction has been included as a reference for the scatterplots.

Reviewer 2 Report

The research is interesting. It is recommended to accept after minor revision.

In section 3.3.3, what is the purpose of regression analysis? Is it only used to indicate the correlation between the predicted score and the actual score? It seems too simple and is suggested that the authors should study it further.

Author Response

Thank you for your thoughtful comment and thorough review.

1. In section 3.3.3, what is the purpose of regression analysis? Is it only used to indicate the correlation between the predicted score and the actual score? It seems too simple and is suggested that the authors should study it further.

Line 97-102, 325-330, and 400-402 have been added/modified to clarify the purpose of including the regression modeling results.

Reviewer 3 Report

Dear Author:

I did not see R² in figure 3

Regards

Author Response

Thank you for your thoughtful comment and thorough review.

1. I did not see R² in figure 3

The R² has now been added to the captions for Fig. 3.

Reviewer 4 Report

The Authors addressed my concerns satisfactorily, albeit quite quickly. However, I'm satisfied with the present version of the manuscript and my recommendation is "Accept in present form".

Author Response

Thank you for your comments and thorough review.