Applicability of ANN Model and CPSOCGSA Algorithm for Multi-Time Step Ahead River Streamflow Forecasting

Round 1

Reviewer 1 Report

This article compares a few ANN-based algorithms in a multi-step ahead river flow forecasting application. The below revisions look necessary: 

1- Introduction:

In this section, the authors have conducted a literature review; however, the discussions are not specific and do not contribute to the research gap identification for this study. For example, one of the features of the proposed algorithm is what is called pre-processing. This process is an input selection procedure using the mutual information technique. However, no discussion is made on other input selection methods and why this method is chosen. Please refer to well-known articles on input selection for AI-based algorithms and improve this section:

Maier, H.R., Jain, A., Dandy, G.C., Sudheer, K.P., 2010. Methods used for the development of neural networks for the prediction of water resource variables in river systems: current status and future directions. Environ. Modell. Software 25 (8), 891–909.

Similarly, the study is focused on optimization methods for ANN, such as using the Particle Swarm Optimization algorithm, while no detailed literature review is made on optimization methods and why this algorithm was chosen. It is necessary to conduct a literature review on this topic as well. Perhaps, the below references could be helpful:

- Chau, K. W. (2006). “Particle swarm optimization training algorithm for ANNs in stage prediction of Shing Mun River.” J. Hydrol., 329(3–4), 363–367.

- M Asadnia, LHC Chua, XS Qin, A Talei, Improved particle swarm optimization–based artificial neural network for rainfall-runoff modeling, Journal of Hydrologic Engineering 19 (7), 1320-1329.  

Finally, it is necessary to discuss why the two algorithms Slim Mould Algorithm (SMA-ANN) and Marine Predator Algorithm (MPA-ANN) are selected for this comparative study. 

2- In hydrological modelling using data-driven techniques, working with a coarser time scale generally benefits the model performance. For example, a model for monthly runoff prediction generally gives higher performance measures than a daily prediction model since the variability of the runoff in a finer time resolution is more difficult to simulate. It would be great to explain if there was a specific reason for working on monthly runoff prediction in this study. 

3- Section 2: A map of the study site is needed. 

4-  Section 3.2, line 161-162: 

ANN model has recently become 161 popular in water resources and hydrology fields.

I am not sure whether this is a valid statement since ANNs have been used since the late 1990's in hydrological and water resources modelling and prediction. 

5- Line 233: What do you mean by "∈ is a small coefficient"? Some elaboration about this parameter is needed. 

6- Figure 4, please change "denoise" to "denoised" in the third boxplot. 

7- Section 4.1:

In this section, AMI is used to select the streamflow lags as the model inputs. Have the authors tried correlation analysis as well? It is a known fact that flow antecedents are highly correlated with the flow. Does CA also give the same results?  or maybe better or worse. Since CA is a benchmark method for input selection, it is suggested to add such comparison also. 

8- Table 2: RMSE and MAE have the same dimension of the evaluated parameter, in this case, flow. Please add the unit to the table. 

9- Figure 7 needs some formatting. The CC values are big in font size. 

10- The article needs careful proofreading. Several sentences had typos or grammatical issues. A few samples are listed below: 

- Line 60-61: an incomplete sentence

Despite several successes in the use of the ANN approach 60 employing a single ANN.

- Line 170: To be with Zubaidi, et al. [52] 170 this research will used ANN...

what do you mean by "to be with"?

- Line 179:

 The entire data set were divided into three groups:

Need to be changed to: 

The entire data set was divided into three groups

Besides, perhaps it is better to use "portions" or "chunks" rather than "groups"

Author Response

Reviewer #1 comment:

 

This article compares a few ANN-based algorithms in a multi-step ahead river flow forecasting application. The below revisions look necessary: 

Response: Thank you very much for your feedback. We believe the paper now is much better. All these issues are amended as below (all amended items are highlighted).

1- Introduction:

In this section, the authors have conducted a literature review; however, the discussions are not specific and do not contribute to the research gap identification for this study. For example, one of the features of the proposed algorithm is what is called pre-processing. This process is an input selection procedure using the mutual information technique. However, no discussion is made on other input selection methods and why this method is chosen. Please refer to well-known articles on input selection for AI-based algorithms and improve this section:

Maier, H.R., Jain, A., Dandy, G.C., Sudheer, K.P., 2010. Methods used for the development of neural networks for the prediction of water resource variables in river systems: current status and future directions. Environ. Modell. Software 25 (8), 891–909.

Response: In the current version of the paper, this has now been addressed in lines 72-82 (Page 2). Thanks for the useful reference.

Similarly, the study is focused on optimisation methods for ANN, such as using the Particle Swarm Optimization algorithm, while no detailed literature review is made on optimisation methods and why this algorithm was chosen. It is necessary to conduct a literature review on this topic as well. Perhaps, the below references could be helpful:

- Chau, K. W. (2006). “Particle swarm optimisation training algorithm for ANNs in stage prediction of Shing Mun River.” J. Hydrol., 329(3–4), 363–367.

- M Asadnia, LHC Chua, XS Qin, A Talei, Improved particle swarm optimisation–based artificial neural network for rainfall-runoff modeling, Journal of Hydrologic Engineering 19 (7), 1320-1329.  

Finally, it is necessary to discuss why the two algorithms Slim Mould Algorithm (SMA-ANN) and Marine Predator Algorithm (MPA-ANN) are selected for this comparative study. 

Response: Thank you for pointing this out; that clarifies the novelty. In the current version of the paper, this has now been addressed in:

• lines 94-106 (Pages 2 and 3).
• lines 114-141 (Page 3).
• lines 158-159 and lines 161-162 (Page 4).

Thanks for the valuable references.

2- In hydrological modelling using data-driven techniques, working with a coarser time scale generally benefits the model performance. For example, a model for monthly runoff prediction generally gives higher performance measures than a daily prediction model since the variability of the runoff in a finer time resolution is more difficult to simulate. It would be great to explain if there was a specific reason for working on monthly runoff prediction in this study. 

Response: In developing countries, the main challenge faced by many researchers is the data. So, our manuscript used monthly time series.

3- Section 2: A map of the study site is needed. 

Response: A study site map was added (Figure 1, Page 4).

4-  Section 3.2, line 161-162: 

ANN model has recently become 161 popular in water resources and hydrology fields.

I am not sure whether this is a valid statement since ANNs have been used since the late 1990's in hydrological and water resources modelling and prediction. 

Response: Agree; This has now been addressed in lines 218-219 (Page 7).

5- Line 233: What do you mean by "∈ is a small coefficient"? Some elaboration about this parameter is needed. 

Response: Agree; This has now been addressed in lines 295-296 (Page 9).

6- Figure 4, please change "denoise" to "denoised" in the third boxplot. 

Response: Thanks. This has been addressed in Figure 5.

7- Section 4.1:

In this section, AMI is used to select the streamflow lags as the model inputs. Have the authors tried correlation analysis as well? It is a known fact that flow antecedents are highly correlated with the flow. Does CA also give the same results?  or maybe better or worse. Since CA is a benchmark method for input selection, it is suggested to add such comparison also. 

Response: In the current version of the paper, we have clarified this point in lines 77-82, Section 1. “Another substantial aspect of data pre-processing is choosing the best scenario of predictors, such as mutual information (MI)  [30], for a univariate model. Using non-linear statistical dependence metrics, such as MI, is more suitable for selecting inputs to ANN techniques than a correlation, which has the limitation of only assessing the linear relationship between variables [31].”

Also, we used correlation analysis to show the importance of pre-processing techniques, lines 338- 341 (Page 10).

8- Table 2: RMSE and MAE have the same dimension of the evaluated parameter, in this case, flow. Please add the unit to the table. 

Response: Amendments have been made accordingly.

9- Figure 7 needs some formatting. The CC values are big in font size. 

Response: Amendments have been made accordingly (Figure 8).

10- The article needs careful proofreading. Several sentences had typos or grammatical issues. A few samples are listed below: 

Response: Thanks, we have taken this advice, and the current version has now been thoroughly revised.  All these issues are amended as below:

- Line 60-61: an incomplete sentence

Despite several successes in the use of the ANN approach 60 employing a single ANN.

Response: The sentence was corrected; please see lines 59-61 (Page 2).

- Line 170: To be with Zubaidi, et al. [52] 170 this research will used ANN...

what do you mean by "to be with"?

Response: Thanks, this has now been addressed in lines 231-232 (Page 7).

- Line 179:

 The entire data set were divided into three groups:

Need to be changed to: 

The entire data set was divided into three groups

Besides, perhaps it is better to use "portions" or "chunks" rather than "groups"

Response: In the current version of the paper, this has now been addressed in line 241 (Page 7).

 

Author Response File: Author Response.docx

Reviewer 2 Report

The authors have applied the ANN model with Coefficient-based Particle Swarm Optimisation and Chaotic Gravitational Search for optimization of the model for the monthly runoff. The paper is useful from the water resource management point of view and has the capability of publishing in this journal.   

Author Response

Reviewer #2 comment:

 

The authors have applied the ANN model with Coefficient-based Particle Swarm Optimisation and Chaotic Gravitational Search for optimisation of the model for the monthly runoff. The paper is useful from the water resource management point of view and has the capability of publishing in this journal.

Response: Many thanks for your positive opinion of the manuscript.

Author Response File: Author Response.docx

Reviewer 3 Report

While the paper is based on a good concept, and some of the results are interesting, I believe that there is not sufficient novelty here to warrant publication, as there have been much work on streamflow prediction using statistical models and neural networks of various flavours and "depths" over recent years. If there is sufficient novelty in the forecasting algoirthm, it is not made explicit to the reader here. Moreover, the authors do not explain why for instance, we should use a 4-layer artificial neural network as opposed to some type of Long Short-Term Memory (LSTM) machine learning model that has been shown to be very effective for the streamflow prediction problem by certain groups. I enjoyed reading the CPSOCGSA algoirthm and some of the techniques for finding appropriate fits to the ML hyper-parameters, but this point seems to be secondary to the primary goals and purported findings of the manuscript.

The claims of the manuscript with respect to carrying out skillful medium-term forecasts, especially in the era of accelerating climate change impacts, warrants deeper thought, consideration, and discussion within the text. The claims made in this paper about the predictive neural network offering preditive capabilities under modern circumstances is most certainly not backed up by the results. For example, the well-known 'stationarity problem' of climate signals under continued warming and/or "business-as-usual" scenarios is not called out or taken into account here. While it appears the lagged multi-steps approach to informing the ANN "just works", so it may be useful, it is not clear how useful it will be in a broader applied sense and over the passage of time. The authors may also be advised to consider a different approach that absorbs other time-series based explanatory forcing data to better 'train' the ANN on physical characteristics, e.g., historical precipitation records over the given time period, correlations with upstream river flows, groundwater sources, etc.

Some of the figures are poorly scaled/shaped/presented, and the reader may be (if only subliminally) turned off by these issues. The bizarre labels with overlapping characters in the Taylor diagram should also be addressed. The paper also appears poorly edited from an English-language perspective, and perhaps help should be sought from a fluent/native-English speaker/writer.

Other general comments:

Sec 3.1: The advantages of the SSA approach to signal denoising are not made clear to me. What advantages does this approach offer over say, low-pass Fourier filtering or a projection approach based upon principal component analysis or empirical orthogonal functions (EOF).

Sec 3.3.2: "m_px and m_ay denote attractive and passive masses", respectively -- I think this might be backwards, as 'p' indicates passive to me while 'a' indicates attractive.

Sec 4: The manuscript would benefit greatly from a discussion of overfitting and how the issues associated with overfitting are avoided here, if at all.

Author Response

Reviewer #3 comment:

 

While the paper is based on a good concept, and some of the results are interesting, I believe that there is not sufficient novelty here to warrant publication, as there have been much work on streamflow prediction using statistical models and neural networks of various flavours and "depths" over recent years. If there is sufficient novelty in the forecasting algoirthm, it is not made explicit to the reader here. Moreover, the authors do not explain why for instance, we should use a 4-layer artificial neural network as opposed to some type of Long Short-Term Memory (LSTM) machine learning model that has been shown to be very effective for the streamflow prediction problem by certain groups. I enjoyed reading the CPSOCGSA algoirthm and some of the techniques for finding appropriate fits to the ML hyper-parameters, but this point seems to be secondary to the primary goals and purported findings of the manuscript.

Response: Thank you very much for your feedback. We believe the paper now is much better. All these issues are amended as below (all amended items are highlighted).

• Thank you for pointing this out; that clarifies the novelty. We considered the recommendations of potential future directions and recommendations of several recent articles. Also, based on the reviewer's comment, we made an amendment to highlight this point (Please see section 1).
• Agree, this issue is explained in section 3.2 (lines 226-232, page 7). Also, about the LSTM model, this is a good point and will consider in future. Thanks.
• For CPSOCGSA algorithm, this has now been addressed in Section 1.

 

The claims of the manuscript with respect to carrying out skillful medium-term forecasts, especially in the era of accelerating climate change impacts, warrants deeper thought, consideration, and discussion within the text. The claims made in this paper about the predictive neural network offering preditive capabilities under modern circumstances is most certainly not backed up by the results. For example, the well-known 'stationarity problem' of climate signals under continued warming and/or "business-as-usual" scenarios is not called out or taken into account here. While it appears the lagged multi-steps approach to informing the ANN "just works", so it may be useful, it is not clear how useful it will be in a broader applied sense and over the passage of time. The authors may also be advised to consider a different approach that absorbs other time-series based explanatory forcing data to better 'train' the ANN on physical characteristics, e.g., historical precipitation records over the given time period, correlations with upstream river flows, groundwater sources, etc.

Response: Thank you very much. These issues are explained below:

• Different statistical criteria used to assess the models include relative error (MARE), absolute error (MAE, RMSE, and MBE), and unit less (R²). Also, several graphical plots, such as a Taylor diagram, Bland–Altman, and scatter plot. All these criteria indicate that CPSOCGSA-ANN outperforms the other models. Moreover, SI was used for examining the CPSOCGSA-ANN. Furthermore, Shapiro–Wilk and Kolmogorov–Smirnov tests were used to test residual normality, and the ADF and KPSS tests were applied to examine the stationary of residual. Accordingly, the results show that CPSOCGSA-ANN is accurate, and the residual is normal distributed and stationary.
• This is a good point, and we added this sentence accordingly, lines 457-460 (page 16).
• Agree, we will consider these variables as a model input in future work. In Iraq, we suffer from a lack of data. So, we will consider another area of study in future. Thanks.

Some of the figures are poorly scaled/shaped/presented, and the reader may be (if only subliminally) turned off by these issues. The bizarre labels with overlapping characters in the Taylor diagram should also be addressed. The paper also appears poorly edited from an English-language perspective, and perhaps help should be sought from a fluent/native-English speaker/writer.

Response: We have taken this advice, and in the current version, the figures have been amended accordingly. Also, the manuscript has been thoroughly revised. 

 

Other general comments:

Sec 3.1: The advantages of the SSA approach to signal denoising are not made clear to me. What advantages does this approach offer over say, low-pass Fourier filtering or a projection approach based upon principal component analysis or empirical orthogonal functions (EOF).

Response: Thank you, amendments have been made accordingly in the paragraph (lines 199-205, pages 6 and 7).

Sec 3.3.2: "m_px and m_ay denote attractive and passive masses", respectively -- I think this might be backwards, as 'p' indicates passive to me while 'a' indicates attractive.

Response: Agree; This has now been addressed in line 294 (Page 9).

Sec 4: The manuscript would benefit greatly from a discussion of overfitting and how the issues associated with overfitting are avoided here, if at all.

Response: This issue is related to the effectiveness of data pre-processing techniques, which improve the raw data quality and choice of the best predictors’ scenario. Also, the optimisation procedure that offers the model with less error scale.

Author Response File: Author Response.docx

Reviewer 4 Report

Review of  “Applicability of ANN Model and CPSOCGSA Algorithm for 1 Multi-Time Step Ahead River Streamflow Forecasting”

By Hugh Whiteley  August 25 2022

 

The paper is very well written and documented . The research is novel and important.

However there is one major factual statement that must be corrected or justified before the paper is published.

In Table 2 and Line 395 the RSME is said to be 0.0704 m^3/s.   This is impossibly small since the monthly mean flow which is being predicted is about 70 m^3/s.    I think the error in statement comes from not recognizing the log transformation.   In any case the correct RSME must be stated for the actual flow prediction in m^3/s.

 

The following minor edits should also be made.

 

13        Abstract     Precise  Accurate  streamflow prediction …..   {Precise is the wrong word to use because it refers to the number of significant figures. Accurate is correct because it means how close a number is to the true magnitude}

Line 23             precise   accurate     check for this throughout the document for example line 327

Line 46             various techniques  forms of analysis and scenarios

Line 56-57       The more frequently artificial intelligence (AI) models most frequently used were  are   used in streamflow forecasting are …

Line 61             However,      {merge the sentence with the preceeding sentence}

Author Response

Reviewer #4 comment:

 

The paper is very well written and documented. The research is novel and important. However, there is one major factual statement that must be corrected or justified before the paper is published.

Response: Many thanks for your positive opinion of the manuscript. (all amended items are highlighted).

 

In Table 2 and Line 395 the RSME is said to be 0.0704 m^3/s.   This is impossibly small since the monthly mean flow which is being predicted is about 70 m^3/s.    I think the error in statement comes from not recognising the log transformation.   In any case the correct RSME must be stated for the actual flow prediction in m^3/s.

Response: Thank you for pointing this out; In the current version of the paper, this has now been addressed in lines 456-457 (Page 16).

 The following minor edits should also be made.

• Line13 Abstract Precise  Accurate  streamflow prediction …..   {Precise is the wrong word to use because it refers to the number of significant figures. Accurate is correct because it means how close a number is to the true magnitude}

Response: This has now been addressed in line 13.

 

• Line 23 precise accurate     check for this throughout the document for example line 327

Response: In the current version of the paper, this has now been addressed throughout the document.

 

• Line 46 various techniques forms of analysis and scenarios

Response: Agree; This has now been addressed in line 46 (Page 2).

 

• Line 56-57 The more frequently artificial intelligence (AI) models most frequently used were  are   used in streamflow forecasting are …

Response: Agree; This has now been addressed in line 59-61 (Page 2).

 

• Line 61 However,   {merge the sentence with the preceeding sentence}

Response: Thank you, amendments have been made accordingly in line 62 (Page 2).

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

The authors have made commendable improvements following the given comments by reviewers. Therefore, I am happy to support the acceptance of this article.  

Author Response

Reviewer #1 comment:

The authors have made commendable improvements following the given comments by reviewers. Therefore, I am happy to support the acceptance of this article.

Response: Thank you very much.

Author Response File: Author Response.docx

Reviewer 3 Report

I feel the major structural changes to the manuscript's text adequately address my concerns regardingly the novelty of the paper and with respect to climate variations.

One comment - Avoid the use of the wording 'exact' in reference to a prediction/forecast, as this implies perfect predictive skill. You may want to consider using 'accurate' or 'very accurate', or 'skillful' / 'very skillful'

Thank you.

 

 

Author Response

Reviewer #2 comment:

I feel the major structural changes to the manuscript's text adequately address my concerns regardingly the novelty of the paper and with respect to climate variations.

One comment - Avoid the use of the wording 'exact' in reference to a prediction/forecast, as this implies perfect predictive skill. You may want to consider using 'accurate' or 'very accurate', or 'skillful' / 'very skillful'

Thank you.

Response: Thank you very much. This has now been addressed in the current version of the manuscript. (Page 3, Line 148).

Author Response File: Author Response.docx

Reviewer 4 Report

The authors have begun the corrections  needed to the  specification of error range. The error range must be stated in units of m^3/s   everywhere (ie. the originally stated values must be replaced by descaled values)

The changes that must be made are

(1) In Table 2  all three columns of error values must be replaced by the correct rescaled values.  Units of m^3/s should be shown for the right hand column.

(2) The statements in line 456 and 457 should be corrected as follows 

Correction for Line 456 and 457

streamflow by yielding R²=0.91, RSME = 1.07 m³/s. These results can offer useful information ……

 

 equals to 0.0704 m³/s (the actual value of rsme is 1.07 m³/s after recalling).  

If there are any other references to the error range elsewhere in the paper these should be replaced by the reaclaed (correct) values - I have no found any other references to numerical error values.

 

Author Response

Reviewer #3 comment:

The authors have begun the corrections  needed to the  specification of error range. The error range must be stated in units of m^3/s   everywhere (ie. the originally stated values must be replaced by descaled values)

Response: Thank you very much. This has now been addressed in the current version of the manuscript.

The changes that must be made are

(1) In Table 2  all three columns of error values must be replaced by the correct rescaled values.  Units of m^3/s should be shown for the right hand column.

Response: All the error values in Table 2 are rescaled in this version. Also, the MARE (the right hand column) is unitless (please see Equation  16).

(2) The statements in line 456 and 457 should be corrected as follows 

Correction for Line 456 and 457

streamflow by yielding R²=0.91, RSME = 1.07 m³/s. These results can offer useful information ……equals to 0.0704 m³/s (the actual value of rsme is 1.07 m³/s after recalling).  

Response: Agree; This has now been addressed in line 456 (Page 16).

If there are any other references to the error range elsewhere in the paper these should be replaced by the reaclaed (correct) values - I have no found any other references to numerical error values.

Response: Thanks; The authors checked the manuscript carefully and found nothing related to this issue.

 

Author Response File: Author Response.docx