Water Quality Prediction Based on the KF-LSTM Encoder-Decoder Network: A Case Study with Missing Data Collection

Round 1

Reviewer 1 Report

In this manuscript, the results of this research are conveyed thoughtfully and completely, and they are consistent with the experimental findings. However, the authors failed to explain and draw out the novelty of the work, this aspect needs to be improved. This work is worthwhile to be publish in this journal after minor revision. The following issues should be addressed:

 

 

1. Introduction part, if possible, some important and relative reports should be added.

 

https://doi.org/10.1007/s10904-023-02604-0, https://doi.org/10.1016/j.est.2023.107168

2. Abstract and conclusion not targeted; the authors should rephrase it.

 

3. The author should better improve the beauty and quality of the figures.

 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

This study examines the complexities of real-time water quality prediction in China, where data collection and interpretation are hindered by elaborate laboratory methods and incomplete data sets. A solution is put forth, introducing a model that synergistically blends a Kalman filter with an attention mechanism within a Long Short-Term Memory (LSTM) framework, enabling swift data reconstruction and preprocessing. Benchmarking tests against existing deep learning methods such as Recurrent Neural Networks (RNNs), XGBoost, and Multi-Layer Perceptrons (MLPs) show that the proposed model decreases the Root Mean Square Error (RMSE) by 13.2% compared to a standard LSTM model.

Generally, the paper presents a relevant topic and holds general interest for the journal's readership. However, it appears to lack depth in data details and analysis, which impedes a comprehensive understanding of the database. A key critique lies in the methodology, particularly concerning hyperparameter optimization; it is unclear whether this crucial step was adequately executed. Additionally, the paper falls short in providing an evaluation based on a test or unseen dataset, which is a significant flaw and hampers its credibility. Therefore, these issues need to be addressed to improve the quality and validity of the study. Other major comments are listed below.

Major comments

 

1. Please ensure all abbreviations are defined prior to their usage (for instance, 'seq2seq' in Line 60, 'LSTM' in Line 61).
2. The methods and algorithms deployed in this study are sourced from existing literature. Consequently, it is essential to give appropriate credit to the original authors by accurately citing the relevant papers.
3. Please, clarify how the hyperparameter settings listed in Table 1 were determined? Did the authors undertake hyperparameter optimization? If so, what technique was utilized? Also, mention whether cross-validation was employed during hyperparameter optimization and model training to mitigate overfitting. For further context, please, refer to Section 5 of the following study: doi.org/10.1016/j.engstruct.2022.113903
4. A significant strength of the current study lies in the employment of machine learning models ranging from simple to complex. Such an investigative approach aids in pinpointing the most efficient and accurate model. To elevate the paper's quality and highlight the importance of this study, please expand the literature review pertaining to the application of these models.
5. The results of this study have substantial implications for the livability and resilience of smart cities by enhancing living conditions. Please discuss these impacts to underscore the study's significance, referring to the recent study doi.org/10.1016/j.jclepro.2022.134203, where a novel machine learning approach is used to investigate the livability and resilience of smart cities.
6. The database analysis appears cursory and lacks necessary details. How were input parameters selected? Was there any analysis of correlation among all pairs of input parameters? Kindly furnish such details.
7. The title of Table 3 appears misleading. Please, revise.
8. Section 4.4: This section should incorporate a discussion on the optimized hyperparameters of each model. Hyperparameter optimization is a fundamental step in developing machine learning models, and it seems to be overlooked or improperly executed in this study. Please expound on the optimization processes and methods for each model in Section 4.4.
9. The models must be evaluated on both the training and the unseen or test dataset. However, this study doesn't present results from the test or unseen dataset, marking a significant oversight. A comprehensive evaluation of the models should be provided for both the training and test datasets. Moreover, please, present the evaluation metrics graphically for better understanding. For guidance, consider referring to Section 4 of the following study: doi.org/10.1016/j.istruc.2022.08.023.
10. The conclusions section requires revision to accurately represent the findings of the current study. Additionally, providing a list of the main conclusions could enhance clarity.
11. Kindly identify and discuss the limitations of the present study.
12. What is the practical application of the current study considering the fact that the developed models have not been deployed into a practical tool.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments to authors are listed below:

1. The authors have used a small sample size with their deep learning model, are there any precautions taken to avoid overfitting issues. It is not clear in the manuscript.

2. The authors state that they have used 80:20 ratio for training and testing which is not considered ideal for machine learning tasks. Can this be justified?

3. Table 3 shows the training set results or test set? Please mention.

4. Abstract must be enriched via valuable results which pave the way for understanding the audiences.

5. The novelty and applications of this work are not clear. How does Kalman Filter introduce novelty in the proposed architecture.

6. Please include more recent literature in the manuscript and differentiate the use of Kalman and other filters that have been used for water quality.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

This referee appreciates the authors effort to address some of the comments; however, upon a comprehensive examination of the revised paper and responses to comments, it appears that several critical concerns have not been addressed, as detailed below.

In addition, I strongly advise the authors to thoroughly proofread the entire paper due to the presence of numerous sentences that lack clarity and are potentially misleading. For example, the phrase "base on" in the paper's title should be corrected to "based on".

General comment: One fundamental critique is in regards to the methodology, specifically the approach to hyperparameter optimization. The revised manuscript still inadequately addresses hyperparameter optimization. Additionally, the paper fails to provide a comprehensive evaluation on both the training and test datasets, which significantly compromises its credibility. Moreover, it is puzzling to note that the hyperparameter settings reported in Table 1 for KF-LSTM are almost identical to those reported in the original manuscript, despite the introduction of grid search and k-fold cross-validation in the revised manuscript based on my previous comments. To enhance the quality and validity of this study, these issues must be addressed. Other major comments are outlined below.

Comment #4: The response to this comment and the supplementary references do not correspond with the comment itself. Please elaborate on the importance of exploring various machine learning models, ranging from simpler white box models to more complex black box models. To improve the paper's quality and emphasize the importance of this study, please expand the literature review related to the application of these models (e.g., doi.org/10.1016/j.istruc.2022.08.023).

Comment #8: This comment has not been satisfactorily addressed. As hyperparameter optimization forms the basis for any machine learning development, the study's validity entirely depends on the accuracy of the hyperparameter optimization methodology. Please discuss all the hyperparameters and their significance in the model, the range of the hyperparameters considered during optimization, and the optimized hyperparameters for all models outlined in Table 3. Please refer to Section 4.6, Figure 3, and Section 5.1 of the following study for guidance: doi.org/10.1016/j.mtcomm.2022.104461. Moreover, it is puzzling to note that the hyperparameter settings reported in Table 1 for KF-LSTM are almost identical to those reported in the original manuscript, despite the introduction of grid search and k-fold cross-validation in the revised manuscript based on my previous comments.

Comment #9: This comment still lacks a comprehensive response. The models should be evaluated on both the training and test datasets. This section requires significant revision. For example, Figure 9 should display results from both the training and test datasets. Likewise, Table 3 and the corresponding discussion should include evaluations from both the training and test datasets.

Comment #10: Please, revise the conclusions section after addressing the above comments to accurately represent the findings of the current study.

Comment #12: The response to this comment does not relate to the comment itself. Given that the developed models have not been incorporated into a practical tool or shared as a public repository, how can other researchers and practitioners utilize the model? The authors are encouraged to refer to previous studies (e.g., doi.org/10.3390/su15064824) that have developed practical software tools from the developed machine learning model for implementation and discuss the current study's limitations in this regard.

Author Response

Please see the attachment

 

Author Response File: Author Response.pdf

Round 3

Reviewer 2 Report

The critical points raised by this reviewer are now addressed in the revised manuscript.