Using Convolutional Neural Networks for Cloud Detection on VENμS Images over Multiple Land-Cover Types

Round 1

Reviewer 1 Report

The authors used Convolutional Neural Networks for Cloud Detection on VENmS Images over Multiple Land-Cover Types. The research topic is quite interesting and it fits the scope of journal very well. After carefully refereeing, it seems to me that the article flow is not smooth, the conclusions are not scientifically supported by the content. Based on these drawbacks, the current manuscript requires intense revision before next refereeing. As the potential for improving the content is high, I would like to recommend major revision and encourage the authors to take some more time to revise and improve the manuscript due to some critical points, as mentioned in the following.

Comments

1.       The research hypothesis is not properly addressed.

2.       The choice of the model should be clearly justified. At times, neural networks-based models are considered as black-box in nature, how authors have addressed this issue?

3.       The latest work on deep learning -CNN and other XAi based models in above mentioned topic should be well justified. Therefore, the literature review needs to be well expanded.

4.       The selection of the study area and parameters used need more justification.

5. Too many figures. Reduce it from main text and accommodate as a supplementary file for less important figures. Bedsides, some of the figures are not properly visible. Make these correct.

6.  Where is the conclusion of the paper?

7.       Is there any limitation of the method? If yes, then stress it in the discussion section.

8.       It will be good if you add a flowchart for the whole work done in the paper in the methodology section.

9.  How you validate your results? I could not find anything related validation. Please stress it in methods and results sections.

 

10. The authors should enhance the discussion section by talking about: (i) the implications of their findings in the context of the current trend, (ii) can the approach of the present paper have practical usage for researchers and how their findings can be communicated?

Author Response

Thank you for your valuable comments. We appreciate the effort dedicated to the provided feedback and are thankful for the recommended improvements to our paper. We have incorporated most of the suggestions. You can find our responses to all of your comments below.

The revised version of the paper has been uploaded. Please find all the changes highlighted in blue, with the complete list of changes being printed right after the paper abstract. If you are interested only in seeing the revised version without the changes highlighted, please only change "\usepackage{changes}" to "\usepackage[final]{changes}" in the uploaded LaTeX file (also, please note that some figures could be mislocated in the "changes" file, but they should be located correctly in the final version).

Here are replies to your comments:

1. The Introduction section has been expanded a bit in order to address the hypothesis a bit more precisely.

2. The models were chosen based on their usage in the field, as mentioned in Section 3.1: "Out of the four CNN architectures that are, as depicted in Figure 6, most commonly employed for image segmentation in the field of remote sensing ([42], [43]), three were chosen to be implemented. The architectures were chosen based on their utilization as backbone models of CNN-based cloud cover studies mentioned in Section 1." It is a consequence of a goal defined in Section 1: "The overarching goal of this study is to investigate the performance of common CNN architectures on the task of cloud detection". Shall we stress this more?

The black-boxness is an issue especially for image classification/information derivation; in our study, it is more or less just a computer vision algorithm for semantic segmentation. However, a small discussion on the black-box nature has been added to the Discussion section and XAIs as a valuable enhancement have been mentioned.

3. XAIs were added to the Discussion section and the literature in the Introduction section was slightly enhanced.

4. The main reason the area (Israel) has been chosen to demonstrate the proposed cloud detection algorithm is that it is the only worldwide region observed by three long strips (Figure 1) and contains a diverse land cover and land use categories, such as water bodies, urban, agriculture, and desert.

Section 2.2 was slightly enhanced to explain this a bit more explicitly

What does the reviewer mean by "parameters"?

5. We have considered this suggestion but found no figure that can be moved to Supplementary Material.

6. A section Conclusion has been added.

7. Limitations of the tested methods added to the Discussion section.

8. The study flowchart has been added to the Methodology section.

9. The validation metrics are described in Section 3. The fact that the values reported in the Results section are coming from the validation set and not from the training set has been more stressed.

10. The architecture that performed the best and the results of the usage of various band(+index) combinations have been stressed in the Conclusion section.

Reviewer 2 Report

This article proposes convolutional neural networks for cloud detection on VENmS images over multiple land-cover types. I suggest the authors bring the following points into their consideration and revise the manuscript accordingly.

1) The abstract should be revised as it does not enough chiefly introduce the area of research along with the research question.

2) Motivation is not clear.

3) I suggest writing the main contributions in points before the last paragraph of the introduction.

4) Proposed method of statistical improvement is missing in the abstract and contributions.

5) I suggest the authors use some recent review papers to summarize the state-of-the-art approaches, discuss the main challenges in Literature Section, and compared their results with the proposed approach in the Experimentation Section.

6) Discussion on Cloud Detection and Atrous Convolution is missing. The following studies may be useful for the paper:  GCDB-UNet: A novel robust cloud detection approach for remote sensing images  Cloud Detection Method Based on All-Sky Polarization Imaging  SD-Net: Understanding overcrowded scenes in real-time via an efficient dilated convolutional neural network

7) The term "we" or "our" is repeated several times. Please, use the passive style.

8) Please explain the proposed method in more detail, what is the novelty of the proposed framework? Also, using distinct words for the method such as proposed approach, the proposed model and proposed framework made ambiguity. Make it consistent for reader understanding.

9) Text inside the figures is not clear such as Fig. 10 such as Fig. 10.

10) It is critical to point out how to set the hyper-parameters of the machine or deep learning methods. how can we know that the tuning of the parameter will not affect the accuracy of the methods?

11) I suggest the authors provide the complexity analysis of their proposed method. For example, model inference time on CPU or GPU and its comparison with state-of-the-arts.

12) To have an unbiased view of the paper, there should be some discussions on the limitations of the proposed method.

13) The language is poor, thus proofreading is necessary.

Author Response

Thank you for your valuable comments. We appreciate the effort dedicated to the provided feedback and are thankful for the recommended improvements to our paper. We have incorporated most of the suggestions. You can find our responses to all of your comments below.

The revised version of the paper has been uploaded. Please find all the changes highlighted in blue, with the complete list of changes being printed right after the paper abstract. If you are interested only in seeing the revised version without the changes highlighted, please only change "\usepackage{changes}" to "\usepackage[final]{changes}" in the uploaded LaTeX file (also, please note that some figures could be mislocated in the "changes" file, but they should be located correctly in the final version).

1. The Abstract has been revised. Now it includes the following common elements: (1) a brief summary of the background and the research question; (2) the research goal; (3) the area of research; (4) methods; (5) results; and (6) conclusions of the research along with its significance.

2. The Introduction section has been modified to stress the motivation more.

3. Added to the Introduction section.

4. We tried to do so by saying "It was found that among the tested architectures, U-Net is the best-performing one in most settings. Its results on a simple RGB-based dataset also indicate its potential value for any satellite system screening at least in the visible spectrum. It is concluded that all of the tested architectures outperform the current VENμS cloud-masking algorithm by lowering the false positive detection ratio by tens of per cent and should be considered an alternative by any user dealing with cloud-corrupted scenes."

5. The literature reviewed throughout the paper has been slightly enhanced, now also mentioning XAIs.

As a new training/validation dataset was created during this study, no other papers report results over the same data. The MAJA algorithm is the only different approach working at least with the same data source (VENuS satellite data) - a comparison with this method is described in Section 4.3.

6. A bit more on atrous convolution has been added to Section 3.1.3. Two of the recommended articles have been newly cited throughout the paper.

7. Replaced with the passive style.

8. There is no novelty in the proposed framework as this study aims to examine the performance of the most common frameworks. However, to the best of our knowledge, there is some originality in the data source used: "no other scientific paper dealing with cloud detection for the VENuS satellite has been published except for the original MAJA proposal paper."

"Proposed approach" has been changed to "proposed method". "Proposed framework" has not been found in the paper.

9. There is no text in Figure 10 (in the revised version of the manuscript, the number of original Fig 10 has been changed to Fig 11 due to the insertion of the work flowchart).

10. In the Methodology, it is said that "[The architectures] are used [...] in their original setting" and all the small differences are mentioned in the same section.

11. Numbers of parameters for each of the tested architectures have been added. Only a short notice on the inference time on CPUs and GPUs has been added, as precise inference time statistics have been lost.

12. Added to the Discussion section.

13. A native UK English speaker has newly checked the paper. His comments have been included in the revised version.

Round 2

Reviewer 1 Report

The authors have adequately addressed my comments. I recommend publication of the revised manuscript.

Reviewer 2 Report

All my comments and suggestions are addressed.