Investigating the Potential of Sentinel-2 MSI in Early Crop Identification in Northeast China

Round 1

Reviewer 1 Report

This paper investigates the potential of Sentinel-2 data in early crop identification. The topic is interesting. Different machine learning methods were compared. However, I mainly have two concerns. First, the presentation needs big improvements. The manuscript need extra and dense editing for English writing. Some (but not all) comments about presentation are listed below. Also, the reviewer is confused in many places. For example, the authors even didn't specify which year they are focusing on, particularly in the Method Section.

More importantly, the authors claimed that this study generated training samples with the help of historical crop maps and remote sensing images. However, remote sensing images in August 2020 were used for generating training samples. As I stated, I am not sure which year of crop type the authors were trying to identify. If it is for 2020, using images in August 2020 to generate training samples will not be considered real-time or early stage. The samples are not retrievable for an early stage identification of crop type. If it is for 2021, the authors should discuss the impacts of crop rotation on the selection of training samples. 

L14-16. make it more concise. 

L87-90: If it is impossible, why the authors are still trying? Do you mean that it is difficult for the current studies? 

L91-94: The sentence looks upside down to me. May switch the goal and method. 

Figure 1: The samples are basically indistinguishable for different types. Regions 1-3 should be labelled in Figure 1. 

L144-146: Please specify which year of data you are using. 

L151-155: Please rephrase these sentences. 

L180-182: Please cite the paper in a good format. 

L192-194: Please divide the long sentence into two. Specify the full name of MDA.

L205-207: change "therefore, " to "so that"

L210: "All the samples were acquired ..."

L318-321: any reference about Fscore? What is the rationale of using Fscore?

L333-334: This should be moved to Method. 

L335-336: You should explain what the range of J-M distance is and which level the value of 1.9 is in. 

 

 

Author Response

Plese see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

 The manuscript is very well written in all its parts. Materials and methods, results, discussions are also clear and well explained. Please consider the following comments.

1. The abstract needs to be simplified as it is a summary of the research.
2. The legend of Figure 1 should be revised, the current version is difficult to distinguish between different categories, and the picture pixels are too low. What do the different colors of the background indicate?
3. Modify the fonts in Table 1. At the same time, fonts in other figures and tables also need to be checked.
4. Line 147, is the cloud removal process performed on the image with cloud cover less than 10%?
5. Line 162, is the ground survey data only used as validation samples? Why not use these for model training?
6. Line 174, how effective is the historical crop map used in the manuscript for the classification of the target crop? This appears to be the key factor limiting the accuracy of the result.
7. Each index in Table 5 should have a separate reference. Also, is the autocorrelation between the different indices taken into account?
8. Has the vegetation index in Figure 5 been processed? e.g, interpolation and smoothing, please in Mapping the Northern Limit of Double Cropping Using a Phenology-Based Algorithm and Google Earth Engine
9. Line 320, why did choose 85% as the threshold? On what basis?
10. I noticed that table 4 and table 6 are exactly the same, please check. There are also Figure 1 and Figure 5.
11. Line 366, how to determine the classification accuracy for each stage? Composite images of each stage? At what time interval were the images composited?
12. The manuscript lacks a detailed description of the input variables of the model. For the process of crop identification in different periods, what are the differences in the input variables? Maybe the start time is fixed and the end time is gradually delayed?
13. Discussions require a more in-depth clarification of the significance and innovations of the manuscript. Necessarily, existing research needs to be summarized to highlight the context of the manuscript.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

The authors assessed the ability to differentiate corn, soybean, and rice in early, mid, and late growth stages using Sentinel-2 imagery over three regions in Heilongjiang Province, China. They used historical crop type maps to quickly and automatically generate training samples, and ground survey data to validate classification models. They found corn and soybean were most differentiable in early July and rice in late April. Gradient Boosting Decision Tree and Random Forest classification algorithms outperformed Support Vector Machine and Minimum Distance Classification algorithms. Questions and comments follow:

1. How valid is the assumption of current crop types being the same as in 2019? Do farmers rotate crops by year or leave fields fallow at certain intervals?
2. Tables 3 and 6 have the same numbers. Does this mean the validation and training sample sizes are the same? If so, are both tables necessary? Also, were the ground survey sample locations different from the training sample locations?
3. Figures 1 and 5 seem to be showing the same thing. Are both necessary? Also, the different symbol colors are difficult to differentiate. Please consider different symbols and a different background.
4. MDA is not defined in the manuscript. Is it the same as MDC and/or MIN? Please use only one throughout the manuscript and figures.
5. Referring to L240, what is “artificial visual interpretation”?
6. Please elaborate on why vegetation indices rather than individual bands were used in classification.
7. How were the parameter values in Table 7 derived? While general guidelines for setting parameter values are a good starting point (L275), the parameters need to be optimized using methods such as grid-search. This is especially important for SVM and might explain its low performance in this study.
8. Since MDC was found to perform poorly as a classifier, how reliable is its performance as a feature optimizer?
9. Referring to L335-336, what is the typical range of J-M values (to provide more context for the reader)? Also, please spell out J-M.
10. It looks like classification accuracies plateau after about 7 images. Is this because the models have sufficient information by then, or because there is no important information in August through October? For example, what would happen if only August through October images were used to classify crops?
11. Are all the bars in Figure 11 supposed to be the same height? If so, what do the authors want the reader to get out of this figure?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report

It is a quite interesting study, dealing with crop recognition by trying to solve the problem of obtaining suitable ground samples, as well as to use different classifiers and compare them to find the more proper to identify crops.

Some issues that must be addressed to be more suitable is the quality of several Figures, such as Figures 1, 5 and 6 that place names, coordinates, north arrows and scale bar are needed. Also, Figure 6 which presents the classification results in a few zoom images showing some interesting classified parts (probably with mixed crops or else) could provide a more interesting perspective for the readers.

Moreover, I think that Table 2 is unnecessary and should be deleted. Some details of S2 images could be incorporated into the text.

Finally, in all the equations of Table 5, the creator of all these indices should be referenced and not the articles that were found (for instance NDVI was referenced from Rouse et al. 1973).

I believe if the authors address these concerns the manuscript could proceed to the next level and be considered for publication.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 5 Report

Dear authors:

I have found your paper quite interesting, but I think some work is needed to improve it.

I do not understand the use of an acronym for the study area. Please use the whole name Heilongjiang in all the paper.

Figure 1 is really poor and does not comply with the journal instructions. The images are not clear enough. ¿Is it not possible to use a better colour composition? The location of the samples very difficult to distinguish. Please include a location map to show where Heilongjiang is in
China. I did some research and I was surprised to discover that it is indeed a large area.

Avoid grey colour in figures and tables as it is very difficult to distinguish. Use black instead.

In general try to use larger fonts in tables and figures. Some of them are very difficult to read.

Although English is not my mother tongue, I think you should review the English language. There are strange sentences that difficult the understanding of the paper (e.g. lines 155-156).

Include the complete names of the acronyms the first time they are used (MDA in line 192, RF and GBDT in line 251) and do not use acronyms in the abstract.

In Figure 3, a reduced version of other figures does not communicate properly  what you mean. Please try a better way to 

In line 192, explain more thoroughly how MDA works.

It is not clear how k-means work, nor how its results are used. Please try to explain better.

Tables 3, 4 are the same. Table 6 is once more repeated but now the caption mentions training instead of validation areas. Did you extract the same number of training samples as validation samples? Please clarify and use just one table.

Figure 5 repeats Figure 1. Please unify.

Table 7 should appear within or near section 3.3. A justification of the parameters used is also needed.

MDC is a very old and naive classification method; today, it is only used for pedagogic purposes. It is not surprise that it gives bad results. Maximum likelihood should be used instead of MDC, as it is also an easy to use method (no parameters) quite more theoretically supported and with contrasted accuracy.

Whereas  Gradient boosting, and specially Random forest, are quite insensitive to the selected parameter values, SVM is very sensitive. It is quite rare that the first parameter set, no matter how thoughtful it may be, gives good accuracy results. However, if the parameters are optimized, the results are usually slighly better than those of RF or GBDT. So, if SVM is used, its parameters should be optimized.

Your conclusions are very conditioned by this problem. Although I prefer RF to SVM, precisely because it is easier to calibrate, I don not thin it is correct to conclude that SVM when the problem is that you have not optimized it correctly.

I think that your findings in relation with the time differences in accuracy for different crops are quite interesting, but our paper need more work in order to be publishable.

Best wishes

 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Thanks for addressing my comments carefully. I appreciate it. 

 I suggest an accept after two minor issues below were addressed. 

1. Line 229-230: Why do you have to put the article title here? Why don't you cite the paper just like how you did for other citations? like using [33] in the main text, and provide the details including title at the end?
2. Line 266: I made a mistake in the first-round comments. Please delete "were" from the sentence and change it back to "All the samples acquired ...." Sorry about it.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Thank you for thoroughly addressing my comments.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report

The authors addressed all comments and remarks. I believe that the manuscript can be accepted in its current form and considered for publication.

Reviewer 5 Report

The paper has improved substantially and I think it is now worth to be published. Good work!

Author Response

Please see the attachment.

Author Response File: Author Response.pdf