Comparison and Assessment of Different Land Cover Datasets on the Cropland in Northeast China

Round 1

Reviewer 1 Report

1、 The authors' investigation of current land use/land cover datasets is still insufficient, for example, ESA has released the World Cover 2020/2021 dataset, ESRI also has multi-year data products, CLCD has been updated to 2022, and CLUD also has data products of 2018. Scientists such as Peng Gong, Liangyun Liu, Bingfang Wu also released various land cover data products, while there are several meter or sub-meter scale land cover datasets are published with articles. The Third National Land Survey led by the Ministry of Natural Resources of China has also accumulated higher-resolution data products, and the authors are advised to add relevant content.

2、 The north arrow in the figure is all messed up and need to update.

3、 There are some blank areas in Figure 1, please check it

4、 The authors compare the spatial consistency of different data products and should also compare the temporal consistency of the long time series data product

5、 The authors need to describe in detail how and on what basis the sample points used for accuracy evaluation are generated, and it is usually unscientific to use sample points for accuracy evaluation of ground cover products, because there is generally no error within the parcel, and errors tend to occur at the boundaries, but the boundaries vary greatly for manual interpretation for different resolution images, and there is also a lot of subjective judgment, which does not ensure that the sample points fall as close as possible to the boundaries of the cropland while correctly determining its type.

6、 Line 397-398, Random Forest is a kind of Machine Learning technology, please rewrite the relevant descriptions

7、 Line 475, these cities are parts of Jilin Province, not Liaoning Province

8、 Please further standardize the figures and tables so that they are clear, concise and easy to understand, with complete information in the figure titles or figure notes.

9、 Different data sources vary greatly, the difference in resolution directly leads to spatial scale problems, the authors directly resampled to the same resolution is somewhat inappropriate, 30 m image can not see the texture of 10 m, and higher resolution is also faced with the problem of subcategory information subsumption, in short, different resolution data have different applicable scenarios. Northeast China has large land parcels with small inter-annual variations and relatively homogeneous crop cultivation, and the differences in different data products in this region are much smaller than in the mountainous areas of southern China with complex topography and fragmented land parcels, and the assessment or production of these highly varied and complex arable land products will be of greater significance for agricultural production and food security.

1、 Xingan——Xing’an

2、 Nen River——Nenjiang River

3、 Figure 1, Capital——Capital City

4、 ESRI should be all capitalized

5、 Please verify subscripts for text writing, such as Line 302/304

6、Daxing'an in Line 132, but Da Xing'an in Line 422, also HulunBuir and Hulunbair, please standardize

Author Response

Dear Reviewer #1:

We would like to express our sincere appreciations for your efforts on reviewing our submission. As shown in the revision, we carefully considered the concerning points and made efforts to improve the rigor, logic, and clarity of this manuscript, and carefully responded to all of the comments and concerns.

Thanks again to the careful inspection and professional comments, the normalization problems of figures and table are corrected accordingly. We made changes to keep it clear, organized, and easy to understand.

According to your comments/ suggestions, we made clarifications in the manuscript and attach the detailed item-by-item response to all comments and suggestions for the evaluation.

Yours sincerely,

Chunqiao Song and co-authors

--------------------------------------------------

Chunqiao Song

Professor, Nanjing Institute of Geography & Limnology (NIGLAS), Chinese Academy of Sciences(CAS)

https://scholar.google.com/citations?user=4N1olTEAAAAJ&hl=en&oi=ao​

http://wsgs.niglas.cas.cn/english/Staff/Professors/201712/t20171221_391509.html

https://www.researchgate.net/profile/Chunqiao-Song

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors performed an accuracy assessment of thirteen sets of global or national-scale land cover datasets on the cropland through visual interpretation of the high-resolution image, and ground “truth” samples data across Northeast China. Overall it is an interesting study and will contribute to the accuracy of different land covers before their use in different applications.

Major Comments

I have a main concern about the use of visual interpretation for accuracy assessment instead of in-situ field visits.

My second comment is the inconsistency between spatial resolutions of coarse land cover products (300m-5km) and randomly generated point validation data obtained by manual interpretation. My third main concern is the use of different random points for every four phases, so authors are using a total of about 8000 points and visually interpreted for crops and non-cropland. Also, it is quite difficult to find same-season historical google earth images.

Minor Comments

Lines 10-11: Please rewrite it for clarity.

In Figure 1, please use the standard north arrow.

Author Response

Dear Reviewer #2:

We would like to express our sincere appreciations for your efforts on reviewing our submission. As shown in the revision, we carefully considered the concerning points and made efforts to improve the rigor, logic, and clarity of this manuscript, and carefully responded to all of the comments and concerns.

Thanks again to the careful inspection and professional comments, the normalization problems of figures and table are corrected accordingly. We made changes to keep it clear, organized, and easy to understand.

According to your comments/ suggestions, we made clarifications in the manuscript and attach the detailed item-by-item response to all comments and suggestions for the evaluation.

Yours sincerely,

Chunqiao Song and co-authors

--------------------------------------------------

Chunqiao Song

Professor, Nanjing Institute of Geography & Limnology (NIGLAS), Chinese Academy of Sciences(CAS)

https://scholar.google.com/citations?user=4N1olTEAAAAJ&hl=en&oi=ao​

http://wsgs.niglas.cas.cn/english/Staff/Professors/201712/t20171221_391509.html

https://www.researchgate.net/profile/Chunqiao-Song

Author Response File: Author Response.pdf

Reviewer 3 Report

The article text presented for the reviewing leaves a very positive impression. Authors conducted a massive work. The text is well-structured and has a high quality in the meaning of language using.

Criticism:

- To exclude confusing of a reader it is better to point directly “Supplementary Table S1” instead of “Table S1” (same for tables S2 and S3);

- Tables S2 and S3 are not remarked in any way in the Supplementary materials section;

- The color selected in the legend to mark the study area can not be observed (is not presented?) in the map; Some white-color artifacts are observed in the map at the river meanders;

- Table 1 is better to be repositioned to exclude its cut by page brake in the printable version of the article;

- When inserting in the text symbols used in the formulas, please format the symbols according to formulas format (lines 300-332) – both, in the meaning of using italic style, and in the meaning of using subscripted and superscripted elements – the symbols in the text paragraph have to have the same view to the symbols used in a formula;

- Figure 2 and 3 – the “±” symbol is the upper left angle is confusing, is it the artifact appeared in some way instead of north arrow(?); The labels at the west map frame have to be added;

- Figure 5 is better to be separated into four figures to gain higher readability, and to ensure easier formatting of printable version of the article (as now it is cut by page break); Same for Figure 6 (two figures – nine plates and six plates separately); Same for Figure 8 (two figures)

-  The scale bar are needed in satellite imagery fragments in Figure 7;

- The labels at the west map frames have to be added for the maps in Figure 8;

- Figure 9 is better to be repositioned to exclude its cut by page brake in the printable version of the article.

Observed misprints:

- Line 348 – not-needed dot after the “coefficient”.

Author Response

Dear Reviewer #3:

We would like to express our sincere appreciations for your efforts on reviewing our submission. As shown in the revision, we carefully considered the concerning points and made efforts to improve the rigor, logic, and clarity of this manuscript, and carefully responded to all of the comments and concerns.

Thanks again to the careful inspection and professional comments, the normalization problems of figures and table are corrected accordingly. We made changes to keep it clear, organized, and easy to understand.

According to your comments/ suggestions, we made clarifications in the manuscript and attach the detailed item-by-item response to all comments and suggestions for the evaluation.

Yours sincerely,

Chunqiao Song and co-authors

--------------------------------------------------

Chunqiao Song

Professor, Nanjing Institute of Geography & Limnology (NIGLAS), Chinese Academy of Sciences(CAS)

https://scholar.google.com/citations?user=4N1olTEAAAAJ&hl=en&oi=ao​

http://wsgs.niglas.cas.cn/english/Staff/Professors/201712/t20171221_391509.html

https://www.researchgate.net/profile/Chunqiao-Song

Author Response File: Author Response.pdf

Reviewer 4 Report

This study undertakes a comprehensive and in-depth assessment of cultivated land classification accuracy within Northeast China, utilizing key metrics including Overall Accuracy (OA), Producer Accuracy, and User Accuracy across a diverse array of 13 global and national-scale land cover datasets. The precision of cultivated land delineation assumes paramount significance in the realms of food security, agricultural strategizing, and resource management, thus conferring a substantive research imperative upon this study.

However, as a study centered on the evaluation of product accuracy, it is imperative for the authors to comprehensively address potential factors that could influence the evaluation results during the experimental phase itself, rather than reserving their impact for the discussion section. For instance, challenges related to divergent field definitions, classifications, and mismatches between validation sample intervals and product data can be effectively resolved through an expansion of the experimental design.

Furthermore, although this study successfully identified datasets that exhibited high accuracy, there is a pressing need for a more extensive comparative analysis. This entails offering a more comprehensive description and analysis of the strengths and weaknesses inherent in each dataset, accompanied by a robust discussion elucidating the underlying reasons for the disparities in dataset performance.

Minor:

Line 135: Figure 1.

Figure 1: The quality of Figure 1 needs to be improved, such as the use of map symbols, and the proper use of legends.

Section 2.2.1: In addition to the temporal coverage summary provided in Table 1, it is essential to include a corresponding explanation within the main body of the description.

Line 225: 2.2.2 Other auxiliary dataset

Line 245: What is the rationale behind selecting 2,000 samples for each year? Please provide an explanation for the appropriateness of this sample size. Additionally, could you discuss whether the distribution of samples across different crops has an impact on verification accuracy?

Section 2.4.1 and 2.4.3: The formula does not specify the parameters used (such as X_+i, X_i+, x_i and y_i), and it is possible that the author has misreported or incorrectly described these parameters.

Figure 3: It is advisable to present the distribution of sample points for each period separately. If concerns regarding text length arise, this information can alternatively be included in an appendix.

Author Response

Dear Reviewer #4:

We would like to express our sincere appreciations for your efforts on reviewing our submission. As shown in the revision, we carefully considered the concerning points and made efforts to improve the rigor, logic, and clarity of this manuscript, and carefully responded to all of the comments and concerns.

Thanks again to the careful inspection and professional comments, the normalization problems of figures and table are corrected accordingly. We made changes to keep it clear, organized, and easy to understand.

According to your comments/ suggestions, we made clarifications in the manuscript and attach the detailed item-by-item response to all comments and suggestions for the evaluation.

Yours sincerely,

Chunqiao Song and co-authors

--------------------------------------------------

Chunqiao Song

Professor, Nanjing Institute of Geography & Limnology (NIGLAS), Chinese Academy of Sciences(CAS)

https://scholar.google.com/citations?user=4N1olTEAAAAJ&hl=en&oi=ao​

http://wsgs.niglas.cas.cn/english/Staff/Professors/201712/t20171221_391509.html

https://www.researchgate.net/profile/Chunqiao-Song

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Thanks to the authors for making detailed revisions according to the review comments, but there are some small issues that need to be corrected.

Line 254, t This ----This

Line 536, 600*10³-----6*10⁵

Author Response

Dear Reviewer #1:

We would like to express our sincere appreciations for your efforts on reviewing our submission. Thanks again to the careful inspection and comments, the normalization problems of figures are corrected accordingly. We made changes to keep it clear, organized, and easy to understand.

According to your comments/ suggestions, we made clarifications in the revised manuscript and attach the detailed item-by-item response to all comments and suggestions for the evaluation.

Yours sincerely,

Chunqiao Song and co-authors

Author Response File: Author Response.pdf

Reviewer 2 Report

The authors have revised the manuscript by incorporating my suggestions. I will recommend it for publication.

Author Response

Dear Reviewer #2:

We would like to express our sincere appreciations for your efforts on reviewing our submission. In response to the comments raised by other reviewers, we made additions in the discussion section to keep it clear, organized, and easy to understand.

Thank you again for your recognition of this study and recommendation for acceptance.

Yours sincerely,

Chunqiao Song and co-authors

Author Response File: Author Response.pdf

Reviewer 4 Report

The authors have addressed most of my previous concerns. Nevertheless, it is still advisable for the authors to further improve the accuracy of the evaluation results by taking into account the experimental design aspect when addressing the two major comments mentioned in the last review. Once this is done, I believe the article can be confidently considered for acceptance.

Author Response

Dear Reviewer #4:

We would like to express our sincere appreciations for your efforts on reviewing our submission. As shown in the revision, we carefully considered your concerned points and also made efforts to improve the rigor, logic, and clarity of this manuscript. According to your comments/suggestions, we reorganized a few sections in Results and made additions in the discussion section of the manuscript. The detailed explanations can be found in the response letter below.

Yours sincerely,

Chunqiao Song and co-authors

Author Response File: Author Response.pdf