Round 1

Reviewer 1 Report

The authors borrowed the Global 30-m LC maps dataset to conduct a comparative analysis and accuracy evaluation of land cover  in the arid regions of northwest China before and after 2020.

In order to make it better for the reader, my suggestions are as follows:

1.The contribution and novelty of the paper should be described in the introduction.

2.The significance of the author's research in the introduction section needs to be added. For the section on global climate change, the climate change and pollution caused should be further described. The author's review of current research in the literature is not comprehensive enough in the introduction section, for example, citing research on PM2.5 pollutuion. See for example “PM2.5 volatility prediction by XGBoost-MLP based on GARCH models”and “Prediction of Air Pollutant Concentration Based on One-Dimensional Multi-Scale CNN-LSTM Considering Spatial-Temporal Characteristics”.

3. Section 3.1 should belong to the data section and section 3.2 Consistency analysis to the results section. It is recommended that the authors reprogram the third section rather than grouping it under methods.

4.What does the author mean by "However, (3)" in the second natural paragraph of section 4.4?

5.It is not sufficient for the authors to compare only FROM-GLC30, GLC_FCS30, and GlobeLand30 in the results section to illustrate the application of previous studies and to compare the results tested in this paper. A more in-depth discussion is needed to compare the accuracy of the three types of data applied in other regions and to highlight the contribution of this study.

Minor issues are as follows:

1. 2.2 Inconsistent formatting of section headings.

2. There is an extra comma in the last sentence of the first paragraph.

3. Inconsistent size of figure and table headings.

Please check the text carefully for details.

Author Response

Dear Editors and Reviewers:

Thank you for giving us the opportunity to revise the manuscript (ID: sustainability-2037243), entitled “Contrastive analysis and Accuracy assessment of three global 30-m land cover maps circa 2020 in arid land”.

We would like to thank the editors and reviewers for their careful works. The valuable comments and constructive suggestions are definitely helpful and we sincerely appreciate them for enhancing our paper. We have carefully considered all the points raised in the reviews and proofread our manuscript accordingly.

In the point-to-point response letter attached below, we detail out all the changes made during the revision. We hope that you will find this revised version satisfactory.

Thank you again for your consideration, and we look forward to your response.

 

Sincerely,

 

Qiang Bie (Corresponding author)

 

Reviewer 1

The authors borrowed the Global 30-m LC maps dataset to conduct a comparative analysis and accuracy evaluation of land cover in the arid regions of northwest China before and after 2020.

In order to make it better for the reader, my suggestions are as follows:

1.The contribution and novelty of the paper should be described in the introduction.

Response: Done as suggested.

We have highlighted the contribution and novelty of this paper in line 170-174.

 

“The present paper aimed to compare and validate three widely used global LC maps in arid land over Northwest China, including FROM30, GLC_FCS30, and GolbeLand30, in the terms of areal similarity, spatial consistency, and qualitative accuracy through error matrices. The study results can guide future improvements of LC mapping. Moreover, they also give advice for users to select the best LC map in arid regions.”

 

2.The significance of the author's research in the introduction section needs to be added. For the section on global climate change, the climate change and pollution caused should be further described. The author's review of current research in the literature is not comprehensive enough in the introduction section, for example, citing research on PM2.5 pollutuion. See for example “PM2.5 volatility prediction by XGBoost-MLP based on GARCH models”and “Prediction of Air Pollutant Concentration Based on One-Dimensional Multi-Scale CNN-LSTM Considering Spatial-Temporal Characteristics”

 

Response: Done as suggested.

We have added a discussion on the study of global land cover products in terms of environmental pollution, and cited another reference (Dai, H., Huang, G., Zeng, H., Zhou, F., 2022. PM_(2.5) volatility prediction by XGBoost-MLP based on GARCH models. J. Clean Prod., 356.) in introduction section.

 

3. Section 3.1 should belong to the data section and section 3.2 Consistency analysis to the results section. It is recommended that the authors reprogram the third section rather than grouping it under methods.

Response: Clarified and revised. 

Thank you for point them out. We have integrated the Section 2 “Materials and methods” and section 3 “Methodology” into one section “Materials and Methodology”. We hope that such reorganization will make the structure of the manuscript clearer and more reasonable. The revised version was shown in the Section 2. Meantime, according to Reviewer 2, we have simplified the description of datasets and methods.

 

4.What does the author mean by "However, (3)" in the second natural paragraph of section 4.4?

Response: Done as suggested. 

The “However” is inappropriate to put it here. We have deleted it and reshaped the paragraphs.

5.It is not sufficient for the authors to compare only FROM-GLC30, GLC_FCS30, and GlobeLand30 in the results section to illustrate the application of previous studies and to compare the results tested in this paper. A more in-depth discussion is needed to compare the accuracy of the three types of data applied in other regions and to highlight the contribution of this study.

Response: Done as suggested.

The contribution of this study is to evaluate performance of the classification accuracy of these global land cover products in arid areas, where the classification accuracy of some categories is not ideal. In the second part of Discussion section, we have supplemented the globally accuracy of the FROM-GLC30, GLC_FCS30, and GlobeLand30 reported from their corresponding producers. The comparison of accuracy of the three LC products in arid and globe were analyzed in section 4.2.

“The overall accuracy of GlobeLand 30 data in globe was 85.72% ^[18], and that of GLC_FCS30 was 82.5% ^[19]. In northwest China, the overall accuracy of Globeland 30 and GLC_FCS30 were 85.69% and 87.07, respectively, slightly higher than that in globe. The patterns of misclassification errors for LC maps…”

Minor issues are as follows:

1. 2.2 Inconsistent formatting of section headings.

Response: Done as suggested.

We have rewritten the section headings as “The 30-m global LC datasets”

 

2. There is an extra comma in the last sentence of the first paragraph.

Response: Done as suggested.

We have deleted the extra comma and reviewed the full text for similar problems.

 

3. Inconsistent size of figure and table headings.

Response: Done as suggested.

We have rewritten the headings of Figure 3 in Line 245, Figure 5 in Line 374.

“Figure 3. The consistency of the spatial distributions of (a) two LC maps and (b) three LC maps based on spatial superposition.”

“Figure 5. The spatial (in)consistency of distribution of (a) three LC maps, (b) FROM vs. FCS, (c) FROM vs. Globe, and (d) FCS vs. Globe.”

 

Please check the text carefully for details.

Response: Thank you for your suggestion

We have checked the grammar, spelling and punctuation of the full text in detail.

Author Response File: Author Response.docx

Reviewer 2 Report

Comments for Paper sustainability-2037243

The authors evaluated the performance of three LC maps and analyzed the consistency between them in Northwestern China. This is a meaningful work due to fine-resolution LC maps are vital fundamental datasets for designing urban planning under the call of sustainable development. The comparative works are relatively detailed and should be encouraged. However, the manuscript should be further improved before publication. The importance and necessity of the work should be enhanced in the [Introduction] section. The description of datasets and methods should be clearer and brief. And the results and key findings are suggested to be clarified or explained. Other comments are listed as follows.

1.     The page numbers and line numbers should be added. It is not convenient for peer review without line numbers.

2.     [Abstract] à The abstract is much longer than the maximum limitation. It should be revised according to the journal's guidelines.

3.     The second paragraph of [Introduction]à It’s no need to list the coarse-resolution LC maps one by one. I suggest to summary their similarity and difference, as well as advantages and disadvantages. Based on the information, please clarify the necessity of your work. In addition, the opinion should be objective, e.g., “some studies pointed out that…” which studies? Why they thought the products couldn’t meet the requirement? At least you should supplement the citation or example.

4.     “In contrast to 10 m resolution LC, the 30m resolution global LC maps such as GlobeLand30, GLC_FCS30, and FROM30 are commonly used in many fields [17, 18, 24-26].” à In this paragraph, you introduce many LC products at 10 m spatial resolution. As you mentioned above that finer resolution and more accurate data is of great concern, it seems that the LC maps in 10m spatial resolution is better. The reason you focus on LC maps in 30 m, but not 10 m, should be stronger or more convincing. Maybe you can explain the point from the aspect of data attributes, such as the long time series of Landsat images, and among others.

5.     Figure 1à The map scales are missed.

6.     Section 2.2à Please supplement the dates you acquired the datasets.

7.     “level 1 types” and “level 2 types”à Please replace them by “first-level types” and “second-level types”. You can refer to the terms applied in http://dx.doi.org/10.1016/j.rse.2014.04.004.

8.     Table 1à Please change “primary LC types” into “First-level types”.

9.     [Methodology]à Please supplement the flowchart involving the conceptual framework and procedures of the harmonization, consistency analysis and accuracy quantifying. Accordingly, the description of contents could be simplified, and the sub-graphs (e.g., Figure2, Figure3 and Figure 4) could be moved into the flowchart.

10.  [Formula]à The explanation of variables in formulas doesn’t need any space before the term “where”.

11.  [Formula (2)] à The sign “×” is missed.

12.  “According to the accommodation”à Please change the term “accommodation” to a more appropriate one.

13.  Figure 4à “buile-up” or “built-up”?

14.  Discussionà Please try to explain the different performance of three LC maps in similarity and accuracy. Meantime, please supplement a simple outlook for the future application of existing LC maps, as well as the outlooks for the generation of future LC maps.  

Author Response

Dear Editors and Reviewers:

Thank you for giving us the opportunity to revise the manuscript (ID: sustainability-2037243), entitled “Contrastive analysis and Accuracy assessment of three global 30-m land cover maps circa 2020 in arid land”.

We would like to thank the editors and reviewers for their careful works. The valuable comments and constructive suggestions are definitely helpful and we sincerely appreciate them for enhancing our paper. We have carefully considered all the points raised in the reviews and proofread our manuscript accordingly.

In the point-to-point response letter attached below, we detail out all the changes made during the revision. We hope that you will find this revised version satisfactory.

Thank you again for your consideration, and we look forward to your response.

 

Sincerely,

 

Qiang Bie (Corresponding author)

 

Reviewer 2

The authors evaluated the performance of three LC maps and analyzed the consistency between them in Northwestern China. This is a meaningful work due to fine-resolution LC maps are vital fundamental datasets for designing urban planning under the call of sustainable development. The comparative works are relatively detailed and should be encouraged.

Response: Thank you for your positive comments and encouragement.

 

However, the manuscript should be further improved before publication. The importance and necessity of the work should be enhanced in the [Introduction] section.

Response: Done as suggested.

We have highlighted the contribution and novelty of this paper in line 170-174.

 

“The present paper aimed to compare and validate three widely used global LC maps in arid land over Northwest China, including FROM30, GLC_FCS30, and GolbeLand30, in the terms of areal similarity, spatial consistency, and qualitative accuracy through error matrices. The study results can guide future improvements of LC mapping. Moreover, they also give advice for users to select the best LC map in arid regions.”

 

The description of datasets and methods should be clearer and brief.

Response: Done as suggested.

We have deleted the description of methods for generation of the Globe, FROM, and FCS and data used to simplify the data description in Section 2.2. Furthermore, we have integrated the methodology to section 2 “Materials and Methodology”.

 

And the results and key findings are suggested to be clarified or explained.

Response: Done as suggested.

we have highlighted the results of this study in Abstract, Discussion, and Conclusion sections.

Abstract: “The results revealed that: (1) the consistent areas of the three maps accounted for 65.96% of the total area, and that of any two maps exceeded 75%; (2) The dominant land cover types, bare land and grassland, were the most consistent land cover types across the three products. In contrast, the spatial inconsistency of wetland, shrubland, and built-up area was relatively high, with the disagreement mainly occurring in heterogeneous regions; (3) the overall accuracy of the GLC_FCS30 map was the highest with a value of 87.07%, followed by GlobeLand30 (85.69%) and FROM-GLC30 (83.49%). Overall, all three LC maps were found to be consistent and have good performance in classification in the arid regions, but their ability to accurately classify specific types varied.”

Discussion: “First, the spatial consistencies between these three maps were conducted by area-based comparison and pixel-based comparison. The results showed that 65.96% of the pixels on three maps had identical classification labels. Regarding the LC types, the bare land, cropland, forest, and water were consistent in space. Secondly, the accuracy measures, including OA, PA, and UA, were obtained from error matrices using a validation dataset. The FCS product had the highest overall accuracy within the territory of northwestern China (87.07%), followed by Globe (85.69%) and FROM (84.39%).”

 

Other comments are listed as follows.

1.The page numbers and line numbers should be added. It is not convenient for peer review without line numbers.

Response: Done as suggested.

We have added the page numbers and line number for the convenience of peer review.

 

2.[Abstract] The abstract is much longer than the maximum limitation. It should be revised according to the journal's guidelines.

Response: Done as suggested.

Thank you for your suggestion. We have rewritten the abstract according to the journal’s guidelines. The revised version has 238 words.

“Fine-resolution land cover (LC) products are critical for studies of urban planning, global climate change, earth’s energy balance, and the geochemical cycle as fundamental geospatial data products. It is important and urgent to evaluate the performance of the updated global land cover mapsIn this study, three wildly used LC maps with 30 m spatial resolution (FROM-GLC30-2020, GLC_FCS30, and GlobeLand30) published around 2020 were evaluated in terms of degree of consistency and accuracy metrics. First, we compared their similarities and difference in area ratio and spatial patterns over land cover types. Second, the sample and response protocol was proposed, and validation samples were collected. Based on this, the overall accuracy, producer’s accuracy, and user’s accuracy were analyzed. The results revealed that: (1) the consistent areas of the three maps accounted for 65.96% of the total area, and that of any two maps exceeded 75%; (2) The dominant land cover types, bare land and grassland, were the most consistent land cover types across the three products. In contrast, the spatial inconsistency of wetland, shrubland, and built-up area was relatively high, with the disagreement mainly occurring in heterogeneous regions; (3) the overall accuracy of the GLC_FCS30 map was the highest with a value of 87.07%, followed by GlobeLand30 (85.69%) and FROM-GLC30 (83.49%). Overall, all three LC maps were found to be consistent and have good performance in classification in the arid regions, but their ability to accurately classify specific types varied.” 

3.The second paragraph of [Introduction] It’s no need to list the coarse-resolution LC maps one by one. I suggest to summary their similarity and difference, as well as advantages and disadvantages. Based on the information, please clarify the necessity of your work. In addition, the opinion should be objective, e.g., “some studies pointed out that…” which studies? Why they thought the products couldn’t meet the requirement? At least you should supplement the citation or example.

Response: Done as suggested.

Thank you for your suggestion. Due to space limitations and limited relevance, we deleted the detailed description of seven coarse resolution LC maps one by one, and retained a brief introduction to relevant work. In addition, we have cited references ^{[14, 15]} to support statements in the manuscript. The revised version was shown in the second paragraph as follows:

 

“The first satellite-based global LC map dates back to the 1990s ^[7]. Since then, with the improvement of satellite techniques and computer facilities, various global/regional land-cover maps with different resolutions based on specific classification schemes have been developed and released. Several types of LC maps with relatively coarse-resolution at 1,000 and 300 meter scales ^{[8, 9, 10, 11, 12, 13]} are available. While the coarser resolution global LC maps have given valuable information for various related applications, some studies ^{[14, 15]} pointed out that these products cannot meet the requirement of accuracy in regions with heterogeneous landscapes, which calls for finer resolution and more accurate data in these areas.”

 

4.“In contrast to 10 m resolution LC, the 30m resolution global LC maps such as GlobeLand30, GLC_FCS30, and FROM30 are commonly used in many fields [17, 18, 24-26].” In this paragraph, you introduce many LC products at 10 m spatial resolution. As you mentioned above that finer resolution and more accurate data is of great concern, it seems that the LC maps in 10m spatial resolution is better. The reason you focus on LC maps in 30 m, but not 10 m, should be stronger or more convincing. Maybe you can explain the point from the aspect of data attributes, such as the long time series of Landsat images, and among others.

Response: Done as suggested.

Thank you for pointing them out. we have added explanatory notes to the revised version.

“In contrast to 10 m resolution LC, the 30 m resolution global LC maps such as GlobeLand30, GLC_FCS30, and FROM30 are commonly used in many fields because of their long time series and robustness in land cover mapping ^{[17, 18, 24-26]}.”

 

5.Figure 1 The map scales are missed.

Response: Done as suggested.

We have redrawn the Figure 1 and added the map scales, which was shown as follows:

6.Section 2.2 Please supplement the dates you acquired the datasets.

Response: Done as suggested.

We have provided the dates we acquired the datasets in the manuscript. The revised version was shown in Line 158-19 as follows:

 

“In the present study, three widely used LC maps that are available freely were chosen for evaluation and analysis. These are the Finer Resolution Observation and Monitoring Global LC dataset based on Landsat images in 2017(abbr. FROM), available at: http://data.ess.tsinghua.edu.cn [16], Global land cover datasets in 2020 from National Geomatics Center of China (abbr. Globe), available at: www.globallandcover.com [17], and Global land-cover product with fine classification system at 30m using time-series Landsat imagery in 2020, (abbr. FCS) available at: https://doi.org/10.5281/zenodo.3986872 [18, 19]. All three datasets were downloaded in May 2022.”

7.“level 1 types” and “level 2 types” Please replace them by “first-level types” and “second-level types”. You can refer to the terms applied in http://dx.doi.org/10.1016/j.rse.2014.04.004.

Response: Done as suggested.

Thank you for your recommendation, we have replaced the “level 1 types” and “level 2 types” with “first-level types” and “second-level types” in revised version.

 

8.Table 1 Please change “primary LC types” into “First-level types”.

Response: Done as suggested.

Thank you for point them out. We have replaced “primary LC types” in Table 1 by “First-level types”.

 

9.[Methodology] Please supplement the flowchart involving the conceptual framework and procedures of the harmonization, consistency analysis and accuracy quantifying. Accordingly, the description of contents could be simplified, and the sub-graphs (e.g., Figure2, Figure3 and Figure 4) could be moved into the flowchart.

Response: Clarified and revised.

Thank you for point them out. The procedures of the harmonization, consistency analysis and accuracy measures calculation were conducted sequentially. We have integrated the Section 2 “Materials and methods” and section 3 “Methodology” into one section “Materials and Methodology”. we hope that such reorganization will make the structure of the manuscript clearer and more reasonable. The revised version was shown in the Section 2.

 

10.[Formula] The explanation of variables in formulas doesn’t need any space before the term “where”.

Response: Done as suggested.

Thank you for pointing them out. We have deleted the redundant space before the term “where” in the explanation of variables in formulas. The revised version was shown as follows:

“where X_i is the number of pixels of the i-th LC type in map X, Y_i is the number of pixels of the i-th LC type in map Y,…”

11.[Formula (2)] The sign “×” is missed.

Response: Done as suggested.

We have added the sign“×”as shown in formula (2):

 （2）

 

12.“According to the accommodation” Please change the term “accommodation” to a more appropriate one.

Response: Done as suggested.

We corrected the misspelling and replaced it with “recommendation”. The revised version was shown:

“According to the recommendation that”

13.  Figure 4 “buile-up” or “built-up”?

Response: Done as suggested.

The revised figure 4 was shown in Line 273.

 

14. Discussion Please try to explain the different performance of three LC maps in similarity and accuracy. Meantime, please supplement a simple outlook for the future application of existing LC maps, as well as the outlooks for the generation of future LC maps.

Response: Done as suggested.

We have added the statement to explain the performance of three LC maps in similarity and accuracy in the first paragraph of the Discussion section.

Author Response File: Author Response.docx

Reviewer 3 Report

You should not assume that all readers will be familiar with terminology such as producer's accuracy and user's accuracy.  Define all metrics you use with an appropriate formula.  You just might draw readership from outside the subdiscipline.

Author Response

Dear Editors and Reviewers:

Thank you for giving us the opportunity to revise the manuscript (ID: sustainability-2037243), entitled “Contrastive analysis and Accuracy assessment of three global 30-m land cover maps circa 2020 in arid land”.

We would like to thank the editors and reviewers for their careful works. The valuable comments and constructive suggestions are definitely helpful and we sincerely appreciate them for enhancing our paper. We have carefully considered all the points raised in the reviews and proofread our manuscript accordingly.

In the point-to-point response letter attached below, we detail out all the changes made during the revision. We hope that you will find this revised version satisfactory.

Thank you again for your consideration, and we look forward to your response.

 

Sincerely,

 

Qiang Bie (Corresponding author)

 

Reviewer 3

You should not assume that all readers will be familiar with terminology such as producer's accuracy and user's accuracy. Define all metrics you use with an appropriate formula. You just might draw readership from outside the sub-discipline.

Response: Done as suggested.

Thank you for your comments. We have provided the formulas for calculating the overall accuracy (OA), producer’s accuracy (PA), user’s accuracy (UA) and Kappa coefficients. The revised version was shown in section 3.3.3 as follows:

 

“The confusion matrix is a very effective way to represent thematic map accuracy, because it provides a clear way of deriving the individual accuracies of each class along with both the errors of inclusion (commission errors) and exclusion (omission errors) presented in the classification ^[41]. The overall accuracy (OA), producer’s accuracy (PA), user’s accuracy (UA) and Kappa coefficients were calculated to show the accuracy of three LC products ^{[27, 47]}. The formulae for calculating each indicator are as follows:

 

Where  is the correctly classified pixel number of type i, n is the total pixel number in the study area,  is total pixel number of type i in the map (data to be verified),  is the total pixel number of type j in the reference data (truth data), r is the number of rows in confusion matrix.”

Author Response File: Author Response.docx

Round 2

Reviewer 2 Report

Comments for Paper sustainability-2037243

Thanks to the authors for their efforts to modify the manuscript. Most of the issues have been resolved. However, there are still several problems existed. I suggest to public the manuscript after a further revision. The detailed comments are listed as follows.

1.     [Methodology]à “Please supplement the flowchart involving the conceptual framework and procedures of the harmonization, consistency analysis and accuracy quantifying. Accordingly, the description of contents could be simplified, and the sub-graphs (e.g., Figure2, Figure3 and Figure 4) could be moved into the flowchart.” I meant that a figure of the flowchart is necessary and the description of contents could be simplified.

2.     Discussionà” Please try to explain the different performance of three LC maps in similarity and accuracy.” I meant that you should explain why the LC maps show similarity in some land types and why they show difference in some other land types. I suggest to clarify the reason from the aspects of interpreting criteria, classification methods, datasets, and among others. The current description reads more like a summary of your results, but not a real discussion.

3.     Please supplement a simple outlook for the future application of existing LC maps, as well as the outlooks for the generation of future LC maps.

4.     [Formula] The explanation of variables in formulas should begin from the term “where” in lower case. Please check your entire manuscript, including but not limited to Line 305.

Author Response

Please see the attachment.

Author Response File: Author Response.docx