A Comparison of Random Forest Algorithm-Based Forest Extraction with GF-1 WFV, Landsat 8 and Sentinel-2 Images

Round 1

Reviewer 1 Report

This paper compared the performance of three data (GF1, L8 and S2) in FNF, which has some certain meaning to classification. However, as far as the article itself is concerned, there are still many aspects to be improved. 

1. It is insufficient in data analysis. In addition to the product processing levels, the factors that affect the classification results also include the spectral range, the accuracy of the preprocessing algorithm, and the time phase difference. In this paper, only time phase difference is considered.

2. The testing and training samples of 23 study areas are determined by visual interpretation? or  from GFC30? If from GFC30, how is the accuracy of the product in these areas? As far as I know, the data used in GFC30 comes from Landsat and GF. Whether it will affect the comparison results?

3. Line 27. Table 1. The time phase difference may cause large classification error. The FNF may have changed on images in different dates. Phenology and fire may be caused by the change of FNF in a certain period. In my opinion, the images with a date interval of more than 10 days is no longer meaningful.

4. Figure 3, 4, 5, 6, 7 all have no axis labels, please add.

5. Line 165, Table 2. Please add the calculation formulas of these VIs.

6.  Line 234 and 235. VOA and VF1 should be displayed with subscripts.

7. Line 251, Table 5. 'w.' should be changed by 'w/' ?

8. Line 286. This is 'Table 6', not 'Table 1' , and the title of the table is wrong.

9. Figure 8-11, what data source and date does the true images of 'i' come from? If add details of 'ii', 'iii' and 'iv', the results would be more obvious. In the following text analysis of these Figures, time phases, abnormal events like mountain fire, hillside shadows caused by observation angles  (Figure 10) should also be taken into account.

Author Response

Response to Reviewer 1 Comments

Dear Reviewer,

Thank you very much for your suggestions and comments on the manuscript “A Comparison of Random Forest Algorithm Based Forest Extraction with GF-1 WFV, Landsat 8 and Sentinel-2 Images”.

We have carefully considered all comments and suggestions from you and incorporated them in the revised version. In this letter, your comments and suggestions and point-to-point response were provided for your quick reference.

Note in the response document: Questions/comments of the reviewers were copied for a quick reference, and they were shown in italic red.

 

This paper compared the performance of three data (GF1, L8 and S2) in FNF, which has some certain meaning to classification. However, as far as the article itself is concerned, there are still many aspects to be improved.

 

1. It is insufficient in data analysis. In addition to the product processing levels, the factors that affect the classification results also include the spectral range, the accuracy of the preprocessing algorithm, and the time phase difference. In this paper, only time phase difference is considered.

Response:

Many thanks for your comments. As you mentioned, there are many factors affecting the classification accuracy, something like the spectral range, the accuracy of the preprocessing algorithm, and the time phase difference. In this study, we focus on the influence of factors such as processing level, band setting, and vegetation index。

For the accuracy of preprocessing algorithms are concerned, they do have a large impact on the classification. However, for each satellite data, we chose the standard preprocessed products or the preprocessing algorithm officially published by the satellite operating agency.How does this affect the classification results?how to address such kind of affecting? It really would be an issue that deserves to be explored in depth in the future.

Since the wavelengths of electromagnetic waves radiated and reflected by the ground objects are different, the spectrum might have a very strong influence on the classification accuracy. In this paper, we did not pay much attention on the influnce of spectral range, intead, the influence of band setting,which is much relating to spectral range, was discussed. In particular, we analyze the effect of different band settings on the classification accuracy based on Sentinel-2 and Landsat-8 data in Section 3.2.

 

2. The testing and training samples of 23 study areas are determined by visual interpretation? or from GFC30? If from GFC30, how is the accuracy of the product in these areas? As far as I know, the data used in GFC30 comes from Landsat and GF. Whether it will affect the comparison results?

Response:

Essentially, the test and training samples in this paper are determined by visual interpretation. In fact, the spatial and quantitative distribution of the samples in this paper is based on a stratified random sampling strategy. This strategy takes into account the spatial inconsistency of existing forest products to ensure the spatial and quantitative rationality of the sample point distribution. And the sampling sample points are visually interpreted (we will introduce the sample point generation strategy in the subsequent work).

 

3. Line 27. Table 1. The time phase difference may cause large classification error. The FNF may have changed on images in different dates. Phenology and fire may be caused by the change of FNF in a certain period. In my opinion, the images with a date interval of more than 10 days is no longer meaningful.

Response:

We strongly agree with you that the time phase difference do have an impact on forest classification accuracy. In terms of the time phase difference, phenology may be changed and fire might happen for the forest in a certain area, which may result in classification error. We took this aspect into consideration and therefore selected images with similar time phase as much as possible when selecting images, but it is really difficult to obtain cloud-free data from multiple sensors on the similar dates,especially in regions with lower latitudes strongly influenced by the monsoon climate,particularly for those satellites having longer revisit time interval,like Landsat 8.

Forest changes may happen due to forest fire, human activities, etc,.In Section 3.3, we compare the FNF classification accuracy of different satellite data. We selected four regions with representative classification differences and analyzed the reasons for the differences in classification accuracy among the selected regions for the three satellite data. We checked carefully that the study areas selected were not affected by fire and human activities during the study period. 

Images acquired at different time phases may reflect the characteristics of forest at different phenology, which might have a certain degree of influence on FNF classification. For this consideration, we therefore selected images in the green season of the forest with similar acquisition dates as far as possible to reduce the influence on the classification results. However,  it is still impossible to collect data from all three satellites in an expected time interval.Therefore we have relaxed the data collection time to the whole growing season in some areas,even back to the similar season the year last. Here, in order to address the influence of the forest phenology,we divide the test dataset in to two parts,one is the time interval of images from different sensor less than or equal to 10 days and the other is the time interval grater than 10days. The classification accuracy shows some differences in accuracy between the two test sets with different time intervals, as shown in Table 1 and Table 2. However, the results show that the two test sets agree in the conclusion that images form Sentinel-2 achieve the highest classification accuracy, followed by GF-1 WFV and Landsat 8 the lowest, although the differences are not remarkable.

 

Table 1 Classification of images from different sensors with time interval less than or equal to 10 days.

	OSR		30m
	mOA	mF1	mOA	mF1
GF	91.47	77.92	91.18	76.04
L8	91.63	74.32	91.63	74.32
S2	92.30	77.49	91.84	78.36

 

Table 2 Classification of images from different sensors with time interval greater than 10 days.

	OSR		30m
	mOA	mF1	mOA	mF1
GF	88.33	81.90	87.96	81.43
L8	86.20	81.49	86.20	81.49
S2	87.32	84.03	87.32	83.90

 

However, the influence of phenology on FNF still seem to be unexplained. Thus, we re-select images from Sentinel-2 for its strong data acquisition ability and maintained an interval of > 10 days from the original images, as shown in the Table3. The classification results show that there is indeed an effect of phenology on classification accuracy. However, in our experiments, phenology did not have a particularly strong influence on our conclusions, probably because we chose images that coincidentally avoided periods of intense phenological change.

In-depth exploration of the effect of phenology on the classification accuracy of FNF requires a large number of remote sensing images of different phenological periods. This is a significant challenge for this paper. However, how much does phenology affect classification accuracy? What is the extent of the effect of phenology on the classification accuracy for different latitudes or types of forests? These issues are deserve to be explored in the future.

 

Table 3 Acquisition time of images from Sentinel-2 for some study area.

Study areas	7	9	12	23
Time 1	2020/9/4	2020/10/23	2020/11/11	2020/8/24
Time 2	2020/9/19	2020/11/12	2020/11/6	2020/9/11

 

Figure 1 The Classification accuracy of 4 study areas. (a) and (b) represent OA and F1 score respectively.

 

4. Figure 3, 4, 5, 6, 7 all have no axis labels, please add.
5. Line 165, Table 2. Please add the calculation formulas of these VIs.
6. Line 234 and 235. VOA and VF1 should be displayed with subscripts.
7. Line 251, Table 5. 'w.' should be changed by 'w/' ?
8. Line 286. This is 'Table 6', not 'Table 1' , and the title of the table is wrong.

Response to question 4-8:

Thank you very much for your suggestions and for pointing out our mistakes. We have made changes in the corresponding parts of the manuscript.

 

9. Figure 8-11, what data source and date does the true images of 'i' come from? If add details of 'ii', 'iii' and 'iv', the results would be more obvious. In the following text analysis of these Figures, time phases, abnormal events like mountain fire, hillside shadows caused by observation angles (Figure 10) should also be taken into account.

Response:

Thank you very much for your suggestions,which provides new ideas for our analysis.

i denotes the high-resolution satellite map (ESRI satellite map or Google Satellite map) built into QGIS software, except for Figure 8 (study area 21, where the high-resolution satellite map is not an image of the growing season) from Sentinel-2.

Based on your suggestions, we have reworked Figures 8-11 to provide more detail and have updated them in the manuscript. In addition, we add an analysis of the classification errors due to shading in study area 23 (Figure 10).

 

 

Author Response File: Author Response.docx

Reviewer 2 Report

The authors analyzed and compared the performance of imagery from GF-1 WFV, Landsat 8, and Sentinel-2 satellites in forest/non-forest classification tasks using random forest algorithm (RF). The topic is highly significant for forest monitoring and fits well into journal's scope.

The introduction clearly explains the state of the art and the purpose of the work.

The work was carried out with in-depth bibliographic analysis and in a good scientific way.

 Some comments and suggestions:

Figure 1: Please add the brown color area in the legend. Brown is a non-forest area?

And I seriously suggest the authors remove the Nine-dash line from the sub-location map. This issue can cause reactions when it has not been recognized by the international community. The authors can refer to the boundary according to the world atlas https://www.worldatlas.com/maps/china.

3. Methods: The authors should provide what platform or software were used for images processing? 

In addition, it is also necessary to describe more clearly how to implement and use the input dataset in the process of analyzing and classifying images?

Figures 3, 4, 5, 6, 7: adding units to the vertical and horizontal axes

Line 23: Keywords: GF-1 WFV;

Line 41: more convenient and faster

Line 69: the forest fires that occurred in 2018 in

Line 160-161: including 4 bands with 10m resolution (B, G, R and NIR bands) and 6 bands with 20m resolution…

Line 181: σ and X mean

Line 224: The accuracies

Line 340: three sensors

Line 386: three satellite images

Lines 405, 467: accessed on ??

Author Response

Response to Reviewer 2 Comments

Dear Reviewer,

Thank you very much for your suggestions and comments on the manuscript “A Comparison of Random Forest Algorithm Based Forest Extraction with GF-1 WFV, Landsat 8 and Sentinel-2 Images”.

We have carefully considered all comments and suggestions from you and incorporated them in the revised version. In this letter, your comments and suggestions and point-to-point response were provided for your quick reference.

Note in the response document: Questions/comments of the reviewers were copied for a quick reference, and they were shown in italic red.

 

Comments to the Author

The authors analyzed and compared the performance of imagery from GF-1 WFV, Landsat 8, and Sentinel-2 satellites in forest/non-forest classification tasks using random forest algorithm (RF). The topic is highly significant for forest monitoring and fits well into journal's scope.

The introduction clearly explains the state of the art and the purpose of the work.

The work was carried out with in-depth bibliographic analysis and in a good scientific way.

Some comments and suggestions:

 

1. Figure 1: Please add the brown color area in the legend. Brown is a non-forest area?

Response:

Thanks to your suggestions. We have refined Figure 1 and updated it in the manuscript.

 

2. And I seriously suggest the authors remove the Nine-dash line from the sub-location map. This issue can cause reactions when it has not been recognized by the international community. The authors can refer to the boundary according to the world atlas https://www.worldatlas.com/maps/china.

Response:

Thanks a lot for your pertinent suggestions. Figure 1 is intended to illustrate the geographical location of our study area, and we do not intend to cause any dispute. Chinese law states that China's borders contain the nine-dash line, and the the boundary of the world atlas https://www.worldatlas.com/maps/china conflicts with the Legal China Map.

We have considered your suggestion very seriously. Therefore we investigated articles published in Remote Sensing and other journals. In the articles that dealt with the Chinese border (articles published in Remote Sensing [1-4] and articles published in other journals [5-7]), all of them included the nine-dash line.

 

3. Methods: The authors should provide what platform or software were used for images processing?

Response:

All the experiments were implemented using libraries such as Scikit-learn, Rasterio and Geopandas based on Python.

We have added some descriptions to the manuscript.

 

4. In addition, it is also necessary to describe more clearly how to implement and use the input dataset in the process of analyzing and classifying images?

Response:

The samples in this paper are stored as Shapefile in the form of points. Before inputting dataset into the RFC model (including training and testing), we calculate the corresponding pixel coordinates (P_pxl) of samples based on their spatial coordinates and the affine transformation matrix of the corresponding image. Then the image data is read and the value of each bands of P_pxl is taken out.

 

5. Figures 3, 4, 5, 6, 7: adding units to the vertical and horizontal axes
6. Line 23: Keywords: GF-1 WFV;
7. Line 41: more convenient and faster
8. Line 69: the forest fires that occurred in 2018 in
9. Line 160-161: including 4 bands with 10m resolution (B, G, R and NIR bands) and 6 bands with 20m resolution…

Response to question 5-9:

Thanks a lot for pointing out our mistakes. We have made corrections accordingly in the manuscript.

 

10. Line 181: σ and X mean

Response:

σ means the standard deviation of X. X donates a set of numbers, here indicating OA or F1 score for different study areas.

We have added some descriptions to the manuscript.

 

11. Line 224: The accuracies
12. Line 340: three sensors
13. Line 386: three satellite images
14. Lines 405, 467: accessed on ??

Response to question 11-14:

Thanks a lot for pointing out our mistakes. We have made corrections accordingly in the manuscript.

 

 

References：

 

1. Yu, H.; Ni, W.; Zhang, Z.; Sun, G.; Zhang, Z. Regional Forest Mapping over Mountainous Areas in Northeast China Using Newly Identified Critical Temporal Features of Sentinel-1 Backscattering. Remote Sensing 2020, 12, doi:10.3390/rs12091485.
2. Song, Q.; Hu, Q.; Zhou, Q.; Hovis, C.; Xiang, M.; Tang, H.; Wu, W. In-Season Crop Mapping with GF-1/WFV Data by Combining Object-Based Image Analysis and Random Forest. Remote Sensing 2017, 9, doi:10.3390/rs9111184.
3. Liu, L.; Zhou, L.; Ao, T.; Liu, X.; Shu, X. Flood Hazard Analysis Based on Rainfall Fusion: A Case Study in Dazhou City, China. Remote Sensing 2022, 14, doi:10.3390/rs14194843.
4. Xie, L.; Zhang, R.; Zhan, J.; Li, S.; Shama, A.; Zhan, R.; Wang, T.; Lv, J.; Bao, X.; Wu, R. Wildfire Risk Assessment in Liangshan Prefecture, China Based on An Integration Machine Learning Algorithm. Remote Sensing 2022, 14, doi:10.3390/rs14184592.
5. Liu, D.; Chen, N.; Zhang, X.; Wang, C.; Du, W. Annual large-scale urban land mapping based on Landsat time series in Google Earth Engine and OpenStreetMap data: A case study in the middle Yangtze River basin. ISPRS Journal of Photogrammetry and Remote Sensing 2020, 159, 337-351, doi:10.1016/j.isprsjprs.2019.11.021.
6. Sun, Q.; Zhang, P.; Sun, D.; Liu, A.; Dai, J. Desert vegetation-habitat complexes mapping using Gaofen-1 WFV (wide field of view) time series images in Minqin County, China. International Journal of Applied Earth Observation and Geoinformation 2018, 73, 522-534, doi:10.1016/j.jag.2018.07.021.
7. Peng, L.; Liu, K.; Cao, J.; Zhu, Y.; Li, F.; Liu, L. Combining GF-2 and RapidEye satellite data for mapping mangrove species using ensemble machine-learning methods. International Journal of Remote Sensing 2019, 41, 813-838, doi:10.1080/01431161.2019.1648907.

 

 

 

Author Response File: Author Response.docx

Reviewer 3 Report

In the paper “A Comparison of Random Forest Algorithm Based Forest Extraction with GF-1WFV, Landsat 8 and Sentinel-2 Images” by Xueli Peng the analysis and comparison of the performance of the forest classification from GF-1 WFV, L8 and S2 images was made. The factors affecting the accuracy of recognition of forest areas were noted. A decent number of samples were considered.

The article is clearly written, and the results presented in it are practically useful.

I think that it can be accepted to be published in Remote Sensing in the present form.

Author Response

Response to Reviewer 3 Comments

Dear Reviewer,

 

Thank you very much for your appreciation of our work. We have carefully considered all comments and suggestions from the reviewers and incorporated them in the revised version.

 

Best regards,

The authors.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

This paper do have certain guiding significance for subsequent applications, although there still some aspects that can be improved.