Responding to Large-Scale Forest Damage in an Alpine Environment with Remote Sensing, Machine Learning, and Web-GIS

Round 1

Reviewer 1 Report

i'm satisfied with changes and the manuscript is now ready for publication

Author Response

we thank the reviewer for her/his time.

Reviewer 2 Report

The article presents a semi-automated workflow for detecting and quantifying forest damage from windthrow in an Alpine region using VIs and machine learning; and shows results of testing NDVI, EVI, RGI, EWDI, NDMI and CI indices to identify which ones are more suitable for quantifying the damage.

 

The topic of the paper is suitable for Remote Sensing journal. However, the article seems a mixture of other works because this kind of machine learning approach was used previously in forests and crops, and the topic is not new, as it is well-known there are vegetation issues due to weather events, and this paper can not be considered a novelty due to the already mentioned well-known effect of weather events on NDVI or other indices.

The main problem is that it remembers more a technical report mixing current knowledge than scientific research. Therefore, the paper should be rejected.

 

Additional comments:

What was the "visual" methodology of the "field survey"? How was the ground data measured? In line 157: "visual inspection directly on site": The meaning is that technicians have taken ground-truth data in the forest/field? How? Did they select some trees randomly? The ground-truth dataset is essential in order to validate the results.

 

A revision of English is necessary. e.g.

Line 246: 'September is a month where photosynthesis rate, i.e. tree growth, is high in Alpine regions' sounds strange. It should be "The photosynthesis rate is higher in September…".

Line 263: values seem[s]

Line 354: 'having slightly better results' instead of 'having a slight better results'

Line 366: Can be  caused [by]

Etc.

 

What are the paper's novelty and the differences concerning previous works? It should be explained clearer in the article.

  

Explanations should link the results with technical/scientific interpretations (biology, physical, chemical…). e.g. Line 282: 'because the NDVI value reflects vegetation which can be either from existing surviving shrub (understorey) or from recovery'.

 

What do the authors think should be the next research steps? A 'further research' paragraph is needed in the conclusions.

 

The authors can consider time-series analysis articles, such as the following articles, which are related to the topic even when they use different techniques:

Verbesselt, J.; Hyndman, R.; Newnham, G.; Culvenor, D. Detecting Trend and Seasonal Changes in Satellite Image Time Series. Remote Sensing of Environment 2010, 114, 106–115, doi:10.1016/j.rse.2009.08.014.

Shikhov, A.N.; Perminova, E.S.; Perminov, S.I. Satellite-Based Analysis of the Spatial Patterns of Fire- and Storm-Related Forest Disturbances in the Ural Region, Russia. Nat Hazards 2019, 97, 283–308, doi:10.1007/s11069-019-03642-z.

Author Response

What was the "visual" methodology of the "field survey"? How was the ground data measured? In line 157: "visual inspection directly on site": The meaning is that technicians have taken ground-truth data in the forest/field? How? Did they select some trees randomly? The ground-truth dataset is essential in order to validate the results.

Answer: expert evaluators from the forestry sector went on-site to inspect and validate the severity of the damage over each area. No specific trees were selected as the value of interest was the overall severity of damage per area, not tree-specific. 

 

A revision of English is necessary. e.g.

Line 246: 'September is a month where photosynthesis rate, i.e. tree growth, is high in Alpine regions' sounds strange. It should be "The photosynthesis rate is higher in September…".

Line 263: values seem[s]

Line 354: 'having slightly better results' instead of 'having a slight better results'

Line 366: Can be  caused [by]

Etc.

Answer: The above modifications were applied and the paper was revised for English. 

 

What are the paper's novelty and the differences concerning previous works? It should be explained clearer in the article.

Answer: This paper uses vegetation indices (VIs) which are not a  novel approach, but the specificity of the work is in the assessment of this well-known approach to windthrow damage that caused different severity scenarios. In particular the effect of understorey, that decreases the potential of VIs for discrimination of severity. This has a cascading effect over the choice of better VIs that are less sensible to this factor and can be used for further analyses, e.g. through the mentioned online web-GIS platform and also further application of machine learning for  severity prediction.  

 

 

Explanations should link the results with technical/scientific interpretations (biology, physical, chemical…). e.g. Line 282: 'because the NDVI value reflects vegetation which can be either from existing surviving shrub (understorey) or from recovery'.

Answer: changed to “...NDVI value reflects vegetation properties thanks to the absorption of the red area of the spectrum by photosynthesis and the strong reflectance of near infrared light by the mesophyll. ”

 

What do the authors think should be the next research steps? A 'further research' paragraph is needed in the conclusions.

Answer: we have added a 'further research' paragraph at the end of the “conclusion” section.

 

The authors can consider time-series analysis articles, such as the following articles, which are related to the topic even when they use different techniques:

Verbesselt, J.; Hyndman, R.; Newnham, G.; Culvenor, D. Detecting Trend and Seasonal Changes in Satellite Image Time Series. Remote Sensing of Environment 2010, 114, 106–115, doi:10.1016/j.rse.2009.08.014.

Shikhov, A.N.; Perminova, E.S.; Perminov, S.I. Satellite-Based Analysis of the Spatial Patterns of Fire- and Storm-Related Forest Disturbances in the Ural Region, Russia. Nat Hazards 2019, 97, 283–308, doi:10.1007/s11069-019-03642-z.

Answer: we added these reference

Reviewer 3 Report

I have found the reading of this manuscript very interested. However, to be considered for publication it requires a discussion section where the authors compare their findings with relevant literature, and also would kindly ask the authors to consider the following comments for a potential improvement.

Line 28- There is not level 2B data from Sentinel-2. 

Line 153-158: This paragraph shouldn't be part of the study area. Authors are explaining how the severity was assessed, so I suggest authors to consider to add this into the next section "Field survey" or create a new one to explain it.

Line 171- In this section authors should explain a bit more about the Sentinel-2. They stated that cloud-free Level-2 Sentinel-2 data were acquired, then authors should know that Sentinel-2 Level-2A products are an an orthoimage Bottom-Of-Atmosphere (BOA) corrected reflectance product which means it has been corrected atmospherically, in this case, the software and the algorithm used should explain. Also, they should clarify the meaning of cloud-free, is it 0% cloud cover? If so, then Line 229 should be clarified.

Line 219-Please change section "2.4" to "2.5".

REFERENCES:

Line 520 - Reference 38 - Please cite this reference properly:
P. Cortez, Data Mining with Neural Networks and Support Vector Machines using the R/rminer Tool. In P. Perner (Ed.), Advances in Data Mining -Applications and Theoretical Aspects, 10th Industrial Conference on Data Mining, LNAI 6171, Springer, pp. 572-583, Berlin, Germany, July, 2010.

Author Response

I have found the reading of this manuscript very interested. However, to be considered for publication it requires a discussion section where the authors compare their findings with relevant literature, and also would kindly ask the authors to consider the following comments for a potential improvement.

Line 28- There is not level 2B data from Sentinel-2. 

Answer: Corrected the type to level 2A

Line 153-158: This paragraph shouldn't be part of the study area. Authors are explaining how the severity was assessed, so I suggest authors to consider to add this into the next section "Field survey" or create a new one to explain it.

Answer: we agree and have changed position of the paragraph in the following section “Field survey”

 

Line 171- In this section authors should explain a bit more about the Sentinel-2. They stated that cloud-free Level-2 Sentinel-2 data were acquired, then authors should know that Sentinel-2 Level-2A products are an an orthoimage Bottom-Of-Atmosphere (BOA) corrected reflectance product which means it has been corrected atmospherically, in this case, the software and the algorithm used should explain. Also, they should clarify the meaning of cloud-free, is it 0% cloud cover? If so, then Line 229 should be clarified.

 

Answer: We used Level-2 Sentinel-2 images that were corrected for atmospheric effects using ESA’s sen2cor - we added a line describing this and a reference to the method. We were able to find cloud-free data in our area as referred to the study area, not the whole image. In the web-GIS only images with <20% cloud coverage are addressed, as a general threshold for any analysis. Scene classification is used for masking clouds - this will be improved in the future (see last part of conclusions, with added “future work” paragraph)

 

Line 219-Please change section "2.4" to "2.5".

Answer: Done

 

REFERENCES:

Line 520 - Reference 38 - Please cite this reference properly:
P. Cortez, Data Mining with Neural Networks and Support Vector Machines using the R/rminer Tool. In P. Perner (Ed.), Advances in Data Mining -Applications and Theoretical Aspects, 10th Industrial Conference on Data Mining, LNAI 6171, Springer, pp. 572-583, Berlin, Germany, July, 2010.

Answer: Reference was corrected

Reviewer 4 Report

The authors have tested the performance of some well-known vegetation indices in wind storm damage recognition when using optical area satellite images, here Sentinel-2 images. The authors state the objectives as follows: “ The specific goal of the paper is to report on results of testing NDVI, EVI, RGI, EWDI, NDMI and CI indices through machine learning and to identify which indices are more suitable for quantifying damage in areas in the Agordino forest hit by Vaia storm, considering also the disturbance and the recovery stage.”  The also present practical tools to carry out the analyses and write “The results have been integrated into a web-GIS application for supporting the public administration and stakeholders involved in forestry management.” The practical application seems to be one aspect in the manuscript.

The authors have used six Sentinel-2 images, two before the wind damage and four after the damage. The damaged areas were identified and delineated visually with aerial photos. The severity of the damage was assessed in the field in the terms of tree canopy loss with the scale of 0-100. The remaining understorey vegetation causes extra challenges in the analyses when affecting the spectral values of the bands. The number of the damage sites is 22 (a rather low). The areas of the sites range from 1 ha to 61.58 ha.

The authors have used the well-known estimation methods Support vector machine (SVM), Random Forest (RF), and nearest neighbours (KNN) in evaluating the performances of the Vis and in predicting the damages.

The effect of the damage on the VIs is demonstrated in several ways, e.g., calculating the statistics of the VIs for each site with one image after the damage and calculating the Spearman’s rank correlation coefficients between the average VIs and severity category for each scene (month). The authors demonstrates the temporal behaviour of the VIs using several graphs, also the effect of the understorey.

The VI statistics are derived from pixel level VIs. The correlation coefficients have been calculated using site level averages of the VIs if I have understood correctly are calculated (Table 2). Please clarify how the significance tests for the correlation coefficients have been calculated (Table 2).

Please also clarify which data are used with SVM, RF and KNN analyses. It is not clear whether you have used pixel level data or site level data. Using pixel level may cause a problem of spatial dependence between the test data and training data depending on how do you split the data into training data and validation data. The use of site level data causes other problems, a problem of a small number of observations in both training data and validation data and also the problem of the very different areal extent of the observations, different weight of the observations in the analyses. Similar may hold also when calculating the Spearman’s rank correlation coefficients.

More detailed descriptions would also be needed on how SVM, RF and KNN have been used due the fact that there are many parameters in these methods and many variations of the methods. Please clarify the calculation of the correlation coefficient and the adjusted R square (Table 3).

It would be reader-friendly to describe also how the importance of the variables are calculated although a reference is given.

The emphasis of the manuscript is in analysing the importance of the different vegetation indices in wind storm damage monitoring. The VIs indicate clear evidence of the canopy losses and correlation coefficient show dependence on the severity of the damage (the amount of the canopy loss). Some detailed information is, however, needed how the data sets are used and how exactly the estimation methods are employed. The test data represent one special area in Italy. Discussions would be interesting on how general the achieved results are. Could they be generalized to other conditions?  One more interesting aspect could be how accurately the VIs and the methods used predict the total area damaged, perhaps by severity categories. However, it seems to be beyond of the objectives of the manuscript.

The analyses are quite straightforward although multifaceted. The novelty aspects of the manuscript should be clarified and stated clearly.

I have only two detailed comments at this phase:

The material from section 3.3 Software development may suit better to be presented into an Appendix. It includes material not necessarily valid in a scientific article although it is  interesting for the users.

In the section Conclusions, you write “Finally, the graphs summarize the spatial statistic.” If you refer to Figure 4, it shows the temporal behavior of the VIs by the sites.

 

Author Response

The effect of the damage on the VIs is demonstrated in several ways, e.g., calculating the statistics of the VIs for each site with one image after the damage and calculating the Spearman’s rank correlation coefficients between the average VIs and severity category for each scene (month). The authors demonstrates the temporal behaviour of the VIs using several graphs, also the effect of the understorey.

The VI statistics are derived from pixel level VIs. The correlation coefficients have been calculated using site level averages of the VIs if I have understood correctly are calculated (Table 2). Please clarify how the significance tests for the correlation coefficients have been calculated (Table 2).

Answer: significance values are provided by R CRAN’s output, we used Spearman’s rank correlation for calculating R-squared and p-values were provided.

 

Please also clarify which data are used with SVM, RF and KNN analyses. It is not clear whether you have used pixel level data or site level data. Using pixel level may cause a problem of spatial dependence between the test data and training data depending on how do you split the data into training data and validation data. The use of site level data causes other problems, a problem of a small number of observations in both training data and validation data and also the problem of the very different areal extent of the observations, different weight of the observations in the analyses. Similar may hold also when calculating the Spearman’s rank correlation coefficients.

Answer: more info was provided in the text in the last paragraph in section 2.4. Site-level data were used - the full dataset consists of 132 measurements, that corresponds to 22 sites in each of the 6 available images. The full dataset has been randomly split in two subset of 60% for training and 40% for validation. Training and testing have been done using K-fold (K=10) cross validation.  

More detailed descriptions would also be needed on how SVM, RF and KNN have been used due the fact that there are many parameters in these methods and many variations of the methods. 

Answer: We have used the automatic “search” method used for tuning the (hyper)parameters in rminer package (see reference from P. Cortez). We searched over 10 combinations and a polynomial kernel.  

Please clarify the calculation of the correlation coefficient and the adjusted R square (Table 3).

Answer: R-square is the accuracy metric that is provided by using the linear regression of observed and predicted values of the test set. It is an average value from ten predictions over each fold of the K-fold cross validation method.

 

It would be reader-friendly to describe also how the importance of the variables are calculated although a reference is given.

Answer: we prefer to leave a reference for the readers, as another paragraph would bring off-topic to the main focus of the paper.

 

The emphasis of the manuscript is in analysing the importance of the different vegetation indices in wind storm damage monitoring. The VIs indicate clear evidence of the canopy losses and correlation coefficient show dependence on the severity of the damage (the amount of the canopy loss). Some detailed information is, however, needed how the data sets are used and how exactly the estimation methods are employed. The test data represent one special area in Italy. Discussions would be interesting on how general the achieved results are. Could they be generalized to other conditions?  One more interesting aspect could be how accurately the VIs and the methods used predict the total area damaged, perhaps by severity categories. However, it seems to be beyond of the objectives of the manuscript.

Answer: The study is local to the Agordino Region, which is representative of the severity values and of the morphology of the terrain and species distribution of these areas. Different areas should be further tested to assess the replicability, but it is beyond the specific objective of this work. 

The analyses are quite straightforward although multifaceted. The novelty aspects of the manuscript should be clarified and stated clearly.

Answer: This paper uses vegetation indices (VIs) which are not a  novel approach, but the specificity of the work is in the assessment of this well-known approach to windthrow damage that caused different severity scenarios. In particular the effect of understorey, that decreases the potential of VIs for discrimination of severity. This has a cascading effect over the choice of better VIs that are less sensible to this factor and can be used for further analyses, e.g. through the mentioned online web-GIS platform and also further application of machine learning for  severity prediction.  

 

I have only two detailed comments at this phase:

The material from section 3.3 Software development may suit better to be presented into an Appendix. It includes material not necessarily valid in a scientific article although it is  interesting for the users.

Answer: Done, we have moved the section 3.3 in Appendix A

In the section Conclusions, you write “Finally, the graphs summarize the spatial statistic.” If you refer to Figure 4, it shows the temporal behavior of the VIs by the sites.

Answer: we changed to “...the plots in figure 4 summarize the temporal behavior of the VIs at the sites”

Round 2

Reviewer 2 Report

The added information is enough. Now the article is more precise.
I strongly recommend extensive editing of the English language prior to publication.

Examples:

line 207, Hard to understand-sentence:

The values at each date in each area are aggregated to the average and standard deviation of the index values inside the polygon

line 13:

Vaia storm [in] October 2018

line 67:

in particular [were] many trees [were] left standing

line 75:

to [the] public...

line 221:

consists [of] the VIs...

 

and so on...

Author Response

The authors thank the reviewer for her/his time in evaluating the manuscript.

===

line 207, Hard to understand-sentence:

The values at each date in each area are aggregated to the average and standard deviation of the index values inside the polygon

Answer: sentence was changed to “Average and standard deviation of the index values inside the polygon were aggregat-ed for each date”

 

line 13:

Vaia storm [in] October 2018

Answer: changed

 

line 67:

in particular [were] many trees [were] left standing

Answer: changed

 

line 75:

to [the] public…

Answer: changed

 

line 221:

consists [of] the VIs...

Answer: changed

Reviewer 3 Report

Thank you very much for the correction and the changes made. Although, after reviewing the manuscript I haven't seen any discussion between authors results and relevant literature as I suggested. Authors only changed the heading of section 3 from "Results" to "Results and discussion" but there is not comment in this section that suggest any discussion of results.  

Author Response

Comment: Thank you very much for the correction and the changes made. Although, after reviewing the manuscript I haven't seen any discussion between authors results and relevant literature as I suggested. Authors only changed the heading of section 3 from "Results" to "Results and discussion" but there is not comment in this section that suggest any discussion of result.

 

Answer: we have reported relevant literature in introduction section, and have recalled in specific parts of the discussion section, adding specific references.

Reviewer 4 Report

The authors have clarified the text to some extent based on my comments 
on the first version.
One open question is how the KNN method was employed, what is the value of K, was any weighting of the features used or not.

My second remaining comment concerns the use of data sets.
In Section 2.4 the authors write:
"The dataset consists in the VIs from the 6 images over the 22 areas, for a total of 132 observations with VIs as predictors and severity as the target variable to predict.  All severity values before the event were set to zero. The dataset was split in 60% as a training
set and 40% as a testing set. The training consisted in using 10-k fold cross-validation, ..."

To me, the number of observations is 22, not 132, although 22 sites is used with 6 images and thus 6 times.
Please clarify also how did you use "10-k fold cross-validation".
I think you mean 10-fold cross-validation, but still clarify 
how this small data set was divided in 10 subsets.

 

Author Response

We thank the reviewer for her/his time in evaluating the manuscript.

 

Comment: One open question is how the KNN method was employed, what is the value of K, was any weighting of the features used or not.

Answer: the default values of the kknn R package was used, with  k=7 and no pre-weighting of features. This was added in the paragraph in section 2.4

 

Comment:  In Section 2.4 the authors write:

"The dataset consists in the VIs from the 6 images over the 22 areas, for a total of 132 observations with VIs as predictors and severity as the target variable to predict.  All severity values before the event were set to zero. The dataset was split in 60% as a training set and 40% as a testing set. The training consisted in using 10-k fold cross-validation, ..."

To me, the number of observations is 22, not 132, although 22 sites is used with 6 images and thus 6 times.

Answer: it is correct to state that we have 22 areas, thus 22 severity values, but each area is sampled by the imagery at 6 different dates, therefore we have 132 different rows of predictors that point to 22 possible severity values.

 

Comment:  Please clarify also how did you use "10-k fold cross-validation".

I think you mean 10-fold cross-validation, but still clarify how this small data set was divided in 10 subsets.

Answer: true, 10-fold is correct, K=10 - this part was rewritten to make it more clear. 14 rows were used for testing and 118 rows for training.

This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.

Round 1

Reviewer 1 Report

The manuscript with the title “Responding to large-scale forest damage in Alpine environment with remote sensing, machine learning, and webGIS” reported a semi-automated workflow for detecting and quantifying forest damage severity from windthrow in an Alpine region using spectral vegetation indices calculated from Sentinel-2 data. The manuscript takes a worthy topic. However, before publication, there are some questions/revisions that need to be addressed.

Major comments

• Why did the author not use the topographical variables (slope and aspect) along with the spectral indices? It is essential to evaluate all possible factors that may affect the difference in spectral signature over healthy and wind damages forest stands.
• I recommend using red-edge based indices as well, like (NDRE). This will help detect if the changes in NDVI value are due to seasonal variation or damage.

Minor comments:

Abstract: Please indicate the remote sensing data (Sentinel-2). So the reader can easily understand what type of remote sensing data has been used in this study.

L 16: add ‘data’ after ground truth

L16-17: Results show that NDVI and NDMI average values decrease in the damaged areas and have a strong negative correlation with damage severity  change to Results show that the mean value of NDVI and NDMI decreased in the damaged area and have a strong negative correlation ( ? )  with damage severity index.

L19: How much Increased? please indicate

Introduction:

L35-41: I recommend moving this paragraph. First, you need to talk about the problem and its background and then monitor and detect it.

L42: Wind following by bark beetle infestation are ………..

L45 According to…. Check the citation style

L 113: which pest? I assume it's bark beetle as it has three different sages!

L113-120: I don’t think we need such detailed information about bark beetle infestation! Because the primary goal of this study is wind storm damage, not bark beetle outbreaks!

Materials and Methods

L147: Please indicate the year in which the field data has been collected!

L148: Did the authors use any criteria to calculate the damage severity degree? e.g., counting the number of fallen trees within particular plots or ….?

L150: What did the author mean by canopy vigor?

L150-154: How many plots were collected? And what was the size of the collected plots? It's not clear how the ground truth data linked to the satellite data?

L188-194: This section has to be explained in more detail.

Results

L215-220: I recommend comparing the results of both pre, and after damage together; also, the name of the selected plots, such as VOA_02 and TA_02, are not explained before in the methodology section.

Reviewer 2 Report

The authors presented an assessment of forest damage severity from windthrow using several vegetation indices and a web application to aid forest management. I think this paper is not suitable for publication with Remote Sensing. My main concern is that the novelty level of this ms very low. It is indeed quite simple, and I do not see any “new” interest findings. In my opinion, this is a presentation of an application rather than a research article.

Other comments:

1. The authors should describe clearly how they determined severity level.
2. Why not integrate Sentinel-1 data? It would help separate shrub and tree canopy better (due to the use of vegetation indices).
3. Is the web application complete? I do not see its link here.
4. I would use histogram of NDVI instead of Figure 5