Statistical Stability and Spatial Instability in Mapping Forest Tree Species by Comparing 9 Years of Satellite Image Time Series

Round 1

Reviewer 1 Report

I reviewed the manuscript entitled “Statistical Stability and Spatial Instability in Mapping Forest Tree Species by Comparing 9 Years of Satellite Image Time Series“. The authors performed a tree species classification (13 species) on a study site in southern France using time series of satellite data, repeatedly over nine years (11 to 43 images per year). They compared accuracies obtained by a standard leave-one-out cross validation (LOOCV) with a spatial LOOCV, where autocorrelation between validation samples was avoided. Moran’s I was used to calculate spatial autocorrelation distance, and validation samples were selected applying the calculated distance as minimum distance between sample locations. Also, they investigated how accuracies vary across years when the same reference training and validation samples are used.

The problem of spatial autocorrelation between samples located close to each other is known and not new, and other approaches to handle this have been presented by other authors, what the authors of the present study correctly address in the discussion. However, this should be addressed in the introduction as well, and pointed out why it is interesting to do this using Moran’s I. The amount of satellite data (large number of satellite images per year, and time series for nine years) is impressive, even though the area of the test site is limited. However, it would be interesting to analyse, which and how many of the images in each time series are most important and relevant for classification (principal component analysis could be one option). Other authors reported good accuracies with only three or four images in a time series (compared to 11 to 43 in this study), what the authors address in the discussion, also.

In the introduction, the objectives should be clearly stated, and repetition should be avoided.

Despite these limitations, the conducted analyses are thorough, and the manuscript is well written. Figures and tables are of good quality.

Specific comments:

L64-70: paragraph repeated, delete

L75: add Latin names of tree species

L76: yes it differs, but it makes sense and is expected: better results are obtained with images from 4 dates and only 5 tree species compared to images from only 2 dates and 7 tree species

L79: 0.2 points of what – OA or kappa?

L83: add “T”

L102-103: “We also hypothesize…” This you already mentioned in describing your main contribution.

L107: Fig. or Figure. Follow journal guidelines and be consistent throughout the manuscript

Figure 3: explain acronym “TOC”

Section 3.2: SVM: which predictors were used? Only the bands, or as well certain indices or band rations? Were all images from each year used, or only a selection (e.g. spring, summer winter)?

L187: with “reference sample”, are you referring to the 1263 pixels verified in the field? Find a clear name and use it consistently throughout the manuscript.

Table 2: “Nb of imiages” Do you mean No.?

L256: repetition of results for Aspen.

L322-324: what are the differences between these approaches? Should one be preferred over others in specific situation? Or would they all result in similar accuracies?

L377: This is a new paragraph. What are you referring to with “this disturbance factor”? Clouds and their shadows?

Figure 12 caption last line. “To” instead of “top”?

 L411 “we” instead of “e”?

L 435-428: yes that would be very interesting!

Author Response

Dear reviewer,

We are very grateful of your review for your constructive comments on the first version of the manuscript.

We did our best to take into account your suggestions and we hope that the new version has been improved in term of quality.

We hope this new version is sufficiently improved to be published in Remote Sensing.

Please find below our answers to your comments. Our replies are written after each '>>'.

To make it easier for you to review, please refer to the attachment where the changes are colored in blue.

Kind regards,

Nicolas Karasiak & co-authors.

-----------------------------------------------

I reviewed the manuscript entitled “Statistical Stability and Spatial Instability in Mapping Forest Tree Species by Comparing 9 Years of Satellite Image Time Series“. The authors performed a tree species classification (13 species) on a study site in southern France using time series of satellite data, repeatedly over nine years (11 to 43 images per year). They compared accuracies obtained by a standard leave-one-out cross validation (LOOCV) with a spatial LOOCV, where autocorrelation between validation samples was avoided. Moran’s I was used to calculate spatial autocorrelation distance, and validation samples were selected applying the calculated distance as minimum distance between sample locations. Also, they investigated how accuracies vary across years when the same reference training and validation samples are used.

The problem of spatial autocorrelation between samples located close to each other is known and not new, and other approaches to handle this have been presented by other authors, what the authors of the present study correctly address in the discussion. However, this should be addressed in the introduction as well, and pointed out why it is interesting to do this using Moran’s I. The amount of satellite data (large number of satellite images per year, and time series for nine years) is impressive, even though the area of the test site is limited. However, it would be interesting to analyse, which and how many of the images in each time series are most important and relevant for classification (principal component analysis could be one option). Other authors reported good accuracies with only three or four images in a time series (compared to 11 to 43 in this study), what the authors address in the discussion, also.

In the introduction, the objectives should be clearly stated, and repetition should be avoided.

Despite these limitations, the conducted analyses are thorough, and the manuscript is well written. Figures and tables are of good quality.

>> We would like to thank you for the time you spent to review the manuscript and the suggestions you proposed.
We improved the introduction by explicitely stating the objectives of the paper (L99-101). We also enriched the part related to the spatial autocorrelation with the existing works (L92-95). The most important change in the manuscript is the additional analysis about the ranking based feature-selection of image dates, as you suggested (L427-457 + appendix E). We have chosen to put the information in the discussion section only because the detailed analysis was not the main objective of the paper. The conclusion has been updated accordingly (L497-504).

--------------------

Specific comments:

L64-70: paragraph repeated, delete
>> Deleted

L75: add Latin names of tree species
>> Added

L76: yes it differs, but it makes sense and is expected: better results are obtained with images from 4 dates and only 5 tree species compared to images from only 2 dates and 7 tree species
>> We agree with your comment. The sentence has been updated.

L79: 0.2 points of what – OA or kappa?
>> We specified it

L83: add “T”
>> Done

L102-103: “We also hypothesize…” This you already mentioned in describing your main contribution.
>> Done (sentence removed)

L107: Fig. or Figure. Follow journal guidelines and be consistent throughout the manuscript
>> We followed the MDPI/RS guidelines (E.g. "Figure 1. name of the figure").

Figure 3: explain acronym “TOC”
>> Updated in the Figure

Section 3.2: SVM: which predictors were used? Only the bands, or as well certain indices or band rations? Were all images from each year used, or only a selection (e.g. spring, summer winter)?
>> We clarified it (spectral bands only, L184)

L187: with “reference sample”, are you referring to the 1263 pixels verified in the field? Find a clear name and use it consistently throughout the manuscript.
>> We now specified that we use 'reference samples' to refer to the 1262 pixels in the rest of the paper (see section 2.4, L157-158).

Table 2: “Nb of images” Do you mean No.?
>> Changed to "Number"

L256: repetition of results for Aspen.
>> Deleted.

L322-324: what are the differences between these approaches? Should one be preferred over others in specific situation? Or would they all result in similar accuracies?
>> We specified that spatial autocorrelation was not quantified in the other approches (L335-336).

L377: This is a new paragraph. What are you referring to with “this disturbance factor”? Clouds and their shadows?
>> We corrected the sentence (L389 "misdetections of clouds and cloud shadows").

Figure 12 caption last line. “To” instead of “top”?
>> Changed to 'to'.

L411 “we” instead of “e”?
>> Changed to 'we'.

L 435-428: yes that would be very interesting!
>> Sorry, it was unclear for us on which paragraph/idea it was about.

Author Response File: Author Response.pdf

Reviewer 2 Report

The manuscript presents a piece of research of multi-year forest mapping based on one-year multiple dates remotely sensed image data. It might interest the forest management departments and remote sensing research. However, the follow aspects need to clarify during the revising period.

(1) I guess the feature sets for the classification are the four band reflectance, which should be addressed clearly in section 3. In addition, some derivatives from the optical images such as NDVI may be included into the classification feature sets (not just using the original spectral bands), to reflect the phenological difference among the different tree types.

(2) As for the data selection of different dates in the study, is it too simple just based on the availability? How about only select those image data with obvious phenological features? If the disparity is caused mainly by the difference of image data sets in different years, I doubt the meaning of the comparison study.

Author Response

Dear reviewer,

We are very grateful of your review for your constructive comments on the first version of the manuscript.

We did our best to take into account your suggestions and we hope that the new version has been improved in term of quality.

We hope this new version is sufficiently improved to be published in Remote Sensing.

To make it easier for you to review, please refer to the attachment where the changes are colored in blue.

Please find below our answers to your comments.

Kind regards,

Nicolas Karasiak & co-authors.

Review
-----------------------------------------------

The manuscript presents a piece of research of multi-year forest mapping based on one-year multiple dates remotely sensed image data. It might interest the forest management departments and remote sensing research. However, the follow aspects need to clarify during the revising period.

(1) I guess the feature sets for the classification are the four band reflectance, which should be addressed clearly in section 3. In addition, some derivatives from the optical images such as NDVI may be included into the classification feature sets (not just using the original spectral bands), to reflect the phenological difference among the different tree types.

(2) As for the data selection of different dates in the study, is it too simple just based on the availability? How about only select those image data with obvious phenological features ? If the disparity is caused mainly by the difference of image data sets in different years, I doubt the meaning of the comparison study.

Answers
----------
We would like to thank you for the time you spent to review the manuscript and the suggestions you proposed.

(1) We now specified that we used only spectral bands (the four Formosat-2 spectral bands at 8m spatial resolution) as predictors (L184).
Spectral index such as NDVI can reflect the phenological differences but their contribution to the model quality should not be quiet different, as recentely stated in the exhaustive Fassnacht et al's review about tree species mapping (RSE Volume 186, 1 December 2016, Pages 64-87): "Focusing on the application of vegetation or spectral indices, none of the reviewed studies demonstrated a clear advantage of their application. Generally, the application of the original reflectance or features derived from a feature extraction approach always led to at least similar but in most cases notably better results."

(2) The dates we used were all the dates available in the time series on the study site. We discusssed about the effect of the available dates in the SITS in section 5.4 (but see also section 5.3. as another factor). Now, we have introduced an additional analysis about the ranking based feature-selection of image dates (L427-457 + appendix E). Globally, the analysis confirmed the difficulty to draw robust conclusions about the tangible contribution of seasonal variations in species discrimination. The most important dates were highly variable from one year to another because of the irregular acquisition dates and clouds contamination. However, we found a clear benefit of using fewer images containing the maximum discrimination information about the tree species classes (ignoring noisy data and reducing the Hughes phenomenon). The conclusion has been updated accordingly (L497-504).

Author Response File: Author Response.pdf

Reviewer 3 Report

The authors should review the introduction to include a few more papers that are relevant to their work. Overall, the paper is well written and presented. 

Author Response

Dear reviewer,

We are very grateful of your review for your constructive comment on the first version of the manuscript.
We did our best to take into account your suggestion and we enhanced the introduction as specified.

As you recommended, we have added a new reference in the introduction (L56). In order to focus on high temporal and spatial resolution sensors, we have chosen not to list papers using hyperspectral sensors because references can be found in the Fassnacht's review. However if we have missed an important paper, please let us know.

To make it easier for you to review, please refer to the attachment where the changes are colored in blue.

We hope this new version is sufficiently improved to be published in Remote Sensing.

Kind regards,

Nicolas Karasiak & co-authors.

------------------------------------------

The authors should review the introduction to include a few more papers that are relevant to their work. Overall, the paper is well written and presented.

Author Response File: Author Response.pdf