Comparison of Methods for Modeling Fractional Cover Using Simulated Satellite Hyperspectral Imager Spectra

Round 1

Reviewer 1 Report

The paper "Comparison of Methods for Modeling Fractional  Cover using Simulated Satellite Hyperspectral  Imager Spectra"is an interesting research paper, But,  major issues are required to be modified and explained in order to give more credibility to this research work such as:

1-in Line 112 To 113 what is meant by oversampled to1 nm?  How this could be done from coarser spectral resolution?

2-when reading the article specifically the data set source it is hard to understand whether the spectral signatures and other related data are obtained from the same source. Please indicate clearly  whether these data are yours or the another dataset source .

3- Is using different ASD and any other  spectroradiometer with different foreoptic has any effect on the differences between synthetic hyperspectral data and between the different results of the experiments?

4-Authors indicated in lines 180 to181 that they have convolved the reflectance spectra from low resolution of 12 or lower to 10 nm how this is considered accurate for use in the experiments?

5-Authors are required to explain clearly the sentence in lines 219 and 221 "traditional...dataset" why there could be inflated accuracy due to spatial and temporal autocorrelation?  and what is the meaning of last statement?

6-The discussion in paragraph from line 268 to 272 does not support the objective of this research why an index wavelengths should match that of the image and not closely match to it (i.e. Red should be 600 nm and not 610nm as an example). In addition, why broadband should be used if the authors'  goal is to prove the hyperspectral simulation?

7-Paragraph from lines 297 to 303 and  including table 3, it is not clear how these indexes are used to create a narrow band soil index?  based on what rules?

8-Authors are describing the continuum removal and they have used two variables to indicate below and above line. the authors must prove the idea between lines 328 to 330 with references. 

9-Remove lines 330 to 331 or explain in details what is vegetation analysis!

10-MEMSA used by the authors is an old method and there are other up to date and better method such as A Spectral Unmixing Method by Maximum Margin Criterion and Derivative Weights to Address Spectral Variability in Hyperspectral Imagery, Remote Sens. 2019, 11(9), 1045; https://doi.org/10.3390/rs11091045

11-no need to add 8 references to prove MESMA use reduce the number of references with most up to date and reliable one 

12-The experiments show that the PLS   have low RMSE compared to  other methods this proves that the relationship between simulated and real hyperspectral data is a simple linear one. In other words there is only a duplication of the original spectral signatures data and really a real simulation To prove the success of the simulation use classification methods

12-Many references are outdated refine these references or use more recent one. In addition some references are not included such as 

Marcel Schwieder , Pedro J. Leitão, Stefan Suess, Cornelius Senf and Patrick Hostert,

Estimating Fractional Shrub Cover Using Simulated EnMAP Data: A Comparison of Three Machine Learning Regression Techniques,   Remote Sens. 2014, 6, 3427-3445; doi:10.3390/rs6043427

Author Response

We thank the reviewers for their helpful comments. We made several changes to the manuscript to address the concerns of all three reviewers. Important changes include:

Sampling by ASD instruments is better explained Actual wavelengths of bands used in each index are now listed in Tables 1-3, with better justification for the bands used for indices in the text Improved description of spectral feature analysis Testing of six additional indices, with results provided in supplemental materials

A point-by-point response follows.

1-in Line 112 To 113 what is meant by oversampled to1 nm?  How this could be done from coarser spectral resolution?

All ASD instruments oversample spectra to 1 nm using cubic spline interpolation before recording a spectrum. To clarify this, we added the following details to our description:

“All reflectance spectra in the field experiments described below were measured using Analytical Spectral Devices (ASD) field spectrometers. The native sampling intervals of ASD spectrometers range from approximately 1.4 nm for the visible/near-infrared detector (VNIR; 350 to 1,000 nm) to approximately 2.2 nm for the SWIR detectors (1,001 to 2,500 nm). ASD instruments resample the native channels using cubic spline interpolation before recording 2,151 channels at standardized wavelengths (350 to 2,500 nm) at a 1 nm interval (ASD Inc., 1999). The spectral bandpass of ASD spectrometers vary across the wavelength range of each detector and differs between spectrometer models, ranging from approximately 3 to 12 nm (Kokaly et al., 2017).”

2-when reading the article specifically the data set source it is hard to understand whether the spectral signatures and other related data are obtained from the same source. Please indicate clearly whether these data are yours or the another dataset source .

The data were contributed from six prior field experiments by the authors of the manuscript. To clarify that the data come from previous experiments, we changed the data descriptions in the abstract and methods:

“We used six field spectroscopy datasets collected in prior experiments from sites with partial crop, grass, shrub, and low-stature resprouting tree cover to simulate satellite hyperspectral data, including sensor noise and atmospheric correction artifacts.”

“Field spectra collected in six separate prior experiments were used to simulate spectra measured by a VSWIR satellite imaging spectrometer.”

3- Is using different ASD and any other  spectroradiometer with different foreoptic has any effect on the differences between synthetic hyperspectral data and between the different results of the experiments?

The field of view was different for each experiment due to the use of different instruments and the presence or absence of a foreoptic. We have provided a full description of the instrument and field of view for each experiment, but once convolved to 10 nm spectral resolution, we did not see any systematic differences in our spectra collected during different experiments. The field of view was fairly similar across experiments, with a range between 18 and 25 degrees.

4-Authors indicated in lines 180 to181 that they have convolved the reflectance spectra from low resolution of 12 or lower to 10 nm how this is considered accurate for use in the experiments?

This is a good question with a very complicated answer. The true spectral resolution (FWHM) of each field spectrometer depends on detector (VNIR, SWIR1, or SWIR2), wavelength, and the generation of each device. All datasets have a spectral resolution that is claimed by the manufacturer to be 10 nm or better. One of us (Kokaly) has done an assessment of the SWIR bandpass of several ASD models using noble gas emission lamps, and has determined that the SWIR FWHM of ASD FR spectrometers is actually approximately 12 nm. Rather than provide the manufacturer’s specifications, we have put the range of bandpass values in our revised text and provided a supporting reference:

“The spectral bandpass of ASD spectrometers vary across the wavelength range of each detector and differs between spectrometer models, ranging from approximately 3 to 12 nm (Kokaly et al., 2017).”

Acknowledging the difference between the manufacturer’s specifications and what one of us has empirically found, the question then becomes whether the difference between 10 nm and 12 nm spectral resolution makes a difference for our analysis. Given that chlorophyll, liquid water, lignin, and cellulose absorption features are much wider than 10 nm, we do not believe the difference in spectral resolution affected our results. If mineral absorption features were present in our high soil cover spectra, the coarser SWIR spectral resolution would have more plausibly affected our analysis, but we did not find any such spectra in our dataset.

5-Authors are required to explain clearly the sentence in lines 219 and 221 "traditional...dataset" why there could be inflated accuracy due to spatial and temporal autocorrelation?  and what is the meaning of last statement?

We changed this sentence to clarify our meaning:

“Random assignment of data into training and validation subsets could inflate accuracy compared to real-world application, since spectra from each prior experiment are not independent observations, but rather are correlated in space and in time.”

6-The discussion in paragraph from line 268 to 272 does not support the objective of this research why an index wavelengths should match that of the image and not closely match to it (i.e. Red should be 600 nm and not 610nm as an example). In addition, why broadband should be used if the authors'  goal is to prove the hyperspectral simulation?

Based on a point made by Reviewer 2, we changed our red band used in NDVI to 670 nm, coinciding with a reflectance minimum. For EVI, we kept MODIS band centers, since EVI is commonly applied to MODIS data. To be clear, convolution to broad multispectral bands was never used in the manuscript, but was mentioned in passing in case this was of interest to readers. We have deleted these sentences in our revisions to avoid confusion.

7-Paragraph from lines 297 to 303 and  including table 3, it is not clear how these indexes are used to create a narrow band soil index?  based on what rules?

Since fractional cover sums to one, the estimates for GV cover and NPV cover can be combined and subtracted from 1 to estimate soil cover. We have edited the text in this section to explain this concept better:

“Instead, the indices with the lowest RMSE for predicting validation library GV and NPV fractional cover were used to create a narrowband index soil approximation (Table 3). This metric was calculated as 1- (GV_NDVI + NPV_CAI), where GV_NDVI is GV cover estimated using the best-fit polynomial function for NDVI and NPV_CAI is NPV cover estimated using the best-fit linear function for CAI.”

8-Authors are describing the continuum removal and they have used two variables to indicate below and above line. the authors must prove the idea between lines 328 to 330 with references. 

We have revised our text and updated our supporting references. The section describing this technique has been changed to:

“Spectral feature analysis (SFA) with continuum removal was applied to selected absorption features for estimating GV and NPV fractional cover, including the 670 nm chlorophyll absorption feature to estimate GV cover (Table 1) and the 2100 nm lignocellulose feature to estimate NPV cover (Table 2). A second lignocellulose absorption feature at 2300 nm has been previously linked to plant biochemical composition (Kokaly and Clark, 1999; Kokaly et al., 2009), but the 2100 nm feature was selected for use with SFA due to higher signal-to-noise in the simulated VSWIR spectra. Continuum removal is a method used to isolate and remove the influence of the other absorptions present in the spectrum, not including the absorption feature of interest (Clark and Roush, 1984). In linear continuum removal, the continuum is defined by the line connecting bands at points on the left and right sides of an absorption feature. Area between the spectrum and the continuum line, both above (area_above) and below (area_below) the line, can be calculated, using the continuum line to normalize the reflectance values (Kokaly and Skidmore, 2015). Parameters such as feature depth and area have been correlated with the amount or concentration of an absorbing chemical (Kokaly and Clark, 1999; Kokaly and Skidmore, 2015).

Since the spectra analyzed in this study included the full range of GV and NPV cover, including spectra containing no GV or NPV cover, some spectra did not have absorption features at 670 and 2100 nm and reflectance could exceed the continuum line. To account for area both above and below the continuum line, net area was calculated as area_below – area_above. This net area has higher positive values when an absorber in the continuum range is present, and allows for negative values when an absorber is not present and spectral shape leads to a net area above the continuum line.

The two net area metrics, area_670nm and area_2100nm, were calculated using USGS PRISM (Processing Routines in IDL for Spectroscopic Measurements) software (Kokaly, 2011). Both net area metrics were calculated for the training spectral library, and then regressed against GV cover and NPV cover respectively, using a second degree polynomial model for GV and a linear model for NPV. The resulting regression coefficients were then applied to the validation spectral library to predict both GV and NPV fractional cover. Similar to spectral indices, SFA was unable to directly estimate soil cover due to the lack of a characteristic absorption feature. To approximate soil fractional cover for the validation library using SFA, the sum of SFA-estimated GV cover (SFA_GV) and NPV cover (SFA­_NPV) was subtracted from one (Table 3).”

9-Remove lines 330 to 331 or explain in details what is vegetation analysis!

Vegetation analysis has been removed from the sentence. The new sentence is:

“The two net area metrics, area_670nm and area_2100nm, were calculated using USGS PRISM (Processing Routines in IDL for Spectroscopic Measurements) software.”

10-MEMSA used by the authors is an old method and there are other up to date and better method such as A Spectral Unmixing Method by Maximum Margin Criterion and Derivative Weights to Address Spectral Variability in Hyperspectral Imagery, Remote Sens. 2019, 11(9), 1045

Since it was not possible to compare all available methods (which number in the many dozens, if not exceeding 100), we chose methods commonly used in the literature. MESMA is very commonly used for fractional modeling, even in the recent literature. We have provided links to the spectral libraries used in our manuscript in the hopes that others will demonstrate methodological improvements using their own preferred algorithms.

11-no need to add 8 references to prove MESMA use reduce the number of references with most up to date and reliable one 

We deleted all but one reference, leaving the most recent comprehensive source.

12-The experiments show that the PLS have low RMSE compared to  other methods this proves that the relationship between simulated and real hyperspectral data is a simple linear one. In other words there is only a duplication of the original spectral signatures data and really a real simulation To prove the success of the simulation use classification methods

We apologize, but we do not understand this criticism. We are not sure what you mean by the relationship between simulated and real hyperspectral data being linear, since only used simulated spectra in the manuscript. The spectra in the validation library do have unique features, such as soil type and time-of-season, that were not included in the training library. However, if the reviewer means that the spectral signatures in the validation dataset (e.g. chlorophyll and lignocellulose absorption features) should be the same as the spectral signatures in the training dataset, we agree with that assessment. The suggestion of using a classification method does not seem to fit fractional cover modeling, since fractional cover is a continuous variable.

12-Many references are outdated refine these references or use more recent one. In addition some references are not included such as… 

Thank you for pointing out Schwieder et al. 2014. We have added it to the manuscript. We have used the original references to the field experiments, techniques, and findings appropriate for our analysis. Since we chose commonly used methods, the original references for those methods can reach back many years. Overall, it’s hard to address such a vague criticism, but we did add a few more recent references. If the reviewer can be more specific as to which references he or she feels are inappropriate, we are happy to address those.

Reviewer 2 Report

The manuscript presents the results of an analysis of methods for calculating fractional vegetation cover, with both green and non-photosynthesizing vegetation included, using hyperspectral data.  Inclusion of some attention to NPV is a strength of this paper, since such analyses are often overlooked.  The analysis purports to use spectra from a variety of cover types at six different locations to test index and spectroscopic approaches for determining vegetation fractions.  The spectra used in the analysis are modified to match the probably parameters of future hyperspectral orbital missions, hence the term 'simulated.'

 

The questions addressed, methods used, and results obtained from this study are all relatively straightforward, and I think the paper is suitable for publication.  I have  a few minor suggestions, mostly arriving from some confusing methods and minor inconsistencies between the narrative and the tables.  

Line 297 -- can you elaborate a bit on the single diagnostic absorption feature that vegetation has (and soil doesn't)?  Is it the 680nm feature mentioned a few lines later?  If so, then this is a chlorophyll feature, so is it still applicable to NPV? Line 308 -- The 680 nm feature is mentioned here and Table 1 is referenced, but I can see no index in the table that uses reflectance at this exact wavelength.  R at 645 is used.  Is this a typo, or a different index?  It's a small point, but in my own experience, the features between the green peal and red trough can be quite narrow, and a even a small ~35nm discrepency could make a difference. Similar to question #2, the 2100 and 2300nm bands referenced in the text (lines 309-311) don't quite match values in Table 2.   In table 4, and line 414 (and after) some analysis is done by averaging the RMSE from each of the four empirical modeling methods.  This seems like an awkward may of summarizing the results, especially since it is unlikely that the results of several spectroscopic methods would be averaged in an actual analysis (it seems more likely that the best one would be used).  It might make more sense to just discuss each of the RMSE results (for each method) in turn, the draw an overall conclusion about them?

 

Author Response

We thank the reviewers for their helpful comments. We made several changes to the manuscript to address the concerns of all three reviewers. Important changes include:

Sampling by ASD instruments is better explained Actual wavelengths of bands used in each index are now listed in Tables 1-3, with better justification for the bands used for indices in the text Improved description of spectral feature analysis Testing of six additional indices, with results provided in supplemental materials

A point-by-point response follows.

Line 297 -- can you elaborate a bit on the single diagnostic absorption feature that vegetation has (and soil doesn't)?  Is it the 680nm feature mentioned a few lines later?  If so, then this is a chlorophyll feature, so is it still applicable to NPV?

We changed this sentence to improve clarity. It now reads:

“Soil does not have characteristic absorption features like GV (chlorophyll) or NPV (lignin and cellulose).”

Line 308 -- The 680 nm feature is mentioned here and Table 1 is referenced, but I can see no index in the table that uses reflectance at this exact wavelength.  R at 645 is used.  Is this a typo, or a different index?  It's a small point, but in my own experience, the features between the green peal and red trough can be quite narrow, and a even a small ~35nm discrepency could make a difference.

Thank you for pointing this out. We had used MODIS band centers, which we failed to mention in our original manuscript. We recalculated NDVI using the 670 nm reflectance minimum. This produced some small changes in results, and we updated our description to reflect these changes. We kept 860 nm as the NIR band, but simplified the notation in the Table 2 from 857 nm (a MODIS band center) to simply 860 nm. For EVI, we kept MODIS band centers to reflect the most common use of that index. We also calculated a large number of additional indices as the request of Reviewer 3, all using the 670 nm reflectance minimum as the red band. We have provided these results in the supplemental material.

Similar to question #2, the 2100 and 2300nm bands referenced in the text (lines 309-311) don't quite match values in Table 2.  

The lignocellulose absorption feature is centered at 2100 nm, but as we mention earlier in this section, we use bands optimized by Serbin et al. to calculate LCA:

“CAI was developed to contrast lignocellulose absorption at 2100 nm with two reference bands at 2000 and 2200 nm (Daughtry, 2001; Nagler et al., 2003). We used wavelengths further refined by Serbin et al. (2009) to take advantage of 10 nm bandwidth and avoid the effects of carbon dioxide absorption closer to 2000 nm (Table 2).”

In table 4, and line 414 (and after) some analysis is done by averaging the RMSE from each of the four empirical modeling methods.  This seems like an awkward may of summarizing the results, especially since it is unlikely that the results of several spectroscopic methods would be averaged in an actual analysis (it seems more likely that the best one would be used).  It might make more sense to just discuss each of the RMSE results (for each method) in turn, the draw an overall conclusion about them?

Based on the reviewer’s suggestion, we removed instances of averaging across methods from the manuscript.

Reviewer 3 Report

The present study demonstrated the incorporation of spectral indices for GV, NPV and SFC. This is a interesting points to raise the difficulties in obtaining accurate results related to GV, NPV, and Soil fractional cover. However, I am concerned about the outcome provided by authors.  There are several studies demonstrating the improved results for the above three parameters retrieval.  

Minor changes required-  to include below indices for better results and performance 

I will suggest authors to include other indices before concluding that hyperspectral images are still unable to perform better discrimination. 

1. Use SAVI, OSAVI (Optimised soil vegetation index),  MSAVI, ATSAVI (Adjusted transformed soil index), MSRI (Misra Soil brightness index). 

2. There are also more studies about the soil, vegetation and greenness index, incorporate TVi (transformed Vegetation index), TVI (triangular vegetation index), TGI (triangular greeness index). 

 

3. Use of selective bands are also nowadays widely employed in the studies. Any attempt to include in the present study?? 

 

I recommend incorporating above indices for better outcome and comparison. 

 

 

Author Response

We thank the reviewers for their helpful comments. We made several changes to the manuscript to address the concerns of all three reviewers. Important changes include:

Sampling by ASD instruments is better explained Actual wavelengths of bands used in each index are now listed in Tables 1-3, with better justification for the bands used for indices in the text Improved description of spectral feature analysis Testing of six additional indices, with results provided in supplemental materials

A point-by-point response follows.

 

Use SAVI, OSAVI (Optimised soil vegetation index),  MSAVI, ATSAVI (Adjusted transformed soil index), MSRI (Misra Soil brightness index). 

This point is addressed with (2) below.

There are also more studies about the soil, vegetation and greenness index, incorporate TVi (transformed Vegetation index), TVI (triangular vegetation index), TGI (triangular greeness index). 

We had originally tried a larger suite of vegetation indices, but narrowed it down to four common indices that we thought were good examples of the range of performance from vegetation indices. All other vegetation indices we have tried performed worse than NDVI. For our revised analysis, we examined SAVI, MSAVI, OSAVI, ATSAVI, TVI, and TGI. All performed more poorly than NDVI. We also looked at TVi, but its results were essentially identical to NDVI since it merely corrects for negative values. We did not try MSRI, since it is a PC transform with band weightings intended to be used with Landsat MSS data. We have included an expanded table with results for SAVI, MSAVI, OSAVI, ATSAVI, TVI, and TGI in the supplemental material.

Use of selective bands are also nowadays widely employed in the studies. Any attempt to include in the present study?? 

Good point. We did not employ band selection in this study, but we added a sentence and references to the discussion to make it clear that this is a good avenue for future comparison:

“Techniques for optimal band selection could also lead to improved results from MESMA (Somers et al., 2010; Somers & Asner, 2013; Tane et al., 2018).”

Round 2

Reviewer 1 Report

Authors have responded to the reviewers' comments in details except few minor and necessary issues which must be corrected or answered to make the research paper better scientifically.

1- Reduce the number of literature by removing old and redundant references. This high number of literature are more convenient for literature review paper.

 

2-  same  numbers are repeated for different datasets i.e. see lines 128 and 152

3-Some of the compared datasets have unsuccessful result in  separating NPV and soil cover such as [19]. Could it be  using different ASD technologies  and different forepotic angles?

In that case should the authors exclude these datasets?

 

Author Response

1- Reduce the number of literature by removing old and redundant references. This high number of literature are more convenient for literature review paper.

We have removed references from approximately fifteen locations in the revised submission, resulting in a net reduction of six references. We note that this manuscript requires a larger number of references than the average research paper, since it uses six prior datasets and nine unique metrics.

2-  same  numbers are repeated for different datasets i.e. see lines 128 and 152

These sentences are correct. The first instance of “second” refers to the second agricultural dataset. The second instance of “second” refers to the second dataset containing only GV and NPV. In both cases, we are drawing a link to the dataset described in the previous paragraph.

3-Some of the compared datasets have unsuccessful result in  separating NPV and soil cover such as [19]. Could it be  using different ASD technologies  and different forepotic angles? In that case should the authors exclude these datasets?

The subset of spectra that we used from the Numata et al. dataset (reference 19) only included NPV. The issue in not being able to separate NPV and soil cover for this dataset was due to the reference photographs, not due to an issue with the spectra. Numata et al. could not separate NPV from soil using their photo classification method, and we used their reference data for this study. The Coal Oil Point and Roth spectra had the same issue.

The lignocellulose absorption features primarily used for NPV-soil separation are quite broad, and thus differences in ASD model or field of view should not have a major impact on NPV-soil separation.