Validation and Error Minimization of Global Ecosystem Dynamics Investigation (GEDI) Relative Height Metrics in the Amazon

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Summary

East et al. present a study focused on evaluating and minimizing error in GEDI Level 2A relative height metrics in Amazonian forests. With a particular emphasis on performance of understory (a.k.a. “lower canopy”) heights, they compare ALS-driven simulated waveforms to real waveforms, performing a series of filters and data augmentations to attempt to reduce disagreement between the two. The paper is well-presented, and while others have conducted similar studies, the focus on understory structure in a novel ecosystem justifies its scientific novelty. I have included a few major and minor comments below that, if addressed, should make the paper worthy of publication in Remote Sensing.

 

Major comments

-          In the Abstract, it needs to be made more clear what parts of the analysis rely on simulated waveforms and what parts of the analysis rely on real GEDI waveforms. After several reads, I’m still not clear. In L20-21 you say that you compare simulated waveforms to ALS data – this makes sense to me. But then you talk about sensor conditions and geolocation correction, both of which would imply real GEDI data. After having read further, I now understand the roles of simulated and real GEDI data, but the Abstract should stand alone and clearly present a summary of the study’s methods, which in its present form, it does not.

-          Throughout this paper, I think it is important to be a little more precise with the language surrounding what types of accuracy/error/uncertainty you’re evaluating. For one example, the Abstract (L18) says “to understand GEDI error”. For another, L151 says “To assess the accuracy of GEDI data”. These could be referring to geolocation error, waveform intensity error, relative height error, canopy cover error, etc. etc.

-          The Introduction would benefit from a greater emphasis on the value of having understory relative height measurements in discrete footprints. In other words, why do we care if GEDI can accurately measure understory structure? You spend an appropriate amount of time discussing GEDI data as a whole, but less on what GEDI can actually tell us about the understory.

-          If the bias in these RH metrics is so persistently negative, then perhaps it’s worth at least discussing (and perhaps testing) a bias correction approach?

 

Minor comments

L11 & L16: Is it fair to say that GEDI is “emergent” and that “…uses of GEDI data are beginning to appear” when it was launched over 6 years ago? These are early days for GEDI, I suppose, but this language makes it sound like it was launched in the last year or two. Your reference in L68 to 10k+ refs of GEDI data sort of stands in contrast to this language as well.

L25: I think there should be equals signs after R2 and RMSE.

L41: “…structure *within* ~25m diameter…”

L42: Minor semantic point, but I would describe 51.6 S to 51.6 N as being “over half” rather than “near global”. Might be more precise to say something about nearly covering all vegetated land on Earth, or something.

L44-45: If you’re going to mention the hibernation years, you might also mention the launch year, or when data first became available.

L86: The use of “adjacent” is a bit odd here – implies proximity to (but not overlapping with) GEDI data. Perhaps “spatially concurrent”?

L93-95: This sentence would be stronger if bolstered by references – calling the Amazon Basin “one of the most challenging ecosystems”, while likely true, suggests some a priori evidence of this difficulty.

L142: Revise to “…were excluded (Figure 1C).”

Figure 1: Suggested improvements: (1) scale bars for your three maps to give us a sense of how far apart/close together these ALS/GEDI data are; (2) it’s not clear what the point of the zoomed in area in Figure 1A is as no references are made in the text/caption; (3) some ALS areas in Figure 1C do not have GEDI footprints? You clarify this in L159, but were these ALS data still used in some way? If not, perhaps remove from the map?

L168: This number of footprints (2693) does not agree with the number from the previous paragraph. Can you clarify?

L220: I’m not clear on what you mean by “points with less than 2/3 ALS cover”. Does this mean a GEDI footprint that straddled the edge of an ALS scan and it had to be at least 2/3 within the scan area? If so, it would seem more appropriate to ensure that the entirety of the footprint area (plus a ~10m buffer, probably, for geolocation uncertainty) should fall within the ALS scan.

L221: Densities less than 3 *pts/sqm*, I assume?

L238: Can you provide more justification for and explanation of this seemingly atypical R-squared measure?

L248: Eq. needs a number.

L250: The bar should be on top of y.

L265-271: Figure caption should follow figure. Applies to all subsequent figures with the same issue as well.

Figure 3: Perhaps it’s just me, but this figure does more to confuse me than to help me understand the process.

L293: Points per *square* meter.

L295-297: Feels like it belongs in Results.

Figure 4: Some points appear to be cut off – are the full x and y axes being shown or are these zoomed in somewhat?

L310: Revise to “data are” here and elsewhere.

Figures 5 and 6: Shouldn’t the trend in Figure 5 mirror the first subplot (bias) in Figure 6? The general trend is similar, but in Figure 5, the RH100 bias is strongly negative, whereas in Figure 6, it looks like the lower RHs (0, 1, 2, etc.) have the most negative bias.

L352-360: Important to note that all of these statements assume that you’re ignoring RH100.

L363: This is yet a different number of footprints (2420) than previously reported (2693 in L168 and 2033 in L162).

L461-462: This is a vague statement (“previous applications…different questions”).

L468-469: Please specify *hemlock* woolly adelgid.

 

Comments on the Quality of English Language

English is fine -- I've suggested a handful of grammatical revisions.

Author Response

Comment 1 -          In the Abstract, it needs to be made more clear what parts of the analysis rely on simulated waveforms and what parts of the analysis rely on real GEDI waveforms. After several reads, I’m still not clear. In L20-21 you say that you compare simulated waveforms to ALS data – this makes sense to me. But then you talk about sensor conditions and geolocation correction, both of which would imply real GEDI data. After having read further, I now understand the roles of simulated and real GEDI data, but the Abstract should stand alone and clearly present a summary of the study’s methods, which in its present form, it does not.

Response 1: That's a good point. We have updated the abstract to clarify the role of real on-orbit GEDI data and ALS simulated GEDI data.

Comment 2 -          Throughout this paper, I think it is important to be a little more precise with the language surrounding what types of accuracy/error/uncertainty you’re evaluating. For one example, the Abstract (L18) says “to understand GEDI error”. For another, L151 says “To assess the accuracy of GEDI data”. These could be referring to geolocation error, waveform intensity error, relative height error, canopy cover error, etc. etc.

Response 2: We have updated language in the abstract specifically to reflect the terms used in the introduction and methods, including highlighting specific terms for error metrics introduced in the methods section and references throughout the paper.

Comment 3 -          The Introduction would benefit from a greater emphasis on the value of having understory relative height measurements in discrete footprints. In other words, why do we care if GEDI can accurately measure understory structure? You spend an appropriate amount of time discussing GEDI data as a whole, but less on what GEDI can actually tell us about the understory.

Response 3: We address the importance of understory to previous GEDI research in the second paragraph. We have reframed and added to this paragraph to draw more attention to the importance of understory data, and the gap in knowledge surrounding it (L54-59)

Comment 4 -          If the bias in these RH metrics is so persistently negative, then perhaps it’s worth at least discussing (and perhaps testing) a bias correction approach?

Response 4: Thank you for this comment. We tried to do this in previous versions of this paper with little success. We had previously reported those methods and results, but given the length and complexity of this paper, we ultimately had to drop those efforts for clarity and length in a previous round of review. We completely agree that it is an important area of work to follow up on in subsequent research. 

Minor comments

Comment 5 L11 & L16: Is it fair to say that GEDI is “emergent” and that “…uses of GEDI data are beginning to appear” when it was launched over 6 years ago? These are early days for GEDI, I suppose, but this language makes it sound like it was launched in the last year or two. Your reference in L68 to 10k+ refs of GEDI data sort of stands in contrast to this language as well.

Response 5: Thank you for highlighting this. We have updated the language to reflect the current state of the field. 

Comment 6 L25: I think there should be equals signs after R2 and RMSE.

Response 6: Corrected.

Comment 7 L41: “…structure *within* ~25m diameter…”

Response 7: Corrected at to within, thank you.

Comment 8 L42: Minor semantic point, but I would describe 51.6 S to 51.6 N as being “over half” rather than “near global”. Might be more precise to say something about nearly covering all vegetated land on Earth, or something.

Response 8: We have added the caveat “of terrestrial vegetation” to this sentence. 

Comment 9 L44-45: If you’re going to mention the hibernation years, you might also mention the launch year, or when data first became available.

Response 9: The previous sentence mentions the launch in March 2019. 

Comment 10 L86: The use of “adjacent” is a bit odd here – implies proximity to (but not overlapping with) GEDI data. Perhaps “spatially concurrent”?

Response 10: Thank you for pointing out that this is confusing. Here we are discussing geolocation correction, so we do in fact need the adjacent ALS data as well as the spatially concurrent data to simulate alternate waveforms to simulate geolocation correction. The text has been updated to read “from spatially concurrent and adjacent ALS data” to speak to this dynamic more directly without going into specific methods level detail. 

Comment 11 L93-95: This sentence would be stronger if bolstered by references – calling the Amazon Basin “one of the most challenging ecosystems”, while likely true, suggests some a priori evidence of this difficulty.

Response 11: We have updated this statement to read “likely one of the most challenging ecosystems for GEDI applications due to the confluence of canopy height, density, and persistent cloud cover” to remove the implications of previous evidence and highlight the region's physical challenges. 

Comment 12 L142: Revise to “…were excluded (Figure 1C).”

Response 12: Corrected.

Comment 13 Figure 1: Suggested improvements: (1) scale bars for your three maps to give us a sense of how far apart/close together these ALS/GEDI data are; (2) it’s not clear what the point of the zoomed in area in Figure 1A is as no references are made in the text/caption; (3) some ALS areas in Figure 1C do not have GEDI footprints? You clarify this in L159, but were these ALS data still used in some way? If not, perhaps remove from the map?

Response 13: Updates made.

Comment 14 L168: This number of footprints (2693) does not agree with the number from the previous paragraph. Can you clarify?

Response 14: Thank you for noting the discrepancies throughout this manuscript in sample size. Given the effect of different data quality filtering and geolocation correction on sample size, these numbers can be challenging to report appropriately in different sections. To address this we have updated the first mention to percentages land cover type, added a note about quality filtering to the second mention, and checked for accuracy of all other sample sizes mentioned throughout. 

Comment 15 L220: I’m not clear on what you mean by “points with less than 2/3 ALS cover”. Does this mean a GEDI footprint that straddled the edge of an ALS scan and it had to be at least 2/3 within the scan area? If so, it would seem more appropriate to ensure that the entirety of the footprint area (plus a ~10m buffer, probably, for geolocation uncertainty) should fall within the ALS scan.

Response 15: This is an additional quality flag input option for the simulator algorithm. It omits footprints where coverage is low in assessing geolocation corrections. The problem with just relying on footprint overlap within some buffer is that with geolocation simulations, many of the hundreds of footprint shifts that it simulates for are larger than 10m, and due to the randomness of the shifts, some end up being quite large. This flag is an extra check to eliminate the effects of those larger shifts moving individual points outside of the ALS boundary from the overall geolocation assessment. Due to the fact that this is an algorithmic quality control flag (active or inactive), and not a parameter that we can modify, we document that we utilize the flag in the simulations but not a broader discussion of its utility to the algorithm. We have tried to make it more clear that this is an algorithm parameter in the text (L221).

Comment 16 L221: Densities less than 3 *pts/sqm*, I assume?

Response 16: Yes, updated in text

Comment 17 L238: Can you provide more justification for and explanation of this seemingly atypical R-squared measure?

Response 17: Yes, this is because we do not want to measure the R-squared for a best fit line with a variable slope when a 1:1 relationship is what we are expecting. The text now reads "such that R2 is measured for the expected 1:1 relationship as opposed to a line of best fit with a variable slope; thus we assume a slope of 1 and intercepting the origin”

Comment 18 L248: Eq. needs a number.

Response 18: Corrected

Comment 19 L250: The bar should be on top of y.

Response 19: Corrected

Comment 20 L265-271: Figure caption should follow figure. Applies to all subsequent figures with the same issue as well.

Response 20: Corrected, thank you for catching this. 

Comment 21 Figure 3: Perhaps it’s just me, but this figure does more to confuse me than to help me understand the process.

Response 21: Thanks for this comment. This figure was developed to satisfy previous peer review comments. We are open to modification of the figure for clarity but are hesitant to omit the workflow diagram as some previous reviewers deemed it helpful/necessary. 

Comment 22 L293: Points per *square* meter.

Response 22: Yes, thank you. Corrected in text. 

Comment 23 L295-297: Feels like it belongs in Results.

Response 23: Great point. I have moved it to the start of section 3.3 Geolocation Correction.

Comment 24 Figure 4: Some points appear to be cut off – are the full x and y axes being shown or are these zoomed in somewhat?

Response 24:The full axis is being shown with no expansion. The points that appear cut off sit on or near the lower bounds of the axis and, thus, due to pixel size, appear to be cut in half. 

Comment 25: L310: Revise to “data are” here and elsewhere.

Response 25: Corrected, thank you

Comment 26 Figures 5 and 6: Shouldn’t the trend in Figure 5 mirror the first subplot (bias) in Figure 6? The general trend is similar, but in Figure 5, the RH100 bias is strongly negative, whereas in Figure 6, it looks like the lower RHs (0, 1, 2, etc.) have the most negative bias.

Response 26: This comment has been immensely helpful, thank you.. You are entirely corect that the Bias plot in figure 6 is incorrectly showing bias at RH100, this is due to an error on our part in the y-axis bounds. This has been updated in the revised manuscript, and the bias results in figure 6 mirror thos in figure 7 where the scale was appropriate. In regards to the lower RH (0-5), the results in Figure 5 lose the trend in the lower RH values in Figures 6 and 7 due to binning (combining RH0-5 into 1 boxplot). This influence is largely due to the influence of the less negative bias values of RH0 (Which was not previously shown in Figures 6 and 7 but is now included as well). 

Comment 27 L352-360: Important to note that all of these statements assume that you’re ignoring RH100.

Response 27: Thank you for pointing this out, we now reiterate this in this section.

Comment 28: L363: This is yet a different number of footprints (2420) than previously reported (2693 in L168 and 2033 in L162).

Response 28: Updates have been made throughout.

Comment 29: L461-462: This is a vague statement (“previous applications…different questions”).

Response 29: Added details to L462

Comment 30: L468-469: Please specify *hemlock* woolly adelgid.

Response 30: Corrected, Thank you

Reviewer 2 Report

Comments and Suggestions for Authors

Dear authors,

thank you for your fascinating article.

Research of validation and error minimization of GEDI relative height metrics is very relevant for sustainable forestry development on a global scale.

Most of the questions for this study arise from the lack of comparison with ground-based, actual data from permanent sample plots, as well as the lack of analysis of the influence of the growth rate of young forest stands on the accuracy of height measurement based on remote sensing data from different period.

You can find comments and remarks below.

L. 32 – Authors should change the keywords “GEDI” and “Validation”. All keywords should not be similar to the words of the article title. Also, authors can add a few more keywords.

L. 350 – Table 3 must be preceded by a reference to Table 3.

L. 380 – Figure 7 must be preceded by a reference to Figure 7.

L.25-26 – the equals sign is missed

L. 174 - Reference to Figure or Table is part of the sentence  - ....(Figure 2).

L. 157 – If ALS data were collected within 1-2 years of GEDI Footprint acquisition, does it mean that the database has parameters of stands on plots that were changing their height during this period? Tropic forests are growing so fast that authors could use incorrect data collected over 2 years.

L. 257 – Did you compare it with field data from permanent plots?

L. 265-271 – How workflow of validation and error minimization did connect with field data? Did you use some calibration?

Comments on the Quality of English Language

Minor editing of English language required.

Author Response

Comment 1: L. 32 – Authors should change the keywords “GEDI” and “Validation”. All keywords should not be similar to the words of the article title. Also, authors can add a few more keywords.

Response 1: 

Comment 2: L. 350 – Table 3 must be preceded by a reference to Table 3.

Response 2: Added to L341, thank you for catching this. 

Comment 3: L. 380 – Figure 7 must be preceded by a reference to Figure 7.

Response 3: Added to L371, thank you for catching this. 

Comment 4: L.25-26 – the equals sign is missed

Resonance 4: Corrected, thank you.

Comment 5: L. 174 - Reference to Figure or Table is part of the sentence  - ....(Figure 2).

Response 5: Corrected, thank you.

Comment 6: L. 157 – If ALS data were collected within 1-2 years of GEDI Footprint acquisition, does it mean that the database has parameters of stands on plots that were changing their height during this period? Tropic forests are growing so fast that authors could use incorrect data collected over 2 years.

Response 6: This is a challenge in GEDI validation, and we have elected to use the smallest temporal window possible to ensure high-quality comparisons. We also extensively filter for temporal change to try to mitigate this effect, but it is a limitation of the current data availability.

Comment 7: L. 257 – Did you compare it with field data from permanent plots?

Response 7: No, permanent plot data was not available to us. 

Comment 8: L. 265-271 – How workflow of validation and error minimization did connect with field data? Did you use some calibration?

Response 8: added "of on-orbit GEDI data to fine-scale ALS"

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have addressed my concerns in the revised manuscript.