How High to Fly? Mapping Evapotranspiration from Remotely Piloted Aircrafts at Different Elevations

Round 1

Reviewer 1 Report

Spatially explicit irrigation control is the holy grail of agricultural and horticultural remote sensing Stronger evidence to demonstrates ease and accuracy of fast and inexpensive spatial methods is highly needed. However, this paper may not serve this overarching aim as it makes promises that are not substantiated. Barely any of the conclusions are underpinned with clear evidence from this analysis. The analysis can therefore not be interpreted as a study in support of the presented approach, despite excellent data collection and processing. The paper cannot be published with it’s current aims and conclusions.

 

Below, I give more detail about the discrepancy between results and conclusions (original conclusions in italics).

 

Drone based remote sensing offers the ability to rapidly generate ET maps within a season that can be used to make in-season decisions. The use of these maps with precision irrigation has the potential to reduce water use and nitrogen loss from agricultural fields. The High Resolution Mapping of Evapotranspiration (HRMET) model was found to be a promising model for further development in potatoes with drone imagery collected under optimal conditions.

This is a superficial claim for which the paper does not provide clear evidence. The accuracy of the maps cannot be assessed quantitatively as there is no spatially explicit reference data. An overestimate between approx. 50 and 150% in 5 out of 8 carefully selected mission dates cannot give land managers confidence in the method. The management relevance is grossly overrated.

 

We evaluated whether there was an increase in insightful information form ET maps created using imagery collected at 90, 60, or 30 m. Though there were some additional detectable features low elevation flights, the tradeoff in resources and computation does not justify these low elevation flights for annual vegetable crops in the Mid-west USA.

The data shows lower variability (higher min, lower max) in the low resolution data. This may be entirely due to smoothing. However, this is a rather trivial observation and there is no information that may elucidate this further, i.e. show to what degree the whole may be more than the sum of it’s parts (i.e. the variability can be derived by simple averaging).

 

It is surprising to see the discrepancy between pattern at different scales, excluding the central access road. Mean and median are very similar, i.e there seems to be no bias effect of scale. This is an important observation. However, it may invalidate the approach as this suggests a very high confidence in the erroneous means. It also implies that there are “additional detectable features” at all scales. This needs to be given a stronger consideration as this may show that there are potential artefacts affecting the results. Pattern may - or may not - be related to ET. Spatio-temporal differences (i.e. incoming solar radiation, wind, humidity, BRDF) during the flight time may generate differences in ET estimates, without any possibility to judge if this a true effect of plant stomatal control or an error in the approach. On the other hand, if these patterns represent actual differences in stomatal control and hence ET, in other words, if the method can detect high spatio-temporal differences in EC-ET, the problem is worse. In this case, the results would imply that the pattern reflect instantaneous estimates of highly variable factors and should probably not be interpreted as representative for longer-term (hourly, daily, weekly) water loss. Later is crucial for management relevance of the drone data.  In the latter case, simple monitoring NDVI differences via drones, manned aircraft, or satellites may suffice and may provide equivalent or better information (if calibrated). This analysis does not demonstrate any superiority of the proposed approach compared to simpler models based on satellite NDVI and models using weather station data.

The interpretation of this spatial variability as management-relevant differences in ET is not convincing. The elephant in the room, that there is unexplained variability of dependent or independent variables within flight paths as well as amongst flights has not been sufficiently addressed.

 

For perennial or high value crops, how high to fly is ultimately dependent on the size of the smallest target feature.

Size of target features has not been a core aim/objective of this paper and this conclusion should not be made.

 

Spatial data were the predominant source of uncertainty in the HRMET model. High solar radiation variability or low wind speeds were the biggest driver in the difference between HRMET and Eddy Covariance (EC) ET estimates.

Unfortunately, this rather limits the usefulness of the approach. Solar radiation and wind speed are likely to change within the 20 minute time interval of flights. This implies that pattern in drone imagery may be most strongly related to instantaneous differences of wind speeds, high clouds producing variation in incoming radiation or specific humidity changing with fetch conditions rather than reflecting spatial differences in imagery-derived ET.

 

Uncertainty maps are an underutilized tool that have the potential to evaluate strengths and weakness of high-resolution ET models.

There is believable but there is little evidence if the uncertainty maps depict modelling error or variability in input variables.

 

Going forward, further application to the HRMET model would prove beneficial, especially for precision irrigation crops that may not have the ideal ‘hot’ and ‘cold’ pixels. Further study is needed to understand how HRMET performs in other climates outside the Midwest United States.

 

This could be true, but the results may also be interpreted as a research dead-end due to inaccessible factors introducing bias and variability. The conclusions of this paper are too strongly biased by the belief that this method should be working.

 

Author Response

1. Spatially explicit irrigation control is the holy grail of agricultural and horticultural remote sensing Stronger evidence to demonstrates ease and accuracy of fast and inexpensive spatial methods is highly needed. However, this paper may not serve this overarching aim as it makes promises that are not substantiated. Barely any of the conclusions are underpinned with clear evidence from this analysis. The analysis can therefore not be interpreted as a study in support of the presented approach, despite excellent data collection and processing. The paper cannot be published with it’s current aims and conclusions.

 

We would like to thank Reviewer 1 for their comments and appreciate their acknowledgement of the ‘excellent data collection and processing’ that went into this work. We share Reviewer 1’s long term vision for spatially explicit, informed irrigation control of agricultural crops. Additionally, we agree that spatially explicit maps of evapotranspiration (ET) are a necessary step towards this goal. Though we share the same long-term vision as Reviewer 1, the goal of this study was to assess and explore one such ET model for use with UAV imagery in a Midwestern potato crop. Our revision further clarifies and differentiates between this long-term vision for spatially-explicit irrigation control and the goals and findings of this study as an incremental step in the direction of drone-based ET model development. We have addressed Reviewer 1’s suggestions and have narrowed the scope of conclusions accordingly.

 

2. “Drone based remote sensing offers the ability to rapidly generate ET maps within a season that can be used to make in-season decisions. The use of these maps with precision irrigation has the potential to reduce water use and nitrogen loss from agricultural fields. The High Resolution Mapping of Evapotranspiration (HRMET) model was found to be a promising model for further development in potatoes with drone imagery collected under optimal conditions.” This is a superficial claim for which the paper does not provide clear evidence. The accuracy of the maps cannot be assessed quantitatively as there is no spatially explicit reference data. An overestimate between approx. 50 and 150% in 5 out of 8 carefully selected mission dates cannot give land managers confidence in the method. The management relevance is grossly overrated.

 

We assessed HRMET against the Eddy Covariance (EC) values with the specific spatial footprint in mind (a 27.4m radius circle around the tower). This was clarified in section 2.6 (lines 248-250). The issue of using the EC tower footprint is also addressed in section 4.1 (lines 430-436). The carefully selected mission dates show the importance of the optimal climate conditions and following best practices for drone imagery collection. This is rephrased in line 541-545 to reflect its importance. The management relevance was not part of what we conducted our research on, but what the field of precision agriculture can do. The language of the conclusion was changed to reflect this difference in outlook vs. scope on lines 555-556.

 

 

3. “We evaluated whether there was an increase in insightful information form ET maps created using imagery collected at 90, 60, or 30 m. Though there were some additional detectable features low elevation flights, the tradeoff in resources and computation does not justify these low elevation flights for annual vegetable crops in the Mid-west USA.” The data shows lower variability (higher min, lower max) in the low resolution data. This may be entirely due to smoothing. However, this is a rather trivial observation and there is no information that may elucidate this further, i.e. show to what degree the whole may be more than the sum of it’s parts (i.e. the variability can be derived by simple averaging).

 

We appreciate these observations and comments from Reviewer 1. Our photogrammetry process did not include smoothing. We agree that the variability can be derived through averaging as the mean/median values were similar throughout the different mission sets. The additional detectable features in the low elevation flights did not disappear at higher elevation flights, but were incorporated into larger mixed pixels. However, finer detectable features may be important for applications like the creation of management zones. This would require further study and was outside of the scope of this work. Clarification regarding these observations and issues was added to section 4.3, lines 476-480.

 

4. It is surprising to see the discrepancy between pattern at different scales, excluding the central access road. Mean and median are very similar, i.e there seems to be no bias effect of scale. This is an important observation. However, it may invalidate the approach as this suggests a very high confidence in the erroneous means. It also implies that there are “additional detectable features” at all scales. This needs to be given a stronger consideration as this may show that there are potential artefacts affecting the results. Pattern may – or may not – be related to ET.

 

We evaluated our confidence in the mean values by conducting the Monte Carlo analysis. The similarity between the mean and median values signified a symmetrical distribution of values. This supports our decision of how we formed our data ensembles used in the Monte Carlo analysis. We agree that the pattern may or may not be related to ET and have added that stipulation to section 3.1 in lines 293-294.

 

 

5. Spatio-temporal differences (i.e. incoming solar radiation, wind, humidity, BRDF) during the flight time may generate differences in ET estimates, without any possibility to judge if this a true effect of plant stomatal control or an error in the approach.

 

Thank you, this was a great point for us to consider. We were able to consider if the differences in ET estimates were an effect of plant stomatal control or an error in the approach using the Monte Carlo Analyses. On July 22^nd we saw higher uncertainty and were able to attribute it to the high variability in the meteorological data. On the other hand, on June 21^st and July 8^th, we saw lower uncertainty with lower meteorological variability, so those differences may be more based in plant stomatal control. We added this observation to section 3.5 in lines 366-370.

 

6. On the other hand, if these patterns represent actual differences in stomatal control and hence ET, in other words, if the method can detect high spatio-temporal differences in EC-ET, the problem is worse. In this case, the results would imply that the pattern reflect instantaneous estimates of highly variable factors and should probably not be interpreted as representative for longer-term (hourly, daily, weekly) water loss. Later is crucial for management relevance of the drone data. In the latter case, simple monitoring NDVI differences via drones, manned aircraft, or satellites may suffice and may provide equivalent or better information (if calibrated). This analysis does not demonstrate any superiority of the proposed approach compared to simpler models based on satellite NDVI and models using weather station data.

 

We appreciate these considerations and ideas from Reviewer 1. NDVI can become green saturated in the late season when there is high LAI. This is the case for potatoes where the LAI can get as high as 6 or 7 [2]. Thermal imagery has the been shown to have stronger relations to stomatal conductance than NDVI [3] and can detect water stress sooner than NDVI [4]. Other Vegetation indices avoid this issue, but these are analogs for stress, not for water use. ET maps, like the ones generated with HRMET, need to be scaled to daily or beyond to generate a value that can be prescribed for irrigation. This background has been added to the introduction in lines 73-77.

7. The interpretation of this spatial variability as management-relevant differences in ET is not convincing. The elephant in the room, that there is unexplained variability of dependent or independent variables within flight paths as well as amongst flights has not been sufficiently addressed.

We thank Reviewer 1 for making this point and bringing up the elephant in the room. We agree that there is more variability to address and it will be important to understand this variability going forward. However, we conducted an uncertainty analysis of our flights and provided additional context to the imagery. Spatially-explicit based measurements of stomatal conductance and photosynthesis would greatly complement tower-based comparisons [5]. This was context was added to section 4.5 in lines 533-536.

8. “For perennial or high value crops, how high to fly is ultimately dependent on the size of the smallest target feature.” Size of target features has not been a core aim/objective of this paper and this conclusion should not be made.

This was intended as an extension of our findings but we agree that this is outside the purview of the study and its methods. The line was removed.

 

9. “Spatial data were the predominant source of uncertainty in the HRMET model. High solar radiation variability or low wind speeds were the biggest driver in the difference between HRMET and Eddy Covariance (EC) ET estimates.” Unfortunately, this rather limits the usefulness of the approach. Solar radiation and wind speed are likely to change within the 20 minute time interval of flights. This implies that pattern in drone imagery may be most strongly related to instantaneous differences of wind speeds, high clouds producing variation in incoming radiation or specific humidity changing with fetch conditions rather than reflecting spatial differences in imagery-derived ET.

 

While it is true that the drone imagery is impacted by the instantaneous changes in wind speed and high clouds, this speaks to more importance of using best practices for collecting remote imagery mentioned in sections 2.2 and 4.4. This is also addressed in section 4.3 with the overarching discussion on how high to fly, and how lower elevation flights take longer and introduce more sources of error.

 

10. “Uncertainty maps are an underutilized tool that have the potential to evaluate strengths and weakness of high-resolution ET models." There is believable but there is little evidence if the uncertainty maps depict modelling error or variability in input variables.

We changed ‘models’ to ‘maps’ to reflect the strength and weakness of both the models and of their input variables, in line 555.

11. “Going forward, further application to the HRMET model would prove beneficial, especially for precision irrigation crops that may not have the ideal ‘hot’ and ‘cold’ pixels. Further study is needed to understand how HRMET performs in other climates outside the Midwest United States.” This could be true, but the results may also be interpreted as a research dead-end due to inaccessible factors introducing bias and variability. The conclusions of this paper are too strongly biased by the belief that this method should be working.

We appreciate Reviewer 1’s concerns about method bias and our intent is to offer HRMET as one of many possible models that can be used. The tone of this section was changed in lines 556 to reflect this.

Author Response File: Author Response.docx

Reviewer 2 Report

General comment:

The study presents the effect of three different UAV fly heights (30m, 60m, and 90m) on estimating evapotranspiration over an irrigated potato field. The research is impressive, well-written manuscript, highlighting the potential future research directions. Please see the minor comments below.

 

Specific comments:

Line: 124: add an explanation for a 0.65 ha subsection (flux tower footprint?)

Section 2.2: what were the research design criteria for eight flight mission sets? Based on growing seasons, weather conditions, soil moisture conditions, ….

Section 2.7: Uncertainty of fluxes from EC towers varies with several factors; looks like energy balance was not closed before comparing with HERMET ET.  If net radiation and soil heat flux data are available, better to close the energy balance or at least report the closure error

Lines 265-266: probably the 2.5m tall flux tower has larger footprints than 27.4m radius

Line 342: report the spatial data (canopy temperature, LAI, height), or is it just the model is more sensitive to these spatial inputs

Section 4.3: also comment on potential differences from different fly heights over tall crops and high water-consuming crops

Lines 468-474: these sentences could fit in the methods section

Author Response

Please see the attachment

Author Response File: Author Response.docx

Reviewer 3 Report

Plese find my minor comments included in the pdf attached.

Comments for author File: Comments.pdf

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

Unfortunately, the authors were unable to address concerns.

In fact, some of the modifications have further enhanced the concern that the conclusions were based on a biased preconception rather than the evidence provided in the results of this study. The added sentence to the start of the conclusions (lines 541-542) “The High Resolution Mapping of Evapotranspiration (HRMET) model has already been found to be a promising model for Midwestern potatoes using aerial imagery” shows this preconception, which is carried through the entire paper. The added qualifier (line 544) “under optimal conditions” is insufficient. I would have at least expected some evidence elucidating which parameters or conditions are optimal and hence produce reliable results.

Interpreting the results show in Fig 12 I would have expected a sentence in the conclusion along the lines: Absolute magnitude of ET predicted using the HRMET approach differed substantially from EC-ET.

I would have expected some conclusions related to the observations presented in Fig 7 that relative ET raster maps flown at different heights produced inconsistent pattern. While patches of low ET (service road) were mapped consistently at all heights and dates, highest relative ET patches vary substantially amongst different heights within a day. Does this mean that the HRMET approach may miss critical factors/model components? On a positive note, pattern seem consistent across flight paths (except for June 21, 90m height, which shows stripes of lowest relative ET parallel to the service track. Could such observations shed light on the low correspondence between HRMET magnitudes and EC-ET?

I would have expected some interpretation of what these results may mean with respect to the feasibility of the HRMET approach. The key sentence of the conclusions “Drone based remote sensing offers the ability to rapidly generate ET maps within a season that can be used to make in-season decisions.” is unsupported. The conclusions are superficial and avoid any potential negative comment about HRMET in spite of presented evidence to the contrary.

 

Detailed comments to response items:

1-2) The response to 1 and 2 does not sufficiently address the issue. I would not consider an overestimate between approx. 50 and 150% in 5 out of 8 carefully selected mission dates sufficient evidence for success, irrespective of potential future uses.  I would suspect that results from this figure could produce a negative Nash–Sutcliffe model efficiency coefficient, i.e. stating that a simple average would explain more of the variance explained by applying the HRMET model.

3) This does not sufficiently address the issue. Spatial smoothing of the signal (ET) is effectively achieved by increasing flight heights.

4) I see hugely different pattern in the ET models from different flight heights, indicative of other factors potentially obscuring the ET signal. This was also stated in the original document. Simply removing this statement does not help to address the issue that other factors may cause flawed maps.

5-7) It is good to see that the uncertainty approach shows relationship with meteorological variability However, I am less concerned about average uncertainty and what it means than from pattern shown in Fig. 7, which shows inconsistency amongst heights within a single day. The changes made to the document do not address my issues raised.

9) This also relates to the “elephant in the room” and strengthens my concerns that the entire approach could potentially be flawed given the low correspondence and high spatial signal in the uncertainty analysis. I think this paper does not sufficiently support the presumption that the approach is useful. On a minor note, I am not clear if a conclusion from this paper would be that in future ET mapping common practice should be changed to fly high and fast?  Why then do you get these strange linear patterns parallel to the service track shown in in Fig 7 only at 90m flight heights with a low uncertainty? This shows error that is undetected by the uncertainty analysis.

11) The change to “model or models like it could prove beneficial” (line 556) broadens the conclusions to methods similar to HRMET. Such a statement requires mentioning on what evidence this is based on.