Mapping Burn Extent of Large Wildland Fires from Satellite Imagery Using Machine Learning Trained from Localized Hyperspatial Imagery
Round 1
Reviewer 1 Report
While several helpful changes were made, this paper does not seem to be ready for publication. Many of the original writing problems persist and most of the revisions have new writing problems, leaving the reader guessing what the authors intended to say.
Authors do not appear to have a clear understanding of the manned and unmanned wildfire imagery acquisition and analysis that is occurring at this point.
The addition of the implications for management were helpful, but actual implementation of these methods still remain somewhat unclear, especially given the lack of adequate detail in the methods.
It is still unclear what results are from this research effort and which are from citations 19 and 22. This should be written more clearly than what is provided in the two short paragraphs of results.
If this is published I suggest the editors help the authors clean this writing up substantially.
Author Response
Open Review
(x) I would not like to sign my review report
( ) I would like to sign my review report
English language and style
(x) Extensive editing of English language and style required
( ) Moderate English changes required
( ) English language and style are fine/minor spell check required
( ) I don't feel qualified to judge about the English language and style
|
Yes |
Can be improved |
Must be improved |
Not applicable |
|
|
Does the introduction provide sufficient background and include all relevant references? |
( ) |
(x) |
( ) |
( ) |
|
Is the research design appropriate? |
( ) |
(x) |
( ) |
( ) |
|
Are the methods adequately described? |
( ) |
(x) |
( ) |
( ) |
|
Are the results clearly presented? |
( ) |
(x) |
( ) |
( ) |
|
Are the conclusions supported by the results? |
( ) |
(x) |
( ) |
( ) |
Comments and Suggestions for Authors
While several helpful changes were made, this paper does not seem to be ready for publication. Many of the original writing problems persist and most of the revisions have new writing problems, leaving the reader guessing what the authors intended to say.
Authors do not appear to have a clear understanding of the manned and unmanned wildfire imagery acquisition and analysis that is occurring at this point.
The addition of the implications for management were helpful, but actual implementation of these methods still remain somewhat unclear, especially given the lack of adequate detail in the methods.
DH: We gave more detail regarding the study areas. More than willing to discuss further with the editors, but would like to know what additional detail they’re looking for.
It is still unclear what results are from this research effort and which are from citations 19 and 22. This should be written more clearly than what is provided in the two short paragraphs of results.
EL: Make another comment to why are results sections has two sources
If this is published I suggest the editors help the authors clean this writing up substantially.
EL: Ill go through all the formatting and and use grammarly
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The author's adequately addressed my concerns. Thank you.
Author Response
Open Review
(x) I would not like to sign my review report
( ) I would like to sign my review report
English language and style
( ) Extensive editing of English language and style required
( ) Moderate English changes required
(x) English language and style are fine/minor spell check required
( ) I don't feel qualified to judge about the English language and style
|
Yes |
Can be improved |
Must be improved |
Not applicable |
|
|
Does the introduction provide sufficient background and include all relevant references? |
( ) |
(x) |
( ) |
( ) |
|
Is the research design appropriate? |
(x) |
( ) |
( ) |
( ) |
|
Are the methods adequately described? |
( ) |
(x) |
( ) |
( ) |
|
Are the results clearly presented? |
( ) |
(x) |
( ) |
( ) |
|
Are the conclusions supported by the results? |
( ) |
(x) |
( ) |
( ) |
Comments and Suggestions for Authors
The author's adequately addressed my concerns. Thank you.
DH: The authors concerns were already addressed.
Submission Date
28 October 2020
Date of this review
05 Nov 2020 23:05:23
Reviewer 3 Report
Dear editor and authors,
The manuscript “Mapping Burn Extent of Large Wildland Fires from Satellite Imagery Using Machine Learning Trained from Localized Hyperspatial Imagery” provides interesting insights into the use of drones to improve understanding of the data currently acquired in the various large-scale satellite monitoring programs.
Nonetheless, this study seems more than anything else to support previous analyzes and studies (some of them not clearly peer-reviewed) reported by the authors themselves in the References (in a sort of continuous self-citation).
The topic is clear but the manuscript is constructed in a rather unusual way with a lot of space for methods (almost never sufficiently reported in full, but most of them are referred to previous unverifiable publications).
The introduction is rather cumbersome, does not always follow a logical thread and above all it does not clearly outline the objectives of the study.
The objectives are unclear and never declared.
In addition to what has already been mentioned above, the methods include some avoidable repetitions and they do not define any statistical analysis or other processing on the dataset (other than pre-processing procedures).
The results are consequently insufficient, unclear and do not follow the usual process of acquisition, analysis, verification and exposure.
The references seem poor with many references to non-peer reviewed sources found on websites or personal communications.
My general suggestions are outlined below, in support of the future authors' work.
INTRODUCTION
Rows 30-37: this part should be moved to the conclusions rather than as an introductory incipit.
Rows 116-125: a little confusing, it needs synthesis and a clearer description.
Row 132: “FAA” the first time requires at least a description of the acronym.
Rows 158-161: scientific name of the vegetation species should be reported in italics format.
Figure 3 and 4: I suggest to add a description of the values reported in the X axis, e.g. (px%).
Row 205: “Biomass consumption” instead of “Burn extent”.
At the end of the introduction chapter, the objectives of the work were not explicitly defined.
METHODS
The methods chapter should include the descriptions or the references to the acquisition protocol and therefore to the data analysis; instead part of the methods is repeated and often refers to studies cited in the references that have not been validated with peer-reviewed procedures, so in my opinion they should be specified better and in full in this section. The descriptions of the study sites are not available and the characteristics of the wildfires are never explained (forest structure, burn severity, size).
Row 215: the objective indicated here is not clear to me.
Rows 230-232: this sentence is confusing and the objectives are not well defined.
Rows 275-276: in my opinion burn severity is a crucial parameter that deserves a descriptive mention of the classes considered and how they were obtained. The “denoised” procedure requires a clear description of the method (or a citation to a verifiable reference).
Rows 281-285: in the caption of the Figure 10, colours of the burned and unburned pixels are associated exactly in reverse with respect to the legend of the Figure 10.
Rows 286-335: this paragraph seems a repetition of the paragraph 1.1 (rows 178-213).
Author Response
Open Review
(x) I would not like to sign my review report
( ) I would like to sign my review report
English language and style
( ) Extensive editing of English language and style required
( ) Moderate English changes required
(x) English language and style are fine/minor spell check required
( ) I don't feel qualified to judge about the English language and style
|
Yes |
Can be improved |
Must be improved |
Not applicable |
|
|
Does the introduction provide sufficient background and include all relevant references? |
( ) |
( ) |
(x) |
( ) |
|
Is the research design appropriate? |
( ) |
( ) |
(x) |
( ) |
|
Are the methods adequately described? |
( ) |
( ) |
(x) |
( ) |
|
Are the results clearly presented? |
( ) |
( ) |
( ) |
(x) |
|
Are the conclusions supported by the results? |
( ) |
( ) |
( ) |
(x) |
Comments and Suggestions for Authors
Dear editor and authors,
The manuscript “Mapping Burn Extent of Large Wildland Fires from Satellite Imagery Using Machine Learning Trained from Localized Hyperspatial Imagery” provides interesting insights into the use of drones to improve understanding of the data currently acquired in the various large-scale satellite monitoring programs.
Nonetheless, this study seems more than anything else to support previous analyzes and studies (some of them not clearly peer-reviewed) reported by the authors themselves in the References (in a sort of continuous self-citation).
DH: This paper describes the latest step in the line of research that this team has been working on for the past few years, so yes, it will cite previous work published by this team that this particular effort built on top of. In the introduction, there are citations to non-peer reviewed website, but they are primarily websites that are maintained by the wildland fire leadership in the United States. We feel these are appropriate for how they are used in the introduction. The only one of our papers that was cited that could possibly be questioned as being peer-reviewed would be the dissertation, but those citations where supported by citations of associated peer reviewed articles. The Dissertation citations were included because the dissertation in certain select instances may contain minor additional detail that was not included in the associated peer reviewed publication.
The topic is clear but the manuscript is constructed in a rather unusual way with a lot of space for methods (almost never sufficiently reported in full, but most of them are referred to previous unverifiable publications).
DH: Addressed citations previously. We welcome an opportunity to further refine the structure of the document if the editors request us to.
The introduction is rather cumbersome, does not always follow a logical thread and above all it does not clearly outline the objectives of the study.
DH: The first paragraph of the introduction addresses what we are doing. The second paragraph of the introduction describes why this is important and what the implications and benefits of this work are. Beyond that, the background sub-section contains summaries of the background work that this project builds upon. We welcome an opportunity to further refine this if the editors request us to.
The objectives are unclear and never declared.
DH: The objectives are stated in the opening paragraph. We welcome an opportunity to work with the editors to refine and more clearly state the objectives.
In addition to what has already been mentioned above, the methods include some avoidable repetitions and they do not define any statistical analysis or other processing on the dataset (other than pre-processing procedures).
DH: The Results and Discussion sections addresse the statistical improvements that were obtained through this study. I’m not seeing unavoidable repetition in how we present the data.
The results are consequently insufficient, unclear and do not follow the usual process of acquisition, analysis, verification and exposure.
DH: The acquisition, analysis, verification and exposure pathway is fairly close to what we were following with the exception that we also had to insert after acquisition steps which allowed for the development, training and utilization of methods and associated tools which enabled the generation of burn severity mapping products which then were assessed with the analysis, verification and exposure steps of the pathway.
The references seem poor with many references to non-peer reviewed sources found on websites or personal communications.
DH: Already assessed that these citations are primarily from websites utilized by the US wildfire leadership and were appropriate for their use in the introduction.
My general suggestions are outlined below, in support of the future authors' work.
INTRODUCTION
Rows 30-37: this part should be moved to the conclusions rather than as an introductory incipit.
DH: This introductory paragraph says what we’re doing, the following paragraph says why it’s important.
Rows 116-125: a little confusing, it needs synthesis and a clearer description.
DH: edit made per reviewer’s suggestion.
Row 132: “FAA” the first time requires at least a description of the acronym.
DH: edit made per reviewer’s suggestion.
Rows 158-161: scientific name of the vegetation species should be reported in italics format.
DH: edit made per reviewer’s suggestion.
Figure 3 and 4: I suggest to add a description of the values reported in the X axis, e.g. (px%).
DH: edit made per reviewer’s suggestion.
Row 205: “Biomass consumption” instead of “Burn extent”.
DH: edit made per reviewer’s suggestion.
At the end of the introduction chapter, the objectives of the work were not explicitly defined.
DH: Objectives are defined in the opening paragraph. We are more than willing to restructure the Introduction in the editors request.
METHODS
The methods chapter should include the descriptions or the references to the acquisition protocol and therefore to the data analysis; instead part of the methods is repeated and often refers to studies cited in the references that have not been validated with peer-reviewed procedures, so in my opinion they should be specified better and in full in this section. The descriptions of the study sites are not available and the characteristics of the wildfires are never explained (forest structure, burn severity, size).
DH: The references cited in the methods section have been peer-reviewed, except for the dissertation. This issue was addressed above.
Row 215: the objective indicated here is not clear to me.
DH: The objective looks clear to me and my co-authors, but we would be willing to further clarify if requested by the editors.
Rows 230-232: this sentence is confusing and the objectives are not well defined.
DH: This looks clear to me and my co-authors, but we would be willing to further clarify if requested by the editors.
Rows 275-276: in my opinion burn severity is a crucial parameter that deserves a descriptive mention of the classes considered and how they were obtained. The “denoised” procedure requires a clear description of the method (or a citation to a verifiable reference).
DH: Burn severity was addressed and defined in the introduction. Gonzalez, 2008 Reference will be added addressing morphology.
Rows 281-285: in the caption of the Figure 10, colours of the burned and unburned pixels are associated exactly in reverse with respect to the legend of the Figure 10.
DH: edit made per reviewer’s suggestion.
Rows 286-335: this paragraph seems a repetition of the paragraph 1.1 (rows 178-213).
DH: This looks appropriate to me.
Submission Date
28 October 2020
Date of this review
09 Nov 2020 11:22:09
Round 2
Reviewer 3 Report
Dear authors,
I leave to the editor the advice to reconsider the manuscript prior, on your part, an important effort in the direction already suggested.
I would suggest that you first submit to a peer-reviewed journal the parts of the cited doctoral dissertation that is so important to this study.
Author Response
The portions of the dissertation that were cited have already been peer-reviewed and published.
Hamilton, D; Pacheco, R; Myers, B; Peltzer, B, (2020) “kNN vs. SVM: a Comparison of Algorithms”, Proceedings of the Fire Continuum-Preparing for the future of wildland fire; 2018 May 21-24; Missoula, MT. Proceedings RMRS-P-78. Fort Collins, CO: U.S. Department of Agriculture, Forest Service, Rocky Mountain Research Station. 358 p.
Hamilton D., Hamilton N., Myers B. (2019) Evaluation of Image Spatial Resolution for Machine Learning Mapping of Wildland Fire Effects. In: Arai K., Kapoor S., Bhatia R. (eds) Intelligent Systems and Applications. IntelliSys 2018. Advances in Intelligent Systems and Computing, vol 868. Springer
Hamilton, D; Myers, B; Branham, J, (2017) “Evaluation of Texture as an Input of Spatial Context for Machine Learning Mapping of Wildland Fire Effects”. Signal and Image Processing: An International Journal, 8(5).
Hamilton,D; Bowerman, M; Colwell, J; Donahoe, G; Myers B, (2017) “A Spectroscopic Analysis for Mapping Wildland Fire Effects from Remotely Sensed Imagery”, Journal of Unmanned Vehicle Systems, 5(4), 146-158.
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
This paper could make a valuable contribution with some improvements. The most important missing element is how managers can change their actions as a result of superior information about fire extent and severity. This bears directly on the modeling accuracy assessment. For example do you expect they would change the areas that are replanted, change the hydrologic response predictions and therefore the sizes of culverts replaced, etc. to avoid secondary fire impacts? Without this key linkage to management the resulting accuracies are meaningless.
Another problem is the lack of clariry about the validation. It appears that the paper shows how high resolution data can be resampled to match lower resolution (30m cells) using fuzzy logic. But it remain unclear after reading how the authors assess validation. What is the truth in this case? Do you have a detailed map of each of these fires that has classified the post fire environment into your unburned, black ash and white ash categoires. How reliable is that data, and did it use some fuzzy logic or something else. If you are instead assessing accuracy by comparing your 30 of data that was left out, what is the comparison exactly.
After reading it seems like the most important comparison would be between 30 meter satellite imagery classified with fuzzy logic and 30 meter data that was resampled from 5 cm data using fuzzy logic. THis would tell you the value of leveraging the 5 cm data from part of the fire (from sUAS). Otherwise, of course the more granular data will have a higher accuracy rate, assuming you can actually map and validate.
Another problem is that the authors are only looking at the two far ends of the spectrum, low elevation sUAS and satellites. What about full size UAS, and manned sensors. These can cover large fires completely now, quickly after a fire or while it is burning. Why aren't these addressed? They are flying fires all over the place now.
The results are confusing. Table 1 shows two citations, one appears to be conference proceedings and the other is not clear if it was a presentation or a paper. Are these results from those papers or from your own work here? There are only six, apparently small fires in the results tables. If the intent is to showcase this resampling with fuzzy logic, why not use many lare fires. You mention a mean accuracy rate, is this computed by cell across all fires or by averaging fires of various sizes? It is not shown in the table. Your discussion and conclusion are very repetitive, but they don't really describe what is shown in the tables. for example the 30 meter severity has as many superior accuracy scores to the 30 meter for the color results. You spend way too discussion the IR comparison which you admit you did not take the time to do correctly.
The writing towards the end of the draft becomes quite sloppy, even including a phrase from the journal template. You have background in methods, you have future work in discussion, the paper needs quite a bit of work. Here are some specific comment to help you get started.
38, suppression is not the entire explanation, consider adding other contributors.
58, you are clearly citing other work from this team, no need to describe it in the paper
70, severity does not refer to how intensely a fire is burning that is called fire intensity, and severity does not just refer to the response over subsequent years, it refers to the impact from the fire, which can last from the combustion for several years
79, omit unburned at first use so it only appears once
92, extinguished fire can be not is a dangerous, plenty of them don’t have trees or any hazards afterwards.
94- if it is hard to see from a helicopter at 100’ it will also be hard to capture from any imagery from space
129 sub hertz temporal resolution, not minutes
134 regulatory requirements are to submit an Burned Area Emergency Response assessment and proposal within 14 days of containment, not to acquire imagery. That is normally done but not required.
160 what about full size UAS and manned aircraft, they can be used?
198 You have not described why you don’t use the actual 30m imagery in this comparison
Figure 9, explain what is white and what is black
271, these are background not methods.
291, has the previous work described how well the methods work across various terrain, fuel type, fuel loading etc?
298, consumption only needed to be evaluated…
303, capitalize IF if you are going to capitalize AND and OR functions
320, striped or stripped
332 why did you eliminate unburned from your accuracy assessment, you can use more than two classes
338 are there better references for 19 and 22 available?
346- delete journal template language before submitting for review
367-mean is not shown or adequately described to know what you did for your calculations
372 smaller than what
373-374 remove thus
Figure 11, caption actually seems like one of the most important findings
378 writing gets very sloppy starting here
420 do you really want to end talking about IR that you admit you did not really address properly. IR works better for detecting heat and some burned conditions but not all, I would just say it once that you didn’t test that properly yet and why, other than you didn’t have time.
Reviewer 2 Report
Authors attempted to model fire extent and severity with very high resolution images of sUAS by representing fractional density of fire pixels and trained it to Landsat images using machine learning approach. As authors mentioned, studies about fire extent and severity with unmanned vehicle are important to clarify how many areas are overestimated by simplified information in relatively coarse resolution of satellite-based images. They used machine learning approach, SVM.
1. However, I couldn't find any novelty, scientific finding, and meaningful discussions. The structures and formats of paper did not meet to the journal of remote sensing.
2. For examples, many of results are located in Introduction, or Materials and Methods.
3. Introduction includes sentences related to the methodology.
4. Main body of the Result part is too short and the information seems to be poor.
5. Introductions need to be objectively written adding more citations. Now it seems to be excluding substantial literatures.
6. Tables and Figures are not fully describing legend, target locations, data acquisition time, and so on, although those are essential.
7. In the part of machine learning model developments, the method how to collect reference data have not been clearly written.
8. Logics why white and black colors can be a proxy of burning severity or completeness were not showed.
9. Imbalanced data issue was mentioned in method part, however, data distributions are not shown.
10. Table 1 and 2 includes the names of Elephant, MM106, and so on, however, the meaning of those names was not presented in any part of the manuscript.
Accordingly, in my opinion, this study needs to be re-constructed. For the rewriting, exceeding time will be taken. Therefore, my decision is reject.
Reviewer 3 Report
- There should be a section explaining the study areas. The first time anything about the study is mentioned is in the results. Most of the paper is about theory and methods, which is fine, but we have no idea where this is being applied until the "Results" section.
- More discussion of the results are needed. For example, explain why there are differing results for the study area.
All other comments can be found in the attachment.
Comments for author File: Comments.pdf
