Round 1

Reviewer 1 Report

The paper demonstrates the use of freely available resources for estimating land cover types, particularly those involving trees, in Rwanda. The findings are potentially valuable for policymakers and planners, e.g., for understanding livelihoods and/or potential for carbon sequestration. The writing is clear and the graphics are attractive. The most persuasive explanation for the value of this paper, land use change detection, is not attempted in this paper, unfortunately.  

The principal weakness of the statistical approach is that the authors do not articulate a theory behind model selection. Because they do not use an approach that acknowledges a bias-variance tradeoff (such as information criterion approaches), they end up selecting the most complex models. The three ‘best’ models incorporate large numbers of data layers—it’s hard to know how many, but possibly 199—to predict land use cover. An information-criterion approach might select models with many fewer variables, and this would be a service to future analysts. A simpler more robust model would have wider applicability. Further, because the authors have no criteria for selecting a best model, much time is spent discussing miniscule differences in predictions among the three favored models.

 

L18-21. Consider switching the last two sentences. The final sentence should be a conclusion, not an expression of concern about the accuracy of the approach.

L40. Unclear what is meant by ‘discrepancies’

L51. Also please provide a definition of agroforestry.

L52. What is meant by ‘associated with’? Does agroforestry have these beneficial properties? If so, say so.

Agroforestry paragraph 52-67 is rambling. Interesting ideas are introduced but none are developed. What is the central idea of this paragraph,  agroforestry or shrublands? Define shrublands

L111. Define ‘lazy evaluation’

L117. Here it is stated that the goal is to use state-of-the-art machine learning methods, but at the end of the manuscript you say that GEE limited the tools you could use and that “next-generation methods” should be used for future studies. I would delete the ‘state-of-the-art’ cliché.

L144. Add names of districts, at least for the study area; district names are referred to frequently later in the manuscript, but without a map they are meaningless

L180. Is it correct that dry season data were filtered by maximum cloud cover of 50%? Shouldn’t the 50% and 20% on L181 be reversed?

192. This table is long and dates are repeated multiple times. You could make it more compact, e.g., by having one line for each month, and superscripts for days indicating which tiles were used, e.g.

January 1^αβδ, 8 ^αβδ, 11 ^αβγδ, etc.

L206. In Fig. 3b, it is unclear what the dark purple area indicates because dark purple is not shown on the legend scale.

L224. I believe these indices are referred to as vegetation indices later, so it would be helpful to use that term here.

L229. Explain what a gray level co-occurrence matrix is more clearly.

L232. It is unclear how the PCA was done, explain. Inputs to PCA were the values for the various spectra at the location of each pixel?

L242. The implication here seems to be that the government studies did not properly define characteristics used to identify land cover types, is that what is meant?

L258. This is more of a figure than a table. I suggest reordering to have forest, then, agroforestry, then shrubland, i.e. order by increasing intensity of exploitation. These descriptions should be in the text rather than here.

L262. Provide more detail. How was biomass estimated?

Table 4. I don’t see the value of this table; one could easily just say that each cover type had > 4000 test pixels and validation pixels.

Fig. 2. Axis labels would be more readily interpreted if they stated the quantity described, e.g., ‘Elevation (m)’ and ‘Precipitation (mm 5d^-1).

L278-279. Move sentence to discussion.

L315. Explain briefly how the Random Forest algorithm works

L330. Re: confusion, the authors mention user and producer accuracy, then state that conclusion was also assessed using confusion matrices in GEE—then explain how UA and PA were calculated from the confusion matrices. It’s the use of the word also that is confusing: were UA and PA calculated in two different ways?

L331. Explain briefly how Kappa value works

L340. Table 5 needs a more informative heading. E.g. remind reader of difference between models whose names are prefixed by ‘P’ vs. ‘Sp’.

L344. Define impurity

L367. I question the usefulness of Fig. 5 – it seems mainly to show that the spectral bands alone do a poor job of distinguishing among land cover types.

Fig. 6, 7. These figures show a similar pattern, of forest being detected more accurately than agroforestry or shrubland, over and over.

L407. It’s unclear to me how all the bands mentioned add up to 199, so it may also be for the average reader. This would be a good place to briefly state where the 199 figure comes from.

Table 7. The differences in classification among models seem infinitesimally small.

Figure 8. (a) is an RGB image; I would not call it ‘actual forest cover’. It is not clear why panel (b), the model agreement graph, has portions that show RGB imagery; consider making that part a uniform color (e.g. white). Also unclear how you can have ‘one model’ agreement; doesn’t that mean there is no agreement? It is odd to find the image showing location in study area as the 3^rd graph, I would expect to find it first. It is not very feasible to find differences among d, e, and f; why not just show the best model? Also, in 8b the term ‘forest cover’ is used to describe forest cover only, whereas in 8 d,e,f, ‘forest cover’ indicates not only forest cover but also agroforestry, cropland, shrubland, etc. More descriptive accuracy is needed.

Figure 9. Most comments for 8 also apply here.

L501. Again, ‘1 model’ agreement means no agreement, correct? If so it should be represented as such

L619. Sounds interesting: do any of the figures illustrate the eastward decline in agroforestry and increase in shrublands?

L668. Wasn’t ‘expensive and costly fieldwork’ done as part of the present study? So how can that be cited as reason for superiority of the approach used in this paper compared to high-resolution imagery approaches?

L689-693. The last two sentences of the paper is not the place to introduce new technologies. If ‘fusion’ might be a better option than the approach adopted by the authors, explain why in the body of the discussion.  It would be better to end the paper by emphasizing what was learned, or its significance.

Editorial

L8. A semi-colon should separate two independent clauses, but the second part of the sentence (starting with ‘with’) is not independent

L14, 18. Change semi-colon to period

L35. Put a comma after ‘practices’ (enclose parenthetic expressions between commas)

L40. Second ‘with’ in sentence. Use of ‘with’ in this way, as a generic connector, is unsatisfactory because it avoids making a connection between the two clauses. Replace with either ‘and’, ‘but’, ‘although’, or a semi-colon. (See also L160 ‘while’; L165;  )

L42. ‘trees outside forests’ is awkward. ‘Isolated trees’ would be better.

L43. Delete ‘purposes’ (redundant)

L45. Run-on sentence. End it at ‘incomes’

L46. Delete ‘regularly’ (redundant)

L50. The use of ‘source’ as a verb is a usage adopted by marketers. Good scientific writing involves precise choice of words, and I recommend using a traditional word such as ‘obtained’ here.

L54-56. Consider rewriting in subject-predicate form, e.g. ‘Shrubs are a key source…’

L68. Check spelling of ‘hindrance’

L69. Delete ‘provision’ (redundant)

L70. Change ‘for’ to ‘of’

L97-100. Instead of a run-on sentence, I recommend you end the sentence on ‘algorithms’. Then make simple declarative statement, e.g. “Machine learning methods improve results…” etc.

L129. It is preferable not to end a paragraph with a list. Can you write a summary sentence?

L160. ‘Perennial’ might be a better word the ‘permanent’ for crops

L176, 224 ‘Pre-processed’ is redundant, just use ‘processed’

L177. Place comma after ‘reflectance’ (parenthetical expressions are placed between commas)

L184. Delete ‘in order’, insert ‘to’

L186. Change ‘less’ to ‘fewer’

L190. It’s unclear whether ‘they’ refers to clouds or artefacts

L217. Check the use of colon here. The second part of the sentence is not grammatical. E.g., put a period there, then start the next sentence “All visible spectrum bands were included…”

L324-327. See previous (L217) comment.

L337. See comment on L40.

L663 and throughout. If using ‘as’ to mean ‘because’, using ‘because’ would be better because it has only one meaning and therefore can be scanned rapidly without confusion.

L775. Use full journal title (do not abbreviate)

Author Response

The authors like to thank the reviewers for carefully reading our manuscript and the helpful and detailed comments, which helped to improve the manuscript with e.g. now the inclusion of a feature selection methodology, and an elaboration of the applications.

The text was rephrased in the respective parts and corrections made. Below we take back the comments made by the reviewers and we respond to each comment one by one.

Please see attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

The article: “Supervised classification of tree cover classes in the complex mosaic landscape of eastern Rwanda” describes the classification of land cover in Easter Rwanda. The manuscript fits the aim and scope of Remote Sensing, but it is one of many other articles about classification. Before publication, more profound insight into feature selection useful for classification should be added. In all models, all layers were used, but is it possible to have a good model using only a few selected features? From a practical point of view, information about recommended features necessary for classifying selected land cover classes is crucial. Also, in my opinion, the title is a misleading term: "classification of tree cover classes" indicates classifications of different types/species of trees, which is not the subject of research presented in the manuscript.

Author Response

The authors like to thank the reviewers for carefully reading our manuscript and the helpful and detailed comments, which helped to improve the manuscript with e.g. now the inclusion of a feature selection methodology, and an elaboration of the applications.

The text was rephrased in the respective parts and corrections made. Below we take back the comments made by the reviewers and we respond to theme one by one.

Please see attachment for more details.

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

The authors did a very nice and thorough job of responding to my comments. I approve the manuscript for publication in Remote Sensing

 

Author Response

Thank you, no additional comments.

Reviewer 2 Report

In general, the authors' work is interesting and worth publication. However, feature selection was added to this work without a deeper understanding. What is the point of 20 different classifications with different feature inputs (don't describe in methodology) and performing feature selection on all features? After revising the methodology (and thus the results and discussion), the article may be considered for publication in the Remote Sensing journal.

Abstract: there is no information about feature selection in the abstract. Also conclusion: “mixed pixels and fragmented tree patches presented challenges for the accurate delineation of some tree cover types. Nonetheless, the methods used in this study were capable of delivering accurate results across the study area” is quite generic.

Introduction:

The authors should add a paragraph about feature selection in this section. More detailed comments you can find below.

L76: 30m - 30 m

L78, L86-89: meters – why not m?

L:100-101: The combination of both multispectral datasets along with vegetation indices and 100

other layers - what other layers are authors referring to?  

L105: why RF abbreviation is after the second use of this term not the first one just the sentence before?

L:107-109: authors write about the pros of RF classification but what about the cons? It also should be added.

 

L:120-134: what about using features selection? In the introduction, the authors do not mention the selection of features in the introduction when describing the methodology, and this should be an important part of the article (according to the comments in the first review). 

Material and methods:

L203, L214, L219, L220: no space before the unit

L241-243: Finally, a principal component analysis (PCA) was conducted on each spectral S2 band for both seasons in order to capture the variability in spectral signals per band and season. From each PCA conducted, the first principal component (PC1) was used as an input layer – according to this text PCA was conducted for each S2 BAND, PCA is conducted for a set of bands not for one band, please clarified PCA calculations. Also why only PC1 was used? If PCA was conducted for the whole S2 product (10 bands in this study) more than one PC might be significant.

L265-297: What images from Google satellite were used in the polygon creation process? How did the time of taking them compare to the S2 images used for classification? A difference in the time of collection can significantly affect the results, so it should be carefully described.

L323-325: Their review? – whom? Please rewrite this sentence.

L330-340: Authors mention of using PCA layers before authors write: “the first principal component (PC1) was used as an input layer” – please clarify the methodology.

L356, L361: please use one term consistently (the best, in my opinion, is feature), and then the text will be clearer.

 

L368, L376: Finally should be used twice in the same paragraph.

Results:

Table 4. – In materials and methods, the authors did not mention classification scenarios using different sets of layers. The methodology should be supplemented with this information, preferably with a description of why such sets were selected.

 

I have a major concern about Feature Selection: it was performed based on P10 model (which as I understand uses all features). Hence, what is the point of all other models in the first place? At this point, I think the authors should extensively revise the methodology before the publication of this interesting study. 

Author Response

Please see the attachment.

Author Response File: Author Response.docx