Using a Machine Learning Approach to Classify the Degree of Forest Management
Round 1
Reviewer 1 Report
In general, the research is interesting and relevant, the structure of the article is clear, the presentation of the research is of high quality and friendly to the reader.
I have only a few comments.
1. In my opinion, the title of the article does not quite correspond to the content, namely, it is not clear what "Predicting the Level" is. Also, the use of the plural "Algorithms" is not clear, since the authors only use one Elastic Net.
2. The use of this algorithm is not sufficiently substantiated. Why Elastic Net? Why did the authors solve the problem of regression and not, for example, classification?
3. The modeling process is poorly described, namely: input and output variables, what error function was used, its value during training?
Author Response
Dear Reviewer,
Thank you for your helpful comments, which we have addressed accordingly.
Please see revised MS in the attachment for full details.
(At the end of the revised MS you will also find the version with all corrections!)
Best for both authors
Andreas Floren
Point by point answer to all reviews.
Review 1:
In general, the research is interesting and relevant, the structure of the article is clear, the presentation of the research is of high quality and friendly to the reader.
I have only a few comments.
- In my opinion, the title of the article does not quite correspond to the content, namely, it is not clear what "Predicting the Level" is. Also, the use of the plural "Algorithms" is not clear, since the authors only use one Elastic Net.
The title has been changed.
- The use of this algorithm is not sufficiently substantiated. Why Elastic Net? Why did the authors solve the problem of regression and not, for example, classification?
Logistic regression is a classification method in statistic and machine learning. Elastic net is a direct generalization of logistic regression, which can deal with many, also correlated features. Please see detailed information in the Material and Methods part.
- The modelling process is poorly described, namely: input and output variables, what error function was used, its value during training?
We added the respective information in the Material and Methods part.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Reviewer(s)' Comments to Author:
In this paper, the authors use machine learning-based methods (elastic net) to identify a beetle signature to distinguish forest types that were managed in different ways. The proposed model identifies a signature of 60 species that allowed trees to be correctly assigned to forest types. The species signature provides new ecological information on how the diversity of primary forests changes as a result of forest management.
In general, the idea of this paper technically makes sense; and the manuscript is easy to follow. Hence, I may suggest a minor revision for this work. However, the following issues should be addressed:
Detailed comments:
1- There is a need to strengthen the Abstract. The author should improve the abstract to include the following in its body: a brief background, brief description of methods and results. It lacks some important details that could make it more informative. For instance, the abstract does not provide any quantitative evaluation metrics.
2- The introduction should briefly indicate the main contributions of this paper in bullets, before the end of the introduction section.
3- The comparative analysis in the discussion section is not fully discussed, so kindly elaborate discussion about comparative analysis.
4- How did the authors set the parameters for the experimental results of the proposed model?
5- The authors did not discuss the limitations of the proposed approach, and the potential future directions for improvement.
6- Language in the whole manuscript requires improvements. There are some grammatical issues throughout the paper. Some grammatical errors and expressions need to be further improved.
7- The references are sufficient, recent and within the field of specialization.
8- References citations and list are not in the required style. Kindly, check the instructions for authors of the Sustainability journal.
English language is fine. some minor editing is required.
Author Response
Point by point answer to all reviews.
Review 2
Reviewer(s)' Comments to Author:
In this paper, the authors use machine learning-based methods (elastic net) to identify a beetle signature to distinguish forest types that were managed in different ways. The proposed model identifies a signature of 60 species that allowed trees to be correctly assigned to forest types. The species signature provides new ecological information on how the diversity of primary forests changes as a result of forest management.
In general, the idea of this paper technically makes sense; and the manuscript is easy to follow. Hence, I may suggest a minor revision for this work. However, the following issues should be addressed:
Detailed comments:
1- There is a need to strengthen the Abstract. The author should improve the abstract to include the following in its body: a brief background, brief description of methods and results. It lacks some important details that could make it more informative. For instance, the abstract does not provide any quantitative evaluation metrics.
We revised the whole abstract accordingly.
2- The introduction should briefly indicate the main contributions of this paper in bullets, before the end of the introduction section.
We restructured and rephrased the last paragraph in the introduction.
3- The comparative analysis in the discussion section is not fully discussed, so kindly elaborate discussion about comparative analysis.
These aspects were addressed in detail in the discussion.
4- How did the authors set the parameters for the experimental results of the proposed model?
We have rephrased the information on setting the model parameters and the loss function in the Methods section.
5- The authors did not discuss the limitations of the proposed approach, and the potential future directions for improvement.
6- Language in the whole manuscript requires improvements. There are some grammatical issues throughout the paper. Some grammatical errors and expressions need to be further improved. 5 and 6: The MS has been revised accordingly.
7- The references are sufficient, recent and within the field of specialization.
8- References citations and list are not in the required style. Kindly, check the instructions for authors of the Sustainability journal.
Comments on the Quality of English Language corrected
English language is fine. some minor editing is required. The language has been revised.
Author Response File: Author Response.pdf
Reviewer 3 Report
General comments:
1) It's common to use square brackets [] instead of parentheses () for citing articles numerically.
2) I noticed a few minor grammar and punctuation issues that I suggest the authors proofread to identify and rectify them.
3) I suggest including an Abbreviations Table right before the references, following the MDPI template. Doing so can enhance the manuscript's readability.
4) The titles for the subfigures appear to be misaligned and aligning them properly would greatly improve the figure's presentation.
5) In Figure 3, I suggest removing the blue background in the ROG curve to enhance visibility, as the gray y=x line is currently not easily distinguishable. Additionally, relocating the length information for this subplot to the bottom right corner would improve the overall clarity and presentation of the figure.
6) For all figures: I highly recommend exporting the figures in 300 DPI (in R or Python) before importing them into the manuscript using Word/LaTeX. To ensure consistency, please use a uniform font size across all figures. Currently, the axis tick labels appear too small, while the axis titles are disproportionately large. It would be beneficial to adjust these sizes for better readability. Moreover, it seems that some figures contain unwanted lines on the sides. I kindly request removing these lines to achieve a cleaner appearance. Additionally, it's regrettable that the figures were initially generated in low quality; however, exporting them at a higher DPI should significantly improve their resolution and overall visual appeal. You might be interested in checking this link: https://www.r-bloggers.com/2013/03/high-resolution-figures-in-r/
Specific comments:
1) I'm pleased to see that cross-validation was utilized to determine the alpha parameter. As we're aware, selecting the appropriate alpha value depends on the particular dataset and the desired level of regularization. Typically, techniques like cross-validation are employed to discover the optimal alpha value, striking the right balance between model complexity and accurate data fitting.
2) While the Receiver Operating Characteristic (ROC) score is a widely used metric for evaluating the performance of classification models, it does have some downsides and limitations. Here are some of the key downsides of using the ROC score:
I. Sensitivity to Class Imbalance: The ROC score can be affected by class imbalance in the dataset, where one class has significantly more samples than the other. In such cases, the ROC curve may not provide an accurate representation of the classifier's performance, especially when the positive class is rare.
II. Insensitivity to Class Probabilities: The ROC score only considers the order of predicted probabilities and not their actual values. It treats all predicted probabilities above a threshold as positive and all below as negative. This can lead to a lack of sensitivity to the actual probabilities assigned to each class, making it less informative about the model's confidence in its predictions.
III. Limited to Binary Classification: The ROC score is designed for binary classification tasks and may not be directly applicable to multi-class classification problems without some modifications, such as one-vs-rest or one-vs-one approaches.
IV. Does Not Consider Decision Threshold: The ROC score does not take into account the specific decision threshold chosen for classifying instances as positive or negative. Different thresholds can lead to different trade-offs between true positive and false positive rates, and the ROC curve does not capture these nuances.
V. Not Suitable for Imbalanced Datasets: In scenarios with imbalanced datasets, where one class is much smaller than the other, the ROC score might give an overly optimistic view of the model's performance, especially if the positive class is rare.
VI. Focuses on Ranking, Not Magnitude: The ROC score primarily evaluates the ranking of predictions rather than the actual magnitudes of predicted probabilities. Consequently, it might not fully capture the true prediction quality when the magnitude of probabilities is essential for decision-making.
It is indeed possible and highly recommended to include additional accuracy metrics for your classification results, such as Precision, Recall, and F1-score. These metrics provide valuable insights into the performance of your classifier beyond just accuracy.
I. Precision: Precision measures the proportion of true positive predictions out of all positive predictions made by the model. It helps assess the model's ability to avoid false positives, making it particularly important when the cost of false positives is high.
II. Recall (Sensitivity or True Positive Rate): Recall measures the proportion of true positive predictions out of all actual positive instances in the dataset. It indicates the model's ability to capture positive instances and is crucial when the cost of false negatives is significant.
III. F1-score: The F1-score is the harmonic mean of Precision and Recall and provides a balanced view of the classifier's performance. It is helpful when you want to strike a balance between Precision and Recall.
3) Including these metrics in addition to accuracy will give a more comprehensive evaluation of your classification model's effectiveness and help you better understand its strengths and weaknesses.
4) I suggest refraining from using the general term "Machine learning" throughout the manuscript. Instead, feel free to include it in the title or keywords, but when discussing specific methodologies, such as ElasticNet, Lasso, and Ridge, please refer to them explicitly.
5) In response to my earlier comment, I recommend revising the discussion section to focus on the actual statistical methods employed instead of using the term "machine learning approach." The discussion should delve into specific methodologies such as ElasticNet, Lasso, and Ridge, providing detailed explanations of how these techniques were applied to the dataset. Furthermore, a complete rewrite of the discussion section is warranted to better align it with the key findings highlighted in the results section through various figures. By overlaying the key points from the results section, the discussion can become more insightful and precise, offering a clearer and more comprehensive interpretation of the study's outcomes. Taking these improvements into account will enhance the clarity and coherence of the discussion section, ensuring that the statistical methods and their implications are conveyed in a more focused and impactful manner.
Comments for author File: Comments.pdf
Upon close examination, I observed a handful of minor grammar and punctuation issues within the text. As a helpful recommendation to the esteemed authors, I propose a thorough proofreading session to accurately identify and promptly rectify these discrepancies. Such meticulous attention to detail will undoubtedly enhance the overall clarity and coherence of the content, ensuring that it is presented in its most polished and professional form.
Author Response
|
Dear Reviewer, Thank you for your helpful comments, which we have addressed accordingly. Please see revised MS in the attachment for full details. (At the end of the revised MS you will also find the version with all corrections!)
Best for both authors Andreas Floren
Point by point answer to all reviews.
Review 3 |
|
|
|
|
General comments:
1) It's common to use square brackets [] instead of parentheses () for citing articles numerically.
MS had been formatted.
2) I noticed a few minor grammar and punctuation issues that I suggest the authors proofread to identify and rectify them.
The language has been revised.
3) I suggest including an Abbreviations Table right before the references, following the MDPI template. Doing so can enhance the manuscript's readability.
We have explained all the abbreviations used in the MS and in the figure captions, which has improved readability. There are only few abbreviations and we have decided not to include a separate table of abbreviations.
4) The titles for the subfigures appear to be misaligned and aligning them properly would greatly improve the figure's presentation.
Corrected as proposed.
5) In Figure 3, I suggest removing the blue background in the ROG curve to enhance visibility, as the gray y=x line is currently not easily distinguishable. Additionally, relocating the length information for this subplot to the bottom right corner would improve the overall clarity and presentation of the figure.
Corrected as proposed.
6) For all figures: I highly recommend exporting the figures in 300 DPI (in R or Python) before importing them into the manuscript using Word/LaTeX. To ensure consistency, please use a uniform font size across all figures. Currently, the axis tick labels appear too small, while the axis titles are disproportionately large. It would be beneficial to adjust these sizes for better readability. Moreover, it seems that some figures contain unwanted lines on the sides. I kindly request removing these lines to achieve a cleaner appearance. Additionally, it's regrettable that the figures were initially generated in low quality; however, exporting them at a higher DPI should significantly improve their resolution and overall visual appeal. You might be interested in checking this link: https://www.r-bloggers.com/2013/03/high-resolution-figures-in-r/
Corrected as proposed.
All figures have a grid for easier comparison. There was actually one figure without a grid, which has been corrected.
Specific comments:
1) I'm pleased to see that cross-validation was utilized to determine the alpha parameter. As we're aware, selecting the appropriate alpha value depends on the particular dataset and the desired level of regularization. Typically, techniques like cross-validation are employed to discover the optimal alpha value, striking the right balance between model complexity and accurate data fitting.
We included the following sentence in the Material part:
“For all analyses the parameter alpha, which choses the mixture of chooses between lasso (alpha = 1) and ridge (alpha = 0) penalty in the elastic net model, was set a priori to alpha = 0.5 to balance between the benefits of both methods”
The regularisation parameter lambda was chosen by cross validation as described in the Material part. Please see Material and Methods part.
Please see revised Material and Methods part
2) While the Receiver Operating Characteristic (ROC) score is a widely used metric for evaluating the performance of classification models, it does have some downsides and limitations. Here are some of the key downsides of using the ROC score:
- Sensitivity to Class Imbalance: The ROC score can be affected by class imbalance in the dataset, where one class has significantly more samples than the other. In such cases, the ROC curve may not provide an accurate representation of the classifier's performance, especially when the positive class is rare.
- Insensitivity to Class Probabilities: The ROC score only considers the order of predicted probabilities and not their actual values. It treats all predicted probabilities above a threshold as positive and all below as negative. This can lead to a lack of sensitivity to the actual probabilities assigned to each class, making it less informative about the model's confidence in its predictions.
III. Limited to Binary Classification: The ROC score is designed for binary classification tasks and may not be directly applicable to multi-class classification problems without some modifications, such as one-vs-rest or one-vs-one approaches.
- Does Not Consider Decision Threshold: The ROC score does not take into account the specific decision threshold chosen for classifying instances as positive or negative. Different thresholds can lead to different trade-offs between true positive and false positive rates, and the ROC curve does not capture these nuances.
- Not Suitable for Imbalanced Datasets: In scenarios with imbalanced datasets, where one class is much smaller than the other, the ROC score might give an overly optimistic view of the model's performance, especially if the positive class is rare.
- Focuses on Ranking, Not Magnitude: The ROC score primarily evaluates the ranking of predictions rather than the actual magnitudes of predicted probabilities. Consequently, it might not fully capture the true prediction quality when the magnitude of probabilities is essential for decision-making.
It is indeed possible and highly recommended to include additional accuracy metrics for your classification results, such as Precision, Recall, and F1-score. These metrics provide valuable insights into the performance of your classifier beyond just accuracy
- Precision: Precision measures the proportion of true positive predictions out of all positive predictions made by the model. It helps assess the model's ability to avoid false positives, making it particularly important when the cost of false positives is high.
- Recall (Sensitivity or True Positive Rate): Recall measures the proportion of true positive predictions out of all actual positive instances in the dataset. It indicates the model's ability to capture positive instances and is crucial when the cost of false negatives is significant.
III. F1-score: The F1-score is the harmonic mean of Precision and Recall and provides a balanced view of the classifier's performance. It is helpful when you want to strike a balance between Precision and Recall.
3) Including these metrics in addition to accuracy will give a more comprehensive evaluation of your classification model's effectiveness and help you better understand its strengths and weaknesses.
Corrected as proposed. We included as an additional performance measure, the F1 score.
4) I suggest refraining from using the general term "Machine learning" throughout the manuscript. Instead, feel free to include it in the title or keywords, but when discussing specific methodologies, such as ElasticNet, Lasso, and Ridge, please refer to them explicitly.
We replaced “Machine learning” by elastic net where we had considered it appropriate.
5) In response to my earlier comment, I recommend revising the discussion section to focus on the actual statistical methods employed instead of using the term "machine learning approach."
The discussion should delve into specific methodologies such as ElasticNet, Lasso, and Ridge, providing detailed explanations of how these techniques were applied to the dataset.
Please find now a detailed description in the Material and Method part. We also have revised the discussion part.
Furthermore, a complete rewrite of the discussion section is warranted to better align it with the key findings highlighted in the results section through various figures. By overlaying the key points from the results section, the discussion can become more insightful and precise, offering a clearer and more comprehensive interpretation of the study's outcomes. Taking these improvements into account will enhance the clarity and coherence of the discussion section, ensuring that the statistical methods and their implications are conveyed in a more focused and impactful manner.
Discussion part was revised.
Author Response File: Author Response.pdf
Round 2
Reviewer 3 Report
Thank you for the revision. The manuscript has improved significantly.
Author Response
Dear reviewer,
also thanks again for the comments that improved the MS
andreas floren
