Multi-Agent Reinforcement Learning for Stand Structure Collaborative Optimization of Pinus yunnanensis Secondary Forests
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsReviewer’s comments to the manuscript “Multi-Agent Reinforcement Learning for Stand Structure Collaborative Optimization of Pinus yunnanensis Secondary Forests in Southwest China " (Authors: Shuai Xuan, JianmingWang, Jiting Yin, Yuling Chen, and BaoguoWu).
The article proposes to investigate the potential and advantages of multi-agent reinforcement learning (MARL) in forest management, offering innovative insights and methodologies for achieving sustainable management of forest ecosystems. The authors formulated the objective function and constraints based on both spatial and non-spatial structural indices of the forest stand structure (FSS) based on the Pinus yunnanensis secondary forests in Southwest China. The value of the objective function (VOF) served as an indicator for assessing FSS. Leveraging the random selection method (RSM) to select harvested trees, we proposed the replanting foreground index (RFI) to enhance replanting optimization. The decision-making processes involved in selection harvest optimization and replanting were modeled as actions within MARL. Through iterative trial-and-error and collaborative strategies, MARL optimized agent actions and collaboration to address the collaborative optimization problem of FSS. The authors conducted optimization experiments for selection felling and replanting across four circular sample plots, comparing MARL with traditional combinatorial optimization (TCO) and single-agent reinforcement learning (SARL). So, the authors present in the study a novel approach to optimizing FSS, contributing to the sustainable management of Pinus yunnanensis secondary forests in southwestern China.
There are some other points to correct or to make the information more exact:
Essential drawbacks.
Remark 1. The novelty of the work is not clear from the article. The authors have already published an article on this topic [https://doi.org/10.3390/f14122456], and most of the material is duplicated in the new article. The previous article was also devoted to reinforcement learning. It is necessary to clearly formulate the novelty and difference from the work already done.
Remark 2. The paper does not describe the software tools with which the authors implemented the proposed algorithms. All algorithms were implemented manually or ready-made libraries were used. There is a significant gap in this part of the article.
Technical drawbacks.
Remark 1. Figures are of poor quality. The font in the pictures is very small and unreadable. It is necessary to increase the font in proportion to the text of the article.
Remark 2. When deciphering the variables included in the equation, “Where” it is used with a capital letter. The style should be: “x*x, where x is …”.
Author Response
Comment1- The novelty of the work is not clear from the article. The authors have already published an article on this topic [https://doi.org/10.3390/f14122456], and most of the material is duplicated in the new article. The previous article was also devoted to reinforcement learning. It is necessary to clearly formulate the novelty and difference from the work already done.
Response1: While our previous research successfully applied single-agent reinforcement learning to the multi-objective optimization of stand structure, it primarily focused on optimizing single cutting measures and did not comprehensively consider the synergistic effects of multiple regulatory measures in model construction and solution. Although single-agent reinforcement learning has advantages in solving single cutting optimization models, it faces challenges when addressing models that require the coordination of multiple measures.
In this study, we have constructed and designed the stand structure optimization model and algorithm by introducing the Replanting Foreground Index, which fully considers the impact of neighboring trees on the growth of replanted trees. We have designed a cutting and replanting synergistic optimization model, which integrates measures such as cutting and replanting. Furthermore, we have incorporated multi-agent reinforcement learning (MARL) to address the complex optimization problems involving multiple coordinated measures. Therefore, it is necessary to further explore the application of MARL in the synergistic optimization of stand structure.
We have clearly formulated the novelty and differences of our current work from our previous study in the revised manuscript.
Comment2- The paper does not describe the software tools with which the authors implemented the proposed algorithms. All algorithms were implemented manually or ready-made libraries were used. There is a significant gap in this part of the article.
Response2: We utilized R4.2.0 to calculate the relevant indices and Python to develop the main experimental program. All algorithms were manually implemented without the use of ready-made libraries. We have now included this information in the revised manuscript’s 2.4. Stand Structure Indexes and 2.7.5. Parameter Settings to address this significant gap.
Comment3- Figures are of poor quality. The font in the pictures is very small and unreadable. It is necessary to increase the font in proportion to the text of the article.
Response3: Regarding the image quality issue you mentioned, we have made the necessary improvements, including increasing the image resolution and enlarging the font size. These changes ensure that the images are clearer and easier to read.
Comment4- When deciphering the variables included in the equation, “Where” it is used with a capital letter. The style should be: “x*x, where x is …”.
Response4: We have addressed this issue to ensure the format is more appropriate. When explaining the variables in the formula, we now use “Eq:” at the beginning of the sentence and “where” within the sentence.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsA Review on a research paper entitled
Multi-Agent Reinforcement Learning for Stand Structure Collaborative Optimization of Pinus yunnanensis Secondary Forests in Southwest China (ID: forests-3042595)
The present study explores the application of multi-agent reinforcement learning (MARL) to managing forest ecosystems, specifically targeting the Pinus yunnanensis secondary forests in Southwest China. The research aims to enhance sustainable forest management practices by integrating innovative insights and methodologies. The authors framed an objective function and constraints based on both spatial and non-spatial structural indices of the forest stand structure (FSS), with the value of the objective function (VOF) serving as an indicator for assessing FSS.
The study used a random selection method (RSM) to select harvested trees and introduced the replanting foreground index (RFI) to optimize replanting processes. The decision-making for selection harvest optimization and replanting was modeled as actions within a MARL framework. The MARL agents engaged in iterative trial-and-error and collaborative strategies to optimize actions and address the collaborative optimization problem of FSS. Optimization experiments were conducted across four circular sample plots (P1, P2, P3, and P4), comparing the performance of MARL with traditional combinatorial optimization (TCO) and single-agent reinforcement learning (SARL). The experiments focused on selection felling and replanting optimization, with performance assessed based on maximum VOF improvements.
Reviewer’s Verdict:
The study presents a novel and effective approach to optimizing forest stand structure through the application of MARL. The superior performance of MARL in both selection harvesting and replanting optimization emphasizes its potential in advancing sustainable management practices for Pinus yunnanensis secondary forests in Southwest China. By achieving significant improvements in VOF across various sample plots, MARL proves to be a promising tool for enhancing forest ecosystem management. Further, the successful application of MARL in this study suggests several broader implications, including, scalability, collaboration, and sustainability.
The above aspects impressed me a lot and I appreciate the efforts of the authors.
Nevertheless, there are a few minor issues that need authors’ attention. I request the authors to respond to them, then, I will decide the fate of this paper.
Minor Issues:
a) The title looks too lengthy. Please curtail it or write in a refined manner.
b) Once an acronym is expanded no need to repeat it. I found such instances several times in this manuscript (Ex: RFI, MARL, FSS, SARL). Try to avoid them.
c) A few Figures are low in resolution, particularly, Figures 6 and 9. The font sizes are too low to be read. Kindly enhance them.
d) Section 2- Materials and Methods, 2.1 Study Areas: ‘This geographical area falls within the subtropical climate belt, characterized by an average annual temperature of approximately 15°C and influenced by prevailing southwest monsoon winds’, which needs a reference.
e) The Future Research section might enhance the gravity of this research. I request the authors write a few lines about future research in the modified manuscript.
f) I also wish to see the role played by the GIS when it is integrated with MARL. I request the authors to discuss that combination in a few lines at the appropriate place.
g) In my personal opinion, the paper is too lengthy, and it lacks lots of patience to read. Kindly look into this critical issue, if the authors concur with my opinion.
Author Response
Comment1- The title looks too lengthy. Please curtail it or write in a refined manner.
Response1: We have simplified the title while ensuring it accurately summarizes the content of the full text.
Comment2- Once an acronym is expanded no need to repeat it. I found such instances several times in this manuscript (Ex: RFI, MARL, FSS, SARL). Try to avoid them.
Response2: We have carefully reviewed and revised the manuscript to explain acronyms only at their first appearance in the abstract and body. We believe this approach ensures that acronyms are not repeated after their initial expansion, allowing for smoother reading.
Comment3- A few Figures are low in resolution, particularly, Figures 6 and 9. The font sizes are too low to be read. Kindly enhance them.
Response3: Regarding the image quality issue you mentioned, we have made the necessary improvements, including increasing the image resolution and enlarging the font size. These changes ensure that the images are clearer and easier to read.
Comment4- Section 2- Materials and Methods, 2.1 Study Areas: ‘This geographical area falls within the subtropical climate belt, characterized by an average annual temperature of approximately 15°C and influenced by prevailing southwest monsoon winds’, which needs a reference.
Response4: In the revised manuscript, we have added references to this section to enhance its scientific accuracy and reliability.
Comment5- The Future Research section might enhance the gravity of this research. I request the authors write a few lines about future research in the modified manuscript.
Response5: We have added content about future research to the Conclusion section to enhance the depth and breadth of this research.
Comment6- I also wish to see the role played by the GIS when it is integrated with MARL. I request the authors to discuss that combination in a few lines at the appropriate place.
Response6: After an extensive review of relevant literature and internal team discussions, we have concluded that integrating GIS with MARL is essential for improving the accuracy and efficiency of MARL algorithm decision-making, given the detailed spatial information that GIS can provide. This integration will be a key focus for future research. We have added the relevant discussion in the revised Conclusion section.
Comment7- In my personal opinion, the paper is too lengthy, and it lacks lots of patience to read. Kindly look into this critical issue, if the authors concur with my opinion.
Response7: We appreciate your feedback regarding the length of the manuscript. We will carefully assess the content and aim to streamline the text by relocating non-essential details to the supplementary materials. This adjustment will enhance the overall readability and maintain readers' interest.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThe authors took into account the comments on the article; there are no new comments.
