Multi-Agent Reinforcement Learning for Stand Structure Collaborative Optimization of Pinus yunnanensis Secondary Forests

Abstract

This study aims to investigate the potential and advantages of multi-agent reinforcement learning (MARL) in forest management, offering innovative insights and methodologies for achieving sustainable management of forest ecosystems. Focusing on the Pinus yunnanensis secondary forests in Southwest China, we formulated the objective function and constraints based on both spatial and non-spatial structural indices of the forest stand structure (FSS). The value of the objective function (VOF) served as an indicator for assessing FSS. Leveraging the random selection method (RSM) to select harvested trees, we propose the replanting foreground index (RFI) to enhance replanting optimization. The decision-making processes involved in selection harvest optimization and replanting were modeled as actions within MARL. Through iterative trial-and-error and collaborative strategies, MARL optimized agent actions and collaboration to address the collaborative optimization problem of FSS. We conducted optimization experiments for selection felling and replanting across four circular sample plots, comparing MARL with traditional combinatorial optimization (TCO) and single-agent reinforcement learning (SARL). The findings illustrate the superior practical efficacy of MARL in collaborative optimization of FSS. Specifically, replanting optimization based on RFI outperformed the classical maximum Delaunay generator area method (MDGAM). Across different plots (P1, P2, P3, and P4), MARL consistently improved the maximum VOFs by 54.87%, 88.86%, 41.34%, and 22.55%, respectively, surpassing those of the TCO (38.81%, 70.04%, 41.23%, and 18.73%) and SARL (54.38%, 70.04%, 41.23%, and 18.73%) schemes. The RFI demonstrated superior performance in replanting optimization experiments, emphasizing the importance of considering neighboring trees’ influence on growth space and replanting potential. Following selective logging and replanting adjustments, the FSS of each sample site exhibited varying degrees of improvement. MARL consistently achieved maximum VOFs across different sites, underscoring its superior performance in collaborative optimization of logging and replanting within FSS. This study presents a novel approach to optimizing FSS, contributing to the sustainable management of Pinus yunnanensis secondary forests in southwestern China.

1. Introduction

Secondary forests generally exhibit issues such as unsustainable forest stand structures (FSSs), diminished biodiversity, heightened vulnerability to forest fires, and susceptibility to natural calamities such as pest infestations, diseases, and wildfires [1,2,3]. The optimization and adjustment of FSS represent pivotal technical interventions in forest management and planning, offering indispensable strategies for managing secondary forests effectively [4,5]. Central to this endeavor is the meticulous determination of FSS indices, the formulation of optimization models, and the design of solution algorithms. Initially, the optimization model is crafted through the judicious selection of FSS indexes, informed by the unique characteristics of FSS. Subsequently, relevant algorithms are deployed to resolve the optimization model, thereby facilitating management optimization through a spectrum of adjustment measures including selective cutting, replanting, tending, and pruning. FSS indices encompass both spatial and non-spatial dimensions, encompassing non-spatial metrics such as diameter scales counts [6], species counts, and plant density [7], alongside spatial indices including the mingling index (M) [8,9], canopy competition index (CI) [10,11], angle index (W) [12,13], story index (S) [14], open comparison (OP) [15], and neighborhood comparison (U) [16].

Selective cutting emerges as a pivotal strategy in the optimization of FSS. This approach entails the targeted removal of trees with limited growth potential, thereby orchestrating a more conducive distribution pattern of trees, enhancing understory light conditions, and alleviating competitive pressures within the forest milieu, ultimately culminating in FSS optimization. However, the efficacy of singular selective cutting endeavors often falls short in achieving the desired optimization outcomes. Complementarily, replanting constitutes another indispensable mechanism for optimization and regulation, endeavoring to bolster the stability and biodiversity of forest ecosystems through the strategic introduction of young trees of indigenous species in judiciously chosen spatial domains. The crux of replanting optimization resides in the discernment of pivotal indicators such as location and species.

Conventional replanting methodologies typically rely on techniques like the Voronoi diagram [17] or Delaunay triangulation [18], Kriging interpolation [19], among others, for determining replanting locations. However, these methodologies often overlook the intricate interplay between replanted trees and their neighboring counterparts, resulting in rigid replanting locations and potentially exacerbating inter-tree competitive pressures. Moreover, the scholarly discourse surrounding the optimization of stand structure based on replanting strategies remains relatively sparse. Existing studies predominantly rely on statistical analyses to ascertain replanting positions and pertinent information pertaining to the replanted flora [17,18,19], with scant attention devoted to replanting research underpinned by intelligent algorithms.

Numerous scholars have devised optimization and adjustment models, such as multi-objective operation and spatial structure, building upon the aforementioned optimization and adjustment strategies [17,20,21,22]. These models typically serve to simulate and optimize individual interventions, such as selective cutting or replanting, or iteratively refine stand structure through a sequence of actions, notably selective cutting followed by replanting (note: specific references are provided for sequential adjustments). However, none of the aforementioned optimization frameworks have comprehensively considered the collaborative synergies among multiple adjustment measures during model formulation and solution. Particularly within the realm of replanting adjustments, addressing complex multidimensional information encompassing spatial coordinates of replanted trees, tree species, age, height, and related parameters poses a significant challenge. Integrating these multidimensional attributes within the model and devising solution algorithms to capture their nuances are essential for leveraging the synergistic effects of multiple measures in optimizing and controlling forest stand structure. Notably, in scenarios typified by secondary forests dominated by a single species and characterized by extensive forest cover, a holistic approach encompassing replanting, tending, and additional measures becomes imperative to enhance spatial segregation within the forest stand and optimize pertinent indices.

The optimization of FSS is a nonlinear multi-objective optimization problem [23,24,25]. Existing literature predominantly explores algorithms for solving FSS optimization models grounded in single measures. These encompass heuristic methodologies like Monte Carlo (MC) [26,27,28,29] and bionic algorithms such as genetic algorithms (GAs) [30,31,32], simulated annealing (SA) [33,34,35], and particle swarm optimization (PSO) [17,36]. While MC offers simplicity and strong programmability, its outcomes often lack precision due to inherent algorithmic limitations. Conversely, bionic algorithms like GAs and PSO frequently encounter challenges such as local optimization pitfalls and volatility in the values of the objective function (VOFs) during solution processes.

Reinforcement learning (RL), characterized by trial-and-error strategies, emerges as a promising intelligent algorithm owing to its conceptual simplicity, absence of a model, dynamic decision-making, and robust adaptability [37]. In a previous study, we addressed the multi-objective logging optimization problem for FSS using single-agent reinforcement learning (SARL), which excelled in single-measure multi-objective optimization. However, FSS optimization necessitates the integration of multiple measures beyond selective logging alone. Moreover, SARL is constrained by individual knowledge and experience, often requiring prolonged training periods to adapt to environments and learned strategies, thus posing challenges in balancing exploration and exploitation.

Multi-agent reinforcement learning (MARL) presents a novel approach by leveraging multiple agents to collaboratively address complex optimization problems. Each agent interacts with the environment, receiving rewards and adapting in response to the behaviors of other agents. However, practical applications of MARL encounter challenges concerning the balance between collaboration and competition among agents, as well as the intricate design of reward functions. Despite its prominence in fields like autonomous driving and smart grids, MARL remains largely unexplored in collaborative optimization of FSS.

Overall, the construction and design of FSS optimization models and algorithms necessitate a comprehensive consideration of measures such as selective cutting and replanting, effective representation of multidimensional tree characteristics during replanting regulation, and a thorough integration of neighboring tree influences on replanted tree growth. Synchronized optimization and adjustment simulations are imperative for solving the optimal configuration of FSS. While SARL offers advantages over traditional heuristic and bionic algorithms in addressing single-measure optimization models, it proves inadequate for addressing synergistic optimization models involving multiple measures. Hence, this study introduces the replanting foreground index (RFI), a logging and replanting collaborative optimization model, and a MARL to address the collaborative optimization problem of FSS based on Pinus yunnanensis secondary forest data in Yunnan Province, China. This elucidates the potential and advantages of the multi-agent approach in forest management, providing novel insights and methodologies for sustainable forest ecosystem management.

2. Materials and Methods

2.1. Study Areas

The forest inventories were undertaken within the geographical vicinity of Cangshan Mountain, located between longitude ${99}^{\circ }{55}^{\prime }$–${100}^{\circ }{12}^{\prime }$ E and latitude ${25}^{\circ }{34}^{\prime }$–${26}^{\circ }{00}^{\prime }$ N, situated in Yunnan Province, China (Figure 1). The surveyed regions predominantly situated on the eastern slope of Cangshan Mountain, include prominent features such as Lan Peak, Malong Peak, Foding Peak, and Zhonghe Peak. This geographical area lies within the subtropical climate belt, with an average annual temperature of around 15 °C, influenced by prevailing southwest monsoon winds [38,39]. Notably, the region experiences abundant annual precipitation exceeding 1000 mm, with distinct seasonal variations marked by pronounced dry spells interspersed with heavy rainfall. The wet season predominantly spans from May to October, accounting for approximately 84% of the total annual precipitation [40]. Predominantly, the area is characterized by red soil, contributing to its unique ecological landscape.

2.2. Study Site and Data Collection

Table 1. Essential Details of Sample Plot Characteristics.

Sample Plots	East Long.	North Lat.	Elevation (m)	Slope (∘)	Slope Dir.	Sample Plot Radius (m)	Tree Species Composition	Stand Density (trees/ha)
P1	100°08.2149″	25°41.5280″	2254	13.45	East	35	8 PY-2 PA-BA-TG	1481
P2	100°10.9639″	25°38.1518″	2271	16.15	South	32	7 PY-3 PA	1822
P3	100°09.3947″	25°39.9506″	2195	17.70	NE	20	7 PY-3 PA-QAC-VBT-GGW-BA	1830
P4	100°07.1906″	25°43.5923″	2138	5.10	NE	19	10 PY-QAC	1975

Note: NE, North-East; PY, Pinus yunnanensis; PA, Pinus armandii; QAC, Quercus acutissima Carrut; VBT, Vaccinium bracteatum Thunb; GGW, Gaultheria griffithiana Wight; BA, Betula alnoide; TG, Ternstroemia gymnanthera. The column ‘Tree Species Composition’ delineates the distribution of tree species within each plot. The numerical values preceding each species denote the relative abundance of that particular species for every 10 trees sampled within the plot.

presents key statistical data pertaining to the sample plots utilized in this study. The primary focus of the investigation centered on the dominant tree species, Pinus yunnanensis, spanning four circular sample plots, each strategically positioned atop distinct peaks. These circular sample plots boasted radii measuring 19 m (Lan Peak), 20 m (Malong Peak), 32 m (Foding Peak), and 35 m (Zhonghe Peak).

Between July and December 2022, comprehensive forestry operations were executed within the delineated sample plots. This phase involved meticulous measurements and recording of geographical coordinates, elevation, slope, slope direction, and plot radii. Furthermore, a systematic survey was conducted encompassing live standing trees with a diameter at breast height (DBH) equal to or exceeding 5 cm (DBH ≥ 5 cm) within the sample plots. Each tree underwent a thorough assessment, documenting essential forestry attributes including species, DBH, tree height (TH), crown width (CW), and crown length (CL), facilitated by specialized altimeters and distance measurer. Using a Topcon GTS-2002 autofocus total station (Topcon, Tokyo, Japan), we precisely determined the relative coordinates of each tree’s base with respect to the center of the sample plots.

2.3. Determination of Spatial Structure Units and Edge Correction

The Voronoi method was used to delineate spatial structure units within the forest stands [17]. This method constructs Voronoi diagrams based on the measured relative positions of individual trees, creating polygons that represent spatial structure units for each tree and its neighboring trees [41,42].

However, the spatial structure units at the edges of stands may be affected by sample boundaries, potentially leading to errors in calculating spatial structure indices. To address this issue, the study utilized the buffer zone method [21,43]. For example, for a circular plot with a diameter of a meters, the buffer zone is created by extending b meters inward from the edge of the plot towards the center. In other words, a m ring area is used as the buffer zone [44]. When calculating the spatial structure index for a unit composed of a central tree and its adjacent trees, the trees within the buffer area are only considered as adjacent trees in forming the spatial structure unit.

The determination of the buffer zone width depends on several factors, including plot size, methods for analyzing stand structure indices, and geographical location. This study carefully considered these factors and, based on existing research and experience [45], set the buffer zone width at 2 m.

2.4. Stand Structure Indexes

Quantifying stand structure is essential for optimizing FSS. In this study, the number of tree diameter classes, the diversity of tree species, cutting intensity and plant density were selected as indicators to quantify the non-spatial structure of the stand. To assess the spatial structure of the forest stand, indicators such as uniform angle index, mingling index, crown competition index, story index, and open comparison were utilized.

2.4.1. Non-Spatial Structural Indexes

(1)

Tree Diameter Classes [6]

Categorizing trees into diameter classes is crucial for our analysis, as it correlates directly with stand growth. In this study, we categorized trees based on their DBH, starting from 6 cm and increasing in 2 cm increments. This systematic approach ensured consistency in the number of diameter classes throughout FSS optimization:

$$D=D_0$$

(1)

$D_0$ denotes the count of diameter classes before selection cutting, and D signifies the count post-selection cutting.

(2)

Diversity of Tree Species

During the selection cutting process, it is crucial to preserve tree species diversity to prevent unintentional extinction. We diligently ensured that the count of tree species remained unchanged throughout the process:

$$T=T_0$$

(2)

$T_0$ signifies the initial count of tree species, and T denotes the count post-selection cutting.

(3)

Cutting Intensity

The vigor of the stand after optimization relies on the cutting intensity. Ideally, the annual cutting volume should not exceed the annual growth rate of the stand. Previous research [46,47] has indicated that the ideal cutting intensity for secondary forests of Pinus yunnanensis should be limited to a maximum of 35%:

$$N\ge N_0\left(1-35\%\right)$$

(3)

$N_0$ signifies the total count of trees before selection cutting, and N represents the count after selection cutting.

(4)

Plant Density ($PD$) [7]

$PD$ is the key factor influencing the replanting effect. Previous studies [48] showed that the reasonable range of $PD$ of Pinus yunnanensis is 1667∼3333 trees/hm². After replanting optimization, the $PD$ of the sample plots should be within the range of [1667, 3333]:

$$1667\le PD\le 3333$$

(4)

2.4.2. Spatial Structural Indexes

(1)

M

The spatial segregation of tree species, represented by M, is calculated as the ratio of neighboring tree j stem count, excluding the same species as the object tree i, to the total neighboring tree stem count. It is mathematically expressed as [8,9]:

$$M_i=\frac{1}{n}\sum _{j=1}^n{v}_{ij}$$

(5)

$M_i$ symbolizes the mingling index of the object tree i, and $v_{ij}$ is a discrete variable. When the neighboring tree j is not of the same species as tree i, $v_{ij}=1$; otherwise, $v_{ij}=0$.

(2)

$CI$

To measure the competitive pressures among trees, we designed a $CI$ that employs the overlap area of canopies. The index is calculated as follows [10,11,44]:

$$C{I}_i=\frac{1}{Z_i}\times \sum _{j=1}^{n}A{O}_{ij}\times \frac{L_j}{L_i}$$

(6)

$C{I}_i$ represents the canopy competition index for object tree i, $Z_i$ denotes the projected canopy area of object tree i. $L_i=H_i$ × $C{W}_i$ × $C{L}_i$, where $H_i$, $C{W}_i$ and $C{L}_i$ represent the height, canopy width, and canopy length of the object tree i, respectively. $L_j=H_j$ × $C{W}_j$ × $C{L}_j$, where $H_j$, $C{W}_j$ and $C{L}_j$ denote the height, canopy width, and canopy length of the competing tree i, respectively. $A{O}_{ij}$ indicates the overlap area between the canopies of object tree i and competitor tree j, with $A{O}_{ij}=1$ when there is no overlap.

(3)

W

The characterization of the stand’s horizontal distribution pattern is represented by the index W. This index measures the proportion of angles $\alpha$ between the object tree i and its nearest neighbors that are smaller than a predefined standard angle ${\alpha }_0$. It is mathematically expressed as [12,13]:

$$W_i=\frac{1}{n}\sum _{j=1}^n{z}_{ij}$$

(7)

$W_i$ denotes the angle index of tree i, and $z_{ij}$ is a discrete variable. If the angle $\alpha$ between trees i and j is less than ${\alpha }_0$, $z_{ij}=1$; otherwise, $z_{ij}=0$.

(4)

S

The vertical diversity and complexity within a stand are encapsulated by the index denoted as S, which quantifies the proportion of neighboring trees j at the same height level as the object tree i. It is calculated as follows [14]:

$$S_i=\frac{1}{n}\sum _{j=1}^n{v}_{ij}$$

(8)

$S_i$ represents the story index of tree i, and $v_{ij}$ is a binary variable. If tree i shares the same height level as tree j, $v_{ij}=0$; otherwise, $v_{ij}=1$.

(5)

$OP$

The open comparison represents the light environment and available growing space for tall trees within the stand. This index measures the degree to which the object tree i within a spatial structure unit is overshadowed by its neighboring tree j. This index is represented as follows [15]:

$$O{P}_i=\frac{1}{n}\sum _{j=1}^n{t}_{ij}$$

(9)

$O{P}_i$ is the open comparison corresponding to object tree i, $t_{ij}$ is a discrete variable. If the horizontal distance between object tree i and neighboring tree j exceeds their height difference, $t_{ij}=1$; contrarily, $t_{ij}=0$.

(6)

U

The size differentiation in the diameter of individual forest trees is described by the proportion of neighboring trees j nearest to object tree i that are larger than object tree i among the n neighboring trees. This comparison was calculated using the formula [16]:

$$U_i=\frac{1}{n}\sum _{j=1}^n{k}_{ij}$$

(10)

$U_i$ is neighborhood comparison of object tree i, $k_{ij}$ is characterized as a discrete variable. When the DBH of neighboring tree j is larger than that of object tree i, $k_{ij}$ = 1; otherwise, $k_{ij}$ = 0.

The aforementioned indices were calculated and analyzed using data from the stand survey with R version 4.2.0.

2.5. Methods for Selecting Trees for Cutting

The RSM (random selection method) [24,44], QVM (Q-value method) [49], and VMM (V-map method) [50] are widely employed techniques for selecting trees for cutting. RSM identifies trees for removal through rapid random sampling from the initial pool of retained trees (comprising all trees within the original stand), and ensuring selection intensity remains within bounds. QVM constructs a single-tree composite index $Q_i$ using five spatial structural parameters (W, M, $CI$, $OP$, and S), then ranks these $Q_i$ values within the harvesting limit to determine the trees to be harvested. Previous research [49] suggests that the probability of FSS achieving optimization is higher when trees corresponding to the top $Q_i$ values are felled. Under ideal conditions, the average angle index of the stand ($\overline{W}$) falls within the range [0.475, 0.517], with a central value of 0.496. In VMM, the initial selection of structural units for harvesting prioritizes the nearest neighboring trees of the reference tree with the highest W value (0.496). These neighboring trees are then used to identify trees for felling, with particular focus on those exhibiting weak mixing, moderate mixing, and suppressed competition based on their M values.

In our prior study, we conducted experimental comparisons among these three selection methods, revealing the RSM as the optimal choice for optimizing stand structure when integrated with RL algorithm [44]. Consequently, the RSM has been selected as the preferred method for tree selection in the cutting optimization segment of this research.

2.6. RFI

2.6.1. Number of Replanting Trees and Species Configuration

The literature suggests that the optimal planting density for Pinus yunnanensis ranges from 1667 to 3333 trees per hectare. In the study, we set the upper limit of planting density for the sample plots after replanting to 3333 trees/hm². Considering various sample plot sizes, we rounded down to determine the upper limit of tree numbers in P1, P2, P3, and P4, resulting in 1282, 1072, 418, and 378 trees, respectively.

In mixed forests, different proportions of replanting species may lead to varying degrees of species segregation, while differences in DBH, H (tree height), CW (crown width), and CL (crown length) of replanted trees may affect the competitive relationships among neighboring trees post-replanting. The four sample plots in this study were secondary forests dominated by Pinus yunnanensis, with other native or companion species present, including Pinus armandii, Vaccinium bracteatum, Quercus acutissima, Betula alnoides, Gaultheria griffithiana, and Ternstroemia gymnanthera. We set the average DBH of replanted trees at 5 cm, and trees within the [5, 6) cm DBH interval were used to calculate an average H of 4.79 m, an average CW of 6.05 m, and an average CL of 1.71 m, which determined the size of replanted trees.

2.6.2. Maximum Delaunay Generator Area Method (MDGAM)

Ideal replanting locations are usually in relatively wide “forest gaps”. Traditional methods for pinpointing these sites include the MDGAM [18], maximum null circle method (MNCM) [17], and Kriging method [19]. These methods assess various areas to determine the optimal replanting location rooted in maximal area calculation. The MDGAM, based on Delaunay triangulation, represents each tree as a node and the distances between neighboring trees as side lengths. This approach effectively captures both the “forest gaps” and the forest stand’s distribution pattern [51]. In contrast, the MNCM and Kriging method lack the Delaunay triangulation’s characteristics and cannot adequately consider the forest stand’s distribution pattern, thereby presenting significant limitations in determining the replanting location. Consequently, this study relied on the generating element area of Delaunay triangulation as the primary criterion for determining replanting locations.

2.6.3. RFI

Replanted trees establish a new spatial structure within the stand alongside their neighboring trees. While the basic MDGAM primarily considers growth space and stand distribution patterns, merely positioning replanting sites within the maximal “forest gaps” overlooks the impact of neighboring trees on the growth of newly planted ones. This oversight may exacerbate competitive relationships within the stand. It becomes imperative to factor in spatial relative coordinates, tree species, tree age, tree height, and articulate these multidimensional characteristics within the replanting optimization model and solution algorithm.

Therefore, this study introduces a novel RFI to tackle these issues. Leveraging Delaunay triangulation, the RFI characterizes the relative coordinates of replanted trees and their neighbors, the mingling index, the impact of replanting species on replanting efficacy, DBH in neighborhood comparison to denote age, and the crown competition index to consider tree crown and height influences. The specific formula for the RFI is outlined as follows:

$$RF{I}_i=\frac{\frac{1+DA{A}_i}{{\delta }_{DAA}}·\frac{1+M_i}{{\delta }_M}·\frac{1+U_i}{{\delta }_U}}{\frac{1+C{I}_i}{{\delta }_{CI}}}$$

(11)

$RF{I}_i$ represents the replanting prospect index of the tree to be replanted i, $DA{A}_i$ denotes the area of the Delaunay triangulation generation element where tree i is situated, $M_i$ signifies the mingling index of tree i, $U_i$ denotes the neighborhood comparison of tree i, and $C{I}_i$ represents the canopy competition index of tree i. Additionally, ${\delta }_{DAA}$, ${\delta }_M$, ${\delta }_U$, and ${\delta }_{CI}$ denote the standard deviation of each structural parameter.

By introducing the RFI, we comprehensively consider the impacts of various factors on the growth of replant trees, including growth space, mingling index, neighborhood comparison, and canopy competition index. This integrated index enhances the accuracy and effectiveness of determining replanting locations. During replanting operations, priority is assigned to locations with higher RFIs, thereby enhancing the efficiency of stand adjustment and optimization.

2.7. Forest Stand Structure Optimization Model

FSS optimization represents a multi-objective optimization challenge [17,24,44], where the optimization model aligns with predefined objectives and constraints. This model entails selective logging and replanting strategies to enhance the overall FSS. Constructing an objective function in accordance with optimization goals is pivotal for effective FSS optimization.

2.7.1. Objective Function

Solving a multi-objective optimization problem involves finding the optimal solution considering multiple objectives while adhering to constraints [52,53]. Typically, these objectives are interconnected and constrained, making it challenging to achieve optimal solutions for each individual objective. Therefore, it’s crucial to integrate and synthesize multiple sub-objectives into an overall objective function to find the optimal solution.

When optimizing FSS, a higher VOF signifies a better spatial structure for the forest stand. This study employed the concept of “multiplication and division” [54] to select and combine five spatial structure indices: angle index, mingling index, crown competition index, story index, and open comparison into the objective function for the multi-objective optimization model of FSS. This objective function was calculated using the formula:

$$maxOF=\frac{1}{N}\sum _{i=1}^N\frac{\frac{1+M_i}{{\delta }_M}·\frac{1+O{P}_i}{{\delta }_{OP}}·\frac{1+S_i}{{\delta }_S}}{\frac{1+C{I}_i}{{\delta }_{CI}}·\frac{1+\left|W_i-0.496\right|}{{\delta }_{\left|W-0.496\right|}}}$$

(12)

$M_i$, $O{P}_i$, $S_i$, $C{I}_i$, and $W_i$ denote the mingling index, open comparison, story index, canopy competition, and uniform angle index of the central tree i, respectively. Additionally, ${\delta }_M$, ${\delta }_{OP}$, ${\delta }_S$, ${\delta }_{CI}$, and ${\delta }_W$ represent the standard deviations of their respective structural parameters. The midpoint of the range [0.475, 0.517] is 0.496, where a smaller value of $\left|W_i-0.496\right|$ indicates that the forest stand’s horizontal distribution pattern is closer to randomness.

To ensure consistency in model evaluation and comparison, and effectively explore performance differences among different methods under optimization objectives, the same objective function was employed for all optimization models in this study.

2.7.2. Traditional Combinatorial Optimization (TCO) Model

Achieving effective optimization of FSS often requires the collaboration of multiple optimization measures, such as selective cutting and replanting strategies. The traditional logging and replanting combinatorial optimization model serves as a common approach for optimizing FSS, involving a dual optimization process. Initially, a portion of the trees within the sample plot is selectively cut according to a predefined harvesting strategy to attain a relatively optimal FSS. Subsequently, native tree species are replanted at suitable locations within the plot post-harvesting, aiming for a double optimization of the FSS.

In a previous study [44], simulated selective logging experiments on four sample plots yielded relatively optimal FSS outcomes. Building upon these findings, replanting optimization experiments were conducted, based on the improved FSS after selective cutting in each sample plot, to assess the practical application of the logging-first-then-replanting optimization concept.

The optimization model is pivotal for FSS optimization, encompassing the objective function, constraints, and other factors. The constraints of the TCO model are outlined as follows:

(1)

Constraints

Following optimization and adjustments, the quality of each sub-objective should not deteriorate compared to its pre-optimization state, ensuring that the spatial structure diversity of the forest stand remains intact. This necessitates a closer-to-random horizontal distribution pattern and an enhanced mixing degree. During replanting, the plant density within the sample plot must be maintained within a reasonable range. Research findings [48,55,56] suggest that the suitable planting density for Pinus yunnanensis ranges between 1667∼3333 trees/hm². The constraints of the TCO model are formulated as follows:

$$s.t.\left\{\begin{array}{c}\left|\overline{W_2}-0.496\right|\le \left|\overline{W_1}-0.496\right|\\ \overline{M_2}\ge \overline{M_1}\\ 1667\le PD\le 3333\end{array}\right.$$

(13)

$\overline{W_1}$ and $\overline{M_1}$ represent the average values of the angle index and mingling index of the stand after selective logging, respectively. $\overline{W_2}$ and $\overline{M_2}$ denote the average values of each parameter after replanting, and $PD$ indicates the plant density of the stand in the sample plot after replanting optimization.

(2)

Solving Algorithm

Based on the findings of prior research [44], this study derived the FSS of each sample plot following selection cutting optimization via RL. Then, the RFI was employed to designate the replanting locations. Considering the predetermined upper limit for the number of replanting trees in each plot, tree species for replanting were allocated in equal proportions, and the dimensions of individual replanted trees were determined accordingly. This process culminated in the compilation of the replanting tree set. Various quantities of trees, ranging from 0 to the predefined upper limit, were selected for replanting from this set. Consequently, their corresponding structural indices for each forest stand were calculated. Figure 2 depicts the flowchart of the TCO process, while the algorithmic pseudocode is elaborated in Appendix A.1.

2.7.3. Collaborative Optimization Model

The collaborative optimization model for logging and replanting aims to optimize the FSS through replanting while adhering to the objectives and constraints of selection cutting, thereby achieving dual optimization of the FSS. This model emphasizes real-time collaboration between logging and replanting, enhancing flexibility and adaptability to varying site conditions. RL presents advantages in addressing dynamic optimization problems like this, yet the performance of SARL and MARL differs significantly in collaborative optimization scenarios. SARL, constrained by individual knowledge and experience, necessitates extensive training time for complex problems, whereas MARL, leveraging collaboration among agents, encounters challenges in balancing cooperation and competition and designing intricate reward functions. In this study, the objective function of the collaborative optimization model aligns with that of the TCO model, facilitating effective comparison across different optimization approaches. The constraints of the collaborative optimization model are detailed as follows:

(1)

Constraints

During the selection cutting process, it is imperative to ensure that the horizontal distribution pattern of the forest stand approaches random distribution, enhancing the mingling index, reducing stand competition, promoting vertical diversity, and increasing openness and light penetration. Regarding non-spatial structure constraints, it is essential to maintain the number of diameter classes and tree species in the stand during selective logging optimization, with the cutting intensity not exceeding 35%. Replanting optimization constraints are the same as those of the TCO model’s replanting optimization segment. In the collaborative optimization model, the VOF post-selective logging $F_1$ must exceed the initial VOF $F_0$, while the VOF post-replanting $F_2$ must surpass $F_1$. A higher $F_2$ indicates superior structure in the cooperative optimization. In summary, the constraints of the collaborative optimization model are expressed as follows:

$$s.t.\left\{\begin{array}{c}\left|\overline{W_1}-0.496<\overline{W_0}-0.496\right|\\ \overline{M_1}>\overline{M_0}\\ \overline{C{I}_1}<\overline{C{I}_0}\\ \overline{S_1}>\overline{S_0}\\ \overline{O{P}_1}>\overline{O{P}_0}\\ D_1=D_0\\ T_1=T_0\\ N_1\ge N_0\left(1-35\%\right)\\ \left|\overline{W_2}-0.496<\overline{W_1}-0.496\right|\\ \overline{M_2}>\overline{M_1}\\ 1667\le PD\le 3333\\ F_1>F_0\\ F_2>F_1\end{array}\right.$$

(14)

$\overline{W_0}$, $\overline{M_0}$, $\overline{C{I}_0}$, $\overline{S_0}$, $\overline{O{P}_0}$ represent the average values of the uniform angle index, mingling index, canopy competition index, story index, and open comparison, respectively, in the forest stand before the selective felling; $\overline{W_1}$, $\overline{M_1}$, $\overline{C{I}_1}$, $\overline{S_1}$, $\overline{O{P}_1}$ denote the average values of each parameter after selective cutting; $D_0$, $T_0$, $N_0$ indicate the number of diameter classes, tree species, and the total number of trees in the stand before selective logging, while $D_1$, $T_1$, and $N_1$ represent the values of each parameter after selective cutting; $\overline{W_2}$, $\overline{M_2}$ represent the values of the angle index, crown competition index, story index, and open comparison of the forest stand after replanting; $PD$ denotes the density of forest stand plants after logging and replanting collaborative optimization.

(2)

Solving Algorithm Based on SARL

This study employs the RSM (random selection method) to determine the logging trees and assesses the replanting potential using the RFI in solving the collaborative optimization problem of selective cutting and replanting of FSS using SARL. The actions of selecting logging trees and determining the number of replanting trees are translated into agent actions in RL. Agents can choose between selective and non-selective logging in the logging optimization phase, and between replanting and non-replanting in the replanting optimization phase. Four joint optimization actions are generated, including ① selective cutting and replanting, ② no selective cutting and no replanting, ③ selective cutting but no replanting, and ④ no selective cutting but replanting. Actions ③ and ④ are equivalent to single optimization actions and have been extensively discussed in previous research. This study focuses on actions ① and ②, selective logging and replanting, and non-selective logging and no replanting, as the co-optimization actions of SARL. Agents select actions successively to maximize the reward value. Figure 3 illustrates the flowchart of SARL for addressing the co-optimization problem of FSS logging and replanting, with detailed algorithmic pseudocode provided in Appendix A.2.

(3)

Solving Algorithm Based on MARL

Figure 4 illustrates the flowchart of MARL for addressing the collaborative optimization problem of FSS logging and replanting, while detailed algorithmic pseudocode is provided in Appendix A.3. At each iteration, Agent1 and Agent2 are positioned at the “start point” and “end point” respectively. Agent1 is responsible for logging optimization, while Agent2 achieves replanting optimization. Using trial-and-error strategy, Agent1 is trained to find the set of selective logging trees satisfying the constraints, and Agent2 is trained to determine the replanting points and the optimal number of replanting plants. The collaborative optimization strategy motivates Agent1 and Agent2 to collaborate effectively through cross-collaboration actions and ultimately obtain the optimal spatial structure of the forest stand.

Designing an effective reward function is crucial in RL to find better results efficiently. In this solving algorithm, the reward function in the selective harvesting optimization part follows the result of previous study. For replanting optimization, a method based on curve trend for sample point selection is designed, rewarding or penalizing based on the presence of extreme points. Agent2 receives rewards and penalties in the replanting process based on the rewards and penalties obtained in the first and second rounds, with specific rules outlined in

Table 2. Reward and punishment rules in MCO.

Initial Rewards/Penalties			Subsequent Rewards/Penalties		Final Rewards/Penalties
	Increase	Little reward	$max(f_{x+b},f_{x+3b})>$ $max(f_x,f_{x+2b},f_{x+4b})$	Little reward	Little reward + Little reward
			Otherwise	Little reward	Little reward + Little reward
	Decrease	Little reward	$max(f_{x+b},f_{x+3b})>$ $max(f_x,f_{x+2b},f_{x+4b})$	Little reward	Little reward + Little reward
			Otherwise	Little reward	Little reward + Little reward
	Convex	Large reward	$max(f_{x+b},f_{x+3b})>$ $max(f_x,f_{x+2b},f_{x+4b})$	Little reward	Large reward + Little reward
			Otherwise	Large reward	Large reward + Large reward
	Concave	Penalty	$max(f_{x+b},f_{x+3b})>$ $max(f_x,f_{x+2b},f_{x+4b})$	Little reward	Penalty + Little reward
			Otherwise	Large penalty	Penalty + Large penalty

. Agent1 and Agent2 collaborate to move towards each other by efficiently identifying the FSS that maximizes the objective function. The earlier meet of Agent1 and Agent2 signifies the higher efficiency of MARL in solving multi-objective forest stand structure (MOFSS) optimization.

2.7.4. Experimental Scheme

Firstly, this study utilized an iterative algorithm to simulate replanting optimization based on the initial and optimal stands after cutting in four sample plots. The effectiveness of the RFI was compared with the MDGAM (maximum Delaunay generator area method) to validate the efficacy of RFI.

Next, to assess the applicability and effectiveness of the TCO scheme employing RFI, the stands of the four sample plots were subjected to logging optimization. The top three ranked stand structures, determined by the VOF, were selected and further optimized through replanting according to RFI. This aimed to validate the generalizability and efficacy of the TCO model.

Finally, to evaluate the performance of SARL and MARL in solving the multi-objective collaborative optimization model of FSS, SARL and MARL were employed. SARL and MARL utilized RSM for selective logging determination, while RFI guided replanting location determination, achieving collaborative optimization of forest stand structure (COFSS) (

Table 3. Experimental schemes for FSS optimization.

Experiment			Cutting Optimization			Replanting Optimization
Name	Type	Purpose	Cutting Method	Solution Algorithm	Resulting Stand	Replanting Method	Solution Algorithm	Resulting Stand
SR	Comparison Experiment	Compare the effectiveness of the RFI with the MDGAM	None	None	Initial stand	MDGAM RFI		Optimal replanting stand based initial stand
OCR			RSM QVM VMM	MC PSO RL	Optimal cutting stand		Iterate	Optimal replanting stand based optimal cutting stand
TCO		Validate the effectiveness and generalizability of RFI-based TCO			First three optimal cutting stands			Optimal replanting stand based first three optimal cutting stand
SCO	Optimization experiment	Evaluate the performance of SARL and MARL in solving multi-objective co-optimization models with FSSs	RSM	SARL	Optimal cutting stand based on RSM-SARL	RFI	SARL	Optimal replanting stand using RFI after the RSM-SARL optimal cutting was conducted
MCO				MARL	Optimal cutting stand based on RSM-MARL		MARL	Optimal replanting stand using RFI after the RSM-MARL optimal cutting was conducted

Note: SR: Single Replanting, OCR: Optimal Cutting–Replanting, TCO: Traditional Combinatorial Optimization, SCO: Collaborative Optimization based SARL, MCO: Collaborative Optimization based MARL

).

2.7.5. Parameter Settings

The parameter settings of each solution algorithm in the experiment are shown in

Table 4. Optimizing algorithm parameter configuration.

Algorithms	Parameters and Parameter Values	Value Meaning
Iterate	$I=0$	The initial iteration count
	$I_{max}=10,000$	Upper limit of iterations
	$U=0$	Initial count of successive iterations without OF improvement
	$U_{max}=500$	Upper limit of successive iterations without OF improvement
SARL	$I=0$	The initial iteration count
	$I_{max}=10,000$	Upper limit of iterations
	$state=1$	Starting position of the agent
	$stat{e}_{max}=100$	Farthest position of the agent
	$r1=150$, $r2=10$, $r3=-1$, $r4=-1$, $r5=1$	Reward and penalty values
RL	$I=0$	The initial iteration count
	$I_{max}=10,000$	Upper limit of iterations
	$stat{e}_1=1$	Starting position of the agent1
	$stat{e}_2=100$	Starting position of the agent2
	$stat{e}_{max1}=100$	Farthest position of the agent1
	$stat{e}_{max2}=1$	Farthest position of the agent2
	$r1=200$, $r2=20$, $r3=10$, $r4=1$, $r5=-1$, $r6=-50$	Reward and penalty values

. The main experimental program was developed in Python version 3.10, with all algorithms implemented manually. This approach allowed for precise control and customization of the algorithmic processes without relying on pre-built libraries.

3. Results

3.1. RFI

To ascertain the effectiveness of RFI, replanting optimization experiments were conducted on eight distinct FSSs. These FSSs encompassed both the optimal stands following selective logging and the initial stands. The study compared the effects of different methods for determining replanting locations in terms of the objective function and the number of iterations. The findings are summarized in Figure 5.

Regarding the VOF, the RFI-based replanting location determination method (RBRLDM) outperformed the basic MDGAM in achieving higher maximum VOFs across five FSSs (P1 initial, P2 initial, P2 optimal, P3 optimal, and P4 initial). For one FSS (P4 optimal), the maximum VOF was equal to that of MDGAM, while for two FSSs (P1 optimal and P3 initial), the maximum VOF equaled that achieved by MDGAM. Overall, RBRLDM yielded superior VOFs compared to basic MDGAM.

As to the number of iterations, RBRLDM demonstrated significantly faster convergence compared to basic MDGAM. Basic MDGAM was prone to “local optimization” pitfalls. Notably, in the replanting optimization experiments of the P3 initial stand, basic MDGAM outperformed RBRLDM in terms of both the number of iterations and the objective function.

3.2. TCO

The TCO model achieves dual optimization of cutting and replanting by optimizing replanting on the optimal FSS after selective logging. To validate its effectiveness, this study initially optimized the FSS of four sample plots through selective logging. Subsequently, the top three FSS sets were ranked in descending order according to the VOF, and replanting optimization was conducted on these sets based on RFIs.

Table 5. The results of TCO.

P1	Initial VOF	0.3515	0.3515	0.3515
	VOF after Cutting	0.41211st	0.40092nd	0.39753rd
	VOF after Replanting	0.47373rd	0.48791st	0.48292nd
P2	Initial VOF	0.2814	0.2814	0.2814
	VOF after Cutting	0.40781st	0.39832nd	0.38773rd
	VOF after Replanting	0.47182nd	0.43483rd	0.47851st
P3	Initial VOF	0.3748	0.3748	0.3748
	VOF after Cutting	0.50471st	0.48872nd	0.47293rd
	VOF after Replanting	0.52931st	0.51862nd	0.49083rd
P4	Initial VOF	0.4812	0.4812	0.4812
	VOF after Cutting	0.56351st	0.55932nd	0.55373rd
	VOF after Replanting	0.48523rd	0.51572nd	0.57131st

illustrates the results of TCO. In sample plot P1, the best replanting outcome of $FSSAC{A}_{1st}$ (the best FSS after cutting adjustment) was inferior to $FSSAC{A}_{2nd}$ and $FSSAC{A}_{3rd}$ (second and third best FSS after cutting adjustment). Similar trends were observed in P2. However, in P3, replanting optimization yielded the best outcome on $FSSAC{A}_{1st}$, achieving optimal harvesting and replanting simultaneously, thereby fulfilling the TCO model’s ultimate objective. The optimization result in P4 was sub-optimal, with replanting optimization failing to achieve the best outcome on $FSSAC{A}_{1st}$ and $FSSAC{A}_{2nd}$, while $FSSAC{A}_{3rd}$ achieved the desired result.

Overall, experimental results suggest that the TCO model, based on optimal FSS after selective logging, may not always attain optimal results and exhibits certain limitations.

3.3. Collaborative Optimization (CO)

This study conducted experiments on selective cutting and replanting collaborative optimization (SCRCO) using SARL and MARL on four sample plots to compare the practical application of the CO and TCO models. As shown in Figure 6 and

Table 6. Specific VOFs for each solution algorithm.

	P1	P2	P3	P4	Average	Lifting Amplitude	Average Lifting Amplitude
Initial	0.3515	0.2814	0.3748	0.4812	0.3722
TCO	0.4879	0.4785	0.5293	0.5713	0.5168	38.83%
SCO	0.5426	0.5273	0.5196	0.5751	0.5412	45.38%	43.88%
MCO	0.5444	0.5315	0.5297	0.5897	0.5488	47.44%

, both SARL and MARL under the SCRCO model can solve the multi-objective SCRCO problem of FSS, similar to the TCO model. Except for P3, SARL achieved better VOFs than TCO, with SARL improving the FSS of all four sample plots more than TCO on average. The optimization results of MARL in the four sample plots were better than those of TCO and SARL, with MARL achieving a higher average improvement in FSS than the other two methods.

3.4. Changes in Stand Structure Parameters

A single VOF cannot comprehensively assess the actual effect of the optimization scheme. This study compared in detail the changes in stand structure indices for each sample plot under each optimization scheme to obtain a more accurate and comprehensive assessment of the optimization effect.

Figure 7 shows the optimization results of different FSS optimization schemes. The spatial structure indicators of each stand were improved to varying degrees after the combination of selective harvesting and replanting in different schemes. Overall, the gap between the average angle index of the stand and the randomly distributed angle index of 0.496 was slightly reduced, indicating a closer alignment with random distribution. The canopy competition index of the stands was reduced, suggesting alleviated pressure among trees. The mingling index of each sample site improved, particularly in P4, which likely benefited from its initially weak mixing state. Additionally, the story index and open comparison of each sample site increased, indicating an enriched vertical structure and improved light transmittance. Overall, the optimization schemes effectively improved the structure of all sample sites.

3.5. Algorithm Performance

This research compares the performance of each algorithm in terms of FSS optimization degree and convergence speed. The FSS’s superiority or inferiority is measured by the VOF. As shown in Table 6, the maximum VOF of MARL (0.5444, 0.5315, 0.5297, 0.5897, respectively) was generally better than that of SARL (0.5426, 0.5273, 0.5196, 0.5751, respectively) in the cutting-replanting combination experiments in the four sample plots.

Regarding the convergence speed, MARL outperformed SARL and the TCO scheme. SARL is prone to “local optimization”, and the TCO scheme requires numerous iterations in harvesting optimization, resulting in slower convergence. Overall, MARL exhibits superior convergence speed and achieves better VOFs compared to SARL and the TCO scheme. (Figure 8 and Figure 9).

4. Discussion

Optimizing FSS is crucial for sustainable forest management, and developing a scientific and efficient quantitative model for FSS and enhancing model-solving efficiency are urgent challenges. Selective cutting and replanting are common measures for FSS optimization. However, the TCO model faces limitations such as not flexible replanting positions and difficulty in simultaneously optimizing selective cutting and replanting. To address these issues, this study proposes a new optimization scheme, aiming to introduce a multi-agent co-optimization strategy in RL to achieve the collaborative optimization of selective cutting and replanting for FSS. Simulated optimization experiments were conducted using data from four circular sample plots of Pinus yunnanensis plantation secondary forest on the eastern slope of Cangshan Mountain in Dali City, Yunnan Province, China, to verify the effectiveness and feasibility of this new scheme.

4.1. Superiority of RFI

The results of simulation optimization using different replanting location determination methods demonstrate that the RFI-based method can achieve better VOFs with fewer replanting iterations. This highlights the practical superiority of the RFI-based method in application schemes.

The basic MDGAM only considers the growth space of replanted trees in replanting optimization, selecting the maximum “forest gap” as the optimal location for replanting without considering other FSS indicators. In contrast, the RFI-based replanting optimization comprehensively considers various factors affecting the growth of replanted trees, including the relative coordinates of replanted trees and their neighbors, mingling degree, effects of replanting species, and tree age represented by DBH size ratio, as well as canopy competition index to account for tree crown and height effects. By incorporating multiple stand structure indices into the objective function, the RFI-based method aims to maximize VOF by improving the quality of as many stand structure indices as possible. Therefore, the RFI offers a significant advantage in replanting optimization for enhancing FSS optimization.

4.2. TCO and CO

In both P1 and P2, despite utilizing the optimal FSS adjusted after harvesting for replanting optimization experiments, the best replanting outcomes were not achieved. However, the $FSSAC{A}_{2nd}$ and $FSSAC{A}_{3rd}$ led to more substantial improvements in replanting optimization. This indicates that relying solely on the results from $FSSAC{A}_{1st}$ may not always yield the best overall optimization outcomes, suggesting the need for more comprehensive optimization schemes. Conversely, in P3, replanting based on $FSSAC{A}_{1st}$ achieved double optimization by combining selective cutting and replanting, resulting in the most optimal effect. This aligns with findings from other studies [17], emphasizing the collaborative synergy between selective logging and replanting. In P4, replanting optimization experiments based on FSS after $FSSAC{A}_{1st}$ and $FSSAC{A}_{2nd}$ did not yield expected results, whereas $FSSAC{A}_{3rd}$ achieved the best replanting optimization effect. This further highlights the potential limitations of relying solely on $FSSAC{A}_{1st}$ results, emphasizing the need for more flexible consideration of alternative optimization paths.

In simulated co-optimization experiments across the four sample plots, SARL outperformed the TCO model in terms of VOFs in all plots except P3. Moreover, the average enhancement of FSS achieved by SARL was higher than that of the TCO model. Notably, the optimization results of MARL across the sample plots were significantly superior to those of TCO and SARL. MARL exhibited a greater enhancement of FSS compared to the other methods. This underscores the exceptional performance of MARL in cooperative optimization, offering an effective approach for tackling complex FSS optimization problems.

Overall, relying solely on $FSSAC{A}_{1st}$ results for replanting optimization may not always lead to the best outcomes in FSS cutting and replanting optimization. The TCO model may exhibit limitations in certain scenarios, necessitating the adoption of more comprehensive and flexible optimization schemes tailored to specific circumstances. SARL and MARL demonstrate significant advantages over the TCO model in solving collaborative FSS optimization problems, providing novel insights and methodologies for addressing such challenges.

4.3. Changes in Stand Structure Parameters and Algorithm Performance

After the dual optimization of selective logging and replanting, several improvements in the FSS were observed. These include the maintenance of tree diameter classes, an increase in tree species diversity, closer adherence to a random distribution pattern, improved spatial segregation of tree species, reduced competitive pressure among trees, enriched vertical structure, and enhanced understory light conditions. Overall, under the applied constraints, the FSS showed improvements compared to its pre-optimization state, validating the efficacy of RL in addressing the multi-objective logging and replanting co-optimization problem for FSS. Additionally, the study confirmed that the optimization approach of selective logging followed by replanting, as seen in the TCO model, effectively addresses such challenges, consistent with findings from previous research [17].

In terms of algorithm performance, the MARL optimization scheme consistently yielded higher maximum VOFs compared to the TCO and SARL schemes, indicating superior FSS optimization outcomes with the multi-agent approach. Furthermore, the MARL algorithm required significantly fewer iterations to converge to the optimal solution compared to SARL and TCO schemes, underscoring its faster convergence speed and enhanced efficiency. SARL exhibited a tendency towards local optima, necessitating a higher number of iterations to achieve desired results. Conversely, the TCO scheme demonstrated a slower convergence speed overall, requiring more iterations to reach the optimal solution. Regarding algorithmic time consumption, MARL performed comparably to SARL and outperformed the TCO scheme, suggesting its ability to maintain high efficiency and computational performance.

Overall, the MARL approach exhibits clear advantages over the TCO and SARL schemes in solving the optimization problem of forest stand structure through cutting and replanting combinations. Its faster convergence speed, superior FSS optimization outcomes, and competitive computational efficiency highlight its effectiveness in addressing complex optimization challenges in forestry management.

5. Conclusions

Although our previous study successfully applied SARL to the multi-objective optimization of FSS, it only considered single cutting measures and did not integrate the synergistic effects of multiple regulatory measures in the model construction and solution process. While SARL has advantages in solving single cutting optimization models, it is less effective in addressing models that require the coordination of multiple measures.

This study introduces the RFI, which comprehensively considers various factors influencing the growth of replanted trees, enhancing the accuracy and effectiveness of replanting location determination. The RFI-based method demonstrates superior performance in optimizing FSS. Moreover, this research pioneers the application of MARL to collaborative optimization of FSS cutting and replanting. By leveraging the collaborative strategy of MARL, the study successfully achieves collaborative optimization of logging and replanting in FSS. In comparison to TCO and SARL, MARL exhibits significant advantages in solving this problem. The detailed comparison of experimental results quantitatively verifies the superiority of MARL in collaborative optimization of stand structure cutting and replanting, providing strong theoretical support for addressing stand structure optimization challenges in actual forest management scenarios.

While this study successfully applied MARL to the multi-objective optimization of stand structure, the experimental results may not fully reflect real-world forest conditions due to the specific sample data and simulation settings used. Additionally, although the optimization scheme based on MARL showed better results, there is still room for performance improvement. Based on the above research results and shortcomings, future research can focus on the following aspects:

(1) To enhance the accuracy and efficiency of MARL decisions, it is essential to integrate GIS (Geographic Information System) data and tools to obtain more detailed three-dimensional spatial information and accurate canopy calculations. By combining GIS with MARL, it is possible to construct constraints or objective functions that incorporate spatial three-dimensional features. This integration ensures that MARL makes more informed and precise decisions.

(2) Improving and optimizing the algorithms and techniques used in the stand structure optimization scheme based on MARL. Enhancing its performance and efficiency in processing large-scale data and complex environments will improve its generalization ability and practical applicability.

(3) Explore and design more flexible and integrated multi-dimensional optimization strategies to balance various objectives and interests in forest management, achieving a more comprehensive and sustainable optimization of stand structure. For instance, design more efficient and appropriate methods for determining the felling and replanting trees.