**1. Introduction**

Straw rotting fungi are fungi that absorb the decayed humus of grass straws (such as straw and wheat straw) as the main source of nutrition [1], mainly including Agaricus bisporus, Agaricus blazei, straw mushrooms, capsule mushrooms, etc. Its factory production usually adopts the bed planting method. The growth of its fruit is similar to a round or spherical shape, easy to cluster, the combination of fruit and culture medium (soil) is relatively rigid and compact, the ripening time of each fruit is different, and the fruit is tender and vulnerable. This kind of fruit usually needs to be harvested by selective harvesting, and the aggregated mature fruits need to be harvested in the specified order; otherwise, the fruit is easy to be damaged. Therefore, it is difficult to achieve effective

**Citation:** Yang, S.; Jia, B.; Yu, T.; Yuan, J. Research on Multiobjective Optimization Algorithm for Cooperative Harvesting Trajectory Optimization of an Intelligent Multiarm Straw-Rotting Fungus Harvesting Robot. *Agriculture* **2022**, *12*, 986. https://doi.org/10.3390/ agriculture12070986

Academic Editor: Francesco Marinello

Received: 30 May 2022 Accepted: 6 July 2022 Published: 8 July 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

automatic nondestructive harvesting [2]. Under the trend of serious labor shortages, although the cultivation of such crops can achieve factory-like and intensive production, the labor-intensive harvesting process [3] still relies on manual labor, which has become the main bottleneck affecting further production and efficiency improvement. Therefore, there is an urgent need for harvesting robots that can adapt to the factory and intensive cultivation environment for intelligent and efficient harvesting.

In the 1990s, Reed, J. N. and Tillett, R.D. [4] proposed the first selective harvesting robot for Agaricus bisporus, which made selective autonomous harvesting of straw-rotting fungus feasible. Since then, scholars have carried out many researches to on the edible fungus harvesting robot technology but mainly focused on visual recognition and endeffectors to improve the harvesting success rate [5–13], and few studies on trajectory planning. Yang [14] aimed at the issue of mushroom harvesting path planning. GA was used to optimize the harvesting path to improve the harvesting efficiency after obtaining the location coordinates of all the mushrooms that can be harvested. In order to improve the harvesting efficiency of the mushroom harvesting robot, Hu et al. [15] proposed an improved simulated annealing algorithm to find the optimal path, which can increase the harvesting efficiency by 14–18%. However, both of the above studies are all optimized for the path of the single-arm harvesting robot, which can improve the harvesting efficiency to a certain extent, but far from manual efficiency. In addition, these methods do not consider the specific harvesting order of the aggregated fruits and are merely suitable for harvesting the fruits, which are relatively sparse. When the fruits grow densely, take Agaricus bisporus as an example, as shown in Figure 1, for the clustered fruits in the red circled area, the height of fruit body A is higher than B and C. If B or C is harvested before A, B and C will be damaged or even be harvested unsuccessfully because their cap is covered by A. Moreover, a may be pushed down, which will cause its cup center to deviate greatly, resulting in failure when picking A. So, the aggregated fruits should be harvested in order of height. Otherwise, the success rate of non-destructive harvesting and the harvesting quality will be reduced. So, it is significant to greatly improve harvesting efficiency; meanwhile, to take into account the harvesting order of aggregated fruits to improve the success rate of non-destructive harvesting further.

**Figure 1.** Schematic diagram of aggregated fruits.

Because the fruit is delicate and vulnerable and the harvesting environment is complex, the harvesting operation is usually limited to slow and time-consuming [16]. So, the harvesting efficiency is normally much lower than manual efficiency, which leads to the situation that the harvesting robot cannot be widely used in actual production. To solve this issue, using multiple harvesting arms is a typical approach.

A cotton harvesting robot with multiple robotic arms has been developed to achieve multiple plucking of crops, which increases the yield by 20–25% [17].

Zion [18] developed a melon harvesting robot with multiple Cartesian arms to accelerate the speed. The robot travels along a two-dimensional field at a constant velocity. The multiarm assignment is modeled as a k-colorable sub-graph problem and uses a greedy algorithm to achieve an optimal solution. Because the greedy algorithm focuses on local optimization, the effect of global optimization may not be very good.

A strawberry harvesting robot with dual cartesian arms was developed to reduce the cost and optimize the harvesting efficiency. The fruits were partitioned into several subsections equally for each arm to harvest. The harvesting speed can be reduced to 4.6 s [19].

An oyster mushroom harvesting robot with four harvesting arms connected in parallel on a common mobile platform is provided to improve efficiency [20]. Each arm is allocated to harvest in the divided area independently, which is divided from left to right with an approximately equal number of target fruits, and the mature fruit closest to the end actuator is picked first in each area.

A multiple robot arm system for kiwifruit harvest has been designed to reduce the harvesting cycle time, thereby increasing the efficiency to meet the requirements of commercial applications [21]. The multiarm harvesting robot sorts and partitions target fruits according to their x coordinates and assigns them to the harvesting arms so that the amount of the harvested fruit is approximately equal for each arm. However, for fruits growing in clusters, the clustered fruits are completely allocated to the same arm to be harvested from low to high to avoid touching or moving other fruits' positions. Although this method can improve the picking success rate of fruit clusters, it will affect the distribution uniformity of each arm, thereby reducing the picking efficiency.

Most of the above multiarm harvesting trajectory planning uses the traditional method, which partitions target fruits approximately evenly according to the arrangement direction of the harvesting arms and allocates them to the robot arms, and each arm harvests fruits from left to right or from right to left. Normally, this method can achieve a good result. However, the trajectory planning effect of this method will be greatly reduced when the fruits are distributed seriously ununiformly and even with many fruit clusters. Moreover, this method assumes that each fruit can be picked by each arm; that is, the accessibility of the task to be executed to the individual is consistent. While the fact is that not every fruit can be picked by each arm due to the size of the arm, which indicates that the accessibility of the task to be executed in the instance is inconsistent. This shortcoming will also increase the difficulty of equal allocation among each arm. Therefore, a more global and flexible optimization approach is required to resolve the above issues. In order to effectively improve harvesting efficiency and adapt to the environment of straw-rotting fungus factory bed planting, the harvesting robot can be designed as a highly cost-effective Cartesian coordinate harvesting robot as shown in Figure 2, equipped with multiple harvesting arms [22]. Here, because the mature fruits of straw-rotting fungus are little difference in height, it can be assumed that the time spent in the height direction (Z-axis) of each fruit during harvesting is the same, so that the dimension of the harvesting trajectory planning of such fruits can be reduced as a two-dimensional trajectory planning problem in the XY plane. In addition, since the shape of the straw-rotting fungus is basically spherical, the projection of the shape of the fruit on the XY plane is further approximated as a circle. Just as the harvesting sequence planning problem of a single-arm harvesting robot can be regarded as a TSP problem [23]. Given the above assumptions, the multiarm cooperative harvesting trajectory planning problem studied in this paper can be simplified as a typical MTSP as well.

Similar to most fruit and vegetable picking robots mentioned above, the harvesting time for a straw-rotting fungus fruit is also time-consuming (about 5 s) due to the operation of grasping the fruit and detaching it from the culture medium or soil cannot be too fast to avoid damaging the fruit. Compared with the harvesting time (time required for harvesting at the target fruit position), the moving time (time used to move from the picked fruit to the next target fruit) is much shorter. Therefore, it is much more significant to allocate the harvesting tasks to each arm as uniformly as possible before optimizing the shortest path for each arm [24–26]. In the meantime, it is also necessary to take into account the harvesting sequence of aggregated fruits to improve the success rate of non-destructive harvesting.

$$\mathbf{a})$$

(**a**) (**b**)

**Figure 2.** Detailed mechanical design of MFA. (**a**) Detailed mechanical structure of MFA; (**b**) detailed mechanical structure of end-effector.

For the sake of the uniform assignment of tasks for each harvesting arm, in addition to the region segmentation method described in the previous literature analysis, the stepwise algorithm used for MTSP is more appropriate to solve the issue, which is also superior to the general heuristic algorithm [27–30]. A stepwise algorithm is proposed to resolve the MTSP of multi-UAV cooperative airport bird repelling, which adopts a genetic algorithm to divide the MTSP into M independent TSP. This paper shows that GA is suitable to solve the task balance assignment of MTSP [31]. Lu et al. [32] combined the K-means clustering algorithm with GA to solve multiobjective MTSP, although a better-balanced task assignment is obtained, but it still has poor trajectory searchability.

The ant colony algorithm is widely used in combinatorial optimization problems due to its strong search ability and fast convergence speed [33]. Necula et al. [34] used ACO as the bi-standard surface for solving the multitraveling salesman problem. Changdar et al. [35] adopted ACO to resolve the multi-stop multitraveling salesman problem with non-random parameters. Although both of them achieve a good result, it is easier to fall into the local optimal solution prematurely in MTSP with high complexity problems.

In addition to solving the problem of equal allocation of multiarm tasks under SSIS restrictions to improve efficiency, it is also necessary to take into account the harvesting order of aggregated fruits to improve the success rate of non-destructive harvesting mentioned above. To achieve this aim, the above algorithm for solving similar MTSP should be further improved by combining a multiobjective optimization method. More than this, another challenge is how to improve the algorithm so that the fruit clusters that need to be picked in a specific order can be split and allocated to multiple different arms, to overcome the shortcoming of the whole fruits in the same cluster being merely allocated to the same arm, so as to further increase the efficiency even though the mature fruits are seriously unevenly distributed on a culture medium (soil) and with many fruit clusters.

Through the above analysis, this paper proposes an improved genetic ant colony multiobjective optimization algorithm, which makes comprehensive use of the advantages of both the genetic algorithm and the ant colony algorithm to resolve the difficulties of multiarm cooperative harvesting of straw-rotting fungus and achieve both high harvesting efficiency and a high success rate. The main contributions of this approach are as follows:


algorithm is improved by designing a new method for calculating local density, which can better meet the clustering analysis of straw-rotting fungus;


#### **2. Description of Multiarm Cooperative Harvesting Trajectory Planning Problem**

#### *2.1. Intelligent Multiarm Straw-Rotting Fungus Harvesting Robot*

As shown in Figure 2, an intelligent multiarm straw rotting fungus harvesting robot consists of the following four main parts: (i) a mobile platform (MP) with a conveyor belt; (ii) a visual position system (VPS), and (iii) multiple flexible arms (MFA); (iv) control system (CS). The MP is placed on the rails of the multistory shelves to move along the rail with the help of the MP, the VPS can recognize and locate all mature fruits. The CS can plan the harvesting trajectory and control MFA to harvest the mature fruits and send them to the convey for blanking.

MFA consists of at least two Cartesian arms with three DOF. As shown in Figure 2a, the X-axis adopts the gear rack motor traverse multi-axis mechanism, and each is mounted on a guide rail. The X-axis motor drives the gears in the gear rack through a synchronous belt drive structure to move the end-effector along the X-direction. The Y-axis uses the synchronous belt drive module to move the end-effector in the Y direction. The end-effector is designed into a two-stage driving structure, as shown in Figure 2b, to adapt to the narrow layer height of the edible fungus culture rack (available design height is only about 250 mm while travel should be 160 mm). The first stage is driven by a rodless cylinder, and the second stage is driven by a pen cylinder. The suction cup is connected to the pen cylinder, which is connected to the rodless cylinder. During harvesting, the rodless cylinder drives the pen cylinder lower. Firstly, the rodless cylinder inflates to drive the pen cylinder down. Then, the pen cylinder deflates to cause the suction cup to drop under the action of gravity until it touches the surface of the target fruit. Finally, vacuum to grasp the target fruit, rotate or wobble, and pull up to detach the fruit from the soil. The structure can better adapt to the large height difference in fruits and realize picking action by wobble or rotation motion.

The working flow of the proposed robot is shown in Figure 3, as follows: First, the camera traverses all fruit images within the current visual area; Secondly, the image is transferred to the host computer, and the positions of the mature fruits are identified by image processing; Thirdly, all of the coordinates of fruits to be picked are scheduled and allocated to multiple harvesting arms; Fourth, the controller drives the arms to harvest fruits after receiving the assigned task; Finally, the robot moves forward as a whole by MP to start the next harvesting cycle until the harvest of one layer is finished. In addition to the use of three arms, in order to improve efficiency, the following are also performed: (i) Multiple economical depth cameras are used to shorten the photographing and identification time; (ii) Visual processing and harvesting operation work in parallel rather than in series. In the current cycle, the multiarm harvester harvests the fruits identified in the previous cycle.

As shown in Figure 4, the control system takes the motion controller (TRIO MC4N) as the core. The motion controller communicates with the PC through the Ethercat bus to obtain the picking task for each arm, then drives the motor of each axis and controls the cylinder, sucker, and other actuators of the end-effector to work monitor the working status of each actuator in real-time and returns it to the upper computer. The economical and compact stepping servo motor is used for each axis motor, and the communication between the motor and the controller is also via the Ethercat bus.

˛ 1

˄

<

**Figure 3.** Workflow of the multiarm harvesting robot.

**Figure 4.** Hardware composition diagram of control system.

### *2.2. Accessibility Analysis of Multiple Harvesting Arms*

Generally, the MTSP problem assumes that all cities can be visited by each traveler, but this is not the case with the multiple harvesting arm structure in this paper. As shown in Figure 2a, multiple arms are arranged in a series along the X-direction. Due to the unignorable width of the end-effector, each arm has a certain inaccessible area, and the reachable range of each arm is different.

The accessibility of the picking arm in this paper is characterized by exclusive area, partially shared area, and fully shared area, and they are symmetrically distributed as shown in Figure 5, which is called the serial increasing symmetric shared (SISS) area. The definition of the SSIS is described in detail in [36]. Such an SSIS area makes it more difficult to assign the harvesting tasks equally among each arm.

**Figure 5.** Distribution of accessible area.

The range of each harvesting region can be described as follows:

$$S\_k = (X\_{k-1}, X\_k), \ k \in \{1, 2, \dots, 2M - 1\} \tag{1}$$

$$X\_k = \begin{cases} \begin{array}{c} W \times k, \\\ L - (2M - 1 - k) \times W, \ k \in \{M + 1, M + 2, \dots, 2M - 1\} \end{array} \tag{2}$$

where *L* is the length of the harvesting region, *W* is the width of arm and end-effector, *M* is the amount of harvesting arm, *Sk* is the *k*-th harvesting region, and *Xk* is the coordinate of the end point in the X-direction of the *k*-th harvesting area.

Equation (3) shows the accessible harvesting arms per area as follows:

$$S\_k(j1, j2) = \begin{cases} S\_k(1, k) \, , & k \in \{1, 2, \dots, M\} \\ S\_k(k - M + 1, M) \, , & k \in \{M + 1, M + 2, \dots, 2M - 1\} \end{cases} \tag{3}$$

where (*j*1,*j*2) indicate harvesting arms *j*1 to *j*2 can reach the *k*-th area, 1 ≤ *j*1 ≤ *j*2 ≤ *M*.

#### *2.3. Description of Trajectory Planning for Multiarm Cooperative Harvesting*

As described in Section 1, the multiarm trajectory planning problem of this paper can be abstracted to an MTSP problem with an SSIS Area. In which case, the aggregated fruits should be harvested in the order from high to low in the Z direction. The goal of trajectory optimization is to achieve both high harvesting efficiency and a high success rate with the following constraints:


#### **3. Mathematical Model of Multiarm Cooperative Harvesting Trajectory Planning Problem**

It can be known from Section 2.3 that the trajectory optimization problem of an intelligent multiarm straw-rotting fungus harvesting robot is regarded as a multiobjective optimization problem. Researchers mainly use multiobjective optimization methods such as the weighted coefficient method, multiobjective genetic algorithm, multiobjective particle swarm optimization algorithm, etc. [37,38]. Among them, because the optimization problem in this paper is a two-dimensional, that is, a low-dimensional objective optimization problem, and to reduce the complexity of the problem, the simple and easy-to-use weighted coefficient method is adopted, which decomposes multiobjective into a single objective and then optimizes this single objective.

Suppose the robot has M arms and N fruits to be harvested. The average harvesting efficiency of the robot is C pcs/h and the average manual working efficiency is M pcs/h; R is the ratio of the average working efficiency of the robot and man. In order to obtain the optimal solution to double the objectives of harvesting efficiency and success rate, take

R and success rate P to solve the objective function E by the objective weighting method as follows:

$$\mathbf{Max}\,\mathbf{E} = \mathbf{R} \times \mathbf{K}\_1 + \mathbf{P} \times \mathbf{K}\_2 \tag{4}$$

$$R = \frac{\mathcal{C}}{\mathcal{M}} = \frac{\mathcal{N}}{\mathcal{T} \times \mathcal{M}} \tag{5}$$

$$P = \frac{\text{N} - \text{N}\_{\text{lose}}}{\text{N}} \times 100\text{\%} \tag{6}$$

$$\mathbf{T} = \left[ \max(\mathbf{t}\_{\text{-}} \text{ARM}\_{\text{i}}) + \mathbf{T}\_{\text{c}} \right] / 3600 \tag{7}$$

$$\mathbf{t}\\_\text{ARM}\_\mathbf{i} = \sum\_{\mathbf{j}=1}^{N} \sum\_{\mathbf{k}=1}^{N} \mathbf{t}\_{\mathbf{jk}} \mathbf{x}\_{\mathbf{ijk}} + \sum\_{\mathbf{j}=1}^{N} \mathbf{t}\_{\mathbf{d},\mathbf{j}} \mathbf{x}\_{\mathbf{id},\mathbf{j}} + \sum\_{\mathbf{j}=1}^{N} \mathbf{t}\_{\mathbf{j}\mathbf{d}\_i} \mathbf{x}\_{\mathbf{ijd}\_i}, \forall \mathbf{i} \in \mathcal{M} \tag{8}$$

$$\chi\_{\rm ijk} = \begin{cases} 1, & \text{when Arm i from fruit j to k} \\ 0, & \text{other} \end{cases} \tag{9}$$

where,

K1 and K2 are weighting coefficients, K1 and K2 > 0 and K1 + K2 = 1;

Nlose is the number of fruits in the clusters that failed to be harvested in the order from high to low in the Z direction when the fruits were gathered;

T is the total harvesting time required for harvesting all mature fruits;

TC is the avoidance time;

xidi <sup>j</sup> indicates that Arm i from the starting point di to fruit j;

xijdi indicates that Arm i from fruit j to the starting point di

tjk is the time taken by the harvesting arm to perform the task from fruit j to k;

The objective function must satisfy the following constraints:

Given the mature fruits set is V = {1,2, ... ,N}; U is the set of harvesting arms (1 ≤ M ≤ N), U = {1,2, ... ,M); Since the working range of each harvesting arm is limited, di is the starting point of Arm i.

$$y\_{i\mathbf{j}} = \begin{cases} 1, \mathbf{j} \in I\_{\mathbf{i}} \text{ when Arm i harvests fruit j} \\ 0, \text{ other} \end{cases} \tag{10}$$

where *Ii* means the mature fruits subset must be harvested by Arm i, Ii∈V

The sum of fruits picked by each arm should be equal to N as follows:

$$\sum\_{\vec{\mathbf{j}}=1}^{n} \mathbf{y}\_{\vec{\mathbf{i}}\dagger} = \mathbf{Q}\_{\vec{\mathbf{i}}\prime} \,\forall \mathbf{i} \in \mathbf{M}\_{\prime} \tag{11}$$

$$\sum\_{\mathbf{i}=1}^{\mathbf{m}} \mathbf{Q}\_{\mathbf{i}} = \mathbf{n} \tag{12}$$

where Qi indicates the number of fruits allocated to Arm i, i∈U;

Each fruit can only be harvested by one arm as follows:

$$\sum\_{\mathbf{i}=1}^{m} \mathbf{y}\_{\mathbf{i}\mathbf{j}} = 1, \forall \mathbf{j} \in \mathbf{V} \tag{13}$$

Each arm should start from and come back to its own start point after harvesting all assigned fruits as follows:

$$\sum\_{\mathbf{j}=1}^{n} \mathbf{x}\_{\text{id}\_{\mathbf{j}}} = \sum\_{\mathbf{j}=1}^{n} \mathbf{x}\_{\text{ijd}\_{\mathbf{i}}} = 1 \,, \forall \mathbf{i} \in \mathcal{M} \tag{14}$$
