*5.1. Parameter Setting*

In order to give full play to the performance of MaOEAs on MaOPs with different objective number, different *FEs* and population size *N* should be set for different objective number *M*. Taking the WFG [89,90] test suite as an example, the number of dimensions *D* needs to be dynamically changed. Here is set as recommended *D* = *M* + 9. In addition, the number of objectives in the experiments conducted in this paper is divided into five groups, and the number of objectives is 3, 5, 8, 10, and 15, respectively. In terms of population size setting, since reference points are used in both MOEA/D-PaS and NSGA-III, the original reference points need to be generated in a certain way. In this case, Das and Dennis's approach [91] is used to generate the original reference points on the hyperplane, while the other algorithms should have the same initial population size to ensure fairness. In addition, the number of generated reference points is the same with set in NSGA-III [5,6]. So, the corresponding population size *N* is set to 91, 210, 156, 275, and 135, respectively. The corresponding number of *FEs* is 104–104 × 5. The detailed parameter settings are shown in Table 1.

**Table 1.** The parameter settings of experiments.


In the experiments of dynamic convergence factor, *α* in the base DLS are set to the recommended 0.9. In the setting of dynamic convergence factors, various functions monotonically increasing in the interval [0, 1] are adopted for dynamic adjustment, which will be described in detail in Section 5.2. For all comparative algorithms in experiments, the parameter settings on each objective were also consistent with those in Table 1.

In addition, the running device is PC, the system version is Windows 10 enterprise version, the processor is Intel(R) Core (TM) i3-8100 CPU 3.6 GHz, and the RAM is 8 GB.

#### *5.2. Experiments on Dynamic Convergence Factors*

This paper proposes the concept of dynamic convergence factor in Section 4.3.2. The main approach is to determine the size of convergence factor dynamically based on the basic DLS and the state of individual exploration. The purpose of dynamic convergence factor is to make DLS adapt better to the evolutionary state of the population, so as to achieve the optimal convergence factor setting. Since the optimal value range of the convergence factor *α* is the interval [0.8, 0.9], we take the value of a monotone increasing function in the interval [0, 1] as shown in Equation (13), multiply it by a dynamic scaling factor *ω* by the function mapping of proportional coefficient *ExRatio*, and then subtract the corrected value from the original. In this way, it is possible to dynamically change the value of *α'* according to the proportionality coefficient *ExRatio*. In addition, when the *ExRatio* is relatively large (more diversity-related solutions need to be explored), the convergence factor can be appropriately reduced to better satisfy the convergence and diversity balance of the population.

In order to give full play to the optimal performance of the DL-TPCEA, certain work require to select the monotone increasing function in the interval [0, 1] of Equation (13). Columns 3 through 11 of the first row in Table 2 show some common monotone increasing functions in the interval [0, 1] as a comparative experiment. For example, the corresponding formula of Tan in the fourth column is as follows:

$$
\omega t = \alpha - \omega \ast \tan(\frac{\text{ExRatio} \ast \pi}{4}) \tag{14}
$$

**Table 2.** The results of various monotone increasing functions on the WFG test suite, and the inverted generational distance (IGD) values of the results are tested by Friedman test.


In addition, column 2 corresponds to the original DLS that the value of *α* is set to 0.9. The last column is a control group, and the method used here is that when *ExRatio* > 0, the value of *α* is multiplied by a value less than 1 (set as 0.9 here). This is equivalent to setting up a fixed set of transformations instead of making dynamic changes through functional mapping and the proportional coefficient *ExRatio*. This group is designed to analyze and compare the advantages and disadvantages of fixed transformations over functional mappings.

According to the parameter settings in Table 1, the different algorithms were run independently 30 times on each WFG test suite. The average inverted generational distance (IGD) values of the 30 runs were performed using the Friedman test (the smaller the better) and presented in Table 2. The last row is the average of the five sets of Friedman tests. Dark gray represents the best result and light gray represents the second-best result. From the results, the best performance is obtained when the monotone increasing function is taken as the sine function, and the optimal value (minimum value) is obtained on the 3- and 15-objective WFG, respectively. The second-best result is obtained on the 10-objective WFG. Although the results on 5- and 8-objective WFG are not so good, they are also relatively small in terms of numerical values. At the same time, the sinusoidal results were also the best among the average Friedman results of the five experiments. In addition, when the monotone increasing function is *x*1/3, it performs second-best, and obtains second best results on the 5- and 8-objective WFG, respectively, and also obtains second best results in the average Friedman test.

When the monotone increasing function is selected as *x<sup>M</sup>* and *x*1/2, the optimal value is obtained on 5- and 8-objective WFG, respectively. However, the average Friedman test results for these two functions are not very good. It is worth noting that the original DLS obtained the optimal result on 10-objective WFG, followed by the mapping of sine function. In addition, the set of fixed transformations also showed the second-best result on 3-objective WFG. The average Friedman test results of these two strategies are not much different, but they are not as good as the average Friedman test results of the sine function.

Therefore, the sine function mapping of *ExRatio* is finally selected in dynamic convergence factor. In the experiments in Section 5.3 below, the dynamic convergence factor in DL-TPCEA is calculated in the form shown in Equation (13).

#### *5.3. Experiments on Comparative Algorithms*

To verify the performance of the proposed DL-TPCEA, we compare it with five stateof-the-art algorithms: MOEA/D-PaS [14], NSGA-III [5], CMOPSO [92], Two\_Arch2 [88], and DLEA. The brief introduction to these comparative algorithms is given below.

In MOEA/D-PaS, a Pareto adaptive scalarizing (PaS) approximation method was proposed, which approximated the optimal *p* value of the commonly used scalarizing method. This is the key to balancing Pareto optimal selection pressure and algorithm robustness to PF geometries. It guarantees that any solution can be found along PF for given some weight. PaS is combined with the decomposition-based algorithm (MOEA/D) to increase the ability of balanced convergence and diversity.

NSGA-III is an improved version based on the framework of NSGA-II [4]. The crowding distance operator that was used to balance diversity in NSGA-II is modified into a diversity keeping strategy based on weight vector guidance. NSGA-III used a set of pre-generated uniformly distributed weight vectors to simulate the distribution of PF. When selecting solutions, the candidate solutions with the shortest vertical distance to these weight vectors will be selected.

CMOPSO is an improved version of the multi-objective particle swarm optimization (MOPSO [93]) by adding a competition mechanism. CMOPSO makes particles pairwise competitions to select particles in each generation of population. This makes the performance of CMOPSO less dependent on global and local optimal particles stored in an external archive.

Two\_Arch2 uses two external archives, where each archive promotes convergence (CA) and diversity (DA). The two archives use different selection principles, where CA is indicator-based and DA is Pareto-based. At the same time, *Lp*-norm-based diversity maintenance scheme was also proposed in Two\_Arch2 to improve the diversity of the population.

DLEA mainly uses the DLS mentioned in Section 4.1. The algorithm mainly used DLS to enhance the balance of convergence and diversity in environmental selection. Two different indicators are used to improve the performance by maintaining the convergence and diversity, respectively. Meanwhile, the convergence factor *α* in DLEA was fixed at 0.9. Compared with DLEA, DL-TPCEA proposes the concept of dynamic convergence factor. The comparison of these two algorithms is mainly to highlight the performance improvement brought by the dynamic convergence factors and the coevolution of the two populations.

The five comparative algorithms selected have their own characteristics, including operators frequently used in the many-objective optimization field: decomposition-based operator, Pareto-based operator, indicator-based operator, external-archive-based operator, and weight-vector-based operator. Comparing these algorithms can show the performance advantage of an algorithm more significantly.

For DL-TPCEA and other five comparative algorithms, Tables 3 and 4 give the mean and standard deviation (in parentheses) of HV values run on five sets of WFG test suites, and the results in Tables 5 and 6 are corresponding IGD values. Wilkerson Rank-Sum test (*α* = 0.05) was used to test the significant difference between HV values and IGD values of these six algorithms. The symbols −, +, and ≈ stand for that the indicator values of the comparative algorithms were significantly worse than, better than, and similar to that of DL-TPCEA, respectively. In addition, for each test instance, the best (maximum) HV value and the best (minimum) IGD value were highlighted in gray.


**Table 3.** HV results of six algorithms on benchmarks WFG1-WFG9 with 3, 5, and 8 objectives.




**Table 5.** IGD results of six algorithms on benchmarks WFG1-WFG9 with 3, 5, and 8 objectives.


**Table 6.** IGD results of six algorithms on benchmarks WFG1-WFG9 with 10 and 15 objectives.

As shown in Tables 3 and 4, in terms of HV, the proposed DL-TPCEA was significantly better than the other five algorithms on 26 out of 45 test instances, and performed similarly to them on two test instances. Specifically, DL-TPCEA generated higher HV values than MOEA/D-PaS, NSGA-III, CMOPSO, Two\_Arch2, and DLEA on 39, 33, 39, 38, and 36 out of the 45 test instances, respectively. As shown in Tables 5 and 6, in terms of IGD, the proposed DL-TPCEA was significantly better than the other five algorithms on 21 out of 45 test instances, and performed similarly to them on one test instance. DL-TPCEA generated smaller IGD values than MOEA/D-PaS, NSGA-III, CMOPSO, Two\_Arch2, and DLEA on 44, 31, 26, 29, and 29 out of the 45 test instances, respectively. The results demonstrated that it was a promising way to approximate the PFs of WFGs via coevolution and dynamic learning strategy in the proposed DL-TPCEA. CMOPSO and DLEA showed better results for IGD values than for HV values, indicating that these two algorithms also had a good ability to maintain the trade-off between convergence and diversity. In addition, from the comprehensive results of HV values and IGD values, NSGA-III was also a good algorithm. However, compared with these three MaOEAs, the proposed DL-TPCEA also showed much better performance with respect to both convergence and diversity.

The superiority of DL-TPCEA can be explained as follows. The other five comparative algorithms, with the exception of Two\_Ach2, attempted to simulate PFs of WFGs through balanced convergence and diversity using a single population. However, as the number of objectives increases, the balance between convergence and diversity became more difficult. This was because the increasing number of objectives led to more serious conflicts on multiple objectives, so that the selection pressure of the MOEAs was not as good as when there were fewer objectives. When the number of objectives kept increasing, the solutions generated by these algorithms may only be single convergence-related solutions or diversity-related solutions, but there was no compromise between convergence and diversity over the whole PF. In addition, Two\_Arch2 used two different external archives to store convergence-related solutions or diversity-related solutions, respectively. Moreover, Two\_Arch2 promoted the evolution between the two archives so that the population maintained a compromise between convergence and diversity. However, these two external archiving methods had poor performance in dealing with MaOPs, especially the objective conflicts were serious. The proposed DL-TPCEA used two populations for coevolution and the shortcomings of each population will be compensated by BCE. It kept a good balance between convergence and diversity, and used dynamic learning strategy to further strengthen the balance. As a result, the proposed DL-TPCEA did not degrade the performance because of the conflicts caused by the number of objectives increased.

From the HV values in Tables 3 and 4, we can see that the HV value obtained by other related algorithms except MOEA/D-PaS on 8-, 10-, and 15-objective WFG3 was zero. This was caused by the calculation of the HV results using a set of reference points set on the corresponding test instance. When the corresponding algorithms failed to obtain any candidate solution dominating the reference point on those test instances, the value of the hypervolume (HV value) formed by the non-dominated population and the reference point was zero. In these three test instances, only MOEA/D-PaS obtained HV results, which also indicates that MOEA/D-PaS had its own advantages in dealing with WFG3. In addition to the three test instances, all the algorithms obtained HV values (non-zero) on the other test instances.

It can also be concluded from the results that DL-TPCEA mainly showed poor performance on WFG1 and WFG3. In terms of the characteristics of the problem, WFG1 is convex and mixed, while WFG3 is linear and degenerate. DL-TPCEA may not be able to find boundary individuals well on such problems like WFG1 or WFG3, thus, resulting in poor performance. In addition, WFG2 is convex and disconnected. However, DL-TPCEA was still able to obtain the optimal HV results, indicating that the two-population coevolution of DL-TPCEA is capable of dealing with disconnected MaOPs. Finally, WFG4-9 is concave, and DL-TPCEA also has the best performance. The performance of DL-TPCEA on 10- and

15-objective WFG4-9 was lower than DLEA, indicating that dynamic learning strategies played a significant role in dealing with concave MaOPs.

In order to more intuitively observe the ability of the six algorithms to balance convergence and diversity on the WFG test suite, the parallel coordinates of the solution set obtained by the six algorithms on the 5-objective WFG2 and 10-objective WFG9 were given in Figures 3 and 4, respectively. In parallel coordinates, the ordinate represents the objective value, and the convergence information can be obtained. An algorithm has good convergence if it can converge to the range of PF. At the same time, the vertical height can also reflect the performance in the diversity. The horizontal axis corresponds to each objective, which can reflect the diversity information of MaOEAs. It is an algorithm that maintains solutions for every objective, and the denser the lines, the better the diversity. Therefore, using the parallel coordinates of the solution set can better compare the performance of MaOEAs.

For 5-objective WFG2, the range of PF on each objective dimension *m* is from 0 to *m* \*2(*m* = 1, ... , *M*). As shown in Figure 3, although the solution sets obtained by all the six algorithms can successfully converged to the range of the corresponding objective dimension on PF, their diversity was significantly different. Among the six algorithms, MOEA/D-PaS and DLEA had the worst performance in diversity. MOEA/D-PaS had a poor diversity in the second objective, while DLEA had a poor diversity in the second to fourth objectives. By contrast, NSGA-III, CMOPSO, and Two\_Arch2 algorithms performed better in diversity. However, the diversity of the solution sets obtained by these three algorithms was not as good as that of DL-TPCEA. It can be seen from in Figure 3f that the diversity of the solution set obtained by DL-TPCEA was good in each objective dimension. This indicated that the output solution set of the proposed DL-TPCEA was better than the other five comparative algorithms in terms of convergence and diversity. This result was also consistent with the maximum HV value and minimum IGD value of DL-TPCEA on 5-objective WFG2, as shown in Tables 3 and 5.

**Figure 3.** The final solution set obtained by the six MOEAs on 5-objectives WFG2, shown by parallel coordinates. (**a**) The final solution set obtained by MOEA/D-Pas; (**b**) The final solution set obtained by NSGA-III; (**c**) The final solution set obtained by CMOPSO; (**d**) The final solution set obtained by Two\_Arch2; (**e**) The final solution set obtained by DLEA; (**f**) The final solution set obtained by DL-TPCEA.

**Figure 4.** The final solution set obtained by the six MOEAs on 10-objectives WFG9, shown by parallel coordinates. (**a**) The final solution set obtained by MOEA/D-Pas; (**b**) The final solution set obtained by NSGA-III; (**c**) The final solution set obtained by CMOPSO; (**d**) The final solution set obtained by Two\_Arch2; (**e**) The final solution set obtained by DLEA; (**f**) The final solution set obtained by DL-TPCEA.

As shown in Figure 4, the solution set obtained by DL-TPCEA was superior to the five comparative algorithms in terms of convergence and diversity. For 10-objective WFG9, the range of PF on each objective dimension *m* is from 0 to *m* \*2(*m* = 1, ... , *M*). Among the six algorithms, MOEA/D-PaS converged to few solutions on PF of 10-objective WFG9, so it cannot approach PF well. As shown in Figure 4c,e, the solution set obtained by CMOPSO had a relatively poor diversity on the sixth objective, while DLEA had a relatively poor diversity on the seventh and ninth objectives. NSGA-III and Two\_Arch2 algorithms performed well, second only to the convergence and diversity of the solution set obtained by DL-TPCEA on 10-objective WFG9. This was consistent with the HV values and IGD values in Tables 4 and 6.

Figure 5 showed the IGD value trajectories obtained by running six algorithms on the 5-objective WFG test suite. The algorithm for each trajectory was identified in the bottom legend, and DL-TPCEA was specifically highlighted in red. Each subgraph was marked with a different problem, and its abscissa was the number of evaluations during algorithm iteration, and its ordinate was the IGD value. As can be seen from Figure 5, the IGD value trajectory of DL-TPCEA generally declines fastest (except for WFG1 and WFG3), which indicated that DLEA converged very quickly. In addition, from the final result, the IGD value obtained by DL-TPCEA is usually the minimum or not far from the minimum. On 5-objective WFG1, the final IGD value obtained by DL-TPCEA was second only to DLEA. And DLEA obtained the best IGD values on WFG3, while DL-TPCEA performed only at the mid-range level on this issue. Except for WFG1 and WFG3, DL-TPCEA performed very well on the other seven 5-objective WFGs. In general, DL-TPCEA had the best performance among the six algorithms.

**Figure 5.** The IGD values convergence trajectories obtained by the six MOEAs on 5-objectives WFG test suite.

Combined with results of HV values and IGD values in Tables 3–6, convergence and diversity effects of solution sets in Figures 3 and 4, as well as IGD value trajectory shown in Figure 5, DL-TPCEA had the best performance in these six algorithms. DL-TPCEA had grea<sup>t</sup> advantages in many-objective optimization, both in terms of the convergence and diversity of the final solution set and the convergence speed.

Table 7 shows the average running time of the six comparative algorithms on 3-, 5-, 8-, 10-, and 15-objective WFG1. The last one is the results of Wilcoxon test (the smaller the value is, the shorter the corresponding running time is). The shortest time is NSGA-III, which is due to the simple structure. However, DL-TPCEA is only ranked fourth, which is also a shortcoming. However, given the performance gains, it is worth it, especially for problems that require a lot of accuracy.

**Table 7.** Comparison of runtime (s) among the six algorithms on 3-, 5-, 8-, 10-, and 15-objective WFG1.


#### *5.4. Comparison Experiments of DL-TPCEA and Two Weight-Sum Based Algorithms*

In this section, we compared DL-TPCEA with two weight-sum based algorithms. The weighted sum method is characterized by fast running speed, simple structure, and easy operation. MaOPs in industrial production tend to have more complex PF, so it may be very limited to solve such problems only by weighted sum method. For this purpose, DL-TPCEA is compared with the weight-sum based approach to verify the advantages of the proposed algorithm.

This paper provides two weight-sum based algorithms for comparison. The first algorithm is a modification of the classic NSGA-II framework, which is called WSEA. The environment selection of the WSEA starts with a non-dominated sort, and then the rest of the solutions are selected by using weight-sum method in the layer *MaxFNo* mentioned in Section 4.1.2. The environment selection of the second algorithm only selects individuals by weight-sum method, which is called WSEA2. From the point of minimizing the problem, the way to select individuals here is to pick out the *N* individuals with smallest weighted sum to the next generation. In addition, both algorithms use crossover and mutation operators to generate offspring. Finally, since there is no preference for an objective, the weights of the two algorithms on each objective are set to the same value of 1/*M*. However, in order to consider the impact of each objective size on the algorithm, a normalization operation should be carried out for each objective before calculating the weighted sum. DL-TPCEA is compared with the two weight-sum based algorithms mentioned above, and the results are shown in Tables 8–11.

**Table 8.** HV results of DL-TPCEA and two weight-sum based algorithms on benchmarks WFG1-WFG9 with 3, 5, and 8 objectives.



**Table 9.** HV results of DL-TPCEA and two weight-sum based algorithms on benchmarks WFG1-WFG9 with 10 and 15 objectives.

> As can be seen from the results in Tables 8–11, DL-TPCEA obtained the optimal results in all the other instances except for the HV results of seven instances on WFG1 and WFG3. Regardless of HV or IGD indicator, DL-TPCEA has the best performance among the three algorithms. It also reflects that the complexity of MaOPs cannot be well adapted to only relying on a single weighted sum method. The reasons are as follows: without considering the objective preference, it is not guaranteed that the solution in the population will converge to PF only by the magnitude of the weighted sum. A smaller weighted sum may just be that the individual retains a smaller objective value for some objective, but whether the individual is a non-dominated solution is unknown. In addition, the method based on the weighted sum is linearly convergent. However, different MaOPs have different characteristics, making it difficult to apply this method to all problems.

> From the point of the feature of the problem, WFG1 is convex and mixed, while WFG3 is linear and degenerate. WSEA has just obtained the optimal HV results in several examples of these two problems, indicating the weighted sum method is promising to deal with these problems. However, the IGD values obtained by WSEA and WSEA2 on these examples are very poor, which also indicates that the convergence ability is not strong. The calculation of HV will consider some boundary individuals in the population, so DL-TPCEA may not ge<sup>t</sup> good HV results because of the boundary individuals in the population. However, combining the results of the two indicators, DL-TPCEA performed best in 83 out of 90 instances, which is an overwhelming advantage. These results reflect the limitations of using the weighted sum method to solve MaOPs, and show that DL-TPCEA has good advantages.


**Table 10.** IGD results of DL-TPCEA and two weight-sum based algorithms on benchmarks WFG1-WFG9 with 3, 5, and 8 objectives.

**Table 11.** IGD results of DL-TPCEA and two weight-sum based algorithms on benchmarks WFG1-WFG9 with 10 and 15 objectives.



**Table 11.** *Cont.*
