**5. Results**

Tables 7–18 compare the error values (when the error value was smaller than <sup>10</sup>−8, the corresponding value was considered optimal) obtained by the original algorithms (SHADE, L-SHADE, and jSO) and their improved versions using turning-based mutation (Tb-SHADE, TbL-SHADE and Tb-jSO, respectively). The results of a comparison are showed in the last column of each table. If the performance of the original version was significantly better, uses the "−" sign; if the performance of the improved version was significantly better, uses the sign "+"; if their performances were similar, "=" is used. The better performance values are displayed in bold, and the last row of these tables shows the results of an overall comparison. Tables 19–22 provide the error values obtained by the advanced algorithms DISH, jDE100 and j2020 on CEC2020. All tables provide the best, mean and std (standard deviation) values of 30 independent repetitions of the experiments.

**Table 7.** SHADE vs. Tb-SHADE on CEC2020 in 5*D*.


**Table 8.** SHADE vs. Tb-SHADE on CEC2020 in 10*D*.



**Table 9.** SHADE vs. Tb-SHADE on CEC2020 in 15*D*.

**Table 10.** SHADE vs. Tb-SHADE on CEC2020 in 20*D*.


**Table 11.** L-SHADE vs. TbL-SHADE on CEC2020 in 5*D*.


**Table 12.** L-SHADE vs. TbL-SHADE on CEC2020 in 10*D*.



**Table 13.** L-SHADE vs. TbL-SHADE on CEC2020 in 15*D*.

**Table 14.** L-SHADE vs. TbL-SHADE on CEC2020 in 20*D*.


**Table 15.** jSO vs. Tb-jSO on CEC2020 in 5*D*.


**Table 16.** jSO vs. Tb-jSO on CEC2020 in 10*D*.



**Table 17.** jSO vs. Tb-jSO on CEC2020 in 15*D*.

**Table 18.** jSO vs. Tb-jSO on CEC2020 in 20*D*.


**Table 19.** DISH and jDE100 on CEC2020 in 5*D*.


**Table 20.** DISH and jDE100 on CEC2020 in 10*D*.



**Table 21.** DISH and jDE100 on CEC2020 in 15*D*.

**Table 22.** DISH and jDE100 on CEC2020 in 20*D*.


Convergence diagrams are shown in Figures 1–12. Figures 1–4 shows the convergence curves of SHADE and Tb-SHADE, respectively, for some test functions in 5*D*, 10*D*, 15*D*, and 20*D*, Figures 5–8 shows those of L-SHADE and TbL-SHADE for some test functions in 5*D*, 10*D*, 15*D*, and 20*D*. and Figures 9–12 shows those of the jSO and Tb-jSO, respectively, for some test functions in 5*D*, 10*D*, 15*D* and 20*D*. It is apparent that the red line of the turning-based mutation version of the algorithm was often slower to converge but attained better objective function values.

**Figure 1.** The selected average convergence of SHADE and Tb-SHADE on CEC2020 in 5*D* is compared. From left to right f3, f9 and f10.

**Figure 2.** The selected average convergence of SHADE and Tb-SHADE is compared on CEC2020 in 10*D*. From left to right f8, f9 and f10.

**Figure 3.** The selected average convergence of SHADE and Tb-SHADE is compared on CEC2020 in 15*D*. From left to right f8, f9 and f10.

**Figure 4.** The selected average convergence of SHADE and Tb-SHADE is compared on CEC2020 in 20*D*. From left to right f3, f5 and f9.

**Figure 5.** The selected average convergence of L-SHADE and TbL-SHADE is compared on CEC2020 in 5*D*. From left to right f3, f4 and f10.

**Figure 6.** The selected average convergence of L-SHADE and TbL-SHADE is compared on CEC2020 in 10*D*. From left to right f8, f9 and f10.

**Figure 7.** The selected average convergence of L-SHADE and TbL-SHADE is compared on CEC2020 in 15*D*. From left to right f8, f9 and f10.

**Figure 8.** The selected average convergence of L-SHADE and TbL-SHADE is compared on CEC2020 in 20*D*. From left to right f3, f5 and f9.

**Figure 9.** The selected average convergence of jSO and Tb-jSO is compared on CEC2020 in 5*D*. From left to right f3, f4 and f10.

**Figure 10.** The selected average convergence of jSO and Tb-jSO is compared on CEC2020 in 10*D*. From left to right f3, f8 and f9.

**Figure 11.** The selected average convergence of jSO and Tb-jSO is compared on CEC2020 in 15*D*. From left to right f3, f8 and f9.

**Figure 12.** The selected average convergence of jSO and Tb-jSO is compared on CEC2020 in 20*D*. From left to right f3, f4 and f10.

Tables 23–34 shows the number of runs (#runs) of population aggregation, the average generation (Mean CO) of the first cluster during these runs, and the average population diversity (Mean PD) of these generations.


**Table 23.** Clustering and population diversity of SHADE and Tb-SHADE on the CEC2020 in 5*D*.

**Table 24.** Clustering and population diversity of SHADE and Tb-SHADE on the CEC2020 in 10*D*.


**Table 25.** Clustering and population diversity of SHADE and Tb-SHADE on the CEC2020 in 15*D*.


**Table 26.** Clustering and population diversity of SHADE and Tb-SHADE on the CEC2020 in 20*D*.



**Table 27.** Clustering and population diversity of L-SHADE and TbL-SHADE on the CEC2020 in 5*D*.

**Table 28.** Clustering and population diversity of L-SHADE and TbL-SHADE on the CEC2020 in 10*D*.


**Table 29.** Clustering and population diversity of L-SHADE and TbL-SHADE on the CEC2020 in 15*D*.


**Table 30.** Clustering and population diversity of L-SHADE and TbL-SHADE on the CEC2020 in 20*D*.



**Table 31.** Clustering and population diversity of jSO and Tb-jSO on the CEC2020 in 5*D*.

**Table 32.** Clustering and population diversity of jSO and Tb-jSO on the CEC2020 in 10*D*.


**Table 33.** Clustering and population diversity of jSO and Tb-jSO on the CEC2020 in 15*D*.


**Table 34.** Clustering and population diversity of jSO and Tb-jSO on the CEC2020 in 20*D*.


The rankings of the Friedman test [52] were obtained by using the average value (Mean) of each algorithm on all 10 test functions in Tables 7–22, and are shown in Tables 35–38. The related statistical

values of the Friedman test are shown in Table 39. If the chi-square statistic was greater than the critical value, the null hypothesis was rejected. *p* represents the probability of the null hypothesis obtaining. The null hypothesis here was that there is no significant difference in performance among the nine algorithms considered here on CEC2020.


**Table 35.** The Friedman ranks of comparative algorithms on CEC2020 in 5*D*.

**Table 36.** The Friedman ranks of comparative algorithms on CEC2020 in 10*D*.


**Table 37.** The Friedman ranks of comparative algorithms on CEC2020 in 15*D*.


**Table 38.** The Friedman ranks of comparative algorithms on CEC2020 in 20*D*.



**Table 39.** Related statistical values obtained of Friedman test for α = 0.05.

#### **6. Results and Discussion**

The results on the CEC2020 benchmark sets are first discussed. As shown in Tables 7–18, the scores were two improvements against two instances of worsening (5*D*), four improvements and two instances of worsening (10*D*), five improvements and two instances of worsening (15*D*), and four improvements no instances of worsening (20*D*) in the case of SHADE; three improvements against zero instances of worsening (5*D*), four improvements and one worsening (10*D*), six improvements no worsening (15*D*), and five improvements and no worsening (20*D*) in the case of L-SHADE; and one improvement against no worsening (5*D*), four improvements and one worsening (10*D*), four improvements and one worsening (15*D*), and two improvements two instances of worsening (20*D*) in the case of jSO. In some test functions, the improved algorithm even escaped the local optimum and found the optimal value (if the error was smaller than <sup>10</sup>−8, the relevant value was considered optimal). Examples are f3 in Tables 10 and 13, and Table 14, f8 in Tables 9, 13 and 16, and Table 17, f9 in Table 12, and f10 in Table 11. In most cases, the improved version was clearly better than the original algorithm except for Tb-SHADE (5*D*) and Tb-jSO (20*D*).

According to the convergence curves in Figures 1–12, in most cases, the improved algorithm showed similar convergence to the original in the early stage of the optimization process, but it clearly maintained a longer exploration phase and achieved better values of the objective function in the middle and late stages; in a few cases (such as *f* 4 in Figure 5), the improved algorithm had slower convergence but did not achieve a better objective function value than the original.

As the numerical analyses in Tables 23–34 show, in most cases, the improved algorithms exhibited fewer clusters (#runs), later clustering (mean CO), and higher population density (mean PD) than the original algorithm. But Tb-SHADE (5*D*) had a lower population density on *f* 6–*f* 9, as did TbL-SHADE (all dimensions) on *f* 2–*f* 7, where this might have been related to the linear decrease in the population size. Tb-jSO showed similar numbers of clusters in all dimensions and a lower population density on some test functions in 5*D*. Therefore, in most cases, the improved versions maintained the diversity of population and a longer exploration phase in the optimization process.

The significant improvements in Tables 7–18 and the clustering analysis in Tables 23 and 24 can be linked. The results in the former set of tables with the "+" symbol were always connected with the occurrence of later clustering, none at all, or fewer instances of clusters of 30 (for the last option, see, for example, column #runs in Tables 24–26, *f* 3). Consequently, the improvement in the performance effected by the updated version was related to the maintenance of population diversity and a longer exploration phase.

According to the Friedman ranking in Tables 35–38, Tb-SHADE, TbL-SHADE, and Tb-jSO were clearly better than the original algorithms and the advanced DISH and jDE100 in 10*D*, 15*D,* and 20*D*. But Tb-SHADE did not perform as well as SHADE in 5*D* and did not perform as well as DISH in 5*D*, 10*D* and 20*D*. In addition, the j2020 algorithm delivered the best performance and ranked first in 10*D*, 15*D* and 20*D* and one of the improved versions, TbL-SHADE, only delivered the best performance and ranked first in 5*D*. And jDE100 (winner of CEC2019), which ranks last in Tables 35–38, did not seem suitable for CEC2020. Table 39 shows that the null hypothesis was rejected in all dimensions, and thus the Friedman ranking was correct. All in all, the three improved algorithms obtained good optimization results in contrast to the original algorithm as well as the advanced DISH and jDE100 algorithms but were slightly worse than the advanced j2020 algorithm.
