**5. Results**

#### *5.1. Optimization of Functions and Parameter Settings*

In this section, to check and verify the efficacy DMQL-CS algorithm, it is thoroughly investigated through benchmark evaluations from various respects. We tested our algorithms on two function groups: Group *A* and Group *B*. Group *A* contains fourteen different global optimization problems, as shown in Table 1. Group *B* is the CEC 2013 test suite including 28 benchmark functions. To make a fair comparison, all experiments were carried out on a P4 Dual-core platform with a 1.75 GHz processor and 4 GB memory, running under the Windows 7.0 operating system. The algorithms were written in MATLAB R2017a. The following were set: maximum number of evaluation *MAX\_FES* = *NP* × 105, population size *NP* = 30, run time *T* = 30, and probability of foreign eggs *pa* = 0.25.


**Table 1.** Brief description of fifteen functions.

#### *5.2. Comparison with Other CS Variants and Rank Based Analysis*

We compared the performance of DMQL-CS with four improved CS variants: CCS [68], GCS [96], CSPSO [97], and OLCS [71]. CCS is a modified Chaos enhanced Cuckoo search algorithm. GCS introduces Gaussian disturbance into the CS algorithm. CSPSO is a kind of algorithm combining CS with PSO. A new search strategy based on orthogonal learning strategy is used in OLCS to enhance the exploitation ability of CS algorithm. The parameter configurations of these algorithms are shown in Table 2 according to corresponding references. Fifteen benchmark functions are shown in Tables 3–6 at *D* = 30 and *D* = 50. All optimization algorithms were tested by using the same parameter settings: population size *NP* = 30, *MAX*\_*FES* = 100,000 × *D*, probability switching parameter *pa* = 0.25, and run time *T* = 30.

As shown in Table 3, the DMQL-CS find global optima 0.00 on the four benchmark functions F1, F6, F7, and F14 when *D* = 30. For unimodal functions F1–F5, the DMQL-CS algorithm achieves higher accuracy than other CS variants on functions F2, F4, and F5. DMQL-CS is only inferior to OLCS on F2. For multimodal problems F6–F11, DMQL-CS algorithm shows higher performance than the other CS variants on functions F6, F7, F8, and F11. For F10, the same solution is found by the four algorithms (CCS, GCS, CSPSO, and OLCS). For the shifted unimodal functions F13–F15, DMQL-CS is significantly better than CCS, GCS, OLCS, and CSPSO on F13, F14 and F15. For F12, CCS performs the best.


**Table 2.** The personal parameters of different algorithms.



**Table 4.** The ranking of different strategies according to the Friedman test.




**Table 6.** The ranking of different strategies according to the Friedman test.


DMQL-CS still has outstanding optimization performance when *D* = 50, as shown in Table 5. From the results, it is apparent that the convergence precision of other algorithms drops rapidly, while the DMQL-CS algorithm achieves better performance than other CS variants on most functions. DMQL-CS and OLCS achieve the global optimum on function F7. DMQL-CS cannot ge<sup>t</sup> the minimum; even then, it is not inferior to other algorithms on F4, F5, F10, F12, F13, and F15. In addition, the DMQL-CS demonstrates a remarkable accuracy on benchmark F1 and F2. Comparing with the optimization results, we can conclude that the DMQL-CS optimization algorithm explored a larger search space than other CS variants. Moreover, it is important to point out that, regardless of the problem's dimensionality, the DMQL-CS converges to the better solution on the shifted multimodal functions F13, F14 and F15. Therefore, these statistical tests confirmed that DMQL-CS algorithm with *Q*-Learning step size and genetic operators has a better overall performance than all other tested competitors. For a clearer observation that DMQL-CS performs best, Table 4 shows the ranking of the strategies in Table 3 according to the Friedman test. We can see that DMQL-CS obtains the best rank, OLCS ranks second, followed by CCS, GCS, and CSPSO. Table 6 shows the ranking of the five strategies according to the Friedman test. OLCS obtains the best rank, DMQL-CS ranks second, followed by GCS, CSPSO, and CCS.

To further demonstrate the convergence of DMQL-CS, the median convergence properties of five algorithms are illustrated in Figure 5. There is no obvious "evolution stagnation" for all algorithms. For the same population size and number of generations, the optimization performance of the four algorithms declines rapidly. However, DMQL-CS can ge<sup>t</sup> better convergence curve than CCS, GCS, CSPSO, and OLCS on F1–F2, F5–F6, F12, and F14. In Figure 5, DMQL-CS algorithm converged to the specified error threshold on function F1, which suggests that DMQL-CS algorithm has a faster convergence rate for the specified error threshold. Generally speaking, when *M* is too small, useful step size information will not be learned. When *M* is too large, the speed of *Q*-Learning will be slowed down. When the value of *M* is 3 or 5, the convergence performance of DMQL-CS can be improved for the ill-condition function F2, complex multimodal functions F5–F6, and Shifted multimodal functions F12 and F14. It is worth mentioning that the accuracy of OLCS is similar to that of DMQL-CS, but the convergence speed of DMQL-CS is much faster than that of OLCS. For multimodal function, all algorithms converge to the specified error threshold with the same number of successes. However, DMQL-CS has good reliability, stability, and faster convergence rate on functions F5–F6. For function F14, DMQL-CS algorithms can find the global optimum with 50,000 FES. As mentioned above, it can be clearly observed that DMQL-CS provided better performance than the four other CS versions, and achieves a promising solution on most test functions.

**Figure 5.** Convergence curves of different algorithms on test functions when *D* = 30.

A series of comparisons proved the high efficiency of DMQL-CS. The performance ranking of the multiple algorithms of the test suite is listed in Tables 7–9. The average rankings of the five CS variants for functions F1–F8 are shown in Table 7 (*D* = 30 and *D* = 50). The average rankings of the five CS variants for functions F9–F15 are shown in Table 8 (*D* = 30 and *D* = 50). In competition ranking, if performances of algorithms are the same, they received the same rank. It can be seen in Tables 7–9 that the average ranking value of DMQL-CS on *D* = 30 is smaller than that of CCS, GCS, OLCS, and CSPSO. Therefore, the performance of DMQL-CS is better than the other CS variants. When *D* = 50, the results are similar to those when *D* = 30, with the average ranking value of DMQL-CS being smaller than those of CCS, GCS, OLCS, and CSPSO.


**Table 7.** Rank results of each algorithm on F1–F8 for *D* = 30 and *D* = 50.

**Table 8.** Rank results of each algorithm on F9-F15 for *D* = 30 and *D* = 50.


**Table 9.** Total rank and final rank on F1–F15 for *D* = 30 and *D* = 50.


In Table 9, DMQL-CS has the best total rank when *D* = 30 and *D* = 50, i.e., 25 and 29, which means that DMQL-CS has the best performance on most of the test functions compared with other algorithms. OLCS has the second-best total rank at *D* = 30 and *D* = 50, i.e., 29 and 30. Obviously, OLCS has better performance than the three other algorithms on high-dimension test functions. Similarly, in Table 9, the order can be clearly observed: DMQL-CS > OLCS > CCS > GCS > CSPSO at *D* = 30; and DMQL-CS > OLCS > GCS > CCS > CSPSO at *D* = 50. Based on the analysis of the above, DMQL-CS has the best performance among all the algorithms on both *D* = 30 and *D* = 50.

#### *5.3. Statistical Analysis of Performance for the CEC 2013 Test Suite*

In this section, the CEC 2013 test suite is selected to test the effectiveness of three different algorithms (jDE [98], SaDE [99], and CLPSO [100]). These algorithms can be seen as representatives of the state-of-the-art algorithms for comparison, and the parameter configurations of these algorithms were set according to the corresponding references, as listed in Table 10.


**Table 10.** The personal parameters of di fferent algorithms.

Table 11 summarizes the results of CEC 2013 test problems on 28 benchmark functions for 30-dimensional case. The rank was used to obtain the ranking of di fferent algorithms on all problems, as shown in Table 12. This means that DMQL-CS gets the first rank and outperforms jDE, SaDE, and CLPSO. The results in Table 11 indicate that with 80% certainty DMQL-CS has statistically higher accuracy than the other algorithms. Note that DMQL-CS obtains the global optimal value 0.00 on F1 and F11. DMQL-CS is significantly better than the three other algorithms, especially on functions CEC 2013-F1, CEC 2013-F2, and CEC 2013-F4. About basic Multimodal Function and composition Functions (CEC 2013-F6–CEC 2013-F28), the ability of DMQL-CS to find the optimal solution is slightly better than that of CLPSO. For functions CEC 2013-F5, CEC 2013-F7, CEC 2013-F17, CEC 2013-F22, and CEC 2013-F25, the performance of DMQL-CS is slightly worse than the other algorithms. For the Unimodal problem CEC 2013-F3, jDE obtains the best solution 2.99 × 106. On Shift Rastrigin Function, SaDE and jDE can ge<sup>t</sup> better solutions of 1.10 × 10<sup>1</sup> and 1.06 × <sup>10</sup>−4, respectively. For CEC 2013-F25–CEC 2013-F26, DMQL-CS is obviously better than SaDE and jDE. CLPSO has the weakest ability to find the optimal solution for 28 functions. From the above results, it can be seen that DMQL-CS with *Q*-Learning and genetic operations has a better overall performance than all other tested competitors on the CEC 2013 test suite. Table 13 reports the rankings of the results between DMQL-CS and other algorithms. In Table 13, it can be seen that DMQL-CS performs the best among the four algorithms. DMQL-CS exhibits consistent ranks of the first in optimizing most of the functions. For a clearer observation that DMQL-CS performs best, Table 12 shows the ranking of the algorithms in according to the Friedman test. DMQL-CS obtains the best rank, jDE ranks second, followed by SaDE and CLPSO. **Table 11.** The mean and standard deviation (STD) of CEC 2013 test suite with four algorithms.



**Table 12.** The ranking of different strategies according to the Friedman test.

**Table 13.** Comparisons between DMQL-CS and other algorithms for CEC 2013 test suite.


*5.4. Application in the Problem of Logistics Distribution Center Location*
