4.3.2. Mutation Process

A parameter *Cm* is defined as the mutation probability. The number *r* is randomly generated in the interval [0, 1]. If *r* < *Cm*, the *i*th chromosome G1 is selected to perform the mutation operation and this process is repeated at each iteration. For illustration, we continue to use the problem of logistics distribution center location with 40 cities and 10 distribution centers. Two genes located on chromosome G1 are randomly selected and their positions swapped to obtain a possible child. Then, the optimal is found as an arc between any two nodes by using enumeration method, which keeps the child obtaining the lowest objective value. Finally, we ge<sup>t</sup> chromosome R1, as shown in Figure 3. If R1 has a smaller objective value than G1, G1 is replaced with R1, else G1 is retained. A new generation of population is generated after the evaluation, crossover, and mutation operations.

**Figure 3.** Mutation process of chromosomes.

4.3.3. Cuckoo Search Algorithm with *Q*-Learning Model and Genetic Operator

Introducing *Q*-Learning into CS algorithm helps to learn the optimal step size strategy and transform. Crossover and mutation strategies enable the nest to approach the historical optimal nest quickly, which can speed up the global convergence rate. The structure of the genetic operator cuckoo search algorithm with *Q*-Learning model (DMQL-CS) is described in Algorithm 3.

**Algorithm 3** DMQL-CS Algorithm.

**Input:** Population size, *NP;* Maximum number of function evaluations, *MAX\_FES*, *LP* (1) Randomly initialize position of *NP* nest, *FES* = *NP*; (2) Calculate the fitness value of each initial solution; (3) **while** (stopping criterion is not meet do) (4) Select the best step size control strategy according to Algorithm 2; (5) Generate new solution *xt*+<sup>1</sup> *i* with the new step size by <sup>L</sup>évy flights; (6) Randomly choose a candidate solution *xti*; (7) **if** *f*(*xti*) > *f*(*xt*+<sup>1</sup> *i* ) (8) Replace *xti* with new solution *xt*+<sup>1</sup> *i* ; (9) **end if** (10) Generate new solution *xt*+<sup>1</sup> *i* by using crossover operator and mutation operator; (11) Throw out a fraction (*pa*) of worst nests, generate solution *k<sup>t</sup>*+<sup>1</sup> *i* using Equation (3); (12) **if** *f*(*xti*) > *f*(*xt*+<sup>1</sup> *i* ) (13) Replace *xti* with new solution *xt*+<sup>1</sup> *i* ; (14) **end if** (15) Rank the solution and find the current best. (16) **end while**

#### 4.3.4. Analysis of Algorithm Complexity

To show the convergence effect of the algorithm, typical function Rastrigrin was selected to analyze the convergen<sup>t</sup> process of DMQL-CS algorithm. Figure 4 shows the location distribution of cuckoo individuals in the search area with a population size of 10. Figure 4a describes the individual distribution at the first generation, Figure 4b describes the individual distribution at the 30th generation, Figure 4c describes the individual distribution at the 50th generation, and Figure 4d describes the individual distribution at the 80th generation. In Figure 4, it can be seen that the activity area of individuals keeps changing and gradually draws closer to the optimal solution during the evolution of the algorithm. It is worth noting that algorithm converged at the 80th generation, which indicates that *Q*-learning and genetic operation expand activity area of the population and improve the convergence performance of the DMQL-CS algorithm.

**Figure 4.** Analysis of algorithm complexity.
