**3. Cuckoo Search**

The cuckoo search algorithm [69] is a stochastic optimization algorithm that models brood parasitism of cuckoo birds. The algorithm is based on the obligate brood parasitic behavior found in some cuckoo nests by combining a model of this behavior with the principles of <sup>L</sup>évy flights, which discard worst solutions and generate new ones after some certain iteration.

According to the mentioned characteristics, CS can be expressed as three idealized rules:

(1) Each cuckoo lays one egg at a time, and places it in a randomly chosen nest.


The CS algorithm is equiponderant to the integration of <sup>L</sup>évy flights. The position of the *i*th nest is indicated by using *D*-dimensional vector *Xi* = (*xi*1, *xi*2, ... , *xid*), 1 ≤ *i* ≤ *n*;aLévy flight is performed:

$$X\_{i}^{t+1} = x\_{i}^{t} + a \otimes levy(\boldsymbol{\lambda}) \ (i = 1, 2, \dots, n), \tag{1}$$

$$a = a\_0 \otimes (\mathbf{x}\_j^t - \mathbf{x}\_i^t) \pm \tag{2}$$

where α > 0 is the step size that is used to control the range of the random search, which should be related to the scales of the problem of interests, and step size information is more useful can be computed by Equation (2). The product ⊗ means entry-wise multiplications. *xti* and *xtj* are two different solutions selected randomly. A new solution with the same number of cuckoos is generated after partial solutions are discarded. *levy*(λ) with the random walk can be expressed in terms of a simple power-law equation.

$$\text{lev}y(\beta) \sim \mu = \text{t}^{-1-\beta}, \; 0 < \beta \le 2 \tag{3}$$

where μ and *t* are two random numbers following the normal distribution and β often takes a fixed value of 1.5.

$$\log \text{w}(\beta) \sim \frac{\phi \times \mu}{|\upsilon|^{1/\beta}} \tag{4}$$

$$\phi = \left[ \frac{\Gamma(1+\beta) \times \sin(\frac{\pi \times \beta}{2})}{\Gamma(\frac{1+\beta}{2}) \times \beta \times 2^{\frac{\beta-1}{2}}} \right]^{1/\beta} \tag{5}$$

where Γ is gamma function. μ and *v* are random numbers drawn from a normal distribution with mean of 0 and standard deviation of 1, which have an infinite variance with an infinite mean. Here, the consecutive jumps/steps of a cuckoo essentially form a random walk process that obeys a power-law step length distribution with a heavy tail. In <sup>L</sup>évy flights random walk component, the new solution *Xi* is generated through Equation (6).

$$X\_{\mathfrak{F}^{+1,i}} = X\_{\mathfrak{F}^{,i}} + \alpha\_0 \frac{\phi \times \mu}{|\upsilon|^{1/\beta}} (X\_{\mathfrak{g},i-} X\_{\mathfrak{g},\text{best}}) \tag{6}$$

where *Xg,best* represents the best solution obtained thus far and α0 is a scaling factor. The <sup>L</sup>évy distribution is a process of random walk; after a series of smaller steps, <sup>L</sup>évy flights can suddenly obtain a relatively larger step size. <sup>L</sup>évy distribution is implemented at the initial stage of algorithm, which helps to jump out of the local optimum.

$$X\_{i}^{t+1} = x\_{i}^{t} + r(X\_{m}^{t} - X\_{n}^{t}) \tag{7}$$

where *Xtm* and *Xtn* are random solutions at the *t*th generation. *r* generates a random number between −1 and 1. The basic steps of the CS algorithm are summarized in Algorithm 1.


(1) randomly initialize population of n host nests (2) calculate fitness value for each solution in each nest (3) **while** (stopping criterion is not meet do) (4) Generate *xt*+<sup>1</sup> *i* as new solution by using <sup>L</sup>évy flights; (5) Choose candidate solution *xti;* (6) **if** *f*(*xti*) > *f*(*xt*+<sup>1</sup> *i* ) (7) Replace *xti* with new solution *xt*+<sup>1</sup> *i* ; (8) **end if** (9) Throw out a fraction (*pa*) of worst nests; (10) Generate solution *k<sup>t</sup>*+<sup>1</sup> *i* using Equation (3); (11) **if** *f*(*xti*) > *f*(*xt*+<sup>1</sup> *i* ) (12) Replace *xti* with new solution *xt*+<sup>1</sup> *i* ; (13) **end if** (14) Rank the solution and find the current best. (15) **end while**

#### **4. Cuckoo Search Algorithm with** *Q***-Learning and Genetic Operations**
